Construction site image processing method and device, equipment and medium

Abstract

The invention relates to the technical field of image processing, and provides a construction site image processing method, device and equipment, and a medium, which can identify a non-target object in a target construction site image, generate a mask in the target construction site image according to an identification result, roughly position the non-target object, and generate a mask in the target construction site image according to the identification result. Accurate target segmentation is carried out according to positioning, so that the processing efficiency and precision can be effectively balanced; the target construction site image is erased according to the mask to obtain a first intermediate image with a blank area, and content completion is performed on the blank area based on an environment perception algorithm to obtain a second intermediate image, so that the problem of feature matching failure caused by dynamic interferents can be solved, and meanwhile, the structural integrity of a building main body is ensured; and detail reduction processing is performed on the second intermediate image based on the spatial semantic information of the building information model, so that the influence of illumination variation and shielding interference can be reduced, and a high-quality image which is more suitable for executing a construction site monitoring task is generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a construction site image processing method, device, equipment and medium. BACKGROUND

[0002] With the accelerated promotion of the digital transformation of the construction industry, intelligent construction monitoring technology based on the fusion of BIM (Building Information Modeling) and computer vision has become a research hotspot. However, the complexity and dynamics of the construction site environment pose great challenges to image processing, and therefore, an efficient construction site image processing method is urgently needed to solve the following technical bottlenecks: 1) Feature matching failure caused by dynamic interference.

[0003] Traditional construction monitoring mainly relies on the comparison of manual image inspection and BIM rendering images, but dynamic interference such as scaffolding and temporary equipment is common on construction sites. These unstructured background noises can severely damage the accuracy of image feature alignment, resulting in a significant decrease in subsequent change detection accuracy. 2) Light changes and occlusion interference.

[0004] The lighting conditions (such as shadows and reflections) and local occlusions (such as equipment movement and material stacking) on the construction site can significantly reduce image quality, making it difficult to extract effective features using traditional threshold segmentation algorithms or edge detection algorithms. For example: low-contrast areas are prone to broken contours, directly affecting the reliability of change detection. SUMMARY

[0005] In view of the above, it is necessary to provide a construction site image processing method, device, equipment and medium, aiming to solve the problem of poor construction site image processing effect caused by dynamic interference, light changes and occlusion interference.

[0006] A construction site image processing method, the construction site image processing method comprising: In response to a processing instruction for a target construction site image, identifying non-target objects in the target construction site image to obtain an identification result; Generating a mask in the target construction site image according to the identification result; Erasing the target construction site image according to the mask to obtain a first intermediate image with a blank area; Performing content completion on the blank area in the first intermediate image based on an environment perception algorithm to obtain a second intermediate image; Performing detail restoration processing on the second intermediate image based on the spatial semantic information of the building information model to obtain a target image.

[0007] According to a preferred embodiment of the present application, the identifying the non-target object in the target construction site image to obtain an identification result comprises: identifying a region to which the non-target object in the target construction site image belongs as the identification result; wherein the region to which the non-target object belongs is a rectangular frame for reflecting position information of the non-target object.

[0008] According to a preferred embodiment of the present application, the generating a mask in the target construction site image according to the identification result comprises: processing an image in the rectangular frame by using a U-Net model to obtain a probability map of each pixel; performing binary conversion on the probability map to obtain the mask.

[0009] According to a preferred embodiment of the present application, the content completion of the blank area in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image comprises: performing multi-scale feature extraction on the first intermediate image by using a feature extractor to obtain feature maps of different levels; retaining context features in the feature maps except the blank area; aggregating the context features to each pixel of the blank area based on a context attention mechanism to obtain aggregated features corresponding to the blank area; converting the aggregated features into pixel values to obtain completion features; optimizing the completion features to obtain optimized features; replacing the blank area in the first intermediate image with the optimized features to obtain the second intermediate image.

[0010] According to a preferred embodiment of the present application, the optimization of the completion features to obtain optimized features comprises: extracting edge features in the context features based on an edge detection operator; performing smooth connection on edges of the completion features and the edge features as a constraint to obtain first features; eliminating edge jaggies in the first features to obtain the optimized features.

[0011] According to a preferred embodiment of the present application, the detail restoration processing of the second intermediate image based on spatial semantic information of a building information model to obtain a target image comprises: retaining an image region in the second intermediate image that spatially overlaps with a component of the building information model as a to-be-processed region; acquire an infrared image and a depth image corresponding to the to-be-processed region by using a sensor; acquire a low-brightness region in the to-be-processed region, and add an infrared pixel value corresponding to the low-brightness region in the infrared image to the low-brightness region as a supplement to obtain a first image; acquire a textureless region in the to-be-processed region, and supplement depth information corresponding to the textureless region in the depth image to the textureless region to obtain a second image; fuse the first image and the second image to the second intermediate image to obtain a fused image; perform adaptive gamma correction on the fused image to obtain a third image; perform edge enhancement processing on the third image by using an edge enhancement algorithm to obtain the target image.

[0012] According to the preferred embodiment of the present application, after the target image is obtained, the method further comprises: in response to an execution instruction of a target construction site monitoring task triggered based on the target image, acquire a model pool constructed in advance; wherein the model pool is used to store an algorithm model corresponding to each construction site monitoring task; call a target model corresponding to the target construction site monitoring task from the model pool; process the target image by using the target model to obtain a processing result; generate a monitoring report of the target construction site monitoring task according to the processing result; encrypt and send the monitoring report to a triggerer of the execution instruction.

[0013] A construction site image processing apparatus, comprising: an identification unit configured to identify a non-target object in a target construction site image in response to a processing instruction of the target construction site image to obtain an identification result; a generation unit configured to generate a mask in the target construction site image according to the identification result; an erasing unit configured to perform erasing processing on the target construction site image according to the mask to obtain a first intermediate image with a blank region; a completion unit configured to perform content completion on the blank region in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image; a restoration unit configured to perform detail restoration processing on the second intermediate image based on spatial semantic information of a building information model to obtain a target image.

[0014] A computer device, comprising: a memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the construction site image processing method.

[0015] A computer readable storage medium, the computer readable storage medium stores at least one instruction, the at least one instruction is executed by a processor in a computer device to implement the construction site image processing method.

[0016] From the above technical solutions can be seen, the present application can identify the non-target object in the target construction site image, and according to the identification result in the target construction site image, the mask is generated, first rough non-target object positioning, then according to the positioning, the target segmentation is carried out accurately, which can effectively balance the processing efficiency and precision; according to the mask, the target construction site image is erased to obtain the first intermediate image with blank area, and based on the environment perception algorithm, the content of the blank area in the first intermediate image is completed to obtain the second intermediate image, which can solve the feature matching failure problem caused by dynamic interference, and ensure the integrity of the building main structure; based on the spatial semantic information of the building information model, the second intermediate image is processed to restore the details to obtain the target image, which can reduce the influence of light change and shielding interference, so as to generate a high-quality image more suitable for performing construction site monitoring task. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flow chart of a preferred embodiment of the construction site image processing method of the present application; Figure 2 is a function module diagram of a preferred embodiment of the construction site image processing device of the present application; Figure 3 is a structural schematic diagram of a computer device for implementing a preferred embodiment of the construction site image processing method of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be described in detail below in combination with the drawings and specific embodiments.

[0019] As shown in Figure 1 is a flow chart of a preferred embodiment of the construction site image processing method of the present application. According to different needs, the order of steps in the flow chart can be changed, and some steps can be omitted.

[0020] The construction site image processing method is applied to one or more computer devices, which is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions. The hardware thereof includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0021] The computer device can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smartphone, a personal digital assistant (PDA), a game console, an interactive Internet Protocol Television (IPTV), a smart wearable device, etc.

[0022] The computer device can also include network devices and / or user devices. The network devices include, but are not limited to, single network servers, server groups composed of multiple network servers, or clouds composed of a large number of hosts or network servers based on cloud computing.

[0023] The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc. Basic cloud computing services.

[0024] Among them, artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application system.

[0025] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. Several major directions.

[0026] The network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0027] S10, in response to a processing instruction of a target construction site image, identifying a non-target object in the target construction site image to obtain an identification result.

[0028] In the embodiment, the processing instruction can be triggered by a safety inspector of the construction site according to actual needs.

[0029] In the embodiment, the target construction site image can be collected by an image collection device deployed in the construction site, or collected by a mobile device such as a drone.

[0030] In the embodiment, the non-target object can include interference objects in the construction site that do not belong to the building content.

[0031] For example, the non-target object can include, but is not limited to, one or a combination of the following objects: workers, slogans, garbage, pull-up banners, construction tools, scaffolding, temporary equipment, etc.

[0032] Since the non-target object does not belong to the building content, when it is necessary to use the construction site image to perform a safety inspection or other task, it will adversely affect the inspection result, so it is necessary to identify the non-target object first.

[0033] Specifically, the identification of the non-target object in the target construction site image to obtain an identification result includes: identifying the area to which the non-target object in the target construction site image belongs as the identification result; The area to which the non-target object belongs is a rectangular frame reflecting the position information of the non-target object.

[0034] For example, the area to which the non-target object in the target construction site image belongs can be identified by a target detection model such as a YOLO (You Only Look Once) model, an SSD (Single Shot MultiBox Detector) model, etc., to obtain a rectangular frame with a class probability.

[0035] Through the above embodiment, the non-target object can be roughly located first, preparing for subsequent more refined processing.

[0036] S11, generating a mask in the target construction site image according to the identification result.

[0037] In the embodiment, the generating a mask in the target construction site image according to the identification result comprises: processing the image in the rectangular frame by using a U-Net (Convolutional Networks for Biomedical Image Segmentation) model to obtain a probability map of each pixel; performing binary conversion on the probability map to obtain the mask.

[0038] The probability map is used to reflect the probability distribution of each pixel belonging to the foreground or other categories.

[0039] Through the above embodiment, after roughly positioning the non-target object, the mask contour of the non-target object can be further accurately identified by using the U-Net model, and the pixel-level accurate segmentation of the non-target object is realized. Since the rough positioning has high processing efficiency but low accuracy, and the U-Net model has low efficiency but high accuracy, the combination of the two can effectively balance the efficiency and accuracy.

[0040] S12, performing erasing processing on the target construction site image according to the mask to obtain a first intermediate image with a blank area.

[0041] In the embodiment, a neural network model or an open-source image processing tool (such as a rubber eraser tool) can be used to perform erasing processing on the target construction site image according to the mask.

[0042] S13, performing content completion on the blank area in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image.

[0043] In the embodiment, the performing content completion on the blank area in the first intermediate image based on the environmental perception algorithm to obtain the second intermediate image comprises: performing multi-scale feature extraction on the first intermediate image by using a feature extractor to obtain feature maps of different levels; retaining context features in the feature maps except the blank area; aggregating the context features to each pixel of the blank area based on a context attention mechanism to obtain aggregated features corresponding to the blank area; converting the aggregated features into pixel values to obtain completion features; performing optimization processing on the completion features to obtain optimized features; replacing the blank area in the first intermediate image with the optimized features to obtain the second intermediate image.

[0044] The feature extractor can include, but is not limited to, a convolutional neural network (CNN) or other model having a multi-scale feature extraction function.

[0045] The feature map can include hierarchical features such as shallow texture details and deep structure semantics.

[0046] The context features can include architectural structure features such as building edges, contours, and wall textures, and can also include architectural component features such as sofas and coffee tables.

[0047] The context attention mechanism can ensure the similarity of blank areas to surrounding spatial features and semantic features when aggregating.

[0048] The above embodiments can solve the feature matching failure problem caused by dynamic interference based on an environmental perception algorithm, while ensuring the integrity of the main building structure.

[0049] In this embodiment, the optimization processing of the completed features to obtain optimized features includes: An edge feature in the context features is extracted based on an edge detection operator; The edges of the completed features are smoothly connected to the edge feature based on a structure consistency loss as a constraint to obtain first features; Edge jaggies in the first features are eliminated to obtain the optimized features.

[0050] The edge detection operator can be a Canny operator or the like.

[0051] The structure consistency loss can be an L1 norm or L2 norm between the edges of the completed features and the edge feature, used to constrain the difference between the two edge features.

[0052] The edge jaggies in the first features can be eliminated using morphological operations such as erosion and dilation, or smoothing filter operations such as Gaussian filtering, to make the completed area and the surrounding transition more natural.

[0053] The above embodiments can avoid destroying the original structure of the building (such as wall perpendicularity, roof slope, door and window position, etc.) in the completion result, thereby ensuring the rationality of the completed building structure.

[0054] S14, based on the spatial semantic information of the building information model (BIM), the second intermediate image is subjected to detail restoration processing to obtain a target image.

[0055] In the embodiment, the space semantic information based on the building information model is used to perform detail restoration processing on the second intermediate image to obtain a target image. A region of the second intermediate image that overlaps with the building information model component in space is reserved as a to-be-processed region. An infrared image and a depth image corresponding to the to-be-processed region are acquired by using a sensor. A low-brightness region in the to-be-processed region is acquired, and an infrared pixel value corresponding to the low-brightness region in the infrared image is added to the low-brightness region as a supplement to obtain a first image. A textureless region in the to-be-processed region is acquired, and depth information corresponding to the textureless region in the depth image is added to the textureless region as a supplement to obtain a second image. The first image and the second image are fused to the second intermediate image to obtain a fused image. Adaptive gamma correction is performed on the fused image to obtain a third image. Edge enhancement processing is performed on the third image by using an edge enhancement algorithm to obtain the target image.

[0056] The space semantic information is used to reflect a geometric attribute of a component. For example, the space semantic information can be a component space coordinate.

[0057] The to-be-processed region is used to reflect an intersection in three-dimensional space between a component (i.e., a geometric body in three-dimensional space, such as a tea table) in the building information model and a real scene (i.e., the target construction site image) that is photographed.

[0058] The sensor includes an infrared sensor, a depth sensor, and the like.

[0059] The infrared image is used to reflect heat radiation of an object and is not affected by light, and can make up for a deficiency of visible light in a dark or blocked region.

[0060] By adding, as a supplement, an infrared pixel value corresponding to the low-brightness region in the infrared image to the low-brightness region, the contour visibility of the region can be enhanced.

[0061] The depth image is used to reflect a three-dimensional structure of a scene, and can provide object contour and distance information to make up for a detail deficiency of visible light in a textureless region. For example, the textureless region can be a smooth wall surface.

[0062] By adding, as a supplement, depth information corresponding to the textureless region in the depth image to the textureless region, gradient information of the depth image can be used to supplement contour details.

[0063] Through the infrared image and the depth image, information missing of visible light in extreme conditions can be filled.

[0064] The adaptive gamma correction algorithm based on brightness can be used to perform adaptive gamma correction on the fusion image. After the adaptive gamma correction based on brightness, the third image not only retains the saliency of the infrared target, but also repairs the light and dark imbalance of the visible light image, so that the overall contrast is more balanced.

[0065] In addition, through adaptive gamma correction for local brightness dynamic adjustment, the problem of global gamma correction that compromises one side to benefit the other side in high and low contrast regions can be solved, and the loss of details caused by global processing can be avoided.

[0066] The edge enhancement algorithm based on the Canny operator can be used to perform edge enhancement processing on the third image. Through the edge enhancement algorithm, the edge enhancement processing on the third image can further enhance the edge and texture details of objects in low-contrast regions, make the edge clearer, and enhance the edge continuity, thereby improving the recognition degree of details.

[0067] Through the above embodiments, the influence of light changes and shielding interference can be reduced, thereby generating high-quality images more suitable for performing construction site monitoring tasks.

[0068] In this embodiment, after obtaining the target image, the method further includes: In response to an execution instruction of a target construction site monitoring task triggered based on the target image, a pre-constructed model pool is obtained; wherein the model pool is used to store algorithm models corresponding to each construction site monitoring task; A target model corresponding to the target construction site monitoring task is called from the model pool; The target image is processed using the target model to obtain a processing result; A monitoring report of the target construction site monitoring task is generated according to the processing result; The monitoring report is encrypted and sent to the triggerer of the execution instruction.

[0069] The target construction site monitoring task can include, but is not limited to, one or a combination of the following tasks: Crack detection, tilt detection, construction specification detection, construction phase detection, construction safety detection, construction quality detection, and the like.

[0070] The model pool can use a "hierarchical storage + index management" architecture. Specifically, the model pool can include a physical storage layer, a metadata management layer, and a task matching layer.

[0071] The physical storage layer is used for actual storage of model files (weights, configurations, inference code), supports local server and cloud backup; The metadata management layer is used for storing description information of the model (such as applicable tasks, precision indicators, and update time), and enables quick retrieval through a database. The task matching layer is used for receiving construction site monitoring tasks (such as crane posture recognition), calling the optimal model through metadata index matching, and loading the model file from the physical storage layer.

[0072] The metadata of the model pool can be stored in a relational database, associated through "task type-model ID", and supports multi-dimensional queries.

[0073] The models in the model pool can use a hybrid storage strategy, i.e., local storage, cloud storage, and storage in the same format specification (such as unified model file naming method, unified compression and encryption method).

[0074] By constructing the model pool, the model can be quickly called, thereby improving the execution efficiency of the construction site monitoring task.

[0075] The triggerer can be a safety inspector, etc.

[0076] Through the above embodiments, high-quality images that have been pre-processed can be used to perform more accurate and rapid task processing.

[0077] In this embodiment, efficient data processing is achieved through an end-to-end layered architecture, which can integrate various processing links in a lightweight manner and achieve a response in seconds.

[0078] This embodiment can also synchronously integrate a privacy desensitization module to automatically blur sensitive information such as faces and license plates, thereby improving construction safety and providing a solution for smart construction sites that combines real-time performance and compliance.

[0079] This embodiment can also provide a visual interface, so that relevant staff can view the image processing in real time.

[0080] From the above technical scheme can be seen, the application can identify non-target objects in the target construction site image, and generate a mask in the target construction site image according to the identification result, and then perform rough non-target object positioning, and then perform accurate target segmentation according to the positioning, so as to effectively balance the processing efficiency and accuracy; the target construction site image is erased according to the mask to obtain a first intermediate image with a blank area, the content of the blank area in the first intermediate image is completed based on an environmental perception algorithm to obtain a second intermediate image, which can solve the feature matching failure problem caused by dynamic interference, and at the same time ensure the integrity of the building main structure; the second intermediate image is processed for detail restoration based on the spatial semantic information of the building information model to obtain a target image, which can reduce the influence of light changes and shielding interference, so as to generate a high-quality image more suitable for performing a construction site monitoring task.

[0081] As Figure 2 shown, it is a functional module diagram of a preferred embodiment of the construction site image processing device. The construction site image processing device 11 comprises an identification unit 110, a generation unit 111, an erasing unit 112, a completion unit 113, and a restoration unit 114. The module / unit referred to in the application refers to a series of computer program segments that can be executed by a processor and can complete a fixed function, which is stored in a memory. In this embodiment, the functions of each module / unit will be described in detail in the subsequent embodiments.

[0082] The identification unit 110 is configured to identify non-target objects in the target construction site image in response to a processing instruction of the target construction site image, and obtain an identification result. The generation unit 111 is configured to generate a mask in the target construction site image according to the identification result. The erasing unit 112 is configured to erase the target construction site image according to the mask to obtain a first intermediate image with a blank area. The completion unit 113 is configured to complete the content of the blank area in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image. The restoration unit 114 is configured to process the second intermediate image for detail restoration based on the spatial semantic information of the building information model to obtain a target image.

[0083] From the above technical scheme can be seen, the application can identify non-target objects in the target construction site image, and generate a mask in the target construction site image according to the identification result, and then perform rough non-target object positioning, and then perform accurate target segmentation according to the positioning, so that the processing efficiency and accuracy can be effectively balanced; the target construction site image is erased according to the mask to obtain a first intermediate image with a blank area, the content of the blank area in the first intermediate image is completed based on an environmental perception algorithm to obtain a second intermediate image, so that the feature matching failure problem caused by dynamic interference objects can be solved, and the integrity of the building main structure can be ensured; the second intermediate image is processed based on the spatial semantic information of the building information model to obtain a target image, so that the influence of light changes and shielding interference can be reduced, and a high-quality image more suitable for performing a construction site monitoring task can be generated.

[0084] As Figure 3 shown, it is a structural schematic diagram of a computer device of a preferred embodiment of the construction site image processing method.

[0085] The computer device 1 can include a memory 12, a processor 13 and a bus (the arrow in the figure is the bus), and can also include a computer program stored in the memory 12 and executable on the processor 13, such as a construction site image processing program.

[0086] Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1, and does not constitute a limitation on the computer device 1, and the computer device 1 can be a bus type structure or a star type structure, and the computer device 1 can also include more or less other hardware or software, or different component arrangements, for example, the computer device 1 can also include an input / output device, a network access device, etc.

[0087] It should be noted that the computer device 1 is only an example, and other existing or future electronic products, such as those adaptable to the application, should also be included within the protection scope of the application and included by reference.

[0088] The memory 12 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a mobile hard disk of the computer device 1. In other embodiments, the memory 12 can also be an external storage device of the computer device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 12 can include both an internal storage unit and an external storage device of the computer device 1. The memory 12 can be used to store application software and various data installed in the computer device 1, such as codes of a construction site image processing program, etc., and to temporarily store data that has been output or will be output.

[0089] The processor 13 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 13 is a control unit of the computer device 1, which connects various components of the computer device 1 through various interfaces and lines, executes programs or modules stored in the memory 12 (e.g., a construction site image processing program, etc.), and calls data stored in the memory 12 to perform various functions and process data of the computer device 1.

[0090] The processor 13 executes an operating system and various application programs installed in the computer device 1. The processor 13 executes the application programs to implement the steps in each of the above construction site image processing method embodiments, such as Figure 1 the steps shown in the above embodiments.

[0091] By way of example, the computer program can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units can be a series of computer-readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program can be divided into an identification unit 110, a generation unit 111, an erasing unit 112, a completion unit 113, and a restoration unit 114.

[0092] The integrated units in the form of software function modules described above can be stored in a computer-readable storage medium. The software function modules described above are stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute part of the construction site image processing method described in various embodiments of the present application.

[0093] The modules / units integrated in the computer device 1, if implemented in the form of software function modules and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the present application can also be implemented by a computer program to instruct related hardware devices to complete all or part of the processes in the above-described embodiments. The computer program can be stored in a computer-readable storage medium and can implement the steps of each method embodiment described above when executed by a processor.

[0094] The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0095] Further, the computer-readable storage medium can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; and the data storage area can store data created according to the use of the blockchain node, etc.

[0096] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods. Each data block contains information about a batch of network transactions, and is used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0097] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one straight line is used in the drawings, but this does not mean that there is only one bus or only one type of bus. The bus is provided to realize the connection and communication between the memory 12, the at least one processor 13, etc. Figure 3

[0098] Although not shown, the computer device 1 can also include a power supply (such as a battery) for powering the various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, so that the power management device can realize functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The computer device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described here.

[0099] Further, the computer device 1 can also include a network interface. Optionally, the network interface can include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0100] ​Optionally, the computer device 1 can further comprise a user interface, which can be a display, an input unit such as a keyboard, and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED touch, etc. The display can also be appropriately referred to as a display screen or a display unit, for displaying information processed in the computer device 1 and for displaying a visualized user interface.

[0101] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by the structure.

[0102] It can be understood by those skilled in the art that, Figure 3 The structure shown does not constitute a limitation on the computer device 1, which can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0103] In combination Figure 1 The memory 12 in the computer device 1 stores a plurality of instructions to implement a construction site image processing method, and the processor 13 can execute the plurality of instructions to implement: In response to a processing instruction for a target construction site image, identifying non-target objects in the target construction site image to obtain an identification result; Generating a mask in the target construction site image according to the identification result; Erasing the target construction site image according to the mask to obtain a first intermediate image with a blank area; Performing content completion on the blank area in the first intermediate image based on an environment perception algorithm to obtain a second intermediate image; Performing detail restoration processing on the second intermediate image based on spatial semantic information of a building information model to obtain a target image.

[0104] Specifically, the processor 13 can refer to the description of the specific implementation method of the above instructions Figure 1 The description of the related steps in the corresponding embodiments will not be repeated here.

[0105] It should be noted that the data involved in the present case are all legally obtained. The non-company software tools or components appearing in the embodiments of the present application are only examples for introduction and do not represent actual use.

[0106] In several embodiments provided by the present application, it should be understood that the disclosed system, apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. For example, the division of the modules is merely logical function division. In actual implementation, other division manners can be adopted.

[0107] The application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0108] The modules described as separate components can or can not be physically separate, and the components shown as modules can or can not be physical units, i.e. can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0109] In addition, the functional modules in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0110] It is obvious to those skilled in the art that the application is not limited to the details of the above exemplary embodiments, and the application can be implemented in other specific forms without departing from the spirit or essential characteristics of the application.

[0111] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the application. Any additional reference signs in the claims should not be considered as limiting the claims involved.

[0112] Furthermore, the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural and vice-versa, unless the context clearly requires these exclusions. The mere fact that different features are recited in mutually different dependent claims does not indicate that the

[0113] Finally, it should be noted that the above-mentioned embodiments illustrate rather than limit the application, since the skilled person, having regard to the above-mentioned teaching, will be able to carry out equivalent modifications or alternative techniques without departing from the application.

Claims

1. A construction site image processing method characterized by, The construction site image processing method comprises: In response to a processing instruction of a target construction site image, non-target objects in the target construction site image are identified to obtain an identification result; According to the identification result, a mask is generated in the target construction site image; According to the mask, the target construction site image is subjected to erasing processing to obtain a first intermediate image with a blank area; Based on an environmental perception algorithm, the blank area in the first intermediate image is subjected to content completion to obtain a second intermediate image; Based on spatial semantic information of a building information model, the second intermediate image is subjected to detail restoration processing to obtain a target image.

2. The construction site image processing method of claim 1, wherein The identification of the non-target objects in the target construction site image to obtain the identification result comprises: The region to which the non-target objects belong in the target construction site image is identified as the identification result; The region to which the non-target objects belong is a rectangular frame reflecting the position information of the non-target objects.

3. The construction site image processing method of claim 2, wherein The generation of the mask in the target construction site image according to the identification result comprises: A U-Net model is used to process the image in the rectangular frame to obtain a probability map of each pixel; The probability map is subjected to binary conversion to obtain the mask.

4. The construction site image processing method of claim 1, wherein The content completion of the blank area in the first intermediate image based on the environmental perception algorithm to obtain the second intermediate image comprises: A feature extractor is used to perform multi-scale feature extraction on the first intermediate image to obtain feature maps of different levels; Contextual features in the feature maps except the blank area are retained; Based on a contextual attention mechanism, the contextual features are aggregated to each pixel of the blank area to obtain aggregated features corresponding to the blank area; The aggregated features are converted into pixel values to obtain completed features; The completed features are subjected to optimization processing to obtain optimized features; The blank area in the first intermediate image is replaced with the optimized features to obtain the second intermediate image.

5. The construction site image processing method of claim 4, wherein The optimization processing of the completed features to obtain the optimized features comprises: Edge features in the contextual features are extracted based on an edge detection operator; With structural consistency loss as a constraint, edges of the completed features are smoothly connected with the edge features to obtain first features; Edge jaggies in the first features are eliminated to obtain the optimized features.

6. The construction site image processing method of claim 1, wherein The detail restoration processing of the second intermediate image based on the spatial semantic information of the building information model to obtain the target image comprises: Image regions in the second intermediate image that spatially overlap with components of the building information model are retained as a to-be-processed region; Infrared images and depth images corresponding to the to-be-processed region are acquired by using sensors; Low-brightness regions in the to-be-processed region are acquired, and infrared pixel values in the infrared images corresponding to the low-brightness regions are added to the low-brightness regions as supplements to obtain first images; Textureless regions in the to-be-processed region are acquired, and depth information in the depth images corresponding to the textureless regions is supplemented to the textureless regions to obtain second images; fusing the first image and the second image to the second intermediate image to obtain a fusion image; performing adaptive gamma correction on the fusion image to obtain a third image; performing edge enhancement processing on the third image by an edge enhancement algorithm to obtain the target image.

7. The construction site image processing method of claim 1, wherein After the target image is obtained, the method further comprises: in response to an execution instruction of a target construction site monitoring task triggered based on the target image, obtaining a model pool constructed in advance; wherein the model pool is used to store an algorithm model corresponding to each construction site monitoring task; calling a target model corresponding to the target construction site monitoring task from the model pool; processing the target image by using the target model to obtain a processing result; generating a monitoring report of the target construction site monitoring task according to the processing result; encrypting and sending the monitoring report to a triggerer of the execution instruction.

8. A construction site image processing apparatus characterized by comprising: The construction site image processing device comprises: an identification unit configured to identify a non-target object in a target construction site image in response to a processing instruction of the target construction site image to obtain an identification result; a generation unit configured to generate a mask in the target construction site image according to the identification result; an erasing unit configured to perform erasing processing on the target construction site image according to the mask to obtain a first intermediate image with a blank area; a completion unit configured to perform content completion on the blank area in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image; a restoration unit configured to perform detail restoration processing on the second intermediate image based on spatial semantic information of a building information model to obtain a target image.

9. A computer device, comprising: The computer device comprises: a memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the construction site image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the construction site image processing method according to any one of claims 1 to 7.

Images