Construction site image processing method, device, equipment and medium

Abstract

The present application relates to the technical field of image processing, and provides a construction site image processing method, device, equipment and medium, which can identify non-target objects in a target construction site image, generate a mask in the target construction site image according to the identification result, roughly position the non-target objects, and accurately segment the target according to the positioning, so as to effectively balance the processing efficiency and accuracy; the target construction site image is subjected to erasing processing according to the mask to obtain a first intermediate image with a blank area, the blank area is subjected to content completion based on an environmental perception algorithm to obtain a second intermediate image, which can solve the feature matching failure problem caused by dynamic interference objects and ensure the integrity of the main building structure; the second intermediate image is subjected to detail restoration processing based on the spatial semantic information of the building information model, which can reduce the influence of light changes and shielding interference, so as to generate a high-quality image more suitable for performing a construction site monitoring task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a construction site image processing method, device, equipment and medium. BACKGROUND

[0002] With the accelerated promotion of the digital transformation of the construction industry, intelligent construction monitoring technology based on the fusion of BIM (Building Information Modeling) and computer vision has become a research hotspot. However, the complexity and dynamics of the construction site environment pose great challenges to image processing, and therefore, an efficient construction site image processing method is urgently needed to solve the following technical bottlenecks:

[0003] 1) Feature matching failure caused by dynamic interference.

[0004] Traditional construction monitoring mainly relies on manual image inspection and comparison with BIM rendered images, but dynamic interference such as scaffolding and temporary equipment is common on construction sites. These unstructured background noises can severely damage the accuracy of image feature alignment, resulting in a significant decrease in subsequent change detection accuracy.

[0005] 2) Illumination changes and occlusion interference.

[0006] The lighting conditions (such as shadows and reflections) and local occlusions (such as equipment movement and material stacking) on the construction site can significantly reduce image quality, making it difficult to extract effective features using traditional threshold segmentation algorithms or edge detection algorithms. For example: in low-contrast areas, broken contours are easily produced, directly affecting the reliability of change detection. SUMMARY

[0007] In view of the above, it is necessary to provide a construction site image processing method, device, equipment and medium, aiming to solve the problem of poor construction site image processing effect caused by dynamic interference, illumination changes and occlusion interference.

[0008] A construction site image processing method, the construction site image processing method comprising:

[0009] In response to a processing instruction for a target construction site image, identifying non-target objects in the target construction site image to obtain an identification result;

[0010] Generating a mask in the target construction site image according to the identification result;

[0011] Erasing the target construction site image according to the mask to obtain a first intermediate image with a blank area;

[0012] completing content of the blank area in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image;

[0013] restoring details of the second intermediate image based on spatial semantic information of a building information model to obtain a target image.

[0014] According to the preferred embodiment of the present application, the identifying the non-target object in the target construction site image to obtain an identification result comprises:

[0015] identifying a region to which the non-target object in the target construction site image belongs as the identification result;

[0016] The region to which the non-target object belongs is a rectangular frame for reflecting position information of the non-target object.

[0017] According to the preferred embodiment of the present application, the generating a mask in the target construction site image according to the identification result comprises:

[0018] processing an image in the rectangular frame by using a U-Net model to obtain a probability map of each pixel;

[0019] performing binary conversion on the probability map to obtain the mask.

[0020] According to the preferred embodiment of the present application, the completing content of the blank area in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image comprises:

[0021] extracting multi-scale features of the first intermediate image by using a feature extractor to obtain feature maps of different levels;

[0022] reserving context features in the feature maps except the blank area;

[0023] aggregating the context features to each pixel of the blank area based on a context attention mechanism to obtain aggregated features corresponding to the blank area;

[0024] converting the aggregated features into pixel values to obtain completion features;

[0025] optimizing the completion features to obtain optimized features;

[0026] replacing the blank area in the first intermediate image with the optimized features to obtain the second intermediate image.

[0027] According to the preferred embodiment of the present application, the optimizing the completion features to obtain optimized features comprises:

[0028] extracting an edge feature in the context feature based on an edge detection operator;

[0029] performing smooth connection on the edge of the completion feature and the edge feature with structural consistency loss as a constraint to obtain a first feature;

[0030] eliminating edge jaggies in the first feature to obtain the optimized feature.

[0031] According to a preferred embodiment of the present application, the spatial semantic information based on the building information model performs detail restoration processing on the second intermediate image to obtain a target image, which includes:

[0032] reserving an image region in the second intermediate image that spatially overlaps with the building information model component as a to-be-processed region;

[0033] obtaining an infrared image and a depth image corresponding to the to-be-processed region collected by a sensor;

[0034] obtaining a low-brightness region in the to-be-processed region, and adding an infrared pixel value corresponding to the low-brightness region in the infrared image to the low-brightness region as a supplement to obtain a first image;

[0035] obtaining a textureless region in the to-be-processed region, and supplementing depth information corresponding to the textureless region in the depth image to the textureless region to obtain a second image;

[0036] fusing the first image and the second image to the second intermediate image to obtain a fused image;

[0037] performing adaptive gamma correction on the fused image to obtain a third image;

[0038] performing edge enhancement processing on the third image through an edge enhancement algorithm to obtain the target image.

[0039] According to a preferred embodiment of the present application, after the target image is obtained, the method further includes:

[0040] in response to an execution instruction of a target construction site monitoring task triggered based on the target image, obtaining a model pool constructed in advance; wherein the model pool is used to store an algorithm model corresponding to each construction site monitoring task;

[0041] calling a target model corresponding to the target construction site monitoring task from the model pool;

[0042] processing the target image using the target model to obtain a processing result;

[0043] generating a monitoring report of the target construction site monitoring task according to the processing result;

[0044] encrypting and sending the monitoring report to a trigger of the execution instruction.

[0045] A construction site image processing apparatus, comprising:

[0046] an identification unit configured to identify a non-target object in a target construction site image in response to a processing instruction of the target construction site image, to obtain an identification result;

[0047] a generation unit configured to generate a mask in the target construction site image according to the identification result;

[0048] an erasing unit configured to perform erasing processing on the target construction site image according to the mask, to obtain a first intermediate image with a blank area;

[0049] a completion unit configured to perform content completion on the blank area in the first intermediate image based on an environmental perception algorithm, to obtain a second intermediate image;

[0050] a restoration unit configured to perform detail restoration processing on the second intermediate image based on spatial semantic information of a building information model, to obtain a target image.

[0051] A computer device, comprising:

[0052] a memory configured to store at least one instruction; and

[0053] a processor configured to execute the instruction stored in the memory to implement the construction site image processing method.

[0054] A computer readable storage medium, wherein at least one instruction is stored in the computer readable storage medium, and the at least one instruction is executed by a processor in a computer device to implement the construction site image processing method.

[0055] From the above technical scheme can be seen, the application can identify non-target objects in the target construction site image, and generate a mask in the target construction site image according to the identification result, and then perform rough non-target object positioning, and then perform accurate target segmentation according to the positioning, so as to effectively balance the processing efficiency and accuracy; the target construction site image is erased according to the mask to obtain a first intermediate image with a blank area, the blank area in the first intermediate image is content-completed based on an environmental perception algorithm to obtain a second intermediate image, which can solve the feature matching failure problem caused by dynamic interference objects, and at the same time ensure the integrity of the building main structure; the second intermediate image is subjected to detail restoration processing based on the spatial semantic information of the building information model to obtain a target image, which can reduce the influence of light changes and shielding interference, so as to generate a high-quality image more suitable for performing a construction site monitoring task. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a flowchart of a preferred embodiment of the construction site image processing method of the application;

[0057] Figure 2 is a functional module diagram of a preferred embodiment of the construction site image processing device of the application;

[0058] Figure 3 is a structural schematic diagram of a computer device for implementing a preferred embodiment of the construction site image processing method of the application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be described in detail below with reference to the drawings and specific embodiments.

[0060] As shown in Figure 1 is a flowchart of a preferred embodiment of the construction site image processing method of the application. The order of steps in the flowchart can be changed according to different needs, and some steps can be omitted.

[0061] The construction site image processing method is applied in one or more computer devices, and the computer device is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and the hardware thereof includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0062] The computer device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive Internet Protocol Television (IPTV), a smart wearable device, and the like.

[0063] The computer device can also include a network device and / or a user device. The network device includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

[0064] The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, and the like.

[0065] Artificial intelligence (AI) is a theory, method, technology, and application system that uses digital computers or computer-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain optimal results.

[0066] Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, and the like. Artificial intelligence software technologies mainly include computer vision technology, robotics technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, and the like.

[0067] The network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), and the like.

[0068] S10, in response to a processing instruction of a target construction site image, identifying a non-target object in the target construction site image to obtain an identification result.

[0069] In this embodiment, the processing instruction can be triggered by a safety inspector of the construction site according to actual needs.

[0070] In the embodiment, the target construction site image can be collected by an image collection device deployed inside the construction site, or collected by a mobile device such as a drone.

[0071] In the embodiment, the non-target object can include an interfering object in the construction site that does not belong to the building content.

[0072] For example, the non-target object can include, but is not limited to, one or a combination of the following objects:

[0073] Workers, slogans, garbage, pull-up banners, construction tools, scaffolding, temporary equipment, etc.

[0074] Since the non-target object does not belong to the building content, when it is necessary to perform a safety inspection task by using the construction site image, the inspection result will be adversely affected, and therefore, the non-target object needs to be identified first.

[0075] Specifically, the identification of the non-target object in the target construction site image includes:

[0076] Identifying the area to which the non-target object belongs in the target construction site image as the identification result.

[0077] The area to which the non-target object belongs is a rectangular frame reflecting the position information of the non-target object.

[0078] For example, the area to which the non-target object belongs in the target construction site image can be identified by using a target detection model such as a YOLO (You Only Look Once) model, an SSD (Single Shot MultiBox Detector) model, etc., to obtain a rectangular frame with a class probability.

[0079] Through the above embodiment, the non-target object can be roughly positioned first, and preparation is made for subsequent more refined processing.

[0080] S11, generating a mask in the target construction site image according to the identification result.

[0081] In the embodiment, the generation of the mask in the target construction site image according to the identification result includes:

[0082] Processing the image in the rectangular frame by using a U-Net (Convolutional Networks for Biomedical Image Segmentation) model to obtain a probability map of each pixel;

[0083] The probability map is binarized to obtain the mask.

[0084] The probability map is used to reflect the probability distribution of each pixel belonging to the foreground or other categories.

[0085] Through the above embodiment, after roughly positioning the non-target object, the U-Net model is used to further accurately identify the mask contour of the non-target object, and the pixel-level accurate segmentation of the non-target object is realized. Since the rough positioning process is efficient but low in accuracy, the U-Net model is low in efficiency but high in accuracy, so the combination of the two can effectively balance efficiency and accuracy.

[0086] S12, according to the mask, erasing processing is performed on the target construction site image to obtain a first intermediate image with a blank area.

[0087] In this embodiment, a neural network model or an open-source image processing tool (such as an eraser tool) can be used to perform erasing processing on the target construction site image according to the mask.

[0088] S13, based on an environment perception algorithm, content completion is performed on the blank area in the first intermediate image to obtain a second intermediate image.

[0089] In this embodiment, the content completion of the blank area in the first intermediate image based on the environment perception algorithm to obtain the second intermediate image includes:

[0090] A feature extractor is used to perform multi-scale feature extraction on the first intermediate image to obtain feature maps of different levels;

[0091] The context features in the feature maps except the blank area are retained;

[0092] Based on a context attention mechanism, the context features are aggregated to each pixel of the blank area to obtain an aggregated feature corresponding to the blank area;

[0093] The aggregated feature is converted into a pixel value to obtain a completion feature;

[0094] The completion feature is optimized to obtain an optimized feature;

[0095] The blank area in the first intermediate image is replaced with the optimized feature to obtain the second intermediate image.

[0096] The feature extractor can include, but is not limited to, a convolutional neural network (CNN) and other models with multi-scale feature extraction function.

[0097] The feature map can include hierarchical features such as shallow texture details, deep structure semantics, and the like.

[0098] The context feature can include architectural structure features such as building edges, contours, wall textures, and the like, and can also include architectural component features such as sofas, coffee tables, and the like.

[0099] The context attention mechanism can ensure the similarity of the blank area to surrounding spatial features and semantic features when aggregating.

[0100] The above embodiments can solve the feature matching failure problem caused by dynamic interference based on an environmental perception algorithm, while ensuring the integrity of the main building structure.

[0101] In this embodiment, the optimization processing of the completed feature to obtain the optimized feature includes:

[0102] The edge feature in the context feature is extracted based on an edge detection operator;

[0103] The edges of the completed feature and the edge feature are smoothly connected to obtain a first feature, with structural consistency loss as a constraint;

[0104] The edge sawtooth in the first feature is eliminated to obtain the optimized feature.

[0105] The edge detection operator can be a Canny operator or the like.

[0106] The structural consistency loss can be an L1 norm or an L2 norm between the edges of the completed feature and the edge feature, used to constrain the difference between the two edge features.

[0107] The edge sawtooth in the first feature can be eliminated by morphological operations such as erosion and dilation, or smoothing filter operations such as Gaussian filtering, to make the completed area and the surrounding transition more natural.

[0108] The above embodiments can avoid destroying the original structure of the building (such as wall perpendicularity, roof slope, door and window position, etc.) in the completion result, thereby ensuring the rationality of the completed building structure.

[0109] S14, based on the spatial semantic information of the building information model (BIM), the second intermediate image is processed to restore details to obtain a target image.

[0110] In this embodiment, the spatial semantic information of the building information model is used to process the second intermediate image to restore details to obtain a target image, including:

[0111] reserving an image region in the second intermediate image that spatially overlaps with the building information model component as a to-be-processed region;

[0112] acquiring an infrared image and a depth image corresponding to the to-be-processed region by using a sensor;

[0113] acquiring a low-brightness region in the to-be-processed region, and adding, as a supplement, an infrared pixel value corresponding to the low-brightness region in the infrared image to the low-brightness region to obtain a first image;

[0114] acquiring a textureless region in the to-be-processed region, and supplementing, to the textureless region, depth information corresponding to the textureless region in the depth image to obtain a second image;

[0115] fusing the first image and the second image to the second intermediate image to obtain a fused image;

[0116] performing adaptive gamma correction on the fused image to obtain a third image;

[0117] performing edge enhancement processing on the third image by using an edge enhancement algorithm to obtain the target image.

[0118] The spatial semantic information is used to reflect a component geometric attribute. For example, the spatial semantic information can be a component spatial coordinate.

[0119] The to-be-processed region is used to reflect an intersection in a three-dimensional space between a component (i.e., a geometric body in a three-dimensional space, such as a tea table) in the building information model and a photographed real scene (i.e., the target construction site image).

[0120] The sensor includes an infrared sensor, a depth sensor, and the like.

[0121] The infrared image is used to reflect object heat radiation and is not affected by light, and can make up for a deficiency of visible light in a dark or blocked region.

[0122] By adding, as a supplement, an infrared pixel value corresponding to the low-brightness region in the infrared image to the low-brightness region, the contour visibility of the region can be enhanced.

[0123] The depth image is used to reflect a scene three-dimensional structure, and can provide object contour and distance information and make up for a detail deficiency of visible light in a textureless region. For example, the textureless region can be a smooth wall surface.

[0124] By supplementing, to the textureless region, depth information corresponding to the textureless region in the depth image, contour details can be supplemented by using gradient information of the depth image.

[0125] Through the infrared image and the depth image, information missing of visible light in extreme conditions can be filled in the image.

[0126] The adaptive gamma correction algorithm based on brightness can be used to perform adaptive gamma correction on the fusion image. After the adaptive gamma correction based on brightness, the third image retains the saliency of the infrared target and repairs the light and dark imbalance of the visible light image, so that the overall contrast is more balanced.

[0127] In addition, through adaptive gamma correction for local brightness dynamic adjustment, the problem that global gamma correction cannot simultaneously consider high and low contrast regions can be solved, and detail loss caused by global processing can be avoided.

[0128] The edge enhancement algorithm based on the Canny operator can be used to perform edge enhancement processing on the third image. Through the edge enhancement algorithm, the edge of the object, texture and other details in the low-contrast region can be further enhanced, so that the edge is clearer and the edge continuity is enhanced, thereby improving the recognition degree of details.

[0129] Through the above embodiments, the influence of illumination changes and occlusion interference can be reduced, so that a high-quality image more suitable for performing a construction site monitoring task can be generated.

[0130] In the embodiment, after the target image is obtained, the method further includes:

[0131] In response to an execution instruction of a target construction site monitoring task triggered based on the target image, a pre-constructed model pool is obtained; wherein the model pool is used to store an algorithm model corresponding to each construction site monitoring task;

[0132] A target model corresponding to the target construction site monitoring task is called from the model pool;

[0133] The target image is processed by using the target model to obtain a processing result;

[0134] A monitoring report of the target construction site monitoring task is generated according to the processing result;

[0135] The monitoring report is encrypted and sent to a triggerer of the execution instruction.

[0136] The target construction site monitoring task can include, but is not limited to, one or a combination of the following tasks:

[0137] Crack detection, inclination detection, construction specification detection, construction phase detection, construction safety detection, construction quality detection, and the like.

[0138] The model pool can adopt a "hierarchical storage + index management" architecture. Specifically, the model pool can include a physical storage layer, a metadata management layer, and a task matching layer.

[0139] The physical storage layer is used for actual storage of model files (weights, configurations, and inference code), and supports local server and cloud backup.

[0140] The metadata management layer is used to store description information of the model (such as applicable tasks, precision indicators, and update time), and enables fast retrieval through a database.

[0141] The task matching layer is used to receive construction site monitoring tasks (such as crane posture recognition), call the optimal model matched by the metadata index, and load the model file from the physical storage layer.

[0142] The metadata of the model pool can be stored in a relational database, and is associated through "task type-model ID", supporting multi-dimensional queries.

[0143] The models in the model pool can adopt a hybrid storage strategy, i.e., local storage and cloud storage, and are stored in the same format specification (such as unified model file naming method and unified compression encryption method).

[0144] By constructing the model pool, the model can be quickly called, thereby improving the execution efficiency of the construction site monitoring task.

[0145] The triggerer can be a safety detection personnel, etc.

[0146] Through the above embodiments, high-quality images that have been pre-processed can be used to perform more accurate and rapid task processing.

[0147] In this embodiment, efficient data processing is achieved through an end-to-end hierarchical architecture, which can integrate various processing links in a lightweight manner to achieve a second-level response.

[0148] This embodiment can also synchronously integrate a privacy desensitization module to automatically blur sensitive information such as faces and license plates, thereby improving construction safety and providing a solution for smart construction sites that combines real-time performance and compliance.

[0149] This embodiment can also provide a visual interface, so that relevant staff can view the image processing in real time.

[0150] From the above technical scheme can be seen, the application can identify non-target objects in the target construction site image, and generate a mask in the target construction site image according to the identification result, first roughly locate the non-target objects, and then accurately segment the target according to the location, which can effectively balance the processing efficiency and accuracy; the target construction site image is erased according to the mask to obtain a first intermediate image with a blank area, the content of the blank area in the first intermediate image is completed based on an environmental perception algorithm to obtain a second intermediate image, which can solve the feature matching failure problem caused by dynamic interference, and ensure the integrity of the building main structure; the second intermediate image is processed for detail restoration based on the spatial semantic information of the building information model to obtain a target image, which can reduce the influence of light changes and shielding interference, thereby generating a high-quality image more suitable for performing a construction site monitoring task.

[0151] As Figure 2 shown, it is a functional module diagram of a preferred embodiment of the construction site image processing device. The construction site image processing device 11 includes an identification unit 110, a generation unit 111, an erasing unit 112, a completion unit 113, and a restoration unit 114. The module / unit referred to in the application refers to a series of computer program segments that can be executed by a processor and can complete a fixed function, which is stored in a memory. In this embodiment, the functions of each module / unit will be described in detail in the subsequent embodiments.

[0152] The identification unit 110 is configured to identify non-target objects in the target construction site image in response to a processing instruction of the target construction site image, and obtain an identification result.

[0153] The generation unit 111 is configured to generate a mask in the target construction site image according to the identification result.

[0154] The erasing unit 112 is configured to erase the target construction site image according to the mask to obtain a first intermediate image with a blank area.

[0155] The completion unit 113 is configured to complete the content of the blank area in the first intermediate image based on an environmental perception algorithm to obtain a second intermediate image.

[0156] The restoration unit 114 is configured to process the second intermediate image for detail restoration based on the spatial semantic information of the building information model to obtain a target image.

[0157] From the above technical scheme can be seen, the application can identify non-target objects in the target construction site image, and generate a mask in the target construction site image according to the identification result, and then perform rough non-target object positioning, and then perform accurate target segmentation according to the positioning, so that the processing efficiency and accuracy can be effectively balanced; the target construction site image is erased according to the mask to obtain a first intermediate image with a blank area, the content of the blank area in the first intermediate image is completed based on an environmental perception algorithm to obtain a second intermediate image, so that the feature matching failure problem caused by dynamic interference objects can be solved, and the integrity of the building main structure can be ensured; the second intermediate image is processed based on the spatial semantic information of the building information model to obtain a target image, so that the influence of light changes and shielding interference can be reduced, and a high-quality image more suitable for performing a construction site monitoring task can be generated.

[0158] As Figure 3 shown, it is a structural schematic diagram of a computer device of a preferred embodiment of the construction site image processing method.

[0159] The computer device 1 can include a memory 12, a processor 13 and a bus (the arrow in the figure is the bus), and can also include a computer program stored in the memory 12 and executable on the processor 13, such as a construction site image processing program.

[0160] Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1, and does not constitute a limitation on the computer device 1, and the computer device 1 can be a bus type structure or a star type structure, and the computer device 1 can also include more or less other hardware or software, or different component arrangements, for example, the computer device 1 can also include an input / output device, a network access device, etc.

[0161] It should be noted that the computer device 1 is only an example, and other existing or future electronic products, such as those adaptable to the application, should also be included within the protection scope of the application and included by reference.

[0162] The memory 12 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a mobile hard disk of the computer device 1. In other embodiments, the memory 12 can also be an external storage device of the computer device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 12 can include both an internal storage unit and an external storage device of the computer device 1. The memory 12 can be used to store application software and various data installed in the computer device 1, such as codes of a construction site image processing program, etc., and to temporarily store data that has been output or will be output.

[0163] The processor 13 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 13 is a control unit of the computer device 1, which connects various components of the computer device 1 through various interfaces and lines, executes programs or modules stored in the memory 12 (e.g., a construction site image processing program, etc.), and calls data stored in the memory 12 to perform various functions and process data of the computer device 1.

[0164] The processor 13 executes an operating system and various application programs installed in the computer device 1. The processor 13 executes the application programs to implement the steps in each of the above construction site image processing method embodiments, such as Figure 1 the steps shown in the above embodiments.

[0165] By way of example, the computer program can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units can be a series of computer-readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program can be divided into an identification unit 110, a generation unit 111, an erasing unit 112, a completion unit 113, and a restoration unit 114.

[0166] The integrated units in the form of software function modules described above can be stored in a computer-readable storage medium. The software function modules described above are stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute part of the construction site image processing method described in various embodiments of the present application.

[0167] The modules / units integrated in the computer device 1, if implemented in the form of software function modules and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the present application can also be implemented by a computer program to instruct related hardware devices to complete all or part of the processes in the above-described embodiments. The computer program can be stored in a computer-readable storage medium and can implement the steps of each method embodiment when executed by a processor.

[0168] The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0169] Further, the computer-readable storage medium can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; and the data storage area can store data created according to the use of the blockchain node, etc.

[0170] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods. Each data block contains information about a batch of network transactions, and is used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0171] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one straight line is used in the Figure 3 , but it does not mean that there is only one bus or one type of bus. The bus is arranged to enable connection and communication between the memory 12, the at least one processor 13, and the like.

[0172] Although not shown, the computer device 1 can also include a power supply (such as a battery) for powering the various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, so that the power management device can realize functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and the like. The computer device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which will not be described here.

[0173] Further, the computer device 1 can also include a network interface. Optionally, the network interface can include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0174] Optionally, the computer device 1 can further include a user interface, which can be a display, an input unit such as a keyboard, and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, for displaying information processed in the computer device 1 and for displaying a visualized user interface.

[0175] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by the structure.

[0176] Those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the computer device 1, which can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0177] In combination Figure 1 The memory 12 in the computer device 1 stores a plurality of instructions to implement a construction site image processing method, and the processor 13 can execute the plurality of instructions to implement:

[0178] In response to a processing instruction for a target construction site image, identifying non-target objects in the target construction site image to obtain an identification result;

[0179] Generating a mask in the target construction site image according to the identification result;

[0180] Performing erasing processing on the target construction site image according to the mask to obtain a first intermediate image with a blank area;

[0181] Performing content completion on the blank area in the first intermediate image based on an environment perception algorithm to obtain a second intermediate image;

[0182] Performing detail restoration processing on the second intermediate image based on spatial semantic information of a building information model to obtain a target image.

[0183] Specifically, the processor 13 can refer to the description of the specific implementation method of the above instructions in the corresponding embodiments, which will not be described here. Figure 1

[0184] It should be noted that the data involved in the present case are all legally obtained. The non-company software tools or components appearing in the embodiments of the present application are only examples for introduction and do not represent actual use.​

[0185] In several embodiments provided by the present application, it should be understood that the disclosed system, apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. For example, the division of the modules is merely logical function division. In actual implementation, other division manners can be adopted.

[0186] The present application can be used in numerous general purpose or special purpose computer system environments or configurations. Examples include personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0187] The modules described as separate components can or can not be physically separate, and the components shown as modules can or can not be physical units, i.e. can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0188] In addition, the functional modules in each of the embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0189] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0190] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended to include all changes falling within the meaning and scope of the equivalent elements of the claims. Any reference signs in the claims should not be regarded as limiting the claims to which they relate.

[0191] Furthermore, the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural and vice-versa, unless the context clearly requires that. The wording "coupled" can also be interpreted as security connection, indirect connection, direct connection or untethered connection. The term "a" or "an" does not exclude a plurality, and a single factor can fulfil the functions of several factors listed in the claims. The term "one" or "said" does not exclude a plurality and a single factor can fulfil the functions of several factors listed in the claims. Numerous references are made to patent applications of the applicant and other documents which are open to the public. The content of all documents cited is incorporated herein by reference, in particular the documents cited in the description and claims.

[0192] Finally, it should be noted that the above-mentioned embodiments illustrate rather than limit the application, since the skilled person, having the benefit of the present disclosure will be able to devise appropriate modifications and / or alternative constructions of the application without departing from the spirit and scope thereof.

Claims

1. A method for processing images at a construction site, characterized in that, The construction site image processing method includes: In response to a processing instruction for a target construction site image, non-target objects in the target construction site image are identified, and an identification result is obtained; A mask is generated in the target construction site image based on the recognition results; The target construction site image is erased according to the mask to obtain a first intermediate image with blank areas; To obtain a second intermediate image, a second intermediate image is obtained by completing the blank areas in the first intermediate image using an environment-aware algorithm. The process includes: extracting multi-scale features from the first intermediate image using a feature extractor to obtain feature maps at different levels; retaining contextual features in the feature maps excluding the blank areas; aggregating the contextual features to each pixel of the blank area using a contextual attention mechanism to obtain aggregated features corresponding to the blank area; converting the aggregated features into pixel values ​​to obtain completion features; optimizing the completion features to obtain optimized features; and replacing the blank areas in the first intermediate image with the optimized features to obtain the second intermediate image. The second intermediate image is processed to restore details based on the spatial semantic information of the building information model, thus obtaining the target image.

2. The construction site image processing method as described in claim 1, characterized in that, The identification of non-target objects in the target construction site image yields the following identification results: The region in the target construction site image to which the non-target object belongs is identified as the identification result; The region to which the non-target object belongs is a rectangular box used to reflect the position information of the non-target object.

3. The construction site image processing method as described in claim 2, characterized in that, The step of generating a mask in the target construction site image based on the recognition result includes: The image within the rectangle is processed using the U-Net model to obtain a probability map for each pixel; The probability map is binarized to obtain the mask.

4. The construction site image processing method as described in claim 1, characterized in that, The optimization process for the completed features to obtain optimized features includes: Edge features are extracted from the context features based on the edge detection operator; Using structural consistency loss as a constraint, the edges of the completed feature are smoothly connected to the edge feature to obtain the first feature; Eliminate the jagged edges in the first feature to obtain the optimized feature.

5. The construction site image processing method as described in claim 1, characterized in that, The spatial semantic information based on the building information model is used to perform detail restoration processing on the second intermediate image to obtain the target image, which includes: The image region in the second intermediate image that overlaps with the spatial components of the building information model is retained as the region to be processed; Acquire infrared and depth images corresponding to the area to be processed, using sensors; The low-brightness region in the area to be processed is obtained, and the infrared pixel value corresponding to the low-brightness region in the infrared image is added to the low-brightness region as a supplement to obtain the first image; Obtain the textureless region in the area to be processed, and supplement the textureless region with the depth information corresponding to the textureless region in the depth image to obtain the second image; The first image and the second image are merged into the second intermediate image to obtain a merged image; Adaptive gamma correction is performed on the fused image to obtain a third image; The target image is obtained by performing edge enhancement processing on the third image using an edge enhancement algorithm.

6. The construction site image processing method as described in claim 1, characterized in that, After obtaining the target image, the method further includes: In response to the execution command of the target construction site monitoring task triggered based on the target image, a pre-built model pool is obtained; wherein, the model pool is used to store the algorithm model corresponding to each construction site monitoring task; Retrieve the target model corresponding to the target construction site monitoring task from the model pool; The target image is processed using the target model to obtain the processing result; A monitoring report for the target construction site monitoring task is generated based on the processing results; The monitoring report is encrypted and sent to the triggerer of the execution command.

7. A construction site image processing device, characterized in that, The construction site image processing device includes: The recognition unit is used to recognize non-target objects in the target construction site image in response to the processing instruction of the target construction site image, and obtain the recognition result; A generation unit is used to generate a mask in the target construction site image based on the recognition result; The erasing unit is used to erase the target construction site image according to the mask to obtain a first intermediate image with blank areas; The completion unit is used to complete the blank areas in the first intermediate image based on an environment-aware algorithm to obtain a second intermediate image. The process includes: extracting multi-scale features from the first intermediate image using a feature extractor to obtain feature maps at different levels; retaining contextual features in the feature maps excluding the blank areas; aggregating the contextual features to each pixel of the blank area based on a contextual attention mechanism to obtain aggregated features corresponding to the blank area; converting the aggregated features into pixel values ​​to obtain completion features; optimizing the completion features to obtain optimized features; and replacing the blank areas in the first intermediate image with the optimized features to obtain the second intermediate image. The restoration unit is used to perform detail restoration processing on the second intermediate image based on the spatial semantic information of the building information model to obtain the target image.

8. A computer device, characterized in that, The computer device includes: Memory, storing at least one instruction; and The processor executes instructions stored in the memory to implement the construction site image processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in a computer device to implement the construction site image processing method as described in any one of claims 1 to 6.

Images