Building construction image processing method and device, equipment and storage medium

The construction images are collected through high-altitude cameras and used improved target detection models for identification and classification processing, which solves the problem of low efficiency in safety behavior detection in the construction field, and achieves accurate detection of safety behavior of construction personnel and reduces safety accidents.

CN118918389APending Publication Date: 2024-11-08CHINA CONSTR THIRD ENG BUREAU GRP CO LTD

Patent Information

Application Number
CN202411246771.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing management methods in the construction field are inefficient and have safety hazards, making it difficult to accurately detect the safety behavior of construction personnel.

Method used

High-altitude cameras are used to collect construction images, identify and classify the training object detection model, and generate safety behavior detection results, including building an improved object detection model to improve the accuracy of small object recognition, and send safety when dangerous behavior is detected. alarm.

Benefits of technology

It improves the accuracy and supervision efficiency of safety behavior detection of construction personnel, and effectively reduces the frequency of safety accidents in construction sites.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918389A_ABST
    Figure CN118918389A_ABST
Patent Text Reader

Abstract

The invention discloses a building construction image processing method, device and equipment and a storage medium, and the method comprises the steps: collecting a construction image of a building construction site through a high-altitude camera of the building construction site, calling a trained target detection model to carry out the recognition processing of the construction image, and obtaining a target image containing a target object in the construction image, and performing classification processing on the target image to obtain a classification result of the target image, and generating a safety behavior detection result of the building construction site based on the classification result. According to the embodiment of the invention, the trained target detection model is adopted to carry out recognition processing on the construction image shot at high altitude, so that small target objects in the construction image can be recognized, the recognized target objects can be classified, the detection precision of unsafe behaviors of constructors at a high-altitude view angle can be improved, and the detection accuracy of the unsafe behaviors of the constructors at a high-altitude view angle can be improved. The supervision effect of the building construction site is improved, and the occurrence frequency of safety accidents of the building construction site is effectively reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a construction image processing method, device, equipment and storage medium. Background Art

[0002] In recent years, with the rapid development of technologies such as artificial intelligence, big data, and cloud computing, digital technology has made great progress and has been successfully applied in many scenarios. At present, most construction fields are still using a more traditional management method: passive monitoring with manual on-duty supervision for abnormal supervision. This supervision method is extremely inefficient and prone to inadequate supervision, posing safety hazards. In order to improve the management efficiency of the construction field, taking the path of sustainable and high-quality development of digital construction is the inevitable direction for the transformation and upgrading of the construction industry.

[0003] To effectively manage the construction industry, it is necessary to ensure that construction workers on the construction site comply with safety regulations and use personal protective equipment correctly. Therefore, how to accurately detect the safety behavior of construction workers has become the key to effective management of the construction industry. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a construction image processing method, device, equipment and storage medium to accurately detect the safety behavior of construction workers in the construction field.

[0005] In a first aspect, to achieve the above-mentioned purpose, an embodiment of the present invention provides a construction image processing method, comprising:

[0006] Collecting a construction image of the construction site through a high-altitude camera at the construction site;

[0007] Calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image;

[0008] Performing classification processing on the target image to obtain a classification result of the target image;

[0009] Based on the classification result, a safety behavior detection result of the construction site is generated.

[0010] Furthermore, before the step of calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image, the method further includes:

[0011] Constructing an initial target detection model, wherein the initial target detection model is sequentially connected by an input layer, a backbone network, a neck network, a head network, and an output layer, wherein the backbone network is composed of three convolutional layers connected in sequence, a global attention mechanism is introduced after the last convolutional layer of the backbone network, and the head network is composed of three prediction heads;

[0012] A new convolutional layer is added before the three convolutional layers of the backbone network to extract a large feature map from the input image, a new prediction head is added to the head network to process the large-scale variance of the target, and the neck network is replaced with a bidirectional feature pyramid network to obtain an improved target detection model;

[0013] The improved target detection model is trained using a target detection data set until convergence to obtain a trained target detection model, wherein the target detection data set includes multiple first training images and first annotation data, wherein the first training images are construction images of a construction site taken from an aerial perspective, and the first annotation data are data after annotating target objects in the first training images.

[0014] Furthermore, before the step of calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image, the method further includes:

[0015] Based on the trained image segmentation model, the construction image is segmented to obtain a construction area image belonging to the construction area in the construction image;

[0016] The step of calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image includes:

[0017] The trained target detection model is called to perform recognition processing on the construction area image to obtain a target image containing the target object in the construction image.

[0018] Furthermore, the step of classifying the target image to obtain the classification result of the target image includes:

[0019] The trained classification model is called to classify the target image to obtain a first classification result of whether the target image belongs to a first category, a second classification result of whether the target image belongs to a second category, and a third classification result of whether the target image belongs to a third category.

[0020] Furthermore, the step of generating the safety behavior detection result of the construction site based on the classification result includes:

[0021] When the first classification result is that the target image belongs to the first category, and the second classification result and / or the third classification result is that the target image belongs to the corresponding category, a detection result indicating the presence of dangerous construction behavior at the construction site is generated.

[0022] Furthermore, before the step of calling the trained classification model to classify the target image to obtain a first classification result of whether the target image belongs to the first category, a second classification result of whether the target image belongs to the second category, and a third classification result of whether the target image belongs to the third category, the method further includes:

[0023] Acquire a plurality of images to be processed, wherein the images to be processed are construction images of a construction site taken from a high altitude;

[0024] Calling the trained target detection model to perform recognition processing on the image to be processed to obtain a second training image containing the target object in the image to be processed;

[0025] Annotating a target category to which a target object in the second training image belongs to obtain second annotation data, wherein the target category includes the first category, the second category, and the third category;

[0026] The second training image and the corresponding second labeled data are used to train the classification model to be trained until convergence, thereby obtaining a trained classification model.

[0027] Furthermore, the method further comprises:

[0028] When the detection result indicates that dangerous construction behavior exists at the construction site, a safety alarm message is generated and sent to a corresponding safety manager.

[0029] In a second aspect, in order to solve the same technical problem, an embodiment of the present invention provides a construction image processing device, comprising:

[0030] A collection module, used to collect construction images of the construction site through a high-altitude camera at the construction site;

[0031] A recognition module, used to call the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image;

[0032] A classification module, used to classify the target image to obtain a classification result of the target image;

[0033] A detection module is used to generate a safety behavior detection result of the construction site based on the classification result.

[0034] In the third aspect, in order to solve the same technical problem, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the memory is coupled to the processor, and when the processor executes the computer program, the steps in the construction image processing method described in any one of the above items are implemented.

[0035] In the fourth aspect, in order to solve the same technical problem, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to perform the steps in any one of the above-mentioned construction image processing methods.

[0036] The embodiments of the present invention provide a construction image processing method, device, equipment and storage medium. The method can identify small target objects in the construction images by using a trained target detection model to identify and process the construction images taken from a high altitude, so as to facilitate classification and processing of the identified target objects, thereby improving the detection accuracy of unsafe behaviors of construction workers from a high-altitude perspective, improving the supervision effect of construction sites, and effectively reducing the frequency of safety accidents at construction sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of a construction image processing method provided by an embodiment of the present invention;

[0038] Figure 2 It is a flowchart of a method for constructing and training a trained target detection model provided by an embodiment of the present invention;

[0039] Figure 3a is a schematic diagram of a model structure of an initial target detection model provided by an embodiment of the present invention;

[0040] Figure 3b is a schematic diagram of a model structure of an improved target detection model provided by an embodiment of the present invention;

[0041] Figure 4 is a schematic diagram of a construction area image belonging to a construction area in the construction image provided by an embodiment of the present invention;

[0042] Figure 5 is a structural schematic diagram of a construction image processing device provided by an embodiment of the present invention;

[0043] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present invention;

[0044] Figure 7 This is another structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0047] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0048] In the relevant technologies, most construction fields still use a more traditional management method: passive monitoring with manual supervision for abnormal supervision. This supervision method is extremely inefficient and prone to inadequate supervision, which poses a safety hazard. In order to improve the management efficiency in the construction field, taking the path of sustainable and high-quality development of digital construction is the inevitable direction for the transformation and upgrading of the construction industry.

[0049] To effectively manage the construction industry, it is necessary to ensure that construction workers on the construction site comply with safety regulations and use personal protective equipment correctly. Therefore, how to accurately detect the safety behavior of construction workers has become the key to effective management of the construction industry.

[0050] In order to solve the technical problems existing in the related art, this embodiment provides a construction image processing method, see Figure 1 , Figure 1 is a flow chart of a construction image processing method provided by an embodiment of the present invention. Figure 1 As shown, the construction image processing method provided by the embodiment of the present invention includes steps 101 to 104, which are specifically as follows:

[0051] Step 101: collect a construction image of the construction site through a high-altitude camera at the construction site.

[0052] In this embodiment, in order to improve the management efficiency of the construction site, this embodiment uses a high-altitude camera installed at the construction site to collect construction images of the construction site, thereby reducing the number of times the construction images of the construction site are collected, and further effectively improving the efficiency of safety behavior detection at the construction site.

[0053] The high-altitude camera provided in this embodiment can be set on an object with a preset height in or around the construction site, such as a tower crane at the construction site, a high-rise building with a preset height around the construction site, or a drone with a preset flight height, etc., as long as the camera's shooting angle is at the preset height. The preset height can be set according to the image captured by the high-altitude camera, as long as the resolution of the construction personnel in the captured image can be less than 50*50 pixels, and the specific height value of the preset height is not limited here.

[0054] Step 102 , calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image.

[0055] The target object provided in this embodiment may be a construction worker, or a designated person or object to be identified. The target detection model provided in this embodiment may be a YOLO target detection model, such as a TPH-YOLOv5 model, a PP-YOLOE-SOD model, or other target detection models.

[0056] In some embodiments, in order to improve the efficiency of safety behavior detection at construction sites, this embodiment can call the trained target detection model to identify the construction images taken by the high-altitude camera to identify the construction workers in the construction images, and judge whether each construction worker complies with the safety regulations and uses personal protective equipment correctly. In this way, the recognition efficiency of the construction image recognition process can be effectively improved, thereby improving the safety behavior detection efficiency of the construction site.

[0057] In another embodiment, due to the small proportion of pixels of personnel in the image from a high-altitude perspective, the pixel proportion of personal protective equipment such as helmets and work clothes is even smaller. In this case, using a simple image recognition method to identify construction images is prone to missed detections or false detections, and thus cannot effectively ensure the safe operation of construction workers in construction sites. Therefore, in order to solve this technical problem, this embodiment may pre-build a target detection model and train it before the step of calling the trained target detection model, identifying and processing the construction image, and obtaining a target image containing the target object in the construction image, so that the target detection model can accurately identify small targets in the construction image, which are mainly construction workers with an area smaller than 50*50 pixels in the construction image. For details, please refer to Figure 2 , Figure 2 is a flow chart of a method for constructing and training a trained target detection model provided by an embodiment of the present invention, such as Figure 2 As shown, it includes steps 201 to 203;

[0058] Step 201, construct an initial target detection model, which is composed of an input layer, a backbone network, a neck network, a head network and an output layer connected in sequence, the backbone network is composed of three convolutional layers connected in sequence, a global attention mechanism is introduced after the last convolutional layer of the backbone network, and the head network is composed of three prediction heads.

[0059] In this embodiment, the initial target detection model provided in this embodiment is the PP-YOLOE-SOD model. For details, see Figure 3a , Figure 3a is a schematic diagram of a model structure of an initial target detection model provided by an embodiment of the present invention, such as Figure 3a As shown, the initial target detection model includes a backbone network, a neck network, a head network and an output layer (not shown in the figure).

[0060] Among them, the backbone network is mainly used to extract features from the input image to obtain a feature map, specifically including three convolution layers C3, C4, and C5 to form a multi-level pyramid structure, so that feature extraction through the multi-level pyramid structure can increase the fusion of features of different scales and improve the detection effect. At the same time, the use of a grouped convolution structure can not only reduce the amount of calculation, but also improve the detection speed; the head network is mainly used to predict the feature map and transmit the prediction results to the output layer for output, specifically including three prediction heads P3, P4, and P5; the neck network is mainly placed between the backbone network and the head network to better utilize the features extracted by the backbone network. In addition, the Transformer global attention mechanism is introduced after the last convolution layer C5 of the backbone network, so that the model can pay more attention to the important parts of the object.

[0061] In order to further improve the recognition accuracy of small targets at a long-distance high-altitude perspective, this embodiment can improve the initial target detection model provided in the above embodiment so that the improved target detection model can accurately identify the specified small target at a long-distance high-altitude perspective. Specifically, the method of improving the initial target detection model is shown in step 202:

[0062] Step 202, adding a new convolutional layer before the three convolutional layers of the backbone network to extract a large feature map from the input image, adding a new prediction head in the head network to process the large-scale variance of the target, and replacing the neck network with a bidirectional feature pyramid network to obtain an improved target detection model.

[0063] For details, see Figure 3b , Figure 3b is a schematic diagram of a model structure of an improved target detection model provided by an embodiment of the present invention, such as Figure 3b As shown, the improved target detection model also includes a backbone network, a neck network, a head network and an output layer (not shown in the figure), but the backbone network of the improved target detection model has 4 convolutional layers: C2, C3, C4, and C5, which has an additional convolutional layer C2 compared to the initial target detection model; the head network of the improved target detection model has 4 prediction heads, which has an additional prediction head P2 compared to the initial target detection model; the neck network is replaced by a bidirectional feature pyramid network, such as a BiFPN network.

[0064] Through the above improvements to the initial target detection model, the improved target detection model has the following advantages over the initial target detection model: the backbone network extracts and fuses the P2 small target detection head from the C2 convolutional layer, so that the model output layer is four heads P2, P3, P4, and P5, so that the 160x160 P2 large feature map can more keenly detect and locate small targets, thereby improving the accuracy of small target detection; the neck network is replaced with a BiFPN network. The introduction of the BiFPN network allows information to propagate bidirectionally between different resolution levels, better integrating multi-scale information, and helping the model to more comprehensively understand targets of different sizes and improve the detection performance of multi-scale objects. At the same time, it also improves the contextual understanding of objects and reduces false positives and negatives.

[0065] Step 203, use the target detection data set to train the improved target detection model until convergence to obtain a trained target detection model, the target detection data set includes multiple first training images and first annotation data, the first training image is a construction image of the construction site taken from a high altitude, and the first annotation data is data after the target object in the first training image is annotated.

[0066] After obtaining the improved target detection model provided in the above embodiment, this embodiment can use the training data set taken from high altitude, that is, the target detection data set to train the improved target detection model until convergence, so as to obtain the trained target detection model.

[0067] In one case, the method for judging the convergence of the improved target detection model in this embodiment can be: when the number of training times of the improved target detection model reaches a preset number, such as 50,000 times, the improved target detection model is determined to be converged. In other cases, the method for judging the convergence of the improved target detection model in this embodiment can also be: when the recognition accuracy of the improved target detection model for the target object reaches a preset accuracy, such as 98%, the improved target detection model is determined to be converged.

[0068] At this point, the construction and training methods of the trained object detection model are described in detail.

[0069] As an optional embodiment, in order to further improve the efficiency of safety behavior detection on the construction site, this embodiment can also only identify the construction area in the construction site. Therefore, before the step of calling the trained target detection model, identifying the construction image, and obtaining the target image containing the target object in the construction image, the construction image processing method provided by this embodiment can also include: based on the trained image segmentation model, segmenting the construction image to obtain the construction area image belonging to the construction area in the construction image.

[0070] Among them, the trained image segmentation model can be obtained based on the DeepLab image segmentation model training, and the specific training process can be: collecting construction training images of the construction site through a high-altitude camera, and marking the construction areas in the construction training images to generate a construction area data set. Then, the DeepLab image segmentation model to be trained is trained based on the construction area data set until convergence, so as to obtain the trained image segmentation model provided in this embodiment. In this way, the trained image segmentation model can be used to segment the construction image to obtain a mask area belonging to the construction area, and then the mask area is morphologically processed to screen out the effective construction area, and the effective construction area is polygonally fitted to obtain a polygonal construction area, as shown in the following figure. Figure 4 As shown, Figure 4 It is a schematic diagram of a construction area image belonging to a construction area in the construction image provided by an embodiment of the present invention.

[0071] After using the trained image segmentation model provided in the above embodiment to segment the construction image and obtain the construction area image belonging to the construction area in the construction image, the step of calling the trained target detection model provided in this embodiment to perform recognition processing on the construction image and obtain the target image containing the target object in the construction image may include: calling the trained target detection model to perform recognition processing on the construction area image and obtain the target image containing the target object in the construction image.

[0072] By calling the trained target detection model to perform recognition processing on the target image containing the target object in the construction image, the amount of calculation required for recognition processing can be reduced, thereby improving the efficiency of safety behavior detection on the construction site.

[0073] Step 103: classify the target image to obtain a classification result of the target image.

[0074] In this embodiment, after obtaining the target image containing the target object, this embodiment can classify the target image to determine the classification result of each target image, so as to facilitate the subsequent safety behavior detection processing of the construction site based on the classification result. Specifically, the step of classifying the target image and obtaining the classification result of the target image provided in this embodiment can include: calling the trained classification model, classifying the target image, and obtaining a first classification result of whether the target image belongs to the first category, a second classification result of whether the target image belongs to the second category, and a third classification result of whether the target image belongs to the third category.

[0075] Among them, the first category provided in this embodiment is used to determine whether the target object in the target image is a person, the second category is used to determine whether the target object in the target image is wearing a safety helmet, and the third category is used to determine whether the target object in the target image is wearing work clothes.

[0076] Specifically, the trained classification model provided in this embodiment may include a backbone network, an avgpool average pooling layer, and a fully connected layer. Specifically, the backbone network of the trained classification model may adopt a MobileNet structure. After the avgpool average pooling layer, the fully connected layer connected after the pooling layer is divided into three fully connected layers, fc1, fc2, and fc3, to respectively determine whether the target object in the target image is a person, whether it is wearing a safety helmet, and whether it is wearing work clothes. At the same time, the loss function of the trained classification model provided in this embodiment is a combination of CrossEntropyLoss of LabelSmooth, which can reduce the high confidence influence brought by Softmax, allowing the model to focus on the weights of low probability distributions, thereby reducing overfitting.

[0077] As an optional embodiment, the present embodiment may call upon the target detection model trained in the above-mentioned embodiment to perform recognition processing on the construction images of the construction site taken from the aerial perspective to obtain a second training image containing the target object, thereby using the second training image to train the classification model to be trained until convergence to obtain the trained classification model of the present embodiment. That is, before the step of calling the trained classification model, classifying the target image, and obtaining a first classification result of whether the target image belongs to the first category, a second classification result of whether the target image belongs to the second category, and a third classification result of whether the target image belongs to the third category, the construction image processing method provided in this embodiment may also include: acquiring multiple images to be processed, wherein the images to be processed are construction images of the construction site taken from a high altitude; calling the trained target detection model, identifying the images to be processed, and obtaining a second training image containing the target object in the image to be processed; labeling the target category to which the target object in the second training image belongs, and obtaining second labeling data, wherein the target category includes the first category, the second category, and the third category; using the second training image and the corresponding second labeling data, training the classification model to be trained until convergence, and obtaining a trained classification model.

[0078] Among them, the method of judging whether the classification model to be trained has converged in this embodiment can be found in the embodiment corresponding to the method of judging whether the improved target detection model has converged, which will not be repeated here.

[0079] Step 104: generating a safety behavior detection result of the construction site based on the classification result.

[0080] After obtaining the classification result of the target image, the safety behavior detection result of the construction site can be generated accordingly based on the category to which the target image belongs. Among them, the safety behavior detection result includes a safety result and a detection result of dangerous construction behavior. Specifically, since the trained classification model provided in this embodiment includes three fully connected layers fc1, fc2, and fc3, the three fully connected layers fc1, fc2, and fc3 are respectively used to determine whether the target object in the target image is a person, whether it is wearing a safety helmet, and whether it is wearing work clothes. Therefore, if the classification result is: fc1 output is a person, and fc2 output is not wearing a safety helmet and / or fc3 output is not wearing work clothes, then a detection result of dangerous construction behavior is generated; otherwise, a safety result is generated. That is, the step of generating the safety behavior detection result of the construction site based on the classification result provided in this embodiment can be: when the first classification result is that the target image belongs to the first category, and the second classification result and / or the third classification result is that the target image belongs to the corresponding category, a detection result of dangerous construction behavior is generated at the construction site.

[0081] As an optional embodiment, after generating the detection result of the existence of dangerous construction behavior at the construction site, the construction image processing method provided by this embodiment may also include: generating safety alarm information when the detection result is the detection result of the existence of dangerous construction behavior at the construction site, and sending the safety alarm information to the corresponding safety manager. The safety alarm information may be text or voice alarm information indicating the existence of dangerous construction behavior at the construction site, or may be alarm information including a target image that triggers the generation of the detection result of the existence of dangerous construction behavior, so that the safety manager can quickly find the target object in the target image to remind him to comply with safety regulations and wear work clothes and hats correctly, thereby effectively reducing the frequency of safety accidents at the construction site.

[0082] In summary, the construction image processing method provided in this embodiment includes collecting construction images of a construction site through a high-altitude camera of the construction site, calling a trained target detection model to identify and process the construction image, obtaining a target image containing a target object in the construction image, classifying and processing the target image, obtaining a classification result of the target image, and generating a safety behavior detection result of the construction site based on the classification result. Using the embodiment of the present invention, by using a trained target detection model to identify and process construction images taken from a high altitude, small target objects in the construction image can be identified, so as to facilitate the classification and processing of the identified target objects, thereby improving the detection accuracy of unsafe behaviors of construction personnel from a high-altitude perspective, improving the supervision effect of the construction site, and effectively reducing the frequency of safety accidents at the construction site.

[0083] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a construction image processing device. The construction image processing device can be implemented as an independent entity or integrated into an electronic device, such as a terminal. The terminal may include a mobile phone, a tablet computer, etc.

[0084] In order to solve the same technical problem, this embodiment also provides a construction image processing device. Figure 5 , Figure 5 is a schematic diagram of a structure of a construction image processing device provided by an embodiment of the present invention, such as Figure 5 As shown, the construction image processing device 500 specifically includes: a collection module 501, a recognition module 502, a classification module 503 and a detection module 504;

[0085] The acquisition module 501 is used to acquire the construction image of the construction site through a high-altitude camera of the construction site;

[0086] The recognition module 502 is used to call the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image;

[0087] The classification module 503 is used to classify the target image to obtain a classification result of the target image;

[0088] The detection module 504 is used to generate a safety behavior detection result of the construction site based on the classification result.

[0089] In some embodiments, the construction image processing device 500 provided in this embodiment further includes a construction module, an improvement module and a first training module;

[0090] Among them, the construction module is used to construct an initial target detection model, which is sequentially connected by an input layer, a backbone network, a neck network, a head network and an output layer. The backbone network is composed of three convolutional layers connected in sequence. A global attention mechanism is introduced after the last convolutional layer of the backbone network, and the head network is composed of three prediction heads;

[0091] An improvement module is used to add a new convolution layer before the three convolution layers of the backbone network to extract a large feature map from the input image, add a new prediction head in the head network to process the large-scale variance of the target, and replace the neck network with a bidirectional feature pyramid network to obtain an improved target detection model;

[0092] The first training module is used to use a target detection data set to train the improved target detection model until convergence to obtain a trained target detection model, wherein the target detection data set includes multiple first training images and first annotation data, wherein the first training image is a construction image of a construction site taken from an aerial perspective, and the first annotation data is data after annotating the target object in the first training image.

[0093] In some embodiments, the construction image processing device 500 provided in this embodiment also includes a segmentation module, which is used to perform segmentation processing on the construction image based on a trained image segmentation model to obtain a construction area image belonging to the construction area in the construction image.

[0094] The recognition module 502 is specifically used to call the trained target detection model to perform recognition processing on the construction area image to obtain a target image containing the target object in the construction image.

[0095] In some embodiments, the classification module 503 provided in this embodiment is specifically used to call the trained classification model to perform classification processing on the target image to obtain a first classification result of whether the target image belongs to the first category, a second classification result of whether the target image belongs to the second category, and a third classification result of whether the target image belongs to the third category.

[0096] In some embodiments, the detection module 504 provided in this embodiment is specifically used to generate a detection result that indicates the presence of dangerous construction behavior at the construction site when the first classification result is that the target image belongs to the first category, and the second classification result and / or the third classification result is that the target image belongs to the corresponding category.

[0097] Optionally, the construction image processing device 500 provided in this embodiment further includes an acquisition module, a call module, a labeling module and a second training module;

[0098] The acquisition module is used to acquire a plurality of images to be processed, wherein the images to be processed are construction images of a construction site taken from a high altitude;

[0099] The calling module is used to call the trained target detection model to perform recognition processing on the image to be processed to obtain a second training image containing the target object in the image to be processed;

[0100] The labeling module is used to label the target category to which the target object in the second training image belongs to obtain second labeling data, wherein the target category includes the first category, the second category and the third category;

[0101] The second training module is used to use the second training image and the corresponding second annotation data to train the classification model to be trained until convergence, so as to obtain a trained classification model.

[0102] In some embodiments, the construction image processing device 500 provided in this embodiment also includes an alarm module, which is used to generate safety alarm information when the detection result is a detection result that dangerous construction behavior exists at the construction site, and send the safety alarm information to the corresponding safety management personnel.

[0103] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0104] Also, see Figure 6 , Figure 6 1 is a schematic diagram of a structure of an electronic device provided by an embodiment of the present invention, and the electronic device may be a mobile terminal such as a smart phone, a tablet computer, or the like. Figure 6 As shown, the electronic device 600 includes a processor 601 and a memory 602. The processor 601 is electrically connected to the memory 602.

[0105] The processor 601 is the control center of the electronic device 600. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device 600 and processes data by running or loading applications stored in the memory 602 and calling data stored in the memory 602, thereby monitoring the electronic device 600 as a whole.

[0106] In this embodiment, the processor 601 in the electronic device 600 will load the instructions corresponding to the processes of one or more applications into the memory 602 according to the following steps, and the processor 601 will run the applications stored in the memory 602, thereby realizing the various functions that can be realized in the above embodiments.

[0107] The electronic device 600 can implement the steps in any embodiment of the construction image processing method provided in the embodiments of the present invention. Therefore, it can achieve the beneficial effects that can be achieved by any construction image processing method provided in the embodiments of the present invention. Please refer to the previous embodiments for details and will not be repeated here.

[0108] See also Figure 7 , Figure 7is another structural schematic diagram of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, Figure 7 The electronic device 700 is a block diagram of a specific structure of an electronic device provided by an embodiment of the present invention, and the electronic device can be used to implement the construction image processing method provided in the above embodiment. The electronic device 700 can be a mobile terminal such as a smart phone or a laptop computer.

[0109] The RF circuit 710 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 710 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 710 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0110] The memory 720 can be used to store software programs and modules, such as the program instructions / modules corresponding to the construction image processing method in the above-mentioned embodiment. The processor 780 executes various functional applications and construction image processing by running the software programs and modules stored in the memory 720, that is, realizes the various functions that can be realized in the above-mentioned embodiment.

[0111] The memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 720 may further include a memory remotely arranged relative to the processor 780, and these remote memories may be connected to the electronic device 700 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0112] The input unit 730 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 730 may include a touch-sensitive surface 731 and other input devices 732. The touch-sensitive surface 731, also known as a touch display screen or touch pad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 731), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 731 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 780, and can receive and execute commands sent by the processor 780. In addition, the touch-sensitive surface 731 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 731, the input unit 730 may further include other input devices 732. Specifically, the other input devices 732 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0113] The display unit 740 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device 700, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 740 may include a display panel 741, and optionally, the display panel 741 may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 731 may cover the display panel 741, and when the touch-sensitive surface 731 detects a touch operation on or near it, it is transmitted to the processor 780 to determine the type of the touch event, and then the processor 780 provides corresponding visual output on the display panel 741 according to the type of the touch event. Although in the figure, the touch-sensitive surface 731 and the display panel 741 are implemented as two independent components to implement input and output functions, in some embodiments, the touch-sensitive surface 731 and the display panel 741 can be integrated to implement input and output functions.

[0114] The electronic device 700 may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 741 according to the brightness of the ambient light, and the proximity sensor may generate an interrupt when the flip cover is closed or closed. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured in the electronic device 700, they will not be repeated here.

[0115] The audio circuit 760, the speaker 761, and the microphone 762 can provide an audio interface between the user and the electronic device 700. The audio circuit 760 can transmit the electrical signal converted from the received audio data to the speaker 761, which is converted into a sound signal for output; on the other hand, the microphone 762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 760 and converted into audio data, and then the audio data is output to the processor 780 for processing, and then sent to another terminal through the RF circuit 710, or the audio data is output to the memory 720 for further processing. The audio circuit 760 may also include an earplug jack to provide communication between an external headset and the electronic device 700.

[0116] The electronic device 700 can help the user receive requests, send information, etc. through the transmission module 770 (such as a Wi-Fi module), which provides the user with wireless broadband Internet access. Although the transmission module 770 is shown in the figure, it is understandable that it is not a necessary component of the electronic device 700 and can be omitted as needed without changing the essence of the invention.

[0117] The processor 780 is the control center of the electronic device 700. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 720, and calling data stored in the memory 720, it executes various functions of the electronic device 700 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 780 may include one or more processing cores; in some embodiments, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 780.

[0118] The electronic device 700 also includes a power supply 790 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 780 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 790 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0119] Although not shown, the electronic device 700 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors to include various functions that can be implemented in the above embodiments.

[0120] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0121] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, which stores multiple instructions, and the instructions can be loaded by a processor to execute the steps of any embodiment of the construction image processing method provided in the embodiment of the present invention.

[0122] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0123] Since the instructions stored in the storage medium can execute the steps in any embodiment of the construction image processing method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any construction image processing method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0124] The above is a detailed introduction to a construction image processing method, device, equipment and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application. Moreover, for those of ordinary skill in the art, without departing from the principles of the present invention, several improvements and modifications can be made, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A construction image processing method, characterized in that: include: Collecting a construction image of the construction site through a high-altitude camera at the construction site; Calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image; Performing classification processing on the target image to obtain a classification result of the target image; Based on the classification result, a safety behavior detection result of the construction site is generated.

2. The construction image processing method according to claim 1, characterized in that: Before the step of calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image, the method further includes: Constructing an initial target detection model, wherein the initial target detection model is sequentially connected by an input layer, a backbone network, a neck network, a head network, and an output layer, wherein the backbone network is composed of three convolutional layers connected in sequence, a global attention mechanism is introduced after the last convolutional layer of the backbone network, and the head network is composed of three prediction heads; A new convolutional layer is added before the three convolutional layers of the backbone network to extract a large feature map from the input image, a new prediction head is added to the head network to process the large-scale variance of the target, and the neck network is replaced with a bidirectional feature pyramid network to obtain an improved target detection model; The improved target detection model is trained using a target detection data set until convergence to obtain a trained target detection model, wherein the target detection data set includes multiple first training images and first annotation data, wherein the first training images are construction images of a construction site taken from an aerial perspective, and the first annotation data are data after annotating target objects in the first training images.

3. The construction image processing method according to claim 1, characterized in that: Before the step of calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image, the method further includes: Based on the trained image segmentation model, the construction image is segmented to obtain a construction area image belonging to the construction area in the construction image; The step of calling the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image includes: The trained target detection model is called to perform recognition processing on the construction area image to obtain a target image containing the target object in the construction image.

4. The construction image processing method according to claim 1, characterized in that: The step of performing classification processing on the target image to obtain the classification result of the target image includes: The trained classification model is called to classify the target image to obtain a first classification result of whether the target image belongs to a first category, a second classification result of whether the target image belongs to a second category, and a third classification result of whether the target image belongs to a third category.

5. The construction image processing method according to claim 4, characterized in that: The step of generating the safety behavior detection result of the construction site based on the classification result includes: When the first classification result is that the target image belongs to the first category, and the second classification result and / or the third classification result is that the target image belongs to the corresponding category, a detection result indicating the presence of dangerous construction behavior at the construction site is generated.

6. The construction image processing method according to claim 4, characterized in that: Before the step of calling the trained classification model to classify the target image to obtain a first classification result of whether the target image belongs to the first category, a second classification result of whether the target image belongs to the second category, and a third classification result of whether the target image belongs to the third category, the method further includes: Acquire a plurality of images to be processed, wherein the images to be processed are construction images of a construction site taken from a high altitude; Calling the trained target detection model to perform recognition processing on the image to be processed to obtain a second training image containing the target object in the image to be processed; Annotating a target category to which a target object in the second training image belongs to obtain second annotation data, wherein the target category includes the first category, the second category, and the third category; The second training image and the corresponding second labeled data are used to train the classification model to be trained until convergence, thereby obtaining a trained classification model.

7. The construction image processing method according to claim 5, characterized in that: The method further comprises: When the detection result indicates that dangerous construction behavior exists at the construction site, a safety alarm message is generated and sent to a corresponding safety manager.

8. A construction image processing device, characterized in that: include: A collection module, used to collect construction images of the construction site through a high-altitude camera at the construction site; A recognition module, used to call the trained target detection model to perform recognition processing on the construction image to obtain a target image containing the target object in the construction image; A classification module, used to classify the target image to obtain a classification result of the target image; A detection module is used to generate a safety behavior detection result of the construction site based on the classification result.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the memory is coupled to the processor, and when the processor executes the computer program, the steps in the construction image processing method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the steps in the construction image processing method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Multi-scale attention-fused traffic helmet small target detection system and method

    CN116665156A

  • Logistics safe driving detection method and device based on lightweight improved YOLOv5

    CN117612074A

  • Construction monitoring method and device, electronic equipment and machine readable storage medium

    CN117690092A

  • Violation detection method, system, and device for building construction site

    WO2020056677A1

Cited By

  • Gas leakage detection method, device, equipment and medium

    CN119958769A