Construction site multiple unsafe behavior intelligent identification system based on image recognition

Abstract

The application relates to the technical field of image recognition, and discloses a construction site multiple unsafe behavior intelligent recognition system based on image recognition; an image labeling classification module is arranged, analog behavior data is divided into behavior images and behavior labels; an image weight fusion module is arranged, channel attention weight and spatial attention weight are fused with the behavior images; a model training compression module is arranged, an original model is compressed and optimized into an image behavior recognition model; a real-time image recognition module is arranged, and real-time behavior labels are recognized; the application can give corresponding light attention mechanisms in the recognition process of the image behavior recognition model, ensures that the image behavior recognition model is more focused on feature recognition in small target and key feature areas when intelligently recognizing mechanism behavior images, suppresses noise interference caused by a complex environment background, and effectively improves the recognition precision of the image behavior recognition model on small targets and unsafe behaviors in a complex background.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to an intelligent recognition system for various unsafe behaviors at construction sites based on image recognition. Background Technology

[0002] As a pillar industry of the national economy, the construction industry is characterized by labor-intensive, complex, and dynamic construction processes. Unsafe behaviors of construction site personnel have always been a major factor contributing to the high incidence of safety accidents. Therefore, timely and accurate monitoring and early warning of unsafe behaviors of construction site personnel are crucial for preventing and reducing safety accidents.

[0003] The patent application with publication number CN120599519A discloses a method and system for detecting unsafe acts at construction sites based on image recognition. It improves the detection accuracy of unsafe acts at construction sites by combining Mosaic data augmentation and adaptive anchor box generation algorithm to train the YOLOv5 model. It also achieves accurate tracking of the behavior trajectory of construction personnel through the DeepSORT algorithm and extracts 128-dimensional feature vectors for subsequent behavior analysis and risk assessment. This enhances the system's recognition stability and behavior analysis depth in complex environments, thereby effectively supporting the automation and intelligence of construction site safety management.

[0004] When existing intelligent recognition systems identify unsafe behaviors at construction sites, they typically process all feature channels and spatial locations in key feature areas equally. However, when complex environmental backgrounds appear in key feature areas, these backgrounds can cause noise interference to the final recognition results. Consequently, the recognition model cannot accurately identify features in small targets and key feature areas based on the different importance of feature channels and spatial locations, thus reducing the recognition accuracy of unsafe behaviors in small targets and complex backgrounds.

[0005] In view of this, the present invention proposes an intelligent identification system for various unsafe behaviors at construction sites based on image recognition to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: an intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition, comprising:

[0007] The image annotation and classification module is used to collect simulated behavioral data of unsafe behaviors at the construction site in a simulated construction scenario, parse the data attributes of the simulated behavioral data, and divide the simulated behavioral data into behavioral images and behavioral annotations.

[0008] The image weight fusion module is used to identify key feature regions in the behavior image, collect the channel attention weights and spatial attention weights of the key feature regions, and fuse the channel attention weights and spatial attention weights with the behavior image to generate a mechanism behavior image.

[0009] The model training compression module is used to combine construction scene and mechanism behavior images into image behavior data, train the original model through the image behavior data, and compress and optimize the original model into an image behavior recognition model.

[0010] The real-time image recognition module is used to combine real-time construction scene and mechanism behavior images into real-time image behavior data. Through the image behavior recognition model, real-time behavior labels are identified, and after determining the construction status of the construction site, corresponding construction prompt information is issued.

[0011] Furthermore, the construction scenario includes weather conditions, lighting conditions, and spatial conditions; the data attributes include independent attributes and dependent attributes.

[0012] The method for parsing data attributes is as follows:

[0013] Import the B simulated behavioral data generated from the A simulated construction scenario one by one into the blank dataset to generate a simulated dataset;

[0014] At the same time, extract the metadata of B simulated behavioral data from the simulated dataset, record the metadata with the annotation word "structure" as the target metadata, and query the structure text of B target metadata one by one;

[0015] When the structure text of the target metadata is a multidimensional sequence, the data attributes are recorded as independent attributes;

[0016] When the structure text of the target metadata is a one-dimensional sequence, the data attributes are recorded as dependent attributes.

[0017] Furthermore, when the data attribute of the simulated behavioral data is an independent attribute, the simulated behavioral data is recorded as a behavioral image, resulting in D behavioral images;

[0018] When the data attribute of the simulated behavioral data is a dependent attribute, the simulated behavioral data is recorded as behavioral labels, and D behavioral labels are obtained.

[0019] Furthermore, when identifying key feature regions, computer vision technology is used to identify construction workers and equipment in the behavioral images and mark the feature points corresponding to the construction workers and equipment.

[0020] Draw the unit rectangle in the behavior image with the geometric center of the behavior image as the intersection of the diagonals of the unit rectangle;

[0021] Continuously adjust the length and width of the unit rectangle until the adjusted unit rectangle first encompasses all feature points. Then, the area inside the adjusted unit rectangle is recorded as the key feature region.

[0022] Furthermore, the method for collecting channel attention weights is as follows:

[0023] The key feature regions are imported into the convolutional neural network for multi-level convolution operations to obtain C feature channels.

[0024] The first average eigenvalue of each of the C feature channels is calculated one by one using the global average pooling formula, and the first maximum eigenvalue of each of the C feature channels is calculated one by one using the global max pooling formula.

[0025] The first average eigenvalue and the first maximum eigenvalue of the C feature channels are concatenated to obtain C first concatenation results. The C first concatenation results are then processed by a multilayer perceptron to obtain C sub-channel weights.

[0026] The average of the weights of the C sub-channels is obtained by summing the weights. The average channel weight is then normalized using the Sigmoid activation function to obtain the channel attention weight.

[0027] Furthermore, the method for collecting spatial attention weights is as follows:

[0028] Channel attention weights are applied to each of the C feature channels to transform key feature regions into enhanced feature regions.

[0029] The second average eigenvalue of the C feature channels in the enhanced feature region is calculated using the global average pooling formula, and the second maximum eigenvalue of the enhanced feature region is calculated one by one using the global max pooling formula.

[0030] The C second average eigenvalues ​​and the second maximum eigenvalues ​​of the enhanced feature region are concatenated on the C feature channels to obtain C second concatenation results. The C second concatenation results are then processed by a 7×7 convolutional layer to obtain C subspace weights.

[0031] The average of the weights of the C subspaces is obtained by summing them up. The mean of the spatial weights is then normalized by the Sigmoid activation function to obtain the spatial attention weights.

[0032] Furthermore, when generating the behavior image of the mechanism, a note box is created on the left and right sides of the behavior image, respectively, and denoted as the first note box and the second note box;

[0033] The channel attention weights and spatial attention weights are sequentially imported into the first and second note boxes, causing the first and second note boxes to generate channel notes and spatial notes. The behavior image with channel notes and spatial notes is recorded as the mechanism behavior image.

[0034] Furthermore, when compressing and optimizing the original model into an image behavior recognition model, the KL divergence loss between the output distribution of the image behavior recognition model and the original model, as well as the cross-entropy loss and localization loss between the predicted output of the image behavior recognition model and the actual behavior label, are minimized. The total loss is calculated by combining the KL divergence loss, cross-entropy loss and localization loss, and the process stops when the total loss converges.

[0035] Furthermore, the construction status includes safe status, low-risk status, and high-risk status;

[0036] When the number of unsafe behaviors corresponding to real-time behavior labels is 0, the construction status is safe.

[0037] When the number of unsafe behaviors corresponding to the real-time behavior label is 1, the construction status is low-risk.

[0038] When the number of unsafe behaviors corresponding to real-time behavior labels is greater than 1, the construction status is high-risk.

[0039] Furthermore, the construction notification information includes normal construction notification information, hazard warning notification information, and hazard correction notification information;

[0040] When the construction status is safe, a normal construction prompt message will be issued.

[0041] When the construction status is low-risk, a hazard warning message will be issued;

[0042] When the construction status is high-risk, a hazard correction prompt message will be issued.

[0043] The technical advantages of this invention, an intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition, are as follows:

[0044] (1): By identifying key feature regions in behavior images and collecting channel attention weights and spatial attention weights of key feature regions, this invention can comprehensively collect factors in two parallel dimensions that affect the feature recognition accuracy of key feature regions: feature channels and spatial location. At the same time, based on the importance of feature channels and spatial location in recognition, a corresponding lightweight attention mechanism can be assigned to the image behavior recognition model during the recognition process. This ensures that the image behavior recognition model focuses more on feature recognition of small targets and key feature regions when intelligently recognizing behavior images, suppressing noise interference caused by complex environmental backgrounds, and effectively improving the recognition accuracy of image behavior recognition models for unsafe behaviors in small targets and complex backgrounds.

[0045] (2): This invention compresses and optimizes the original model into an image behavior recognition model by using the knowledge distillation method. This method can distill the knowledge of the original model into a lighter image behavior recognition model, achieving a reasonable lightweight transformation effect of the image behavior recognition model. This ensures that the image behavior recognition model can achieve or even surpass the recognition accuracy of the original model with a significant reduction in data volume and computational volume, and also improve the recognition speed of the image behavior recognition model, effectively meeting the intelligent recognition needs of unsafe behaviors at construction sites. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of a module of an intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition, provided in Embodiment 1 of the present invention.

[0047] Figure 2 This is a logical diagram of an intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition, provided in Embodiment 1 of the present invention.

[0048] Figure 3 This is a flowchart illustrating the intelligent identification method for various unsafe behaviors at construction sites based on image recognition, provided in Embodiment 2 of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Example 1: Please refer to Figures 1-2 As shown in this embodiment, the intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition includes:

[0051] The image annotation and classification module collects simulated behavioral data of unsafe behaviors occurring at the construction site under simulated construction scenarios, parses the data attributes of the simulated behavioral data, and divides the simulated behavioral data into behavioral images and behavioral annotations.

[0052] A simulated construction scenario refers to a construction environment that contains only one comprehensive environmental data type, which is simulated through experiments. This simulated construction scenario can serve as a prerequisite for the subsequent collection of relevant images and data at the construction site. In this embodiment, since there is more than one simulated construction scenario, it is necessary to simulate each different construction scenario in order to ensure the accuracy of subsequent identification of unsafe behaviors.

[0053] Specifically, the construction scenarios include weather conditions, lighting conditions, and spatial conditions;

[0054] Weather conditions are used to specifically describe the different weather conditions at the construction site location. Weather conditions include, but are not limited to, sunny days, rainy days, and snowy days.

[0055] Lighting conditions are a specific representation of different lighting conditions at the location of a construction site. Lighting conditions include, but are not limited to, strong light, no light, and weak light.

[0056] Spatial conditions are used to specifically represent the conditions of different construction sites. Spatial conditions include, but are not limited to, indoor, rooftop, and outdoor locations.

[0057] By arbitrarily selecting weather conditions, lighting conditions, and spatial conditions, and combining them, a construction scene can be obtained; for example, when the selected weather conditions, lighting conditions, and spatial conditions are sunny, strong light, and outdoor, respectively, the construction scene is an outdoor scene with sunny weather and strong light.

[0058] Unsafe acts refer to behaviors that endanger the personal safety of construction workers during construction, and are directly represented by the final result obtained after image recognition in this embodiment.

[0059] Because construction workers at construction sites are subject to safety impacts from multiple dimensions during their work, the number and dimensions of unsafe behaviors are usually quite large. In this embodiment, to ensure the accuracy of unsafe behavior identification in image recognition, it is necessary to use the standard that an image contains only one unsafe behavior as the basis and premise for subsequent image recognition.

[0060] Specifically, unsafe behaviors include, but are not limited to, not wearing a safety helmet, not wearing a safety rope, entering a dangerous area, and smoking in the construction area.

[0061] Simulated behavior data refers to image and text data of unsafe behaviors of construction workers collected in simulated construction scenarios, thus providing a comprehensive representation of unsafe behaviors of construction workers. Since simulated behavior data includes not only image data but also text data, in order to facilitate the classification and integration of simulated behavior data, the data attributes of simulated behavior data need to be analyzed, so as to facilitate the effective differentiation of images and text of unsafe behaviors in the future.

[0062] Data attributes are used to represent the results of data identification logic represented by simulated behavioral data. Specifically, data attributes include independent attributes and dependent attributes.

[0063] The method for parsing data attributes is as follows:

[0064] Import the B simulated behavioral data generated from the A simulated construction scenario one by one into the blank dataset to generate a simulated dataset;

[0065] At the same time, the metadata of B simulated behavioral data in the simulated dataset is extracted respectively. The metadata with the annotation words as structures is recorded as target metadata, and the structure text of B target metadata is queried one by one. The annotation words are used to represent the true meaning contained in the metadata corresponding to the simulated behavioral data, so that metadata with different meanings can be effectively identified and distinguished.

[0066] When the structure text of the target metadata is a multidimensional sequence, the data attribute of the simulated behavior data corresponding to the target metadata is recorded as an independent attribute.

[0067] When the structure text of the target metadata is a one-dimensional sequence, the data attributes of the simulated behavior data corresponding to the target metadata are recorded as dependent attributes.

[0068] After obtaining the data attributes of the simulated behavior data, the simulated behavior data within the simulated dataset can be distinguished according to the different data attributes, so that the simulated behavior data can be divided into behavior images that represent unsafe behaviors visually and behavior labels that represent unsafe behaviors textually.

[0069] Specifically, when dividing simulated behavior data, the data attributes of the simulated behavior data should be used as the basis for division. When the data attribute is an independent attribute, the simulated behavior data belongs to the independent variable data of unsafe behavior identification operation, and the simulated behavior data is recorded as a behavior image, resulting in D behavior images. When the data attribute is a dependent attribute, the simulated behavior data belongs to the dependent variable data of unsafe behavior identification operation, and the simulated behavior data is recorded as a behavior label, resulting in D behavior labels.

[0070] In this embodiment, a behavior image is image data that represents unsafe behaviors at the construction site, and a behavior label is text data that represents unsafe behaviors at the construction site. One behavior image corresponds to one behavior label, and the number of behavior images and behavior labels is the same. Changes in behavior images will cause changes in behavior labels.

[0071] The image weight fusion module identifies key feature regions in the behavior image, collects mechanism influence parameters of the key feature regions, including channel attention weights and spatial attention weights, and fuses the mechanism influence parameters with the behavior image to generate a mechanism behavior image.

[0072] Key feature regions refer to the regions in a behavior image that represent the features corresponding to unsafe behaviors with the smallest possible area. This allows key feature regions to maximize the area ratio of the features corresponding to unsafe behaviors, thereby laying the foundation for accurate identification of unsafe behaviors in the future.

[0073] Since behavioral images contain diverse feature regions such as construction workers, unsafe behaviors, and construction backgrounds, they may contain a large number of useless features that affect the identification of unsafe behaviors. Therefore, it is necessary to identify key feature regions in smaller areas from large behavioral images.

[0074] Specifically, when identifying key feature regions, computer vision technology is used to identify construction workers and equipment in the behavior image, and the corresponding feature points of the construction workers and equipment in the behavior image are marked. The geometric center of the behavior image is used as the intersection of the diagonals of the unit rectangle. The unit rectangle is drawn in the behavior image, and the length and width of the unit rectangle are continuously adjusted until the adjusted unit rectangle first wraps all the feature points. The area in the behavior image located inside the adjusted unit rectangle is recorded as the key feature region.

[0075] It should be noted that a unit rectangle is a rectangle drawn based on a unit length and a unit width. In this case, the area of ​​the unit rectangle is much smaller than the overall area of ​​the behavioral image. When adjusting the unit rectangle, the length and width of the unit rectangle are usually increased, so that the area of ​​the unit rectangle is continuously expanded until all feature points are included.

[0076] Because there are many types of unsafe behaviors, the feature forms of unsafe behaviors in behavior images are also different. This results in different importance of feature channels and spatial location in key feature regions for different unsafe behaviors. In order to more accurately identify the features contained in the key feature regions corresponding to different unsafe behaviors and improve the recognition accuracy of unsafe behaviors in small target areas and complex backgrounds, it is necessary to collect the mechanism influencing parameters that represent the importance of feature channels and spatial location in key feature regions.

[0077] In this embodiment, the mechanism influence parameter can not only represent the importance of feature channels and spatial location, but also serve as an important parameter for subsequent models to identify unsafe behaviors in behavioral images, thereby improving the accuracy of identifying unsafe behaviors in small targets and complex environments.

[0078] Specifically, the mechanism-affected parameters include channel attention weights and spatial attention weights;

[0079] Channel attention weight refers to the degree of importance of the feature channels in the key feature region during the process of identifying unsafe behavior. It can play a role in the accuracy of identifying unsafe behavior in the key feature region in terms of feature channels.

[0080] The method for collecting channel attention weights is as follows:

[0081] The key feature regions are imported into the convolutional neural network for multi-level convolution operations to obtain C feature channels.

[0082] The first average eigenvalue of each of the C feature channels is calculated one by one using the global average pooling formula.

[0083] The formula for calculating the first average eigenvalue is:

[0084] ;

[0085] In the formula, The first average feature value of the key feature region. The height of key regional features, Width of key region features, The number of feature channels, =1,2...C, Key feature regions;

[0086] The first maximum eigenvalue of each of the C feature channels is calculated one by one using the global max pooling formula.

[0087] The first average eigenvalue and the first maximum eigenvalue of the C feature channels are concatenated to obtain C first concatenation results. The C first concatenation results are then processed by a multilayer perceptron to obtain C sub-channel weights.

[0088] The average of the weights of the C sub-channels is obtained by summing the weights. The average channel weight is then normalized using the Sigmoid activation function to obtain the channel attention weight.

[0089] Spatial attention weight refers to the degree of importance of the spatial location in the key feature region during the identification of unsafe behaviors, which can have a spatial influence on the accuracy of identifying unsafe behaviors in the key feature region.

[0090] The method for collecting spatial attention weights is as follows:

[0091] Channel attention weights are applied to each of the C feature channels to transform key feature regions into enhanced feature regions.

[0092] The second average eigenvalues ​​of the C feature channels in the enhanced feature region are calculated using the global average pooling formula.

[0093] The formula for calculating the second average eigenvalue is:

[0094] ;

[0095] In the formula, To enhance the second average eigenvalue of the feature region, To enhance the feature regions;

[0096] The second maximum feature value of the enhanced feature region is calculated one by one using the global max pooling calculation formula.

[0097] The C second average eigenvalues ​​and the second maximum eigenvalues ​​of the enhanced feature region are concatenated on the C feature channels to obtain C second concatenation results. The C second concatenation results are then processed by a 7×7 convolutional layer to obtain C subspace weights.

[0098] The average of the weights of the C subspaces is obtained by summing them up. The mean of the spatial weights is then normalized by the Sigmoid activation function to obtain the spatial attention weights.

[0099] It should be noted that the values ​​of channel attention weight and spatial attention weight are ultimately distributed between 0 and 1. When the value of channel attention weight or spatial attention weight is larger, it means that the key feature region is more important in the feature channel dimension, and vice versa.

[0100] After collecting the channel attention weights and spatial attention weights of the key feature regions, it is necessary to effectively fuse the channel attention weights and spatial attention weights with the behavior image to generate a mechanism behavior image. This allows the mechanism behavior image for unsafe behavior recognition to have attention mechanisms in both channel and spatial dimensions, ensuring that the model can focus more on small targets and key regions in the mechanism behavior image. This suppresses noise interference from complex environmental backgrounds and improves feature representation capabilities and the accuracy of unsafe behavior recognition.

[0101] Specifically, when generating the mechanism behavior image, a note box is created on the left and right sides of the behavior image, denoted as the first note box and the second note box. Channel attention weights and spatial attention weights are then imported into the first note box and the second note box in sequence, so that the first note box and the second note box generate channel notes and spatial notes. The behavior image with channel notes and spatial notes is then denoted as the mechanism behavior image.

[0102] In this embodiment, the mechanism behavior image is the direct object that is finally input into the model for unsafe behavior identification. At this time, the mechanism behavior image has the same behavior label as the previous behavior image.

[0103] The model training compression module combines construction scene and mechanism behavior images into image behavior data, trains the original model using the image behavior data, and compresses the original model to generate an image behavior recognition model.

[0104] Image behavior data is direct image data used for intelligent identification of unsafe behaviors at construction sites and is subsequently input into the model.

[0105] Since image behavior data is not obtained directly, it is necessary to combine different construction scenarios with corresponding mechanism behavior images to form image behavior data that can provide intelligent recognition basis for unsafe behaviors in different construction scenarios.

[0106] Specifically, when combining image behavior data, arbitrarily select weather conditions, lighting conditions, and spatial conditions to form a construction scene, obtaining E construction scenes, and combine the mechanism behavior images corresponding to the E construction scenes to obtain E image behavior data.

[0107] The original model is a machine learning model based on the YOLOv8n model and a large amount of image behavior data, which can accurately identify the behavior annotations in the mechanism behavior images under different construction scenarios.

[0108] To improve the accuracy of the original model in recognizing unsafe behaviors in mechanism behavior images, a large amount of training and optimization processing is required for the original model.

[0109] Specifically, the training method for the original model is as follows:

[0110] Convert the construction scene into the corresponding number. For example, convert the number of the outdoor scene with strong sunlight on a sunny day to 1, the number of the indoor scene with weak sunlight on a sunny day to 2, and the number of the indoor scene with weak sunlight on a rainy day to 3.

[0111] Each set of mechanism behavior images is labeled as a training feature, and the behavior annotations of each set of training features are annotated.

[0112] The labeled training features are divided into a training set and a test set; 70% of the training features are used as the training set and 30% of the training features are used as the test set; the original model is trained using the training set and tested using the test set.

[0113] A preset error threshold is set. When the mean of the prediction errors of all training features in the test set is less than the error threshold, the original model is output.

[0114] In this embodiment, the preset error threshold is set in advance according to the accuracy actually required by the original model. For example, the preset error threshold is 0.92.

[0115] After obtaining the original model, the original model at this time is not the final model for unsafe behavior recognition. Instead, the original model is compressed by knowledge distillation to obtain an image behavior recognition model that can directly recognize the behavior annotations corresponding to image behavior data.

[0116] Compared to the original model, the image behavior recognition model does not require a large amount of image behavior data and behavior annotations as training data. By training to convergence on the basis of the original model, it can improve the recognition speed of behavior annotations corresponding to image behavior data while maintaining the recognition accuracy of image behavior data.

[0117] Specifically, when training the original model into an image behavior recognition model, not only is the total loss calculated using real behavior annotations, but the predicted distribution of the original model's output is also introduced as a supervision signal. By minimizing the KL divergence loss between the output distributions of the image behavior recognition model and the original model, as well as the cross-entropy loss and localization loss between the predicted output of the image behavior recognition model and the real behavior annotations, and combining the KL divergence loss, cross-entropy loss, and localization loss, the total loss is calculated. The process stops when the total loss converges. This allows the knowledge of the original model to be distilled into a lighter image behavior recognition model. Ultimately, the image behavior recognition model can achieve recognition accuracy close to or even surpass that of the original model with a significant reduction in data volume and computational cost.

[0118] The formula for calculating the loss function is:

[0119] ;

[0120] In the formula, For the total loss, For cross-entropy loss, To pinpoint the loss, For KL divergence loss, , , These are the weighting coefficients for cross-entropy loss, localization loss, and KL divergence loss, respectively.

[0121] The real-time image recognition module combines real-time construction scene and mechanism behavior images into real-time image behavior data. It identifies real-time behavior annotations through the image behavior recognition model, determines the construction status of the construction site, and issues corresponding construction prompt information.

[0122] After obtaining the image behavior recognition model, the real-time collected mechanism behavior images and real-time construction scenes can be combined to obtain real-time image behavior data, which can then be used as the basis for recognizing real-time behavior annotations.

[0123] Specifically, when collecting real-time construction scene data, weather and lighting conditions are obtained through weather forecast software and light sensors, respectively, while spatial conditions are obtained through the spatial location of the camera capturing the construction site, thereby achieving the effect of real-time construction scene data collection.

[0124] Mechanism behavior images are based on behavior images captured by cameras at the construction site. After identifying and drawing key feature regions of the behavior images, the mechanism influence parameters are collected and fused to obtain an image that can be used as the direct object for real-time behavior labeling at the current moment.

[0125] After identifying real-time behavior labels, the safety level of the unsafe behavior at the construction site can be determined based on the specific results of the unsafe behavior corresponding to the identified behavior labels.

[0126] Construction status is used to represent the degree of danger corresponding to the number of unsafe acts at the construction site. Specifically, construction status includes safe status, low-risk status, and high-risk status. Among them, the number of unsafe acts corresponding to safe status, low-risk status, and high-risk status increases from few to many, and the degree of danger of construction operations at the construction site increases from low to high.

[0127] In this embodiment, multiple cameras at different angles and positions at the construction site collect behavioral images, and the real-time behavioral labels are identified by the image behavior recognition model. Therefore, there are multiple real-time behavioral labels.

[0128] For example, when the number of unsafe behaviors corresponding to real-time behavior labels is 0, the construction status of the construction site is safe; when the number of unsafe behaviors corresponding to real-time behavior labels is 1, the construction status of the construction site is low-risk; when the number of unsafe behaviors corresponding to real-time behavior labels is greater than 1, the construction status of the construction site is high-risk.

[0129] Construction alerts are safety warnings issued for different construction conditions. Specifically, construction alerts include normal construction alerts, hazard warnings, and hazard correction alerts.

[0130] In this embodiment, when the construction status is safe, there are no unsafe behaviors at the construction site, and the issued construction prompt message is a normal construction prompt message.

[0131] When the construction status is low-risk, if an unsafe behavior occurs at the construction site, the issued construction warning message will be a hazard warning message.

[0132] When the construction status is high-risk, and at least two unsafe behaviors occur at the construction site, the issued construction warning message is a hazard correction warning message.

[0133] Example 2: Please refer to Figure 3 As shown, parts not described in detail in this embodiment are described in Embodiment 1. This embodiment provides an intelligent identification method for multiple unsafe behaviors at construction sites based on image recognition, implemented through an intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition, including:

[0134] S01: In a simulated construction scenario, collect simulated behavioral data of unsafe behaviors at the construction site, parse out the data attributes of the simulated behavioral data, and divide the simulated behavioral data into behavioral images and behavioral annotations;

[0135] S02: Identify key feature regions in the behavior image, collect the channel attention weights and spatial attention weights of the key feature regions, and fuse the channel attention weights and spatial attention weights with the behavior image to generate a mechanism behavior image;

[0136] S03: Combine construction scene and mechanism behavior images into image behavior data, train the original model using the image behavior data, and compress and optimize the original model into an image behavior recognition model;

[0137] S04: Combine real-time construction scene and mechanism behavior images into real-time image behavior data, identify real-time behavior labels through image behavior recognition model, and issue corresponding construction prompt information after determining the construction status of the construction site.

[0138] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition, characterized in that: include: The image annotation and classification module is used to collect simulated behavioral data of unsafe behaviors at the construction site in a simulated construction scenario, parse the data attributes of the simulated behavioral data, and divide the simulated behavioral data into behavioral images and behavioral annotations. The image weight fusion module is used to identify key feature regions in the behavior image, collect the channel attention weights and spatial attention weights of the key feature regions, and fuse the channel attention weights and spatial attention weights with the behavior image to generate a mechanism behavior image. When identifying key feature regions, computer vision technology is used to identify construction workers and equipment in behavioral images and mark the feature points corresponding to the construction workers and equipment. Draw the unit rectangle in the behavior image with the geometric center of the behavior image as the intersection of the diagonals of the unit rectangle; Continuously adjust the length and width of the unit rectangle until the adjusted unit rectangle first encloses all feature points. Then, the area inside the adjusted unit rectangle is recorded as the key feature area. The model training compression module is used to combine construction scene and mechanism behavior images into image behavior data, train the original model through the image behavior data, and compress and optimize the original model into an image behavior recognition model. The real-time image recognition module is used to combine real-time construction scene and mechanism behavior images into real-time image behavior data. Through the image behavior recognition model, real-time behavior labels are identified, and after determining the construction status of the construction site, corresponding construction prompt information is issued.

2. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 1, characterized in that, Construction scenarios include weather conditions, lighting conditions, and spatial conditions; data attributes include independent and dependent attributes. The method for parsing data attributes is as follows: Import the B simulated behavioral data generated from the A simulated construction scenario one by one into the blank dataset to generate a simulated dataset; At the same time, extract the metadata of B simulated behavioral data from the simulated dataset, record the metadata with the annotation word "structure" as the target metadata, and query the structure text of B target metadata one by one; When the structure text of the target metadata is a multidimensional sequence, the data attributes are recorded as independent attributes; When the structure text of the target metadata is a one-dimensional sequence, the data attributes are recorded as dependent attributes.

3. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 2, characterized in that, When the data attribute of the simulated behavior data is an independent attribute, the simulated behavior data is recorded as a behavior image, and D behavior images are obtained; When the data attribute of the simulated behavioral data is a dependent attribute, the simulated behavioral data is recorded as behavioral labels, and D behavioral labels are obtained.

4. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 3, characterized in that, The method for collecting channel attention weights is as follows: The key feature regions are imported into the convolutional neural network for multi-level convolution operations to obtain C feature channels. The first average eigenvalue of each of the C feature channels is calculated one by one using the global average pooling formula, and the first maximum eigenvalue of each of the C feature channels is calculated one by one using the global max pooling formula. The first average eigenvalue and the first maximum eigenvalue of the C feature channels are concatenated to obtain C first concatenation results. The C first concatenation results are then processed by a multilayer perceptron to obtain C sub-channel weights. The average of the weights of the C sub-channels is obtained by summing the weights. The average channel weight is then normalized using the Sigmoid activation function to obtain the channel attention weight.

5. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 4, characterized in that, The method for collecting spatial attention weights is as follows: Channel attention weights are applied to each of the C feature channels to transform key feature regions into enhanced feature regions. The second average eigenvalue of the C feature channels in the enhanced feature region is calculated using the global average pooling formula, and the second maximum eigenvalue of the enhanced feature region is calculated one by one using the global max pooling formula. The C second average eigenvalues ​​and the second maximum eigenvalues ​​of the enhanced feature region are concatenated on the C feature channels to obtain C second concatenation results. The C second concatenation results are then processed by a 7×7 convolutional layer to obtain C subspace weights. The average of the weights of the C subspaces is obtained by summing them up. The mean of the spatial weights is then normalized by the Sigmoid activation function to obtain the spatial attention weights.

6. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 5, characterized in that, When generating the behavior image of the mechanism, a note box is created on the left and right sides of the behavior image, respectively, and denoted as the first note box and the second note box; The channel attention weights and spatial attention weights are sequentially imported into the first and second note boxes, causing the first and second note boxes to generate channel notes and spatial notes. The behavior image with channel notes and spatial notes is recorded as the mechanism behavior image.

7. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 6, characterized in that, When compressing and optimizing the original model into an image behavior recognition model, the KL divergence loss between the output distribution of the image behavior recognition model and the original model is minimized, as well as the cross-entropy loss and localization loss between the predicted output of the image behavior recognition model and the actual behavior label. The total loss is calculated by combining the KL divergence loss, cross-entropy loss and localization loss, and the process stops when the total loss converges.

8. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 7, characterized in that, Construction status includes safe status, low-risk status, and high-risk status; When the number of unsafe behaviors corresponding to real-time behavior labels is 0, the construction status is safe. When the number of unsafe behaviors corresponding to the real-time behavior label is 1, the construction status is low-risk. When the number of unsafe behaviors corresponding to real-time behavior labels is greater than 1, the construction status is high-risk.

9. The intelligent identification system for multiple unsafe behaviors at construction sites based on image recognition according to claim 8, characterized in that, Construction notification information includes normal construction notification information, hazard warning notification information, and hazard correction notification information; When the construction status is safe, a normal construction prompt message will be issued. When the construction status is low-risk, a hazard warning message will be issued; When the construction status is high-risk, a hazard correction prompt message will be issued.

Images