A construction safety early warning method and device based on a saliency neural network model

Abstract

The application discloses a construction safety early warning method and device based on a saliency neural network model, generates training sample data by performing image preprocessing on a construction equipment and personnel image set, constructs an initial saliency neural network model, trains a target saliency neural network model, when receiving a to-be-recognized construction site image, inputs the to-be-recognized construction site image into the target saliency neural network model, generates a target refined feature map, determines a plurality of corresponding target body regions, and identifies the target body regions, judges whether the target body data associated construction personnel exist construction safety hazards based on a judgment result of whether a preset condition is met, and solves the technical problems that the existing convolutional neural network cannot accurately identify the pixel point feature difference between the construction personnel and the construction equipment in the picture, highlights the salient features of the local region of the construction personnel, and thus cannot provide safety early warning for the construction personnel in time, and construction safety hazards exist.

Description

Technical Field

[0001] This invention relates to the field of construction site safety monitoring technology, and in particular to a construction safety early warning method and device based on a saliency neural network model. Background Technology

[0002] With the rapid development of artificial intelligence technology, the field of computer vision has been promoted. As a large amount of image data is transmitted on the Internet, the available datasets are increasing, making feature extraction of image information increasingly important.

[0003] Early saliency detection mostly focused on detecting human attention points, that is, detecting the most attractive parts of an image to human eyes. The detection result was a region of interest composed of a group of bright spots. With the further development of computer vision, the detection of single visual attention points has gradually become insufficient to meet its needs.

[0004] Therefore, researchers proposed viewing saliency detection as an image segmentation problem, which involves segmenting salient target regions from the background of an image, and called this task salient target detection.

[0005] Currently, while the efficient feature extraction capabilities of convolutional neural networks have greatly improved the performance of salient target detection algorithms, simply relying on convolutional neural networks for feature concatenation or addition is insufficient to effectively enhance features. This results in excessive image noise and redundant pixel input in construction scenarios, making it difficult to accurately identify the pixel feature differences between construction workers and equipment in images, and to highlight the salient features of local areas of construction workers. Consequently, timely safety warnings are not provided to construction workers, posing potential safety hazards. Summary of the Invention

[0006] This invention provides a construction safety early warning method and device based on a saliency neural network model. It solves the technical problem that existing convolutional neural networks cannot effectively enhance features, resulting in excessive image noise and redundant pixel input in construction scenarios. This makes it impossible to accurately identify the pixel feature differences between construction workers and construction equipment in the image and highlight the salient features of local areas of construction workers, thus failing to provide timely safety warnings to construction workers and posing potential construction safety hazards.

[0007] The first aspect of this invention provides a construction safety early warning method based on a saliency neural network model, comprising:

[0008] When a set of images of construction equipment and personnel is received, image preprocessing is performed on the set of images to generate training sample data;

[0009] A convolutional neural network model with an intersection-gap spatial attention mechanism is constructed by using a pre-set VGGnet16 convolutional neural network model and a pre-set intersection-gap spatial neural network model, and an initial saliency neural network model is constructed by combining a pre-set LSTM neural network model.

[0010] The initial saliency neural network model is trained using the training sample data to generate the target saliency neural network model;

[0011] When a construction site image to be identified is received, the image is input into the target saliency neural network model to generate a refined feature map of the target.

[0012] Extract the corresponding target ontology data from the target refinement feature map, determine multiple corresponding target ontology regions based on the target ontology data, and identify the target ontology regions;

[0013] Based on the judgment results of whether multiple target body regions meet the preset conditions, it is determined whether there are construction safety hazards associated with the construction personnel associated with the target body data.

[0014] Optionally, the step of training the initial saliency neural network model using the training sample data to generate the target saliency neural network model includes:

[0015] The training sample data is input into the initial saliency neural network model to generate the corresponding training refined feature map;

[0016] Calculate the training loss value between the trained refined feature map and the associated standard refined feature map;

[0017] Compare the training loss value with a preset training standard threshold;

[0018] If the training loss value is less than or equal to the training standard threshold, then training is stopped and a target saliency neural network model is generated.

[0019] Optionally, it also includes:

[0020] If the training loss value is greater than the training standard threshold, the parameters of the initial saliency neural network model are adjusted according to the preset gradient, and the process jumps to the step of inputting the training sample data into the initial saliency neural network model to generate the corresponding training refinement feature map.

[0021] Optionally, the step of receiving a construction site image to be identified and inputting the image into the target saliency neural network model to generate a refined target feature map includes:

[0022] When a construction site image to be identified is received, the image is input into the convolutional neural network model with the intersection-joint spatial attention mechanism to train the model and extract multiple corresponding convolutional image feature maps.

[0023] Normalization and mapping are performed on multiple convolutional image feature maps to obtain multiple corresponding image feature probability maps;

[0024] The image feature probability map is used as input to the LSTM neural network model for training, and multiple corresponding target saliency maps are extracted.

[0025] A linear fitting operation is performed on multiple target saliency maps to obtain corresponding target refinement feature maps.

[0026] Optionally, the target ontology data includes construction equipment ontology data and construction personnel ontology data, and the target ontology region includes a construction equipment ontology region and a construction personnel ontology region. The steps of extracting corresponding target ontology data from the target refinement feature map, determining multiple corresponding target ontology regions based on the target ontology data, and identifying the target ontology regions include:

[0027] Extract the corresponding construction equipment body data from the target refined feature map, and divide multiple construction equipment body regions with a preset radius, taking the location of each construction equipment body data as the region center.

[0028] The construction equipment body area is identified by a preset category color according to the equipment type of the construction equipment associated with it.

[0029] Extract the corresponding construction worker body data from the target refined feature map, and divide multiple construction worker body regions with the location of each construction worker body data as the region center.

[0030] A pre-defined prominent border is set for the construction worker identifier associated with the construction worker body area.

[0031] Optionally, the step of determining whether there are construction safety hazards associated with the construction personnel based on the judgment results of whether multiple target ontology regions meet preset conditions includes:

[0032] If there is an overlap between any of the construction equipment body areas and each of the construction personnel body areas, it is determined that the construction personnel associated with the construction personnel body area pose a construction safety hazard.

[0033] If there is no overlap between the areas of all the construction equipment and the areas of each of the construction personnel, then it is determined that there is no construction safety hazard associated with the construction personnel in the area of ​​the construction personnel.

[0034] A second aspect of the present invention provides a construction safety early warning device based on a saliency neural network model, comprising:

[0035] The image preprocessing module is used to perform image preprocessing on the construction equipment and personnel image set when the image set of construction equipment and personnel is received, and to generate training sample data.

[0036] The initial saliency neural network model construction module is used to construct an intersection-gap spatial attention mechanism convolutional neural network model by using a preset VGGnet16 convolutional neural network model and a preset intersection-gap spatial neural network model, and to construct an initial saliency neural network model by combining a preset LSTM neural network model.

[0037] The target saliency neural network model generation module is used to train the initial saliency neural network model using the training sample data to generate the target saliency neural network model;

[0038] The target refinement feature map acquisition module is used to generate a target refinement feature map by inputting the target saliency neural network model with the construction site image to be identified when a construction site image to be identified is received.

[0039] The target ontology region acquisition module is used to extract corresponding target ontology data from the target refinement feature map, determine multiple corresponding target ontology regions based on the target ontology data, and identify the target ontology regions.

[0040] The safety hazard judgment module is used to determine whether there are any construction safety hazards associated with the construction personnel in the target body data based on the judgment results of whether multiple target body regions meet preset conditions.

[0041] Optionally, the target refinement feature map acquisition module includes:

[0042] The convolutional image feature map acquisition submodule is used to train the intersection-and-joint spatial attention mechanism convolutional neural network model by inputting the construction site image to be identified into the convolutional neural network model, and to extract multiple corresponding convolutional image feature maps.

[0043] The image feature probability map acquisition submodule is used to perform normalization and mapping operations on multiple convolutional image feature maps to obtain multiple corresponding image feature probability maps.

[0044] The target saliency map acquisition submodule is used to train the LSTM neural network model by inputting the image feature probability map into it, and to extract multiple corresponding target saliency maps.

[0045] The linear fitting submodule is used to perform a linear fitting operation on multiple target saliency maps to obtain the corresponding target refinement feature maps.

[0046] Optionally, the target ontology data includes construction equipment ontology data and construction personnel ontology data, the target ontology region includes a construction equipment ontology region and a construction personnel ontology region, and the target ontology region acquisition module includes:

[0047] The construction equipment body area acquisition submodule is used to extract the corresponding construction equipment body data from the target refined feature map, and divide multiple construction equipment body areas with a preset radius, with the location of each construction equipment body data as the area center.

[0048] The category color identification submodule is used to identify the construction equipment body area with a preset category color according to the equipment type of the construction equipment associated with the construction equipment body area;

[0049] The submodule for obtaining the body region of construction workers is used to extract the corresponding body data of construction workers from the target refined feature map, and to divide multiple body regions of construction workers with the location of each body data of construction workers as the center of the region.

[0050] The prominent border identifier submodule is used to preset a prominent border for the construction worker identifier associated with the construction worker body area.

[0051] Optionally, the safety hazard assessment module includes:

[0052] The first judgment submodule is used to determine that if there is regional overlap between any of the construction equipment body areas and each of the construction personnel body areas, the construction personnel associated with the construction personnel body area have a construction safety hazard.

[0053] The second judgment submodule is used to determine that if there is no regional overlap between all the construction equipment body areas and each of the construction personnel body areas, the construction personnel associated with the construction personnel body area do not pose a construction safety hazard.

[0054] As can be seen from the above technical solutions, the present invention has the following advantages:

[0055] Upon receiving an image set of construction equipment and personnel, image preprocessing is performed to generate training sample data. A convolutional neural network model with an intersection-gap spatial attention mechanism is constructed using a pre-set VGGnet16 convolutional neural network model and a pre-set intersection-gap spatial neural network model. An initial saliency neural network model is then constructed by combining this with a pre-set LSTM neural network model. The initial saliency neural network model is trained using the training sample data to generate a target saliency neural network model. When an image of the construction site to be identified is received, it is input into the target saliency neural network model to generate a target refinement feature map. Corresponding target ontology data is extracted from the target refinement feature map. Multiple corresponding target ontology regions are determined based on the target ontology data and labeled. Based on the judgment results of whether multiple target ontology regions meet pre-set conditions, it is determined whether there are any hidden safety hazards associated with the construction personnel in the target ontology data. This paper addresses the problem that existing convolutional neural networks cannot effectively enhance features, leading to excessive noise and redundant pixels in images during construction scenes. This results in inaccurate identification of pixel feature differences between construction workers and equipment, failing to highlight salient features in localized areas and thus failing to provide timely safety warnings, posing potential safety hazards. By combining eye-tracking prediction and the memory function of LSTM neural networks, and mimicking the biological theory of human visual saliency, a new model for salient target location detection is developed. This model progressively refines the target feature map and finally uses a classification function to classify the image, achieving image recognition. This significantly improves the performance of saliency detection. In a dataset of construction workers and equipment images, by detecting pixel feature differences between the human body and other types of equipment in the feature map, the model significantly amplifies human features, ultimately achieving hazard warnings and protecting construction workers. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 The flowchart illustrates the steps of a construction safety early warning method based on a saliency neural network model provided in Embodiment 1 of the present invention.

[0058] Figure 2 The following is a flowchart of the steps of a construction safety early warning method based on a saliency neural network model provided in Embodiment 2 of the present invention;

[0059] Figure 3 This is a structural block diagram of a construction safety early warning device based on a saliency neural network model provided in Embodiment 3 of the present invention. Detailed Implementation

[0060] This invention provides a construction safety early warning method and apparatus based on a saliency neural network model. It addresses the technical problem that existing convolutional neural networks cannot effectively enhance features, resulting in excessive noise and redundant pixel inputs in construction scenes. This makes it difficult to accurately identify the pixel feature differences between construction workers and construction equipment in images, highlight the salient features of local areas of construction workers, and thus fail to provide timely safety warnings to construction workers, leading to potential construction safety hazards.

[0061] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a construction safety early warning method based on a saliency neural network model, as provided in Embodiment 1 of the present invention.

[0063] This invention provides a construction safety early warning method based on a saliency neural network model, comprising:

[0064] Step 101: When the image set of construction equipment and personnel is received, perform image preprocessing on the image set of construction equipment and personnel to generate training sample data.

[0065] The image set of construction equipment and personnel refers to an image set consisting of pictures of construction sites, including excavators, cranes, trucks, bulldozers, cars, and construction workers.

[0066] Image preprocessing refers to collecting images according to their type and compiling them into a preliminary image dataset. This typically requires hundreds or thousands of images, which can be obtained from online sources or taken from real-world scenarios. Image cleaning is crucial; during the collection or shooting process, some images may not accurately represent the intended meaning of the dataset. These images need to be removed from the dataset to ensure feature consistency. Annotation is then performed, using different annotation methods based on the dataset type. The images and annotation files are then read and saved sequentially, forming the training sample data.

[0067] Training sample data refers to the sample data used as input for training the initial saliency neural network model.

[0068] In this embodiment of the invention, when a set of images of equipment and personnel is received, the set of images of construction equipment and personnel is cleaned and labeled to generate training sample data.

[0069] Step 102: Construct an inter-joint spatial attention mechanism convolutional neural network model using a preset VGGnet16 convolutional neural network model and a preset inter-joint spatial neural network model, and construct an initial saliency neural network model by combining it with a preset LSTM neural network model.

[0070] The VGGnet16 convolutional neural network model refers to a neural network model composed of VGGnet16 convolutional neural networks. It mainly includes convolutional layers, pooling layers, and fully connected layers. Each convolutional layer consists of several convolutional units, and the weights and biases of each unit are optimized using the backpropagation algorithm. The purpose of convolutional operations is to extract different features. The first layer extracts low-level features such as lines, angles, and edges, while deeper layers iteratively extract more complex features from these low-level features. Typically, after convolutional layers, high-dimensional features are obtained. Because pooling layers lose image feature details, the extracted image feature maps are ignored. Fully connected layers combine all local features into global features, which are used to calculate the final classification score. The hierarchical structure of the convolutional neural network first extracts features through multiple alternating layers of convolution and / or pooling, then transforms and maps the features, and finally uses a classification function to output the result.

[0071] Intersection-and-merge spatial neural network model refers to a neural network model composed of intersection-and-merge spatial neural networks.

[0072] LSTM neural network model refers to Long Short Term Network (LSTM), which is a special type of RNN that can learn long-term dependency information. LSTM is deliberately designed to avoid long-term dependency problems.

[0073] In this embodiment of the invention, a convolutional neural network model with an intersection-and-gap spatial attention mechanism is constructed by using a preset VGGnet16 convolutional neural network model and a preset intersection-and-gap spatial neural network model, and then an initial saliency neural network model is constructed by combining a preset LSTM neural network model.

[0074] Step 103: Train the initial saliency neural network model using training sample data to generate the target saliency neural network model.

[0075] The target saliency neural network model refers to a neural network model used to receive and extract images of construction sites to be identified, and generate the corresponding target saliency model.

[0076] In this embodiment of the invention, the initial saliency neural network model is trained based on the training sample data obtained after image preprocessing, thereby generating the corresponding target saliency neural network model.

[0077] Step 104: When the construction site image to be identified is received, the image is input into the target saliency neural network model to generate a refined feature map of the target.

[0078] The target refinement feature map refers to the refined feature map of the construction site output by inputting the image of the construction site to be identified into the target saliency neural network model.

[0079] In this embodiment of the invention, when a construction site image to be identified is received, the image is input into a target saliency neural network model to generate a refined feature map of the target.

[0080] Step 105: Extract the corresponding target ontology data from the target refinement feature map, determine multiple corresponding target ontology regions based on the target ontology data, and identify the target ontology regions.

[0081] Target ontology data refers to the feature data of construction personnel and construction equipment ontology extracted from the target refinement feature map.

[0082] The target ontology region refers to the region data where the construction personnel and construction equipment are located, extracted from the target refinement feature map.

[0083] Identification refers to color-coding and prominent border marking based on the obtained target body region.

[0084] In this embodiment of the invention, feature data of the corresponding construction worker body and construction equipment body are extracted from the target refined feature map. Based on the feature data of the construction worker body and construction equipment body, multiple corresponding regional data of the construction worker body and construction equipment body are determined, and the target body region is marked with color and prominent border.

[0085] Step 106: Based on the judgment results of whether multiple target body regions meet the preset conditions, determine whether there are construction safety hazards associated with the construction personnel of the target body data.

[0086] In this embodiment of the invention, if there is regional overlap between any construction equipment body area and each construction worker body area, it is determined that the construction worker associated with the construction worker body area has a construction safety hazard; if there is no regional overlap between all construction equipment body areas and each construction worker body area, it is determined that the construction worker associated with the construction worker body area does not have a construction safety hazard.

[0087] In this embodiment of the invention, when a set of images of construction equipment and personnel is received, image preprocessing is performed on the images to generate training sample data. A convolutional neural network model with an intersection-gap spatial attention mechanism is constructed using a preset VGGnet16 convolutional neural network model and a preset intersection-gap spatial neural network model. An initial saliency neural network model is constructed by combining this with a preset LSTM neural network model. The initial saliency neural network model is trained using the training sample data to generate a target saliency neural network model. When a construction site image to be identified is received, the image is input into the target saliency neural network model to generate a target refinement feature map. Corresponding target ontology data is extracted from the target refinement feature map. Multiple corresponding target ontology regions are determined based on the target ontology data and are identified. Based on the judgment results of whether multiple target ontology regions meet preset conditions, it is determined whether the construction personnel associated with the target ontology data are present. This paper addresses the technical problem of existing convolutional neural networks failing to effectively enhance features, leading to excessive noise and redundant pixels in images during construction scenes. This results in inaccurate identification of pixel feature differences between construction workers and equipment, hindering the identification of salient features in localized areas and preventing timely safety warnings. By combining eye-tracking prediction with the memory function of LSTM neural networks, and mimicking the biological theory of human visual saliency, a novel model for salient target location detection is developed. This model progressively refines the target feature map and finally uses a classification function to classify the image, achieving image recognition and effectively improving saliency detection performance. In a dataset of construction workers and equipment images, by detecting pixel feature differences between the human body and other types of equipment in the feature map, the model significantly amplifies human features, ultimately providing hazard warnings and protecting construction workers.

[0088] Please see Figure 2 , Figure 2 The flowchart illustrates the steps of a construction safety early warning method based on a saliency neural network model, as provided in Embodiment 2 of the present invention.

[0089] This invention provides a construction safety early warning method based on a saliency neural network model, comprising:

[0090] Step 201: When the image set of construction equipment and personnel is received, perform image preprocessing on the image set of construction equipment and personnel to generate training sample data.

[0091] In this embodiment of the invention, the specific implementation process of step 201 is similar to that of step 101, and will not be repeated here.

[0092] It's worth noting that the image set of construction equipment and personnel (including excavators, cranes, trucks, bulldozers, cars, and people) comprises 6 classes, with 4200 training images and 800 test images. The CIFAR-10 image dataset was also used for training and testing. CIFAR-10 is a commonly used image dataset in the field of deep learning. The CIFAR-10 dataset contains 60,000 32×32 color images across 10 classes, with 50,000 training images and 10,000 test images. The CIFAR-10 dataset is divided into 5 training blocks and 1 test block, with 10,000 images in each block. The test block contains 1000 images randomly selected from each class, and the training blocks contain these images in a random order.

[0093] Step 202: Construct an inter-joint spatial attention mechanism convolutional neural network model using a preset VGGnet16 convolutional neural network model and a preset inter-joint spatial neural network model, and construct an initial saliency neural network model by combining it with a preset LSTM neural network model.

[0094] In this embodiment of the invention, the specific implementation process of step 202 is similar to that of step 102, and will not be repeated here.

[0095] Step 203: Train the initial saliency neural network model using training sample data to generate the target saliency neural network model.

[0096] Furthermore, step 203 may include the following sub-steps:

[0097] S11. Input the training sample data into the initial saliency neural network model to generate the corresponding training refined feature map.

[0098] The training refinement feature map refers to the feature map generated by inputting training sample data into the initial saliency neural network model, which is used to determine whether the initial saliency neural network model has been trained successfully.

[0099] In this embodiment of the invention, the generated training sample data is input into the initial saliency neural network model to generate the corresponding training refined feature map.

[0100] S12. Calculate the training loss value between the trained refined feature map and the associated standard refined feature map.

[0101] The standard refined feature map refers to the standard refined feature map corresponding to the training sample data.

[0102] The training loss value refers to the loss value between the trained refined feature map and the standard refined feature map.

[0103] In this embodiment of the invention, the training loss value between the trained refined feature map and the associated standard refined feature map is calculated.

[0104] S13. Compare the training loss value with the preset training standard threshold.

[0105] The training standard threshold refers to the threshold used to determine whether the initial saliency neural network model has been successfully trained.

[0106] In this embodiment of the invention, the training loss value is compared with a preset training standard threshold.

[0107] S14. If the training loss value is less than or equal to the training standard threshold, stop training and generate the target saliency neural network model.

[0108] In this embodiment of the invention, if the training loss value is less than or equal to the training standard threshold, training is stopped and a target saliency neural network model is generated.

[0109] Furthermore, step 203 may also include the following sub-steps:

[0110] S15. If the training loss value is greater than the training standard threshold, adjust the parameters of the initial saliency neural network model according to the preset gradient, and jump to the step of inputting the training sample data into the initial saliency neural network model to generate the corresponding training refinement feature map.

[0111] In this embodiment of the invention, if the training loss value is greater than the training standard threshold, the parameters of the initial saliency neural network model are adjusted according to the preset gradient, and the process jumps to the step of inputting the training sample data into the initial saliency neural network model to generate the corresponding training refined feature map.

[0112] Step 204: When the construction site image to be identified is received, the convolutional neural network model with the intersection-gathering spatial attention mechanism of the construction site image to be identified is trained to extract multiple corresponding convolutional image feature maps.

[0113] In this embodiment of the invention, when a construction site image to be identified is received, a convolutional neural network model with an intersection-joint spatial attention mechanism is trained to extract multiple corresponding convolutional image feature maps.

[0114] It is worth mentioning that a convolutional neural network model with an intersection-gap spatial attention mechanism is constructed by combining a pre-defined VGGnet16 convolutional neural network model with a pre-defined intersection-gap spatial neural network model. A spatial attention mechanism is added to each convolutional layer to enhance the extraction of image features, especially highlighting the features of people in salient images.

[0115] In one example of the present invention, when the training sample data is input into the initial saliency neural network model, the preset VGGnet16 convolutional neural network model in the initial saliency neural network model is first trained. If each convolutional block reaches the maximum number of iterations and obtains the highest accuracy, the training is complete; otherwise, the training continues. The specific training process of the VGGnet16 convolutional neural network model is as follows:

[0116] Given a 224*224 image, the VGGnet16 neural network first extracts depth features using five convolutional blocks. In the VGGnet16 neural network structure, the first convolutional block has two convolutional layers and one pooling layer, denoted as conv1; the second convolutional block has two convolutional layers and one pooling layer, denoted as conv2; the third convolutional block has three convolutional layers and one pooling layer, denoted as conv3; the fourth convolutional block has three convolutional layers and one pooling layer, denoted as conv4; and the fifth convolutional block has three convolutional layers and one pooling layer, denoted as conv5.

[0117] After each convolutional block, the feature layer is fed into a network with an intersection-and-gap spatial attention mechanism. Through the spatial attention mechanism, features are extracted from image pixels to amplify the differences between human features and other construction site equipment features in the input feature map.

[0118] The specific process of using the intersection-union spatial attention mechanism in a convolutional neural network model is as follows:

[0119] The input feature layer F∈(H*W*C) is subjected to max pooling and average pooling along the channel axis to obtain the pooled feature map Fmax∈(H*W*1) and feature map Favg∈(H*W*1), respectively. This step does not change the size of the feature layer, but compresses the number of channels from C to 1.

[0120] F max =Maxpool(F) (1)

[0121] F avg =Avgpool(F) (2)

[0122] F in formulas (1) and (2) max F avg To obtain a feature map containing compressed spatial location information, Maxpool is used for the maximum pooling operation, and Avgpool for the average pooling operation. Furthermore, since F... max F avg Same size, for F max F avg Perform an intersection / union operation (i.e., a logical AND operation), if F max F avgThe two correspond to pixel C in the same spatial location. ith If all weights are not 0, the result is 1. max F avg If one of the corresponding pixel weights is 0, the result is 0. After all pixels have been compared, the mask matrix Mask in formula (3) is obtained.

[0123] Mask = (F max &F avg (3)

[0124] F after pooling operation max F avg There must be items of the same size; otherwise, an intersection / merge operation cannot be performed. F max &F avg In effect, the signals of the same region of interest are amplified, while the weights of different regions are set to 0 and discarded. This suppresses irrelevant regions in the input image while highlighting the salient features of local human regions, so as to better identify construction workers and achieve hazard warning.

[0125]

[0126] Furthermore, in formula (4), the initial input feature layer F is multiplied by the mask matrix Mask to obtain the output feature layer F′, thus obtaining the feature layer with spatial attention mechanism.

[0127] Since pooling layers lose detail in image features, the pooling layers are ignored in the extracted image feature maps. Each time the input image passes through a VGGnet convolutional block, it outputs convolutional image feature maps of sizes 224*224*64, 112*112*128, 56*56*256, 28*28*512, and 14*14*512, respectively, denoted as pic1, pic2, pic3, pic4, and pic5.

[0128] Step 205: Normalize and map multiple convolutional image feature maps to obtain multiple corresponding image feature probability maps.

[0129] In this embodiment of the invention, normalization and mapping are performed on multiple convolutional image feature maps to obtain multiple corresponding image feature probability maps.

[0130] In one example of the present invention, the convolutional feature map extracted from the fifth convolutional block is normalized using a convolutional layer with an activation function to obtain an image feature probability map. This image feature probability map is then mapped to obtain a grayscale viewpoint map, which is used as a visual prior guide. The specific process is as follows:

[0131] The pic5 image is normalized using a 3*3 convolutional layer with a sigmoid activation function, resulting in a 14*14 image feature probability map with values ​​between 0 and 1 (P∈(0,1)). Finally, the normalized image probability map is mapped to obtain a grayscale viewpoint map, which will serve as the visual prior guidance for this network model.

[0132] The sigmoid function is a widely used activation function in neural networks. Different metrics often have different dimensions and units. Directly weighting and summing the raw data of each metric will result in features with larger values ​​having a greater impact on the objective function than those with smaller values. However, features with larger values ​​are not necessarily more important in the evaluation system. Therefore, it is necessary to unify the orders of magnitude of each metric, i.e., feature normalization.

[0133] Step 206: Use the image feature probability map as input to train the LSTM neural network model and extract multiple corresponding target saliency maps.

[0134] In this embodiment of the invention, an image feature probability map is used as input to train an LSTM neural network model to extract multiple corresponding target saliency maps.

[0135] In one example of this invention, the specific process of outputting the corresponding target saliency map through the LSTM neural network model is as follows:

[0136] The image feature probability map obtained by normalization is used as the early memory h0 and the previous cell C0 of the LSTM. The five convolutional image feature maps pic are used as the input information X of the LSTM neural network, denoted as X1, X2, X3, X4, and X5, respectively. They are input into their respective LSTM neural networks for training. The LSTM forget gate, input gate, and output gate are used to retain important feature information and filter out secondary feature information from the five convolutional feature maps to form a target saliency map. From the target saliency map, the detection and localization of salient targets in the input image are roughly learned.

[0137] The term "gate" controls how much information can pass through. All three gates use the sigmoid activation function to map the output to the interval [0, 1]. A value of 1 indicates that all information can pass through, while a value of 0 indicates that no information passes through. The input gate, forget gate, and output gate all receive information X. t and h t-1 The input gate determines which information is added to C. t-1 To update C t The forget gate allows the network to forget useless information, while the output gate and C... t-1 Together they determine the output h at time t t C t-1To C t The renewal of memory cells is key to solving the problem of long-term dependency. Memory cells store useful information from various moments and continuously discard useless messages through the forget gate and receive new information through the input gate, thereby updating the cell state. By using memory cells, LSTM can capture the correlation between information from different points in a time series.

[0138] Forget Gate: This gate reads previous memories h0 and current input information X. t Then, the sigmoid function is used to output a value between 0 and 1, where 1 represents "completely keep" and 0 represents "completely discard". Both are then processed by the sigmoid function to output a weight f between 0 and 1. t This indicates the importance of the feature, so as to determine whether to retain it in the next step of feature selection. The specific calculation formula is shown in formula (5):

[0139]

[0140] In the formula, f t Indicates the importance of features, σ represents the sigmoid layer, and W f and b f h represents the learnable weights corresponding to the forget gate. t-1 For the hidden layer state, x t This is the current input information.

[0141] Input gate: Input the previous cell's prior memory h0 and the current input information x. t The control signal i is obtained by updating the information through the sigmoid function. t The output i t Given a value between 0 and 1, and using the tanh function to generate a vector C. t The specific calculation formulas are shown in formulas (6) and (7):

[0142]

[0143]

[0144] In the formula, σ represents the sigmoid layer, tanh represents the tanh activation layer, and i t For control signals, W f W c The learnable parameter matrix representing the convolution kernel, b f b c Represents the learnable bias parameter matrix. In cellular state, h t-1 This is the hidden layer state.

[0145] The control signal of the input gate is multiplied by the information stored in the cell to obtain the updated matrix, and the previously calculated forgotten state matrix is ​​added to obtain the current time state matrix. Finally, the state matrix C of the output information is generated. t The specific calculation formula is shown in formula (8):

[0146]

[0147] In formula (8), C t-1 This represents the cell state at the previous time step, along with the forgetting vector f. t Multiply point by point, and then combine the value with the output value of the input gate. By adding the information point by point, the new information discovered by the neural network is updated into the cell state, and the final result is C. t For the newly generated cell state, use the previous cell's prior memory h0 and the current input information x. t The sigmoid function outputs a value between 0 and 1, σt, which determines which part of the cell state will be output. The current cell state matrix is ​​then passed through a tanh function and compared with the output control signal σt. t Multiply to determine the output information h of the next cell. t This provides feature information for subsequent target detection, and the specific calculation formulas are as shown in formulas (9) and (10);

[0148] σ t =W0[h t-1 x t ]+b0 (9)

[0149] h t =σ t ×tanh(C t (10)

[0150] h t The hidden layer states are represented by W0, the learnable parameter matrix of the convolutional kernel is represented by b0, and the learnable bias parameter matrix is ​​represented by σ. t This represents the output control signal, h. t This serves as the target saliency map for each convolutional feature map. The process of outputting the corresponding target saliency map through the LSTM neural network is repeated for each convolutional feature map, and so on. The resulting target saliency maps are h1, h2, h3, h4, and h5, respectively.

[0151] Step 207: Perform linear fitting on multiple target saliency maps to obtain the corresponding target refinement feature maps.

[0152] In this embodiment of the invention, a linear fitting operation is performed on multiple target saliency maps to obtain corresponding target refinement feature maps.

[0153] In one example of the present invention, the output information h obtained by LSTM is decoded using a fully connected layer. t The state is thus obtained, which allows us to acquire the encoding matrix M. t It consists only of 0s and 1s, where 1 represents the features of the retained region and 0 represents the features of the forgotten region. After linear transformation, the mask matrix and the input image features are... Figure 1 To.

[0154] Let the original image be 'a', and use the original image and the output information h of each layer... t The decoded encoding matrix M t The dot product operation is denoted as the mask matrix R. t The specific calculation formula is shown in formula (11):

[0155] M t ·a→R t (11)

[0156] The mask matrix obtained from the target saliency map h5 is denoted as R5, and the mask matrix of the target saliency map h4 is denoted as R4. The feature f extracted from the same position in the saliency maps R5 and R4 are respectively... a and f b Perform the outer product operation, that is, use the bilinear pooling function to combine the two sets of features extracted, and the specific calculation formula is shown in formula (12):

[0157] f x =(f a ) T f b (12)

[0158] f x These are the extracted feature parameters, f a and f a These represent two bilinear feature flows. Next, a summation pooling method is used to sum the bilinear features obtained at all positions as the features of this image. The specific calculation formula is shown in formula (13):

[0159] f x_all =∑ l f x (13)

[0160] The bilinear characteristic is calculated using formula (14): l

[0161]

[0162] Y is used to represent the result of bilinear feature fusion. Its negative square root is obtained and regularized. The specific calculation formula is shown in formula (15):

[0163]

[0164] Z represents the result of regularization of the Y parameter. The result is vectorized to obtain the final image feature f. c Then, the feature obtained at the same position in the target saliency map R3 is f. d , will f c and f d The features are integrated using matrix outer product, and the result is denoted as f. e The features obtained from the target saliency map R2 are then denoted as f. f , characteristic function f e and characteristic function f f The feature fusion method is as described above. By analogy, image features are progressively layered and recursively derived to render image details.

[0165] It is worth mentioning that the classifier is also an SVM classifier. The nonlinear SVM is used instead of the Softmax function in the traditional LSTM as the final classifier, which further improves the accuracy of saliency detection. The output is directly from the SVM classifier.

[0166] Step 208: Extract the corresponding target ontology data from the target refinement feature map, determine multiple corresponding target ontology regions based on the target ontology data, and identify the target ontology regions.

[0167] Furthermore, the target ontology data includes construction equipment ontology data and construction personnel ontology data, and the target ontology region includes construction equipment ontology region and construction personnel ontology region. Step 208 may include the following sub-steps:

[0168] S21. Extract the corresponding construction equipment body data from the target refinement feature map, and divide multiple construction equipment body regions with a preset radius, taking the location of each construction equipment body data as the region center.

[0169] Construction equipment body data refers to the characteristics and location data of the construction equipment itself.

[0170] In this embodiment of the invention, corresponding construction equipment body data is extracted from the target refinement feature map. Using the location of each construction equipment body data point as the region center, multiple construction equipment body regions with a preset radius are divided. That is, based on the obtained equipment body location data, a circular region is divided with the current location of the equipment body as the region center. The radius can be set according to requirements. Thus, construction equipment body regions with the same number of equipment can be divided according to the number of equipment involved in the target refinement feature map.

[0171] S22. Identify the pre-defined category color for the construction equipment body area according to the equipment type of the construction equipment associated with the construction equipment body area.

[0172] Category color refers to the pre-set color based on the type of equipment; the specific color can be determined according to requirements.

[0173] In this embodiment of the invention, the colors of the construction equipment body area are pre-defined according to the different equipment types.

[0174] S23. Extract the corresponding construction personnel ontology data from the target refined feature map, and divide multiple construction personnel ontology regions with a preset radius, taking the location of each construction personnel ontology data as the region center.

[0175] Construction worker body data refers to the physical characteristics and location data of construction workers.

[0176] In this embodiment of the invention, corresponding construction worker body data is extracted from the target refinement feature map. Using the location of each construction worker body data point as the region center, multiple construction worker body regions with a preset radius are divided. That is, based on the obtained location data of the construction workers, a circular region is divided with the current location of the construction worker body as the region center. The radius can be set according to requirements. Thus, construction worker body regions with a consistent number of workers can be divided according to the number of workers involved in the target refinement feature map.

[0177] S24. A pre-defined prominent border for the construction worker identifier associated with the construction worker's main body area.

[0178] A salient border is a border used to mark construction workers, making them salient in the target detail feature map.

[0179] In this embodiment of the invention, a prominent border is preset for the construction worker identifier associated with the construction worker body area.

[0180] Step 209: Based on the judgment results of whether multiple target body regions meet the preset conditions, determine whether there are construction safety hazards associated with the construction personnel of the target body data.

[0181] Furthermore, step 209 may include the following sub-steps:

[0182] S31. If there is an overlap between the area of ​​any construction equipment and the area of ​​any construction worker, it is determined that there is a construction safety hazard associated with the construction worker's area.

[0183] In this embodiment of the invention, if there is an overlap between any construction equipment body area and any construction worker body area, that is, if the construction equipment body area and the construction worker body area overlap, then it is determined that the construction worker poses a safety hazard.

[0184] S32. If there is no overlap between the areas of all construction equipment and the areas of each construction worker, then it is determined that there is no construction safety hazard associated with the construction worker's area.

[0185] In this embodiment of the invention, if there is no overlap between the body area of ​​all construction equipment and the body area of ​​each construction worker, that is, if there is no overlap between the body area of ​​all construction equipment and the body area of ​​each construction worker, then it is determined that there is no construction safety hazard associated with the body area of ​​the construction worker.

[0186] In this embodiment of the invention, when a set of images of construction equipment and personnel is received, image preprocessing is performed on the images to generate training sample data. A convolutional neural network model with an intersection-gap spatial attention mechanism is constructed using a preset VGGnet16 convolutional neural network model and a preset intersection-gap spatial neural network model. An initial saliency neural network model is constructed by combining this with a preset LSTM neural network model. The initial saliency neural network model is trained using the training sample data to generate a target saliency neural network model. When a construction site image to be identified is received, the image is input into the target saliency neural network model to generate a target refinement feature map. Corresponding target ontology data is extracted from the target refinement feature map. Multiple corresponding target ontology regions are determined based on the target ontology data and are identified. Based on the judgment results of whether multiple target ontology regions meet preset conditions, it is determined whether the construction personnel associated with the target ontology data are present. This paper addresses the technical problem of existing convolutional neural networks failing to effectively enhance features, leading to excessive noise and redundant pixels in images during construction scenes. This results in inaccurate identification of pixel feature differences between construction workers and equipment, hindering the identification of salient features in localized areas and preventing timely safety warnings. By combining eye-tracking prediction with the memory function of LSTM neural networks, and mimicking the biological theory of human visual saliency, a novel model for salient target location detection is developed. This model progressively refines the target feature map and finally uses a classification function to classify the image, achieving image recognition and effectively improving saliency detection performance. In a dataset of construction workers and equipment images, by detecting pixel feature differences between the human body and other types of equipment in the feature map, the model significantly amplifies human features, ultimately providing hazard warnings and protecting construction workers.

[0187] Please see Figure 3 , Figure 3 This is a structural block diagram of a construction safety early warning device based on a saliency neural network model provided in Embodiment 3 of the present invention.

[0188] This invention provides a construction safety early warning device based on a saliency neural network model, comprising:

[0189] The image preprocessing module 301 is used to perform image preprocessing on the construction equipment and personnel image set when the image set of construction equipment and personnel is received, and to generate training sample data.

[0190] The initial saliency neural network model construction module 302 is used to construct an inter-joint spatial attention mechanism convolutional neural network model by using a preset VGGnet16 convolutional neural network model and a preset inter-joint spatial neural network model, and to construct an initial saliency neural network model by combining a preset LSTM neural network model.

[0191] The target saliency neural network model generation module 303 is used to train the initial saliency neural network model using training sample data to generate the target saliency neural network model.

[0192] The target refinement feature map acquisition module 304 is used to generate a target refinement feature map by inputting the target saliency neural network model with the construction site image to be identified when the image to be identified is received.

[0193] The target ontology region acquisition module 305 is used to extract the corresponding target ontology data from the target refinement feature map, determine multiple corresponding target ontology regions based on the target ontology data, and identify the target ontology regions.

[0194] The safety hazard judgment module 306 is used to judge whether there are construction safety hazards associated with the construction personnel in the target body data based on the judgment results of whether multiple target body areas meet the preset conditions.

[0195] Furthermore, the target saliency neural network model generation module 303 includes:

[0196] The training refinement feature map acquisition submodule is used to input training sample data into the initial saliency neural network model and generate the corresponding training refinement feature map.

[0197] The training loss value acquisition submodule is used to calculate the training loss value between the trained refined feature map and the associated standard refined feature map.

[0198] The comparison submodule is used to compare the training loss value with the preset training standard threshold.

[0199] The first data processing submodule is used to stop training and generate a target saliency neural network model if the training loss value is less than or equal to the training standard threshold.

[0200] Furthermore, the target saliency neural network model generation module 303 also includes:

[0201] The second data processing submodule is used to adjust the parameters of the initial saliency neural network model according to the preset gradient if the training loss value is greater than the training standard threshold, and then jump to the step of inputting the training sample data into the initial saliency neural network model to generate the corresponding training refined feature map.

[0202] Furthermore, the target refinement feature map acquisition module 304 includes:

[0203] The convolutional image feature map acquisition submodule is used to train a convolutional neural network model with the intersection-gathering spatial attention mechanism based on the input image of the construction site to be identified, and to extract multiple corresponding convolutional image feature maps.

[0204] The image feature probability map acquisition submodule is used to perform normalization and mapping operations on multiple convolutional image feature maps to obtain multiple corresponding image feature probability maps.

[0205] The target saliency map acquisition submodule is used to train an LSTM neural network model by inputting the image feature probability map, and extracting multiple corresponding target saliency maps.

[0206] The linear fitting submodule is used to perform linear fitting operations on multiple target saliency maps to obtain the corresponding target refinement feature maps.

[0207] Furthermore, the target ontology data includes construction equipment ontology data and construction personnel ontology data, the target ontology region includes construction equipment ontology region and construction personnel ontology region, and the target ontology region acquisition module 305 includes:

[0208] The construction equipment body area acquisition submodule is used to extract the corresponding construction equipment body data from the target refinement feature map, and divide multiple construction equipment body areas with a preset radius, with the location of each construction equipment body data as the area center.

[0209] The category color identification submodule is used to identify the construction equipment body area with a preset category color according to the equipment type of the construction equipment associated with the construction equipment body area.

[0210] The submodule for acquiring the body region of construction workers is used to extract the corresponding body data of construction workers from the target refined feature map, and to divide multiple body regions of construction workers with a preset radius, with the location of each body data of construction workers as the center of the region.

[0211] The prominent border identifier submodule is used to pre-set prominent borders for construction worker identifiers associated with the construction worker body area.

[0212] Furthermore, the safety hazard assessment module 306 includes:

[0213] The first judgment submodule is used to determine that if there is an overlap between the area of ​​any construction equipment body and the areas of each construction worker body, the construction worker body associated with the construction worker body area has a construction safety hazard.

[0214] The second judgment submodule is used to determine that if there is no overlap between the body area of ​​all construction equipment and the body area of ​​each construction worker, then the construction worker associated with the body area of ​​the construction worker does not pose a construction safety hazard.

[0215] In this embodiment of the invention, when a set of images of construction equipment and personnel is received, image preprocessing is performed on the images to generate training sample data. A convolutional neural network model with an intersection-gap spatial attention mechanism is constructed using a preset VGGnet16 convolutional neural network model and a preset intersection-gap spatial neural network model. An initial saliency neural network model is constructed by combining this with a preset LSTM neural network model. The initial saliency neural network model is trained using the training sample data to generate a target saliency neural network model. When a construction site image to be identified is received, the image is input into the target saliency neural network model to generate a target refinement feature map. Corresponding target ontology data is extracted from the target refinement feature map. Multiple corresponding target ontology regions are determined based on the target ontology data and are identified. Based on the judgment results of whether multiple target ontology regions meet preset conditions, it is determined whether the construction personnel associated with the target ontology data are present. This paper addresses the technical problem of existing convolutional neural networks failing to effectively enhance features, leading to excessive noise and redundant pixels in images during construction scenes. This results in inaccurate identification of pixel feature differences between construction workers and equipment, hindering the identification of salient features in localized areas and preventing timely safety warnings. By combining eye-tracking prediction with the memory function of LSTM neural networks, and mimicking the biological theory of human visual saliency, a novel model for salient target location detection is developed. This model progressively refines the target feature map and finally uses a classification function to classify the image, achieving image recognition and effectively improving saliency detection performance. In a dataset of construction workers and equipment images, by detecting pixel feature differences between the human body and other types of equipment in the feature map, the model significantly amplifies human features, ultimately providing hazard warnings and protecting construction workers.

[0216] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0217] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0218] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0219] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0220] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0221] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A construction safety early warning method based on a saliency neural network model, characterized in that, include: When a set of images of construction equipment and personnel is received, image preprocessing is performed on the set of images to generate training sample data; A convolutional neural network model with an intersection-gap spatial attention mechanism is constructed by using a pre-set VGGnet16 convolutional neural network model and a pre-set intersection-gap spatial neural network model, and an initial saliency neural network model is constructed by combining a pre-set LSTM neural network model. The construction of the intersection-and-merge spatial attention mechanism convolutional neural network model includes: performing max pooling and average pooling on the feature map extracted by the VGGnet16 convolutional neural network model along the channel axis, performing an intersection-and-merge operation on the pooling results to generate a mask matrix, and multiplying the mask matrix with the feature map to obtain the enhanced feature map; The initial saliency neural network model is trained using the training sample data to generate the target saliency neural network model; When a construction site image to be identified is received, the image is input into the target saliency neural network model to generate a refined feature map of the target. Extract the corresponding target ontology data from the target refinement feature map, determine multiple corresponding target ontology regions based on the target ontology data, and identify the target ontology regions; Based on the judgment results of whether multiple target body regions meet the preset conditions, it is determined whether there are construction safety hazards associated with the construction personnel associated with the target body data.

2. The construction safety early warning method based on a saliency neural network model according to claim 1, characterized in that, The step of training the initial saliency neural network model using the training sample data to generate the target saliency neural network model includes: The training sample data is input into the initial saliency neural network model to generate the corresponding training refined feature map; Calculate the training loss value between the trained refined feature map and the associated standard refined feature map; Compare the training loss value with a preset training standard threshold; If the training loss value is less than or equal to the training standard threshold, then training is stopped and a target saliency neural network model is generated.

3. The construction safety early warning method based on a saliency neural network model according to claim 2, characterized in that, Also includes: If the training loss value is greater than the training standard threshold, the parameters of the initial saliency neural network model are adjusted according to the preset gradient, and the process jumps to the step of inputting the training sample data into the initial saliency neural network model to generate the corresponding training refinement feature map.

4. The construction safety early warning method based on a saliency neural network model according to claim 1, characterized in that, The step of receiving a construction site image to be identified and inputting the image into the target saliency neural network model to generate a refined target feature map includes: When a construction site image to be identified is received, the image is input into the convolutional neural network model with the intersection-joint spatial attention mechanism to train the model and extract multiple corresponding convolutional image feature maps. Normalization and mapping are performed on multiple convolutional image feature maps to obtain multiple corresponding image feature probability maps; The image feature probability map is used as input to the LSTM neural network model for training, and multiple corresponding target saliency maps are extracted. A linear fitting operation is performed on multiple target saliency maps to obtain corresponding target refinement feature maps.

5. The construction safety early warning method based on a saliency neural network model according to claim 1, characterized in that, The target ontology data includes construction equipment ontology data and construction personnel ontology data, and the target ontology region includes construction equipment ontology region and construction personnel ontology region. The steps of extracting corresponding target ontology data from the target refinement feature map, determining multiple corresponding target ontology regions based on the target ontology data, and identifying the target ontology regions include: Extract the corresponding construction equipment body data from the target refined feature map, and divide multiple construction equipment body regions with a preset radius, taking the location of each construction equipment body data as the region center. The construction equipment body area is identified by a preset category color according to the equipment type of the construction equipment associated with it. Extract the corresponding construction worker body data from the target refined feature map, and divide multiple construction worker body regions with the location of each construction worker body data as the region center. A pre-defined prominent border is set for the construction worker identifier associated with the construction worker body area.

6. The construction safety early warning method based on a saliency neural network model according to claim 5, characterized in that, The step of determining whether there are construction safety hazards associated with the construction personnel in the target ontology data based on the judgment results of whether multiple target ontology regions meet preset conditions includes: If there is an overlap between any of the construction equipment body areas and each of the construction personnel body areas, it is determined that the construction personnel associated with the construction personnel body area pose a construction safety hazard. If there is no overlap between the areas of all the construction equipment and the areas of each of the construction personnel, then it is determined that there is no construction safety hazard associated with the construction personnel in the area of ​​the construction personnel.

7. A construction safety early warning device based on a saliency neural network model, characterized in that, The construction safety early warning device based on the saliency neural network model is used to implement the construction safety early warning method based on the saliency neural network model as described in any one of claims 1-6, wherein the construction safety early warning device based on the saliency neural network model includes: The image preprocessing module is used to perform image preprocessing on the construction equipment and personnel image set when the image set of construction equipment and personnel is received, and to generate training sample data. The initial saliency neural network model construction module is used to construct an intersection-gap spatial attention mechanism convolutional neural network model by using a preset VGGnet16 convolutional neural network model and a preset intersection-gap spatial neural network model, and to construct an initial saliency neural network model by combining a preset LSTM neural network model. The target saliency neural network model generation module is used to train the initial saliency neural network model using the training sample data to generate the target saliency neural network model; The target refinement feature map acquisition module is used to generate a target refinement feature map by inputting the target saliency neural network model with the construction site image to be identified when a construction site image to be identified is received. The target ontology region acquisition module is used to extract corresponding target ontology data from the target refinement feature map, determine multiple corresponding target ontology regions based on the target ontology data, and identify the target ontology regions. The safety hazard judgment module is used to determine whether there are any construction safety hazards associated with the construction personnel in the target body data based on the judgment results of whether multiple target body regions meet preset conditions.

8. The construction safety early warning device based on a saliency neural network model according to claim 7, characterized in that, The target refined feature map acquisition module includes: The convolutional image feature map acquisition submodule is used to train the intersection-and-joint spatial attention mechanism convolutional neural network model by inputting the construction site image to be identified into the convolutional neural network model, and to extract multiple corresponding convolutional image feature maps. The image feature probability map acquisition submodule is used to perform normalization and mapping operations on multiple convolutional image feature maps to obtain multiple corresponding image feature probability maps. The target saliency map acquisition submodule is used to train the LSTM neural network model by inputting the image feature probability map into it, and to extract multiple corresponding target saliency maps. The linear fitting submodule is used to perform a linear fitting operation on multiple target saliency maps to obtain the corresponding target refinement feature maps.

9. The construction safety early warning device based on a saliency neural network model according to claim 7, characterized in that, The target ontology data includes construction equipment ontology data and construction personnel ontology data; the target ontology region includes a construction equipment ontology region and a construction personnel ontology region; the target ontology region acquisition module includes: The construction equipment body area acquisition submodule is used to extract the corresponding construction equipment body data from the target refined feature map, and divide multiple construction equipment body areas with a preset radius, with the location of each construction equipment body data as the area center. The category color identification submodule is used to identify the construction equipment body area with a preset category color according to the equipment type of the construction equipment associated with the construction equipment body area; The submodule for obtaining the body region of construction workers is used to extract the corresponding body data of construction workers from the target refined feature map, and to divide multiple body regions of construction workers with the location of each body data of construction workers as the center of the region. The prominent border identifier submodule is used to preset a prominent border for the construction worker identifier associated with the construction worker body area.

10. The construction safety early warning device based on a saliency neural network model according to claim 9, characterized in that, The safety hazard assessment module includes: The first judgment submodule is used to determine that if there is regional overlap between any of the construction equipment body areas and each of the construction personnel body areas, the construction personnel associated with the construction personnel body area have a construction safety hazard. The second judgment submodule is used to determine that if there is no regional overlap between all the construction equipment body areas and each of the construction personnel body areas, the construction personnel associated with the construction personnel body area do not pose a construction safety hazard.

Images