An image recognition-based engineering safety hazard intelligent identification system and method

Abstract

This invention belongs to the field of image analysis and visual recognition. It discloses an intelligent identification system and method for engineering safety hazards based on image recognition. The system obtains multi-source engineering hazard image features by collecting, processing, and fusing engineering monitoring images, and also acquires environmental and meteorological features, engineering personnel features, and constructs a knowledge structure graph. It obtains engineering safety association tags and calculates the risk relationship features between the environment and engineering personnel. After feature embedding and fusion processing of the above images and relationship features, it constructs an engineering safety hazard identification and prediction model based on corresponding engineering safety hazard prediction and identification methods. The model is used for risk monitoring of engineering construction safety hazards. The system achieves intelligent identification of engineering safety hazards by constructing a knowledge graph network representing the attribute feature vectors of the environment and personnel, and relying on a voting multi-value fusion classification neural network model for end-to-end fusion feature processing.

Description

Technical Field

[0001] This invention belongs to the field of image analysis and visual recognition technology, and in particular relates to an intelligent identification system and method for engineering safety hazards based on image recognition. Background Technology

[0002] In on-site safety supervision of engineering projects, manual inspections are inefficient, hazard identification is delayed, and risk prediction is insufficient. Existing technologies, such as patent CN121599980A, construct an intelligent identification framework with spatial context awareness and temporal evolution reasoning capabilities by fusing high-resolution image sequences with a pre-set digital twin model of engineering facilities. This framework enables accurate detection, structured analysis, and risk classification of abnormal surface conditions on water conservancy and hydropower engineering facilities. However, this technology is only applicable to water conservancy and hydropower engineering scenarios, which differ from conventional engineering construction scenarios. Furthermore, the aforementioned models use a simplistic deep residual network architecture for hierarchical feature extraction, rather than utilizing knowledge graphs for feature classification of the input model. Given the limited applicability of the model to various environments, the computational load is excessive, and the model construction remains relatively simplistic.

[0003] This invention addresses the problem of how to process and fuse engineering monitoring images to obtain multi-source engineering hazard image features, and how to fully utilize the construction of knowledge graph networks to obtain embeddable fusion features as input to a constructed voting multi-value fusion classification neural network model to achieve real-time prediction and identification of engineering safety hazards. It proposes an intelligent identification method for engineering safety hazards based on image recognition. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes an intelligent identification system and method for engineering safety hazards based on image recognition.

[0005] In a first aspect of the present invention, an intelligent identification method for engineering safety hazards based on image recognition is provided, the method comprising:

[0006] Collect and process fusion engineering monitoring images to obtain multi-source engineering hazard image features, acquire environmental and meteorological features and corresponding engineering personnel features of the engineering site, and acquire system and document features;

[0007] Construct an engineering image recognition knowledge graph that integrates the environmental meteorological features, the engineering personnel features, and the institutional document features; and calculate and obtain the risk relationship features of environmental engineering personnel based on the engineering safety association tags of the engineering image recognition knowledge graph.

[0008] After fusing the embedded features of the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features, a safety hazard identification feature is obtained. A corresponding engineering safety hazard prediction and identification method is then obtained. An engineering safety hazard identification and prediction model is constructed using the safety hazard identification feature and the engineering safety hazard prediction and identification method.

[0009] Based on the aforementioned engineering safety hazard identification and prediction model, the method outputs a cross-scene identification method for predicting engineering safety hazards by recognizing images of differentiated projects and the corresponding engineering image knowledge graphs.

[0010] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features. The site entrance and exit image features are obtained by comparing the image features of engineering workers entering the site with standard clothing image features.

[0011] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features, which are obtained through image filtering enhancement.

[0012] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features, which are obtained through image colorimetry.

[0013] Furthermore, the environmental meteorological characteristics include ambient temperature, ambient humidity, and average wind speed, and the engineering personnel characteristics include personnel density, the proportion of people wearing safety helmets, and the proportion of smokers.

[0014] Furthermore, the method is characterized by:

[0015] Before calculating and obtaining the risk relationship features of environmental engineering personnel based on the engineering safety association tags of the engineering image recognition knowledge graph, an engineering image recognition knowledge graph is constructed that integrates the environmental features, the engineering personnel features, and the institutional document features. The risk relationship features of environmental engineering personnel are then calculated and obtained based on the engineering safety association tags of the engineering image recognition knowledge graph.

[0016] Furthermore, the safety hazard identification features are obtained by embedding and fusing the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features. The embedded feature fusion is obtained by weighted splicing of the environmental meteorological features and the engineering personnel features after dimensional normalization with the multi-source engineering hazard image features and the environmental engineering personnel risk relationship features. The weighted splicing coefficient is obtained based on the ratio of engineering personnel feature nodes to system document feature nodes.

[0017] Furthermore, the engineering safety hazard identification and prediction model adopts a voting multi-value fusion neural network model, which is composed of a multi-source engineering hazard image neural network model constructed by combining the multi-source engineering hazard image features and the engineering safety hazard prediction and identification method, and an engineering safety hazard identification and prediction neural network model constructed by combining the safety hazard identification features and the engineering safety hazard prediction and identification method.

[0018] A system for intelligent identification of engineering safety hazards based on image recognition is also provided. This system includes a multi-source engineering hazard image feature processing and acquisition module, an environmental and meteorological feature data retrieval module, an engineering personnel feature data acquisition module, an engineering image recognition knowledge graph construction and relation feature calculation module, an engineering safety hazard identification and prediction model construction module, and an engineering safety hazard prediction module.

[0019] The multi-source engineering hazard image feature processing and acquisition module is used to collect and process fused engineering monitoring images to obtain multi-source engineering hazard image features.

[0020] The environmental meteorological feature data retrieval module is used to acquire environmental meteorological features.

[0021] The engineering personnel characteristic data acquisition module is used to acquire the corresponding engineering personnel characteristics of the engineering department and to acquire the characteristics of institutional documents;

[0022] The engineering image recognition knowledge graph construction and relationship feature calculation module is used to construct an engineering image recognition knowledge graph that integrates the environmental meteorological features, the engineering personnel features, and the institutional document features, and to calculate and obtain the environmental engineering personnel risk relationship features based on the engineering safety association tags of the engineering image recognition knowledge graph.

[0023] The engineering safety hazard identification and prediction model construction module is used to fuse the embedded features of the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features to obtain safety hazard identification features, acquire the corresponding engineering safety hazard prediction and identification method, and construct the engineering safety hazard identification and prediction model using the safety hazard identification features and the engineering safety hazard prediction and identification method.

[0024] The engineering safety hazard prediction module: Based on the engineering safety hazard identification and prediction model, it outputs a cross-scene identification method for predicting engineering safety hazards by analyzing images of differentiated projects and the corresponding engineering image recognition knowledge graph.

[0025] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features. The site entrance and exit image features are obtained by comparing the image features of engineering workers entering the site with standard clothing image features.

[0026] Therefore, the beneficial effects of this invention are achieved by collecting, processing, and fusing engineering monitoring images to obtain multi-source engineering hazard image features (including site entrance and exit images (identifying personnel safety), material storage area images (identifying environmental safety), and high-altitude work area images (detecting operational safety), etc.), and collecting environmental meteorological features (including temperature, humidity, wind speed, etc.), and correspondingly obtaining engineering personnel features (personnel density, proportion of people wearing safety helmets, proportion of smokers), constructing a knowledge graph that fuses environmental features and construction personnel features, and calculating environmental engineering personnel risk relationship features based on engineering safety association tags (environmental meteorological features, engineering personnel features, and safety risk coefficients to construct the knowledge graph and obtain corresponding node features and edge relationship features). After embedding and fusing the multi-source engineering hazard image features, environmental meteorological features, engineering personnel features, and environmental engineering personnel risk relationship features, and combining them with corresponding engineering safety hazard prediction and identification methods to construct an engineering safety hazard identification and prediction model, the risk monitoring of engineering construction safety hazards is carried out based on the engineering safety hazard identification and prediction model. By constructing a knowledge graph network that represents the attribute feature vectors of the environment and personnel, and relying on a voting multi-value fusion classification neural network model, intelligent identification of engineering safety hazards is achieved. The model realizes intelligent prediction of hazards by an end-to-end embedded fusion of knowledge graph features and image fusion processing features using a voting multi-value fusion classification neural network model. The constructed model has strong generalization ability and fine feature expression granularity. At the same time, combined with the knowledge graph association network, dynamic environmental adaptation of engineering construction safety hazards is achieved.

[0027] Further embodiments and improvements of the present invention will be described in conjunction with the accompanying drawings and specific examples. Attached Figure Description

[0028] Figure 1 This is a partial illustration of the safety hazard risk identification image of the present invention;

[0029] Figure 2 This is a schematic diagram of the intelligent identification method for engineering safety hazards based on image recognition according to the present invention;

[0030] Figure 3 This is a schematic diagram of an intelligent engineering safety hazard identification system based on image recognition, as described in an embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of an engineering image recognition knowledge graph in an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of a portion showing personnel working at height in an embodiment of the present invention;

[0033] Figure 6 This is a schematic diagram of an electronic device structure for implementing the method of the present invention in an embodiment of the present invention. Detailed Implementation

[0034] The invention will now be further described in conjunction with the accompanying drawings and specific embodiments. The voting multi-value fusion classification neural network model used in this invention is a model for multiple judgment voting classification of input data after specific modifications to the neural network model in the scenario of this application. It differs from the single neural network classification model in the processing of input features and the fusion of models.

[0035] The voting multi-value fusion classification neural network model of this invention is used for intelligent identification of engineering safety hazards. It is applied to the field of visual recognition in building engineering and belongs to information system integration services such as artificial intelligence systems in the construction production field. It also belongs to the emerging software and new information technology service industry.

[0036] In a first aspect of the present invention, an intelligent identification method for engineering safety hazards based on image recognition is provided, the method comprising:

[0037] Collect and process fusion engineering monitoring images to obtain multi-source engineering hazard image features, acquire environmental and meteorological features and corresponding engineering personnel features of the engineering site, and acquire system and document features;

[0038] Construct an engineering image recognition knowledge graph that integrates the environmental meteorological features, the engineering personnel features, and the institutional document features; and calculate and obtain the risk relationship features of environmental engineering personnel based on the engineering safety association tags of the engineering image recognition knowledge graph.

[0039] After fusing the embedded features of the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features, a safety hazard identification feature is obtained. A corresponding engineering safety hazard prediction and identification method is then obtained. An engineering safety hazard identification and prediction model is constructed using the safety hazard identification feature and the engineering safety hazard prediction and identification method.

[0040] Based on the aforementioned engineering safety hazard identification and prediction model, the method outputs a cross-scene identification method for predicting engineering safety hazards by recognizing images of differentiated projects and the corresponding engineering image knowledge graphs.

[0041] This application's intelligent identification method for engineering safety hazards based on image recognition, before constructing an engineering image recognition knowledge graph that integrates environmental features, engineering personnel features, and policy document features, uploads the policy document features for the model to process relevant feature data on the client side. The actual display of the policy document features within the client is shown in the form of an enterprise knowledge base interface. When the model calls relevant policy document features, it does so in the form of feature vectors. The feature vectors are represented using natural language processing (NLP) technology to represent the document as a feature space vector. The document is essentially a sequence of natural language text, and one of the core tasks of NLP is to convert unstructured text into computer-computable structured vectors, i.e., feature vectors, to construct the document's feature space and meet the numerical feature requirements of machine learning or deep learning. In this embodiment, the policy document features are obtained by constructing feature vectors based on the frequency of keyword occurrences and the set feature vector values ​​of those keywords. The specific implementation steps of this intelligent identification method for engineering safety hazards based on image recognition are as follows: Figure 2 As shown. The engineering monitoring images in this application can also be obtained by processing a camera at a fixed location.

[0042] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude work area image features. The site entrance and exit image features are obtained by comparing the image features of engineering workers entering the site with standard clothing image features. The calculation formula for the site entrance and exit image features is as follows:

[0043]

[0044] In the formula, g represents the image features of the construction site entrance / exit, N represents the size of the retrieval window containing the personnel image of the construction site entrance / exit, and is the number of pixels in a square retrieval window. To extract the R-channel pixel feature value of the i-th pixel in the window, The R-channel pixel feature value of the i-th pixel in the image of personnel wearing standard work clothes at the construction site entrance is extracted. To extract the G-channel pixel feature value of the i-th pixel in the window, The pixel feature value of the G channel of the i-th pixel in the image of personnel wearing standard work clothes at the construction site entrance is extracted. To extract the B-channel pixel feature value of the i-th pixel in the window, The B-channel pixel feature value of the i-th pixel in the image capture window when personnel are wearing standard work clothes at the construction site entrance.

[0045] In this embodiment, to facilitate the calculation of image features at the construction site entrance and exit, the pixels of the cropped window are represented with the top-left pixel as the origin and the pixel in row a, column a as the a×a-th pixel, where N is the square of a positive integer, and the value of N is less than the total number of pixels in the cropped window of the image when personnel are wearing standard work clothes at the construction site entrance and exit. This embodiment does not perform global image comparison; it only extracts features and calculates similarity for key target areas such as critical human body areas, and then feeds them into the machine learning model. Essentially, this removes invalid background noise, focuses on effective task information, reduces feature dimensionality, improves the accuracy and generalization ability of the subsequent voting multi-value fusion classification neural network model, and reduces computational load. In this embodiment, the construction site entrance and exit image is used to identify personnel safety, the material storage area image is used to identify environmental safety, and the high-altitude work area image is used to detect operational safety.

[0046] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude work area image features. The material storage area image features and the high-altitude work area image features are obtained through image filtering enhancement. The image filtering enhancement formula is as follows:

[0047]

[0048] In the formula, This represents the pixel value at (x, y) of the corresponding pixel point after image filtering and enhancement. This represents the pixel value at (x, y) of the corresponding pixel point before image filtering and enhancement. For brightness correction parameters, when When the value is between 0.4 and 0.7, it is used to enhance the dark areas of the material stacking area image to obtain the image features of the material stacking area. When the value is between 1.2 and 1.5, it is used to suppress the bright parts of the high-altitude work area image to obtain the high-altitude work area image features.

[0049] like Figure 5The image shown is a captured image of personnel working at height, but it does not mean that the safety hazard risk images corresponding to this application are only of this high-altitude operation. It also includes other types of engineering monitoring images, such as unloading points, discharge points, and a series of engineering monitoring images that have an impact on safety hazard risks.

[0050] In this embodiment, since material storage areas and high-altitude operation areas often face a series of parameter features that affect the realism of images, such as uneven lighting, backlighting or strong light, dust and fog, heavy shadows, low contrast, blurred target edges, and a lot of noise, this application performs lighting correction to reduce the influence of lighting in real-world scenes and obtain the real image feature values ​​of material storage area images and high-altitude operation area images in backlighting or shadow.

[0051] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude work area image features. The material storage area image features and the high-altitude work area image features are obtained through image chromaticity calculation. The chromaticity calculation formulas for the material storage area image features and the high-altitude work area image features are as follows:

[0052]

[0053] In the formula, The image features of the material storage area or the high-altitude operation area are defined as follows, where m is the number of pixels in the image of the material storage area or the high-altitude operation area. Let R be the pixel feature value of the j-th pixel in the image of the material storage area or the high-altitude operation area. Let G be the pixel feature value of the j-th pixel in the image of the material storage area or the high-altitude operation area. , where is the B-channel pixel feature value of the j-th material stacking area image or high-altitude operation area image.

[0054] The chromaticity mean is a first-order statistical feature, requiring only pixel summation and averaging, without complex convolutions or deep learning inference. It boasts low computational power consumption and strong real-time performance, making it compatible with surveillance cameras and embedded edge devices. It can be used for real-time assessment of material stacking and personnel work areas to determine anomalies, meeting the needs of online inspection at construction sites. It is unaffected by changes in brightness and lighting, exhibiting strong stability. However, construction site environments are characterized by complex lighting conditions, including backlighting, shadows, strong light, and day-night variations. While brightness is volatile, chromaticity is relatively stable. Using the chromaticity mean as a feature eliminates lighting interference, resulting in more stable image similarity and feature matching results, avoiding model misjudgments caused by brightness variations. Furthermore, based on the aforementioned image filtering and enhancement processing, the influence of light intensity is significantly reduced, thereby obtaining more realistic image features. By using the average color value of the image to represent image features, it can effectively avoid brightness interference from lighting, shadows, backlighting, etc. in construction site scenes. It can extract color information from material stacking areas and high-altitude operation areas using low-dimensional first-order statistics. This feature is simple to calculate and has strong real-time performance. It can be used as a lightweight input feature for similarity matching and machine learning models, improving the stability and efficiency of regional state recognition and compliance detection.

[0055] Furthermore, the environmental meteorological characteristics include ambient temperature, ambient humidity, and average wind speed, and the engineering personnel characteristics include personnel density, the proportion of people wearing safety helmets, and the proportion of smokers.

[0056] In this embodiment, among the environmental meteorological characteristics, excessively high temperatures, humidity, and wind speeds can easily cause physical discomfort to personnel, exacerbate the risks of working at heights, and increase safety hazards. Regarding the characteristics of engineering personnel, excessively high personnel density can easily lead to disorder and difficulties in emergency evacuation; a low rate of helmet wearing and a high rate of smoking directly increase the risks of fall injuries, fires, and violations of regulations. These meteorological and personnel characteristics, from the perspectives of environmental conditions and personnel behavior, jointly reflect the weakness of on-site safety management and can effectively characterize the level of engineering safety hazards.

[0057] Furthermore, before calculating and obtaining the risk relationship features of environmental engineering personnel based on the engineering safety association tags of the engineering image recognition knowledge graph, an engineering image recognition knowledge graph is constructed that integrates the environmental features, the engineering personnel features, and the institutional document features. The risk relationship features of environmental engineering personnel are then calculated and obtained based on the engineering safety association tags of the engineering image recognition knowledge graph. The calculation formula for the risk relationship features of environmental engineering personnel is as follows:

[0058]

[0059] In the formula, The risk relationship characteristics of the environmental engineers mentioned above. For the engineering personnel feature nodes, This refers to the set of neighbors of the engineering personnel feature nodes, which is also the set of policy document feature nodes. The inverse mean of the degrees of the engineering personnel feature nodes and the policy document feature nodes is given. The mean value of the degree of the correlation edges between the engineering personnel feature nodes and the institutional document feature nodes.

[0060] Normalization coefficients, typically used for aggregation features, are chosen to prevent excessively large node values ​​from leading to overly strong features. In this embodiment, each node generates an information feature degree based on its own characteristics and calculates the feature value with the root node to obtain the environmental engineering personnel risk relationship feature. A comprehensive calculation is then performed based on the degree and edges of the engineering personnel feature node and the policy document feature node, providing a more objective comprehensive data on the relationship between the fusion of policy document features and engineering personnel features and engineering hazard relationships. Normalization coefficients are often used as constraint terms for aggregation features to suppress feature weight imbalance caused by excessively large nodes. In this embodiment, each node generates an information feature degree based on its own attributes and constructs an environmental-personnel risk association feature by completing feature interaction operations with the root node. A comprehensive calculation is then performed by combining the degree and association edges of the engineering personnel and policy document feature nodes to objectively quantify the degree of association between the fusion of the two and engineering safety hazards. This method improves the multimodal feature fusion logic, provides effective data support for subsequent model building, and enhances the robustness and scenario applicability of the model. Figure 4 The diagram shown is a schematic diagram of the constructed knowledge graph. In the diagram, a and B represent other corresponding institutional document nodes and engineering personnel feature nodes. It does not mean that the construction of the knowledge graph in this application only includes two other institutional document nodes and engineering personnel feature nodes, a and B. In addition to the engineering safety technical standards mentioned in the diagram, other national standards and corresponding engineering personnel feature nodes can also be named. Furthermore, B1 in the diagram represents the feature size of the corresponding connection edge between the engineering personnel feature node and the other national standard institutional document B.

[0061] Furthermore, the safety hazard identification features are obtained by fusing the embedded features of the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features. The embedded feature fusion is obtained by normalizing the dimensions of the environmental meteorological features and the engineering personnel features, and then weighting and splicing them with the multi-source engineering hazard image features and the environmental engineering personnel risk relationship features. The weighted splicing coefficient is obtained based on the ratio of engineering personnel feature nodes to system document feature nodes. During the splicing process, the ratio of the number of engineering personnel feature nodes to the number of system document feature nodes is used as the splicing coefficient of the multi-source engineering hazard image features and the environmental engineering personnel risk relationship features.

[0062] Embedded feature fusion was performed on multi-source engineering hazard image features, environmental meteorological features, engineering personnel features, and environmental engineering personnel risk relationship features to construct safety hazard identification features. During the fusion process, the environmental meteorological features and engineering personnel features were first normalized to eliminate dimensional differences. Then, the normalized features were weighted and concatenated with the multi-source engineering hazard image features and environmental engineering personnel risk relationship features. The weighting coefficients were determined based on the ratio of engineering personnel feature nodes to policy document feature nodes, thereby achieving effective fusion of multimodal features.

[0063] Furthermore, the engineering safety hazard identification and prediction model adopts a voting multi-value fusion neural network model. This model is constructed by fusing a multi-source engineering hazard image neural network model built using the multi-source engineering hazard image features and the engineering safety hazard prediction and identification method, with an engineering safety hazard identification and prediction neural network model built using the safety hazard identification features and the engineering safety hazard prediction and identification method. The activation function of the hidden layer of the multi-source engineering hazard image neural network model is set as follows:

[0064]

[0065] In the formula, The activation function value of the multi-source engineering hazard image neural network model is given. The weighted sum of the features of the multi-source engineering hazard image is used as input to the neural network model of the multi-source engineering hazard image.

[0066] The hidden layer activation function of the neural network model for identifying and predicting engineering safety hazards is set as follows:

[0067]

[0068] In the formula, The activation function value of the neural network model for identifying and predicting safety hazards in the project. The weighted sum of the safety hazard identification features is used as input to the neural network model for identifying and predicting engineering safety hazards.

[0069] In this embodiment, the multi-source engineering hazard image neural network model has 3 hidden layers, each with 64 neurons, a learning rate of 0.00012, 500 iterations, an Adam optimizer, and cross-entropy loss as the loss function. The engineering safety hazard identification and prediction neural network model has 3 hidden layers, each with 128 neurons, a learning rate of 0.00015, 1000 iterations, an Adam optimizer, and cross-entropy loss as the loss function.

[0070] Both models employ three hidden layers, using 64 and 128 neurons respectively. The multi-source engineering hazard image neural network focuses on extracting basic features such as shallow texture, color, and edges, resulting in lower feature complexity. 64 neurons are sufficient to fully extract image information, avoiding redundancy. The engineering safety hazard identification and prediction model, however, requires the fusion of deep, multi-modal features from images, meteorological data, personnel data, and regulatory documents. This results in higher feature dimensions and more complex coupling relationships, thus the number of neurons is increased to 128. This enhances the model's nonlinear fitting and feature expression capabilities, adapting to the needs of multi-source heterogeneous data fusion.

[0071] Learning rate settings: The learning rate for the image prediction model is 0.00012, and for the multi-range image data prediction model it is 0.00015, both using small learning rates. Engineering hazard samples are characterized by complex scenarios and high noise levels; a small learning rate avoids excessive parameter updates that could lead to training oscillations and local optima. Since the prediction model incorporates richer features, appropriately increasing the learning rate accelerates convergence and balances training stability and efficiency.

[0072] Number of iterations: The image model iterates 500 times to fully learn the features of a single modality; the prediction model needs to explore the deep risk correlation between multimodal features, which is more difficult to train. Therefore, the number of iterations is increased to 1000 times to ensure that the model fully learns the correlation rules of hidden dangers.

[0073] Optimizer and loss function: The Adam optimizer is selected, which can adaptively adjust the learning rate to adapt to imbalanced and complex engineering safety samples; the cross-entropy loss can effectively measure the prediction bias of the hazard category and improve the classification and recognition accuracy.

[0074] In a further embodiment of this application, the voting multi-value fusion neural network model uses the multi-source engineering hazard image features and the engineering safety hazard prediction and identification method, as well as the corresponding engineering safety hazard prediction and identification real-time method generated by the multi-source engineering hazard image neural network model to update the multi-source engineering hazard image neural network model in real time;

[0075] The system utilizes the safety hazard identification features, the engineering safety hazard prediction and identification method, and the training set data comprised of the corresponding real-time engineering safety hazard prediction and identification method generated by the engineering safety hazard identification and prediction neural network model to perform real-time updates of the engineering safety hazard identification and prediction neural network model. After adding training data to the model, the learning rate is adjusted accordingly. In this embodiment, with the increase in model data, the learning rate is set to 0.00015, and the number of iterations after updating the corresponding training set data is set to 1000 to accommodate the expansion of training data. In this embodiment, the model uses the training set data to train the hyperparameters of each model, and then expands the training data to improve the hyperparameters accordingly, thereby achieving adaptive improvement of the output of the engineering safety hazard prediction and identification method.

[0076] Due to the uninterpretability of neural networks, activation functions can address the shortcomings of nonlinear model representation and solve linearly inseparable problems. However, when it comes to predicting engineering hazards and risks, the activation functions of general single neural network models have certain limitations in handling nonlinear changes and are not well-suited for the multimodal data fusion engineering hazard and risk prediction scenario described in this application. To ensure the full utilization of multimodal data from various image datasets when using multi-source data, this application adopts a voting multi-value fusion classification neural network model. Multiple neural network models are fused to process the training data, and the classification results of the classification neural network model are comprehensively judged to obtain the corresponding engineering hazard and risk level. This ensures the accuracy of engineering hazard and risk prediction by fully utilizing the training data.

[0077] In this embodiment, for the voting multi-value fusion classification neural network model, the two neural network models first determine whether to output an engineering safety hazard prediction and identification method through voting. If the probability values ​​output by both neural networks are above 0.33, the classification accuracy is determined by the average of the final output feature probability values. The classification result is determined by the feature probability values ​​output by the neural networks. In this embodiment, it can be set that if the average of the final output feature probability values ​​is below 0.33, the output engineering safety hazard prediction and identification method corresponds to an identification error. If the average of the final output feature probability values ​​is above 0.33 (excluding 0.33), the output engineering safety hazard prediction and identification method corresponds to a medium risk. If the average of the final output feature probability values ​​is above 0.33 (excluding 0.33) and equal to 1, the output engineering safety hazard prediction and identification method corresponds to a high risk. The corresponding risk level determines the output of the corresponding safety hazard, such as the failure to configure fire extinguishing equipment as required in the stacking area. This includes not only this type of safety hazard, but also, but is not limited to, the identification of a series of safety hazards such as the non-use of safety protection doors at each unloading port.

[0078] In this embodiment, after the voting multi-value fusion classification neural network model outputs the safety hazard risk level, it matches the corresponding feature situation according to the corresponding output feature probability value. For example, a probability value of 0.387 corresponds to the safety hazard of no safety protection shed at the ground feed inlet, which corresponds to a medium risk. Figure 1 The image shown is a partial illustration of the corresponding safety hazard risk identification.

[0079] A system for intelligent identification of engineering safety hazards based on image recognition is also provided. This system includes a multi-source engineering hazard image feature processing and acquisition module, an environmental and meteorological feature data retrieval module, an engineering personnel feature data acquisition module, an engineering image recognition knowledge graph construction and relation feature calculation module, an engineering safety hazard identification and prediction model construction module, and an engineering safety hazard prediction module.

[0080] The multi-source engineering hazard image feature processing and acquisition module is used to collect and process fused engineering monitoring images to obtain multi-source engineering hazard image features.

[0081] The environmental meteorological feature data retrieval module is used to acquire environmental meteorological features.

[0082] The engineering personnel characteristic data acquisition module is used to acquire the corresponding engineering personnel characteristics of the engineering department and to acquire the characteristics of institutional documents;

[0083] The engineering image recognition knowledge graph construction and relationship feature calculation module is used to construct an engineering image recognition knowledge graph that integrates the environmental meteorological features, the engineering personnel features, and the institutional document features, and to calculate and obtain the environmental engineering personnel risk relationship features based on the engineering safety association tags of the engineering image recognition knowledge graph.

[0084] The engineering safety hazard identification and prediction model construction module is used to fuse the embedded features of the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features to obtain safety hazard identification features, acquire the corresponding engineering safety hazard prediction and identification method, and construct the engineering safety hazard identification and prediction model using the safety hazard identification features and the engineering safety hazard prediction and identification method.

[0085] The engineering safety hazard prediction module: Based on the engineering safety hazard identification and prediction model, it outputs a cross-scene identification method for predicting engineering safety hazards based on images of differentiated projects and retrieved knowledge graphs of corresponding engineering images. This application provides an image recognition-based intelligent engineering safety hazard identification system as follows: Figure 3 As shown.

[0086] Furthermore, the multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude work area image features. The site entrance and exit image features are obtained by comparing the image features of engineering workers entering the site with standard clothing image features. The calculation formula for the site entrance and exit image features is as follows:

[0087]

[0088] In the formula, g represents the image features of the construction site entrance / exit, N represents the size of the retrieval window containing the personnel image of the construction site entrance / exit, and is the number of pixels in a square retrieval window. To extract the R-channel pixel feature value of the i-th pixel in the window, The R-channel pixel feature value of the i-th pixel in the image of personnel wearing standard work clothes at the construction site entrance is extracted. To extract the G-channel pixel feature value of the i-th pixel in the window, The pixel feature value of the G channel of the i-th pixel in the image of personnel wearing standard work clothes at the construction site entrance is extracted. To extract the B-channel pixel feature value of the i-th pixel in the window, The B-channel pixel feature value of the i-th pixel in the image capture window when personnel are wearing standard work clothes at the construction site entrance.

[0089] In this embodiment, to facilitate the calculation of image features at the construction site entrance and exit, the pixels of the cropped window are represented with the top-left pixel as the origin and the pixel in row a, column a as the a×a-th pixel, where N is the square of a positive integer, and the value of N is less than the total number of pixels in the cropped window of the image when personnel are wearing standard work clothes at the construction site entrance and exit. This embodiment does not perform global image comparison; it only extracts features and calculates similarity for key target areas such as critical human body areas, and then feeds them into the machine learning model. Essentially, this removes invalid background noise, focuses on effective task information, reduces feature dimensionality, improves the accuracy and generalization ability of the subsequent voting multi-value fusion classification neural network model, and reduces computational load. In this embodiment, the construction site entrance and exit image is used to identify personnel safety, the material storage area image is used to identify environmental safety, and the high-altitude work area image is used to detect operational safety.

[0090] Therefore, the beneficial effects of this invention are achieved by collecting, processing, and fusing engineering monitoring images to obtain multi-source engineering hazard image features (including site entrance and exit images (identifying personnel safety), material storage area images (identifying environmental safety), and high-altitude work area images (detecting operational safety), etc.), and collecting environmental meteorological features (including temperature, humidity, wind speed, etc.), and correspondingly obtaining engineering personnel features (personnel density, proportion of people wearing safety helmets, proportion of smokers), constructing a knowledge graph that fuses environmental features and construction personnel features, and calculating environmental engineering personnel risk relationship features based on engineering safety association tags (environmental meteorological features, engineering personnel features, and safety risk coefficients to construct the knowledge graph and obtain corresponding node features and edge relationship features). After embedding and fusing the multi-source engineering hazard image features, environmental meteorological features, engineering personnel features, and environmental engineering personnel risk relationship features, and combining them with corresponding engineering safety hazard prediction and identification methods to construct an engineering safety hazard identification and prediction model, the risk monitoring of engineering construction safety hazards is carried out based on the engineering safety hazard identification and prediction model. By constructing a knowledge graph network that represents the attribute feature vectors of the environment and personnel, and relying on a voting multi-value fusion classification neural network model, intelligent identification of engineering safety hazards is achieved. The model realizes intelligent prediction of hazards by an end-to-end embedded fusion of knowledge graph features and image fusion processing features using a voting multi-value fusion classification neural network model. The constructed model has strong generalization ability and fine feature expression granularity. At the same time, combined with the knowledge graph association network, dynamic environmental adaptation of engineering construction safety hazards is achieved.

[0091] like Figure 6 As shown, the intelligent identification system and method for engineering safety hazards based on image recognition of the present invention operates through conventional components of a computing device, including a processor, memory, communication interface and bus.

[0092] Of course, it is understood that each embodiment of the present invention can achieve one of the effects individually, and the combination of multiple embodiments of the present invention can achieve all the above effects. However, it is not required that each embodiment of the present invention achieve all the above advantages and effects, because each embodiment of the present invention can constitute a separate technical solution and make one or more contributions to the prior art. Each embodiment of the present invention is not affected, and the solution can still be implemented even if one embodiment is deleted.

[0093] For any module structures not specifically defined in this invention, the existing technical descriptions shall prevail. The prior art mentioned in the foregoing background and specific embodiments sections can be considered part of this invention and used to understand the meaning of certain technical features or parameters.

Claims

1. A method for intelligent identification of engineering safety hazards based on image recognition, characterized in that, The method includes: Collect and process the monitoring images of the fusion project to obtain multi-source engineering hazard image features, acquire environmental and meteorological features and corresponding engineering personnel features of the project site, and acquire system and document features; Construct an engineering image recognition knowledge graph that integrates the environmental meteorological features, the engineering personnel features, and the institutional document features; and calculate and obtain the risk relationship features of environmental engineering personnel based on the engineering safety association tags of the engineering image recognition knowledge graph. After fusing the embedded features of the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features, a safety hazard identification feature is obtained. A corresponding engineering safety hazard prediction and identification method is then obtained. An engineering safety hazard identification and prediction model is constructed using the safety hazard identification feature and the engineering safety hazard prediction and identification method. Based on the aforementioned engineering safety hazard identification and prediction model, the method outputs a cross-scene identification method for predicting engineering safety hazards by recognizing images of differentiated projects and the corresponding engineering image knowledge graphs.

2. The intelligent identification method for engineering safety hazards based on image recognition as described in claim 1, characterized in that: The multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features. The site entrance and exit image features are obtained by comparing the image features of engineering workers entering the site with standard clothing image features.

3. The intelligent identification method for engineering safety hazards based on image recognition as described in claim 2, characterized in that: The multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features. The material storage area image features and the high-altitude operation area image features are obtained through image filtering enhancement.

4. The intelligent identification method for engineering safety hazards based on image recognition as described in claim 2 or 3, characterized in that: The multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features. The material storage area image features and the high-altitude operation area image features are obtained through image colorimetry.

5. A method for intelligent identification of engineering safety hazards based on image recognition as described in claim 1, 2, or 3, characterized in that: The environmental meteorological characteristics include ambient temperature, ambient humidity, and average wind speed, while the engineering personnel characteristics include personnel density, the proportion of people wearing safety helmets, and the proportion of smokers.

6. The intelligent identification method for engineering safety hazards based on image recognition as described in claim 1 or 3, characterized in that: Before calculating and obtaining the risk relationship features of environmental engineering personnel based on the engineering safety association tags of the engineering image recognition knowledge graph, an engineering image recognition knowledge graph is constructed that integrates the environmental features, the engineering personnel features, and the institutional document features. The risk relationship features of environmental engineering personnel are then calculated and obtained based on the engineering safety association tags of the engineering image recognition knowledge graph.

7. The intelligent identification method for engineering safety hazards based on image recognition as described in claim 6, characterized in that: The method involves embedding and fusing the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features. After processing, safety hazard identification features are obtained. The embedded feature fusion is obtained by normalizing the dimensions of the environmental meteorological features and the engineering personnel features, and then weighting and splicing them with the multi-source engineering hazard image features and the environmental engineering personnel risk relationship features. The weighting splicing coefficient is obtained based on the ratio of engineering personnel feature nodes to system document feature nodes.

8. The intelligent identification method for engineering safety hazards based on image recognition as described in claim 7, characterized in that: The engineering safety hazard identification and prediction model adopts a voting multi-value fusion neural network model, which is composed of a multi-source engineering hazard image neural network model constructed by combining the multi-source engineering hazard image features and the engineering safety hazard prediction and identification method, and an engineering safety hazard identification and prediction neural network model constructed by combining the safety hazard identification features and the engineering safety hazard prediction and identification method.

9. An intelligent identification system for engineering safety hazards based on image recognition, comprising a multi-source engineering hazard image feature processing and acquisition module, an environmental meteorological feature data retrieval module, an engineering personnel feature data acquisition module, an engineering image recognition knowledge graph construction and relation feature calculation module, an engineering safety hazard identification and prediction model construction module, and an engineering safety hazard prediction module, characterized in that: The multi-source engineering hazard image feature processing and acquisition module is used to collect and process fused engineering monitoring images to obtain multi-source engineering hazard image features. The environmental meteorological feature data retrieval module is used to acquire environmental meteorological features. The engineering personnel characteristic data acquisition module is used to acquire the corresponding engineering personnel characteristics of the engineering department and to acquire the characteristics of institutional documents; The engineering image recognition knowledge graph construction and relationship feature calculation module is used to construct an engineering image recognition knowledge graph that integrates the environmental meteorological features, the engineering personnel features, and the institutional document features, and to calculate and obtain the environmental engineering personnel risk relationship features based on the engineering safety association tags of the engineering image recognition knowledge graph. The engineering safety hazard identification and prediction model construction module is used to fuse the embedded features of the multi-source engineering hazard image features, the environmental meteorological features, the engineering personnel features, and the environmental engineering personnel risk relationship features to obtain safety hazard identification features, acquire the corresponding engineering safety hazard prediction and identification method, and construct the engineering safety hazard identification and prediction model using the safety hazard identification features and the engineering safety hazard prediction and identification method. The engineering safety hazard prediction module: Based on the engineering safety hazard identification and prediction model, it outputs a cross-scene identification method for predicting engineering safety hazards by analyzing images of differentiated projects and the corresponding engineering image recognition knowledge graph.

10. The intelligent identification system for engineering safety hazards based on image recognition as described in claim 9, characterized in that: The multi-source engineering hazard image features include site entrance and exit image features, material storage area image features, and high-altitude operation area image features. The site entrance and exit image features are obtained by comparing the image features of engineering workers entering the site with standard clothing image features.

Images