Potential safety hazard discovery and early warning method for limited space operation

Abstract

The invention provides a potential safety hazard discovery and early warning method for limited space operation, and the method comprises the steps: collecting a video image through setting a camera at a construction site, carrying out the feature point extraction through a detection model by employing a convolutional neural network, obtaining the operation information in a construction fence, and recognizing a construction effective region; on the basis, the opening condition of the well lid, the detection result of the toxic gas monitoring equipment and the safety feature of well descending operation are recognized; and generating an alarm signal for the detected and identified abnormal information.

Description

Technical Field

[0001] The present invention relates to the field of operation safety and early warning technology, and in particular to a method for discovering and early warning safety hazards in confined space operations. Background Art

[0002] In recent years, safety accidents in confined space operations (manholes) have occurred frequently, including poisoning and suffocation, pedestrian falls, electric shocks, etc. Although relevant safety training and safety publicity are carried out every year, a large number of various safety accidents such as poisoning and suffocation, pedestrian falls, electric shocks, etc. still occur. The main cause of the accidents is that the construction process is not carried out in accordance with the specifications, there is no good supervision method at the construction site, and there is a lack of identification, analysis and early warning capabilities for safety hazards in the construction process.

[0003] How to timely and automatically detect potential safety hazards during the construction process, so as to proactively report incidents to supervisors rather than passively discover abnormalities, and reduce the occurrence of personal injury accidents during confined space operations, is of great importance.

[0004] Therefore, a method for discovering and warning safety hazards in confined space operations (manhole wells) is needed to solve the problem that construction sites lack the ability to identify, analyze and warn of safety hazards in the construction process. Summary of the invention

[0005] The purpose of the present invention is to provide a method for discovering and warning safety hazards in confined space operations (manholes) to solve the problem of lack of identification, analysis and warning capabilities of safety hazards in the construction process proposed in the above-mentioned background technology, and reduce the occurrence of personal injury accidents in the construction process. It sets up a camera at the construction site, which is responsible for collecting video images and transmitting them to the AI ​​algorithm box for AI video analysis, and generates abnormal alarms that are sent to the database server through a router; the database server stores the alarms received from the basic resource module. And an application server is deployed, which includes but is not limited to a data processing module and an alarm module, which is responsible for identifying the alarm information transmitted from the AI ​​edge algorithm box, data cleaning, data analysis and locating safety hazard factors and behaviors, generating alarms and notifying relevant personnel in the form of text messages.

[0006] The present invention specifically adopts the following technical solutions:

[0007] A method for discovering and warning of safety hazards in confined space operations: by setting up a camera at the construction site to collect video images, and then using a convolutional neural network through a detection model to extract feature points, obtain operation information within the construction fence, and identify the effective construction area; on this basis, identify the opening status of the manhole cover, the detection results of the toxic gas monitoring equipment, and the safety features of the downhole operation; generate an alarm signal for the abnormal information detected and identified.

[0008] Furthermore, the preprocessing of the detection model includes: converting the video frame from a standard RGB color space to a YCbCr color space to highlight the enclosure area and standardizing the converted image; and then using a Gaussian mixture model to establish a background model.

[0009] Furthermore, the detection model includes an adaptive convolutional neural network, and the processing process adopted after preprocessing includes:

[0010] Multi-scale convolution feature extraction: Use convolution kernels of different sizes to extract multi-scale image features to identify enclosure areas of different scales;

[0011] Adaptive weight adjustment is used to improve the adaptability of the model in different scenarios;

[0012] Then the activation function is optimized, and the activation function is dynamically selected according to the statistical characteristics of the input data through the adaptive activation function;

[0013] Finally, feature selection and dimensionality reduction are performed using principal component analysis to reduce the dimensionality of the extracted high-dimensional features in order to retain the most important features and reduce the impact of noise.

[0014] Furthermore, after being processed by the adaptive convolutional neural network, the detection model performs spatiotemporal feature fusion, adopts a three-dimensional convolutional neural network to extract spatiotemporal features, uses a three-dimensional convolutional neural network to extract spatiotemporal features in the video, combines the information of time series and spatial dimensions to detect safety hazards in the fenced area, and uses spatiotemporal feature image segmentation to segment the spatiotemporal feature map into the fenced area and the background through a graph cut-based segmentation algorithm.

[0015] Furthermore, the detection model performs dynamic geometric analysis and anomaly detection after spatiotemporal feature fusion, including: first, fitting the geometric features of the enclosure area, using Hough transform to detect the edge features of the enclosure, and describing the geometric shape of the enclosure by fitting a straight line model; then, performing least squares fitting on the detected edges to eliminate noise and enhance the accuracy of the geometric description; then performing local feature analysis of construction hazards, and using a gradient-based feature descriptor to detect potential construction hazards in the enclosure area; calculating the difference between local features and the geometric shape of the enclosure; and finally completing anomaly detection and timing analysis: using a long short-term memory network to analyze the video sequence of the construction site to detect abnormal behavior in timing.

[0016] Furthermore, based on the on-site installation of depth cameras, sound sensors and vibration sensors, the detection model adopts a multimodal data fusion mechanism, including: depth and color feature fusion, in the detection process, the depth information from the depth camera is fused with the color feature to enhance the detection accuracy of the enclosure area; multimodal fusion of sound and vibration data, using the data of the sound sensor and vibration sensor at the construction site for multimodal fusion; extracting frequency domain features from the sound signal and vibration signal through Fourier transform; and finally combining these frequency domain features with visual features to achieve comprehensive analysis of multimodal data.

[0017] Furthermore, the detection model adopts a dynamic feedback mechanism to dynamically adjust model parameters to adapt to different construction scenarios and environmental changes through feedback on detection results.

[0018] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the program.

[0019] A non-transitory computer-readable storage medium stores a computer program, which implements the steps of the method described above when executed by a processor.

[0020] Compared with the prior art, the present invention and its preferred solution use cameras to monitor the on-site construction process in real time, and combine with the innovatively designed machine learning model to establish an early warning mechanism for on-site construction safety hazards, which can quickly and accurately discover on-site construction safety hazards, improve on-site construction standards, and reduce casualties caused by non-standard construction. Therefore, the present invention has strong practicality and broad application prospects in the field of construction safety of confined space operations (manholes). BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0022] Figure 1 It is a system architecture diagram of an embodiment of the present invention.

[0023] Figure 2 It is a flow chart of the algorithm model implementation of the embodiment of the present invention.

[0024] Figure 3 It is a flow chart of the detection method of an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Hereinafter, the specific embodiments of the present application will be described in detail with reference to the accompanying drawings. According to these detailed descriptions, those skilled in the art can clearly understand the present application and can implement the present application. Without violating the principles of the present application, the features in the various embodiments can be combined to obtain new implementations, or certain features in certain embodiments can be replaced to obtain other preferred implementations.

[0026] In order to make the features and advantages of the present invention more clearly understood, the following embodiments are specifically described in detail with reference to the accompanying drawings:

[0027] Embodiment 1

[0028] like Figure 1 As shown, in the basic design scheme provided by the present invention, a confined space operation (manhole) safety hazard discovery and early warning system is provided, which is composed of a basic resource module, a platform module and other parts.

[0029] The basic resource module mainly includes terminal-side monitoring equipment. Cameras are set up at the construction site to collect video images and transmit them to the AI ​​algorithm box for AI video analysis. Abnormal alarms are generated and sent to the platform module database server through routers.

[0030] Therefore, the basic resource module also includes AI algorithm boxes, switches, and routers. The functions are as follows:

[0031] Monitoring equipment: Each construction site is equipped with a main camera and auxiliary cameras to provide comprehensive video monitoring. The main camera is used to monitor the main area, while the auxiliary camera is used to supplement the monitoring of blind spots or specific details.

[0032] Al algorithm box: used for data processing and AI analysis at the edge. Through the built-in AI algorithm, it can quickly process and intelligently analyze the monitoring data of the construction site cameras, and automatically supervise the operations on the construction site.

[0033] Switch: used to connect routers, Al edge algorithm boxes, primary and auxiliary cameras in the network, and forward data to the router device.

[0034] Router: In order to achieve remote data transmission, each construction site is equipped with an industrial router. The router can use the 4G / 5G network to send the abnormal alarm analyzed by the Al algorithm box to the platform module database server through the router.

[0035] The platform module includes a database server, which stores the alarms received from the basic resource module. The alarm analysis and processing platform is deployed on the application server, and includes but is not limited to a data processing module and an alarm module, which is responsible for identifying the alarm information sent by the AI ​​edge algorithm box, data cleaning, data analysis, and locating safety hazards and behaviors, generating alarms and notifying relevant personnel via SMS.

[0036] Embodiment 2

[0037] Based on the device provided in Example 1, an AI video analysis algorithm for automatically discovering safety hazards in confined space operations (manholes) is further provided. This algorithm accurately identifies safety hazards within the construction site enclosure through video analysis. The innovation of the algorithm is reflected in the extraction of multi-level spatiotemporal features, the fusion of multimodal data, dynamic feedback mechanism, and the use of multi-scale deep learning models. The algorithm ensures the accuracy and real-time performance of detection.

[0038] like Figure 2 As shown, the complete detection model includes the following steps, among which the multimodal data fusion mechanism and the dynamic feedback mechanism are the preferred settings:

[0039] S1. Video data preprocessing

[0040] S101. Color space conversion and standardization

[0041] In order to better highlight the enclosure area, the video frame I(t) is first converted from the standard RGB color space to the YCbCr color space. This can effectively reduce the impact of lighting changes and improve contrast:

[0042] Y=0.299R+0.587G+0.114B

[0043] Cb=0.564(BY),Cr=0.713(RY)

[0044] The converted image is normalized to reduce the impact of different lighting conditions. The normalization formula is as follows:

[0045]

[0046] Among them, μ and σ are the mean and standard deviation of the image respectively.

[0047] S102. Background modeling and foreground segmentation

[0048] In order to effectively separate the foreground (i.e. the enclosure area) and the background, a Gaussian mixture model (GMM) is used to build a background model. The probability of each pixel belonging to the background is calculated using the following formula:

[0049]

[0050] in, is the parameter of the model, and K represents the number of mixed Gaussian distributions.

[0051] S2. Adaptive Convolutional Neural Network (Adaptive CNN)

[0052] S201. Multi-scale convolutional feature extraction

[0053] In order to identify enclosures of different scales, convolution kernels K of different sizes are used. s Extract multi-scale image features:

[0054] F s (t) = K s *I norm (t)

[0055] Among them, * represents the convolution operation, and s represents the scale of the convolution kernel.

[0056] S202. Adaptive weight adjustment

[0057] The weights in the convolutional neural network are adaptively adjusted according to the features of the input image to improve the adaptability of the model in different scenarios. The adaptive weight update formula is:

[0058]

[0059] Among them, λ is the learning rate, which controls the step size of weight update.

[0060] S203. Activation function optimization

[0061] Improve the robustness of the network through adaptive activation functions, such as dynamically selecting ReLu or Leaky ReLu activation functions based on the statistical characteristics of the input data:

[0062]

[0063] Among them, τ is the dynamically set threshold.

[0064] S204. Feature selection and dimensionality reduction

[0065] The extracted high-dimensional features are reduced using principal component analysis (PCA) to retain the most important features while reducing the impact of noise:

[0066] F pca (t) = PCA(F S (t))

[0067] S3. Spatial-temporal feature fusion

[0068] S301. Three-dimensional Convolutional Neural Network (3D-CNN) Spatiotemporal Feature Extraction

[0069] Use a 3D convolutional neural network to extract spatiotemporal features in the video, and combine the information of time series and spatial dimensions to detect safety hazards in the fenced area:

[0070] F st (t)=3D-CNN(F pca (t))

[0071] 3D-CNN can effectively capture the motion features that change over time in videos, and is particularly suitable for complex scenes at construction sites. S302. Image segmentation based on spatiotemporal features

[0072] The spatiotemporal feature map is further segmented into the enclosure area B(t) and the background through a segmentation algorithm based on Graph Cut:

[0073] B(t)=GeaphCut(F st (t))

[0074] The algorithm classifies pixels into enclosure areas or background areas by constructing an energy function.

[0075] S4. Dynamic geometry analysis and anomaly detection

[0076] S401. Geometric feature fitting of enclosure area

[0077] Hough Transform is used to detect the edge features of the fence, and the geometric shape of the fence is accurately described by fitting the straight line model L(t):

[0078] L(t)=HoughTransform(B(t))

[0079] The detected edges are fitted with the least squares method to eliminate noise and enhance the accuracy of the geometric description.

[0080] S402. Analysis of local characteristics of construction hazards

[0081] In the enclosure area, a gradient-based feature descriptor is used to detect potential construction hazards. The difference between the local features and the enclosure geometry is calculated:

[0082]

[0083] Among them, D(F st (t), L(t)) represents the Euclidean distance between the feature and the geometric shape, β i is the weight coefficient.

[0084] S403. Anomaly detection and timing analysis

[0085] Use a long short-term memory network (LSTM) to analyze video sequences of construction sites and detect abnormal behaviors in time series:

[0086] A(t)=LSTM(S risk (t))

[0087] When the anomaly score A(t) exceeds the set threshold, the system issues a warning signal.

[0088] S5. Multimodal data fusion

[0089] S501. Fusion of depth and color features

[0090] During the detection process, the depth information D(t) from the depth camera is fused with the color feature C(t) to enhance the detection accuracy of the enclosure area:

[0091] F multi (t) = α·D(t) + β·C(t)

[0092] Among them, α and β are fusion coefficients, which are obtained through experimental optimization.

[0093] S502. Multimodal fusion of sound and vibration data

[0094] Use the data from the on-site sound sensor and vibration sensor for multimodal fusion. audio (t) and vibration signal V vib (t) Extract its frequency domain features through Fourier transform:

[0095] F audio-vib (t) = FFT(S audio (t))+FFT(V vib (t))

[0096] Finally, these frequency domain features are combined with the visual features F multi (t) Combine to achieve comprehensive analysis of multimodal data.

[0097] S6. Real-time feedback and adaptive adjustment

[0098] S601. Dynamic feedback mechanism

[0099] Through feedback from the test results, the model parameters are dynamically adjusted to adapt to different construction scenarios and environmental changes:

[0100]

[0101] in is the loss function, which is used to measure the error between the actual detection result and the expected result, and η is the learning rate.

[0102] like Figure 3 As shown, based on the design of the above detection model, the present invention provides a specific implementation scheme flow for reference, including the following steps:

[0103] Step S701 sets up the fence: Use the feature point extraction method of the convolutional neural network to locally extract the operation information within the construction fence, identify the effective construction area, and solve the problem of false alarms caused by the camera capturing passers-by.

[0104] Step S702: Open the manhole cover:

[0105] The detection model can identify the action of opening the manhole cover and record the time when the manhole cover is opened.

[0106] The model can identify whether the construction workers have performed an electrical inspection on the manhole cover before going down the well.

[0107] Step S703: Ventilate for 30 minutes:

[0108] The model can recognize the manhole cover opening action and record the time.

[0109] The time of going down the well is identified and compared with the time generated in step S2. If it is less than 30 minutes, an alarm is generated and it is determined that the ventilation time is less than 30 minutes.

[0110] Step S704: Toxic gas detection:

[0111] The model can recognize the image of the detection equipment, and then use OCR image detection to identify the detection result image on the toxic gas monitoring equipment, so that the data is converted into a standard detection result format.

[0112] Based on the identified results, it is automatically determined whether the test results are normal. When the test results are abnormal, an alarm will be generated when going down the well.

[0113] Step S705: Downhole operation:

[0114] The model can identify the action of going down the well, and the system records the time of going down the well.

[0115] The model can identify whether the personnel going down the mine are wearing safety helmets, safety belts, and their hands and feet are not exposed.

[0116] Based on the system's records of the time spent downhole, the model can identify whether there is someone holding the safety rope next to the manhole cover during the descent.

[0117] It should be noted that the detection model is pre-trained, and its training process is as follows: obtain image sample data, where the image sample data includes image samples and labels of image samples; input the image sample data into the initial detection model, and train the initial detection model according to the loss between the output of the initial detection model and the corresponding label to obtain the detection model.

[0118] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0119] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0121] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

[0122] The present invention is not limited to the above-mentioned optimal implementation mode. Anyone can derive other forms of a method for discovering and warning safety hazards in confined space operations under the inspiration of the present invention. All equivalent changes and modifications made within the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A method for discovering and warning potential safety hazards in confined space operations, characterized in that: By setting up cameras at the construction site to collect video images, and then using the detection model to extract feature points using a convolutional neural network, the operation information within the construction fence is obtained, and the effective construction area is identified. On this basis, the opening status of the manhole cover, the detection results of the toxic gas monitoring equipment, and the safety features of the downhole operation are identified; and an alarm signal is generated for the abnormal information detected and identified.

2. A method for discovering and warning potential safety hazards in confined space operations according to claim 1, characterized in that: The preprocessing of the detection model includes: converting the video frame from the standard RGB color space to the YCbCr color space to highlight the enclosure area and standardizing the converted image; and then using a Gaussian mixture model to establish a background model.

3. A method for discovering and warning potential safety hazards in confined space operations according to claim 2, characterized in that: The detection model includes an adaptive convolutional neural network, and the processing process adopted after preprocessing includes: Multi-scale convolution feature extraction: Use convolution kernels of different sizes to extract multi-scale image features to identify enclosure areas of different scales; Adaptive weight adjustment is used to improve the adaptability of the model in different scenarios; Then the activation function is optimized, and the activation function is dynamically selected according to the statistical characteristics of the input data through the adaptive activation function; Finally, feature selection and dimensionality reduction are performed using principal component analysis to reduce the dimensionality of the extracted high-dimensional features in order to retain the most important features and reduce the impact of noise.

4. A method for discovering and warning potential safety hazards in confined space operations according to claim 3, characterized in that: After being processed by the adaptive convolutional neural network, the detection model performs spatiotemporal feature fusion, adopts a three-dimensional convolutional neural network to extract spatiotemporal features, uses a three-dimensional convolutional neural network to extract spatiotemporal features in the video, combines the information of time series and spatial dimensions to detect safety hazards in the enclosure area, and uses spatiotemporal feature image segmentation to segment the spatiotemporal feature map into the enclosure area and the background through a segmentation algorithm based on graph cuts.

5. A method for discovering and warning potential safety hazards in confined space operations according to claim 4, characterized in that: The detection model performs dynamic geometric analysis and anomaly detection after spatiotemporal feature fusion, including: firstly, fitting the geometric features of the enclosure area, using Hough transform to detect the edge features of the enclosure, and describing the geometric shape of the enclosure by fitting a straight line model; then, performing least squares fitting on the detected edges to eliminate noise and enhance the accuracy of the geometric description; then, performing local feature analysis of construction hazards, using gradient-based feature descriptors to detect potential construction hazards in the enclosure area; calculating the difference between local features and the geometric shape of the enclosure; and finally, completing anomaly detection and time series analysis: using a long short-term memory network to analyze the video sequence of the construction site to detect abnormal behavior in time series.

6. The method for discovering and warning potential safety hazards in confined space operations according to claim 1, characterized in that: Based on the on-site installation of depth cameras, sound sensors and vibration sensors, the detection model adopts a multimodal data fusion mechanism, including: depth and color feature fusion, during the detection process, the depth information from the depth camera is fused with the color features to enhance the detection accuracy of the enclosure area; multimodal fusion of sound and vibration data, using the data of the sound sensor and vibration sensor at the construction site for multimodal fusion; extracting frequency domain features from the sound signal and vibration signal through Fourier transform; and finally combining these frequency domain features with visual features to achieve comprehensive analysis of multimodal data.

7. The method for discovering and warning potential safety hazards in confined space operations according to claim 1, characterized in that: The detection model adopts a dynamic feedback mechanism to dynamically adjust model parameters to adapt to different construction scenarios and environmental changes through feedback on detection results.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a method for discovering and warning safety hazards in confined space operations as described in any one of claims 1 to 7 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for discovering and warning safety hazards in confined space operations as described in any one of claims 1 to 7 are implemented.

Images