A method and system for intelligently identifying risks of high-altitude operations based on industrial scenarios

Abstract

The present invention discloses a method and system for intelligently identifying risks of high-altitude operations based on industrial scenarios, which relates to the field of AI visual technology, including: collecting real-time video data of the factory, performing frame operations on the video data, and obtaining image frames; establishing a high-altitude operation scene recognition model, identifying the high-altitude operation scene and the body shape of personnel; enhancing the image frames identified as high-altitude operation scenes, establishing a risk assessment model to calculate the risk value, judging the risk level according to the risk value, and performing automatic processing. The present invention establishes a high-altitude operation scene recognition model to identify the high-altitude operation scene and the body shape of personnel, quantifies the risks of the high-altitude operation scene through enhanced processing and risk assessment models, judges whether personnel wear safety helmets, safety clothes and safety hooks correctly, and whether dangerous actions are performed, judges the risk level according to the risk value, and performs automatic processing, thereby realizing real-time monitoring and early warning of safety risks in high-altitude operation scenes.

Description

Technical Field

[0001] The present invention relates to the field of AI vision technology, and specifically to a method and system for intelligently identifying risks of high-altitude operations based on industrial scenarios. Background Art

[0002] With the advancement of smart factory reform, factory management continues to develop and innovate. Climbing work refers to the situation where workers perform high-altitude work on scaffolding, PTZ and other equipment. How to further improve the safety of the work scene, ensure the personal safety of workers, and promote factory safety operations has become a focus of many factories. Therefore, the risk identification of climbing work has become an indispensable part of safe production management.

[0003] Although factories have paid enough attention to the safety management of height operations over the years, previous management methods have mainly focused on eliminating potential risks, and usually adopt methods such as on-site personnel patrolling or personnel monitoring in front of monitoring equipment. Once a problem or potential problem is found, reminders and interventions are made. However, from the perspective of actual management results, this method relies too much on the personal discovery and handling efficiency of inspectors in standardizing the work standards of on-site operators and preventing accidents, and the effect is not satisfactory. Especially in the context of the rapid advancement of smart factory intelligence, the need for safe operations needs to be addressed urgently. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is that the manual management method has the problems of insufficient timeliness and excessive cost and manpower consumption.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: an intelligent identification method for high-altitude work risks based on industrial scenarios, comprising: collecting real-time video data of the factory, performing frame operations on the video data, and obtaining image frames; establishing a high-altitude work scene recognition model, and identifying the high-altitude work scene and the body shape of personnel; enhancing the image frames identified as high-altitude work scenes, establishing a risk assessment model to calculate the risk value, judging the risk level according to the risk value, and performing automatic processing.

[0007] As a preferred solution of the intelligent identification method of risks of high-altitude operations based on industrial scenarios described in the present invention, the frame-taking operation is to extract static image frames from the real-time video stream at time intervals, which is expressed as:

[0008] ,

[0009] in, Represents a set of image frames extracted from a video; Indicates at time Video frame images captured at each moment; Indicates the captured time interval in the video stream.

[0010] As a preferred solution of the intelligent identification method of the risks of high-altitude operations based on industrial scenes described in the present invention, the high-altitude operation scene recognition model includes identifying the image frame, and when there is no high-altitude equipment or no personnel in the image frame, it is identified as a non-high-altitude operation scene; when there are both high-altitude equipment and personnel in the image frame, the overlapping area of ​​the high-altitude equipment and personnel is detected, which is expressed as:

[0011] ,

[0012] in, Indicates the overlap between climbing equipment and personnel; Indicates the area where the climbing equipment is located; Indicates the area where the person is located; uses OpenPose to process the image frame, extracts the key points of the human body, and determines the posture of the person based on the position and angle of the key points; when there are both climbing equipment and people in the image frame, and the overlap between the climbing equipment and the person is If the posture of the person is not climbing or hanging, it is judged as a non-climbing operation scene. If the posture of the person is climbing or hanging, it is judged as a climbing operation scene. When the overlap between the climbing equipment and the person is It is judged as a high-altitude operation scene.

[0013] As a preferred solution of the intelligent identification method for high-altitude operation risks based on industrial scenes described in the present invention, the enhancement processing includes denoising the image frames of the high-altitude operation scene by Gaussian filtering, and enhancing the image edges by using a sharpening filter; according to the key points of the human body extracted by OpenPose, the coordinates of the key points are expressed in the form of a two-dimensional array. , traverse all the key points of a person and record them separately and The minimum and maximum values ​​of are used to construct the minimum rectangular bounding box containing all key points. The coordinates of the upper left corner of the bounding box are , the coordinates of the lower right corner of the bounding box are According to the constructed bounding box, the image area within the bounding box is cropped from the original image frame to obtain the image of each person. The cropped image area maintains the order and timing information of the original image frame; the Farneback optical flow method is used to calculate the inter-frame optical flow and the mean amplitude of the optical flow , identify and extract key frames with obvious action changes, expressed as:

[0014] ,

[0015] ,

[0016] ,

[0017] Pre-set the threshold of the optical flow mean amplitude , when the frame The mean amplitude of When the frame is the key frame and extracted; among them, Representation frame and frame The optical flow field between Representation frame images; Representation frame images; Indicates that the optical flow is Directional component; Indicates that the optical flow is Directional component; Represents the total number of pixels in the image; Represents a set of key frames; Indicates calculation of two consecutive frames and The optical flow field between .

[0018] As a preferred solution of the intelligent identification method for high-altitude operations based on industrial scenarios described in the present invention, the risk assessment model includes the appearance recognition of personnel, and the appearance recognition includes collecting images of personnel who correctly wear safety helmets, safety clothes and safety hooks, and obtaining the color and shape characteristics of key points of personnel when they are correctly worn; by comparing the real-time collected personnel images with the correctly worn personnel images, it is determined whether the personnel are correctly wearing safety helmets, safety clothes and safety hooks, and the risk value of appearance recognition is calculated:

[0019] ,

[0020] in, Represents the risk value of shape recognition; Representative The value of the influencing factor; Representative The weight of the influencing factors.

[0021] As a preferred solution of the intelligent identification method for high-altitude operation risks based on industrial scenarios described in the present invention, the risk assessment model also includes action recognition of personnel. According to the personnel images of the extracted key frames, the spatiotemporal features are extracted using a three-dimensional convolutional neural network to capture the action details in the personnel images. The LSTM module is introduced to enhance the modeling ability of the risk assessment model for long-term action sequences in continuous key frames. The action classification is judged according to the model output, which is expressed as:

[0022] ,

[0023] ,

[0024] in, Represents the action features extracted by the 3D convolutional neural network; Represents the feature extraction function of a three-dimensional convolutional neural network; Represents the action characteristics after temporal modeling; represents the long short-term memory network; dangerous actions are divided into jumping, running, imbalance and falling, and the action classification is compared with the characteristics of dangerous actions to determine whether the person performs dangerous actions, which is expressed as:

[0025] ,

[0026] ,

[0027] in, represents cosine similarity; Indicates the preset Characteristics of a dangerous action; represents the dynamic time warping distance; Representation calculation and Dynamic time regularization distance between the two; pre-set the judgment threshold of dangerous actions and ,when and When it is judged that the person has not performed any dangerous action, the risk value output by the risk assessment model is ;when and When it is judged that the person has performed a dangerous action for a short time, the risk value of the action recognition is output , the risk value output by the risk assessment model is expressed as:

[0028] ,

[0029] when and When it is judged that the person is performing a continuous dangerous action, the risk value of the action recognition is output , the risk value output by the risk assessment model is expressed as:

[0030] ,

[0031] When the dangerous action is identified as falling, the risk value calculation is skipped and an alarm is issued directly; Represents the risk value output by the risk assessment model; Represents the risk value proportionality factor for shape recognition.

[0032] As a preferred solution of the intelligent identification method of industrial scene climbing operation risk according to the present invention, the risk level includes: When the risk level is no risk, no operation is performed; when When , the risk level is low risk, the dangerous behavior is marked in the original image frame, and the original image frame is output to generate a safety hazard record; when When the risk level is general risk, based on the automatic processing of low risk, the personnel will be reminded to stop dangerous behaviors through automatic broadcast notification; when When the risk level is a major risk, based on the automatic processing of general risks, an alarm is issued and the safety personnel are notified to go to the height operation scene for manual processing.

[0033] In the second aspect, the present invention also provides an intelligent identification system for risks of high-altitude operations in industrial scenarios, including: an acquisition module, which collects real-time video data of the factory through in-factory monitoring, extracts static image frames from the real-time video stream at time intervals, and uploads them to a scene recognition module; a scene recognition module, which establishes a high-altitude operation scene recognition model, and determines whether it is a high-altitude operation scene based on the area overlap between the high-altitude equipment and the personnel and the personnel's body shape; a risk identification module, which enhances the image frames of the high-altitude operation scene, establishes a risk assessment model, calculates the risk value by performing appearance and body shape recognition on the personnel, and performs automatic processing.

[0034] In a third aspect, the present invention further provides a computing device, comprising: a memory and a processor;

[0035] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for intelligently identifying risks of high-altitude operations based on industrial scenarios are implemented.

[0036] In a fourth aspect, the present invention further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, can implement the steps of the method for intelligently identifying risks of height operations based on industrial scenarios.

[0037] The beneficial effects of the present invention are as follows: the present invention establishes a climbing work scene recognition model to identify the climbing work scene and the body shape of the personnel, quantifies the risks of the climbing work scene through enhanced processing and risk assessment models, judges whether the personnel wear safety helmets, safety clothing and safety hooks correctly, judges whether the personnel perform dangerous actions through action recognition, judges the risk level according to the risk value and performs automated processing, realizes real-time monitoring and early warning of safety risks in the climbing work scene, and improves the safety of the workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0039] Figure 1 An overall flow chart of an intelligent identification method for risks of high-altitude operations based on industrial scenarios provided by one embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a bounding box of an intelligent risk identification method for high-altitude operations in industrial scenarios provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0042] Example 1, reference Figure 1 , which is an embodiment of the present invention, provides an intelligent identification method for risks of high-altitude operations based on industrial scenarios, comprising:

[0043] S1: Collect real-time video data from the factory, perform frame operations on the video data, and obtain image frames.

[0044] Furthermore, the surveillance cameras connected to the factory are used to collect video frame images in real time as the basic data of the model. The video data has rich spatiotemporal information and can provide a comprehensive perspective of the work site. However, directly processing the real-time video stream has problems such as large computational complexity and poor real-time performance, so it is necessary to perform frame operations on the video.

[0045] The frame grabbing operation refers to the process of extracting static image frames from the real-time video stream at a certain interval. The factory monitoring camera collects video data in real time and performs frame grabbing operations at preset time intervals, which can be expressed as:

[0046] ,

[0047] in, Represents a set of image frames extracted from a video; Indicates at time Video frame images captured at each moment; Indicates the captured time interval in the video stream.

[0048] The extracted image frames will be used as data input for the subsequent high-altitude operation scene recognition model. The frame-taking operation can effectively reduce the redundant information of the video data and retain the image frames at key moments. In the high-altitude operation scene recognition, continuous image frames can provide more stable feature extraction and help improve the recognition accuracy of the model. For example, by analyzing the key frames, the posture changes of the operator and the use of the equipment can be more accurately identified.

[0049] The frame-taking operation can significantly reduce the amount of data that needs to be processed, avoiding the large amount of computing resources required to directly process the full video traffic. Under the premise of ensuring recognition accuracy, the frame-taking operation can reduce the computational complexity of the system and improve processing efficiency.

[0050] S2: Establish a high-altitude operation scene recognition model to identify the high-altitude operation scene and personnel posture.

[0051] Furthermore, a climbing work scene recognition model is established to identify the image frames. When there is no climbing equipment or no personnel in the image frame, it is identified as a non-climbing work scene. The climbing equipment is generally a pan head or a scaffolding.

[0052] When there are both climbing equipment and personnel in the image frame, the area where each person is located is As a benchmark, detect the overlapping area of ​​climbing equipment and personnel. 80% of the regions are Overlapping can be judged as personnel In the climbing operation, it is expressed as:

[0053] ,

[0054] in, Indicates the overlap between climbing equipment and personnel; Indicates the area where the climbing equipment is located; Indicates the area where the person is located.

[0055] Furthermore, OpenPose is used to process image frames, extract key points of the human body, and judge the posture of the person based on the position and angle of the key points. OpenPose is a popular open source human posture estimation library that can detect the position of key points of the human body in real time. OpenPose can be cloned and compiled from the official repository. OpenPose is used to process real-time video frames, extract key points of the human body, parse the key point data returned by OpenPose, and judge the posture of the person based on the position and angle of the key points.

[0056] In the factory scaffolding or PTZ climbing scene, the human body posture of the operator can be divided into five categories: standing, bending, squatting, climbing and hanging. When the recognized human body posture is climbing and hanging, it indicates that the person is in a high-altitude working state.

[0057] When there are both climbing equipment and personnel in the image frame, and the overlap between the climbing equipment and personnel is If the person's posture is not climbing or hanging, it is judged as a non-height working scene; if the person's posture is climbing or hanging, it is judged as a height working scene.

[0058] When the overlap between climbing equipment and personnel It is judged as a high-altitude operation scene.

[0059] S3: Enhance the image frames identified as high-altitude work scenes, establish a risk assessment model to calculate the risk value, determine the risk level based on the risk value and perform automated processing.

[0060] Furthermore, before transmitting the picture frames of the climbing operation scene to the risk assessment model, the image frames of the climbing operation scene are denoised by Gaussian filtering, and the filtered image uses a sharpening filter to enhance the image edge, which is expressed as:

[0061] ,

[0062] in, is the input image pixel; represents the standard deviation of Gaussian filtering; is the radius of the filter kernel; i, j represent the offset within the filter kernel range, traversing all pixels of the filter kernel.

[0063] According to the human key points extracted by OpenPose, the coordinates of the key points are expressed in the form of a two-dimensional array , traverse all the key points of a person and record them separately and The minimum and maximum values ​​are used as the coordinates of the upper left corner of the bounding box, and the maximum value is used as the coordinates of the lower right corner of the bounding box.

[0064] Construct the minimum rectangular bounding box containing all key points, and the coordinates of the upper left corner of the bounding box are , the coordinates of the lower right corner of the bounding box are , use the image cropping function of the OpenCV library to determine the range of the cropping area based on the coordinates of the upper left and lower right corners of each person's bounding box, from the original image The image area within the bounding box is cropped out, and the cropped image is the personnel area of ​​each person , the cropped image area maintains the order and timing information of the original image frame, expressed as:

[0065] ,

[0066] It should be noted that the bounding box segmentation method can effectively crop out the area containing all key points, which can usually completely include the entire body of the operator. When applied, if part of the operator's body exceeds the range of the key points, the bounding box may not be able to fully cover it. At this time, the size of the bounding box can be appropriately expanded to ensure the integrity of the personnel area.

[0067] After obtaining the personnel area, key frames with obvious action changes are extracted from the video frame sequence to reduce the interference of redundant frames and improve the accuracy of action recognition.

[0068] Use the Farneback optical flow method to calculate the inter-frame optical flow and the mean amplitude of the optical flow , identify and extract key frames with obvious action changes, reduce the amount of calculation for action recognition, and extract key frames based on the mean amplitude of optical flow, expressed as:

[0069] ,

[0070] ,

[0071] ,

[0072] Pre-set the threshold of the optical flow mean amplitude , when the frame The mean amplitude of When the frame is the key frame and extracted; among them, Representation frame and frame The optical flow field between Representation frame images; Representation frame images; Indicates that the optical flow is Directional component; Indicates that the optical flow is Directional component; Represents the total number of pixels in the image; Represents a set of key frames; Indicates calculation of two consecutive frames and The optical flow field between .

[0073] It should be noted that enhancement processing can effectively improve the accuracy of risk factor assessment and action recognition. De-noising and sharpening processing can significantly enhance image quality and make subsequent image segmentation more accurate. Key frame extraction reduces redundant frames by calculating optical flow changes, thereby improving the efficiency and accuracy of action recognition.

[0074] Furthermore, the enhanced video frame image is passed to the risk assessment model for evaluation and identification. The evaluation and identification is divided into two parts: appearance recognition and action recognition. Appearance recognition defines three risk factors: 1. Whether the operator wears a safety helmet correctly; 2. Whether the operator wears safety clothing correctly; 3. Whether the operator uses the safety hook correctly.

[0075] Collect images of people who correctly wear safety helmets, safety clothing, and safety hooks, obtain the color and shape features of the key points of the people when they wear them correctly, and judge whether the people wear safety helmets, safety clothing, and safety hooks correctly by comparing the real-time collected images of the people with the images of the people wearing them correctly. Position the key points extracted by OpenPose, and use color and shape features to detect the safety helmet in the detected face and head areas. If the color and shape in the face and head areas correspond to the safety helmet, and the safety helmet covers the top and side areas of the face, it is considered to be worn correctly.

[0076] Safety clothing is detected through color and texture features, and whether the safety clothing is worn correctly is judged based on the detected position of the safety clothing and the position relationship between the key points of the human body; the safety hook is based on the set key points for wearing the safety hook, such as the waist and chest. If the safety hook is connected to the key points of the human body and the connection point is at the appropriate position of the safety belt, it is considered to be used correctly.

[0077] The risk value of shape recognition is calculated based on the three identified risk factors:

[0078] ,

[0079] in, Represents the risk value of shape recognition; Representative The value of the influencing factor ranges from ; Representative The weight of the influencing factors.

[0080] Furthermore, the risk assessment model also includes action recognition of personnel. Based on the extracted key frame personnel images, I3D uses three-dimensional convolution to extract spatiotemporal features in the video, which is very suitable for capturing action details. However, I3D alone has a weak ability to model the dependencies of long-term actions. By introducing the LSTM module, its ability to model long-term action sequences can be enhanced, especially to adapt to the continuous and complex action changes in high-altitude operation scenarios (such as adjusting the body posture multiple times to maintain balance).

[0081] The three-dimensional convolutional neural network is used to extract spatiotemporal features and capture the action details in the personnel image. The LSTM module is introduced to enhance the risk assessment model's modeling ability for long-term action sequences in continuous key frames. The action classification is judged based on the model output, which is expressed as:

[0082] ,

[0083] ,

[0084] in, Represents the action features extracted by the 3D convolutional neural network; Represents the feature extraction function of a three-dimensional convolutional neural network; Represents the action characteristics after temporal modeling; Represents a long short-term memory network.

[0085] Dangerous actions are divided into jumping, running, imbalance and falling. The action classification is compared with the characteristics of dangerous actions to determine whether the person is performing a dangerous action, which is expressed as:

[0086] ,

[0087] ,

[0088] in, represents cosine similarity; Indicates the preset Characteristics of a dangerous action; represents the dynamic time warping distance; Representation calculation and The dynamic time regularization distance between them; It should be noted that when personnel jump or run fast during high-altitude operations, there is a risk of imbalance. Failure to react to imbalance in time or inadequate protective measures may cause personnel to fall. Therefore, the detection of dangerous actions is an important part of risk assessment.

[0089] Pre-set the judgment threshold of dangerous actions and ,when and When it is judged that the person has not performed any dangerous action, the risk value output by the risk assessment model is .

[0090] when and When it is judged that the person has performed a dangerous action for a short time, the risk value of the action recognition is output , the risk value output by the risk assessment model is expressed as:

[0091] ,

[0092] when and When it is judged that the person is performing a continuous dangerous action, the risk value of the action recognition is output , the risk value output by the risk assessment model is expressed as:

[0093] ,

[0094] When the dangerous action is identified as falling, the risk value calculation is skipped and an alarm is issued directly; Represents the risk value output by the risk assessment model; Represents the risk value proportionality factor for shape recognition.

[0095] It should also be noted that the cosine similarity calculation It is used to measure the similarity between the currently identified action features and the preset dangerous action feature vector. The higher the similarity, the closer the current action is to the dangerous action.

[0096] Dynamic Time Warping Used to measure the similarity between the current action sequence and the preset dangerous action sequence, DTW can handle the variable length and speed problems of time series data and accurately match the action sequence.

[0097] At the same time, the two are combined to identify and judge dangerous actions. Using only cosine similarity will ignore the timing characteristics of the action, because cosine similarity only considers the spatial proximity of feature vectors, but does not consider the changes of actions over time. Dynamic time warping (DTW) can better utilize the timing information of the action, which is extremely important for the identification of certain actions (such as imbalance, falling, etc.).

[0098] Furthermore, according to the calculated risk value, automatic processing is performed. When the risk level is zero, no operation is performed.

[0099] when When the risk level is low, the dangerous behavior is marked in the original image frame, and the original image frame is output to generate a safety hazard record.

[0100] when When the risk level is general risk, based on the automatic processing of low risk, automatic broadcast notifications are used to remind personnel to stop dangerous behaviors.

[0101] when When the risk level is a major risk, based on the automatic processing of general risks, an alarm is issued and the safety personnel are notified to go to the height operation scene for manual processing.

[0102] The present embodiment also provides an intelligent identification system for risks of high-altitude operations in industrial scenarios, including: an acquisition module, which collects real-time video data of the factory through in-factory monitoring, extracts static image frames from the real-time video stream at time intervals, and uploads them to a scene recognition module; a scene recognition module, which establishes a high-altitude operation scene recognition model, and determines whether it is a high-altitude operation scene based on the area overlap between the high-altitude equipment and the personnel and the personnel's body shape; a risk identification module, which enhances the image frames of the high-altitude operation scene, establishes a risk assessment model, calculates the risk value by recognizing the appearance and body shape of the personnel, and performs automatic processing.

[0103] This embodiment also provides a computing device, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the method for intelligently identifying risks of high-altitude operations based on industrial scenarios as proposed in the above embodiments.

[0104] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for intelligently identifying risks of high-altitude operations based on industrial scenarios proposed in the above embodiment is implemented.

[0105] The storage medium proposed in this embodiment and the intelligent identification method for risks of high-altitude operations based on industrial scenarios proposed in the above-mentioned embodiments belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above-mentioned embodiments, and this embodiment has the same beneficial effects as the above-mentioned embodiments.

[0106] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccessMemory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

[0107] Example 2, reference Figure 2 , which is an embodiment of the present invention, provides an intelligent identification method for height operation risks based on industrial scenarios. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through simulation experiments.

[0108] Through the real-time video data collected by the factory monitoring, with protective measures taken, personnel simulated the possible risk situations in the climbing scene and conducted simulation experiments. The simulation situations were as follows: Subject 1 wore a safety helmet, safety clothes, and used a safety hook, and his posture was standing; Subject 2 wore a safety helmet, but not a safety suit, but used a safety hook, and his posture was bending over; Subject 3 wore all safety equipment and his posture was climbing; Subject 4 did not wear a safety helmet, but used a safety suit, but did not use a safety hook, and his posture was hanging; Subject 5 did not wear any safety equipment and his posture was standing. The test results are shown in the following table.

[0109] Table 1 Test results

[0110] ,

[0111] Among them, 1 means that the safety equipment is worn correctly, and 0 means that the safety equipment is not used. From the output pictures of the climbing work scene recognition model, 1, 2, 3, and 4 are judged as climbing work scenes. The overlap of 1, 2, and 3 is greater than 80%. Although the overlap of 4 is only 75%, the detected posture is hanging, so it also belongs to climbing work.

[0112] like Figure 2As shown in FIG. 1 , it is a schematic diagram of a bounding box constructed according to key points proposed in the present invention. The × part represents the key points of the human body extracted according to OpenPose, the solid line is the human body schematic diagram formed by connecting the key points, and the dotted line part is the bounding box of the standing portrait. It can be seen that the bounding box constructed according to the method proposed in the present invention can completely include the human body schematic diagram part, and this part is cropped out from the overall image as the image input for risk assessment model recognition.

[0113] The risk values ​​of objects 1, 2, 3, and 4 in the height operation scenario are calculated. Objects 1 and 4 are running, while objects 2 and 3 are stationary. The risk values ​​are shown in the following table.

[0114] Table 2 Risk value table

[0115] ,

[0116] In this embodiment, the weights of the appearance risk value and the action risk value are 50% each. It can be seen from the total risk index data that object 2 is a low risk and is recorded but not alarmed. Objects 1 and 3 are general risks, and the system automatically reminds relevant personnel through broadcasting. Object 4 is a major risk, and the system issues an alarm and notifies staff to rush to the scene to deal with it.

[0117] It can be seen from the experimental data that the advantage of the present invention is that it can accurately identify the safety status and actions of operators in real time, thereby evaluating the risk level. Compared with traditional methods, the present invention combines image processing and machine learning technology to improve the accuracy and real-time performance of recognition. Especially in terms of action recognition, by introducing I3D and LSTM models, the modeling ability of long-term action sequences is enhanced, so that the system can better identify complex working environments and behaviors, and timely warn of potential dangers. Not only has it achieved significant results in the experiment, it also provides an effective new method for industrial safety monitoring.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for intelligently identifying risks of high-altitude operations based on industrial scenarios, characterized in that: include: Collect real-time video data from the factory, perform frame operations on the video data, and obtain image frames; Establish a scene recognition model for high-altitude operations to identify scenes of high-altitude operations and personnel postures; The high-altitude operation scene recognition model includes identifying the image frame, and when there is no high-altitude equipment or no personnel in the image frame, identifying it as a non-high-altitude operation scene; When there are both climbing equipment and personnel in the image frame, the overlapping area of ​​the climbing equipment and personnel is detected, which is expressed as: Among them, R represents the overlap between the climbing equipment and the personnel; A represents the area where the climbing equipment is located; B represents the area where the personnel are located; Use OpenPose to process image frames, extract key points of the human body, and judge the posture of the person based on the position and angle of the key points; When there are both climbing equipment and personnel in the image frame, and the overlap between the climbing equipment and personnel R < 80%, if the posture of the personnel is not climbing or hanging, it is judged as a non-climbing operation scene; if the posture of the personnel is climbing or hanging, it is judged as a climbing operation scene; When the overlap between the climbing equipment and the personnel R ≥ 80%, it is judged as a climbing operation scene; The image frames identified as high-altitude work scenes are enhanced, a risk assessment model is established to calculate the risk value, and the risk level is determined based on the risk value and processed automatically.

2. The intelligent identification method for risks of high-altitude operations based on industrial scenarios according to claim 1 is characterized in that: The frame extraction operation is to extract static image frames from the real-time video stream at time intervals, which is expressed as: F={I(t)|t∈T} Among them, F represents a set of image frames extracted from the video; I(t) represents a video frame image captured at time t; and T represents a captured time interval in the video stream.

3. The intelligent identification method for risks of high-altitude operations based on industrial scenarios according to claim 2 is characterized in that: The enhancement processing includes performing denoising processing on the image frame of the climbing operation scene by Gaussian filtering and enhancing the image edge by using a sharpening filter; According to the key points of the human body extracted by OpenPose, the coordinates of the key points are expressed in the form of a two-dimensional array (x, y). All the key points of a person are traversed, and the minimum and maximum values ​​of x and y are recorded respectively. The minimum rectangular bounding box containing all the key points is constructed. The coordinates of the upper left corner of the bounding box are (x min ,y min ), the coordinates of the lower right corner of the bounding box are (x max ,y max ), according to the constructed bounding box, the image area within the bounding box is cropped from the original image frame to obtain the person image of each person, and the cropped image area maintains the order and timing information of the original image frame; The Farneback optical flow method is used to calculate the inter-frame optical flow and the mean amplitude of the optical flow M(t), and the key frames with obvious motion changes are identified and extracted, which is expressed as: F(t) = Farneback(It, It+1) K={I t ∣M(t)>θ} A threshold θ of the optical flow mean amplitude is set in advance. When the mean amplitude M(t) of frame t>θ, frame t is judged as a key frame and extracted; where F(t) represents the optical flow field between frame t and frame t+1; I t I represents the image of frame t; t+1 represents the image of frame t+1; F x Represents the component of optical flow in the x direction; F y represents the component of optical flow in the y direction; N represents the total number of pixels in the image; K represents the key frame set; Farneback (I t , I t+1 ) represents the calculation of two consecutive frames I t and I t+1 The optical flow field between .

4. The intelligent identification method for risks of high-altitude operations based on industrial scenarios according to claim 3 is characterized in that: The risk assessment model includes the appearance recognition of personnel, which includes collecting images of personnel who are correctly wearing safety helmets, safety clothing and safety hooks, and obtaining the color and shape characteristics of key points of personnel when they are correctly wearing them; By comparing the real-time collected personnel image with the correctly wearing personnel image, it is determined whether the personnel are wearing the helmet, safety clothing and safety hook correctly, and the risk value of appearance recognition is calculated: Among them, G f Represents the risk value of shape recognition; x i represents the value of the i-th influencing factor; w i Represents the weight of the i-th influencing factor.

5. The intelligent identification method for risks of high-altitude operations based on industrial scenarios according to claim 4 is characterized in that: The risk assessment model also includes the recognition of personnel actions. Based on the personnel images of the extracted key frames, the three-dimensional convolutional neural network is used to extract spatiotemporal features, capture the action details in the personnel images, and introduce the LSTM module to enhance the risk assessment model's modeling ability for long-term action sequences in continuous key frames. The action classification is determined based on the model output, which is expressed as: F I3D =ψ ( {I t } ) A=LSTM ( F I3D ) Among them, F I3D represents the action features extracted by the three-dimensional convolutional neural network; ψ represents the feature extraction function of the three-dimensional convolutional neural network; A represents the action features after time series modeling; LSTM (F I3D ) represents long short-term memory network; Dangerous actions are divided into jumping, running, imbalance and falling. The action classification is compared with the characteristics of dangerous actions to determine whether the person is performing a dangerous action, which is expressed as: S DTW =DTW ( A A i ) Among them, S cos represents cosine similarity; A i represents the characteristics of the preset i-th dangerous action; S DTW represents the dynamic time warping distance; DTW(A,A i ) represents the calculation of A and A i The dynamic time-warping distance between Pre-set the judgment threshold θS of dangerous actions cos and θS DTW , when S cos ≤θS cos And S DTW ≥θS DTW When the risk is G, it is judged that the person has not performed any dangerous action, and the risk value output by the risk assessment model is G f ; When S cos >θS cos And S DTW ≥θS DTW When the person performs a short-term dangerous action, the risk value output of the action recognition is G d =100, the risk value output by the risk assessment model is expressed as: G=αG f +(1-α)G d When S cos >θS cos And S DTW <θS DTW When the person is judged to be performing a continuous dangerous action, the risk value of the action recognition is output as G d =200, the risk value output by the risk assessment model is expressed as: G=αG f +(1-α)G d When the dangerous action is identified as falling, the risk value calculation is skipped and an alarm is issued directly; Where G represents the risk value output by the risk assessment model; α represents the risk value proportional coefficient of shape recognition.

6. The method for intelligently identifying risks of high-altitude operations based on industrial scenarios according to claim 5 is characterized in that: The risk levels include: when G = 0, the risk level is risk-free and no operation is performed; when 0 < G ≤ 30, the risk level is low risk, the dangerous behavior is marked in the original image frame, and the original image frame is output to generate a safety hazard record; when 30 < G ≤ 60, the risk level is medium risk, and on the basis of the automatic processing of low risk, the personnel are reminded to stop the dangerous behavior through automatic broadcast notification; when G > 60, the risk level is high risk, and on the basis of the automatic processing of medium risk, an alarm is made and the security personnel are notified to go to the on-site of the working-at-height scenario for manual processing.

7. An intelligent identification system for risks of high-altitude operations based on industrial scenarios using the method according to any one of claims 1 to 6, characterized in that: including: a collection module, which collects the real-time video data of the factory through the in-plant monitoring, extracts static image frames from the real-time video stream at time intervals, and uploads them to the scenario recognition module; a scenario recognition module, which establishes a working-at-height scenario identification model and determines whether it is a working-at-height scenario according to the regional overlap degree between the working-at-height equipment and the personnel and the body posture of the personnel; a risk recognition module, which enhances the image frames of the working-at-height scenario, establishes a risk assessment model, calculates the risk value through the external shape recognition and body posture recognition of the personnel, and performs automated processing.

8. A computing device, characterized in that including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: including: it stores computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 6 are implemented.

Images