High-altitude operation violation behavior identification monitoring method and system based on deep learning

Abstract

The invention provides a deep learning-based high-altitude operation violation behavior identification monitoring method and system, and the method comprises the steps: configuring a signal tracking device for each operator, carrying out the operation tracking of the operator, obtaining an operation track corresponding to each operator, configuring a video shooting device for each operator, and carrying out the video shooting of each operator; the method comprises the following steps: acquiring an operation video stream corresponding to each operator, establishing a plurality of operation actions corresponding to each operator based on an operation track and the operation video stream, performing deep danger identification on each operation action based on an operation height corresponding to each operation action, and determining an illegal action corresponding to each operator according to a danger identification result, and a danger warning corresponding to the violation action is called and transmitted to the information listening and displaying equipment of the corresponding operator for danger early warning, a series of violation actions and standardized operation behaviors of the high-altitude operation personnel are identified and supervised, the climbing danger is effectively reduced, and the safety of the operator is protected.

Description

Technical Field

[0001] The present invention relates to the technical field of violation identification, and in particular to a method and system for identifying and monitoring violation behaviors in aerial work based on deep learning. Background Art

[0002] Falling accidents from heights are the most common of all kinds of safety accidents, and they have the greatest impact on safe construction and operations. Workers need to move frequently when working at heights, and in this case, they need to be supervised for a long time. Therefore, it is necessary to conduct video supervision of the workers. Traditional video surveillance systems are mainly manual monitoring, and video evidence is collected after the event. In addition, the architecture design of the video surveillance system is relatively simple, and it cannot perform real-time intelligent analysis of video images, cannot realize remote monitoring, and cannot promptly alarm abnormal events.

[0003] Therefore, the present invention provides a method and system for identifying and monitoring illegal behaviors in aerial work based on deep learning. Summary of the invention

[0004] The high-altitude work illegal behavior identification and monitoring method and system of the present invention based on deep learning can identify and monitor a series of illegal actions and standardized working behaviors of high-altitude workers, effectively reduce the danger of climbing and protect the safety of workers.

[0005] The present invention provides a method for identifying and monitoring illegal behaviors of aerial work based on deep learning, comprising:

[0006] Step 1: respectively configure a signal tracking device for each operator, and use the signal tracking device to track the operator's operation to obtain the operation trajectory corresponding to each operator;

[0007] Step 2: respectively configure a video capture device for each of the operators, and use the video capture device to collect the operation video stream corresponding to each of the operators;

[0008] Step 3: establishing a plurality of operation actions corresponding to each operator based on the operation trajectory and the operation video stream, and performing deep hazard identification on each operation action based on the operation height corresponding to each operation action;

[0009] Step 4: Determine the illegal action corresponding to each operator according to the hazard identification result, retrieve the hazard warning words corresponding to the illegal actions and transmit them to the information audio-visual display device of the corresponding operator for hazard warning.

[0010] In one practicable manner,

[0011] Also includes:

[0012] Each of the operators is equipped with a signal tracking device, a video shooting device and an information audio-display device.

[0013] In one practicable manner,

[0014] The step 1 comprises:

[0015] Step 11: After configuring a signal tracking device for each operator, respectively, send a test signal to each signal tracking device, obtain a feedback signal corresponding to each signal tracking device, obtain a number of feedback data contained in each feedback signal, and determine the tracking delay of the corresponding signal tracking device according to the data volume corresponding to each feedback data;

[0016] Step 12: During the operation of the operator, the real-time position of each operator is tracked to obtain the tracking position information corresponding to each operator, and the tracking position information is time-corrected using the corresponding tracking delay to obtain the position information corresponding to each operator at different times;

[0017] Step 13: Determine the stay time of each operator at different work positions according to the position information corresponding to each operator at different times, and determine the total work time corresponding to each operator, establish a work timeline of the corresponding operator according to the total work time, mark the corresponding stay time in the work timeline, and determine a number of work positions according to the marked points;

[0018] Step 14: Draw the three-dimensional working range of the corresponding operator according to the corresponding position information of each operator at different times, divide the three-dimensional working range into several working areas according to the working timeline, map each working position into the three-dimensional working range, count the several mapped working positions contained in each working area, connect the mapped working positions according to the working start time of each mapped working position in the working timeline, and generate the working trajectory corresponding to each operator.

[0019] In one practicable manner,

[0020] The step 2 comprises:

[0021] Step 21: After configuring a video capture device for each of the operators, a self-collection range is established based on the appearance information corresponding to each of the operators, and the shooting target of the corresponding video capture device is adjusted using the self-collection range. During the operation of the operator, the shooting area of ​​the corresponding video capture device is adjusted according to the real-time position of the operator with the shooting target as the center, and a real-time video stream of the shooting area is collected;

[0022] Step 22: locating the shooting target in the real-time video stream, performing edge enhancement on the shooting target in each frame of the video image, obtaining a plurality of shooting contour images of the operator, performing global average pooling on each of the shooting contour images, obtaining a plurality of operation features corresponding to each of the shooting contour images, and performing depth marking on each of the operation features;

[0023] Step 23: Based on the order of each of the captured contour images in the real-time video stream, the captured contour images are sorted to generate a reconstructed video stream, each of the depth marks is mapped to the reconstructed video stream, and the mapped video position corresponding to each of the depth marks is motion compensated to generate an operation video stream corresponding to each of the operators.

[0024] In one practicable manner,

[0025] The step 3 comprises:

[0026] Step 31: determining a plurality of key operation positions and non-key operation positions corresponding to the operator according to the operation trajectory, locating each of the key operation positions in the corresponding operation video stream, and dividing the operation video stream into a plurality of video segments according to the positioning results;

[0027] Step 32: Mark each of the video clips in the operation track to obtain a plurality of operation actions of the operator at different key operation positions, and identify the operation object corresponding to each operation action in the video clips;

[0028] Step 33: determining the operation height change information of the operator according to the operation trajectory, determining the operation height corresponding to each operation action based on the operation height change information, respectively retrieving a plurality of operation standards corresponding to each operation height, and establishing corresponding identification conditions according to the operation standards and operation objects;

[0029] Step 34: Perform deep training on each of the identification conditions to obtain a number of non-standard actions corresponding to each of the working heights, perform cluster analysis on a number of working actions corresponding to the same working height to obtain a number of working classes at the same height, use the non-standard actions corresponding to the working heights to perform hazard identification on each of the working classes at the same height, obtain hazard identification results corresponding to each of the operators, and display them separately.

[0030] In one practicable manner,

[0031] The step 4 comprises:

[0032] Step 41: determining a number of illegal actions corresponding to each operator according to the hazard identification result, respectively determining the working height corresponding to each illegal action, matching a corresponding hazard level for the illegal action, and retrieving a corresponding first warning word according to the hazard level;

[0033] Step 42: respectively obtaining the compliance action corresponding to each of the violation actions, determining the violation level corresponding to the violation action according to the difference between the violation action and the compliance action, and retrieving the corresponding second warning word according to the violation level;

[0034] Step 43: semantically reorganizing the first warning word and the second warning word to generate a danger warning word for each operator;

[0035] Step 44: Transmit the danger warning to the information audio-visual display device of the corresponding operator for danger warning.

[0036] In one practicable manner,

[0037] Also includes:

[0038] When each of the operators does not commit any illegal actions, the corresponding information audio-visual display device is controlled to play the corresponding audio according to the operator's instructions.

[0039] In one practicable manner,

[0040] Also includes:

[0041] Counting a number of illegal actions corresponding to each operator and analyzing the violation rate of each operator;

[0042] The operator evaluation list is constructed and displayed in descending order of violation rate.

[0043] The present invention provides a high-altitude work violation behavior recognition and monitoring system based on deep learning, comprising:

[0044] A tracking and acquisition module is used to configure a signal tracking device for each operator, and use the signal tracking device to track the operator to obtain the operation trajectory corresponding to each operator;

[0045] A video acquisition module, configured to configure a video capture device for each operator, and use the video capture device to capture an operation video stream corresponding to each operator;

[0046] A hazard identification module, used to establish a plurality of operation actions corresponding to each of the operators based on the operation trajectory and the operation video stream, and to perform in-depth hazard identification on each of the operation actions based on the operation height corresponding to each of the operation actions;

[0047] The danger warning module is used to determine the illegal action corresponding to each operator according to the danger identification result, retrieve the danger warning words corresponding to the illegal action and transmit them to the information audio-visual display device of the corresponding operator for danger warning.

[0048] In one practicable manner,

[0049] The tracking and acquisition module includes:

[0050] The first tracking unit is used to respectively configure a signal tracking device for each operator, respectively send a test signal to each signal tracking device, obtain a feedback signal corresponding to each signal tracking device, obtain a plurality of feedback data contained in each feedback signal, and determine a tracking delay of the corresponding signal tracking device according to a data volume corresponding to each feedback data;

[0051] The second tracking unit is used to track the real-time position of each of the operators during the operation of the operators, obtain the tracking position information corresponding to each of the operators, and perform time correction on the tracking position information using the corresponding tracking delay to obtain the position information corresponding to each of the operators at different times;

[0052] A third tracking unit is used to determine the stay time of each operator at different working positions according to the position information corresponding to each operator at different times, and determine the total working time corresponding to each operator, establish a working timeline of the corresponding operator according to the total working time, mark the corresponding stay time in the working timeline, and determine a number of working positions according to the marked points;

[0053] The fourth tracking unit is used to draw the three-dimensional working range of the corresponding operator according to the corresponding position information of each operator at different times, divide the three-dimensional working range into a number of working areas according to the working timeline, map each working position into the three-dimensional working range, count the number of mapped working positions contained in each working area, connect the mapped working positions according to the working start time of each mapped working position in the working timeline, and generate the working trajectory corresponding to each operator.

[0054] The achievable beneficial effects of the above technical solution are: in order to supervise and identify multiple operators at the same time, each operator is configured with a signal tracking device, a video shooting device and an information audio-visual device before starting work, and then the signal tracking device is used to track the operator's trajectory, and the video shooting device is used to shoot the operator's actions, so as to identify the danger of each operation action of the operator, thereby determining a number of illegal actions of the operator, and further matching each illegal action with a corresponding warning word, and issuing a danger warning to the operator. In this way, the behavior of multiple operators can be identified, and the operator can be issued a danger warning. The behavior recognition through the dual means of trajectory and video can achieve accurate identification, achieving the effect of simultaneous identification and simultaneous warning.

[0055] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0056] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0058] Figure 1 A schematic diagram of the workflow of a method for identifying and monitoring violations of aerial work based on deep learning in an embodiment of the present invention;

[0059] Figure 2 Schematic diagram of the composition of a high-altitude work violation behavior recognition and monitoring system based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0061] Example 1

[0062] This embodiment provides a method and system for identifying and monitoring illegal behaviors of aerial work based on deep learning, such as Figure 1 As shown, including:

[0063] Step 1: respectively configure a signal tracking device for each operator, and use the signal tracking device to track the operator's operation to obtain the operation trajectory corresponding to each operator;

[0064] Step 2: respectively configure a video capture device for each of the operators, and use the video capture device to collect the operation video stream corresponding to each of the operators;

[0065] Step 3: establishing a plurality of operation actions corresponding to each operator based on the operation trajectory and the operation video stream, and performing deep hazard identification on each operation action based on the operation height corresponding to each operation action;

[0066] Step 4: Determine the illegal action corresponding to each operator according to the hazard identification result, retrieve the hazard warning words corresponding to the illegal actions and transmit them to the information audio-visual display device of the corresponding operator for hazard warning.

[0067] In this example, the operation trajectory represents the trajectory that the operator walks while performing the operation;

[0068] In this example, each operator corresponds to a work track and a work video stream;

[0069] In this example, the work action represents the action performed by the worker when performing the work;

[0070] In this example, the illegal action indicates that the operator has taken an illegal action.

[0071] The working principle and beneficial effects of the above technical solution: in order to supervise and identify multiple operators at the same time, each operator is configured with a signal tracking device, a video shooting device and an information audio-visual device before starting work. The signal tracking device is used to track the operator's trajectory, and the video shooting device is used to shoot the operator's actions, so that each operation action of the operator is identified for danger, thereby determining a number of illegal actions of the operator, and further matching each illegal action with a corresponding warning word, and issuing a danger warning to the operator. In this way, the behavior of multiple operators can be identified, and the operator can be issued a danger warning. The behavior recognition through the dual means of trajectory and video can achieve accurate identification, achieving the effect of simultaneous identification and simultaneous warning.

[0072] Example 2

[0073] On the basis of Example 1, the method for identifying and monitoring violations of aerial work based on deep learning further includes:

[0074] Each of the operators is equipped with a signal tracking device, a video shooting device and an information audio-display device.

[0075] Example 3

[0076] On the basis of Example 1, the method for identifying and monitoring violations of aerial work based on deep learning is characterized in that step 1 comprises:

[0077] Step 11: After configuring a signal tracking device for each operator, respectively, send a test signal to each signal tracking device, obtain a feedback signal corresponding to each signal tracking device, obtain a number of feedback data contained in each feedback signal, and determine the tracking delay of the corresponding signal tracking device according to the data volume corresponding to each feedback data;

[0078] Step 12: During the operation of the operator, the real-time position of each operator is tracked to obtain the tracking position information corresponding to each operator, and the tracking position information is time-corrected using the corresponding tracking delay to obtain the position information corresponding to each operator at different times;

[0079] Step 13: Determine the stay time of each operator at different work positions according to the position information corresponding to each operator at different times, and determine the total work time corresponding to each operator, establish a work timeline of the corresponding operator according to the total work time, mark the corresponding stay time in the work timeline, and determine a number of work positions according to the marked points;

[0080] Step 14: Draw the three-dimensional working range of the corresponding operator according to the corresponding position information of each operator at different times, divide the three-dimensional working range into several working areas according to the working timeline, map each working position into the three-dimensional working range, count the several mapped working positions contained in each working area, connect the mapped working positions according to the working start time of each mapped working position in the working timeline, and generate the working trajectory corresponding to each operator.

[0081] In this example, feedback data refers to data included in the feedback signal;

[0082] In this example, the tracking delay refers to the delay of the signal tracking device during operation.

[0083] The working principle and beneficial effects of the above technical solution: In order to further determine and improve the effectiveness of trajectory drawing, a test signal is sent to the signal tracking device and then data analysis is performed on the feedback signal to determine the tracking delay of each signal tracking device, so that the collected position information is corrected when the real-time position of the operator is collected, and the corresponding position information of the operator at different times is determined. The operator's stay time at different work positions and the operator's total work time are further analyzed, thereby establishing an operation timeline, and marking multiple work positions in the operation timeline. The work positions are connected according to the work area corresponding to each work position and the work start time of each work position, thereby generating the operator's work trajectory. In this way, not only the trajectory generated for the operator can be generated, but also the stay time of each operator at different positions can be determined according to time, and the work trajectory is improved in time and space.

[0084] Example 4

[0085] On the basis of Example 1, the method for identifying and monitoring violations of aerial work based on deep learning, step 2 comprises:

[0086] Step 21: After configuring a video capture device for each of the operators, a self-collection range is established based on the appearance information corresponding to each of the operators, and the shooting target of the corresponding video capture device is adjusted using the self-collection range. During the operation of the operator, the shooting area of ​​the corresponding video capture device is adjusted according to the real-time position of the operator with the shooting target as the center, and a real-time video stream of the shooting area is collected;

[0087] Step 22: locating the shooting target in the real-time video stream, performing edge enhancement on the shooting target in each frame of the video image, obtaining a plurality of shooting contour images of the operator, performing global average pooling on each of the shooting contour images, obtaining a plurality of operation features corresponding to each of the shooting contour images, and performing depth marking on each of the operation features;

[0088] Step 23: Based on the order of each of the captured contour images in the real-time video stream, the captured contour images are sorted to generate a reconstructed video stream, each of the depth marks is mapped to the reconstructed video stream, and the mapped video position corresponding to each of the depth marks is motion compensated to generate an operation video stream corresponding to each of the operators.

[0089] In this example, the photographed object represents the operator;

[0090] In this example, the shooting area indicates the area where the operator is located;

[0091] In this example, the operation characteristics represent the characteristics exhibited by the operator when performing the operation;

[0092] In this example, the depth representation indicates the result of enhancing the operation features;

[0093] In this instance, motion compensation refers to the process of eliminating jitter components in the reconstructed video stream;

[0094] In this example, the real-time video stream is captured with the operator's knowledge.

[0095] The working principle and beneficial effects of the above technical solution are as follows: In order to reduce the impact of the external environment on the video shooting angle when conducting video supervision on each operator, the shooting range of each video shooting device is determined according to the appearance information of the operator, so that the shooting area of ​​the video shooting device is adjusted with the operator as the center during the shooting process, thereby obtaining a real-time video stream, and then the video image in the real-time video stream is enhanced and globally averaged pooled, and a number of operation features of each shooting contour image are determined, and depth marking is performed on it. Further, a reconstructed video stream is constructed based on the acquired shooting contour image, and the depth mark is used to determine the position of the reconstructed video stream for motion compensation, and corresponding compensation is performed. Finally, the operator's operation video stream is generated. In this way, it can be ensured that the video shooting device always shoots the operator, and the operator's behavior can also be preliminarily supervised, thereby improving the effectiveness of the operation video stream.

[0096] Example 5

[0097] On the basis of Example 1, the method for identifying and monitoring violations of aerial work based on deep learning, step 3 includes:

[0098] Step 31: determining a plurality of key operation positions and non-key operation positions corresponding to the operator according to the operation trajectory, locating each of the key operation positions in the corresponding operation video stream, and dividing the operation video stream into a plurality of video segments according to the positioning results;

[0099] Step 32: Mark each of the video clips in the operation track to obtain a plurality of operation actions of the operator at different key operation positions, and identify the operation object corresponding to each operation action in the video clips;

[0100] Step 33: determining the operation height change information of the operator according to the operation trajectory, determining the operation height corresponding to each operation action based on the operation height change information, respectively retrieving a plurality of operation standards corresponding to each operation height, and establishing corresponding identification conditions according to the operation standards and operation objects;

[0101] Step 34: Perform deep training on each of the identification conditions to obtain a number of non-standard actions corresponding to each of the working heights, perform cluster analysis on a number of working actions corresponding to the same working height to obtain a number of working classes at the same height, use the non-standard actions corresponding to the working heights to perform hazard identification on each of the working classes at the same height, obtain hazard identification results corresponding to each of the operators, and display them separately.

[0102] In this example, the key work position refers to the position where the operator performs the work, and the non-key work position refers to the position where the operator passes by, that is, the stay time at the key work position is 2 seconds or more.

[0103] The working principle and beneficial effects of the above technical solution: In order to reduce the supervision time and improve the efficiency of recognition, the operator's work trajectory is divided into key work positions and non-key work positions to locate the video clips of the operator in each key work position, and then the operator's work actions in different key work positions and the corresponding work objects are determined, so as to construct the corresponding recognition conditions in combination with the work standards corresponding to each work height, and further conduct in-depth training on the recognition conditions to obtain several non-standard actions of the operator at a half-day working height. By comparing the operator's work actions with non-standard actions, the hazard recognition results of the operation are generated, thereby completing the hazard identification and laying the foundation for subsequent warning work.

[0104] Example 6

[0105] On the basis of Example 1, the method for identifying and monitoring violations of aerial work based on deep learning, wherein step 4 comprises:

[0106] Step 41: determining a number of illegal actions corresponding to each operator according to the hazard identification result, respectively determining the working height corresponding to each illegal action, matching a corresponding hazard level for the illegal action, and retrieving a corresponding first warning word according to the hazard level;

[0107] Step 42: respectively obtaining the compliance action corresponding to each of the violation actions, determining the violation level corresponding to the violation action according to the difference between the violation action and the compliance action, and retrieving the corresponding second warning word according to the violation level;

[0108] Step 43: semantically reorganizing the first warning word and the second warning word to generate a danger warning word for each operator;

[0109] Step 44: Transmit the danger warning to the information audio-visual display device of the corresponding operator for danger warning.

[0110] The working principle and beneficial effects of the above technical solution are as follows: in order to effectively supervise the operators and make them pay attention to the danger, the corresponding first warning word is called according to the working height corresponding to the operator's illegal action, and the corresponding second warning word is called according to the violation level of the illegal action. Then, the two warning words are semantically reorganized to generate a danger warning word. The danger warning word can be used to warn the operator to achieve the effect of real-time warning, clarify the purpose of the warning, and let the operator understand the direction that needs to be adjusted.

[0111] Example 7

[0112] On the basis of Example 6, the method for identifying and monitoring violations of aerial work based on deep learning further includes:

[0113] When each of the operators does not commit any illegal actions, the corresponding information audio-visual display device is controlled to play the corresponding audio according to the operator's instructions.

[0114] The working principle and beneficial effects of the above technical solution are as follows: the operator can use the information audio-visual equipment for some leisure, which makes him / her feel happy and improves work efficiency.

[0115] Example 8

[0116] On the basis of Example 1, the method for identifying and monitoring violations of aerial work based on deep learning further includes:

[0117] Counting a number of illegal actions corresponding to each operator and analyzing the violation rate of each operator;

[0118] The operator evaluation list is constructed and displayed in descending order of violation rate.

[0119] The working principle and beneficial effects of the above technical solution are as follows: by counting the violation rate of operators, a certain warning effect can be provided to the operators, reminding them to work cautiously.

[0120] Example 9

[0121] This example provides a high-altitude work violation behavior recognition and monitoring system based on deep learning, including:

[0122] A tracking and acquisition module is used to configure a signal tracking device for each operator, and use the signal tracking device to track the operator to obtain the operation trajectory corresponding to each operator;

[0123] A video acquisition module, configured to configure a video capture device for each operator, and use the video capture device to capture an operation video stream corresponding to each operator;

[0124] A hazard identification module, used to establish a plurality of operation actions corresponding to each of the operators based on the operation trajectory and the operation video stream, and to perform in-depth hazard identification on each of the operation actions based on the operation height corresponding to each of the operation actions;

[0125] The danger warning module is used to determine the illegal action corresponding to each operator according to the danger identification result, retrieve the danger warning words corresponding to the illegal action and transmit them to the information audio-visual display device of the corresponding operator for danger warning.

[0126] In this example, the operation trajectory represents the trajectory that the operator walks while performing the operation;

[0127] In this example, each operator corresponds to a work track and a work video stream;

[0128] In this example, the work action represents the action performed by the worker when performing the work;

[0129] In this example, the illegal action indicates that the operator has taken an illegal action.

[0130] The working principle and beneficial effects of the above technical solution: in order to supervise and identify multiple operators at the same time, each operator is configured with a signal tracking device, a video shooting device and an information audio-visual device before starting work. The signal tracking device is used to track the operator's trajectory, and the video shooting device is used to shoot the operator's actions, so that each operation action of the operator is identified for danger, thereby determining a number of illegal actions of the operator, and further matching each illegal action with a corresponding warning word, and issuing a danger warning to the operator. In this way, the behavior of multiple operators can be identified, and the operator can be issued a danger warning. The behavior recognition through the dual means of trajectory and video can achieve accurate identification, achieving the effect of simultaneous identification and simultaneous warning.

[0131] Example 10

[0132] On the basis of Example 9, the high-altitude work violation behavior recognition and monitoring system based on deep learning, the tracking and collection module includes:

[0133] The first tracking unit is used to respectively configure a signal tracking device for each operator, respectively send a test signal to each signal tracking device, obtain a feedback signal corresponding to each signal tracking device, obtain a plurality of feedback data contained in each feedback signal, and determine a tracking delay of the corresponding signal tracking device according to a data volume corresponding to each feedback data;

[0134] The second tracking unit is used to track the real-time position of each of the operators during the operation of the operators, obtain the tracking position information corresponding to each of the operators, and perform time correction on the tracking position information using the corresponding tracking delay to obtain the position information corresponding to each of the operators at different times;

[0135] A third tracking unit is used to determine the stay time of each operator at different working positions according to the position information corresponding to each operator at different times, and determine the total working time corresponding to each operator, establish a working timeline of the corresponding operator according to the total working time, mark the corresponding stay time in the working timeline, and determine a number of working positions according to the marked points;

[0136] The fourth tracking unit is used to draw the three-dimensional working range of the corresponding operator according to the corresponding position information of each operator at different times, divide the three-dimensional working range into a number of working areas according to the working timeline, map each working position into the three-dimensional working range, count the number of mapped working positions contained in each working area, connect the mapped working positions according to the working start time of each mapped working position in the working timeline, and generate the working trajectory corresponding to each operator.

[0137] In this example, feedback data refers to data included in the feedback signal;

[0138] In this example, the tracking delay refers to the delay of the signal tracking device during operation.

[0139] The working principle and beneficial effects of the above technical solution: In order to further determine and improve the effectiveness of trajectory drawing, a test signal is sent to the signal tracking device and then data analysis is performed on the feedback signal to determine the tracking delay of each signal tracking device, so that the collected position information is corrected when the real-time position of the operator is collected, and the corresponding position information of the operator at different times is determined. The operator's stay time at different work positions and the operator's total work time are further analyzed, thereby establishing an operation timeline, and marking multiple work positions in the operation timeline. The work positions are connected according to the work area corresponding to each work position and the work start time of each work position, thereby generating the operator's work trajectory. In this way, not only the trajectory generated for the operator can be generated, but also the stay time of each operator at different positions can be determined according to time, and the work trajectory is improved in time and space.

[0140] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for identifying and monitoring illegal behaviors of aerial work based on deep learning, characterized in that: include: Step 1: respectively configure a signal tracking device for each operator, and use the signal tracking device to track the operator's operation to obtain the operation trajectory corresponding to each operator; Step 2: respectively configure a video capture device for each of the operators, and use the video capture device to collect the operation video stream corresponding to each of the operators; Step 3: establishing a plurality of operation actions corresponding to each operator based on the operation trajectory and the operation video stream, and performing deep hazard identification on each operation action based on the operation height corresponding to each operation action; Step 4: Determine the illegal action corresponding to each operator according to the hazard identification result, retrieve the hazard warning words corresponding to the illegal actions and transmit them to the information audio-visual display device of the corresponding operator for hazard warning.

2. The method for identifying and monitoring violations of aerial work based on deep learning as claimed in claim 1, characterized in that: Also includes: Each of the operators is equipped with a signal tracking device, a video shooting device and an information audio-display device.

3. The method for identifying and monitoring high-altitude work violation behaviors based on deep learning as claimed in claim 1, characterized in that: The step 1 comprises: Step 11: After configuring a signal tracking device for each operator, respectively, send a test signal to each signal tracking device, obtain a feedback signal corresponding to each signal tracking device, obtain a number of feedback data contained in each feedback signal, and determine the tracking delay of the corresponding signal tracking device according to the data volume corresponding to each feedback data; Step 12: During the operation of the operator, the real-time position of each operator is tracked to obtain the tracking position information corresponding to each operator, and the tracking position information is time-corrected using the corresponding tracking delay to obtain the position information corresponding to each operator at different times; Step 13: Determine the stay time of each operator at different work positions according to the position information corresponding to each operator at different times, and determine the total work time corresponding to each operator, establish a work timeline of the corresponding operator according to the total work time, mark the corresponding stay time in the work timeline, and determine a number of work positions according to the marked points; Step 14: Draw the three-dimensional working range of the corresponding operator according to the corresponding position information of each operator at different times, divide the three-dimensional working range into several working areas according to the working timeline, map each working position into the three-dimensional working range, count the several mapped working positions contained in each working area, connect the mapped working positions according to the working start time of each mapped working position in the working timeline, and generate the working trajectory corresponding to each operator.

4. The method for identifying and monitoring high-altitude work violations based on deep learning as claimed in claim 1, characterized in that: The step 2 comprises: Step 21: After configuring a video capture device for each of the operators, a self-collection range is established based on the appearance information corresponding to each of the operators, and the shooting target of the corresponding video capture device is adjusted using the self-collection range. During the operation of the operator, the shooting area of ​​the corresponding video capture device is adjusted according to the real-time position of the operator with the shooting target as the center, and a real-time video stream of the shooting area is collected; Step 22: locating the shooting target in the real-time video stream, performing edge enhancement on the shooting target in each frame of the video image, obtaining a plurality of shooting contour images of the operator, performing global average pooling on each of the shooting contour images, obtaining a plurality of operation features corresponding to each of the shooting contour images, and performing depth marking on each of the operation features; Step 23: Based on the order of each of the captured contour images in the real-time video stream, the captured contour images are sorted to generate a reconstructed video stream, each of the depth marks is mapped to the reconstructed video stream, and the mapped video position corresponding to each of the depth marks is motion compensated to generate an operation video stream corresponding to each of the operators.

5. The method for identifying and monitoring high-altitude work violation behaviors based on deep learning as claimed in claim 1, characterized in that: The step 3 comprises: Step 31: determining a plurality of key operation positions and non-key operation positions corresponding to the operator according to the operation trajectory, locating each of the key operation positions in the corresponding operation video stream, and dividing the operation video stream into a plurality of video segments according to the positioning results; Step 32: Mark each of the video clips in the operation track to obtain a plurality of operation actions of the operator at different key operation positions, and identify the operation object corresponding to each operation action in the video clips; Step 33: determining the operation height change information of the operator according to the operation trajectory, determining the operation height corresponding to each operation action based on the operation height change information, respectively retrieving a plurality of operation standards corresponding to each operation height, and establishing corresponding identification conditions according to the operation standards and operation objects; Step 34: Perform deep training on each of the identification conditions to obtain a number of non-standard actions corresponding to each of the working heights, perform cluster analysis on a number of working actions corresponding to the same working height to obtain a number of working classes at the same height, use the non-standard actions corresponding to the working heights to perform hazard identification on each of the working classes at the same height, obtain hazard identification results corresponding to each of the operators, and display them separately.

6. The method for identifying and monitoring violations of aerial work based on deep learning as claimed in claim 1, characterized in that: The step 4 comprises: Step 41: determining a number of illegal actions corresponding to each operator according to the hazard identification result, respectively determining the working height corresponding to each illegal action, matching a corresponding hazard level for the illegal action, and retrieving a corresponding first warning word according to the hazard level; Step 42: respectively obtaining the compliance action corresponding to each of the violation actions, determining the violation level corresponding to the violation action according to the difference between the violation action and the compliance action, and retrieving the corresponding second warning word according to the violation level; Step 43: semantically reorganizing the first warning word and the second warning word to generate a danger warning word for each operator; Step 44: Transmit the danger warning to the information audio-visual display device of the corresponding operator for danger warning.

7. The method for identifying and monitoring high-altitude work violations based on deep learning as claimed in claim 6, characterized in that: Also includes: When each of the operators does not commit any illegal actions, the corresponding information audio-visual display device is controlled to play the corresponding audio according to the operator's instructions.

8. The method for identifying and monitoring high-altitude work violations based on deep learning as claimed in claim 1, characterized in that: Also includes: Counting a number of illegal actions corresponding to each operator and analyzing the violation rate of each operator; The operator evaluation list is constructed and displayed in descending order of violation rate.

9. A high-altitude work violation behavior recognition and monitoring system based on deep learning, characterized by: include: A tracking and acquisition module is used to configure a signal tracking device for each operator, and use the signal tracking device to track the operator to obtain the operation trajectory corresponding to each operator; A video acquisition module, configured to configure a video capture device for each operator, and use the video capture device to capture an operation video stream corresponding to each operator; A hazard identification module, used to establish a plurality of operation actions corresponding to each of the operators based on the operation trajectory and the operation video stream, and to perform in-depth hazard identification on each of the operation actions based on the operation height corresponding to each of the operation actions; The danger warning module is used to determine the illegal action corresponding to each operator according to the danger identification result, retrieve the danger warning words corresponding to the illegal action and transmit them to the information audio-visual display device of the corresponding operator for danger warning.

10. The high-altitude work violation behavior recognition and monitoring system based on deep learning as claimed in claim 9, characterized in that: The tracking and acquisition module includes: The first tracking unit is used to respectively configure a signal tracking device for each operator, respectively send a test signal to each signal tracking device, obtain a feedback signal corresponding to each signal tracking device, obtain a plurality of feedback data contained in each feedback signal, and determine a tracking delay of the corresponding signal tracking device according to a data volume corresponding to each feedback data; The second tracking unit is used to track the real-time position of each of the operators during the operation of the operators, obtain the tracking position information corresponding to each of the operators, and perform time correction on the tracking position information using the corresponding tracking delay to obtain the position information corresponding to each of the operators at different times; A third tracking unit is used to determine the stay time of each operator at different working positions according to the position information corresponding to each operator at different times, and determine the total working time corresponding to each operator, establish a working timeline of the corresponding operator according to the total working time, mark the corresponding stay time in the working timeline, and determine a number of working positions according to the marked points; The fourth tracking unit is used to draw the three-dimensional working range of the corresponding operator according to the corresponding position information of each operator at different times, divide the three-dimensional working range into a number of working areas according to the working timeline, map each working position into the three-dimensional working range, count the number of mapped working positions contained in each working area, connect the mapped working positions according to the working start time of each mapped working position in the working timeline, and generate the working trajectory corresponding to each operator.

Images