Aerial work personnel behavior detection method

Abstract

The invention discloses a method for detecting behaviors of high-altitude operation personnel. The method comprises the following steps: sampling based on a real-time video of the high-altitude operation personnel on a target communication tower to obtain a sampling picture sequence; determining a current behavior sequence of the high-altitude operation personnel based on the high-altitude operation area of the target communication tower and the position of the high-altitude operation personnel in each sampling picture of the sampling picture sequence, wherein the current behavior sequence represents the operation behavior change of the high-altitude operation personnel when the high-altitude operation personnel operates in the high-altitude operation area at present; based on the current behavior sequence, acquiring a behavior characteristic sequence of the aerial operation personnel, wherein the behavior characteristic sequence is used for representing mathematical statistics of the operation behavior change; and inputting the behavior feature sequence into a high-altitude operation anomaly detection model to determine whether the operation behavior of the high-altitude operation personnel is abnormal, and the high-altitude operation anomaly detection model is obtained by performing unsupervised anomaly detection training based on historical behavior feature sequences of a plurality of high-altitude operation personnel in the high-altitude operation area of the communication tower.

Description

Technical Field

[0001] The present application relates to the technical field of high-altitude work safety, and in particular to a method for detecting the behavior of personnel working at heights. Background Art

[0002] We are currently in a period of rapid development of communication infrastructure construction, which has accelerated the high-quality development of the social economy. The construction and maintenance of communication towers are also particularly important. There are a lot of safety issues such as high-altitude operations in the construction and maintenance of communication towers. Safety accidents occur frequently due to insufficient safety awareness of construction personnel, non-standard construction procedures of construction units, and insufficient on-site supervision. Effectively conducting standardized inspections of construction site operations is an important guiding method for avoiding safety accidents and controlling subsequent construction sites.

[0003] Currently, safety inspections on high-altitude operations on communication towers are mainly carried out manually, but there are problems such as the inability of inspectors to fully monitor the site in real time and the inability to archive records of manual inspections.

[0004] With the rapid development of video surveillance and artificial intelligence technology, intelligent detection methods based on video surveillance have been proposed for the safety detection of high-altitude operations on communication towers. Video surveillance mainly uses machine vision algorithms to conduct real-time detection of personnel in high-altitude work areas and the use of safety equipment (such as safety belts, helmets, safety ropes, and safety nets). However, there is no intelligent detection method for the behavioral norms of high-altitude workers on communication towers, and it is impossible to timely discover the potential safety risks of high-altitude workers and avoid safety accidents.

[0005] How to improve the safety of the behavior of workers working at heights is a technical problem that needs to be solved at present. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a method for detecting the behavior of personnel working at heights, so as to solve the problem of low safety of the working behavior of personnel working at heights on communication towers.

[0007] In order to solve the above technical problems, this specification is implemented as follows: In a first aspect, a method for detecting the behavior of a person working at height is provided, comprising: Sampling is performed based on the real-time video of the high-altitude workers at the target communication tower to obtain a sampling picture sequence; Determine a current behavior sequence of the high-altitude worker based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude worker in each sampling picture of the sampling picture sequence, wherein the current behavior sequence represents the working behavior change of the high-altitude worker when currently working in the high-altitude working area; Acquire a behavior feature sequence of the aerial work personnel based on the current behavior sequence, wherein the behavior feature sequence is used to characterize mathematical statistics of changes in the work behavior; The behavior feature sequence is input into a high-altitude work anomaly detection model to determine whether the work behavior of the high-altitude worker is abnormal. The high-altitude work anomaly detection model is obtained by unsupervised anomaly detection training based on historical behavior feature sequences of multiple high-altitude workers in the high-altitude work area of ​​the communication tower.

[0008] Optionally, the determining of the current behavior sequence of the high-altitude workers based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude workers in each sampled picture of the sampled picture sequence includes: Based on the position of the high-altitude worker in each sampling picture of the sampling picture sequence, determining the position change amount of the high-altitude worker at the time corresponding to each sampling picture in the sampling picture sequence relative to the time corresponding to the previous sampling picture; Based on the detection frame occupied by the high-altitude working personnel in each sampling picture of the sampling picture sequence and the high-altitude working area, determining the overlap degree between the high-altitude working personnel and the high-altitude working area at the time corresponding to each sampling picture; The current behavior sequence is determined based on a sequence consisting of position changes and overlaps corresponding to each sampled picture in the sampled picture sequence.

[0009] Optionally, the position change includes at least one of a horizontal movement, a vertical movement, and a linear movement.

[0010] Optionally, the horizontal movement amount, the vertical movement amount, and the linear movement amount are determined based on the following formulas respectively:

[0011]

[0012]

[0013] in, Indicates the corresponding time The horizontal and vertical coordinates of the target point on the detection frame occupied by the high-altitude worker in the sampling picture, Indicates the corresponding time The moment before The horizontal coordinate and vertical coordinate of the target point on the detection frame occupied by the aerial worker in the sampling picture, Indicates that the target point is at time The horizontal movement of Indicates that the target point is at time The vertical movement of Indicates that the target point is at time The amount of linear movement.

[0014] Optionally, the overlap between the aerial work personnel and the aerial work area is calculated by the following formula:

[0015] in, Indicates the corresponding time The area of ​​the detection box occupied by the high-altitude workers in the sampled image, Indicates the corresponding time The area of ​​the high-altitude working area described in the sampling picture, Indicates time The degree of overlap between the high-altitude working personnel and the corresponding high-altitude working area.

[0016] Optionally, before determining the current behavior sequence of the high-altitude workers based on the high-altitude working area of ​​the target communication tower and the positions of the high-altitude workers in each sampled picture of the sampled picture sequence, the method further includes: Detecting whether the aerial worker in each sampling picture of the sampling picture sequence is in an off-the-ground state or a non-off-the-ground state; The sample pictures in which the aerial workers are not off the ground are filtered out.

[0017] Optionally, before determining the current behavior sequence of the high-altitude workers based on the high-altitude working area of ​​the target communication tower and the positions of the high-altitude workers in each sampled picture of the sampled picture sequence, the method further includes: Based on a preset target segmentation algorithm, identifying the target communication tower in each sampled picture of the sampled picture sequence; Determine the outline of the target communication tower based on the grayscale image corresponding to the color of the target communication tower in each sampled image and the background color other than the target communication tower; Based on the outline of the target communication tower, the high-altitude operation area of ​​the target communication tower in each sampling picture is determined.

[0018] Optionally, the behavior feature sequence includes a maximum frequency, and the step of obtaining the behavior feature sequence of the aerial worker based on the current behavior sequence includes: Calculate the maximum value of the position change and the maximum value of the overlap corresponding to each moment in the current behavior sequence; Determine a first number of position changes exceeding a maximum value of the position changes and a second number of overlaps exceeding a maximum value of the overlap in the current behavior sequence; Based on the first number and the second number, the maximum value frequency of the position change amount and the maximum value frequency of the overlap degree are determined in a one-to-one correspondence.

[0019] Optionally, the behavior feature sequence further includes an absolute total amount of change, and the step of obtaining the behavior feature sequence of the aerial worker based on the current behavior sequence further includes: Respectively determining a first difference between position changes corresponding to adjacent moments in the current behavior sequence and a second difference between coincidence degrees; Based on the sum of the first differences and the sum of the second differences, the total absolute change of the position change and the total absolute change of the overlap are determined one by one.

[0020] Optionally, before inputting the behavior feature sequence into the high-altitude work abnormality detection model, the method further includes: Based on the real-time video of multiple high-altitude workers at the communication tower, sampling is performed to obtain a sampling picture sequence corresponding to each high-altitude worker; Determine the current behavior sequence of each high-altitude worker based on the high-altitude work area of ​​the communication tower and the position of each high-altitude worker in each sampling picture of the sampling picture sequence; Obtaining a behavior feature sequence of each high-altitude worker based on the current behavior sequence of each high-altitude worker; Unsupervised anomaly detection training is performed based on samples composed of each behavioral feature in the behavioral feature sequence of each high-altitude worker to obtain the high-altitude work anomaly detection model.

[0021] In an embodiment of the present application, sampling is performed based on real-time video of high-altitude workers in a target communication tower to obtain a sampling picture sequence; a current behavior sequence of the high-altitude workers is determined based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude workers in each sampling picture of the sampling picture sequence, and the current behavior sequence characterizes the changes in the working behavior of the high-altitude workers when they are currently working in the high-altitude working area; a behavior feature sequence of the high-altitude workers is obtained based on the current behavior sequence, and the behavior feature sequence is used to characterize the mathematical statistics of the changes in the working behavior; the behavior feature sequence is input into a high-altitude work anomaly detection model to determine whether the working behavior of the high-altitude workers is abnormal, and the high-altitude work anomaly detection model is obtained by unsupervised anomaly detection training based on historical behavior feature sequences of multiple high-altitude workers in the high-altitude working area of ​​the communication tower, thereby intelligently detecting the behavioral norms of high-altitude workers in the communication tower, timely discovering potential safety risks in the behavior of high-altitude workers, avoiding safety accidents, and significantly improving the safety of the behavior of high-altitude workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a flow chart of the method for detecting the behavior of personnel working at heights according to an embodiment of the present application.

[0023] Figure 2 It is a schematic diagram of the specific application process of the method for detecting the behavior of high-altitude workers in an embodiment of the present application.

[0024] Figure 3 It is a structural block diagram of the high-altitude work personnel behavior detection device of an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. The numbering of the drawings in this application is only used to distinguish the various steps in the scheme, and is not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0026] In order to solve the problems existing in the prior art, the present application embodiment provides a method for detecting the behavior of high-altitude workers, such as Figure 1 As shown, the following steps 102 to 108 are included.

[0027] Step 102, sampling is performed based on the real-time video of the aerial workers at the target communication tower to obtain a sampled picture sequence.

[0028] At the construction and maintenance site of the target communication tower, cameras are deployed at fixed distances and angles to capture real-time video of high-altitude workers performing high-altitude operations on the target communication tower. The camera is facing the communication tower, and the captured video needs to cover the entire high-altitude workers and the target communication tower. The captured real-time video is sampled at a preset sampling time, for example, the video stream is sampled at a frequency of 2 frames per second, thereby extracting a sampled picture sequence including multiple picture frames of the currently captured video.

[0029] Step 104, determining the current behavior sequence of the high-altitude workers based on the high-altitude working area of ​​the target communication tower and the positions of the high-altitude workers in each sampling picture of the sampling picture sequence, wherein the current behavior sequence represents the changes in the working behavior of the high-altitude workers when they are currently working in the high-altitude working area.

[0030] When high-altitude workers are working in the high-altitude work area of ​​the target communication tower, their behaviors may change at different times, such as moving in or leaving the high-altitude work area. Each sampled picture in the sampled picture sequence will show various behaviors of high-altitude workers. By comparing two different sampled pictures, the specific behavioral differences of high-altitude workers in the two sampled pictures can be reflected.

[0031] Optionally, the determining of the current behavior sequence of the high-altitude working personnel based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude working personnel in each sampling picture of the sampling picture sequence includes: determining the position change of the high-altitude working personnel at the moment corresponding to each sampling picture in the sampling picture sequence relative to the moment corresponding to the previous sampling picture based on the position of the high-altitude working personnel in each sampling picture of the sampling picture sequence; determining the overlap between the high-altitude working personnel and the high-altitude working area at the moment corresponding to each sampling picture based on the detection box occupied by the high-altitude working personnel in each sampling picture of the sampling picture sequence and the high-altitude working area; determining the current behavior sequence based on a sequence consisting of the position change and the overlap corresponding to each sampling picture in the sampling picture sequence.

[0032] In the above embodiment, the current behavior sequence of the aerial workers includes the position change and the overlap between the aerial workers and the aerial work area. Each sampling picture contains aerial workers, and the position of the aerial workers can use the target point in the detection frame occupied by the aerial workers in the sampling picture as the position point of the aerial workers. The target point can be the center of the detection frame occupied by the aerial workers or the center point of the bottom frame of the detection frame, or the target point can be the four corner position points of the detection frame occupied by the aerial workers. The detection frame is a frame used to detect the area occupied by the aerial workers in the preset detection algorithm, and the detection frame can include the entire outline of the aerial workers.

[0033] In two adjacent sampling pictures at the sampling time, the position change of the high-altitude workers in the sampling picture at the later time compared with the sampling picture at the previous time can be determined. For example, if the currently collected sampling picture sequence includes 10 sampling pictures, the positions of the high-altitude workers are compared in adjacent sampling pictures, and 9 position changes can be obtained. The position change of the sampling picture at the initial sampling time can be recorded as 0, so that the 10 position changes constitute the current position change sequence.

[0034] In each sampling picture, the overlap between the high-altitude workers and the high-altitude working area can also be determined, that is, the intersection of the detection frame occupied by the high-altitude workers in the sampling picture and the high-altitude working area in the sampling picture. For example, whether the detection frame occupied by the high-altitude workers falls completely in the high-altitude working area, partially in the high-altitude working area, or not in the high-altitude working area at all. If it falls partially in the high-altitude working area, there is a potential safety risk compared to falling completely in the high-altitude working area. The smaller the high-altitude working area, the higher the safety risk of the current high-altitude workers. Still taking the current collected sampling picture sequence including 10 sampling pictures as an example, 10 overlaps can be obtained, and these 10 overlaps constitute the current overlap sequence.

[0035] The sequence including 10 position changes and the sequence including 10 coincidence degrees are arranged in rows respectively, and the current behavior sequence of the aerial workers is obtained.

[0036] The position change of the aerial work personnel when performing work in the aerial work area of ​​the target communication tower may be along the horizontal direction, the vertical direction, or the horizontal and vertical directions.

[0037] Optionally, the position change includes at least one of a horizontal movement, a vertical movement, and a linear movement.

[0038] Specifically, the horizontal movement amount, vertical movement amount, and linear movement amount are determined based on the following formulas:

[0039]

[0040]

[0041] in, Indicates the corresponding time The horizontal and vertical coordinates of the target point on the detection frame occupied by the high-altitude worker in the sampling picture, Indicates the corresponding time The moment before The horizontal coordinate and vertical coordinate of the target point on the detection frame occupied by the aerial worker in the sampling picture, Indicates that the target point is at time The horizontal movement of Indicates that the target point is at time The vertical movement of Indicates that the target point is at time The amount of linear movement.

[0042] The horizontal movement is determined based on the change in the horizontal coordinate position of the high-altitude workers in the two adjacent sampling pictures, and the vertical movement is determined based on the change in the vertical coordinate position of the high-altitude workers in the two adjacent sampling pictures. The linear movement is determined based on the horizontal and vertical movement corresponding to the two adjacent sampling pictures.

[0043] For example, the current time is The initial sampling time is , then the position change of the aerial worker in the current behavior sequence is expressed as follows:

[0044]

[0045]

[0046] in, Represents the horizontal movement sequence in the current behavior sequence of the aerial worker. Indicates the current time The horizontal movement of Indicates the current time The moment before The horizontal movement of Indicates the horizontal movement at the initial sampling time. Represents the vertical movement sequence in the current behavior sequence of the aerial worker. Represents the linear movement sequence in the current behavior sequence of the aerial worker. For the definition of other parameters, please refer to The relevant parameters are not described here.

[0047] Still taking the above-mentioned currently collected sampling picture sequence including 10 sampling pictures as an example, the 10 horizontal movement amounts, 10 vertical movement amounts, and 10 linear movement amounts corresponding to these sampling pictures can be calculated respectively. The sequence including 10 horizontal movement amounts, the sequence including 10 vertical movement amounts, and the sequence including 10 linear movement amounts are arranged in rows respectively, and the position change amount sequence in the current behavior sequence of the aerial worker is obtained.

[0048] Optionally, the overlap between the aerial work personnel and the aerial work area is calculated by the following formula:

[0049] in, Indicates the corresponding time The area of ​​the detection box occupied by the high-altitude workers in the sampled image, Indicates the corresponding time The area of ​​the high-altitude working area described in the sampling picture, Indicates time The degree of overlap between the high-altitude working personnel and the corresponding high-altitude working area.

[0050] In this embodiment, the corresponding degree of overlap is determined by calculating the ratio of the intersection of the area of ​​the detection frame occupied by the high-altitude workers in the sampled image and the area of ​​the high-altitude working area to the area of ​​the detection frame occupied by the high-altitude workers. In other embodiments, the corresponding degree of overlap may also be determined by calculating the ratio of the intersection of the area of ​​the detection frame occupied by the high-altitude workers in the sampled image and the area of ​​the high-altitude working area to the area of ​​the high-altitude working area.

[0051] For example, the current time is The initial sampling time is , then the overlap degree in the current behavior sequence of the aerial workers is expressed as follows:

[0052] in, Represents the overlap sequence in the current behavior sequence of the aerial workers. Indicates the current time The overlap of Indicates the current time The moment before The overlap of Indicates the degree of coincidence at the initial sampling moment.

[0053] In order to improve the accuracy and speed of high-altitude worker behavior detection, optionally, before determining the current behavior sequence of the high-altitude worker based on the high-altitude work area of ​​the target communication tower and the position of the high-altitude worker in each sampling picture of the sampling picture sequence, it also includes: detecting whether the high-altitude worker in each sampling picture of the sampling picture sequence is in an off-the-ground state or a non-off-the-ground state; and filtering out sampling pictures in which the high-altitude worker is in a non-off-the-ground state.

[0054] To detect whether a person working at high altitude is in an off-the-ground state or in a non-off-the-ground state, a target detection model can be trained by using, for example, a target detection algorithm of the Yolo series, in which the people in the sample are labeled with two labels: off-the-ground state and on-the-ground state, and multiple samples with labeled labels are used to train the target detection model.

[0055] Input each sampled picture in the sampled picture sequence into the trained target detection model, and then according to the output result of off-the-ground or on-the-ground, it can be detected whether the aerial workers in each sampled picture are off-the-ground or not. The aerial workers are not off-the-ground, which means that the personnel in the corresponding sampled picture have not yet been working in the aerial work area of ​​the target communication tower, and are not the objects of the aerial workers behavior detection. Therefore, the sampled pictures detected as aerial workers are not off-the-ground are filtered out. Combined with the sampled pictures that really belong to aerial work, the current behavior sequence of the aerial workers is determined, which can improve the accuracy and speed of the aerial workers behavior detection.

[0056] In order to improve the accuracy of behavior detection of high-altitude workers, optionally, before determining the current behavior sequence of high-altitude workers based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude workers in each sampling picture of the sampling picture sequence, it also includes: based on a preset target segmentation algorithm, identifying the target communication tower in each sampling picture of the sampling picture sequence; determining the outline of the target communication tower based on the color of the target communication tower in each sampling picture and the grayscale image corresponding to the background color except the target communication tower; determining the high-altitude working area of ​​the target communication tower in each sampling picture based on the outline of the target communication tower.

[0057] The high-altitude work area in each sampled image can be determined by the outline of the target communication tower. The outline of the target communication tower can be determined by using, for example, the Yolo series target segmentation algorithm to train a communication tower area segmentation model, wherein the communication tower area in the sample is labeled with two labels: with communication tower and without communication tower. Multiple labeled samples are used for training the communication tower area segmentation model.

[0058] Input each sampled picture in the sampled picture sequence into the trained communication tower area segmentation model, segment the target communication tower in each sampled picture, and output the segmented target communication tower area and the background area of ​​the non-target communication tower in each sampled picture. Fix the color of the target communication tower area to a color, such as 0, and fix the background color except the target communication tower to a color, such as 1, to obtain the grayscale image corresponding to each sampled picture. Calculate the contour of the target communication tower area based on the grayscale image to obtain the high-altitude operation area in each sampled picture. The high-altitude operation area can be represented by the coordinate position set of multiple points corresponding to the contour.

[0059] In one embodiment, for the current behavior sequence of the high-altitude workers determined above, according to the characteristics of high-altitude work on communication towers, that is, staying in a certain area for a long time for construction, the 80th percentiles of various behavior sequences in the current behavior sequence (sequences with the category of position change and sequences with the category of overlap) are counted respectively, and the 80th percentile is used as a threshold to filter each type of behavior sequence in the current behavior sequence. After filtering, only the values ​​that exceed the corresponding threshold are retained for each type of behavior sequence.

[0060] For example, when the position change includes horizontal movement, vertical movement, and linear movement, the 80th percentile of the horizontal movement, the 80th percentile of the vertical movement, the 80th percentile of the linear movement, and the 80th percentile of the overlap in the current behavior sequence are recorded as: The above parameters are used as the thresholds of the corresponding category behavior sequences, and the horizontal movement, vertical movement, linear movement, and overlap in each category of sequences that are lower than the corresponding thresholds are filtered out. The filtered current behavior sequence can more accurately reflect the current work behavior of the high-altitude workers, which is conducive to improving the accuracy of subsequent behavior feature extraction and behavior detection.

[0061] Step 106: acquiring a behavior feature sequence of the aerial work personnel based on the current behavior sequence, wherein the behavior feature sequence is used to characterize mathematical statistics of changes in the work behavior.

[0062] In the embodiment of the present application, the statistical characteristics of the behavior sequence of the high-altitude workers are used to represent the work behavior characteristics of the high-altitude workers.

[0063] In one embodiment, the behavior feature sequence includes a maximum frequency, and obtaining the behavior feature sequence of the height worker based on the current behavior sequence includes: calculating the maximum value of the position change and the maximum value of the overlap corresponding to each moment in the current behavior sequence; determining a first number of position changes in the current behavior sequence that exceed the maximum value of the position change, and a second number of overlaps that exceed the maximum value of the overlap; based on the first number and the second number, determining the maximum frequency of the position change and the maximum frequency of the overlap one by one.

[0064] The maximum value is a local concept, which means that the function value of the point is the largest in a small area. The embodiment of the present application uses the maximum value as a threshold to determine whether the operation behavior of the high-altitude operator in the high-altitude operation area is likely to be risky.

[0065] When the position change includes horizontal movement, vertical movement, and linear movement, the maximum value of the position change includes the maximum value of horizontal movement, the maximum value of vertical movement, and the maximum value of linear movement. Then, the maximum value of horizontal movement, the maximum value of vertical movement, the maximum value of linear movement, and the maximum value of overlap corresponding to each moment in the current behavior sequence are calculated accordingly.

[0066] Specifically, the maximum values ​​of the horizontal movement, vertical movement, linear movement and fit degree corresponding to each moment can be calculated based on the quartile statistics, and the upper quartile value and interquartile range of the horizontal movement, vertical movement, linear movement and coincidence degree corresponding to each moment in the behavior sequence can be calculated respectively. Based on the corresponding upper quartile value and interquartile range, the maximum value of the horizontal movement, the maximum value of the vertical movement, the maximum value of the linear movement and the maximum value of the coincidence degree corresponding to each moment in the behavior sequence are determined.

[0067] The maximum value calculation formula is as follows: Maximum horizontal shift:

[0068] Maximum vertical displacement:

[0069] Maximum value of linear movement:

[0070] Maximum fit:

[0071] in, represents the upper quartile value of the horizontal shift, Represents the upper quartile value of the vertical movement, Indicates the upper quartile value of the linear movement, represents the upper quartile value of overlap, The interquartile range represents the horizontal shift. represents the interquartile range of vertical movement, The interquartile range represents the amount of linear movement. The interquartile range represents the degree of coincidence. The interquartile range is the difference between the upper quartile and the lower quartile, for example , Indicates the lower quartile value of the horizontal shift.

[0072] After calculating each maximum value, the maximum value is used as a threshold to compare the corresponding values ​​of each moment included in the current behavior sequence of the aerial worker. For example, the horizontal movement amount at each moment in the current behavior sequence is compared with the maximum value of the horizontal movement amount, and the number of the horizontal movement amount exceeding the maximum value of the horizontal movement amount is determined; similarly, the number of the vertical movement amount exceeding the maximum value of the vertical movement amount, the number of the linear movement amount exceeding the maximum value of the linear movement amount, and the number of the overlap exceeding the overlap maximum value at each moment in the current behavior sequence can be determined.

[0073] Based on the corresponding quantity, the maximum frequency of the horizontal movement amount, the maximum frequency of the vertical movement amount, the maximum frequency of the linear movement amount and the maximum frequency of the overlap degree are determined respectively.

[0074] At the current moment Taking 10 sample images as an example, the current behavior sequence includes a total of 10 behavior sequences at different moments, as follows:

[0075]

[0076]

[0077]

[0078] For the current moment The behavior feature sequence of high-altitude workers is obtained through the maximum frequency, and the maximum frequency calculation formula is as follows:

[0079] in, represents the maximum frequency, Indicates that except the current time The corresponding values ​​in the sequence of the other nine moments can be the horizontal movement, vertical movement, linear movement or overlap, Indicates the maximum value corresponding to the horizontal movement amount, vertical movement amount, linear movement amount or overlap. For example, in a sequence of horizontal movement amounts corresponding to 10 moments, if there are 6 moments of horizontal movement amounts exceeding the maximum value of the horizontal movement amount, then the maximum frequency of the horizontal movement amount is 6. Similarly, the maximum frequency of the vertical movement amount, the maximum frequency of the linear movement amount and the maximum frequency of the overlap degree can be calculated.

[0080] Optionally, the behavior feature sequence also includes a total absolute change amount, and obtaining the behavior feature sequence of the high-altitude workers based on the current behavior sequence also includes: respectively determining a first difference between the position change amounts corresponding to adjacent moments in the current behavior sequence and a second difference between the overlaps; based on the sum of the first differences and the sum of the second differences, determining the total absolute change amount of the position change amount and the total absolute change amount of the overlap one by one.

[0081] When the position change includes horizontal movement, vertical movement, and linear movement, the difference between the horizontal movement corresponding to each adjacent moment in the current behavior sequence, the difference between the vertical movement corresponding to each adjacent moment, and the difference between the linear movement corresponding to each adjacent moment in the current behavior sequence are further counted.

[0082] Specifically, the maximum values ​​of the horizontal movement, vertical movement, linear movement and fit degree corresponding to each moment can be calculated based on the quartile statistics, and the upper quartile value and interquartile range of the horizontal movement, vertical movement, linear movement and coincidence degree corresponding to each moment in the behavior sequence can be calculated respectively. Based on the corresponding upper quartile value and interquartile range, the maximum value of the horizontal movement, the maximum value of the vertical movement, the maximum value of the linear movement and the maximum value of the coincidence degree corresponding to each moment in the behavior sequence are determined.

[0083] Continue with the current moment above Taking 10 sampled images and a current behavior sequence including 10 behavior sequences at a total of 10 moments as an example, the calculation formula for the total absolute change is as follows:

[0084] in, represents the total absolute change, Indicates that except the current time The corresponding values ​​in the sequence of the other nine moments can be the horizontal movement, vertical movement, linear movement or overlap; Indicates the time in the sequence The previous moment The corresponding value of .

[0085] In addition, the characteristics of the behavior feature sequence may also include the linear change trend, approximate entropy, Fourier transform coefficient, and whether the standard deviation is times the range ( )wait.

[0086] The formula for calculating whether the standard deviation is multiple of the range is as follows:

[0087] in, Indicates the coefficient, you can choose 0.5, 1.0, 1.5, It represents the standard deviation of the corresponding values ​​of the current behavior sequence of the aerial workers at each moment. Indicates the maximum value of the corresponding value of the current behavior sequence of the aerial workers at each moment, Indicates the minimum value of the current behavior sequence of the aerial workers at each moment. Indicates extremely poor.

[0088] Taking the horizontal movement in the current behavior sequence as an example, the standard deviation and range of the horizontal movement at each moment of the current behavior sequence are calculated respectively, so as to determine whether the standard deviation of the horizontal movement at each moment of the current behavior sequence is times the range according to the above formula. Similarly, it can be determined whether the standard deviation of the vertical movement at each moment of the current behavior sequence is times the range, whether the standard deviation of the linear movement is times the range, and whether the standard deviation of the overlap is times the range.

[0089] Thus, through the above different mathematical statistics methods, the behavior feature sequence of the high-altitude workers can be obtained from the current behavior sequence. In one embodiment, the behavior feature sequence of the high-altitude workers at least includes the maximum value frequency.

[0090] Step 108, input the behavior feature sequence into the high-altitude work anomaly detection model to determine whether the work behavior of the high-altitude worker is abnormal. The high-altitude work anomaly detection model is obtained by unsupervised anomaly detection training based on the historical behavior feature sequences of multiple high-altitude workers in the high-altitude work area of ​​the communication tower.

[0091] Through the above steps 102 to 106, the behavioral feature sequence of the aerial workers when they are working in the aerial work area of ​​the target communication tower can be obtained in real time. By inputting the currently obtained behavioral feature sequence into the pre-trained aerial work anomaly detection model, it is possible to detect whether the current working behavior of the aerial workers is normal or abnormal. When the detection result is abnormal, it is determined that the current aerial workers are not behaving in a standardized manner and there is a work safety risk. Furthermore, the behavior records with real-time detection results that are not standardized can be stored in the database, and the pictures of evidence of irregular behavior can be stored in the file system. It is also possible to notify relevant personnel of the irregular behavior of aerial workers in real time through text messages or voice calls, so as to provide timely reminders and corrections.

[0092] Optionally, before inputting the behavior feature sequence into the high-altitude work anomaly detection model, it also includes: sampling based on real-time videos of multiple high-altitude workers at a communication tower to obtain a sampling picture sequence corresponding to each high-altitude worker; determining the current behavior sequence of each high-altitude worker based on the high-altitude work area of ​​the communication tower and the position of each high-altitude worker in each sampling picture of the sampling picture sequence; obtaining the behavior feature sequence of each high-altitude worker based on the current behavior sequence of each high-altitude worker; and performing unsupervised anomaly detection training based on samples consisting of each behavior feature in the behavior feature sequence of each high-altitude worker to obtain the high-altitude work anomaly detection model.

[0093] For videos of different high-altitude workers performing high-altitude operations on communication towers in different scenarios, we sampled image data and extracted the behavioral characteristics of the high-altitude workers at each historical time point. We used an unsupervised anomaly detection algorithm such as the isolation forest algorithm to train the historical samples, thereby obtaining a high-altitude operation anomaly detection model.

[0094] Figure 2 : is a schematic diagram of a specific application process of the method for detecting the behavior of high-altitude workers in an embodiment of the present application, such as Figure 2 The following steps are shown: S1: Install video surveillance equipment such as cameras at the high-altitude operation site of the communication tower at a specific distance and angle, so that the camera of the video surveillance equipment faces the communication tower; S2: Sample the video stream in real time at a frequency of, for example, 2 frames per second, and use the target detection model and the communication tower area segmentation model to predict the sampled images, filter out the sampled images that are not off the ground, and identify the high-altitude operation area of ​​the communication tower in the sampled images; S3: Based on the behavior sequence calculation method of the high-altitude workers, the horizontal movement amount, vertical movement amount, linear movement amount, and overlap degree between the high-altitude workers and the high-altitude working area of ​​the high-altitude workers are calculated to obtain the current behavior sequence of the high-altitude workers; S4: Based on the behavioral characteristics calculation method of high-altitude workers, the behavioral characteristics of high-altitude workers are calculated in real time; S5: Based on the method for determining the standardization of aerial work behavior, the calculated behavior features are input into the aerial work abnormality detection model to determine whether the behavior of the aerial work personnel is standard or abnormal; S6: record and store the judgment results, and issue a warning for any irregular operation behaviors detected; S7: After the construction is completed, the on-site video surveillance equipment will be recovered.

[0095] In an embodiment of the present application, sampling is performed based on real-time video of high-altitude workers in a target communication tower to obtain a sampling picture sequence; a current behavior sequence of the high-altitude workers is determined based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude workers in each sampling picture of the sampling picture sequence, and the current behavior sequence characterizes the changes in the working behavior of the high-altitude workers when they are currently working in the high-altitude working area; a behavior feature sequence of the high-altitude workers is obtained based on the current behavior sequence, and the behavior feature sequence is used to characterize the mathematical statistics of the changes in the working behavior; the behavior feature sequence is input into a high-altitude work anomaly detection model to determine whether the working behavior of the high-altitude workers is abnormal, and the high-altitude work anomaly detection model is obtained by unsupervised anomaly detection training based on historical behavior feature sequences of multiple high-altitude workers in the high-altitude working area of ​​the communication tower, thereby intelligently detecting the behavioral norms of high-altitude workers in the communication tower, timely discovering potential safety risks in the behavior of high-altitude workers, avoiding safety accidents, and significantly improving the safety of the behavior of high-altitude workers.

[0096] Alternatively, if Figure 3 As shown, an embodiment of the present application also provides a device 2000 for detecting the behavior of personnel working at heights, including a processor 2400 and a memory 2200. The memory 2200 stores programs or instructions that can be run on the processor 2400. When the program or instructions are executed by the processor 2400, the various steps of the above-mentioned embodiment of the method for detecting the behavior of personnel working at heights are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0097] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of any of the above-mentioned embodiments of the method for detecting the behavior of high-altitude workers is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the readable storage medium includes a computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0098] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program can be operated to enable a computer to implement each process of any of the above-mentioned embodiments of the method for detecting the behavior of high-altitude workers when executed, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0099] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0101] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for detecting the behavior of personnel working at heights, characterized in that: include: Sampling is performed based on the real-time video of the high-altitude workers at the target communication tower to obtain a sampling picture sequence; Determine a current behavior sequence of the high-altitude worker based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude worker in each sampling picture of the sampling picture sequence, wherein the current behavior sequence represents the working behavior change of the high-altitude worker when currently working in the high-altitude working area; Acquire a behavior feature sequence of the aerial work personnel based on the current behavior sequence, wherein the behavior feature sequence is used to characterize mathematical statistics of changes in the work behavior; The behavior feature sequence is input into a high-altitude work anomaly detection model to determine whether the work behavior of the high-altitude worker is abnormal. The high-altitude work anomaly detection model is obtained by unsupervised anomaly detection training based on historical behavior feature sequences of multiple high-altitude workers in the high-altitude work area of ​​the communication tower.

2. The method according to claim 1, characterized in that The determining of the current behavior sequence of the high-altitude workers based on the high-altitude working area of ​​the target communication tower and the position of the high-altitude workers in each sampling picture of the sampling picture sequence includes: Based on the position of the high-altitude worker in each sampling picture of the sampling picture sequence, determining the position change amount of the high-altitude worker at the time corresponding to each sampling picture in the sampling picture sequence relative to the time corresponding to the previous sampling picture; Based on the detection frame occupied by the high-altitude working personnel in each sampling picture of the sampling picture sequence and the high-altitude working area, determining the overlap degree between the high-altitude working personnel and the high-altitude working area at the time corresponding to each sampling picture; The current behavior sequence is determined based on a sequence consisting of position changes and overlaps corresponding to each sampled picture in the sampled picture sequence.

3. The method according to claim 2, characterized in that The position change includes at least one of a horizontal movement, a vertical movement, and a linear movement.

4. The method according to claim 3, characterized in that: The horizontal movement, vertical movement, and linear movement are determined based on the following formulas: in, Indicates the corresponding time The horizontal and vertical coordinates of the target point on the detection frame occupied by the high-altitude workers in the sampling picture, Indicates the corresponding time The moment before The horizontal coordinate and vertical coordinate of the target point on the detection frame occupied by the aerial worker in the sampling picture, Indicates that the target point is at time The horizontal movement of Indicates that the target point is at time The vertical movement of Indicates that the target point is at time The amount of linear movement.

5. The method according to claim 2, characterized in that: The overlap between the aerial work personnel and the aerial work area is calculated by the following formula: in, Indicates the corresponding time The area of ​​the detection box occupied by the high-altitude workers in the sampled image, Indicates the corresponding time The area of ​​the high-altitude working area described in the sampling picture, Indicates time The degree of overlap between the high-altitude working personnel and the corresponding high-altitude working area.

6. The method according to claim 5, characterized in that Before determining the current behavior sequence of the high-altitude workers based on the high-altitude working area of ​​the target communication tower and the positions of the high-altitude workers in each sampled picture of the sampled picture sequence, the method further includes: Detecting whether the aerial worker in each sampling picture of the sampling picture sequence is in an off-the-ground state or a non-off-the-ground state; The sample pictures in which the aerial workers are not off the ground are filtered out.

7. The method according to claim 5, characterized in that Before determining the current behavior sequence of the high-altitude workers based on the high-altitude working area of ​​the target communication tower and the positions of the high-altitude workers in each sampled picture of the sampled picture sequence, the method further includes: Based on a preset target segmentation algorithm, identifying the target communication tower in each sampled picture of the sampled picture sequence; Determine the outline of the target communication tower based on the color of the target communication tower in each sampled image and the grayscale image corresponding to the background color except the target communication tower; Based on the outline of the target communication tower, the high-altitude operation area of ​​the target communication tower in each sampling picture is determined.

8. The method according to claim 5, characterized in that The behavior feature sequence includes a maximum frequency, and the step of obtaining the behavior feature sequence of the aerial worker based on the current behavior sequence includes: Calculate the maximum value of the position change and the maximum value of the overlap corresponding to each moment in the current behavior sequence; Determine a first number of position changes exceeding a maximum value of the position changes and a second number of overlaps exceeding a maximum value of the overlap in the current behavior sequence; Based on the first number and the second number, the maximum value frequency of the position change amount and the maximum value frequency of the overlap degree are determined in a one-to-one correspondence.

9. The method according to claim 8, characterized in that The behavior feature sequence also includes the absolute change amount, and the behavior feature sequence of the high-altitude operator obtained based on the current behavior sequence also includes: Respectively determining a first difference between position changes corresponding to adjacent moments in the current behavior sequence and a second difference between coincidence degrees; Based on the sum of the first differences and the sum of the second differences, the total absolute change of the position change and the total absolute change of the overlap are determined one by one.

10. The method according to claim 1, characterized in that Before inputting the behavior feature sequence into the high-altitude operation abnormality detection model, the method further includes: Based on the real-time video of multiple high-altitude workers at the communication tower, sampling is performed to obtain a sampling picture sequence corresponding to each high-altitude worker; Determine the current behavior sequence of each high-altitude worker based on the high-altitude work area of ​​the communication tower and the position of each high-altitude worker in each sampling picture of the sampling picture sequence; Obtaining a behavior feature sequence of each high-altitude worker based on the current behavior sequence of each high-altitude worker; Unsupervised anomaly detection training is performed based on samples composed of each behavioral feature in the behavioral feature sequence of each high-altitude worker to obtain the high-altitude work anomaly detection model.

Images