Electrostatic ball touch identification method and system

Abstract

The invention discloses an electrostatic ball touch identification method and system. The method comprises the following steps: extracting human skeleton point information of a person in each frame in a video to be detected; according to the human body skeleton point information, acquiring a hand region of a person; feature extraction is carried out on the hand area of the person, a residual attention mechanism is introduced to identify and position the hand area and the electrostatic ball area, and whether the hand area and the electrostatic ball area in the video to be detected coincide or not is judged; and when the hand region coincides with the electrostatic ball region, performing comprehensive judgment of multi-frame voting decision on the hand region information, and determining whether a person in the to-be-detected video really touches the electrostatic ball or not. And the behavior that a worker touches the static ball can be effectively improved, safety production is guaranteed, and the possibility of safety accidents is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an electrostatic ball touch recognition method and system. Background Art

[0002] In a safe production environment, some scenes require workers to eliminate static electricity before entering, otherwise there may be safety hazards; if static electricity elimination is not performed, it will not only affect the normal operation of the production environment, but also pose a threat to human life.

[0003] Static elimination ball, also known as static discharge instrument, static ball. It is a human static discharge product suitable for flammable, explosive and anti-static places. It can safely discharge the static charge accumulated by the human body, avoid fire and explosion accidents caused by human static electricity, human electric shock and reduce the occurrence of static damage to electronic components. Therefore, static elimination ball is usually used to eliminate human static electricity.

[0004] Therefore, there is an urgent need for a method to determine whether the staff touches the static ball to eliminate static electricity on the human body. Summary of the invention

[0005] The purpose of the present invention is to provide an electrostatic ball touch identification method and system, which can effectively improve the behavior of workers touching the electrostatic ball, ensure safe production, and reduce the possibility of safety accidents.

[0006] To achieve the above object, the present invention provides a method for identifying electrostatic ball touch, comprising:

[0007] Extract the human skeleton point information of each person in each frame of the video to be tested;

[0008] Acquire the hand area of ​​the person according to the human skeleton point information;

[0009] The hand area of ​​the person is feature extracted, and the residual attention mechanism is introduced to identify and locate the hand area and the electrostatic ball area, and determine whether the hand area and the electrostatic ball area in the video to be tested overlap;

[0010] When the hand area overlaps with the electrostatic ball area, a comprehensive judgment of multi-frame voting decision is performed on the hand area information to determine whether the person in this section of the video to be tested actually touches the electrostatic ball.

[0011] Furthermore, human skeleton point information of each person in each frame of the video to be tested is extracted, including:

[0012] Input the video to be tested into the human body detection algorithm model to obtain the human body frame of each frame;

[0013] The human body frame is input into the skeleton point detection model to obtain the human body skeleton point information of the person.

[0014] Further, according to the human skeleton point information, the hand area of ​​the person is obtained, including:

[0015] Using the bone point information of the elbow and wrist of the person, the center point of the person's hand is obtained;

[0016] Get the person's hand area based on the center point of the person's hand.

[0017] Furthermore, the skeleton point information of the elbow and wrist of the person is used to obtain the center point of the person's hand, including:

[0018] With the person's elbow as the starting point of the vector and the wrist as the end point of the vector, the center point of the person's hand is calculated using the following formula:

[0019] P(x h ,y h )=P(x1,y1)+(P(x1,y1)-P(x2,y2))*ratio a (1)

[0020] Among them, P(x h ,y h ) represents the center point of the person's hand, P(x1, y1) represents the person's elbow, i.e. the starting point; P(x2, y2) represents the person's wrist, i.e. the end point; ratio a Indicates the ratio from wrist to elbow.

[0021] Furthermore, based on the center point of the person's hand, the person's hand area is obtained, including:

[0022] According to the center point of the person's hand, the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are calculated to obtain the person's hand area;

[0023] The vector distance from the elbow to the wrist of the person is used as the measurement standard. Assume that the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are P(x a ,y a )、P(x b ,y b ), the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are calculated by the following formula:

[0024]

[0025] Where dis represents the vector distance from the elbow to the wrist, P(x a ,y a) represents the coordinate of the upper left corner of the rectangular box of the hand area, P(x b ,y b ) represents the lower right corner coordinate of the rectangular box of the hand area, P(x h ,y h ) represents the center point of the person’s hand, ratio a Indicates the ratio from wrist to elbow.

[0026] Furthermore, a comprehensive judgment of multi-frame voting decision is performed on the hand area information to determine whether the person in the video to be tested actually touches the electrostatic ball, including:

[0027] If the hand area of ​​the person in the video to be tested touches the electrostatic ball, a multi-frame voting decision is made on the hand area information corresponding to the hands of the person in each frame of the video to determine whether the person meets the time limit for touching the electrostatic ball;

[0028] If the person meets the time requirement of touching the electrostatic ball, it means that the person in this video under test actually touches the electrostatic ball;

[0029] The following formula is used to determine whether the person meets the time limit for touching the electrostatic ball:

[0030]

[0031] In the formula, Y represents the result of judging whether the person meets the time requirement of touching the electrostatic ball. If Y is True, it means that the person meets the time requirement of touching the electrostatic ball. If Y is False, it means that the person does not meet the time requirement. t represents the time requirement of touching the electrostatic ball. f represents the number of frames within the preset time of the video. S lh Indicates whether the person's left hand is touching the electrostatic ball. S rh Indicates whether the person's right hand is touching the electrostatic ball.

[0032] Based on the same inventive concept, the present invention also provides an electrostatic ball touch recognition system, including an extraction unit, an acquisition unit, an identification unit and a determination unit.

[0033] The extraction unit is used to extract the human skeleton point information of the person in each frame of the video to be tested;

[0034] The acquisition unit is used to acquire the hand area of ​​the person according to the human skeleton point information;

[0035] The recognition unit is used to extract features of the hand area of ​​the person, and introduce a residual attention mechanism to recognize and locate the hand area and the electrostatic ball, and determine whether the hand area and the electrostatic ball area in the video to be tested overlap;

[0036] The determination unit is used to perform a comprehensive judgment of multi-frame voting decisions on the hand area information when the hand area coincides with the electrostatic ball area, so as to determine whether the person in this section of the video to be tested actually touches the electrostatic ball.

[0037] Furthermore, the extraction unit is specifically used to input the video to be tested into the human body detection algorithm model to obtain the human body frame of each frame, and input the human body frame into the skeleton point detection model to obtain the human body skeleton point information of the person;

[0038] The acquisition unit is specifically used to obtain the center point of the person's hand by using the bone point information of the person's elbow and wrist; and to obtain the person's hand area based on the center point of the person's hand.

[0039] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the aforementioned electrostatic ball touch recognition method when executing the computer program.

[0040] Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and when the computer executable instructions are executed, the aforementioned electrostatic ball touch recognition method is implemented.

[0041] Technical effects and advantages of the present invention: The present invention uses a deep learning neural network and a multi-frame voting strategy to realize a method for identifying the touch of an electrostatic ball. The method is mainly based on a deep neural network and a multi-frame voting strategy of a residual attention mechanism. The hand area of ​​the human body is extracted through pre-trained skeleton points, and the area is used for feature recognition. Finally, multi-frame voting is performed with the help of the distribution results of the features, thereby realizing a method for identifying the touch of an electrostatic ball in a video.

[0042] The present invention utilizes skeleton point detection, hand area extraction, hand feature behavior recognition, and multi-frame decision-making to construct a complete algorithm and system, which can ensure the operation of the monitoring equipment stably, continuously and efficiently; it can realize real-time detection of electrostatic ball touching behavior in any production area to ensure the necessity of personnel touching the electrostatic ball, and can be widely promoted and applied in businesses with similar needs.

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 This is a flow chart of a method for identifying electrostatic ball touch according to an embodiment of the present invention;

[0046] Figure 2 Schematic diagram of the framework structure of the behavior recognition model in an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of the structure of the residual attention mechanism in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of the structure of an electrostatic ball touch recognition system according to an embodiment of the present invention;

[0049] Figure 5 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] To address the deficiencies of the prior art, the present invention discloses a method for identifying electrostatic ball contact. Figure 1 As shown, the following steps are included:

[0052] Step S1, extracting the human skeleton point information of each person in each frame of the video to be tested;

[0053] Step S2: obtaining the hand area of ​​the person according to the human skeleton point information;

[0054] Step S3, extracting features from the hand area of ​​the person, and introducing a residual attention mechanism to identify and locate the hand area and the electrostatic ball area, and determining whether the hand area and the electrostatic ball area in the video to be tested overlap;

[0055] Step S4: When the hand area overlaps with the electrostatic ball area, a comprehensive judgment of multi-frame voting decision is performed on the hand area information to determine whether the person in this section of the video to be tested actually touches the electrostatic ball.

[0056] In some specific embodiments, step S1, extracting the human skeleton point information of a person in each frame of the video to be tested, specifically includes:

[0057] Step S101: first input the video to be tested into the human body detection algorithm model to obtain the human body frame of each frame.

[0058] Among them, the human body detection algorithm model is trained using the Y0loV5 network structure. The output end of the human body detection algorithm model (i.e., the Head structure of the YoloV5 network) is mainly composed of three different output layers, which are responsible for detecting large, medium and small scale targets respectively.

[0059] The YoloV5 network structure uses CSPDarknet53 as its backbone network. CSPDarknet53 has strong feature extraction capabilities and computational efficiency, which can improve the feature extraction of the human detection algorithm model for the video to be tested; at the same time, the FPN network is used to detect and fuse information from different feature map levels to improve the performance of the video to be tested detection; and the FocalLoss loss function is used to alleviate the problem of category imbalance in the detection process of the video to be tested, thereby improving the performance of the human detection algorithm model; through non-maximum suppression (NMS), after the YOLOv5 network structure outputs the result, it will perform NMS processing on the overlapping target boxes to obtain the final detection result (that is, the human box of the person in each frame).

[0060] Step S102: then input the human body frame into the skeleton point detection model to obtain the human body skeleton point information of the person.

[0061] Among them, HRNet is used to train the skeleton point detection model, which is downsampled by 4 times through two convolutional layers with a convolution kernel size of 3×3 and a stride of 2 (both followed by BN and ReLU), and then Bottleneck is repeatedly stacked through the Layer1 module.

[0062] In some specific embodiments, step S2, obtaining a person's hand region according to the human skeleton point information, specifically includes:

[0063] By using the bone point information of the person's elbow and wrist, the center point of the person's hand is obtained with the person's elbow as the vector starting point and the wrist as the vector end point.

[0064] The center point of the person's hand is obtained by the following calculation:

[0065] P(x h ,y h )=P(x1, y1)+(P(x1, y1)-P(x2, y2))*ratio a (1)

[0066] Among them, P(x h ,y h ) represents the center point of the person's hand, P(x1, y1) represents the person's elbow, i.e. the starting point; P(x2, y2) represents the person's wrist, i.e. the end point; ratio a Indicates the ratio from wrist to elbow.

[0067] According to the center point of the person's hand, the coordinates of the upper left corner and the lower right corner of the rectangular box of the person's hand area are calculated to obtain the person's hand area.

[0068] The vector distance from the elbow to the wrist of the person is used as the measurement standard. Assume that the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are P(x a ,y a )、P(x b ,y b ), the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are calculated by the following formula:

[0069]

[0070] Where dis represents the vector distance from the elbow to the wrist, P(x a ,y a ) represents the coordinate of the upper left corner of the rectangular box of the hand area, P(x b ,y b ) represents the lower right corner coordinate of the rectangular box of the hand area, P(x h ,y h ) represents the center point of the person’s hand, ratio a Indicates the ratio from wrist to elbow.

[0071] In some specific embodiments, step S3, using the behavior recognition model to extract features from the hand area of ​​the person, and introducing the residual attention mechanism to identify and locate the hand area and the electrostatic ball, so as to determine whether the hand area of ​​the person in the video to be tested touches the electrostatic ball; specifically includes:

[0072] like Figure 2 As shown, the overall framework of the behavior recognition model adopts the basic ResNet framework;

[0073] The hand area of ​​the person is input into the behavior recognition model for processing to obtain a 224×224 image; then the image passes through a 7×7 convolution layer, a maximum pooling layer, and three combination modules (a combination module includes: a deep residual convolution Block and a residual attention module), and then passes through a deep residual convolution Block for global average pooling, and finally outputs through two fully connected layers to obtain the positioning of the hand area and the electrostatic ball area.

[0074] Among them, the deep residual convolution Block consists of two 3×3 convolution layers with a stride of 2, and the input of the first 3×3 convolution layer with a stride of 2 is jump-connected to the output of the second 3×3 convolution layer with a stride of 2. The residual attention module consists of multiple 3×3 convolution layers and relu activation functions, which are then output after channel fusion through a 1×1 convolution layer and a sigmoid activation function.

[0075] Since the input hand area image has a low resolution, the residual attention mechanism is used to assist in locating and identifying the features of the hand area. The structural flow of the residual attention mechanism is as follows: Figure 3 As shown in the figure, after multiple 3×3 convolutional layers and relu activation functions are composed, the output is through a 1×1 convolutional layer and a sigmoid activation function after channel fusion.

[0076] In some specific embodiments, in step 4, a comprehensive judgment of multi-frame voting decision is performed on the hand region information to determine whether the person in the video to be tested actually touches the electrostatic ball, specifically including:

[0077] If the hand area of ​​the person in the video to be tested touches the electrostatic ball, a multi-frame voting decision is made on the hand area information corresponding to the hands of the person in each frame of the video to determine whether the person meets the predetermined time of touching the electrostatic ball;

[0078] If the person meets the predetermined time of touching the electrostatic ball, it means that the person in this video to be tested actually touches the electrostatic ball.

[0079] The following formula is used to determine whether the person meets the time limit for touching the electrostatic ball:

[0080]

[0081] Among them, Y represents the result of judging whether the person meets the time requirement of touching the electrostatic ball. If Y is True, it means that the person meets the time requirement of touching the electrostatic ball. If Y is False, it means that the person does not meet the time requirement. t represents the time requirement of touching the electrostatic ball. f represents the number of frames within the preset time of the video. S lh Indicates whether the person's left hand is touching the electrostatic ball. S rhIndicates whether the person's right hand is touching the electrostatic ball (1 for touching, 0 for not touching).

[0082] Based on the same inventive concept, an embodiment of the present invention further provides an electrostatic ball touch recognition system, such as Figure 4 As shown, it includes an extraction unit, an acquisition unit, a recognition unit and a determination unit,

[0083] An extraction unit, used to extract human skeleton point information of a person in each frame of the video to be tested;

[0084] The acquisition unit is used to obtain the hand area of ​​the person according to the human skeleton point information; the recognition unit is used to extract the features of the hand area of ​​the person, and introduce the residual attention mechanism to identify and locate the hand area and the electrostatic ball, and determine whether the hand area and the electrostatic ball area in the video to be tested overlap;

[0085] The determination unit is used to perform multi-frame voting decision judgment on the hand area information when the hand area coincides with the electrostatic ball area, so as to determine whether the person in this section of the video to be tested actually touches the electrostatic ball.

[0086] In some specific embodiments, the extraction unit is specifically used to input the video to be tested into a human body detection algorithm model to obtain a human body frame of each frame of the person, and input the human body frame into a skeleton point detection model to obtain human body skeleton point information of the person;

[0087] The acquisition unit is specifically used to obtain the center point of the person's hand by using the bone point information of the person's elbow and wrist; and to obtain the person's hand area based on the center point of the person's hand.

[0088] Regarding the system in the above embodiment, the specific manner in which each unit module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0089] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, whose structure is as follows: Figure 5 As shown, it includes: a memory and a processor, and the processor is used to read and execute the computer program stored in the memory to implement the above-mentioned electrostatic ball touch recognition method.

[0090] Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and when the computer executable instructions are executed, the aforementioned electrostatic ball touch recognition method is implemented.

[0091] The present invention uses skeleton points to extract the hand area of ​​the human body, and uses a feature recognition model based on the residual attention mechanism to distinguish it; in order to better improve the accuracy and robustness, a multi-frame voting strategy is also used to comprehensively judge whether the human body in a video has been touched by an electrostatic ball, thereby achieving accurate identification and early warning of "electrostatic ball touch", reducing the possibility of safety accidents, and improving the protection of the production environment and human life safety.

[0092] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying electrostatic ball contact, characterized in that: include: Extract the human skeleton point information of each person in each frame of the video to be tested; Acquire the hand area of ​​the person according to the human skeleton point information; The hand area of ​​the person is feature extracted, and the residual attention mechanism is introduced to identify and locate the hand area and the electrostatic ball area, and determine whether the hand area and the electrostatic ball area in the video to be tested overlap; When the hand area overlaps with the electrostatic ball area, a comprehensive judgment of multi-frame voting decision is performed on the hand area information to determine whether the person in this section of the video to be tested actually touches the electrostatic ball.

2. The electrostatic ball touch identification method according to claim 1, characterized in that: Extract the human skeleton point information of each frame of the video to be tested, including: Input the video to be tested into the human body detection algorithm model to obtain the human body frame of each frame; The human body frame is input into the skeleton point detection model to obtain the human body skeleton point information of the person.

3. The electrostatic ball touch identification method according to claim 1 or 2, characterized in that: According to the human skeleton point information, the hand area of ​​the person is obtained, including: Using the bone point information of the elbow and wrist of the person, the center point of the person's hand is obtained; Get the person's hand area based on the center point of the person's hand.

4. The electrostatic ball touch identification method according to claim 3, characterized in that: Using the bone point information of the elbow and wrist of the person, the center point of the person's hand is obtained, including: With the person's elbow as the starting point of the vector and the wrist as the end point of the vector, the center point of the person's hand is calculated using the following formula: P(x h ,y h )=P(x1,y1)+(P(x1,y1)-P(x2,y2))*ratio a (1) Among them, P(x h ,y h ) represents the center point of the person's hand, P(x1,y1) represents the person's elbow, i.e. the starting point; P(x2,y2) represents the person's wrist, i.e. the end point; ratio a Indicates the ratio from wrist to elbow.

5. The electrostatic ball touch identification method according to claim 3, characterized in that: According to the center point of the person's hand, obtain the person's hand area, including: According to the center point of the person's hand, the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are calculated to obtain the person's hand area; The vector distance from the elbow to the wrist of the person is used as the measurement standard. Assume that the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are P(x a ,y a )、P(x b ,y b ), the coordinates of the upper left corner and the lower right corner of the rectangular frame of the hand area are calculated by the following formula: Where dis represents the vector distance from the elbow to the wrist, P(x a ,y a ) represents the coordinate of the upper left corner of the rectangular box of the hand area, P(x b ,y b ) represents the lower right corner coordinate of the rectangular box of the hand area, P(x h ,y h ) represents the center point of the person’s hand, ratio a Indicates the ratio from wrist to elbow.

6. The electrostatic ball touch identification method according to claim 1, characterized in that: A comprehensive judgment of multi-frame voting decisions is performed on the hand area information to determine whether the person in the video to be tested actually touches the electrostatic ball, including: If the hand area of ​​the person in the video to be tested touches the electrostatic ball, a multi-frame voting decision is made for the hand area information corresponding to both hands of the person in each frame of the video to determine whether the person meets the time limit for touching the electrostatic ball; If the person meets the time requirement of touching the electrostatic ball, it means that the person in this video under test actually touches the electrostatic ball; The following formula is used to determine whether the person meets the time limit for touching the electrostatic ball: In the formula, Y represents the result of judging whether the person meets the time requirement of touching the electrostatic ball. If Y is True, it means that the person meets the time requirement of touching the electrostatic ball. If Y is False, it means that the person does not meet the time requirement. t represents the time requirement of touching the electrostatic ball. f represents the number of frames within the preset time of the video. S lh Indicates whether the person's left hand is touching the electrostatic ball. S rh Indicates whether the person's right hand is touching the electrostatic ball.

7. An electrostatic ball touch recognition system, characterized in that: It includes an extraction unit, an acquisition unit, an identification unit and a determination unit, The extraction unit is used to extract the human skeleton point information of the person in each frame of the video to be tested; The acquisition unit is used to acquire the hand area of ​​the person according to the human skeleton point information; The recognition unit is used to extract features of the hand area of ​​the person, and introduce a residual attention mechanism to recognize and locate the hand area and the electrostatic ball, and determine whether the hand area and the electrostatic ball area in the video to be tested overlap; The determination unit is used to perform a comprehensive judgment of multi-frame voting decisions on the hand area information when the hand area coincides with the electrostatic ball area, so as to determine whether the person in this section of the video to be tested actually touches the electrostatic ball.

8. The electrostatic ball touch identification system according to claim 7, characterized in that: The extraction unit is specifically used to input the video to be tested into the human body detection algorithm model to obtain the human body frame of each frame, and input the human body frame into the skeleton point detection model to obtain the human body skeleton point information of the person; The acquisition unit is specifically used to obtain the center point of the person's hand by using the bone point information of the person's elbow and wrist; and to obtain the person's hand area based on the center point of the person's hand.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to read and execute a computer program stored in the memory to implement the electrostatic ball touch identification method described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed, the electrostatic ball touch recognition method according to any one of claims 1 to 6 is implemented.

Images