A power scene dressing specification detection method based on human key points
By processing image information with YOLO V5 and SimplePose, and combining skin color segmentation and interval sampling in the HSV color space, the problem of insufficient generalization ability and misjudgment in clothing compliance detection in power scenarios is solved, achieving fine-grained recognition and high-precision detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI UNIV
- Filing Date
- 2023-03-22
- Publication Date
- 2026-06-19
AI Technical Summary
Existing methods for detecting clothing standards in power scenarios have poor generalization ability, cannot perform fine-grained recognition, and have a high rate of false positives, especially in the skin-colored area of the hands.
YOLO V5 and SimplePose were used to process the image information, obtain human body key point information, perform skin color segmentation in the HSV color space, and perform interval sampling on the human limbs. The skin color segmentation information was combined for judgment, and the hand area was located to eliminate false detections. The comprehensive results were used to judge whether the clothing was in accordance with regulations.
It enables fine-grained recognition of items such as rolled-up sleeves and trouser legs, improving the accuracy of dress code detection and reducing the impact of environmental factors such as occlusion and shadows, thus enhancing detection precision.
Smart Images
Figure CN116311065B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of safety monitoring, and in particular to a method for detecting clothing standards in power scenarios based on key points on the human body. Background Technology
[0002] Power facilities are a crucial component of the national infrastructure. Ensuring the reliable operation of these facilities is paramount, but so is guaranteeing the safety of personnel. The "State Grid Corporation of China Power Safety Work Regulations" stipulate that safety helmets must be worn correctly when entering work sites, along with long-sleeved work clothes, ensuring no large areas of skin are exposed. While safety regulations are primarily supervised manually at production sites, in certain situations, such as hot weather, workers may habitually roll up their sleeves or trousers, violating safety regulations.
[0003] With the rise of computer vision technology, it has become possible to replace manual labor in continuous on-site inspections using visual technology. Existing detection methods can identify targets with different sleeve lengths, but they are not feasible for more granular identification, such as rolled-up sleeves or trouser legs. Furthermore, real-world environments inevitably have the influence of factors such as occlusion and shadows, resulting in poor generalization ability of existing power sector clothing compliance detection algorithms.
[0004] On the other hand, since hands are not required to be covered by clothing, and hands and arms often overlap in daily work (e.g., in a monitoring perspective where the hand is in front and the arm is behind), the skin-colored area exposed by the hand can be misidentified by the system as an arm, and consequently, the target may be misjudged as having unusual attire. This results in a large number of false detections and significantly impacts detection accuracy. Summary of the Invention
[0005] In view of the aforementioned shortcomings of the prior art, the technical problem to be solved by the present invention is that existing safety attire detection methods have poor generalization ability, cannot perform fine-grained identification, and have a high rate of false positives. The present invention provides a power scenario attire compliance detection method based on key human body points, which can perform fine-grained identification and detection, is applicable to various actual power environments, and improves the accuracy of attire compliance detection.
[0006] A preferred embodiment of the present invention provides a method for detecting clothing compliance in power scenarios based on key points of the human body, comprising the following steps:
[0007] Import the image to be tested;
[0008] YOLO V5 and SimplePose were used to process the image information of the image to be detected, and human body key point information was obtained.
[0009] For the image to be detected, obtain the skin color segmentation information of the image to be detected in the HSV color space;
[0010] Interval sampling was performed on the four limbs of the human body, and skin color segmentation information was combined to determine whether the clothing at the sampling points was abnormal;
[0011] Locate the hand area to eliminate false positives caused by hand skin color;
[0012] The overall results are used to determine whether the target's attire is appropriate.
[0013] Furthermore, the images to be detected include single-person images and multi-person images.
[0014] Furthermore, YOLO V5 and SimplePose are used to process the image information of the image to be detected to obtain human body key point information. Specifically, YOLO V5 is used as a target human body detector to detect the target human body in the image to be detected, and SimplePose is used as a single-person key point prediction model to detect the human body key points of the target human body. The human body key points are in COCO2017 format, and a total of 17 key points are detected.
[0015] Furthermore, obtaining skin tone segmentation information within the HSV color space specifically includes the following steps:
[0016] The R, G, and B values of the image to be detected are normalized.
[0017] Calculate the V, S, and H values for the HSV color space respectively;
[0018] Using the optimal threshold, skin-colored regions in the image to be detected in the HSV space are segmented.
[0019] Furthermore, the R, G, and B values of the image to be detected are normalized using the following formula:
[0020]
[0021] The V value of the HSV color space is calculated using the following formula:
[0022] V = max(R, G, B)
[0023] The S value of the HSV color space is calculated using the following formula:
[0024]
[0025] The H value of the HSV color space is calculated using the following formula:
[0026] σ = V - min(R, G, B)
[0027]
[0028] The skin-colored region of the image to be detected in the HSV space is segmented using the optimal threshold obtained by manual selection.
[0029] Furthermore, interval sampling is performed on the limbs of the human body, and skin color segmentation information is combined to determine whether the clothing of the sampling points is abnormal. Specifically, interval sampling is performed on the area of interest, and the sampling points are compared with the skin color segmentation map of the corresponding location and the number of points in the corresponding area is judged as skin, thereby determining whether the clothing of the area is abnormal.
[0030] Furthermore, to locate the hand region and remove false positives caused by hand skin color, this involves using a circle to represent the hand region, determining the center and radius of the circle to define the hand region, and taking two points on one arm, the elbow and wrist, as (x1, y1) and (x2, y2) respectively, and setting the center of the hand circle as (x1, y1). h y h Given a radius of r, the coordinates of the hand's center and radius can be obtained from the following correspondence:
[0031] x h = x² + 0.3 × (x² - x¹)
[0032] y h = y2 + 0.3 × (y2 - y1)
[0033]
[0034] Furthermore, the overall results are used to determine whether the target's clothing is in accordance with regulations, including the results of sampling the area of interest and the results after eliminating false detections of hands, to comprehensively determine whether there are any abnormalities in the clothing.
[0035] Technical effect
[0036] This invention proposes a method for detecting clothing compliance in power industry scenarios based on human key points. It uses YOLO V5 and SimplePose to process image information to obtain human key points, and simultaneously obtains skin color segmentation information in the HSV color space. Then, it performs interval sampling on the limbs and uses the skin color segmentation information to judge the sampled points. Simultaneously, it locates the hand region and eliminates false detections caused by hand skin color. The overall result determines whether the target's clothing conforms to regulations. This detection method, especially the step of eliminating false detections by judging the hand region after interval sampling, allows for fine-grained recognition such as rolled-up sleeves and trouser legs, while improving the accuracy of clothing compliance detection. It can be applied to more scenarios and is less affected by factors such as occlusion and shadows in the actual environment.
[0037] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description
[0038] Figure 1 This is a flowchart illustrating a preferred embodiment of the present invention: a method for detecting clothing standards in power scenarios based on key human body points.
[0039] Figure 2 This is a schematic diagram of 17 key points of the human body in a power scene clothing standard detection method based on human body key points according to a preferred embodiment of the present invention.
[0040] Figure 3 This is a rendering of a preferred embodiment of the present invention, a method for detecting clothing standards in a power scenario based on key human body points, in a real power scenario.
[0041] Figure 4 This is a schematic diagram illustrating the result of processing human body key points in a preferred embodiment of the present invention, which is a method for detecting clothing standards in power scenarios based on human body key points.
[0042] Figure 5 This is a schematic diagram of the skin color binarization segmentation effect of the image to be detected in a preferred embodiment of the present invention, which is a method for detecting clothing standards in power scenes based on human body key points.
[0043] Figure 6 This is a schematic diagram of the preliminary judgment result of sampling points in a power scene clothing standard detection method based on human body key points according to a preferred embodiment of the present invention;
[0044] Figure 7 This is a schematic diagram showing the result of locating the hand area in the image to be detected in a preferred embodiment of the present invention, which is a method for detecting clothing standards in power scenarios based on human body key points. Detailed Implementation
[0045] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
[0046] In the following description, specific details, such as particular internal procedures and techniques, are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will appreciate that the invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of the invention with unnecessary detail.
[0047] like Figure 1As shown, this invention provides a method for detecting clothing compliance in power scenarios based on key points of the human body, including the following steps:
[0048] Step 100: Import the image to be detected;
[0049] Step 200: Use YOLO V5 and SimplePose to process the image information to obtain human body key points;
[0050] Step 300: Obtain skin color segmentation information of the image to be tested in the HSV color space;
[0051] Step 400: Perform interval sampling on the four limbs of the human body and combine the skin color segmentation information to determine the sampling points;
[0052] Step 500: Locate the hand area to remove false detections caused by hand skin color;
[0053] Step 600: Based on the overall results, determine whether the target's attire is appropriate.
[0054] In step 200, the human body key points are in COCO2017 format, which can detect 17 key points, such as... Figure 2 As shown: Key point 0, nose; Key point 1, left eye; Key point 2, right eye; Key point 3, left ear; Key point 4, right ear; Key point 5, left shoulder; Key point 6, right shoulder; Key point 7, left elbow; Key point 8, right elbow; Key point 9, left wrist; Key point 10, right wrist; Key point 11, left hip; Key point 12, right hip; Key point 13, left knee; Key point 14, right knee; Key point 15, left ankle; Key point 16, right ankle.
[0055] Step 300 involves segmenting the skin tone in the HSV space of the image to be detected using a fixed threshold. This includes the following steps: Step 301, to facilitate subsequent conversion from RGB to HSV format, the R, G, and B values of the image to be detected are normalized using the following formula:
[0056]
[0057] Step 302: Calculate the V value of the HSV color space using the following formula:
[0058] V = max(R, G, B)
[0059] Step 303: Calculate the S value of the HSV color space using the following formula:
[0060]
[0061] Step 304: Calculate the H value of the HSV color space using the following formula:
[0062] σ = V - min(R, G, B)
[0063]
[0064] Step 305: Using repeated experiments and manual selection of the optimal threshold corresponding to the best comprehensive performance index, the skin-colored region of the image to be detected in the HSV space is binarized and segmented. From the perspective of the overall image, the skin-colored and non-skin-colored regions are set as bright and dark areas. If the sampling point coordinates of the corresponding limbs are located in the bright area, it can be determined that there is exposed skin, which provides a basis for the judgment of clothing standards in subsequent steps. Among them, the optimal threshold is selected by experimental evaluation indicators such as accuracy, precision, recall, and F1 score to select the optimal threshold corresponding to the best comprehensive performance index.
[0065] Step 400: Since the limbs can be located using the key points of the human body in Step 200, but only the elbows, wrists, knees, and ankles can be located; the areas that need to be focused on for clothing compliance inspection are the arms and lower legs, that is, the areas in the middle of the lines connecting the key points from the elbow to the wrist and from the knee to the ankle. Judging the above-mentioned areas of focus involves performing interval sampling on the human limbs and combining this with skin color segmentation information to judge the sampled points, specifically including:
[0066] From the key points of the human body obtained in step 200, two points located on one arm, the elbow and the wrist, are selected as (x1, y1) and (x2, y2) respectively. Then, the arm length is divided into four equal parts, and three of these division points are taken as sampling points. Since the arm is a straight line, dividing the arm length into four equal parts is equivalent to dividing the difference between the x-coordinate and y-coordinate of the elbow and wrist into four equal parts.
[0067] Δx = 0.25 × (x2 - x1)
[0068] Δy = 0.25 × (y2 - y1)
[0069] The coordinates of the three sampling points obtained after dividing the image into four equal parts are (x1+Δx, y1+Δy), (x1+2Δx, y1+2Δy), and (x1+3Δx, y1+3Δy). Then, it is determined whether these three sampling points are located in the bright areas after image segmentation in step 300. If they are located in the bright areas, it can be preliminarily determined that the clothing in that area is abnormal. Finally, following this process, the other arm and both lower legs are also assessed, and a preliminary judgment on the clothing standards of the limbs is obtained by combining the results.
[0070] In step 500, to eliminate possible misjudgments of the hand region in step 400 (e.g., in a monitoring perspective, if the hand is in front and the arm is behind, the skin color of the hand might cause the arm to be judged as abnormal clothing), the hand region needs to be located, and then sampling points in the hand region are removed. Since the detection results of the key point detection model do not include the hand, the hand region can be inferred using prior knowledge. Since the hand region is an extension of the arm region, the length of the hand is generally proportional to the length of the arm, and the hand region can be represented by a circle, therefore, only the center and radius of the circle need to be determined to determine the hand region. From the human body key points obtained in step 200, take the elbow and wrist points on one arm as (x1, y1) and (x2, y2) respectively, and let the center of the hand circle be (x1, y1) and (x2, y2). h y h Given a radius of r, the coordinates of the hand's center and radius can be obtained from the following correspondence:
[0071] x h = x² + 0.3 × (x² - x¹)
[0072] y h = y2 + 0.3 × (y2 - y1)
[0073]
[0074] The method of the present invention will be illustrated below with specific examples. The images to be detected are selected from actual power operation environments where hand misjudgment is prone to occur, such as... Figure 3 As shown. After importing the image to be detected, the first step is to use YOLO V5 and SimplePose to process the image information to obtain human body key points. The processing result is shown below. Figure 4 As shown. The second step involves processing the image within the HSV color space using a manually selected optimal threshold (H value range of 10-40, S value greater than 0.11, V value greater than 0.2) to obtain the skin tone binarization segmentation information of the image to be detected. The result is shown in the image below. Figure 5 As shown. The third step involves interval sampling of the limbs and combining this with skin color segmentation information to determine the sampling points. It can be concluded that the sampling point (718.75, 800.17) on the left arm is located in the bright area after binarization and segmentation, and is initially judged as having abnormal clothing regulations. The effect is as follows: Figure 6As shown, it can be seen that due to the camera's shooting angle, the workers were mistakenly judged as having abnormal clothing. The fourth step, to remove potential false positives due to hand skin color, involves locating the hand area in the image to be detected: From step 500, the coordinates of the left elbow are (552.78, 833.36), and the coordinates of the left wrist are (884.71, 768.97). Using the formula, the center coordinates of the left hand are (984.29, 747.05), and the radius of the left hand's center is 101.55; the coordinates of the right elbow are (519.58, 883.15), and the coordinates of the right wrist are (702.15, 800.16). Using the formula, the center coordinates of the right hand's center are (756.92, 775.26), and the radius of the right hand's center is 60.16. The effect is as follows: Figure 7 As shown, this method can detect whether the clothing of on-site personnel is standardized in actual power operation environments, and can effectively eliminate misjudgments caused by hand skin color, with high accuracy.
[0075] This invention proposes a method for detecting clothing compliance in power scenarios based on human body key points. It uses YOLO V5 and SimplePose to process image information to obtain human body key points, and simultaneously obtains skin color segmentation information in the HSV color space. Then, it performs interval sampling on the human limbs and judges the sampling points in conjunction with the skin color segmentation information. At the same time, it locates the hand area and removes false detections caused by hand skin color. The comprehensive results determine whether the target's clothing complies with the regulations, thus improving the accuracy of clothing compliance detection.
[0076] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A method for detecting power scene dressing specification based on human key points, characterized in that, Includes the following steps: Import the image to be tested; YOLO V5 and SimplePose were used to process the image information of the image to be detected to obtain human body key point information; the human body key points were in COCO2017 format, and 17 key points were detected: key point 0, nose; Key point 1, left eye; Key point 2, right eye; Key point 3, left ear; Key point 4, right ear; Key point 5, left shoulder; Key point 6, right shoulder; Key point 7, left elbow; Key point 8, right elbow; Key point 9, left wrist; Key point 10, right wrist; Key point 11, left hip; Key point 12, right hip; Key point 13, left knee; Key point 14, right knee; Key point 15, left ankle; Key point 16, right ankle. For the image to be tested, skin color segmentation information of the image to be tested is obtained in the HSV color space; The skin-colored region of the image to be detected in the HSV space is binarized and segmented. From the perspective of the overall image, the skin-colored and non-skin-colored regions are set as bright and dark areas. If the sampling point coordinates of the corresponding limbs are located in the bright area, it can be determined that the skin is exposed, which provides a basis for the subsequent step of judging the dress code. Interval sampling was performed on the four limbs of the human body, and skin color segmentation information was combined to determine whether the clothing at the sampling points was abnormal; Interval sampling is performed on the limbs of the human body, and the sampling points are determined by combining skin color segmentation information. Specifically, this includes: From the obtained key points of the human body, two points are selected: the elbow and the wrist on one arm. and Subsequently, the arm length is divided into four equal parts, and three of these parts are taken as sampling points. Since the arm is a straight line, dividing the arm length into four equal parts is equivalent to dividing the difference between the x-coordinate and y-coordinate of the elbow and wrist into four equal parts. ， ， The coordinates of the three sampling points obtained after dividing the sample into four equal parts are: , and Then, determine whether these three sampling points are located in the bright area after the image is segmented; if they are located in the bright area, it can be preliminarily determined that the clothing in this area is abnormal; finally, follow this process to judge the other arm and the two lower legs respectively, and obtain a preliminary judgment on the clothing standard of the limbs; To eliminate false positives caused by hand skin color, the hand area is located. Specifically, this involves using a circle to represent the hand area, determining the center and radius of the circle to define the hand region, and selecting the elbow and wrist as two points on one arm. and Let the center of the hand be... , radius is The coordinates of the hand's center and radius can be obtained from the following correspondence: ， ， ； The overall results are used to determine whether the target's attire is appropriate.
2. The method for detecting clothing standards in power scenarios based on key human body points as described in claim 1, characterized in that, The images to be detected include single-person images and multi-person images.
3. The method for detecting clothing standards in power scenarios based on key human body points as described in claim 1, characterized in that, The image information of the image to be detected is processed using YOLO V5 and SimplePose to obtain human body key point information. Specifically, YOLO V5 is used as a target human body detector to detect the target human body in the image to be detected, and SimplePose is used as a single-person key point prediction model to detect the human body key points of the target human body. The human body key points are in COCO2017 format, and a total of 17 key points are detected.
4. The method for detecting clothing standards in power scenarios based on key human body points as described in claim 2, characterized in that, Obtaining skin tone segmentation information within the HSV color space involves the following steps: The R, G, and B values of the image to be detected are normalized. Calculate the V, S, and H values for the HSV color space respectively; Using the optimal threshold, skin-colored regions in the image to be detected in the HSV space are segmented.
5. The method for detecting clothing standards in power scenarios based on key human body points as described in claim 4, characterized in that, The R, G, and B values of the image to be detected are normalized using the following formula: ； The V value of the HSV color space is calculated using the following formula: ， The S value of the HSV color space is calculated using the following formula: ， The H value of the HSV color space is calculated using the following formula: ， ， The skin-colored region of the image to be detected in the HSV space is segmented using the optimal threshold obtained by manual selection.
6. The method for detecting clothing standards in power scenarios based on key human body points as described in claim 1, characterized in that, Intermittent sampling is performed on the limbs of the human body, and skin color segmentation information is combined to determine whether the clothing is abnormal. Specifically, this involves intermittent sampling of the area of interest, comparing the sampling points with the skin color segmentation map of the corresponding location, and counting how many points in the corresponding area are identified as skin, thereby determining whether the clothing in the area is abnormal.
7. The method for detecting clothing standards in power scenarios based on key human body points as described in claim 6, characterized in that, The comprehensive results are used to determine whether the target's clothing is in accordance with regulations. This includes judging whether there are any abnormalities in the clothing based on the sampling results of the area of interest and the results after eliminating false detections of hands.