Safety helmet wearing identification method and device

Abstract

The invention discloses a safety helmet wearing recognition method and device, and the method comprises the steps: obtaining a video stream, carrying out the pedestrian recognition of each video frame in the video stream, and obtaining a target picture containing a pedestrian; performing human body recognition on the target picture to obtain a human body recognition result of the target picture; the human body identification comprises human body skeleton key point identification, human body semantic segmentation and / or human body image quality; according to the human body recognition result, performing safety helmet wearing classification recognition on the target picture to obtain a corresponding safety helmet wearing classification recognition result; and determining alarm information according to the target picture, the human body recognition result and / or the safety helmet wearing classification recognition result. Through human body recognition, the false recognition rate caused by the influence of shielding, blurring, dark light, backlight and other factors under the condition that the safety helmet is worn can be greatly reduced. By means of refined recognition processing, safety helmet recognition can be improved, and recognition of various safety helmets is supported. The interpretability of the recognition result is improved based on the alarm information, and the method is suitable for various wearing scenes.

Description

Technical Field

[0001] This invention relates to the field of image recognition, specifically to a method and apparatus for recognizing the wearing of a safety helmet. Background Technology

[0002] Safety accidents in industries such as construction, manufacturing, and petrochemicals often involve issues such as improper helmet wearing. Head injuries are a common type of injury on construction sites, and wearing a safety helmet can effectively prevent them. However, in actual operations, due to inadequate supervision and insufficient worker awareness, improper helmet wearing frequently occurs.

[0003] Traditional monitoring methods rely primarily on manual inspections, which are inefficient and lack comprehensive coverage. Real-time monitoring of construction sites utilizes conventional deep learning-based helmet detection methods, such as directly detecting helmets using fixed models, color-coding helmets based on pedestrian head detection, or matching detection frames with pedestrian head frames. However, these methods often suffer from delayed alarms, false alarms, or missed alarms. Furthermore, recognition performance deteriorates under conditions of lighting, angle, and occlusion, and conventional data iteration models cannot address these issues. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a helmet wearing recognition method and apparatus that overcomes or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a helmet wearing recognition method is provided, the method comprising:

[0006] Acquire a video stream, perform pedestrian recognition on each video frame in the video stream, and obtain target images containing pedestrians;

[0007] Perform human body recognition on the target image to obtain the human body recognition result of the target image; human body recognition includes human skeleton key point recognition, human semantic segmentation and / or human image quality;

[0008] Based on the human body recognition results, the helmet wearing classification and recognition of the target image is performed to obtain the corresponding helmet wearing classification and recognition results;

[0009] The alarm information is determined based on the target image, human body recognition results, and / or helmet wearing classification recognition results.

[0010] According to another aspect of the present invention, a helmet wearing recognition device is provided, comprising:

[0011] The pedestrian recognition module is suitable for acquiring video streams, performing pedestrian recognition on each video frame in the video stream, and obtaining target images containing pedestrians.

[0012] The human body recognition module is suitable for performing human body recognition on target images to obtain human body recognition results for the target images; human body recognition includes human skeleton key point recognition, human semantic segmentation and / or human image quality;

[0013] The helmet recognition module is suitable for classifying and recognizing helmet wearing in target images based on human body recognition results, and obtaining the corresponding helmet wearing classification and recognition results.

[0014] The alarm module is suitable for determining alarm information based on target images, human body recognition results, and / or helmet wearing classification recognition results.

[0015] According to another aspect of the present invention, a computing device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0016] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the helmet wearing recognition method described above.

[0017] According to another aspect of the present invention, a computer storage medium is provided, the storage medium storing at least one executable instruction, the executable instruction causing a processor to perform an operation corresponding to the above-described helmet wearing recognition method.

[0018] According to another aspect of the present invention, a computer program product is provided, including at least one executable instruction that causes a processor to perform an operation corresponding to the helmet wearing recognition method described above.

[0019] The helmet wearing recognition method and apparatus provided in this invention, through human body recognition, can significantly reduce the false recognition rate caused by factors such as obstruction, blurriness, low light, and backlighting when wearing a helmet. Utilizing refined recognition processing, helmet recognition can be improved, supporting the recognition of various helmets. Based on alarm information, the interpretability of the recognition results is enhanced, adapting to various helmet wearing scenarios.

[0020] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more obvious and understandable, specific implementation methods of the embodiments of the present invention are described below. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0022] Figure 1 A flowchart of a helmet wearing recognition method according to an embodiment of the present invention is shown;

[0023] Figure 2 This diagram illustrates a situation where the target image has poor brightness and contrast.

[0024] Figure 3 This diagram illustrates the filtering of target images based on attribute information.

[0025] Figure 4 A schematic diagram of a helmet wearing recognition device according to an embodiment of the present invention is shown;

[0026] Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. Detailed Implementation

[0027] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0028] Figure 1 A flowchart of a helmet wearing recognition method according to an embodiment of the present invention is shown, as follows: Figure 1 As shown, the method includes the following steps:

[0029] Step S101: Obtain the video stream, perform pedestrian recognition on each video frame in the video stream, and obtain the target image containing pedestrians.

[0030] The video stream can be acquired using a capture device such as a camera. It can include a video stream captured when workers clock in, containing a photo of the worker clocking in. This photo can be used to determine if the safety helmet is correctly worn. Alarms can also be triggered based on the worker's posture, lighting conditions, etc., to remind them to clock in correctly. The above is just an example; specific settings will vary depending on the implementation. It can also be used for safety helmet recognition in other scenarios, and is not limited here.

[0031] The video stream contains multiple video frames. Each video frame may contain pedestrians (i.e., the video frame when workers clock in) or may not contain pedestrians (i.e., the video frame when no one clocks in). To facilitate efficient and accurate identification of safety helmet wearing status in this embodiment, pedestrian identification can be performed on each video frame in the video stream first. During identification, pedestrian identification is performed on each video frame in the video stream separately to obtain target images containing pedestrians.

[0032] Pedestrian recognition can be performed using a pre-defined pedestrian detection model, such as the YOLO deep learning-based real-time object detection model. This model is trained on pedestrian sample data to achieve high recall. The pre-defined pedestrian detection model can detect pedestrians in various postures, including partial occlusion, distant and small pedestrians, strong light and backlighting, motion blur, bending, squatting, and sitting. When recognizing video frames, after identifying the target image containing pedestrians, the pre-defined pedestrian detection model marks the pedestrian detection box in the target image and determines the detection score of the target image. The detection score can be used for subsequent alarm information and to determine the recognition status of the video stream. The pedestrian detection box can mark the pedestrians displayed in the target image, such as by using geometric lines like rectangles to select and mark the entire pedestrian. The specific settings depend on the implementation and are not limited here.

[0033] Pedestrian recognition allows for the filtering of target images containing pedestrians from the video stream. Subsequent recognition can then focus solely on these target images, eliminating the need to identify non-target images and improving efficiency. Furthermore, after identifying the pedestrian detection bounding boxes in the target images, the aspect ratio can be calculated. Target images with aspect ratios below a preset threshold undergo initial filtering. This filtering is based on the aspect ratio of the pedestrian detection boxes. If a pedestrian is far away and not yet near the acquisition device (e.g., passing by from a distance and not a worker), further recognition can be omitted. Similarly, if a pedestrian is small, it indicates they are not centered and are not the primary target of the acquisition device, meaning they may not be a worker. When a pedestrian is small and far away, their aspect ratio will be relatively low compared to the normal threshold, allowing for initial filtering and preventing misidentification.

[0034] Step S102: Perform human body recognition on the target image to obtain the human body recognition result of the target image.

[0035] After the target image is determined, human body recognition can be performed on the target image. Human body recognition includes human skeleton key point recognition, human semantic segmentation, and human image quality assessment.

[0036] Human body recognition can perform human body identification on target images using a pre-defined neural convolutional network. This network can be, for example, a neural convolutional network with a single backbone and four branches. By identifying pedestrians within the pedestrian detection bounding box in the target image, the system can obtain key points of the human skeleton. These key points include various points on the human skeleton, such as the top of the head, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. This can cover various joints, facilitating the localization of different body parts. Furthermore, based on the key point recognition results, semantic segmentation can be achieved, such as identifying different body parts through different key points. Semantic segmentation results include parts such as the head, upper body, and lower body. Human image quality information is determined based on image brightness and blur level. For example, by identifying the target image, we can determine whether the target image is dark and determine the blur level of the target image. The blur level can be divided into different levels according to the sharpness of the target image, such as sharp, relatively sharp, intermediate level (between sharp and blurry), blurry, very blurry, etc. Different levels can correspond to different sharpness, etc., which are not limited here.

[0037] In the pre-defined neural convolutional network, human skeleton keypoint recognition can be trained using loss functions such as MSE (Mean Squared Error). Human semantic segmentation can be trained by combining loss functions such as Dice, cross-entropy, and focal loss. Human image quality information can be trained using loss functions such as cross-entropy. The backbone of the pre-defined neural convolutional network can be self-supervised based on a large-scale pedestrian sample dataset, using models such as MoCo v3. Alternatively, the backbone can be replaced with models such as HRNet. The parameters in the model can be frozen during training of branches with different human recognition results and not participate in the training. For example, when training for human skeleton keypoint recognition, parameters related to human semantic segmentation and human image quality information can be frozen. The above is an example; specific settings should be made according to the implementation situation and are not limited here.

[0038] Furthermore, after obtaining the human recognition result from the target image through human recognition, a second filtering process can be applied to target images whose image brightness and blur level are below a preset quality threshold. For example... Figure 2 As shown, the target images are all taken in low light conditions, with brightness information below the preset quality threshold. Furthermore, the first image is very blurry, so it can be filtered out and not used for subsequent recognition, which would affect the recognition effect and easily lead to misidentification. The preset quality threshold can include parameters such as brightness threshold and blur level threshold, and the specific setting depends on the implementation situation; no limitation is made here.

[0039] This embodiment introduces human body recognition. By recognizing key points of the human skeleton, segmenting human semantics, and assessing human image quality, it can more accurately judge and filter the quality of human images and segment the human head more accurately, thereby greatly reducing the false recognition rate in actual work scenarios, which can be reduced to below 1%.

[0040] Step S103: Based on the human body recognition results, perform helmet wearing classification and recognition on the target image to obtain the corresponding helmet wearing classification and recognition results.

[0041] After performing human body recognition on the target image, the results can be used for further helmet wearing classification. During helmet wearing classification, a third filtering process can be performed on the target image based on the human body recognition results. This third filtering process includes filtering based on the distance between the bounding box of the human head segmentation and the pedestrian detection box, filtering based on the contour point set of the human head segmentation, filtering based on the edge pixels of the human head region, and filtering based on the positional relationships of key points on the human skeleton. Specifically, filtering can be performed based on the range selected by the pedestrian detection bounding box and the range of the bounding box of the head segmentation after human semantic segmentation. For example, if the distance from the upper edge of the head segmentation bounding box to the upper edge of the pedestrian detection box is less than a preset distance threshold, it indicates that the head of the human in the target image is incomplete, and the target image only displays a part of the head area. In this case, it is filtered out and not further recognized. Alternatively, the segmented head contour point set can be obtained, and an ellipse function can be fitted using functions such as `fitEllipse` in OpenCV to obtain the ratio of the major and minor axes of the ellipse. The ratio is used to judge whether the adjacent points are greater than the preset ellipse ratio, indicating that the head segmentation may be abnormal. In this case, it is filtered out and not further recognized. Alternatively, the convex hull of the head segmentation contour point set can be calculated using functions such as `convexHull` in OpenCV. The area of ​​the convex hull is determined by comparing the overall segmented area of ​​the head with the area of ​​the convex hull. If the ratio is less than a preset ratio, the head segmentation may be abnormal, such as inaccurate segmentation, and the image is filtered out and not further recognized. Alternatively, the RGB pixel values ​​of the edge pixels of the head region (e.g., within a width of 3-10) are determined, and the percentage of these values ​​that are greater than a preset pixel threshold, such as 220 (near white), is judged. If the percentage is greater than the preset percentage, the target image is considered backlit and is filtered out and not further recognized. Alternatively, the human body... The positional relationship of key points in the skeleton is considered. For example, it is determined whether the key points of the left and right shoulders are identified. If the accuracy score is lower than the preset accuracy score, it indicates that the area above the shoulders of the human body in the target image is occluded, and this is filtered out and not further identified. Alternatively, the distance between the head segmentation bounding box and the left and right sides of the pedestrian detection box is used for judgment. If the distance is less than 0, the head of the human semantic segmentation is outside the pedestrian detection box. The detected pedestrian may have actions such as bending over, or the human semantic segmentation may be inaccurate due to occlusion. This is filtered out and not further identified. The above are just examples. The specific settings can be adjusted according to the implementation situation and are not limited here.

[0042] After applying a third-level filter to the target image, images with issues such as recognition errors, low image quality, or occlusion can be filtered out, reducing the false recognition rate. Based on the target image after the third-level filter, its corresponding attribute information can be determined. Attribute information includes orientation, posture, and occlusion attributes. Orientation attributes include facing away, facing forward, and sideways; posture attributes include looking down, looking up, and looking straight ahead; occlusion attributes include occluded and unoccluded. Each attribute can be determined based on the positional relationships of key points in the human skeleton. For example, based on the different positions of the left and right shoulder key points in the target image, the orientation attribute (facing away, facing forward, or sideways) can be determined. Based on the positional relationship between the top of the head, neck, and left and right shoulders, the posture attribute (looking down, looking up, or looking straight ahead) can be determined. Figure 3 As shown in the first image, the pose can be identified as head tilted back. By recognizing key points of the human skeleton, it can be determined whether there are occlusions in the target image, such as... Figure 3 As shown in the second and third images, occlusion exists in the images. Based on the attribute information, a fourth filtering process can be applied to the target images after the third filtering. For example, according to the attribute information, if the human body is facing away and looking down, or facing forward and looking up, or if there is occlusion, the recognition result will be inaccurate even if it is identified. Therefore, target images unsuitable for subsequent recognition are filtered out. By finely classifying attributes such as orientation, posture, and occlusion, it is possible to effectively distinguish situations where the visibility of certain areas is low when wearing a safety helmet, thus eliminating situations unsuitable for safety helmet analysis and avoiding problems such as low head posture and occlusion encountered in actual operations.

[0043] Through the above multiple filtering processes, images unsuitable for helmet wearing recognition can be filtered out, reducing the false recognition rate. A preset image segmentation convolutional neural network is then used to classify and recognize helmet wearing in the target image after the fourth filtering process. The preset image segmentation convolutional neural network can be networks such as FCN, SegNet, or U-Net; no specific limitation is made here. The sample data used for training the preset image segmentation convolutional neural network can be labeled with the helmet area and other areas. Different helmet areas can be labeled according to different scenarios; no specific limitation is made here. By training the preset image segmentation convolutional neural network using the sample data, a preset image segmentation convolutional neural network applicable to various helmet wearing scenarios can be obtained.

[0044] Based on a pre-defined image segmentation convolutional neural network, the head after semantic segmentation can be further segmented to obtain the segmented helmet region and other regions, such as hair regions, other types of hat regions, and skin regions. Furthermore, considering the possibility of baldness, regions larger than a pre-defined skin area ratio above the head can be identified as baldness regions and also included in the "other regions." Based on the pixel area of ​​the helmet region and other regions, if the ratio of the pixel area of ​​the other regions to the total pixel area of ​​the helmet region is greater than a pre-defined area threshold, it can be determined that no helmet is being worn. In other words, if the pixel area of ​​the segmented helmet region is too small, no helmet is being worn; otherwise, a helmet is being worn. This segmentation of the helmet region and other regions effectively eliminates the limitations of various helmet color classifications, making it more widely applicable and supporting different types and colors of helmets, such as black helmets and helmets with accessories such as ear protectors and face shields.

[0045] Step S104: Determine alarm information based on the target image, human body recognition results, and / or helmet wearing classification recognition results.

[0046] In addition to identifying whether a safety helmet is being worn, this embodiment can also determine the corresponding alarm information based on the identification of the target image in the above steps. That is, in addition to not wearing a safety helmet, the alarm information can also include alarm reminders based on the posture, orientation, light intensity, and clarity of pedestrians (workers) in the target image.

[0047] Specifically, based on the results of the target image recognition in each of the above steps, such as the target image detection score, human body recognition results, target image attribute information, and the pixel area of ​​the helmet area and other areas in the helmet wearing classification recognition results, corresponding alarm information can be determined. For example, based on the target image detection score, if the detection score is lower than the preset detection score, the alarm information could be "Image cannot be detected, please take a new photo"; based on the human body recognition results, the alarm information could be "Backlighting, please take a new photo" or "Image is blurry, please take a new photo"; based on the target image attribute information, the alarm information could be "Please face the camera directly when taking the photo," "Do not bend over to take the photo," or "Do not obstruct the view"; based on the helmet wearing classification recognition results, the alarm information could be "Please wear a helmet," etc. The above are just examples; specific settings depend on the implementation and are not limited here. Alarm information can be recorded and adjusted, improving the interpretability of the recognition results, adapting to multiple scenarios, and enhancing versatility.

[0048] Furthermore, in different scenarios, the thresholds used for filtering or identification in the above steps can be set differently depending on the scenario and the specific implementation situation, so as to be applicable to various scenarios where safety helmets need to be worn.

[0049] The helmet wearing recognition method provided in this embodiment of the invention, through human body recognition, can significantly reduce the false recognition rate caused by factors such as obstruction, blurriness, low light, and backlight when wearing a helmet. Utilizing refined recognition processing, helmet recognition can be improved, supporting the recognition of various helmets. Based on alarm information, the interpretability of the recognition results is enhanced, adapting to various helmet wearing scenarios.

[0050] Figure 4 A schematic diagram of the structure of the helmet wearing recognition device provided in an embodiment of the present invention is shown. Figure 4 As shown, the device includes:

[0051] The pedestrian recognition module 410 is adapted to acquire a video stream, perform pedestrian recognition on each video frame in the video stream, and obtain a target image containing pedestrians.

[0052] The human body recognition module 420 is suitable for performing human body recognition on a target image to obtain the human body recognition result of the target image; human body recognition includes human skeleton key point recognition, human semantic segmentation and / or human image quality;

[0053] The safety helmet recognition module 430 is suitable for classifying and recognizing the wearing of safety helmets in target images based on human body recognition results, and obtaining the corresponding safety helmet wearing classification and recognition results.

[0054] The alarm module 440 is suitable for determining alarm information based on the target image, human body recognition results, and / or safety helmet wearing classification recognition results.

[0055] Optionally, the target image includes pedestrian detection boxes;

[0056] Pedestrian recognition module 410 is further adapted to:

[0057] Acquire the video stream, use a preset pedestrian detection model to identify pedestrians in each video frame of the video stream, determine the target image containing pedestrians, mark the pedestrian detection box in the target image, and determine the detection score of the target image;

[0058] Determine the aspect ratio of the pedestrian detection bounding box in the target image;

[0059] Target images with aspect ratios lower than a preset threshold are subjected to the first filtering process.

[0060] Optionally, the human body recognition module 420 is further adapted to:

[0061] Human recognition is performed on the target image using a pre-defined neural convolutional network, resulting in human recognition results that include human skeleton key point recognition, human semantic segmentation, and / or human image quality information. The pre-defined neural convolutional network includes a neural convolutional network with a single backbone and four branch outputs. The human skeleton key point recognition results include each key point of the human skeleton. The human semantic segmentation results include the head, upper body, and lower body. The human image quality information is determined based on image brightness and / or blur level.

[0062] Optionally, the human body recognition module 420 is further adapted to:

[0063] Based on the human body recognition results obtained from the human body recognition of the target image, the target images whose image brightness and / or blur level in the human body image quality information are lower than the preset quality threshold are subjected to a second filtering process.

[0064] Optionally, the helmet recognition module 430 is further adapted to:

[0065] Based on the human body recognition results, a third filtering process is performed on the target image. The third filtering process includes: filtering based on the distance between the bounding box of the human head segmentation and the pedestrian detection box, filtering based on the human head segmentation contour point set, filtering based on the edge pixels of the human head region, and filtering based on the positional relationship of the key points of the human skeleton.

[0066] The attribute information of the target image after the third filtering process is determined. The attribute information includes orientation attribute, pose attribute and / or occlusion attribute; orientation attribute includes facing away, facing and / or sideways; pose attribute includes looking down, looking up and / or looking straight ahead; occlusion attribute includes occlusion and no occlusion; the attribute information is determined based on the human skeleton key point recognition results.

[0067] The target image after the third filtering process is then subjected to a fourth filtering process based on the attribute information.

[0068] Based on a pre-defined image segmentation convolutional neural network, the target image after the fourth filtering process is used to classify and identify helmet wearing, thereby segmenting the helmet region and determining the corresponding helmet wearing classification and identification result based on the pixel area of ​​the helmet region and other regions.

[0069] Optionally, the alarm module 440 is further adapted to:

[0070] Based on the detection score of the target image, the human body recognition result, the attribute information of the target image, the safety helmet area in the safety helmet wearing classification recognition result, and the pixel area of ​​other areas, the corresponding alarm information is determined.

[0071] The descriptions of the above modules refer to the corresponding descriptions in the method embodiments, and will not be repeated here.

[0072] This invention also provides a non-volatile computer storage medium storing at least one executable instruction that can perform the operation corresponding to the helmet wearing recognition method in any of the above method embodiments.

[0073] This application provides a computer program product, which includes at least one executable instruction or computer program that enables a processor to perform the operation corresponding to the helmet wearing recognition method in any of the above method embodiments.

[0074] Figure 5 The diagram illustrates the structure of a computing device according to an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0075] like Figure 5 As shown, the computing device may include: a processor 502, a communication interface 504, a memory 506, and a communication bus 508.

[0076] in:

[0077] The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508.

[0078] Communication interface 504 is used to communicate with other network elements such as clients or other servers.

[0079] The processor 502 is used to execute program 510, which can specifically execute the relevant steps in the above-described embodiment of the helmet wearing recognition method.

[0080] Specifically, program 510 may include program code that includes computer operation instructions.

[0081] Processor 502 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0082] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0083] Specifically, program 510 can be used to cause processor 502 to execute the helmet wearing recognition method in any of the above method embodiments. The specific implementation of each step in program 510 can be found in the corresponding descriptions of the steps and units in the above helmet wearing recognition embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0084] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the embodiments of the present invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of the present invention.

[0085] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0086] Similarly, it should be understood that, in order to streamline the embodiments of the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0087] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0088] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0089] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The embodiments of the present invention can also be implemented as device or apparatus programs (e.g., computer programs and computer program products) for performing part or all of the methods described herein. Such programs implementing the embodiments of the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0090] It should be noted that the above embodiments are illustrative of the present invention and not restrictive of the invention, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the present invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for identifying helmet wearing, characterized in that, The methods include: Acquire a video stream, perform pedestrian recognition on each video frame in the video stream, and obtain a target image containing pedestrians; Human body recognition is performed on the target image to obtain the human body recognition result of the target image; the human body recognition includes human skeleton key point recognition, human semantic segmentation and / or human image quality; Based on the human body recognition results, the helmet wearing classification and recognition of the target image are performed to obtain the corresponding helmet wearing classification and recognition results. An alarm message is determined based on the target image, the human body recognition result, and / or the helmet wearing classification recognition result.

2. The method according to claim 1, characterized in that, The target image includes pedestrian detection boxes; The step of acquiring the video stream and performing pedestrian recognition on each video frame of the video stream to obtain a target image containing pedestrians further includes: Acquire a video stream, use a preset pedestrian detection model to perform pedestrian recognition on each video frame in the video stream, determine the target image containing pedestrians, mark the pedestrian detection box in the target image, and determine the detection score of the target image; Determine the aspect ratio of the pedestrian detection bounding box in the target image; The target image whose aspect ratio is lower than the preset aspect ratio threshold is subjected to the first filtering process.

3. The method according to claim 1, characterized in that, The step of performing human body recognition on the target image to obtain the human body recognition result of the target image further includes: Human recognition is performed on the target image using a preset neural convolutional network to obtain human recognition results that include human skeleton key point recognition results, human semantic segmentation results, and / or human image quality information; the preset neural convolutional network includes a neural convolutional network with a single backbone and four branch outputs; the human skeleton key point recognition results include each key point of the human skeleton; the human semantic segmentation results include the head, upper body, and lower body; the human image quality information is determined based on image brightness and / or blur level.

4. The method according to claim 3, characterized in that, The step of performing human body recognition on the target image to obtain the human body recognition result of the target image further includes: Based on the human body recognition results obtained by performing human body recognition on the target image, the target images whose image brightness and / or blur level in the human body image quality information are lower than a preset quality threshold are subjected to a second filtering process.

5. The method according to claim 1, characterized in that, The step of classifying and recognizing the helmet wearing status of the target image based on the human body recognition result, and obtaining the corresponding helmet wearing classification and recognition result, further includes: Based on the human body recognition results, a third filtering process is performed on the target image; the third filtering process includes: filtering based on the distance between the bounding box of the human head segmentation and the pedestrian detection box, filtering based on the human head segmentation contour point set, filtering based on the edge pixels of the human head region, and filtering based on the positional relationship of key points of the human skeleton. The attribute information of the target image after the third filtering process is determined. The attribute information includes orientation attribute, posture attribute and / or occlusion attribute; the orientation attribute includes facing away, facing and / or sideways; the posture attribute includes looking down, looking up and / or looking straight ahead; the occlusion attribute includes occlusion and no occlusion; the attribute information is determined based on the human skeleton key point recognition results. The target image after the third filtering process is then subjected to a fourth filtering process based on the attribute information. Based on a preset image segmentation convolutional neural network, the target image after the fourth filtering process is subjected to helmet wearing classification and recognition to segment the helmet region, and the corresponding helmet wearing classification and recognition result is determined based on the pixel area of ​​the helmet region and other regions.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the alarm information based on the target image, the human body recognition result, and / or the helmet wearing classification recognition result further includes: Based on the detection score of the target image, the human body recognition result, the attribute information of the target image, the safety helmet area in the safety helmet wearing classification recognition result, and the pixel area of ​​other areas, the corresponding alarm information is determined.

7. A helmet wearing recognition device, characterized in that, The device includes: A pedestrian recognition module is adapted to acquire a video stream, perform pedestrian recognition on each video frame in the video stream, and obtain a target image containing pedestrians. The human body recognition module is adapted to perform human body recognition on the target image to obtain the human body recognition result of the target image; the human body recognition includes human skeleton key point recognition, human semantic segmentation and / or human image quality; The helmet recognition module is adapted to perform helmet wearing classification and recognition of the target image based on the human body recognition result, and obtain the corresponding helmet wearing classification and recognition result; The alarm module is adapted to determine alarm information based on the target image, the human body recognition result, and / or the helmet wearing classification recognition result.

8. A computing device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the helmet wearing recognition method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The storage medium stores at least one executable instruction, which causes the processor to perform the operation corresponding to the helmet wearing recognition method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes at least one executable instruction that causes the processor to perform the operation corresponding to the helmet wearing recognition method as described in any one of claims 1-6.

Images