Special protective equipment wearing recognition system and method based on multi-dimensional information fusion

Abstract

The application provides a special protective equipment wearing identification system and method based on multi-dimensional information fusion, and relates to the technical field of industrial safety intelligent management and control. It comprises: a front-end image acquisition module that acquires human body images of personnel entering a target scene; a server recognition and analysis module that outputs PPE recognition results, face information and positioning card information based on the human body images; a PPE compliance recognition module that obtains compliance recognition results by comparing the PPE recognition results with preset rules; a face recognition module that obtains face recognition results by comparing the face information with a preset personnel information library; a positioning card information comparison module that obtains positioning card verification results by comparing the positioning card information with a preset binding relationship; and a server recognition and analysis module that fuses and judges the aforementioned three results and generates access control instructions. That is, multi-dimensional information fusion is used to realize intelligent management and control of personnel PPE wearing compliance, identity authority and positioning card status in special scenarios.

Description

Technical Field

[0001] This application relates to the field of intelligent industrial safety management and control technology, and in particular to a special protective equipment wearing recognition system and method based on multi-dimensional information fusion. Background Technology

[0002] PPE (Personal Protective Equipment) refers to protective equipment worn, equipped, and used by workers to protect themselves from physical, chemical, and biological hazards. Currently, a common approach combines fixed video cameras with AI technology to assess PPE wearing status. However, this method suffers from issues such as low overall recognition rate, limited specificity, and a high number of false alarms. Particularly in high-risk operation scenarios involving acids, alkalis, and chlorine, workers face serious threats such as chemical corrosion and toxic gas inhalation. The protective equipment they require differs significantly from standard PPE, and incomplete or improper use can easily lead to serious personal injury accidents. Therefore, accurate identification and effective management of the compliance of special PPE wearing is crucial for ensuring the safety of high-risk workers. However, current technology lacks the ability to effectively identify and assess special PPE wearing, resulting in ineffective management of PPE wearing in these special scenarios.

[0003] Specifically, the existing technologies of the same type generally have the following problems: (1) insufficient identification targeting, unable to effectively identify special PPE required for high-risk operations such as acid and alkali, chlorine-related operations; (2) low identification rate, which is greatly affected by on-site environmental factors such as light and obstruction; (3) high false alarm rate, where non-compliant situations are often misjudged as compliant or compliant situations are often misreported as non-compliant, which seriously affects the actual application effect of the system and the willingness of management personnel to respond.

[0004] Based on the aforementioned shortcomings, a novel technical solution is urgently needed to address the problem that existing technologies fail to effectively identify the compliance of special PPE wear in high-risk operational scenarios. Summary of the Invention

[0005] In view of this, this application provides a special protective equipment wearing identification system and method based on multi-dimensional information fusion to solve the problem that the prior art fails to effectively identify the need for correct wearing of PPE in high-risk operations.

[0006] Firstly, a special protective equipment wearing recognition system based on multi-dimensional information fusion is provided, including: The front-end image acquisition module is set at the entrance of the target scene or the area to be controlled, and is used to acquire human images of people entering the scene. The server identification and analysis module is used to receive the human body image and the associated information corresponding to the human body image. The associated information includes at least the acquisition time, device number, and access control point number. The server identification and analysis module is also used to identify the human body image, output the PPE identification result, extract the facial features of the person in the human body image to identify the face information, and obtain the positioning card information corresponding to the person entering the body. The PPE compliance identification module is connected to the server identification and analysis module and is used to compare the PPE identification result with the preset PPE compliance rules to generate the PPE compliance identification result. A face recognition module, connected to the server recognition and analysis module, is used to identify the identity of the person entering based on the face information, compare it with a preset personnel information database, and generate a face recognition result. The location card information comparison module is connected to the server identification and analysis module and is used to compare the location card information with the preset personnel location card binding relationship to generate a location card verification result. The server identification and analysis module performs a fusion judgment based on the PPE compliance identification result, the face recognition result, and the positioning card verification result, and generates an access control command to drive the access control module to open or remain closed.

[0007] In one possible implementation, the server identification and analysis module includes: A fast front-end recognition unit is used to identify the human body region of the target person in the human body image; wherein the human body region is divided into a head sub-region, a hand sub-region, and a foot sub-region; the fast front-end recognition unit is also used to output the bounding box and PPE candidate region of the target person, wherein the PPE candidate region is derived from the head sub-region, the hand sub-region, and the foot sub-region. The fine recognition unit is used to perform secondary recognition on the PPE candidate region and output the PPE recognition result.

[0008] In one possible implementation, the fast front-end recognition unit uses the YOLOv8 target detection model to detect the human body image; The fine recognition unit uses a preset visual recognition model for secondary recognition to determine whether one or more of the following items exist, whether they are worn correctly, and whether they meet the requirements of the target scenario: safety helmet, goggles, protective face shield, gas mask, face mask, protective gloves, protective clothing, work shoes, protective boots, soundproof earmuffs, gas detector, high-pressure water cleaning protective clothing, shin guards, acid and alkali resistant clothing, chlorine-related work protective equipment, contractor vest or guardian vest.

[0009] In one possible implementation, a management configuration module is also included, which is used to store PPE compliance rules, personnel authorization rules, and personnel location card binding relationships corresponding to different work scenarios. The server identification and analysis module calls the PPE compliance rules, personnel authorization rules, and personnel location card binding relationships corresponding to the target scenario, based on the target scenario.

[0010] In one possible implementation, the PPE compliance identification module compares the PPE identification result with the PPE compliance rules corresponding to the target scenario stored in the management configuration module to generate a PPE compliance identification result; wherein, the PPE compliance identification result includes at least the overall compliance status, a list of missing PPEs, and a list of abnormal PPEs.

[0011] In one possible implementation, the positioning card information comparison module compares the positioning card information with the personnel positioning card binding relationship stored in the management configuration module corresponding to the target scene to determine whether the entering personnel are carrying their own positioning card and whether the location information of the positioning card matches the human body image acquisition location. When the identity of the person entering the location is identified as matching the person bound to the location card, and the location card is within a preset allowable range, the location card verification is deemed successful; otherwise, the location card verification is deemed unsuccessful.

[0012] In one possible implementation, the front-end image acquisition module is installed at a fixed position in front of, to the side of, or in the area to be controlled in the access control channel of the target scene, and is set towards the direction of personnel passage so that the acquisition line of sight covers the head, torso, hands and feet of the personnel entering. The target scenario refers to any one of the following: cabinet rooms, substations, equipment plant entrances / exits, and open area entrances / exits in areas such as the chlor-alkali plant, polyvinyl chloride plant, acetylene plant, and public works center.

[0013] In one possible implementation, a voice prompt module is further included. This voice prompt module is connected to the server recognition and analysis module and is used to broadcast corresponding voice prompt information based on the result of the fusion judgment, including: When the PPE compliance identification result has missing or abnormal items, the corresponding missing or abnormal item prompt will be broadcast. When the facial recognition result fails, an abnormal identity or permission message will be displayed. When the location card verification result fails, a location card error message is broadcast. When all three checks pass, a message indicating that passage is permitted will be broadcast.

[0014] In one possible implementation, a log recording module is also included. This log recording module is connected to the server identification and analysis module and is used to record access events or abnormal events of the personnel entering the premises. The log recording content of the log recording module includes at least the following fields: The data includes: identification time, data collection time, access control point number, device number, personnel identity information, location card number, image storage path, PPE identification result, list of missing and abnormal items in the PPE compliance identification result, face recognition result, location card verification result, access control command, access control feedback status, and voice prompt content.

[0015] Secondly, a method for identifying the wearing of special protective equipment based on multi-dimensional information fusion is provided, which is applied to any of the systems described above, including: Receive human images of people entering the area and associated information corresponding to the human images sent by the front-end image acquisition module. The associated information includes at least the acquisition time, device number and access control point number. The human image is identified to obtain a PPE identification result. The PPE compliance identification module is then invoked so that the PPE compliance identification module compares the PPE identification result with preset PPE compliance rules. The PPE compliance identification result returned by the PPE compliance identification module is then received. Extract the facial features of the person in the human image, call the face recognition module, so that the face recognition module can identify the identity of the person entering based on the facial features and compare it with the preset personnel information database, and receive the face recognition result returned by the face recognition module; Obtain the location card information corresponding to the person entering, call the location card information comparison module so that the location card information comparison module obtains the location card information corresponding to the person entering and compares the location card information with the preset personnel location card binding relationship, and receive the location card verification result returned by the location card information comparison module; The PPE compliance identification result, the face recognition result, and the location card verification result are fused and judged. When the PPE compliance identification result, the face recognition result, and the location card verification result all meet the preset conditions, an access permission command is generated and sent to the access control module to drive the access control device to open; when any one of the preset conditions is not met, an access prohibition command is generated and sent to the access control module to drive the access control device to remain closed.

[0016] Through the above technical solutions, the special protective equipment (PPE) wearing recognition system based on multi-dimensional information fusion provided in this application can realize PPE compliance determination, personnel identification, location card carrying status verification, and access control linkage control for personnel entering the target scene, forming a closed-loop management process from image acquisition, network transmission, server recognition, rule comparison, result return, access control, to voice prompts. Simultaneously, the system of this application can configure differentiated PPE recognition rules for different work scenarios, improving the targeting of special PPE recognition, reducing the risk of false alarms and missed alarms caused by a single video recognition method, and can accurately identify the basic information and location information of personnel entering the target scene, effectively managing the risks of on-site personnel operations.

[0017] The technical solutions provided in this application can achieve the following technical effects: (1) To address the shortcomings of traditional single visual recognition, such as low accuracy and lack of special protection recognition, the system provided in this application adopts a dual-model architecture of YOLOv8 fast front recognition + scene fine-tuning recognition to achieve hierarchical accurate detection.

[0018] (2) To address the shortcomings of traditional control systems, such as poor adaptability to different scenarios and the inability to configure rules in a differentiated manner, the system is equipped with an independent management and configuration module. It supports the custom storage of PPE compliance verification rules by scenario, access control point, and work area. It can be configured with specific control standards for chlorine-related operations, acid and alkali operations, high-pressure water cleaning, and insulation and anti-static measures between cabinets. By automatically matching the corresponding verification rules through the point number, it can achieve precise and differentiated control of various high-risk scenarios throughout the plant. There is no need to manually modify the program code. The rule configuration is automatically updated and switched, solving the technical defects of traditional systems such as fixed rules and insufficient scenario adaptability.

[0019] (3) To solve the shortcomings of traditional access verification with a single dimension and lack of personnel qualification verification, the system integrates a face recognition module and connects to a complete personnel information database. It can simultaneously identify personnel name, employee number, affiliated unit, position, authorized area, training qualification, work permission and other full-dimensional information. The accuracy of personnel identification is high. It can accurately intercept personnel without qualifications, without regional authorization, or with unqualified training, and realize multi-dimensional joint verification of PPE wearing compliance, personnel identity legality and work permission validity.

[0020] (4) To address the shortcomings of traditional management and control systems, such as the lack of closed-loop feedback, violation tracing, and prompting mechanisms, a fully automated closed-loop management and control process is constructed, consisting of "image acquisition - AI recognition - rule comparison - result judgment - voice prompt - access control linkage". The voice module can output corresponding accurate voice prompts for various violation scenarios, such as missing PPE, failed identity recognition, no positioning card, and insufficient permissions, with high accuracy in indicating the cause of violation. The access control module can link with turnstiles, electric gates, and the existing access control system in the factory area to achieve automatic release for compliance and immediate locking for violations. The entire process requires no manual intervention, realizing a closed-loop management and control system with intelligent interception before the event, accurate prompts during the event, and data traceability after the event, completely solving the shortcomings of traditional management and control processes, such as fragmentation and lack of real-time intervention capabilities. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0022] Figure 1 This is a structural diagram of a special protective equipment wearing recognition system based on multi-dimensional information fusion provided in an embodiment of this application.

[0023] Figure 2 This is a structural example diagram of the server identification and analysis module provided in the system embodiment of this application.

[0024] Figure 3 This is a diagram of the dual-model architecture for recognizing human images provided in the system embodiments of this application, which is a server recognition and analysis module.

[0025] Figure 4 This is a flowchart of a special protective equipment wearing recognition method based on multi-dimensional information fusion provided in an embodiment of this application.

[0026] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0029] To achieve joint verification and automated access control of personnel identity, work permissions, compliance of special protective equipment wearing, and location card carrying status in high-risk areas such as server rooms, substations, plant entrances and exits, and open areas in chlor-alkali plants, polyvinyl chloride plants, acetylene plants, and public works centers, this application provides a special protective equipment wearing identification system based on multi-dimensional information fusion. For example... Figure 1 As shown, the system includes a front-end image acquisition module, a network transmission module, a server identification and analysis module, a management and configuration module, a PPE compliance identification module, a face recognition module, a location card information comparison module, an access control module, a voice prompt module, and a log recording module. The following provides a detailed description of each module.

[0030] 1. Front-end image acquisition module The front-end image acquisition module is positioned at the entrance of the target scene or in a controlled area, specifically at a fixed location directly in front of, to the side of, or in the controlled area of ​​the target scene, facing the direction of personnel passage. This ensures that the field of view covers the head, torso, hands, and feet of the personnel entering, facilitating the acquisition of human images. Specifically, the front-end image acquisition module includes a fixed camera, an industrial camera, or a mobile acquisition terminal. When using a fixed camera, the camera is installed directly in front of or to the side of the access control channel, facing the direction of personnel passage, to cover the head, torso, hands, feet, and any PPE worn or carried by the personnel. When using a mobile acquisition terminal, the terminal acquires human images by taking photos or selecting images, and performs image compression, orientation correction, and historical copy saving locally.

[0031] The front-end image acquisition module also collects associated information corresponding to the human body image, and packages this associated information together with the human body image to send to the server recognition and analysis module. The associated information includes at least the acquisition time, device number, and access control point number. The acquisition time refers to the system timestamp when the front-end image acquisition module acquires the current human body image, used to identify the moment the acquisition occurred, facilitating subsequent time-based tracing and statistical analysis of the recognition event. The device number is a unique identifier for the front-end image acquisition module, used to distinguish multiple acquisition devices deployed in the system, facilitating the system's identification of the image source and association with the corresponding device's operating status. The access control point number is a unique identifier for the access control point corresponding to the installation location of the front-end image acquisition module, used to identify the specific physical location where the current acquisition occurred, enabling the server recognition and analysis module to identify the current scene information and invoke the corresponding control rules based on this number.

[0032] The target scenarios mentioned above are any one of the following: cabinet rooms, substations, equipment plant entrances and exits, and open area entrances and exits in areas such as the chlor-alkali plant, polyvinyl chloride plant, acetylene plant, and public works center.

[0033] 2. Network transmission module The network transmission module connects to the front-end image acquisition module and the server recognition and analysis module, and is used to transmit the human body images and corresponding associated information acquired by the front-end image acquisition module to the server recognition and analysis module. Specifically, the network transmission module uses a wired network, a wireless network, or a dedicated network within the factory for data transmission. When using a wired network, stable and high-speed transmission of image data and associated information is achieved through industrial Ethernet or fiber optic networks; when using a wireless network, data interaction between the front-end device and the server is achieved through wireless communication methods such as Wi-Fi, 4G, or 5G; when using a dedicated network within the factory, data transmission is achieved through a private enterprise network or virtual private network to ensure communication security. Of course, the network transmission module can also transmit the image resource address of the human body image to the server recognition and analysis module. The image resource address refers to the storage path identifier of the human body image in the front-end device's local storage, the factory's internal storage system, or a dedicated storage server, enabling the server recognition and analysis module to retrieve the corresponding human body image from the specified storage location based on this address. This is suitable for deployment scenarios where network bandwidth is limited or centralized management of image data is required.

[0034] 3. Server Identification and Analysis Module The server identification and analysis module is used to receive human images and related information sent by the front-end image acquisition module through the network transmission module. It is also used to identify the human images and output PPE identification results, extract facial features of people in the human images to identify face information, and obtain the location card information corresponding to the people entering.

[0035] like Figure 2 and Figure 3 As shown, specifically, the server recognition and analysis module includes an image receiving and processing unit, an image preprocessing unit, a fast pre-recognition unit, a fine recognition unit, a feature extraction unit, a data retrieval unit, a result standardization unit, and a rule comparison unit. Each unit is described in detail below.

[0036] The image receiving and processing unit receives human images and corresponding associated information sent by the front-end image acquisition module via the network transmission module. Upon receiving the data, it verifies data integrity, confirming that the human image can be decoded normally and that the associated information fields are complete. Then, it transmits the human image and associated information to the image preprocessing unit. The image preprocessing unit, connected to the image receiving and processing unit, performs preprocessing operations on the received human image, including size normalization, sharpness verification, orientation correction, and image region organization. The preprocessed human image is then input into the fast pre-recognition unit and the feature extraction unit, respectively.

[0037] The rapid pre-recognition unit uses the YOLOv8 target detection model to quickly detect pre-processed human images, identify target individuals within the images, and then recognize the human body region based on these target individuals. The target individual refers to the person identified as the subject of recognition in the human image. When the human image contains only a single person, that person is identified as the target individual. When the human image contains multiple people, the target individual can be selected from among them based on a preset strategy. This preset strategy includes, but is not limited to, identifying the person occupying the largest area in the human image as the target individual, identifying the person located in the center of the image as the target individual, or identifying the person who enters the preset recognition area first. The human body region can be further divided into head, hand, and foot sub-regions. The human body image is divided into three sub-regions: a head sub-region (head area) and a foot sub-region (foot area). The head sub-region is the local image area corresponding to the head of a person, used to detect head-related PPE such as safety helmets, goggles, face shields, gas masks, face masks, and earmuffs. The hand sub-region is the local image area corresponding to the hands of a person, used to detect hand-related PPE such as protective gloves. The foot sub-region is the local image area corresponding to the feet of a person, used to detect foot-related PPE such as work shoes and protective boots. The fast pre-recognition unit identifies the PPE location in each sub-region, marks the PPE location as a candidate PPE location, and outputs the bounding box of the target person and the candidate PPE region. The bounding box of the target person refers to the coordinate information of the entire area of ​​the target person marked as a rectangle in the human body image. The candidate PPE region refers to the local area determined to contain the candidate PPE location.

[0038] By using the initial detection of the rapid front-end recognition unit, the amount of data processing required for subsequent fine recognition by the fine recognition unit can be reduced, thereby improving the overall recognition efficiency.

[0039] The fine-grained identification unit connects with the rapid pre-identification unit to perform secondary identification of PPE candidate areas and output PPE identification results. The fine-grained identification unit uses a preset visual recognition model for secondary identification. This model is trained or fine-tuned based on PPE-annotated images specific to the enterprise's scenario. It is used to determine the presence, correct wearing, and compliance with the requirements of the current scenario for one or more of the following: safety helmets, goggles, face shields, gas masks, face masks, protective gloves, protective clothing, work shoes, protective boots, earmuffs, gas detectors, high-pressure water cleaning protective clothing, shin guards, acid and alkali resistant clothing, chlorine-related work protective equipment, contractor vests, or guardian vests. Specifically, the PPE identification result refers to the structured data output by the fine-grained identification unit after identifying the PPE candidate areas, including the name, existence status, compliance status, bounding box, confidence level, and explanation for each PPE item. The existence status refers to the judgment result that the PPE item is detected as "present" or "absent" in the image; the compliance status refers to the fine-grained judgment result that the PPE item is judged as "correctly worn", "incorrectly worn", "missing" or "cannot be confirmed"; the confidence level refers to the model's confidence probability value of the judgment result, which ranges from 0 to 1, and the higher the value, the greater the model's confidence in the judgment result; the explanation of the reason refers to the textual explanation of the judgment result.

[0040] The feature extraction unit uses a face detection algorithm to locate the facial region of the target person in a human image, and then extracts facial biometric data from this region for unique identification or comparison of the person's identity; this data is also known as the target person's facial information. The data retrieval unit, based on the target person's facial information, queries the personnel location card binding relationship stored in the management configuration module for the location card number bound to that person to determine the target person's current location, i.e., obtain the location card information.

[0041] The result standardization unit is connected to the fine recognition unit, feature extraction unit, and data retrieval unit respectively. It is used to receive PPE recognition results, face information, and location card information, and output the received data as unified standard structured data. Then, the standardized PPE recognition results, face information, and location card information are sequentially output to the PPE compliance recognition module, face recognition module, and location card information comparison module.

[0042] The rule comparison unit is connected to the PPE compliance identification module, the face recognition module, and the location card information comparison module, respectively. It is used to receive the PPE compliance identification result returned by the PPE compliance identification module based on the PPE identification result, the face recognition result returned by the face recognition module based on the face information, and the location card verification result returned by the location card information comparison module based on the location card information. It then performs a fusion judgment on the PPE compliance identification result, the face recognition result, and the location card verification result to generate an access control command to drive the access control module to open or remain closed. In one possible implementation, the fusion judgment logic is as follows: First, it checks whether the face recognition result is "not recognized." If it is, it is directly determined that the preset conditions are not met. Second, it checks whether the permission judgment result in the face recognition result is "permitted." If it is "no permission," it is determined that the preset conditions are not met. Third, it checks whether the overall compliance status in the PPE compliance recognition result is "compliant." If it is "non-compliant," it is determined that the preset conditions are not met. Finally, it checks whether the location card verification result is "verification passed." If it is "verification failed," it is determined that the preset conditions are not met. When all four judgments are positive, i.e., face recognition is successful, the person has the current area permission, the PPE is generally compliant, and the location card verification is passed, it is determined that all preset conditions are met. At this time, an access permission command is generated and sent to the access control module to drive the access control device to open. At the same time, the fusion judgment result and the access permission command are sent to the voice prompt module and the log recording module. If any of the preset conditions are not met, a prohibition command is generated and sent to the access control module to drive the access control device to remain closed. At the same time, the fusion judgment result, the prohibition command and the reason for non-compliance are sent to the voice prompt module and the log recording module.

[0043] The aforementioned reasons for non-compliance include, but are not limited to, specific information such as "PPE missing: protective gloves, protective face shield", "PPE abnormality: improper wearing of goggles", "identity not recognized", "no area access", "no location card detected", "location card does not match personnel identity", and "location card location mismatch". The voice prompt module will play the corresponding voice prompt content according to the reason for non-compliance, and the log recording module will record all data of this recognition event.

[0044] 4. Management Configuration Module The management configuration module stores PPE compliance rules, personnel authorization rules, and personnel location card binding relationships for different work scenarios. Specifically, it stores PPE compliance rules, personnel authorization rules, location card binding relationships, access control policies, voice prompts, and log fields for different factory areas, workshops, and access control points. Personnel authorization rules refer to a pre-defined set of permission rules that determine whether different personnel are authorized to enter in different target scenarios. These rules include at least personnel identity information, authorized area scope, job qualification requirements, and training completion status. Personnel location card binding relationships are pre-defined one-to-one correspondences between personnel identity information and location card numbers, used to verify whether entering personnel are carrying their own location cards.

[0045] The management configuration module allows administrators to configure differentiated rules for access control points in different scenarios based on varying HSE control requirements. The server recognition and analysis module identifies the current target scenario type based on the access control point number uploaded by the front-end image acquisition module and automatically invokes the corresponding PPE compliance rules, personnel authorization rules, and personnel location card binding relationships, without requiring manual modification of the program code. The management configuration module also supports adding, modifying, deleting, and versioning rules; the rule configuration, update, and switching response time is less than 1 second, adapting to all special operations and high-risk operation access control scenarios.

[0046] 5. PPE Compliance Identification Module The PPE compliance identification module is connected to the server identification and analysis module, specifically to the result standardization unit and the rule comparison unit. The PPE compliance identification module receives the standardized PPE identification results and the PPE compliance rules corresponding to the target scenario from the result standardization unit, compares the PPE identification results with the PPE compliance rules item by item, generates the PPE compliance identification results, and then outputs them to the rule comparison unit.

[0047] Specifically, after receiving the PPE identification results and the corresponding PPE compliance rules for the target scenario, the PPE compliance identification module parses both. The PPE compliance rules refer to a pre-defined set of rules for the PPE items that personnel must correctly wear when entering the target scenario, along with their wearing requirements. These rules are stored in the management configuration module in a structured data format. The PPE compliance identification module compares each PPE item in the PPE identification results with the list of required PPE items for the target scenario item by item. The comparison logic is as follows: For each required PPE item in the rule list, first, check the existence status of the PPE item in the identification results. If the existence status is "not present," the item is considered missing and recorded in the missing PPE list. If the existence status is "present," then further check the compliance status of the item. If the compliance status is "correctly worn," the item is considered compliant. If the compliance status is "incorrectly worn" or "cannot be confirmed," the item is considered abnormal and recorded in the abnormal PPE list. When all required PPE items in the rule list are deemed compliant, the PPE compliance identification module determines the overall compliance status as "compliant." When any required PPE item in the rule list is deemed missing or abnormal, the overall compliance status is determined as "non-compliant." The PPE compliance identification module encapsulates the above comparison results into a PPE compliance identification result and outputs it to the rule comparison unit, which then fuses and judges it with the facial recognition result and the location card verification result.

[0048] 6. Face recognition module The face recognition module is connected to the server recognition and analysis module, specifically to the result standardization unit and the rule comparison unit. The face recognition module receives standardized facial information output by the result standardization unit and personnel authorization rules corresponding to the target scene. It identifies the identity of the person entering based on the facial information and compares the identified identity information with the personnel authorization rules, generating a face recognition result which is then output to the rule comparison unit. Specifically, the face recognition module first compares the received facial feature vector with pre-stored facial feature templates for each person in a preset personnel information database, selecting the person with the highest similarity exceeding a preset threshold as the identity recognition result. If the comparison is successful, it further determines whether the person has permission to enter the current target scene based on the personnel authorization rules' authorization area, training qualifications, and work permissions. If the comparison fails, the identity recognition result is "not recognized," and the permission judgment result is "no permission." The face recognition module encapsulates the comparison result into a face recognition result and outputs it to the rule comparison unit.

[0049] 7. Locator Card Information Comparison Module The location card information comparison module is connected to the server identification and analysis module, specifically to the result standardization unit and the rule comparison unit. The location card information comparison module receives the standardized location card information output by the result standardization unit. This location card information is the current location data of the target person, obtained by querying the binding relationship from the management configuration module based on the target person's identity. The location card information comparison module also receives the location information of the current access control point and compares the target person's current location with the current access control point's location to determine if the target person is within the preset allowable range of the current access control point. After generating the location card verification result, it outputs it to the rule comparison unit. Specifically, the location card information comparison module calculates the distance between the received target person's current location and the current access control point location. If the distance difference is within the preset allowable range, the location card verification is considered successful; if it exceeds the preset allowable range, the location card verification is considered unsuccessful, with the reason being "location card location does not match access control point location". Furthermore, if the data retrieval unit fails to find the location card number bound to the target personnel from the management configuration module, i.e., it cannot obtain the target personnel's current location data, the location card information comparison module determines that the location card verification fails, with the reason being "no location card detected or the location card is not bound". The location card information comparison module encapsulates the comparison result into a location card verification result and outputs it to the rule comparison unit.

[0050] 8. Access Control Module The access control module is connected to the server identification and analysis module, specifically to the rule comparison unit. The access control module receives access control commands generated by the rule comparison unit and controls the access control device to open or remain closed based on these commands.

[0051] Specifically, when the access control module receives a permission instruction, it sends an open signal to the connected electric door, turnstile, or existing access control system in the factory area, driving the access control device to open and allowing personnel to pass. When it receives a prohibition instruction, it does not send an open signal or sends a lock signal, driving the access control device to remain closed and preventing personnel from passing. The access control module is also used to feed back the actual status of the access control device (including open, closed, not closed after timeout, or forcibly opened) to the log recording module. In addition, the access control module is also used to trigger local or background alarms when multiple identifications fail, the access control status is abnormal, or the device is offline.

[0052] 9. Voice Prompt Module The voice prompt module is connected to the server recognition and analysis module, specifically to the rule comparison unit. The voice prompt module receives the fusion judgment results and reasons for non-compliance generated by the rule comparison unit, and broadcasts the corresponding voice prompt information based on the fusion judgment results.

[0053] Specifically, when the fusion judgment result indicates that all verifications have passed, the voice prompt module announces a permission prompt, such as "Recognition passed, please proceed." When the fusion judgment result indicates that any item has failed, the voice prompt module announces the corresponding prompt based on the reason for non-compliance: If the reason for non-compliance is a missing PPE item, it announces the corresponding missing item prompt, such as "Please wear protective gloves" or "Please wear a protective face shield." If the reason for non-compliance is an abnormal PPE item, it announces the corresponding abnormal item prompt, such as "Goggles not worn correctly, please wear them correctly" or "Face shield not fully lowered, please adjust." If the reason for non-compliance is a failed face recognition, it announces "Personnel identity not recognized, please align with the camera again or contact management personnel." If the reason for non-compliance is insufficient permissions, it announces "This person does not have permission to enter the current area, please contact management personnel." If the reason for non-compliance is a failed location card verification, it announces "Location card not detected, please bring your own location card before entering" or "Location card position abnormal, please confirm whether you have your location card with you."

[0054] 10. Log Tracking Module The log recording module connects to the server identification and analysis module, specifically to the rule comparison unit. The log recording module receives the fusion judgment results, access control commands, and reasons for non-compliance generated by the rule comparison unit, and generates structured log records for storage. Specifically, the log recording module's records include at least the following fields: recognition time, acquisition time, access control point number, device number, personnel identification information, location card number, image storage path, PPE recognition result, missing PPE list, abnormal PPE list, face recognition result, location card verification result, access control command, access control feedback status, and voice prompt content. The log recording module also marks and stores abnormal records when abnormal events occur, and pushes them to the management backend for manual review and rule optimization by administrators when necessary.

[0055] The above provides a detailed description of each major module of this application. In addition, this application may also include a front-end display module and a data storage module. The front-end display module is connected to the server recognition and analysis module and is used to receive standardized structured data output by the result standardization unit. It overlays and displays personnel bounding boxes and bounding boxes of each PPE item on the acquired image, using different colors to distinguish normal, missing, abnormal, and unconfirmed states, facilitating real-time viewing of recognition results by management personnel. The data storage module is located at the front end and is connected to the server recognition and analysis module and the log recording module. It is used to receive and classify the standardized structured data and log records generated by the log recording module. The log records include at least the recognition time, acquisition time, access control point number, device number, personnel identity information, image storage path, missing item list, abnormal item list, and access control instructions. Management personnel can retrieve, view, and export historical records through the data storage module, trace the reasons for non-compliance, and optimize scene rules or supplement training samples based on the review results, forming a complete data closed loop from recognition and display to storage and optimization.

[0056] Based on the above embodiments, which provide a special protective equipment wearing recognition system based on multi-dimensional information fusion, this application provides a special protective equipment wearing recognition method based on multi-dimensional information fusion, such as... Figure 4 As shown, the method uses the server identification and analysis module as the main execution body and includes steps S101 to S105.

[0057] Step S101: Receive the human body image of the person entering the room and the associated information corresponding to the human body image sent by the front-end image acquisition module. The associated information includes at least the acquisition time, device number and access control point number.

[0058] Upon arrival at the entrance to the target scene or the area to be controlled, the front-end image acquisition module captures the person's image in real time. This image, along with associated information such as the acquisition time, device number, and access control point number, is then transmitted to the server recognition and analysis module via a wired network, wireless network, or a dedicated network within the factory. The image receiving and processing unit within the server recognition and analysis module receives this data. The image preprocessing unit performs preprocessing operations such as size normalization, sharpness verification, orientation correction, and preparation of the area to be detected to ensure that the input image meets the quality requirements of the subsequent recognition model.

[0059] Step S102: Recognize the human body image to obtain PPE recognition result, call the PPE compliance recognition module so that the PPE compliance recognition module compares the PPE recognition result with the preset PPE compliance rules, and receive the PPE compliance recognition result returned by the PPE compliance recognition module.

[0060] The rapid pre-recognition unit uses the YOLOv8 target detection model to quickly detect human images, identify the target person's body region, and divide it into head, hand, and foot sub-regions. PPE candidate locations are marked in each sub-region, and the PPE candidate regions are output. Then, the fine recognition unit uses a visual recognition model to perform secondary recognition on each PPE candidate region, determining the presence, correct wearing, and compliance with current scenario requirements of each PPE item, outputting the PPE recognition result. The server recognition and analysis module sends the PPE recognition result to the PPE compliance recognition module. The PPE compliance recognition module compares each PPE item in the PPE recognition result with the corresponding PPE compliance rules for the target scenario, determining whether any required PPEs are missing or abnormal, generating an overall compliance status, a list of missing PPEs, and a list of abnormal PPEs, and returning the PPE compliance recognition result to the rule comparison unit.

[0061] Step S103: Extract the facial features of the person in the human image, call the face recognition module so that the face recognition module can identify the identity of the person entering based on the facial features and compare it with the preset personnel information database, and receive the face recognition result returned by the face recognition module.

[0062] The feature extraction unit extracts facial features from the preprocessed human image and generates facial information accordingly, which is then sent to the face recognition module. The face recognition module compares the facial feature vectors in the facial information with pre-stored facial feature templates for each person in a preset personnel database, selecting the person with the highest similarity exceeding a preset threshold as the identity verification result. If the comparison is successful, the module further determines whether the person has permission to enter the target scene based on personnel authorization rules; if the comparison fails, the identity verification result is "unrecognized," and the permission judgment result is "no permission." The face recognition module returns the face recognition result, including the identity verification result and the permission judgment result, to the rule comparison unit.

[0063] Step S104: Obtain the location card information corresponding to the person entering the area, call the location card information comparison module so that the location card information comparison module obtains the location card information corresponding to the person entering the area and compares the location card information with the preset personnel location card binding relationship, and receive the location card verification result returned by the location card information comparison module.

[0064] The data retrieval unit, based on the target person's identity information identified by the face recognition module, queries the personnel location card binding relationship stored in the management configuration module for the location card number bound to that person, obtains the real-time location signal data of that location card, generates location card information, and then sends the location card information comparison module. The location card information comparison module compares the location card number in the received location card information with the location card number that the person should be bound to in the personnel location card binding relationship, and determines whether the real-time location of the location card matches the current access control point. If the location card matches the person's identity and the location of the location card matches, the location card verification is considered successful; otherwise, the verification is considered unsuccessful, and the reason for the failure is recorded. The location card information comparison module returns the location card verification result to the rule comparison unit.

[0065] Step S105: The PPE compliance identification result, the face recognition result, and the location card verification result are fused and judged; when the PPE compliance identification result, the face recognition result, and the location card verification result all meet the preset conditions, an access permission command is generated and sent to the access control module to drive the access control device to open; when any one of the preset conditions is not met, an access prohibition command is generated and sent to the access control module to drive the access control device to remain closed.

[0066] After receiving the PPE compliance identification result, facial recognition result, and location card verification result, the rule comparison unit performs logical judgment on the three results: When the overall compliance status is "compliant," facial recognition identity recognition is successful and the permission judgment is "permitted," and the location card verification is "passed," all preset conditions are met, a permission instruction is generated and sent to the access control module, and the access control device driver unit drives the electric control door or turnstile to open for passage. When any of the above preset conditions are not met, a prohibition instruction is generated and sent to the access control module, the access control device remains closed, and the reason for non-compliance is output to the voice prompt module and the log recording module. The voice prompt module broadcasts the corresponding prompt content according to the reason for non-compliance (such as missing PPE item, identity not recognized, or location card abnormality, etc.), and the log recording module records the complete data of this identification event for subsequent traceability and rule optimization.

[0067] This completes a full closed-loop management process, from image acquisition, recognition and analysis, multi-dimensional verification, fusion judgment to access control linkage and voice feedback, truly achieving "seamless" risk control closed-loop management.

[0068] It should be noted that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In practical applications, all the above possible implementation methods can be arbitrarily combined in a combined manner to form possible embodiments of this application, which will not be described in detail here.

[0069] Based on the same inventive concept, this application also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the special protective equipment wearing recognition method based on multi-dimensional information fusion of any of the above embodiments.

[0070] In an exemplary embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The illustrated electronic device includes a processor and a memory. The processor and memory are connected, for example, via a bus. Optionally, the electronic device may also include a transceiver, which integrates software programs or hardware such as applications, application programming interfaces (APIs), middleware, and kernels. It should be noted that in practical applications, the transceiver is not limited to one unit, and the structure of this electronic device does not constitute a limitation on the embodiments of this application.

[0071] The processor can be a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0072] A bus can include a pathway for transmitting information between the aforementioned components or connecting these devices to hardware such as input / output interfaces, display devices, and communication interfaces. The bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0073] The memory can be ROM (Read Only Memory), which is another type of static storage device that stores static information and instructions; RAM (Random Access Memory), which is another type of dynamic storage device that stores information and instructions; it can also be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0074] The memory stores computer program code that executes the scheme of this application, and its execution is controlled by a processor. The processor executes the computer program code stored in the memory to implement the content shown in the foregoing method embodiments.

[0075] Among them, electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0076] Based on the same inventive concept, this application also provides a storage medium storing a computer program, wherein the computer program is configured to execute the special protective equipment wearing recognition method based on multi-dimensional information fusion of any of the above embodiments when running.

[0077] Those skilled in the art will clearly understand that the specific working process of the systems, devices, and modules described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0078] Those skilled in the art will understand that the technical solution of this application, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several program instructions to cause an electronic device (such as a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of this application when running the program instructions. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0079] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as electronic devices like personal computers, servers, or network devices) associated with program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of this application.

[0080] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of this application, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to leave the protection scope of this application.

Claims

1. A special protective equipment wear recognition system based on multi-dimensional information fusion, characterized in that, include: The front-end image acquisition module is set at the entrance of the target scene or the area to be controlled, and is used to acquire human images of people entering the scene. The server identification and analysis module is used to receive the human body image and the associated information corresponding to the human body image. The associated information includes at least the acquisition time, device number, and access control point number. The server identification and analysis module is also used to identify the human body image, output the PPE identification result, extract the facial features of the person in the human body image to identify the face information, and obtain the positioning card information corresponding to the person entering the body. The PPE compliance identification module is connected to the server identification and analysis module and is used to compare the PPE identification result with the preset PPE compliance rules to generate the PPE compliance identification result. A face recognition module, connected to the server recognition and analysis module, is used to identify the identity of the person entering based on the face information, compare it with a preset personnel information database, and generate a face recognition result. The location card information comparison module is connected to the server identification and analysis module and is used to compare the location card information with the preset personnel location card binding relationship to generate a location card verification result. The server identification and analysis module performs a fusion judgment based on the PPE compliance identification result, the face recognition result, and the positioning card verification result, and generates an access control command to drive the access control module to open or remain closed.

2. The system according to claim 1, characterized in that, The server identification and analysis module includes: A fast front-end recognition unit is used to identify the human body region of the target person in the human body image; wherein the human body region is divided into a head sub-region, a hand sub-region, and a foot sub-region; the fast front-end recognition unit is also used to output the bounding box and PPE candidate region of the target person, wherein the PPE candidate region is derived from the head sub-region, the hand sub-region, and the foot sub-region. The fine recognition unit is used to perform secondary recognition on the PPE candidate region and output the PPE recognition result.

3. The system according to claim 2, characterized in that, The fast front-end recognition unit uses the YOLOv8 target detection model to detect the human body image; The fine recognition unit uses a preset visual recognition model for secondary recognition to determine whether one or more of the following items exist, whether they are worn correctly, and whether they meet the requirements of the target scenario: safety helmet, goggles, protective face shield, gas mask, face mask, protective gloves, protective clothing, work shoes, protective boots, soundproof earmuffs, gas detector, high-pressure water cleaning protective clothing, shin guards, acid and alkali resistant clothing, chlorine-related work protective equipment, contractor vest or guardian vest.

4. The system according to claim 1, characterized in that, It also includes a management configuration module, which is used to store PPE compliance rules, personnel authorization rules, and personnel location card binding relationships for different work scenarios; The server identification and analysis module calls the PPE compliance rules, personnel authorization rules, and personnel location card binding relationships corresponding to the target scenario, based on the target scenario.

5. The system according to claim 4, characterized in that, The PPE compliance identification module compares the PPE identification result with the PPE compliance rules corresponding to the target scenario stored in the management configuration module to generate a PPE compliance identification result; wherein, the PPE compliance identification result includes at least the overall compliance status, the list of missing PPEs, and the list of abnormal PPEs.

6. The system according to claim 4, characterized in that, The positioning card information comparison module compares the positioning card information with the personnel positioning card binding relationship stored in the management configuration module that corresponds to the target scene, to determine whether the person entering is carrying their own positioning card, and to determine whether the location information of the positioning card matches the location of the human body image acquisition. When the identity of the person entering the location is identified as matching the person bound to the location card, and the location card is within a preset allowable range, the location card verification is deemed successful; otherwise, the location card verification is deemed unsuccessful.

7. The system according to claim 1, characterized in that, The front-end image acquisition module is installed at a fixed position in front of, to the side of, or in the area to be controlled in the target scene, and is set towards the direction of personnel passage so that the acquisition line of sight covers the head, torso, hands and feet of the personnel entering. The target scenario refers to any one of the following: cabinet rooms, substations, equipment plant entrances / exits, and open area entrances / exits in areas such as the chlor-alkali plant, polyvinyl chloride plant, acetylene plant, and public works center.

8. The system according to claim 1, characterized in that, It also includes a voice prompt module, which is connected to the server recognition and analysis module and is used to broadcast corresponding voice prompt information based on the result of the fusion judgment, including: When the PPE compliance identification result has missing or abnormal items, the corresponding missing or abnormal item prompt will be broadcast. When the facial recognition result fails, an error message indicating an abnormal identity or permission will be displayed. When the location card verification result fails, a location card error message will be broadcast. When all three checks pass, a message indicating that passage is permitted will be broadcast.

9. The system according to claim 1, characterized in that, It also includes a log recording module, which is connected to the server identification and analysis module and is used to record access events or abnormal events of the personnel entering the premises. The log recording module shall include at least the following fields: The data includes: identification time, data collection time, access control point number, device number, personnel identity information, location card number, image storage path, PPE identification result, list of missing and abnormal items in the PPE compliance identification result, face recognition result, location card verification result, access control command, access control feedback status, and voice prompt content.

10. A method for identifying the wearing of special protective equipment based on multi-dimensional information fusion, applied to the system as described in any one of claims 1-9, characterized in that, include: Receive human images of people entering the area and associated information corresponding to the human images sent by the front-end image acquisition module. The associated information includes at least the acquisition time, device number and access control point number. The human image is identified to obtain a PPE identification result. The PPE compliance identification module is then invoked so that the PPE compliance identification module compares the PPE identification result with preset PPE compliance rules. The PPE compliance identification result returned by the PPE compliance identification module is then received. Extract the facial features of the person in the human image, call the face recognition module, so that the face recognition module can identify the identity of the person entering based on the facial features and compare it with the preset personnel information database, and receive the face recognition result returned by the face recognition module; Obtain the location card information corresponding to the person entering, call the location card information comparison module so that the location card information comparison module obtains the location card information corresponding to the person entering and compares the location card information with the preset personnel location card binding relationship, and receive the location card verification result returned by the location card information comparison module; The PPE compliance identification result, the face recognition result, and the location card verification result are fused and judged. When the PPE compliance identification result, the face recognition result, and the location card verification result all meet the preset conditions, an access permission command is generated and sent to the access control module to drive the access control device to open; when any one of the preset conditions is not met, an access prohibition command is generated and sent to the access control module to drive the access control device to remain closed.

Images