A safety protection product wearing verification method and system based on a patrol video

Abstract

The present application relates to the field of computer vision and intelligent safety inspection, and discloses a safety protection product wearing verification method and system based on inspection video; the method performs cross-frame association on personnel targets in the inspection video, determines the protection product to be verified according to the operation risk information, and establishes a wearing topology template including a human body part node, a protection product node, an attachment connection edge and a shielding relationship edge; generates a verification evidence in combination with the exposed state of the human body surface and the necessary attachment connection state, selects a complementary image frame according to the connection set to be verified, fuses the verification evidences of different perspectives, and outputs a verification result of wearing compliance, product loss, non-standard wearing or insufficient evidence.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent security inspection, specifically to a method and system for verifying the wearing of security protective equipment based on inspection videos. Background Technology

[0002] With the development of intelligent inspection equipment, mobile video terminals, and machine vision technology in industrial sites, the automatic verification of the wearing status of safety protective equipment (PPE) by inspecting video has been gradually applied to equipment maintenance, production operations, warehousing and transportation, and inspection of high-risk areas. Existing technologies typically first detect personnel targets and PPE such as safety helmets, gloves, safety belts, and reflective vests in video images. Then, based on the positional relationship, overlap, or distance between the PPE detection area and the head, hands, torso, or legs, they determine whether the PPE is present and located at a predetermined body part. Some solutions also combine personnel tracking, human key point analysis, multi-frame recognition, or semantic segmentation to accumulate the PPE detection results for the same person in consecutive images.

[0003] However, inspection cameras are typically in motion, and the distance, viewing angle, and relative position between personnel and the camera are constantly changing. Furthermore, the connection points of protective equipment may be obstructed by tools, equipment, personnel's own limbs, or other people. When judging solely based on single-frame detection results or the overlap between the protective equipment detection area and the human body area, it is difficult to distinguish between situations such as correctly worn protective equipment, protective equipment being held by hand, protective equipment being suspended on a non-target part of the human body, protective equipment being near adjacent personnel, and incomplete connections of protective equipment. While using majority voting with consecutive images can reduce fluctuations in single-frame detection, consecutive images from the same viewing direction often contain repetitive information, still failing to provide valid verification evidence for obstructed connection points.

[0004] Furthermore, existing methods typically interpret the absence of protective equipment (PE) as a sign of not being worn, or stop judging when the body part is not visible, lacking verification of whether the human body surface that should be covered by PE under correct wearing conditions is actually exposed. When the human body surface cannot be verified due to obstruction, being outside the frame, insufficient resolution, or motion blur, the absence of PE detection does not directly prove its absence; similarly, when PE is present but its necessary connecting parts are not stably connected to the corresponding body part, the same judgment should not be applied as if the PE were completely missing. Therefore, existing inspection video verification methods, under conditions of dynamic observation, partial obstruction, and complex personnel interactions, suffer from unclear boundaries between missing PE, improper wearing, and insufficient evidence, and cannot select effective supplementary images for specific unverified wearing connection parts. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for verifying the wearing of safety protective equipment based on inspection video, thereby solving the technical problems existing in the prior art.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: A method for verifying the wearing of safety protective equipment based on inspection videos includes the following steps: S1: Acquire the inspection video collected by the inspection equipment, perform cross-frame correlation on the personnel targets in the inspection video, and form personnel trajectories; S2: Determine the protective equipment to be verified based on the operational risk information corresponding to the personnel trajectory, and obtain the wearing topology template corresponding to the protective equipment to be verified. The wearing topology template includes human body part nodes, protective equipment nodes, attachment connection edges representing the wearing connection relationship between the protective equipment and the human body parts, and occlusion relationship edges representing the covering relationship formed by the protective equipment on the human body surface area under the correct wearing state. Among the attachment connection edges, the attachment connection edges that must be formed according to the correct wearing structure requirements are the necessary attachment connection edges. S3: For image frames in the personnel trajectory, based on the personnel posture, human body surface segmentation results and the observation direction of the inspection camera relative to the personnel target, determine whether the human body surface area associated with the occlusion relationship edge meets the exposure judgment condition. S4: Establish a local coordinate system for the human body based on the human body part nodes, and determine the connection status of necessary attached connection edges based on the spatial relationship between the protective equipment nodes and the human body part nodes and the cross-frame relative motion relationship. S5: When a protective equipment node is detected and the corresponding necessary attachment connection edge is in a connected state, positive verification evidence is generated; when no corresponding protective equipment node is detected in at least two temporally adjacent image frames and the corresponding human body surface areas all meet the exposure judgment conditions, reverse verification evidence is generated; when a protective equipment node is detected but at least one necessary attachment connection edge is not formed, evidence of improper wearing is generated; in other states, verification evidence is generated. S6: Establish a set of connections to be verified based on the necessary attached connection edges in the verification data. Select complementary image frames based on the expected visibility of the necessary attached connection edges in the candidate image frames, the expected exposure of the corresponding human surface area, and the degree of overlap of observation view between the candidate image frames and the adopted image frames. S7: Integrate verification evidence from different perspectives and output verification results for compliance with wearing regulations, missing items, improper wearing, or insufficient evidence. Among these, missing items are determined by reverse verification evidence from at least two different perspectives.

[0007] Preferably, the determination of operational risk information includes: Obtain the work task data corresponding to the inspection area, and extract the work area range and work risk category from the work task data; Identify high-risk objects from inspection videos and extract the time sequence of key human body points of personnel targets, the distance changes between personnel targets and high-risk objects, and the contact areas between human body parts and high-risk objects; Based on the time sequence, distance changes, contact area, and duration of contact at key points on the human body, the operational status of the personnel target relative to the risk object is determined; Based on the time interval of personnel entering the work area, the work risk category, risk object, and operation status are associated with the personnel trajectory. The protective equipment to be verified is determined according to the mapping relationship between the protection requirements of the work risk category, risk object, operation status and protective equipment category.

[0008] Preferably, constructing a wearing topology template includes: Human body parts nodes are determined based on key human body points and human body surface segmentation results in correctly worn samples, and protective equipment nodes are determined based on the segmentation results of the components of the protective equipment to be verified. Based on the continuous spatial connection between the protective equipment nodes and human body part nodes in the correctly worn sample, an attachment connection edge is established, and the relative position allowable range, relative direction allowable range, and scale ratio allowable range are determined for each attachment connection edge. Establish occlusion relationships based on the coverage relationships formed by the nodes of protective equipment on the human body surface area in the correctly worn sample; Based on the correct wearing structure requirements of the protective equipment to be verified, the attachment connection edges that must be formed under the correct wearing condition are marked as necessary attachment connection edges.

[0009] Preferably, determining whether a human body surface area meets the exposure criteria includes: Based on the person's posture, the human surface region associated with the occlusion relationship edge is projected onto the current image frame to obtain the surface region to be determined. Based on the segmentation results of other parts of the personnel, risk objects, scene objects and other personnel, determine the occlusion ratio of the surface area to be judged; Determine the effective pixel size, boundary integrity, local image clarity, and pixel ratio of the exposed human body surface of the surface region to be judged; When the occlusion ratio is not greater than the occlusion ratio threshold, the effective pixel size is not less than the verification size threshold, the boundary integrity is not less than the boundary integrity threshold, the local image clarity is not less than the clarity threshold, and the proportion of exposed human body surface pixels is not less than the exposure ratio threshold, the human body surface area is determined to meet the exposure judgment conditions. Each threshold is determined jointly based on the statistical distribution of the corresponding indicators in the labeled calibration images of the correct wearing state, the non-wearing state, and the occluded state.

[0010] Preferably, determining the connection state of the necessary attached connecting edges includes: The origin, reference direction, and scale reference of the local coordinate system of the human body are determined by at least two key human body points that constitute the nodes of the human body parts. The position, orientation, and size of the protective equipment nodes in sequentially adjacent image frames are transformed to the local coordinate system of the human body to obtain the relative position vector, relative orientation difference, and scale ratio of the protective equipment nodes relative to the human body parts nodes. When the relative position vector, relative direction difference, and scale ratio are all within the allowable range of the corresponding attached connection edge, and the changes in the aforementioned data in at least two temporally adjacent image frames are not greater than the stable change threshold, it is determined that the necessary attached connection edge is in a connected state. When a protective equipment node does not form an attachment connection edge with a human body part node defined by the wearing topology template, but forms an attachment connection edge with other human body part nodes of the personnel target, the protective equipment node is determined to be a non-wearing protective equipment node. The stable change threshold is determined based on the labeled calibration videos of the correct wearing state, handheld state, and adjacent placement state.

[0011] Preferably, the personnel trajectory is divided into verification periods according to the moment when the existence status of protective equipment nodes or the connection status of necessary attached connecting edges continuously changes, and verification evidence is generated in each verification period. When the status changes of the protective equipment node and its corresponding necessary attached connection edge continue to reach the status confirmation time, the current verification period ends and the next verification period is established. During the same verification period, when a necessary attachment connection edge obtains positive verification evidence or evidence of improper wearing, the necessary attachment connection edge is removed from the set of connections to be verified; when the protective equipment to be verified obtains negative verification evidence in at least two different view categories, the necessary attachment connection edge associated with the protective equipment to be verified is removed from the set of connections to be verified. If contradictory verification evidence appears within the same verification period and the conditions for switching verification periods are not met, the corresponding image frame will be recorded as verification evidence to be verified.

[0012] Preferably, complementary image frames are selected, including: The expected visibility of the necessary attached connection edge is determined based on the proportion of the two nodes of the necessary attached connection edge falling within the image range, the expected non-occlusion proportion, the projection length of the connection direction, and the effective pixel spacing between the two nodes. The expected exposure level of the human body surface area is determined based on the projected area of ​​the corresponding human body surface area, the degree of orientation towards the inspection camera, and the expected non-obstruction ratio. The verification gain of candidate image frames is generated based on the expected visibility, expected exposure, effective pixel size of personnel targets, and local image sharpness. Based on the angle between the observation direction of the candidate image frame and the adopted image frame and the degree of overlap of the projection of the corresponding human body surface area, the degree of observation view repetition is generated. The difference between the normalized verification gain and the normalized observation view repetition is used as the frame selection score. Complementary image frames are selected from high to low according to the frame selection score, and the verification data generated by the complementary image frames is used to update the set of connections to be verified until the set of connections to be verified is empty or the personnel trajectory ends.

[0013] A safety protection equipment wearing verification system based on inspection video includes a personnel trajectory construction module, a risk information processing module, a wearing topology acquisition module, a human body surface exposure determination module, an attachment connection determination module, a verification data generation module, a complementary image selection module, and a verification result output module. The personnel trajectory construction module is used to acquire inspection videos, perform cross-frame correlation on personnel targets, and form personnel trajectories. The risk information processing module is used to determine the protective equipment to be verified based on the operational risk information corresponding to the personnel's trajectory; The wear topology acquisition module is used to acquire the wear topology template corresponding to the protective equipment to be verified. The wear topology template includes human body part nodes, protective equipment nodes, attachment connection edges and occlusion relationship edges, and determines the necessary attachment connection edges from the attachment connection edges. The human body surface exposure determination module is used to determine whether the human body surface area associated with the occlusion relationship edge meets the exposure determination conditions based on the person's posture, the human body surface segmentation results, and the observation direction of the inspection camera relative to the person target. The attachment connection determination module is used to establish a local coordinate system of the human body based on the human body part nodes, and to determine the connection status of necessary attachment connection edges based on the spatial relationship between the protective equipment nodes and the human body part nodes and the cross-frame relative motion relationship. The verification evidence generation module is used to generate positive verification evidence, reverse verification evidence, evidence of improper wearing, or evidence to be verified based on the node detection results of protective equipment, the exposure status of the human body surface area, and the connection status of necessary attachment and connection edges. The complementary image selection module is used to establish a set of connections to be verified based on the necessary attached connection edges in the data to be verified, and to select complementary image frames based on the expected visibility of the necessary attached connection edges in the candidate image frames, the expected exposure of the corresponding human body surface area, and the degree of repetition of the observation view. The verification result output module is used to integrate verification evidence from different perspectives and output verification results for compliance with wearing regulations, missing items, improper wearing, or insufficient evidence. Among them, missing items are determined by reverse verification evidence from at least two different perspectives.

[0014] Preferably, the risk information processing module includes a task data parsing unit, a video risk identification unit, and a protective equipment determination unit; The task data parsing unit is used to extract the work area range and work risk category from the work task data; The video risk identification unit is used to determine the operational status of the personnel relative to the risk object based on the timing of key human body points in the inspection video, the distance changes between the personnel target and the risk object, the contact area between the human body parts and the risk object, and the duration of the contact state. The protective equipment determination unit is used to determine the protective equipment to be verified based on the mapping relationship between the protective requirements of the operation risk category, the risk object, the operation status and the protective equipment category. The wearable topology acquisition module includes a human body node determination unit, a protective equipment node determination unit, and a topology relationship determination unit. The topology relationship determination unit is used to establish attachment connection edges and occlusion relationship edges, and to determine the necessary attachment connection edges according to the correct wearing structure requirements.

[0015] Preferably, the human body surface exposure determination module outputs the exposure determination result of the human body surface area based on the occlusion ratio of the human body surface area, the effective pixel size, the boundary integrity, the local image clarity, and the proportion of exposed human body surface pixels. The attachment connection determination module outputs the connection status of necessary attachment connection edges based on the relative position vector, relative direction difference, scale ratio, and cross-frame change of the aforementioned data of the protective equipment node relative to the human body part node. The complementary image selection module generates a verification gain based on the expected visibility of the necessary attached connecting edge, the expected exposure of the corresponding human surface area, the effective pixel size of the personnel target, and the clarity of the local image. It generates the observation view repetition degree based on the observation direction angle between the candidate image frame and the adopted image frame and the degree of overlap of the projection of the human surface area. It then selects a complementary image frame based on the verification gain and the observation view repetition degree. The verification result output module updates the set of connections to be verified based on the verification evidence generated from complementary image frames, and outputs the verification result based on the updated set of connections to be verified and the verification evidence in different view categories. In summary, the present invention has the following main beneficial effects: By establishing a wearing topology template that includes human body part nodes, protective equipment nodes, attachment connection edges, and occlusion relationship edges, the system jointly verifies whether protective equipment appears, whether it is located on the target human body part, and whether a correct wearing connection relationship is formed. It also combines the spatial relationship in the local human body coordinate system and the cross-frame relative motion relationship to determine the connection status of necessary attachment connection edges, thereby achieving the effect of distinguishing between correct wearing, equipment being held, equipment being placed nearby, and the state of partial connection missing, reducing erroneous judgments caused by relying solely on the overlap between the protective equipment detection frame and the human body area.

[0016] By using the area of ​​the human body surface that should be covered by protective equipment when it is correctly worn as the reverse verification object, reverse verification evidence is generated only when the corresponding human body surface area meets the verification conditions of occlusion ratio, effective pixel size, boundary integrity, local image clarity and the proportion of exposed human body surface pixels, and no protective equipment nodes are detected in consecutive image frames. This achieves the effect of distinguishing between the actual absence of protective equipment and the inability to identify it due to occlusion, out of frame, blur, or unsuitable viewing angle, and avoids directly judging the absence of equipment as insufficient evidence.

[0017] By establishing a set of connections to be verified based on necessary attachment connections for which valid verification evidence has not yet been obtained, and selecting complementary image frames by combining the expected visibility of necessary attachment connections, the expected exposure of the corresponding human body surface area, and the repetition of observation angles, and fusing verification evidence from different viewpoint categories, the goal is to supplement key wearing parts with limited inspection videos. This enables the verification results to distinguish between wearing compliance, missing items, improper wearing, and insufficient evidence, thereby improving the reliability of verification results under conditions of motion inspection, partial human body occlusion, and changes in observation direction. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] refer to Figure 1 A method for verifying the wearing of safety protective equipment based on inspection videos includes the following steps: S1: Acquire the inspection video collected by the inspection equipment, perform cross-frame correlation on the personnel targets in the inspection video, and form personnel trajectories; S2: Determine the protective equipment to be verified based on the operational risk information corresponding to the personnel trajectory, and obtain the wearing topology template corresponding to the protective equipment to be verified. The wearing topology template includes human body part nodes, protective equipment nodes, attachment connection edges representing the wearing connection relationship between the protective equipment and the human body parts, and occlusion relationship edges representing the covering relationship formed by the protective equipment on the human body surface area under the correct wearing state. Among the attachment connection edges, the attachment connection edges that must be formed according to the correct wearing structure requirements are the necessary attachment connection edges. S3: For image frames in the personnel trajectory, based on the personnel posture, human body surface segmentation results and the observation direction of the inspection camera relative to the personnel target, determine whether the human body surface area associated with the occlusion relationship edge meets the exposure judgment condition. S4: Establish a local coordinate system for the human body based on the human body part nodes, and determine the connection status of necessary attached connection edges based on the spatial relationship between the protective equipment nodes and the human body part nodes and the cross-frame relative motion relationship. S5: When a protective equipment node is detected and the corresponding necessary attachment connection edge is in a connected state, positive verification evidence is generated; when no corresponding protective equipment node is detected in at least two temporally adjacent image frames and the corresponding human body surface areas all meet the exposure judgment conditions, reverse verification evidence is generated; when a protective equipment node is detected but at least one necessary attachment connection edge is not formed, evidence of improper wearing is generated; in other states, verification evidence is generated. S6: Establish a set of connections to be verified based on the necessary attached connection edges in the verification data. Select complementary image frames based on the expected visibility of the necessary attached connection edges in the candidate image frames, the expected exposure of the corresponding human body surface area, and the degree of overlap of observation view between the candidate image frames and the adopted image frames. S7: Integrate verification evidence from different perspectives and output verification results for compliance with wearing regulations, missing items, improper wearing, or insufficient evidence. Among these, missing items are determined by reverse verification evidence from at least two different perspectives.

[0021] The inspection camera continuously captures video of the inspection area and records a timestamp for each image frame. The motion information acquisition unit synchronously acquires the position and attitude data of the inspection camera.

[0022] Motion information can be obtained from changes in static scene feature points in inertial measurement units, odometers, positioning devices, gimbal encoders, or inspection videos. When using two or more types of motion information, time alignment is performed according to image frame timestamps, and the cumulative drift of the inertial measurement unit is corrected using relative motion estimated from visual feature points.

[0023] The data processing device detects personnel targets in the inspection video and obtains the personnel target region, human body key points, human body surface segmentation results, and human body appearance features corresponding to each personnel target. The personnel target region can be represented by a bounding rectangle or a personnel instance segmentation region.

[0024] For sequentially adjacent image frames, candidate pairs of personnel associations are constructed based on changes in the position of personnel targets, appearance features, human posture, and direction of movement. Changes in the position of personnel targets are normalized to the scale of personnel targets to avoid incorrect segmentation into different trajectories when the same person experiences large pixel displacements due to changes in distance from the inspection camera.

[0025] For candidate pairs of individuals, an association cost is generated. The lower the association cost, the higher the probability that the two detection results belong to the same person. The association cost comprehensively considers differences in normalized location, appearance features, human posture, and direction of movement. When the association cost is not greater than the trajectory association threshold, the two detection results are associated with the same person's trajectory.

[0026] When a person target corresponds to more than two candidate targets, the candidate target with the lowest association cost and not occupied by other people's trajectories is selected. If a one-to-one association cannot be formed, ambiguous detection results will not be forcibly merged into the original person trajectory.

[0027] When a person is temporarily obscured by equipment, scene objects, or other people, the search area is predicted based on their position, speed, and direction of movement before the obstruction. If the reappearing person meets the trajectory association conditions and the interruption time does not exceed the trajectory recovery time threshold, the original person trajectory is restored; if the trajectory recovery time threshold is exceeded or there are two or more indistinguishable candidate persons, the original person trajectory is terminated and a new person trajectory is established.

[0028] The trajectory association threshold and trajectory recovery time threshold are determined based on calibration videos labeled with real personnel trajectories. During calibration, priority is given to limiting the situation where different personnel are incorrectly merged into the same personnel trajectory, and then the parameter with the lower trajectory interruption rate is selected from the candidate parameters that meet the error merging rate requirements.

[0029] The generated personnel trajectory records at least the personnel trajectory identifier, image frame timestamp, personnel target area, human body key points, human body surface segmentation results, personnel orientation, inspection camera motion information, and subsequently generated verification evidence.

[0030] The observation direction of the inspection camera relative to the personnel target is determined based on the location of the inspection camera and the location of the personnel target.

[0031] The location of a person can be determined based on the person's foot contact point, the target area, and the mapping relationship between the target area and the scene ground. When scene depth cannot be obtained, the relative position of the person is estimated based on the calibration parameters of the inspection camera, the position of the person in the image, and the pixel height of the person.

[0032] A local coordinate system for the person's torso is established using key points on the left and right shoulders and hips. The observation vector from the person's target to the inspection camera is transformed into the local coordinate system of the person's torso to obtain the azimuth and pitch angles relative to the person's orientation.

[0033] When the human body key points required to establish the local coordinate system of the human torso are missing or the confidence of the key points is insufficient, the image frame will not be used to distinguish the viewpoint category, but it can continue to be used as a normal image frame in the human trajectory.

[0034] The angular boundaries of different viewpoint categories are determined based on the calibration video. Specifically, the visibility of protective equipment nodes, necessary attachment connection edges, and human body surface areas is statistically analyzed from different viewing angles; continuous angular ranges in which essentially the same human body surface area and attachment connection structure can be observed are classified into the same viewpoint category. When a change in the viewing direction causes at least one necessary attachment connection edge or corresponding human body surface area to become visible from invisible, it is classified into a different viewpoint category.

[0035] Therefore, the perspective category is not simply fixed as front, back, and side, but corresponds to the wearing structure and verifiable parts of the protective equipment to be verified.

[0036] The data processing device acquires the work task data corresponding to the inspection area. The work task data can come from work tickets, maintenance task orders, on-site work plans, or work management systems.

[0037] Extract the work area range and work risk category from the work task data, and map the work area range to the actual area corresponding to the inspection video based on equipment identification, area identification or scene coordinates.

[0038] Risk objects are identified from inspection videos. Risk objects are equipment, tools, materials, or work structures that directly correspond to operational risks.

[0039] For the trajectory of personnel entering the work area, extract the time sequence of key human body points, the normalized distance between the personnel target and the risk object, the contact area between human body parts and the risk object, and the duration of the contact state.

[0040] When the normalized distance between the operated human body part and the risk object is not greater than the operation distance threshold, the contact area ratio is not less than the contact ratio threshold, and the duration of this state is not less than the operation duration threshold, the personnel target is determined to be in the operation state of the corresponding risk object.

[0041] For operations that do not require direct contact with the risk object, the operational status is determined based on the reciprocating motion, periodic motion, or continuous pointing relationship of key human body points relative to the risk object.

[0042] If personnel merely pass by the vicinity of the risk object without forming a continuous approach, contact, or action trajectory corresponding to the operation, it is uncertain whether it is an operational state.

[0043] The operating distance threshold, contact ratio threshold, and operating duration threshold are jointly determined based on calibration videos labeled with operating and non-operating states. During calibration, priority is given to preventing situations where personnel passing near the risk object are mistakenly identified as operators.

[0044] Based on the mapping relationship between the protection requirements of the operation risk category, risk object, operation status and the protection equipment category, the protective equipment to be verified corresponding to the personnel trajectory is determined.

[0045] The protective requirement mapping relationship is only used to determine the category of protective equipment that needs to be verified, and is not used directly to determine whether the protective equipment is worn correctly.

[0046] Training data, calibration data, and verification data are established for each type of protective equipment.

[0047] The training data should include at least the following images: correctly worn, not worn at all, improperly worn, handheld protective equipment, protective equipment hanging on a non-target body part, protective equipment located near adjacent personnel, external obstruction, self-obstruction by the body, motion blur, and images from different viewing directions.

[0048] The training data is used to train personnel detection models, human body key point extraction models, human body surface segmentation models, and protective equipment component segmentation models.

[0049] The input to the human body surface segmentation model is the image patch corresponding to the target area of ​​the person and the category of human body parts. The output is the category result of each pixel belonging to the surface of the human body, the surface of basic work clothes, protective equipment, or external occlusion.

[0050] The input to the protective equipment component segmentation model is an image patch of the target area of ​​the person or the corresponding human body part, and the output is the segmented region and detection confidence of the protective equipment component.

[0051] For protective gloves, the human body surface includes the skin of the palms, fingers, and wrists; for reflective vests or safety belts, the human body surface can be the surface of the basic work clothes; for safety helmets, the human body surface can be the hair area and the outer contour area of ​​the head.

[0052] The position of the nodes of protective equipment is determined by the geometric center or skeleton center of the segmented area of ​​the corresponding component, the direction is determined by the main axis direction of the segmented area, and the size is determined by the length of the segmented area along the main axis and the secondary axis.

[0053] When the effective area of ​​the segmented region of the protective equipment component is less than the effective area threshold of the node, or when the proportion of the segmented region exceeding the image boundary is greater than the node incompleteness threshold, a definite protective equipment node is not output, and the node status is recorded as uncertain.

[0054] The effective area threshold and the node incompleteness threshold are determined based on the node samples that can be reliably identified and the node samples that cannot be reliably identified.

[0055] Calibration data is used to determine trajectory association thresholds, operational state thresholds, human surface exposure thresholds, allowable adhesion intervals, stability change thresholds, and complementary image selection weights. Validation data is used only to validate the determined parameters and does not participate in parameter selection.

[0056] The same person or the same continuous video clip must not be included in both calibration and verification data at the same time to avoid distortion of parameter evaluation results.

[0057] Example 5: Establishment of Wearing Topology Template A wearing topology template is created for each type of protective equipment to be verified. The wearing topology template includes human body part nodes, protective equipment nodes, attachment connection edges, and occlusion relationship edges.

[0058] Human body part nodes are determined based on human body key points and segmented regions on the human body surface. A human body part node can be defined by at least two human body key points and their associated human body regions.

[0059] The protective equipment nodes are determined based on the segmentation results of the protective equipment components. For protective equipment that only requires verification of the overall position, an overall protective equipment node can be set; for protective equipment that requires verification of the partial connection status, protective equipment nodes should be set separately for each of its components.

[0060] For correctly worn samples, the relative position, relative direction, and scale ratio between protective equipment nodes and human body part nodes are statistically analyzed. When a certain connection relationship continuously appears in the valid image frames of correctly worn samples, but cannot appear stably in handheld, adjacent, or improperly worn samples, an attachment connection edge is established between the corresponding protective equipment node and human body part node.

[0061] For each attachment connection edge, allowable intervals for relative position, relative direction, and scale are set. These allowable intervals are formed by the parameter distribution in correctly worn samples, and the interval boundaries are corrected using incorrectly worn samples.

[0062] Based on the stable coverage relationship formed by the nodes of protective equipment on the human body surface in the correctly worn sample, establish the occlusion relationship edge, and record the boundary of the corresponding human body surface area in the local coordinate system of the human body.

[0063] Based on the correct wearing structure requirements of protective equipment, candidate necessary attachment connection edges are identified. Then, samples of correct wearing are used to verify whether the candidate connection edges persist, and samples of improper wearing are used to verify whether the absence of the connection edge results in an incomplete wearing state. Candidate connection edges that meet the aforementioned conditions are marked as necessary attachment connection edges.

[0064] For example, a helmet wearing topology template can include a head top node, a chin area node, a helmet body node, and a chin strap node. Necessary attachment edges are established between the helmet body node and the head top node, necessary attachment edges are established between the chin strap node and the chin area node, and occlusion relationship edges are established between the helmet body node and the corresponding human body surface area on the head top.

[0065] The topology template for protective gloves can include hand nodes, wrist nodes, palm cover nodes, finger cover nodes, and wrist-mouth nodes. Necessary attachment edges are established between the palm cover nodes and finger cover nodes and the hand nodes, and necessary attachment edges are established between the wrist-mouth nodes and the wrist nodes.

[0066] The seat belt wearing topology template can include shoulder strap nodes, chest strap nodes, waist belt nodes, leg strap nodes, connecting rope nodes, and their corresponding human body part nodes and connection position nodes.

[0067] Based on the person's posture in the current image frame, the human body surface region associated with the occlusion relationship edge in the wearing topology template is projected onto the current image frame to obtain the surface region to be determined.

[0068] For a human surface region defined by two human key points, projection is performed using translation, rotation, and scaling transformations; for a human surface region defined by three or more human key points, the human surface region is divided into several local regions, and segmented transformation is performed based on the corresponding human key points.

[0069] When the human body key points required to complete the human body surface projection are missing or the confidence level of the key points is lower than the effective threshold of the key points, the exposure determination of the human body surface area is not performed and it is recorded as unverifiable.

[0070] The occlusion status of the surface area to be judged is determined based on the segmented areas of other parts of the person, the risk object, scene objects, and other people. If the order of occlusion of overlapping areas cannot be determined, the overlapping part is included in the occluded area and not in the exposed human body surface area.

[0071] Exposure determination should use at least the following indicators: The proportion of the surface area to be determined that is occluded by other targets; The effective pixel size of the surface region to be determined, which is located within the current image range and is not occluded; The degree of boundary integrity of the surface region to be determined within the image area; The clarity of the local image within the surface area to be determined; The percentage of human body surface pixels identified as not covered by protective equipment to be verified.

[0072] When the occlusion ratio is not greater than the occlusion ratio threshold, the effective pixel size is not less than the verification size threshold, the boundary integrity is not less than the boundary integrity threshold, the local image clarity is not less than the clarity threshold, and the proportion of pixels on the exposed human body surface is not less than the exposure ratio threshold, the human body surface area is determined to meet the exposure judgment conditions.

[0073] Each threshold was determined using a joint calibration method. First, candidate thresholds were formed based on the statistical distribution of each indicator in the correct wearing state, the no-wearing state, and the occluded state; then, the calibration data were processed using a combination of candidate thresholds.

[0074] Among the candidate combinations that satisfy the condition that the error rate of correctly wearing or covering the body surface is incorrectly judged as being exposed does not exceed the upper limit of the exposure misjudgment rate, the threshold combination with the lower actual body surface exposure misjudgment rate is selected.

[0075] Therefore, the determination of human body surface exposure should prioritize avoiding the misinterpretation of situations where protective equipment is temporarily not visible as a lack of equipment.

[0076] For each human body part node, a local coordinate system is established using at least two human body key points. The first reference key point is used as the origin of the coordinate system, the direction from the first reference key point to the second reference key point is used as the reference direction, and the distance between the two reference key points is used as the scale reference.

[0077] Let the position of the protective equipment node in the image coordinate system be... The origin of the local coordinate system of the human body is The direction transformation matrix is The human body scale benchmark is Then the normalized position of the protective equipment node in the local coordinate system of the human body is: ; In the formula, The normalized position of the protective equipment node in the local coordinate system of the human body; This represents the position of the protective equipment node in the image coordinate system. The origin of the local coordinate system of the human body; This is the direction transformation matrix; It serves as a benchmark for human body dimensions.

[0078] The relative position vector between the protective equipment node and the human body part node is determined based on the normalized position. The relative direction difference is determined based on the principal axis direction of the protective equipment node and the reference direction of the human body part. The scale ratio is determined based on the size of the protective equipment node and the human body scale reference.

[0079] The relative direction difference is calculated based on the minimum included angle of the circle to avoid erroneous large angle differences when the direction angle crosses the boundaries of positive and negative angles.

[0080] When the relative position vector, relative direction difference, and scale ratio are all within the allowable range of the corresponding attached connection edge, the current image frame is determined to meet the single-frame attachment candidate condition.

[0081] To rule out accidental overlap between protective equipment nodes and human body parts, further assessment of cross-frame relative motion stability is needed. Let the relative position vectors in adjacent image frames be... and The relative direction differences are respectively and The scale ratios are respectively and Then the change in attachment is: ; In the formula, This represents the amount of attachment variation of the attached connection edge between adjacent image frames; , and These are the weights corresponding to relative position, relative direction, and scale, respectively, all of which are non-negative and their sum is 1; It is the smallest included angle on the circumference.

[0082] When at least two temporally adjacent image frames satisfy the single-frame attachment candidate condition and the attachment change is not greater than the stable change threshold, the attachment connection edge is determined to be in a connected state.

[0083] The allowable ranges for relative position, relative direction, and scale, as well as the weights and stability change thresholds, are determined based on calibration videos of the correct wearing, handheld, and adjacent placement states.

[0084] During calibration, the proportion of correctly worn states incorrectly identified as having no connection was recorded, as well as the proportion of handheld and adjacent placement states incorrectly identified as having a connection. Among the candidate parameters that meet the requirement for a low incorrect wearing misclassification rate, the parameter with the lowest incorrect wearing misclassification rate was selected.

[0085] When a protective equipment node does not form an attachment connection edge with a human body part node defined by the wearing topology template, but forms an attachment connection edge with other human body part nodes of the personnel target, the protective equipment node is identified as a non-wearing protective equipment node.

[0086] For example, if the helmet body node forms a stable attachment connection with the hand node, but does not form an attachment connection with the head node, the helmet is identified as being in a handheld state.

[0087] Considering that the same person may change the wearing status of protective equipment during a single personnel trajectory, the personnel trajectory is divided into more than one verification period.

[0088] When the existence status of the protective equipment node or the connection status of the necessary attached connecting edge changes, and the changed status continues for the status confirmation duration, the current verification period ends, and a new verification period is established from the start time of the status change.

[0089] The status confirmation duration is determined based on calibration videos under normal walking, turning, protective equipment shaking, and short-term obstruction conditions to avoid classifying single-frame recognition fluctuation errors as new verification periods.

[0090] Verification evidence and results are generated separately for different verification periods. Compliance results from a later verification period do not overwrite results from a previous verification period indicating missing or improperly worn items.

[0091] Within the same verification period, evidence shall be generated according to the following mutually exclusive conditions: When a protective equipment node is detected, and all necessary attached edges corresponding to that protective equipment node are in a connected state, positive verification data is generated.

[0092] When no corresponding protective equipment node is detected in at least two temporally adjacent image frames, and the human body surface areas associated with the corresponding occlusion relationship edges all meet the exposure determination conditions, reverse verification evidence is generated.

[0093] Two image frames used to generate the same reverse verification evidence should have different timestamps, and the time interval should not exceed the continuous evidence interval threshold. The continuous evidence interval threshold is determined based on the video sampling frequency, the speed of human movement, and the rate of change of occlusion status.

[0094] When a node for protective equipment is detected, but at least one necessary attachment connection edge is not formed, evidence of improper wearing is generated.

[0095] When the human body surface area does not meet the exposure determination conditions, the node status of protective equipment is uncertain, or the connection status of necessary attachment and connection edges cannot be determined, verification data is generated.

[0096] The necessary attached connection edges whose connection status has not yet been determined are recorded in the verification data, and a set of connections to be verified is established accordingly.

[0097] When a necessary attached connection edge obtains positive verification evidence or is fitted with non-standard evidence, the necessary attached connection edge is removed from the set of connections to be verified.

[0098] When the same protective equipment to be verified has already obtained reverse verification evidence in at least two different view categories, the necessary attachment connection edges associated with that protective equipment will be removed from the set of connections to be verified.

[0099] When conflicting evidence appears within the same verification period, determine whether the condition of the protective equipment meets the conditions for switching verification periods. If the switching conditions are met, establish a new verification period; if the switching conditions are not met, record the conflicting images as evidence to be verified, without directly overwriting existing evidence.

[0100] Candidate image frames are those that have been captured in the trajectory of the person and have undergone person pose extraction, human body surface area projection, basic occlusion analysis and local image sharpness analysis, but have not yet undergone complete wear topology verification.

[0101] For each necessary attached connection edge in the set of connections to be verified, determine its expected visibility in the candidate image frame.

[0102] The expected visibility is determined based on at least the following data: The proportion of the projected area of ​​the nodes at both ends of the attached connecting edge that falls within the image range; The percentage of the two endpoints that are not expected to be occluded by scene objects; The projection length of the attachment connection direction in the imaging plane; The effective pixel spacing between the two endpoints.

[0103] The expected exposure level for a corresponding area of ​​the human body surface should be determined based on at least the following data: The projected area of ​​the human body surface region; The degree to which the surface area of ​​the human body faces the inspection camera; The percentage of the human body surface area that is expected to be unobstructed by other targets.

[0104] The expected exposure level is only used to determine whether the candidate image frame has the observation conditions for further exposure determination. It does not directly indicate that the human body surface is exposed, nor does it directly generate reverse verification evidence.

[0105] The verification gain of candidate image frames is determined by taking into account the expected visibility of the necessary attached connecting edges, the expected exposure of the corresponding human body surface, the effective pixel size of the personnel target, and the local image sharpness.

[0106] Then, based on the angle between the observation direction between the candidate image frame and the already used image frame, and the degree of overlap of the projection of the corresponding human body surface area in the two image frames, the degree of overlap of the observation viewpoint is determined.

[0107] After normalizing the verification gain and the degree of repetition of the observation viewpoint, the frame selection score for the candidate image frame is as follows: ; In the formula, Candidate image frames Frame selection score; This is the normalized verification gain; This represents the degree of repetition of observational perspectives after normalization. Complementary image frames are selected from high to low scores based on frame selection.

[0108] When two candidate image frames have the same frame selection score, the image frame with a larger viewing angle from the already adopted image frame is given priority; if the viewing angle is still the same, the image frame with an earlier timestamp is given priority.

[0109] After selecting a complementary image frame, the system performs segmentation of protective equipment components, determination of human surface exposure, determination of attachment connection status, and generation of verification data on the image frame, and updates the set of connections to be verified.

[0110] When there are repeated verification evidences for the same necessary attached connection edge within the same viewpoint category, the verification evidence generated by the complementary image frame with the higher frame selection score is retained to avoid consecutive frames under the same observation direction being repeatedly accumulated into multiple independent pieces of evidence.

[0111] The complementary image frame selection is continuously performed until the set of connections to be verified is empty, the personnel trajectory ends, or none of the remaining candidate image frames meet the minimum verification gain condition.

[0112] The weights of each data point in the verification gain, the weights of each data point in the observation viewpoint repeatability, and the minimum verification gain condition are all determined based on the calibration video. During calibration, among the parameter combinations that meet the final verification error rate requirements, the combination with the lowest average number of complete image frames processed per person is selected.

[0113] For the current verification period, if all necessary attachment and connection edges of the protective equipment to be verified have positive verification evidence, the wearing compliance will be output.

[0114] Output item is missing when at least two different perspective categories have reverse verification evidence for the same protective equipment to be verified.

[0115] Reverse verification evidence from different viewpoint categories should each meet the criteria for determining human surface exposure. Consecutive image frames within the same viewpoint category cannot be used as evidence from two different viewpoints.

[0116] When a protective equipment node is detected, and at least one necessary attached connection edge has evidence of improper wearing, output "improper wearing".

[0117] If the set of connections to be verified is still not empty after the personnel trajectory or the current verification period ends, the output evidence is insufficient.

[0118] Insufficient evidence means that the inspection video failed to provide sufficient evidence of positive, negative, or improper wearing of the items, and insufficient evidence will not be directly used to determine that the items are missing.

[0119] The verification results should include at least the personnel trajectory identifier, verification time period identifier, category of protective equipment to be verified, verification status, timestamp of evidence image frame, viewpoint category, human body surface area, and status of necessary attached connecting edges.

[0120] The helmet wearing topology template includes nodes for the top of the head, the chin, the helmet body, and the chin strap.

[0121] A first necessary attachment connection edge is established between the hat body node and the head top node, and a second necessary attachment connection edge is established between the chin strap node and the chin area node. An occlusion relationship edge is established between the hat body node and the corresponding human body surface area of ​​the head top.

[0122] When a hat node is detected in the first observation direction and a stable attachment connection is formed between the hat node and the top of the head node, positive verification evidence is generated for the first necessary attachment connection edge.

[0123] When the chin region is occluded by the shoulder, tools, or scene objects, even if no chin band node is detected, no reverse verification evidence is generated. Instead, the second necessary attached connection edge is retained in the set of connections to be verified.

[0124] The complementary image selection process prioritizes image frames that can improve the expected visibility of chin band nodes and the expected exposure of the chin region.

[0125] If a chin band node is detected in the complementary image frame, and the chin band node forms a stable attachment connection with a chin region node, then positive verification evidence is generated for the second necessary attachment connection edge. If both necessary attachment connection edges have positive verification evidence, the output "Helmet wearing compliant" is output.

[0126] If a hat body node is detected, but the chin strap node is missing or the chin strap node does not form an attachment connection with the chin area node, then evidence of improper wearing is generated.

[0127] If no helmet node is detected in either of the two different viewpoint categories, and the human body surface area corresponding to the top of the head meets the exposure criteria, then the output is "Helmet equipment missing".

[0128] For the left and right hands, establish hand nodes, wrist nodes, palm-covering nodes, finger-covering nodes, and wrist-mouth nodes respectively.

[0129] Establish necessary attachment edges between the palm-covering node, the finger-covering node and the corresponding hand node, and establish necessary attachment edges between the wrist-mouth node and the corresponding wrist node.

[0130] When a glove node is detected and all necessary attached connection edges are formed, positive verification evidence is generated.

[0131] When the hand is covered by a tool or device, even if no glove node is detected, no reverse verification evidence is generated. Instead, the corresponding necessary attached connection edges are retained in the set of connections to be verified.

[0132] When no glove node is detected in at least two temporally adjacent image frames, and the human body surface areas corresponding to the palm, fingers, and wrist all meet the exposure determination criteria, reverse verification evidence is generated.

[0133] The corresponding glove item is only output as missing if reverse verification evidence for the same glove is formed again in a different perspective category.

[0134] When a glove node is detected, but the wrist node does not form the necessary attachment connection edge with the wrist node, evidence of improper wearing is generated.

[0135] The seatbelt wearing topology template includes shoulder strap nodes, chest strap nodes, waist belt nodes, leg strap nodes, connecting rope nodes, and their corresponding shoulder nodes, chest nodes, waist nodes, leg nodes, and connection position nodes.

[0136] Based on the correct structural requirements for wearing a seat belt, the connecting edges of the shoulder straps, waist belt, leg straps, and connecting ropes are identified as necessary attachment connecting edges.

[0137] When all necessary attachment connections have positive verification evidence, the output shows that the seat belt wearing is compliant.

[0138] When shoulder strap nodes and waist belt nodes are detected, but leg strap nodes do not form the necessary attachment connection with leg nodes, evidence of improper wearing is generated, and this state is not determined as a complete lack of seat belt.

[0139] When the torso or legs are obscured by the equipment and it is impossible to determine whether the belt exists, the corresponding necessary attachment connection edges are recorded in the verification data and the verification is continued using subsequent complementary perspectives.

[0140] The safety protection equipment wearing verification system based on inspection video includes a personnel trajectory construction module, a risk information processing module, a wearing topology acquisition module, a human body surface exposure determination module, an attachment connection determination module, a verification data generation module, a complementary image selection module, and a verification result output module.

[0141] The personnel trajectory construction module receives inspection videos and motion information from inspection cameras, and outputs personnel trajectories.

[0142] The risk information processing module receives personnel trajectory and work task data, and outputs protective equipment to be verified corresponding to the personnel trajectory and verification time period.

[0143] The wear topology acquisition module receives the category of protective equipment to be verified and outputs the wear topology template, the necessary attachment connection edges, and the human body surface area associated with the occlusion relationship edges.

[0144] The human body surface exposure determination module receives human body key points, human body surface segmentation results, inspection camera observation direction and occlusion relationship edges, and outputs human body surface area exposure determination results.

[0145] The attachment connection determination module receives human body part nodes, protective equipment nodes, and image frames that are adjacent in time sequence, and outputs the attachment connection edge status and the protective equipment node in the non-wearing state.

[0146] The verification evidence generation module receives the results of the human body surface area exposure determination, the results of the protective equipment node detection, and the status of the necessary attachment and connection edges, and outputs positive verification evidence, reverse verification evidence, evidence of improper wearing, or evidence to be verified.

[0147] The complementary image selection module receives the set of connections to be verified and candidate image frames, and outputs complementary image frames.

[0148] The verification result output module receives verification evidence from different perspectives and categories, as well as the updated set of connections to be verified, and outputs whether the wearing is compliant, the supplies are missing, the wearing is not standardized, or the evidence is insufficient.

[0149] Each module transmits data that includes at least the personnel trajectory identifier, verification time period identifier, image frame timestamp, category of protective equipment to be verified, human body part node identifier, protective equipment node identifier, and necessary attached connection edge identifier, in order to ensure the data correspondence when multiple people and multiple types of protective equipment are verified simultaneously.

[0150] Each module can be a logical processing unit running in the same data processing device, or it can be deployed separately in the edge computing device and remote processing server of the inspection equipment.

[0151] This invention does not judge the wearing status solely based on whether the protective equipment detection frame appears, nor does it simply vote on consecutive image frames. Instead, it uses the attachment and connection relationship formed when the protective equipment is correctly worn and the coverage relationship on the human body surface to control the generation of verification evidence.

[0152] Reverse verification evidence is generated only when the corresponding human body surface area meets the exposure determination conditions and the protective equipment node is not detected; for necessary attached connection edges that cannot be determined temporarily, complementary image frames are selected through the set of connections to be verified, and verification is completed using evidence from different perspective categories.

[0153] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for verifying the wearing of safety protective equipment based on inspection video, characterized in that, Includes the following steps: S1: Acquire the inspection video collected by the inspection equipment, perform cross-frame correlation on the personnel targets in the inspection video, and form personnel trajectories; S2: Determine the protective equipment to be verified based on the operational risk information corresponding to the personnel trajectory, and obtain the wearing topology template corresponding to the protective equipment to be verified. The wearing topology template includes human body part nodes, protective equipment nodes, attachment connection edges representing the wearing connection relationship between the protective equipment and the human body parts, and occlusion relationship edges representing the covering relationship formed by the protective equipment on the human body surface area under the correct wearing state. Among the attachment connection edges, the attachment connection edges that must be formed according to the correct wearing structure requirements are the necessary attachment connection edges. S3: For image frames in the personnel trajectory, based on the personnel posture, human body surface segmentation results and the observation direction of the inspection camera relative to the personnel target, determine whether the human body surface area associated with the occlusion relationship edge meets the exposure judgment condition. S4: Establish a local coordinate system for the human body based on the human body part nodes, and determine the connection status of necessary attached connection edges based on the spatial relationship between the protective equipment nodes and the human body part nodes and the cross-frame relative motion relationship. S5: Generate positive verification data when a protective equipment node is detected and the corresponding necessary attached connection edge is in a connected state; When no corresponding protective equipment node is detected in at least two temporally adjacent image frames, and the corresponding human body surface areas all meet the exposure determination conditions, reverse verification evidence is generated; when a protective equipment node is detected but at least one necessary attachment connection edge is not formed, evidence of improper wearing is generated; in other states, verification evidence is generated. S6: Establish a set of connections to be verified based on the necessary attached connection edges in the verification data. Select complementary image frames based on the expected visibility of the necessary attached connection edges in the candidate image frames, the expected exposure of the corresponding human body surface area, and the degree of overlap of observation view between the candidate image frames and the adopted image frames. S7: Integrate verification evidence from different perspectives and output verification results for compliance with wearing regulations, missing items, improper wearing, or insufficient evidence. Among these, missing items are determined by reverse verification evidence from at least two different perspectives.

2. The method for verifying the wearing of safety protective equipment based on inspection video according to claim 1, characterized in that, Identify operational risk information, including: Obtain the work task data corresponding to the inspection area, and extract the work area range and work risk category from the work task data; Identify high-risk objects from inspection videos and extract the time sequence of key human body points of personnel targets, the distance changes between personnel targets and high-risk objects, and the contact areas between human body parts and high-risk objects; Based on the time sequence, distance changes, contact area, and duration of contact at key points on the human body, the operational status of the personnel target relative to the risk object is determined; Based on the time interval of personnel entering the work area, the work risk category, risk object, and operation status are associated with the personnel trajectory. The protective equipment to be verified is determined according to the mapping relationship between the protection requirements of the work risk category, risk object, operation status and protective equipment category.

3. The method for verifying the wearing of safety protective equipment based on inspection video according to claim 2, characterized in that, Constructing a wearing topology template includes: Human body parts nodes are determined based on key human body points and human body surface segmentation results in correctly worn samples, and protective equipment nodes are determined based on the segmentation results of the components of the protective equipment to be verified. Based on the continuous spatial connection between the protective equipment nodes and human body part nodes in the correctly worn sample, an attachment connection edge is established, and the relative position allowable range, relative direction allowable range, and scale ratio allowable range are determined for each attachment connection edge. Establish occlusion relationships based on the coverage relationships formed by the nodes of protective equipment on the human body surface area in the correctly worn sample; Based on the correct wearing structure requirements of the protective equipment to be verified, the attachment connection edges that must be formed under the correct wearing condition are marked as necessary attachment connection edges.

4. The method for verifying the wearing of safety protective equipment based on inspection video according to claim 3, characterized in that, Determining whether a region of the human body surface meets the exposure criteria includes: Based on the person's posture, the human surface region associated with the occlusion relationship edge is projected onto the current image frame to obtain the surface region to be determined. Based on the segmentation results of other parts of the personnel, risk objects, scene objects and other personnel, determine the occlusion ratio of the surface area to be judged; Determine the effective pixel size, boundary integrity, local image clarity, and pixel ratio of the exposed human body surface of the surface region to be judged; When the occlusion ratio is not greater than the occlusion ratio threshold, the effective pixel size is not less than the verification size threshold, the boundary integrity is not less than the boundary integrity threshold, the local image clarity is not less than the clarity threshold, and the proportion of exposed human body surface pixels is not less than the exposure ratio threshold, the human body surface area is determined to meet the exposure judgment conditions. Each threshold is determined jointly based on the statistical distribution of the corresponding indicators in the labeled calibration images of the correct wearing state, the non-wearing state, and the occluded state.

5. The method for verifying the wearing of safety protective equipment based on inspection video according to claim 4, characterized in that, Determine the connection status of the necessary attached connection edges, including: The origin, reference direction, and scale reference of the local coordinate system of the human body are determined by at least two key human body points that constitute the nodes of the human body parts. The position, orientation, and size of the protective equipment nodes in sequentially adjacent image frames are transformed to the local coordinate system of the human body to obtain the relative position vector, relative orientation difference, and scale ratio of the protective equipment nodes relative to the human body parts nodes. When the relative position vector, relative direction difference, and scale ratio are all within the allowable range of the corresponding attached connection edge, and the changes in the aforementioned data in at least two temporally adjacent image frames are not greater than the stable change threshold, it is determined that the necessary attached connection edge is in a connected state. When a protective equipment node does not form an attachment connection edge with a human body part node defined by the wearing topology template, but forms an attachment connection edge with other human body part nodes of the personnel target, the protective equipment node is determined to be a non-wearing protective equipment node. The stable change threshold is determined based on the labeled calibration videos of the correct wearing state, handheld state, and adjacent placement state.

6. The method for verifying the wearing of safety protective equipment based on inspection video according to claim 5, characterized in that, The personnel trajectory is divided into verification periods based on the moment when the existence status of protective equipment nodes or the connection status of necessary attached connecting edges continuously changes, and verification evidence is generated in each verification period. When the status changes of the protective equipment node and its corresponding necessary attached connection edge continue to reach the status confirmation time, the current verification period ends and the next verification period is established. During the same verification period, when a necessary attachment connection edge obtains positive verification evidence or evidence of improper wearing, the necessary attachment connection edge is removed from the set of connections to be verified; when the protective equipment to be verified obtains negative verification evidence in at least two different view categories, the necessary attachment connection edge associated with the protective equipment to be verified is removed from the set of connections to be verified. If contradictory verification evidence appears within the same verification period and the conditions for switching verification periods are not met, the corresponding image frame will be recorded as verification evidence to be verified.

7. The method for verifying the wearing of safety protective equipment based on inspection video according to claim 6, characterized in that, Select complementary image frames, including: The expected visibility of the necessary attached connection edge is determined based on the proportion of the two nodes of the necessary attached connection edge falling within the image range, the expected non-occlusion proportion, the projection length of the connection direction, and the effective pixel spacing between the two nodes. The expected exposure level of the human body surface area is determined based on the projected area of ​​the corresponding human body surface area, the degree of orientation towards the inspection camera, and the expected non-obstruction ratio. The verification gain of candidate image frames is generated based on the expected visibility, expected exposure, effective pixel size of personnel targets, and local image sharpness. Based on the angle between the observation direction of the candidate image frame and the adopted image frame and the degree of overlap of the projection of the corresponding human body surface area, the degree of observation view repetition is generated. The difference between the normalized verification gain and the normalized observation view repetition is used as the frame selection score. Complementary image frames are selected from high to low according to the frame selection score, and the verification data generated by the complementary image frames is used to update the set of connections to be verified until the set of connections to be verified is empty or the personnel trajectory ends.

8. A safety protective equipment wearing verification system based on inspection video, applicable to the safety protective equipment wearing verification method based on inspection video as described in any one of claims 1-7, characterized in that, It includes a personnel trajectory construction module, a risk information processing module, a wearable topology acquisition module, a human body surface exposure determination module, an attachment connection determination module, a verification data generation module, a complementary image selection module, and a verification result output module; The personnel trajectory construction module is used to acquire inspection videos, perform cross-frame correlation on personnel targets, and form personnel trajectories. The risk information processing module is used to determine the protective equipment to be verified based on the operational risk information corresponding to the personnel's trajectory; The wear topology acquisition module is used to acquire the wear topology template corresponding to the protective equipment to be verified. The wear topology template includes human body part nodes, protective equipment nodes, attachment connection edges and occlusion relationship edges, and determines the necessary attachment connection edges from the attachment connection edges. The human body surface exposure determination module is used to determine whether the human body surface area associated with the occlusion relationship edge meets the exposure determination conditions based on the person's posture, the human body surface segmentation results, and the observation direction of the inspection camera relative to the person target. The attachment connection determination module is used to establish a local coordinate system of the human body based on the human body part nodes, and to determine the connection status of necessary attachment connection edges based on the spatial relationship between the protective equipment nodes and the human body part nodes and the cross-frame relative motion relationship. The verification evidence generation module is used to generate positive verification evidence, reverse verification evidence, evidence of improper wearing, or evidence to be verified based on the node detection results of protective equipment, the exposure status of the human body surface area, and the connection status of necessary attachment and connection edges. The complementary image selection module is used to establish a set of connections to be verified based on the necessary attached connection edges in the data to be verified, and to select complementary image frames based on the expected visibility of the necessary attached connection edges in the candidate image frames, the expected exposure of the corresponding human body surface area, and the degree of repetition of the observation view. The verification result output module is used to integrate verification evidence from different perspectives and output verification results for compliance with wearing regulations, missing items, improper wearing, or insufficient evidence. Among them, missing items are determined by reverse verification evidence from at least two different perspectives.

9. A safety protection equipment wearing verification system based on inspection video as described in claim 8, characterized in that: The risk information processing module includes a task data parsing unit, a video risk identification unit, and a protective equipment determination unit; The task data parsing unit is used to extract the work area range and work risk category from the work task data; The video risk identification unit is used to determine the operational status of the personnel relative to the risk object based on the timing of key human body points in the inspection video, the distance changes between the personnel target and the risk object, the contact area between the human body parts and the risk object, and the duration of the contact state. The protective equipment determination unit is used to determine the protective equipment to be verified based on the mapping relationship between the protective requirements of the operation risk category, the risk object, the operation status and the protective equipment category. The wearable topology acquisition module includes a human body node determination unit, a protective equipment node determination unit, and a topology relationship determination unit. The topology relationship determination unit is used to establish attachment connection edges and occlusion relationship edges, and to determine the necessary attachment connection edges according to the correct wearing structure requirements.

10. A safety protection equipment wearing verification system based on inspection video as described in claim 9, characterized in that: The human body surface exposure determination module outputs the exposure determination result of the human body surface area based on the occlusion ratio of the human body surface area, the effective pixel size, the boundary integrity, the local image clarity, and the proportion of exposed human body surface pixels. The attachment connection determination module outputs the connection status of necessary attachment connection edges based on the relative position vector, relative direction difference, scale ratio, and cross-frame change of the aforementioned data of the protective equipment node relative to the human body part node. The complementary image selection module generates a verification gain based on the expected visibility of the necessary attached connecting edge, the expected exposure of the corresponding human surface area, the effective pixel size of the personnel target, and the clarity of the local image. It generates the observation view repetition degree based on the observation direction angle between the candidate image frame and the adopted image frame and the degree of overlap of the projection of the human surface area. It then selects a complementary image frame based on the verification gain and the observation view repetition degree. The verification result output module updates the set of connections to be verified based on the verification evidence generated from complementary image frames, and outputs the verification results based on the updated set of connections to be verified and the verification evidence in different view categories.

Images