Method, system and device for detecting a safety belt for aerial work and storage medium

Abstract

The embodiment of the application discloses a kind of detection method, system, equipment and storage medium of aerial work safety belt, it is related to safety detection technical field, comprising: the visual recognition result and the magnetic sensing recognition result corresponding to the work site are acquired;Wherein, visual recognition result includes personnel presence state, safety belt wearing state and hook visual state;In the personnel presence state indicates that there is personnel in work scene image, and safety belt wearing state indicates that personnel wear safety belt, under the condition, according to hook visual state and magnetic sensing recognition result, the target state of hook is determined;According to the target state of hook and safety belt standard hooking condition, the safety belt hooking compliance detection result of personnel is determined.Combining visual recognition result and magnetic sensing recognition result are detected, even in the case where hook is shielded or visual miss detection, the state of hook can still be judged, and then the accuracy of the safety belt hooking compliance detection of high-altitude work is improved.

Description

Technical Field

[0001] This application relates to the field of safety detection technology, and in particular to a detection method, system, equipment and storage medium for safety belts used at heights. Background Technology

[0002] In high-altitude work scenarios, workers are usually required to wear safety belts and attach hooks to objects that comply with safety regulations to form an effective fall protection link and reduce the risk of falls from heights.

[0003] In related technologies, the compliance inspection of safety belt attachments for high-altitude operations typically relies on manual inspections. However, due to limitations in inspection frequency and site conditions, it is difficult to continuously and comprehensively monitor the operation process. With the development of visual perception technology, some solutions use target detection algorithms to identify hooks in operation images. However, because hooks are small and easily obscured, it is still difficult to accurately identify the attachment status of the hooks.

[0004] It is evident that the current inspection of the compliance of safety belt attachments for high-altitude operations suffers from low accuracy. Summary of the Invention

[0005] In view of this, one of the objectives of this application is to provide a method, system, device and storage medium for testing safety belts for high-altitude operations, which can improve the accuracy of testing the compliance of safety belt attachments for high-altitude operations.

[0006] To achieve the above objectives, the technical solution of this application is implemented as follows: In a first aspect, embodiments of this application provide a method for detecting safety belts used in high-altitude operations, the method comprising: The system acquires visual recognition and magnetic sensing recognition results corresponding to the work site. The visual recognition results are obtained by target detection of the work scene image. The visual recognition results include the presence status of personnel, the wearing status of safety belts, and the visual status of hooks. The visual status of hooks is used to characterize the connection relationship between the hook and the hooked object. Hooks include safety belt hooks and scaffold hooks. The magnetic sensing recognition results are obtained by inertial magnetic time series data collected by inertial magnetic sensors. The inertial magnetic sensors are installed on the safety belt hooks and scaffold hooks. The magnetic sensing recognition results include the hook action stage and the hooked object. When the personnel presence status indicates that there are personnel in the work scene image, and the safety belt wearing status indicates that the personnel are wearing safety belts, the target status of the hook is determined based on the visual status of the hook and the magnetic sensing recognition results. Based on the target status of the hook and the standard conditions for attaching the safety belt, determine the compliance test results of the personnel's safety belt attachment.

[0007] In one possible implementation, the visual recognition results include the presence status of the person and the seat belt wearing status; The visual recognition results are obtained by performing target detection on the work scene image, including: Perform target detection on the work scene image to obtain target detection results, which include at least one of the personnel detection box and the protective equipment detection box; In response to determining that the target detection results include personnel detection boxes and protective equipment detection boxes, the area coverage rate is determined based on the area of ​​the overlapping area between the protective equipment detection box and the personnel detection box, as well as the area of ​​the protective equipment detection box; and the personnel presence status is determined to indicate the presence of personnel in the work scene image. If the area coverage rate is greater than or equal to the preset coverage rate threshold, the seat belt wearing status is determined according to the protective equipment category corresponding to the protective equipment detection box.

[0008] In one possible implementation, determining the seatbelt wearing status based on the protective equipment category corresponding to the protective equipment detection frame includes: When the protective equipment category is the first preset category, the seat belt wearing status is determined to indicate that the person is wearing a seat belt; wherein, the first preset category includes any one of the seat belt category and the reflective vest and seat belt combination category; When the protective equipment category is the second preset category, the seat belt wearing status is determined to indicate that the person is not wearing a seat belt; wherein, the second preset category includes reflective clothing.

[0009] In one possible implementation, the visual state of the hook includes at least one of the following: a first engagement state between the scaffold hook and the wire rope, a second engagement state between the safety belt hook and the D-ring of the safety belt, and a third engagement state between the scaffold hook and the webbing of the safety belt; the first engagement state, the second engagement state, and the third engagement state correspond one-to-one with the wire rope, the D-ring of the safety belt, and the webbing of the safety belt in the engagement objects; Based on the visual state of the hook and the results of magnetic sensing recognition, the target state of the hook is determined, including: When the target attachment status indicates that an attachment relationship has been detected, the attachment object corresponding to the target attachment status is checked for consistency with the attachment object in the magnetic sensing identification result. The target attachment status is any one of the first attachment status, the second attachment status, and the third attachment status. In response to the fact that the hooked object corresponding to the target hooking state is consistent with the hooked object in the magnetic sensing identification result, the target state of the hook is determined to be the hooked object in the magnetic sensing identification result. In response to the discrepancy between the hooked object corresponding to the determined target hooking state and the hooked object in the magnetic sensing identification result, the target state of the hook is determined to be a state pending verification.

[0010] In one possible implementation, the safety belt attachment compliance test results are determined based on the target status of the hook and the safety belt attachment specifications, including: If the target status indicator of the scaffold hook is a wire rope and the target status indicator of the safety belt hook is the D-ring of the safety belt, the safety belt attachment compliance test result is determined to be compliant. If the target status indication of the scaffold hook is the webbing of the safety belt, the safety belt attachment compliance test result is determined to be non-compliant.

[0011] In one possible implementation, before determining the safety belt attachment compliance test results based on the target status of the hook and the safety belt attachment specifications, the method further includes: If the visual status indicator of the hook does not detect a connection between the hook and the object being hooked, the target status of the hook is determined based on the hook's action phase.

[0012] In one possible implementation, the hooking action phase includes at least one of the following: unhooked phase, hooking action phase, stable hooking phase, and unhooking action phase. Based on the hooking action phase, determine the target state of the hook, including: When the hooking action phase is a stable hooking phase, the hooking object in the magnetic sensing identification result is determined as the target state of the hook. If the hooking action phase is the unattached phase, the target state of the hook is determined to be the unattached state; When the hooking action phase is either the hanging action phase or the unhooking action phase, the target state of the hook is determined as the action process state.

[0013] In one possible implementation, the safety belt attachment compliance test results are determined based on the target status of the hook and the safety belt attachment specifications, including: If the target status of the hook is not attached, the result of the seat belt attachment compliance test is determined to be non-compliant.

[0014] In one possible implementation, the magnetic sensing identification result is obtained based on the inertial magnetic time series data collected by the inertial magnetic sensor, including: Feature extraction is performed on the inertial magnetic time series data collected by the inertial magnetic sensor to obtain the window features corresponding to the hook. The window features include at least one of acceleration features, angular velocity features and angle features. Based on the window features corresponding to the hook, determine the hooking action stage of the hook; When the hook action phase is a stable hooking phase, the hooking object is identified based on the window features and magnetic field features corresponding to the hook, and the hooking object is obtained. The magnetic field features are extracted from the magnetic field data in the inertial magnetic time series data. The magnetic sensing recognition result is determined based on the hooking action stage and the object being hooked.

[0015] In one possible implementation, feature extraction is performed on the inertial magnetic time-series data acquired by the inertial magnetic sensor to obtain the window features corresponding to the hook, including: Preprocessing of inertial magnetic time series data includes at least one of timestamp alignment, outlier removal, and data smoothing. The preprocessed inertial magnetic time series data is sliced ​​into sliding slices according to a preset window length to obtain at least two sliding windows. Statistical features were extracted from the acceleration, angular velocity, and angle data within each sliding window to obtain the window features of each sliding window.

[0016] In one possible implementation, the hooking action phase of the hook is determined based on the window features corresponding to the hook, including: The window features corresponding to the hook are input into the first preset classification model to obtain the hook action stage of the hook output by the first preset classification model; wherein, the first preset classification model is trained based on the first inertial magnetic sample data labeled with action stage labels; Based on the window and magnetic field characteristics corresponding to the hook, the hook's attachment object is identified, resulting in the hook's attachment object, including: Input the window features and magnetic field features corresponding to the hook into the second preset classification model to obtain the hook's attachment object output by the second preset classification model. The second preset classification model is trained based on the second inertial magnetic sample data. The second inertial magnetic sample data is the inertial magnetic sample data with the action stage label being the stable attachment stage and labeled with the attachment object label.

[0017] In one possible implementation, before determining the safety belt attachment compliance test results based on the target status of the hook and the safety belt attachment specifications, the method further includes: If no personnel are present in the work scene image indicating the presence of personnel, the safety belt attachment compliance test will not be performed on the work scene image. If a person is present in the work scene image and the safety belt wearing status indicator shows that the person is not wearing a safety belt, the safety belt attachment compliance test result is determined to be non-compliant.

[0018] Secondly, embodiments of this application provide a detection system for safety belts used in high-altitude operations, the system comprising: The acquisition module is used to acquire visual recognition results and magnetic sensing recognition results corresponding to the work site. The visual recognition results are obtained by target detection of the work scene image. The visual recognition results include the personnel presence status, safety belt wearing status, and hook visual status. The hook visual status is used to characterize the hooking relationship between the hook and the hooked object. The hooks include safety belt hooks and scaffolding hooks. The magnetic sensing recognition results are obtained by inertial magnetic time series data collected by inertial magnetic sensors. The inertial magnetic sensors are installed on the safety belt hooks and scaffolding hooks. The magnetic sensing recognition results include the hook action stage and the hooked object. The first determining module is used to determine the target state of the hook based on the visual state of the hook and the magnetic sensing recognition result when the personnel presence status indicates that there are personnel in the work scene image and the safety belt wearing status indicates that the personnel are wearing safety belts. The second determination module is used to determine the compliance test results of the personnel's seat belt attachment based on the target status of the hook and the standard attachment conditions of the seat belt.

[0019] Thirdly, embodiments of this application provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the method provided in the first aspect.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by one or more processors, implements the method provided in the first aspect.

[0021] Fifthly, embodiments of this application provide a computer program product, which includes a computer program that, when executed by one or more processors, implements the method provided in the first aspect.

[0022] The high-altitude work safety belt detection method provided in this application can acquire visual recognition results obtained from target detection based on work scene images, and magnetic sensing recognition results obtained from inertial magnetic time-series data collected by an inertial magnetic sensor installed on the hook. After confirming the presence of personnel and that they are wearing safety belts, the method determines the target state of the hook based on the visual state of the hook and the magnetic sensing recognition results, thereby determining the safety belt attachment compliance detection result. Since the visual recognition results can provide visual semantic information such as the presence of personnel, safety belt wearing, and the connection relationship between the hook and the attached object, and the magnetic sensing recognition results can provide information on the hook's action stage and the attached object from the hook's own perspective, combining the visual recognition results and the magnetic sensing recognition results for detection allows for hook state judgment based on the inertial magnetic time-series data of the hook body even when the hook is obscured or visually missed, thus improving the accuracy of high-altitude work safety belt attachment compliance detection. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. It should be understood that the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] The attached diagram is described below: Figure 1 A schematic diagram of the architecture of the detection system for safety belts used in high-altitude operations provided in an embodiment of this application; Figure 2 One of the wearing diagrams involved in the detection method of a safety belt for high-altitude operations provided in this application embodiment; Figure 3 A flowchart illustrating the detection method for safety belts used in high-altitude operations provided in this application embodiment; Figure 4 The second schematic diagram of wearing a safety belt for high-altitude operations provided in this application embodiment; Figure 5 A functional module diagram of a high-altitude work safety belt detection system provided in an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0025] Explanation of reference numerals in the attached figures: 500. Detection system for safety belts used in high-altitude operations; 510. Acquisition Module; 520. First Determined Module; 530. Second Determination Module; 601. Processor; 602. Memory; 603. Communication interface; 610. Bus. Detailed Implementation

[0026] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0027] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0028] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a high-altitude work safety belt detection system provided in an embodiment of this application. Figure 1 As shown, the architecture of the high-altitude work safety belt detection system includes a computer vision recognition module, an inertial magnetic time sequence recognition module, and a vision and magnetic sensing fusion decision module. The computer vision recognition module performs target detection on the work scene image and outputs visual recognition results; the inertial magnetic time sequence recognition module identifies the inertial magnetic time sequence data collected by the inertial magnetic sensor and outputs magnetic sensing recognition results; the vision and magnetic sensing fusion decision module determines the target state of the hook based on the visual and magnetic sensing recognition results, and further determines the safety belt attachment compliance detection results.

[0029] The aforementioned computer vision recognition module may include an image acquisition device for acquiring images of the work scene at the work site. The image acquisition device may be a camera mounted on a drone, capturing images of the dispersed work area via drone patrol; or it may be a monitoring camera installed at the work site, continuously capturing images of a fixed work area. This application does not limit the specific form of the image acquisition device.

[0030] The aforementioned inertial magnetic timing identification module may include an inertial magnetic sensor, which can be installed on the hook to collect inertial magnetic timing data during the hook's use. In some embodiments, the inertial magnetic sensor may include a first inertial magnetic sensor and a second inertial magnetic sensor. The first inertial magnetic sensor is installed on the outside of the safety belt hook, and the second inertial magnetic sensor is installed on the outside of the scaffold hook, thereby collecting inertial magnetic timing data for two types of hooks with different functions, respectively. For example, the aforementioned inertial magnetic sensor may be a ten-axis inertial magnetic sensor, which outputs three-axis acceleration data, three-axis angular velocity data, three-axis angle data, and three-axis magnetic field data via TTL communication. It should be noted that the inertial magnetic sensor is externally mounted, without altering the original mechanical structure of the hook, facilitating deployment on existing safety equipment.

[0031] The aforementioned visual and magnetic sensing fusion decision module can be connected to the image acquisition device and the inertial magnetic sensor respectively to obtain visual recognition results and magnetic sensing recognition results, and perform fusion judgment and compliance detection.

[0032] For the safety belt hooks and scaffold hooks mentioned above, please refer to [link / reference]. Figure 2 , Figure 2 This is one of the schematic diagrams illustrating the wearing of a safety belt used in a high-altitude work safety belt detection method provided in this application embodiment. Figure 2 As shown, the workers are wearing safety belts, with the safety belt hooks attached to the D-rings on the safety belts, and the scaffolding hooks attached to the wire ropes. The wire ropes can be anchored to load-bearing objects, such as wire ropes with a diameter greater than 8 mm. In actual operation, some workers may, for convenience, attach the scaffolding hooks to non-load-bearing objects such as the webbing of their own safety belts. While the hooks are technically attached, this does not form an effective fall protection link, posing a significant fall hazard. Therefore, continuous and accurate monitoring of the workers' safety belt wearing and hook attachment is necessary.

[0033] The following will provide a detailed description of the detection method for high-altitude work safety belts provided in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a flowchart illustrating a method for detecting safety belts for working at heights provided in an embodiment of this application. The method can be implemented using the safety belt detection system or electronic equipment described in the following embodiments. The method for detecting safety belts for working at heights includes steps 110 to 130.

[0034] Step 110: Obtain the visual recognition results and magnetic sensing recognition results corresponding to the work site; wherein, the visual recognition results are obtained by target detection of the work scene image, and the visual recognition results include the personnel presence status, safety belt wearing status, and hook visual status. The hook visual status is used to characterize the hooking relationship between the hook and the hooked object. The hooks include safety belt hooks and scaffold hooks; the magnetic sensing recognition results are obtained by inertial magnetic time series data collected by inertial magnetic sensors. The inertial magnetic sensors are installed on the safety belt hooks and scaffold hooks. The magnetic sensing recognition results include the hook action stage and the hooked object.

[0035] In this step: The aforementioned work site refers to the area where workers perform high-altitude operations, such as scaffolding work surfaces, cantilevered platforms, or areas near openings or edges in a construction site.

[0036] The visual recognition results corresponding to the above-mentioned work site refer to the results obtained by visually recognizing the work scene images obtained by taking pictures of the work site.

[0037] The magnetic sensing identification results corresponding to the above-mentioned work site refer to the results obtained by identifying the data collected by the inertial magnetic sensor installed on the hook of the safety belt worn by the worker at the work site.

[0038] The aforementioned visual recognition results are obtained by performing target detection on the work scene image. Target detection refers to the process of identifying targets of a preset category in an image and determining their location. For example, target detection can obtain the target's category and its bounding box. Exemplarily, the YOLOv26 target detection model can be used to perform target detection on the work scene image, or other models with target detection capabilities can be used. This application embodiment does not limit the specific type of target detection model.

[0039] The aforementioned personnel presence status can be used to indicate whether personnel are present in the work scene image. For example, the personnel presence status can be represented as Person_State, where Person_State is 1 when personnel are detected in the work scene image, and 0 when no personnel are detected in the work scene image.

[0040] The aforementioned seatbelt wearing status can be used to indicate whether a person in a work scene image is wearing a seatbelt. For example, the seatbelt wearing status may include wearing reflective clothing, wearing both reflective clothing and a seatbelt, or simply wearing a seatbelt.

[0041] The aforementioned visual status of hooks can be used to characterize the attachment relationship between the hook and the attached object. The visual status of a hook does not merely indicate the presence of a hook in the image, but rather indicates the type of attachment relationship between the hook and the object it is attached to. These hooks include safety belt hooks and scaffolding hooks. Specifically, a safety belt hook is a hook connected to one side of the safety belt for attaching to a D-ring on the safety belt; a scaffolding hook is a hook connected to one end of the safety rope for attaching to anchoring objects such as wire ropes on the scaffolding side.

[0042] The aforementioned attachment objects include wire ropes, D-rings of safety belts, and webbing of safety belts. For example, the visual state of the hook can be represented as Visual_Hook_State, which can include the wire rope attachment state of the scaffold hook, the D-ring attachment state of the safety belt hook, and the safety belt webbing attachment state of the scaffold hook itself. Each state can be represented using a binary method, with a value of 1 indicating that the corresponding state is detected, and a value of 0 indicating that the corresponding state is not detected.

[0043] The above magnetic sensing identification results are obtained based on the inertial magnetic time-series data collected by the inertial magnetic sensor. Inertial magnetic time-series data refers to a data sequence continuously collected by the inertial magnetic sensor at a fixed sampling frequency and containing time information. For example, the data output by the inertial magnetic sensor at any sampling moment may include triaxial acceleration data, triaxial angular velocity data, triaxial angle data, and triaxial magnetic field data, which can be represented as S(t), a data set consisting of ax(t), ay(t), az(t), gx(t), gy(t), gz(t), roll(t), pitch(t), yaw(t), mx(t), my(t), and mz(t). Here, ax(t), ay(t), and az(t) are triaxial acceleration data; gx(t), gy(t), and gz(t) are triaxial angular velocity data; roll(t), pitch(t), and yaw(t) are triaxial angle data; and mx(t), my(t), and mz(t) are triaxial magnetic field data.

[0044] The aforementioned magnetic sensing identification results include the hook action stage and the attached object. The hook action stage characterizes the current stage of the hook's action process, such as the unattached stage, the attachment stage, the stable attachment stage, and the unhooking stage. The attached object refers to the object attached when the hook is stably attached, such as a wire rope, a D-ring of a safety belt, or the webbing of a safety belt. For example, the magnetic sensing identification result can be represented as a result set consisting of the hook action stage, the attached object, and time information. For instance, the magnetic sensing identification result Sensor_Result can be represented as Sensor_Result = {Action_State, Object_State, Time}, that is, a set consisting of Action_State, Object_State, and Time.

[0045] Wherein, Action_State represents the current hooking action stage; Object_State represents the hooking object in the stable hooking state, including scaffold hook wire rope hooking, safety belt hook D-ring hooking, and scaffold hook own safety belt webbing hooking; Time represents the timestamp corresponding to the recognition result.

[0046] The aforementioned acquisition of visual recognition and magnetic sensing recognition results corresponding to the work site can be achieved either by electronic equipment receiving data transmitted from the image acquisition device and the inertial magnetic sensor in real time and then recognizing the data, or by electronic equipment retrieving already recognized results from the storage device according to a preset cycle. Both visual recognition and magnetic sensing recognition results can carry time information.

[0047] Step 120: If there are personnel in the work scene image indicating the presence of personnel and the safety belt wearing status indicates that the personnel are wearing safety belts, determine the target status of the hook based on the visual status of the hook and the magnetic sensing recognition results.

[0048] In this step: The presence of personnel in the work scene image, as indicated by the aforementioned personnel status indicator, and the wearing of safety belts, are prerequisites for conducting hook-up compliance checks. It is only meaningful to check the hook-up status of personnel if they are present in the work scene and wearing safety belts.

[0049] The target state of the hook mentioned above refers to the hook state determined after integrating the visual state and magnetic sensing recognition results. The target state of the hook can be used to indicate the object to which the hook is attached, or to indicate whether the hook is in an unattached state, in the process of operation, or in a state pending verification.

[0050] Step 130: Determine the compliance test results of the personnel's safety belt attachment based on the target status of the hook and the standard attachment conditions of the safety belt.

[0051] In this step: The aforementioned safety belt attachment conditions refer to the hook attachment conditions determined according to the safety belt usage specifications for working at heights. For example, the safety belt attachment conditions may include the scaffold hook being attached to the wire rope, and the safety belt hook being attached to the D-ring of the safety belt. When the target condition of the hook meets the safety belt attachment conditions, the safety belt attachment compliance test result can be determined to be compliant; when the target condition of the hook does not meet the safety belt attachment conditions, the safety belt attachment compliance test result can be determined to be non-compliant.

[0052] In addition to compliance and non-compliance, the above-mentioned safety belt attachment compliance test results may also include cases where no test results are output. For example, if there are no personnel in the work scene image, the safety belt attachment compliance test will not be performed; if the hook is in the process of being attached or unhooked, the test results may be temporarily not output.

[0053] It should be noted that the detection method for safety belts used in high-altitude operations provided in this application embodiment can be applied to scenarios involving high-altitude operations, such as building construction, bridge construction, and power maintenance, including scaffolding operations, cantilever platform operations, operations near openings, tower crane operations, and facade work. In such scenarios, the working space is high, the working surface is dispersed, and the environment changes rapidly. Workers need to wear safety belts and complete the hook connection according to the safety belt specification requirements to form an effective fall protection link.

[0054] The high-altitude work safety belt detection method provided in this application can acquire visual recognition results obtained from target detection based on work scene images, and magnetic sensing recognition results obtained from inertial magnetic time-series data collected by an inertial magnetic sensor installed on the hook. After confirming the presence of personnel and that they are wearing safety belts, the method determines the target state of the hook based on the visual state of the hook and the magnetic sensing recognition results, thereby determining the safety belt attachment compliance detection result. Since the visual recognition results can provide visual semantic information such as the presence of personnel, safety belt wearing, and the connection relationship between the hook and the attached object, and the magnetic sensing recognition results can provide information on the hook's action stage and the attached object from the hook's own perspective, combining the visual recognition results and the magnetic sensing recognition results for detection allows for hook state judgment based on the inertial magnetic time-series data of the hook body even when the hook is obscured or visually missed, thus improving the accuracy of high-altitude work safety belt attachment compliance detection.

[0055] In one possible implementation, the visual recognition results include the presence status of the person and the seat belt wearing status; The visual recognition results are obtained by performing target detection on the work scene image, including: Perform target detection on the work scene image to obtain target detection results, which include at least one of the personnel detection box and the protective equipment detection box; In response to determining that the target detection results include personnel detection boxes and protective equipment detection boxes, the area coverage rate is determined based on the area of ​​the overlapping area between the protective equipment detection box and the personnel detection box, as well as the area of ​​the protective equipment detection box; and the personnel presence status is determined to indicate the presence of personnel in the work scene image. If the area coverage rate is greater than or equal to the preset coverage rate threshold, the seat belt wearing status is determined according to the protective equipment category corresponding to the protective equipment detection box.

[0056] The personnel detection boxes mentioned above refer to the detection boxes in the target detection results that are categorized as "personnel". Please refer to [link / reference]. Figure 4 , Figure 4 This is the second schematic diagram of the wearing method involved in the detection method of a safety belt for high-altitude operations provided in this application embodiment. Figure 4 In (a) of the text, the personnel category can be represented as person, i.e., worker.

[0057] The aforementioned protective equipment detection boxes refer to the detection boxes categorized as personal protective equipment in the target detection results. Figure 4 In (b) of the figure, the categories of personal protective equipment can include three types: reflective vests, reflective vests and safety belts, which correspond to the three example images in the figure.

[0058] The aforementioned area coverage ratio can be used to characterize the extent to which the protective equipment detection frame falls within the personnel detection frame. For example, the area coverage ratio can be expressed as IoU, and its calculation method is as follows: IoU=Area(B_ppe∩B_person) / Area(B_ppe).

[0059] Where B_ppe represents the protective equipment detection frame, B_person represents the personnel detection frame, Area(B_ppe∩B_person intersection) represents the area of ​​the overlapping region between the protective equipment detection frame and the personnel detection frame, and Area(B_ppe) represents the area of ​​the protective equipment detection frame.

[0060] In some embodiments, the preset coverage threshold can be set to 0.8, that is, when more than 80% of the area of ​​the protective equipment detection frame overlaps with the personnel detection frame, the protective equipment is considered to be actually worn by the personnel.

[0061] If the area coverage rate is greater than or equal to the preset coverage rate threshold, it indicates that the protective equipment corresponding to the protective equipment detection frame is actually worn by the person. In this case, the seat belt wearing status is determined according to the type of protective equipment corresponding to the protective equipment detection frame. If protective equipment is detected but the area coverage rate is lower than the preset coverage rate threshold, or if the protective equipment detection frame is not detected, it can be determined that a valid protective equipment wearing status has not been detected.

[0062] In some embodiments, the area coverage rate can also be calculated in other ways. For example, the coverage rate can be the ratio of the area of ​​the overlapping area to the area of ​​the personnel detection frame, or the protective equipment can be determined by whether the center point of the protective equipment detection frame is located within the personnel detection frame. This application embodiment does not limit this.

[0063] In this embodiment, the area coverage between the protective equipment detection frame and the personnel detection frame is used to determine whether the protective equipment is actually worn by the personnel. This can exclude equipment that is hung in other locations and equipment that is falsely detected in the background, so that the determination of the seat belt wearing status is based on the premise that the protective equipment is actually worn by the personnel, thus improving the accuracy of the wearing status determination.

[0064] In one possible implementation, determining the seatbelt wearing status based on the protective equipment category corresponding to the protective equipment detection frame includes: When the protective equipment category is the first preset category, the seat belt wearing status is determined to indicate that the person is wearing a seat belt; wherein, the first preset category includes any one of the seat belt category and the reflective vest and seat belt combination category; When the protective equipment category is the second preset category, the seat belt wearing status is determined to indicate that the person is not wearing a seat belt; wherein, the second preset category includes reflective clothing.

[0065] Since reflective vests do not provide fall protection, and personnel wearing only reflective vests do not meet the requirements for hook-attachment testing, reflective vests are classified as the second preset category, while safety belts and combinations of reflective vests and safety belts are classified as the first preset category.

[0066] Please see Figure 4 In (b), the safety harness category can be represented as: safety harness, the combination of reflective vest and safety harness can be represented as: safety vest and harness, and the reflective vest category can be represented as safety vest.

[0067] For example, if a protective equipment detection frame corresponding to the seat belt category is detected and the area coverage is greater than or equal to a preset coverage threshold, it is determined that the person is wearing a seat belt; if a protective equipment detection frame corresponding to the combination category of reflective vest and seat belt is detected at the same time and the area coverage is greater than or equal to a preset coverage threshold, it is determined that the person is wearing both reflective vest and seat belt; if only a protective equipment detection frame corresponding to the reflective vest category is detected, it is determined that the person is not wearing a seat belt.

[0068] In some embodiments, the protective equipment category can be expanded or merged according to actual management needs. For example, a safety helmet category can be further set up to simultaneously detect whether personnel are wearing safety helmets. This application embodiment does not limit this.

[0069] In this embodiment of the application, by dividing the protective equipment into a first preset category and a second preset category, and corresponding to the two states of wearing a seat belt and not wearing a seat belt respectively, the two situations of wearing only reflective clothing and wearing a seat belt can be accurately distinguished, avoiding misjudging the wearing state that does not have the ability to prevent falls as having the judgment conditions for hook attachment detection.

[0070] In one possible implementation, the visual state of the hook includes at least one of the following: a first engagement state between the scaffold hook and the wire rope, a second engagement state between the safety belt hook and the D-ring of the safety belt, and a third engagement state between the scaffold hook and the webbing of the safety belt; the first engagement state, the second engagement state, and the third engagement state correspond one-to-one with the wire rope, the D-ring of the safety belt, and the webbing of the safety belt in the engagement objects; Based on the visual state of the hook and the results of magnetic sensing recognition, the target state of the hook is determined, including: When the target attachment status indicates that an attachment relationship has been detected, the attachment object corresponding to the target attachment status is checked for consistency with the attachment object in the magnetic sensing identification result. The target attachment status is any one of the first attachment status, the second attachment status, and the third attachment status. In response to the fact that the hooked object corresponding to the target hooking state is consistent with the hooked object in the magnetic sensing identification result, the target state of the hook is determined to be the hooked object in the magnetic sensing identification result. In response to the discrepancy between the hooked object corresponding to the determined target hooking state and the hooked object in the magnetic sensing identification result, the target state of the hook is determined to be a state pending verification.

[0071] Please see Figure 4 In (c), the first hooking state can be represented as fastened hook1, which is the hooking state of the scaffold hook wire rope; the second hooking state can be represented as fastened hook2, which is the hooking state of the safety belt hook D-ring; and the third hooking state can be represented as fastened hook3, which is the hooking state of the scaffold hook itself safety belt webbing.

[0072] The aforementioned one-to-one correspondence refers to the following: the first hooking state corresponds to the wire rope in the hooked object, the second hooking state corresponds to the D-ring of the safety belt in the hooked object, and the third hooking state corresponds to the webbing of the safety belt in the hooked object. During consistency verification, the hooked object corresponding to the target hooking state is compared with the hooked object in the magnetic sensing identification result according to this correspondence. If they match, the result is highly reliable, and the hooked object in the magnetic sensing identification result can be identified as the target state of the hook. If they do not match, it indicates that at least one identification method may have misjudged the situation. In this case, it is not advisable to directly output a conclusion, and the target state of the hook can be identified as a state awaiting verification.

[0073] For example, if a first attachment state is detected in the work scene image, that is, if the visual recognition result indicates that the scaffold hook is attached to the wire rope, and the magnetic sensing recognition result indicates that the attachment object is also the wire rope, then the target state of the scaffold hook is determined to be attached to the wire rope. If the visual recognition result indicates that the scaffold hook is attached to the wire rope, but the magnetic sensing recognition result indicates that the attachment object is the webbing of the safety belt, then the target state of the scaffold hook is determined to be a state pending verification.

[0074] In this embodiment, by setting three attachment states that correspond one-to-one with the wire rope, D-ring, and webbing, and verifying the consistency between the target attachment state indicating the detected attachment relationship and the magnetic sensing recognition result, erroneous output caused by misjudgment of a single recognition result can be avoided.

[0075] In one possible implementation, the safety belt attachment compliance test results are determined based on the target status of the hook and the safety belt attachment specifications, including: If the target status indicator of the scaffold hook is a wire rope and the target status indicator of the safety belt hook is the D-ring of the safety belt, the safety belt attachment compliance test result is determined to be compliant. If the target status indication of the scaffold hook is the webbing of the safety belt, the safety belt attachment compliance test result is determined to be non-compliant.

[0076] In some embodiments, if the test result for seat belt attachment compliance is determined to be non-compliant, the processing device may also send an alarm message to the on-site safety management terminal so that safety management personnel can take timely action.

[0077] In this embodiment of the application, by attaching the double hooks to the wire rope and the D-ring respectively for compliance determination, the common and concealed violation of attaching scaffold hooks to one's own safety belt webbing can be accurately identified, thereby improving the accuracy of compliance detection results.

[0078] In some embodiments, determining the safety belt attachment compliance test results based on the target state of the hook and the safety belt attachment specifications further includes: If the target status of the hook is pending verification, the safety belt attachment compliance test result is not output. Instead, the process returns to the step of obtaining visual recognition results and magnetic sensing recognition results, so as to verify the hook attachment status based on the re-obtained visual recognition results and magnetic sensing recognition results.

[0079] The aforementioned pending verification status indicates a conflict between the visual recognition result and the magnetic sensing recognition result. This conflict may stem from false detections in visual recognition, misjudgments by the sensor, or the hook being in a transient motion state. In this case, no detection result is output; instead, the system returns to reacquire both types of recognition results for verification. For example, a consistency check can be performed again on the reacquired results. If the reacquired results are consistent, the target state of the hook is determined based on the consistent results.

[0080] In some embodiments, if the target state of the hook is determined to be in a state pending review after a preset number of consecutive times, the processing device can mark the hook as an abnormal hook and send a prompt message to the on-site safety management terminal to prompt the safety management personnel to conduct manual verification.

[0081] In one possible implementation, before determining the safety belt attachment compliance test results based on the target status of the hook and the safety belt attachment specifications, the method further includes: If the visual status indicator of the hook does not detect a connection between the hook and the object being hooked, the target status of the hook is determined based on the hook's action phase.

[0082] The aforementioned visual indication of no hook connection detected refers to a situation where no hook connection is detected in the first, second, and third hook connection states. This could be caused by factors such as the hook being too small, the hook being located behind or to the side of a person and thus obscured by their body, the hook being obscured by scaffolding components, or the hook being in rapid motion causing image blurring. In this case, the electronic device can determine the target state of the hook based on the hook's movement stage in the magnetic sensing recognition results.

[0083] In this embodiment, when the visual state of the hook is unclear, it is not directly determined that the hook is not hooked. Instead, the true state of the hook is judged based on the hook action stage, so that scenarios where visual perception fails, such as obstruction or missed detection, can still be accurately judged.

[0084] In one possible implementation, the hooking action phase includes at least one of the following: unhooked phase, hooking action phase, stable hooking phase, and unhooking action phase. Based on the hooking action phase, determine the target state of the hook, including: When the hooking action phase is a stable hooking phase, the hooking object in the magnetic sensing identification result is determined as the target state of the hook. If the hooking action phase is the unattached phase, the target state of the hook is determined to be the unattached state; When the hooking action phase is either the hanging action phase or the unhooking action phase, the target state of the hook is determined as the action process state.

[0085] For example, the hooking action phase can also be divided according to the hook type, such as the following eight categories: safety belt hook not attached, scaffold hook not attached, safety belt hook attached, scaffold hook attached, safety belt hook stably attached, scaffold hook stably attached, safety belt hook unhooked, and scaffold hook unhooked, which can be represented by labels 1 to 8 respectively.

[0086] When the hook is in a stable engagement phase during the hooking action phase, the hook is in a reliable engagement state, and the engagement object in the magnetic sensor identification result accurately reflects the object being hooked. Therefore, this engagement object is determined as the target state of the hook. When the hook is in an unengaged phase during the hooking action phase, the target state of the hook is determined as the unengaged state. When the hook is in the engagement or disengagement phase, the contact between the hook and the engagement object may be instantaneous or temporary. In this case, the target state of the hook is determined as the action process state, and the engagement status is not directly determined.

[0087] In this embodiment of the application, by judging the hook state when the visual condition is unclear during the hooking action stage, it is possible to distinguish between the actual unhooked state and the dynamic operation process, and avoid misjudging the momentary contact during the hooking process or the temporary contact during the unhooking process as hooked or unhooked, thereby improving the accuracy and reliability of the target state determination.

[0088] In one possible implementation, the safety belt attachment compliance test results are determined based on the target status of the hook and the safety belt attachment specifications, including: If the target status of the hook is not attached, the result of the seat belt attachment compliance test is determined to be non-compliant.

[0089] The target state of the aforementioned hooks is "not attached," indicating that the hooks are not attached to any object, and can be directly determined as non-compliant. For example, if the target state of a safety belt hook or a scaffolding hook is "not attached," the safety belt attachment compliance test result can be determined as non-compliant.

[0090] In some embodiments, when the target state of the hook is in the action process state, the electronic device may not output the seat belt hooking compliance detection result, and return to the step of obtaining visual recognition result and magnetic sensing recognition result, so as to continuously detect the hook in the hooking action process or the unhooking action process, and output the corresponding detection result after the hook enters the stable hooking stage or the unhooking stage.

[0091] In this embodiment of the application, hooks that are confirmed not to be attached are directly judged as non-compliant, while hooks in the process of operation are continuously monitored, which not only ensures the identification of real violations, but also avoids false alarms for normal attachment operations.

[0092] In one possible implementation, the magnetic sensing identification result is obtained based on the inertial magnetic time series data collected by the inertial magnetic sensor, including: Feature extraction is performed on the inertial magnetic time series data collected by the inertial magnetic sensor to obtain the window features corresponding to the hook. The window features include at least one of acceleration features, angular velocity features and angle features. Based on the window features corresponding to the hook, determine the hooking action stage of the hook; When the hook action phase is a stable hooking phase, the hooking object is identified based on the window features and magnetic field features corresponding to the hook, and the hooking object is obtained. The magnetic field features are extracted from the magnetic field data in the inertial magnetic time series data. The magnetic sensing recognition result is determined based on the hooking action stage and the object being hooked.

[0093] The aforementioned window features refer to features extracted from inertial magnetic timing data segments of a set time length. Acceleration features can be used to reflect changes in the motion intensity of the hook during the processes of picking up, moving, colliding, hooking in, and unhooking; angular velocity features can be used to reflect changes in the rotation of the hook during the processes of rotating, flipping, hooking in, and unhooking; and angle features can be used to reflect changes in the hook's attitude and its degree of stability.

[0094] For example, when the peak value, standard deviation, and range of acceleration are large, it usually indicates that the hook is in the process of moving, attaching, or unattaching; when the range of acceleration and standard deviation are small, it usually indicates that the hook is in an unattached, stationary state or a stable attachment stage; when the peak value and range of angular velocity are large, it usually indicates that the hook is rotating, attaching, or unattaching; when the range of angular velocity is small and the duration is long, it usually indicates that the hook may have entered a stable attachment stage.

[0095] The aforementioned magnetic field characteristics can be used to reflect changes in the magnetic field caused by the surrounding metallic environment and the object being hooked. Steel wire ropes and D-rings are both metallic objects; when a hook approaches or is hooked onto such objects, the surrounding magnetic field changes accordingly. However, the webbing of a safety harness does not possess obvious metallic characteristics, and the magnetic field changes when hooked onto the webbing differ significantly from those when hooked onto a steel wire rope or D-ring. Therefore, magnetic field characteristics can primarily be used to distinguish between different hooked objects.

[0096] It should be noted that during the attaching and unattaching phases, there may be momentary contact between the hook and the attached object, and the hook is in a transient state of motion. Under these circumstances, identifying the attached object can easily lead to misjudgments. However, during the stable attachment phase, the relative positions of the hook and the attached object are fixed, the magnetic field characteristics are stable, and the identification results are more reliable. Therefore, attached object identification should only be performed when the hook action phase is the stable attachment phase.

[0097] In some embodiments, the inertial magnetic sensor may also be a nine-axis or ten-axis sensor capable of outputting acceleration, angular velocity, attitude angle and magnetic field data, or a combined module consisting of an accelerometer, a gyroscope and a magnetometer. The specific model of the inertial magnetic sensor is not limited in the embodiments of this application.

[0098] In this embodiment, by first identifying the hook action stage based on acceleration characteristics, angular velocity characteristics, and angle characteristics, and only combining magnetic field characteristics to identify the hooked object during the stable hooking stage, it is possible to first determine whether the hook is stable and then determine the hooked object, so that the identification of the hooked object avoids transient motion. At the same time, combining magnetic field characteristics can make up for the insufficiency of relying solely on motion characteristics to distinguish between wire rope, D-ring, and webbing.

[0099] In one possible implementation, feature extraction is performed on the inertial magnetic time-series data acquired by the inertial magnetic sensor to obtain the window features corresponding to the hook, including: Preprocessing of inertial magnetic time series data includes at least one of timestamp alignment, outlier removal, and data smoothing. The preprocessed inertial magnetic time series data is sliced ​​into sliding slices according to a preset window length to obtain at least two sliding windows. Statistical features were extracted from the acceleration, angular velocity, and angle data within each sliding window to obtain the window features of each sliding window.

[0100] The timestamp alignment mentioned above can be used to place acceleration data, angular velocity data, angle data, and magnetic field data in the same time series; outlier removal can be used to remove data points with communication abnormalities or obvious unreasonableness; data smoothing can be used to reduce the impact of high-frequency noise.

[0101] The preset window length can be determined based on the sampling frequency of the inertial magnetic sensor and the duration of the hanging action. For example, when the sampling frequency is 100 Hz, the preset window length can be set to 2 seconds, that is, each sliding window contains 200 consecutive sampling points; adjacent sliding windows can partially overlap to ensure that the action process is completely covered.

[0102] The above statistical feature extraction can include at least one of the following: mean, maximum, minimum, standard deviation, range of variation, rate of change, and number of peaks; for angle data, the duration of stability can also be extracted, which can be used to characterize the stability of the posture.

[0103] For example, window features may include at least one of component statistical features, magnitude statistical features, and relative baseline change features. Component statistical features refer to statistical features extracted from single-axis data, such as extracting the mean, standard deviation, etc., from ax(t), ay(t), and az(t); magnitude statistical features refer to statistical features extracted from the magnitudes of multi-axis data, such as acceleration magnitude calculated from triaxial acceleration data, angular velocity magnitude calculated from triaxial angular velocity data, and magnetic field magnitude calculated from triaxial magnetic field data, and then extracting statistical features such as mean, maximum value, and standard deviation from the magnitudes; relative baseline change features refer to the amount of change of the data within the window relative to the unloaded stationary baseline.

[0104] The above-mentioned unloaded static baseline can be determined as follows: Before starting work at the work site, ensure that the hook is not attached, does not contact the wire rope, D-ring, safety belt webbing or other attachment objects, and remains naturally hanging and stationary, and collect inertial magnetic data under this state; select the continuous data segment with the smallest data fluctuation from the collected inertial magnetic data; determine the unloaded static baseline based on the average value of the data of each axis in the continuous data segment.

[0105] For example, the unloaded stationary baseline can be represented as B0, which is a baseline vector composed of the average values ​​of triaxial acceleration, triaxial angular velocity, triaxial angle, and triaxial magnetic field. In the unloaded stationary baseline, the average acceleration reflects the direction of gravity when the hook is stationary, the average angular velocity reflects zero bias or slight drift when stationary, the average angle reflects the initial attitude after sensor installation, and the average magnetic field reflects the background magnetic field state in the working environment.

[0106] In some embodiments, the above-mentioned features can be extracted uniformly for all sliding windows, or different combinations of statistical features can be extracted for different types of data. For example, the number of peak values ​​and the range of variation can be extracted for acceleration data, the number of peak values ​​can be extracted for angular velocity data, and the duration of stability can be extracted for angle data. This application embodiment does not limit this.

[0107] In this embodiment, noise and abnormal data interference are eliminated through preprocessing, continuous timing information of the hanging action, stable hanging and unhooking action is retained by sliding slicing, and the change characteristics relative to the unloaded static baseline are introduced so that the window features can simultaneously reflect the absolute motion state of the hook and the offset relative to the initial static state, thereby reducing the impact of differences in background magnetic fields of different work sites and differences in the installation posture of different hooks on the recognition results.

[0108] In one possible implementation, the hooking action phase of the hook is determined based on the window features corresponding to the hook, including: The window features corresponding to the hook are input into the first preset classification model to obtain the hook action stage of the hook output by the first preset classification model; wherein, the first preset classification model is trained based on the first inertial magnetic sample data labeled with action stage labels; Based on the window and magnetic field characteristics corresponding to the hook, the hook's attachment object is identified, resulting in the hook's attachment object, including: Input the window features and magnetic field features corresponding to the hook into the second preset classification model to obtain the hook's attachment object output by the second preset classification model. The second preset classification model is trained based on the second inertial magnetic sample data. The second inertial magnetic sample data is the inertial magnetic sample data with the action stage label being the stable attachment stage and labeled with the attachment object label.

[0109] In some embodiments, both the first preset classification model and the second preset classification model can be a random forest model.

[0110] In some embodiments, the first preset classification model and the second preset classification model may also adopt models such as support vector machine, decision tree, XGBoost, LightGBM, long short-term memory network, gated recurrent unit network or one-dimensional convolutional network. The specific type of classification model is not limited in the embodiments of this application.

[0111] In this embodiment, the first preset classification model learns the motion change patterns of the hook in each stage—unattached, attaching, stable attachment, and unattaching—based on samples labeled with action stage tags. The second preset classification model is trained using only samples from the stable attachment stage, corresponding to the method of activating the model only when the hook is in the stable attachment stage during online recognition. Because the contact between the hook and the attached object is unstable and the surrounding magnetic field changes chaotically during the attaching and unattaching actions, using such samples for training can easily interfere with the model's learning of different attached objects. However, when training only with samples from the stable attachment stage, the relative position of the hook and the attached object is fixed, allowing the model to learn the stable magnetic field characteristics of different attached objects, thus making the object recognition results more accurate.

[0112] In one possible implementation, before determining the safety belt attachment compliance test results based on the target status of the hook and the safety belt attachment specifications, the method further includes: If no personnel are present in the work scene image indicating the presence of personnel, the safety belt attachment compliance test will not be performed on the work scene image. If a person is present in the work scene image and the safety belt wearing status indicator shows that the person is not wearing a safety belt, the safety belt attachment compliance test result is determined to be non-compliant.

[0113] In this embodiment, when there are no personnel in the work scene image, there are no objects to be detected. At this time, the safety belt hooking compliance test is not performed, which can avoid outputting meaningless test results for empty scenes. When personnel are not wearing safety belts, the fall protection link is missing from the beginning, and the test result can be directly determined as non-compliant without further judging the hook status.

[0114] In this embodiment, if no personnel are present in the work scene image, it indicates that there are no workers on site requiring safety belt attachment checks, and continuing subsequent inspections is meaningless. Therefore, safety belt attachment compliance checks are not performed on the work scene image. When personnel are not wearing safety belts, regardless of the hook's attachment status, effective fall protection cannot be provided. Therefore, there is no need to further assess the hook's condition, and the safety belt attachment compliance check result is directly determined to be non-compliant. In this way, scenarios where inspection is unnecessary can be skipped, and a quick inspection result can be provided even when personnel are not wearing safety belts.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations. The acquisition, storage, use and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0116] Corresponding to the above method embodiments, this application also provides a detection system for safety belts used in high-altitude operations. Please refer to [link to relevant documentation]. Figure 5 , Figure 5 This application provides a functional module diagram of a detection system for safety belts used in high-altitude operations, comprising: The acquisition module 510 is used to acquire visual recognition results and magnetic sensing recognition results corresponding to the work site. The visual recognition results are obtained by target detection of the work scene image, and include personnel presence status, safety belt wearing status, and hook visual status. The hook visual status characterizes the connection relationship between the hook and the attached object. The hooks include safety belt hooks and scaffolding hooks. The magnetic sensing recognition results are obtained based on inertial magnetic time-series data collected by an inertial magnetic sensor installed on the safety belt hook and the scaffolding hook. The magnetic sensing recognition results include the hook action stage and the attached object. The first determining module 520 is used to determine the target state of the hook based on the visual state of the hook and the magnetic sensing recognition result when the personnel presence state indicates that there are personnel in the work scene image and the safety belt wearing state indicates that the personnel are wearing safety belts. The second determining module 530 is used to determine the compliance test result of the personnel's safety belt attachment based on the target state of the hook and the safety belt attachment conditions.

[0117] The high-altitude work safety belt detection system provided in this application embodiment can achieve the following: Figure 3 The various processes implemented in the Chinese method embodiments can achieve similar or the same technical effects, and will not be described again here to avoid repetition.

[0118] In addition to the methods and systems provided in the above embodiments, this application also provides an electronic device, please refer to [link to previous document]. Figure 6 , Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0119] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0120] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0121] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0122] In some embodiments, memory 602 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in the methods provided according to embodiments of this application.

[0123] The processor 601 implements the method provided in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0124] In one example, the electronic device may also include a communication interface 603 and a bus 610. The processor 601, memory 602, and communication interface 603 are connected via the bus 610 and communicate with each other.

[0125] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0126] Bus 610 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0127] Furthermore, in conjunction with the methods provided in the above embodiments, this application embodiment can be implemented using a computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the methods in the above embodiments.

[0128] Furthermore, in conjunction with the methods provided in the above embodiments, this application embodiment can provide a computer program product to implement the methods. This program product is stored in a storage medium and executed by at least one processor to implement the various processes of the embodiments of the methods provided in the above embodiments, achieving similar or identical technical effects. To avoid repetition, further details are omitted here.

[0129] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0130] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0131] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0132] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0133] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for detecting safety belts used in high-altitude operations, characterized in that, include: The system acquires visual recognition results and magnetic sensing recognition results corresponding to the work site. The visual recognition results are obtained by target detection of the work scene image, including personnel presence status, safety belt wearing status, and hook visual status. The hook visual status characterizes the connection relationship between the hook and the attached object. The hooks include safety belt hooks and scaffolding hooks. The magnetic sensing recognition results are obtained based on inertial magnetic time-series data collected by an inertial magnetic sensor installed on both the safety belt hook and the scaffolding hook. The magnetic sensing recognition results include the hook action stage and the attached object. The hook visual status includes at least one of the following: a first connection state between the scaffolding hook and the wire rope; a second connection state between the safety belt hook and the D-ring of the safety belt; and a third connection state between the scaffolding hook and the webbing of the safety belt. The hook action stage includes stages of no attachment, attachment action, stable attachment, and unhooking action. The attached object is identified during the stable attachment stage. When the personnel presence status indicates that there are personnel in the work scene image, and the safety belt wearing status indicates that the personnel are wearing safety belts, the target status of the hook is determined based on the visual status of the hook and the magnetic sensing recognition result. Based on the target status of the hook and the standard conditions for attaching the safety belt, the compliance test result of the personnel's safety belt attachment is determined.

2. The method as described in claim 1, characterized in that, The visual recognition results include the presence status of the person and the seat belt wearing status; The visual recognition result is obtained by performing target detection on the work scene image, including: Target detection is performed on the work scene image to obtain target detection results, which include at least one of personnel detection boxes and protective equipment detection boxes; In response to determining that the target detection result includes the personnel detection box and the protective equipment detection box, the area coverage is determined based on the area of ​​the overlapping area between the protective equipment detection box and the personnel detection box, and the area of ​​the protective equipment detection box; and the personnel presence status is determined to indicate that personnel exist in the work scene image. If the area coverage rate is greater than or equal to a preset coverage rate threshold, the seat belt wearing status is determined according to the protective equipment category corresponding to the protective equipment detection frame.

3. The method as described in claim 2, characterized in that, The step of determining the seatbelt wearing status based on the protective equipment category corresponding to the protective equipment detection frame includes: When the protective equipment category is a first preset category, the seat belt wearing status is determined to instruct the person to wear the seat belt; wherein, the first preset category includes any one of the seat belt category and the reflective vest and seat belt combination category; When the protective equipment category is a second preset category, the seat belt wearing status is determined to indicate that the person is not wearing the seat belt; wherein, the second preset category includes reflective clothing.

4. The method as described in claim 1, characterized in that, The first attachment state, the second attachment state, and the third attachment state correspond one-to-one with the wire rope, the D-ring of the safety belt, and the webbing of the safety belt in the attachment object; Based on the visual state of the hook and the magnetic sensing recognition result, the target state of the hook is determined, including: When a target attachment status indicates that an attachment relationship has been detected, the attachment object corresponding to the target attachment status is checked for consistency with the attachment object in the magnetic sensing identification result. The target attachment status is any one of the first attachment status, the second attachment status, and the third attachment status. In response to determining that the hooking object corresponding to the target hooking state is consistent with the hooking object in the magnetic sensing identification result, the target state of the hook is determined to be the hooking object in the magnetic sensing identification result; In response to the determination that the hooked object corresponding to the target hooking state is inconsistent with the hooked object in the magnetic sensing identification result, the target state of the hook is determined to be a state pending verification.

5. The method for detecting safety belts for high-altitude operations as described in claim 4, characterized in that, The determination of the personnel's seatbelt attachment compliance test result based on the target state of the hook and the standard attachment conditions of the seatbelt includes: If the target status indication of the scaffold hook is a wire rope and the target status indication of the safety belt hook is the D-ring of the safety belt, then the safety belt attachment compliance test result is determined to be compliant. If the target status indication of the scaffold hook is the webbing of the safety belt, the safety belt attachment compliance test result is determined to be non-compliant.

6. The method as described in claim 1, characterized in that, Before determining the safety belt attachment compliance test result based on the target state of the hook and the safety belt attachment specifications, the method further includes: If the visual status indicator of the hook does not detect a connection between the hook and the object being hooked, the target state of the hook is determined based on the hook action phase.

7. The method as described in claim 6, characterized in that, Determining the target state of the hook based on the hook action phase includes: When the hook action phase is the stable hooking phase, the hooking object in the magnetic sensing identification result is determined as the target state of the hook; If the hook action phase is the unattached phase, the target state of the hook is determined to be the unattached state; When the hook action phase is either the hanging action phase or the unhooking action phase, the target state of the hook is determined to be the action process state.

8. The method as described in claim 7, characterized in that, The determination of the personnel's seatbelt attachment compliance test result based on the target state of the hook and the standard attachment conditions of the seatbelt includes: If the target state of the hook is the unattached state, the result of the seat belt attachment compliance test is determined to be non-compliant.

9. The method as described in claim 7, characterized in that, The magnetic sensing identification result is obtained based on the inertial magnetic time series data collected by the inertial magnetic sensor, including: Feature extraction is performed on the inertial magnetic time series data collected by the inertial magnetic sensor to obtain the window feature corresponding to the hook. The window feature includes at least one of acceleration feature, angular velocity feature and angle feature. The hooking action phase of the hook is determined based on the window features corresponding to the hook. When the hook action phase is a stable hooking phase, the hook is identified by the window features and magnetic field features corresponding to the hook to obtain the hook's hooking object. The magnetic field features are extracted from the magnetic field data in the inertial magnetic time series data. The magnetic sensing recognition result is determined based on the hooking action stage of the hook and the object being hooked.

10. The method as described in claim 9, characterized in that, The step of extracting features from the inertial magnetic time-series data acquired by the inertial magnetic sensor to obtain the window features corresponding to the hook includes: The inertial magnetic time series data is preprocessed, and the preprocessing includes at least one of timestamp alignment, outlier removal, and data smoothing; The preprocessed inertial magnetic time series data is sliced ​​into sliding slices according to a preset window length to obtain at least two sliding windows. Statistical features are extracted from the acceleration data, angular velocity data, and angle data within each sliding window to obtain the window features of each sliding window.

11. The method as described in claim 9, characterized in that, The step of determining the hooking action phase based on the window features corresponding to the hook includes: The window features corresponding to the hook are input into a first preset classification model to obtain the hook action stage of the hook output by the first preset classification model; wherein, the first preset classification model is trained based on first inertial magnetic sample data labeled with action stage labels. The step of identifying the hook's attachment object based on the window features and magnetic field features corresponding to the hook, to obtain the hook's attachment object, includes: The window features and magnetic field features corresponding to the hook are input into the second preset classification model to obtain the hook's attachment object output by the second preset classification model. The second preset classification model is trained based on the second inertial magnetic sample data, which is the inertial magnetic sample data with the action stage label being the stable attachment stage and labeled with the attachment object label.

12. The method as described in claim 1, characterized in that, Before determining the safety belt attachment compliance test result based on the target state of the hook and the safety belt attachment specifications, the method further includes: If the personnel presence status indicates that there are no personnel in the work scene image, no safety belt attachment compliance test will be performed on the work scene image; If the personnel presence status indicates that there are personnel in the work scene image, and the safety belt wearing status indicates that the personnel are not wearing safety belts, then the safety belt attachment compliance test result is determined to be non-compliant.

13. A detection system for safety belts used in high-altitude operations, characterized in that, include: The acquisition module is used to acquire visual recognition results and magnetic sensing recognition results corresponding to the work site. The visual recognition results are obtained by target detection of the work scene image, and include personnel presence status, safety belt wearing status, and hook visual status. The hook visual status characterizes the connection relationship between the hook and the attached object. The hooks include safety belt hooks and scaffolding hooks. The magnetic sensing recognition results are obtained based on inertial magnetic time-series data collected by an inertial magnetic sensor installed on the safety belt hook and the scaffolding hook. The magnetic sensing recognition results include the hook action stage and the attached object. The hook visual status includes at least one of the following: a first connection state between the scaffolding hook and the wire rope, a second connection state between the safety belt hook and the D-ring of the safety belt, and a third connection state between the scaffolding hook and the webbing of the safety belt. The hook action stage includes stages of no attachment, attachment action, stable attachment, and unhooking action. The attached object is identified during the stable attachment stage. The first determining module is used to determine the target state of the hook based on the visual state of the hook and the magnetic sensing recognition result when the personnel presence state indicates that there are personnel in the work scene image and the safety belt wearing state indicates that the personnel are wearing safety belts. The second determining module is used to determine the compliance test result of the personnel's seat belt attachment based on the target state of the hook and the standard attachment conditions of the seat belt.

14. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method of any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by one or more processors, implements the method of any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by one or more processors, implements the method of any one of claims 1 to 12.

Images