Detection method for use safety of herringbone ladder, electronic equipment and storage medium

Abstract

The invention relates to the field of safety detection, in particular to a detection method for use safety of a herringbone ladder, electronic equipment and a storage medium. The method comprises the following steps: detecting whether a target image comprises a herringbone ladder or not through a first target detection model; under the condition that the target image comprises the herringbone ladder, determining whether the target image comprises a safety rope or not based on the second target detection model; under the condition that the target image comprises the safety rope, whether the herringbone ladder is equipped with the safety rope or not is determined based on the relative position relation between the safety rope and the herringbone ladder.

Description

Technical Field

[0001] The present disclosure relates to the field of safety detection, and more specifically, to a method for detecting the safety of using a stepladder, an electronic device, and a storage medium. Background Art

[0002] As an important part of the ladder, the safety rope of the ladder plays a key role in the safe use of the safety rope. At the same time, due to the popularity of video surveillance, it is necessary to use artificial intelligence methods to detect the safety rope. However, since most ladders are used in scenes such as construction sites and home decoration, these scenes are characterized by the messy placement of items. At the same time, because the safety rope is too thin and has different colors and shapes, it is easy to use target detection methods to detect the safety rope in the video image. It is easy to misdetect objects such as exposed wires as safety ropes, and the accuracy rate is low. Summary of the invention

[0003] One purpose of the present disclosure is to provide a method for detecting the safety of using a stepladder, so as to solve the problem of low accuracy of safety rope detection caused by complex real-life scenes.

[0004] According to a first aspect of the present disclosure, a method for detecting the safety of using a stepladder is provided, comprising:

[0005] Detecting whether a herringbone ladder is included in the target image by a first target detection model;

[0006] In a case where the target image includes the stepladder, determining whether the target image includes a safety rope based on the second target detection model;

[0007] In a case where the safety rope is included in the target image, it is determined whether the stepladder is equipped with a safety rope based on a relative positional relationship between the safety rope and the stepladder.

[0008] Optionally, before determining whether the target image includes a safety rope based on the second target detection model, the method further includes:

[0009] In the case that the target image includes the herringbone ladder, the image of the herringbone ladder portion in the target image is cropped to remove the image of the non-herringbone ladder portion in the target image.

[0010] Optionally, determining whether the stepladder is equipped with a safety rope based on the relative position relationship between the safety rope and the stepladder comprises:

[0011] Determine the center coordinates of the safety rope detection frame in the height direction of the stepladder;

[0012] When the center coordinates are within the first preset interval, determining that the stepladder in the target image is equipped with a safety rope;

[0013] When the center coordinate is outside the first preset interval, it is determined that the stepladder in the target image is not equipped with a safety rope.

[0014] Optionally, the upper limit of the first preset interval is the product of the first parameter and the height value of the stepladder, and the lower limit of the first preset interval is the product of the second parameter and the height value of the stepladder.

[0015] Optionally, after cropping the image of the herringbone ladder portion in the target image, the method further includes:

[0016] Determining the use status of the stepladder according to the target image;

[0017] When the stepladder is in use, a step of determining whether a safety rope is included in the target image based on the second target detection model is performed.

[0018] Optionally, determining the use status of the stepladder according to the target image includes:

[0019] Determine whether the height value of the stepladder is greater than a first threshold, wherein the first threshold is the product of a third parameter and the height value of the target image;

[0020] In the case that the height value of the herringbone ladder is greater than the first threshold, determining whether the ratio of the width value of the herringbone ladder to the height value of the herringbone ladder is greater than a second threshold;

[0021] When the ratio is greater than the second threshold, it is determined that the stepladder is in use.

[0022] Optionally, the first target detection model and the second target detection model are pre-trained YOLOv5 detection models, and the first target detection model and the second target detection model are pre-trained based on a first total loss function, wherein the first total loss function L is:

[0023] L=L box +λ cls ·L cls +λ obj ·L obj

[0024] Among them, λ cls is the weight of the target category classification loss, λ obj is the weight of target confidence loss, L box is the target box regression loss, L clsis the target category classification loss, L obj is the target confidence loss;

[0025] The target box regression loss L box for:

[0026]

[0027] Among them, N obj is the number of grid cells with targets, λ coord is the coefficient used to balance the position regression loss, x i ,y i 、w i 、h i are the center coordinates and width and height of the prediction box. is the center coordinate of the actual box;

[0028] The target category classification loss L cls for:

[0029]

[0030] Among them, classes is a collection of categories, y i,c is the one-hot encoding of the actual category, is the probability of the predicted category;

[0031] The target confidence loss L obj for:

[0032]

[0033] in, is the predicted probability of the target existing.

[0034] Optionally, the first target detection model and the second target detection model are pre-trained YOLOv5 detection models, and the first target detection model is pre-trained based on a first total loss function, wherein the first total loss function L is:

[0035] L=L box +λ cls ·L cls +λ obj ·L obj

[0036] Among them, λ cls is the weight of the target category classification loss, λ obj is the weight of target confidence loss, L box is the target box regression loss, L cls is the target category classification loss, L obj is the target confidence loss;

[0037] The target box regression loss L box for:

[0038]

[0039] Among them, N obj is the number of grid cells with targets, λ coord is the coefficient used to balance the position regression loss, x i ,y i 、w i 、h i are the center coordinates and width and height of the prediction box. is the center coordinate of the actual box;

[0040] The target category classification loss L cls for:

[0041]

[0042] Among them, classes is a collection of categories, y ic is the one-hot encoding of the actual category, is the probability of the predicted category;

[0043] The target confidence loss L obj for:

[0044]

[0045] in, is the predicted probability of the target existing;

[0046] The second target detection model is trained based on a second total loss function, and the second total loss function L 2 for:

[0047] L 2 =L box +λ cls ·L cls2 +λ obj ·L obj2

[0048] The second target category classification loss L cls2 for:

[0049]

[0050] Among them, N pos is the number of grids without targets, α is the weight of positive and negative samples, is the probability predicted by the model, γ is the focusing parameter;

[0051] The second target confidence loss Lobj2 for:

[0052]

[0053] Among them, N is the number of detection boxes, y i The label of the test result.

[0054] According to a second aspect of the present disclosure, an electronic device is provided, including a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of any one of the methods described in the first aspect are implemented.

[0055] According to a third aspect of the present disclosure, there is provided a storage medium on which computer instructions are stored, and when the computer instructions are executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0056] One of the technical effects of the present disclosure is that a method for detecting the safety of using a stepladder is provided. The stepladder and the safety rope can be detected respectively through two target detection models, and whether a stepladder is provided with a safety rope can be determined based on the relative position relationship between the two, thereby improving the accuracy of safety rope detection in complex construction scenarios.

[0057] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0059] Figure 1 is a flow chart of a method for detecting the safety of using a stepladder according to an embodiment;

[0060] Figure 2 is a schematic structural diagram of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0061] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0062] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0063] Techniques and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the techniques and equipment should be considered part of the specification.

[0064] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0065] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0066] The present application embodiment discloses a method for detecting the safety of using a stepladder, such as Figure 1 As shown, it includes steps S11 to S13.

[0067] Step S11, detecting whether a herringbone ladder is included in the target image by using a first target detection model.

[0068] In an example, the first target detection model may be a target detection model such as YOLO, SSD, or R-CNN. The first target detection model may be pre-trained based on an image of a herringbone ladder so as to recognize the herringbone ladder in an actual image.

[0069] In this embodiment, images containing herringbone ladders can be collected in advance through monitoring probes at various construction sites to obtain training data for the model. At the same time, images blocked by safety ropes can be removed, and data cleaning can be performed on the training data to improve the training effect. Afterwards, the herringbone ladders in the training data are annotated using annotation tools, such as the lablImg annotation tool, and the herringbone ladder types are marked to generate a txt file suitable for the first target detection model training. Each line in the txt file represents information about a herringbone ladder, including the herringbone ladder category, the horizontal coordinate of the center point of the herringbone ladder, the vertical coordinate of the center point of the herringbone ladder, the width of the herringbone ladder, and the height of the herringbone ladder. The training set, validation set, and test set are divided proportionally.

[0070] In an embodiment of the present application, the first target detection model may be a YOLOv5 detection model, and the first target detection model is pre-trained based on a first total loss function, wherein the first total loss function L is:

[0071] L=L box +λ cls ·L cls +λ obj ·L obj

[0072] Among them, λ cls is the weight of the target category classification loss, λ obj is the weight of target confidence loss, L box is the target box regression loss, L cls is the target category classification loss, Lobj is the target confidence loss;

[0073] Target box regression loss L box for:

[0074]

[0075] Among them, N obj is the number of grid cells with targets, λ coord is the coefficient used to balance the position regression loss, x i ,y i 、w i 、h i are the center coordinates and width and height of the prediction box. is the center coordinate of the actual box;

[0076] Target category classification loss L cls for:

[0077]

[0078] Among them, classes is a collection of categories, y i,c is the one-hot encoding of the actual category, is the probability of the predicted category;

[0079] Target confidence loss L obj for:

[0080]

[0081] in, is the predicted probability of the target existing.

[0082] In this example, the smooth L1 loss can be used to calculate the position regression loss of the target box. This loss is used to measure the difference between the position of the predicted box and the position of the true box. For the target category classification loss, the cross entropy loss can be used to calculate the target category classification loss. This loss is used to measure the difference between the model's classification prediction of the target category and the actual category, while for the target confidence loss, the binary cross entropy loss can be used to calculate the target confidence loss. This loss is used to measure the difference between the model's confidence prediction of the existence of the target and the actual existence. The total loss function of the model is obtained based on the above three loss functions.

[0083] Step S12: When the target image includes a stepladder, determining whether the target image includes a safety rope based on the second target detection model.

[0084] In this embodiment, the training data of the first target detection model can be input into the second target detection model for training after being labeled.

[0085] In an example of this embodiment, before determining whether a safety rope is included in the target image based on the second target detection model, the method also includes: when the target image includes a herringbone ladder, cropping the image of the herringbone ladder portion in the target image to remove the image of the non-herringbone ladder portion in the target image.

[0086] In this example, since the scene of the use of a stepladder is usually complicated, and other ropes or rope-like objects may appear, in order to avoid other objects interfering with the detection results, after the first object detection model detects the stepladder, the stepladder part in the target image can be cropped, and only this part is retained as the target image, and other parts are removed. Then it is input into the second object detection model to confirm whether there is a safety rope.

[0087] In this example, the cropping method of the herringbone ladder can be set according to the actual situation. For example, it can be cropped according to the detection frame of the herringbone ladder output by the first target detection model.

[0088] In an example of this embodiment, the type of the second target detection model and the first target detection model may be the same or different. In this embodiment, the training data of the second target detection model may be further acquired based on the training data of the first detection model. For example, the safety rope of the stepladder in the training data may be further labeled by the labelImg labeling tool, the type of safety rope may be marked, and a txt file suitable for the training of the second target detection model may be generated. Each line in the txt file represents the information of a safety rope, including the safety rope category, the horizontal coordinate of the center point of the safety rope, the vertical coordinate of the center point of the safety rope, the width of the safety rope, and the height of the safety rope.

[0089] In this example, the second target detection model may also be a YOLOv5 detection model, which may be trained using the same loss function as the first target detection model.

[0090] In another example, the second target detection model is trained based on a second total loss function, where the second total loss function L 2 for:

[0091] L 2 =L box +λ cls ·L cls2 +λ obj ·L obj2

[0092] The second target category classification loss L cls2 for:

[0093]

[0094] Among them, N pos is the number of grids without targets, α is the weight of positive and negative samples, is the probability predicted by the model, γ is the focusing parameter;

[0095] The second target confidence loss L obj2 for:

[0096]

[0097] Among them, N is the number of detection boxes, y i The label of the test result.

[0098] In this example, due to the small target characteristics of the safety rope, a high-resolution feature layer can be added to the feature extraction module of YOLOv5, and the anchor frame design can be optimized to enable the model to capture more subtle features. At the same time, in view of the fact that the small target of the safety rope is easy to miss, Focal Loss is used to replace part of the standard loss function to increase the model's attention to low-confidence samples, thereby reducing missed detections. The introduction of Focal Loss is mainly aimed at the classification loss and confidence loss parts. This is because the core function of Focal Loss is to reduce the contribution of easy-to-classify samples to the loss and increase the model's attention to difficult samples. It is particularly suitable for processing small targets and low-confidence samples.

[0099] Step S13: in the case where the target image includes a safety rope, determining whether the stepladder is equipped with a safety rope based on the relative position relationship between the safety rope and the stepladder.

[0100] In an example of the present embodiment, after the second target detection model detects the safety rope, it can be determined whether the stepladder is equipped with a safety rope based on the relative position of the safety rope and the stepladder. For example, it can be judged by whether the safety rope is set at the center position of the stepladder. If it is at the center position of the stepladder, it is determined that the stepladder is equipped with a safety rope; if it is not at the center position of the stepladder, it is determined that the stepladder is not equipped with a safety rope.

[0101] In this example, a method for detecting the safety of using a stepladder is provided. The stepladder and the safety rope can be detected respectively through two target detection models, and whether the stepladder is equipped with a safety rope can be determined based on the relative position relationship between the two, thereby improving the accuracy of safety rope detection in complex construction scenarios.

[0102] In an example of the present embodiment, determining whether a stepladder is equipped with a safety rope is based on the relative position relationship between the safety rope and the stepladder, including: determining the center coordinates of the safety rope detection frame in the height direction of the stepladder; when the center coordinates are within a first preset interval, determining that the stepladder in the target image is equipped with a safety rope; when the center coordinates are outside the first preset interval, determining that the stepladder in the target image is not equipped with a safety rope.

[0103] In this embodiment, after detecting the safety rope, the second target detection model can obtain the position coordinates of the detection frame on the target image, further obtain the center coordinates of the detection frame center, and then make a judgment based on the position of the center coordinates. For example, the center coordinates of the detection frame in the height direction of the ladder can be used to determine whether it is within a preset interval. The upper and lower limits of the preset interval can be set based on actual needs or the position of the safety rope in the height direction of the ladder. When the center coordinates of the detection frame are within the first preset interval, it means that the safety rope detected by the second target detection model is the safety rope of the ladder, that is, the ladder is already equipped with a safety rope. When the center coordinates of the detection frame are outside the first preset interval, that is, the safety rope detected by the second target detection model is not the safety rope of the ladder, but other ropes in the current scene. At this time, it can be determined that the ladder in the target image is not equipped with a safety rope.

[0104] In an example of this embodiment, the upper limit of the first preset interval is the product of the first parameter and the height value of the stepladder, and the lower limit of the first preset interval is the product of the second parameter and the height value of the stepladder.

[0105] In this embodiment, the upper and lower limits of the first preset interval can be determined based on the height value of the ladder. In this example, the coordinate value of the center coordinate of the detection frame and the height value of the ladder are values ​​in the same coordinate system, for example, they can both be values ​​in the pixel coordinate system of the target image. In this example, the first parameter and the second parameter can be set based on actual needs. Usually, the safety rope is located in the middle of the ladder. Therefore, in one example, the first parameter value can be set to 0.75 and the second parameter value can be set to 0.35, so that the first preset interval covers the middle of the ladder. In one example, the height value of the ladder is h. a , the center coordinate y of the safety rope detection frame in the height direction of the ladder b , when it is within the first preset interval, that is:

[0106] 0.7h a >y b >0.35h a

[0107] It can be determined that the ladder is equipped with a safety rope when the safety rope detection frame is at the center coordinate y in the height direction of the ladder. b , when it is outside the first preset interval, that is:

[0108] y b ≥0.7h a or b ≤0.35h a

[0109] It can be confirmed that the stepladder is not equipped with a safety rope.

[0110] In another example of this embodiment, it is possible to further determine whether the center coordinate in the width direction is in a second preset interval based on determining whether the center coordinate in the height direction of the stepladder is within the preset interval. In this example, only when the center coordinates in the height direction and the width direction are respectively within the corresponding intervals, the stepladder is considered to be equipped with a safety rope. In other cases, the stepladder is considered to be not equipped with a safety rope.

[0111] In an example of this embodiment, the upper and lower limits of the preset interval can also be calculated based on the product of the first parameter and the height value of the herringbone ladder, plus an additional fourth parameter, where the fourth parameter can be set according to the confidence and noise ratio of the detection frame.

[0112] In an example of the present embodiment, after cropping the image of the herringbone ladder portion in the target image, the method also includes: determining the usage status of the herringbone ladder based on the target image; when the herringbone ladder is in use, executing a step of determining whether a safety rope is included in the target image based on a second target detection model.

[0113] In the embodiment of the present application, after cropping the image, the use status of the ladder in the image can be determined first. For example, when the ladder is in use, that is, when it is opened, it is necessary to monitor the safety of the ladder and continue to perform the step of determining whether the target image includes a safety rope based on the second target detection model. When the ladder is not in use, such as when the ladder is not opened and is in a closed state, it does not need to be monitored, and there is no need to monitor whether it is equipped with a safety rope.

[0114] In an example of the present embodiment, determining the usage status of the stepladder according to the target image includes: determining whether the height value of the stepladder is greater than a first threshold, wherein the first threshold is the product of a third parameter and the height value of the target image; if the height value of the stepladder is greater than the first threshold, determining whether the ratio of the width value of the stepladder to the height value of the stepladder is greater than a second threshold; if the ratio is greater than the second threshold, determining that the stepladder is in use.

[0115] In this embodiment, the height and width of the ladder in the image can be measured first. Assuming that the overall height of the target image is 1000 pixels, the third parameter we pre-set is 0.5, and the first threshold is the product of the third parameter (0.5) and the target image height value (1000 pixels), i.e., 500 pixels. Afterwards, based on the measured height value of the ladder in the image, if the measurement result shows that the height of the ladder exceeds 500 pixels, this indicates that the ladder is a normal opening method in the image, and then, the ratio of the height value to the width value of the ladder is further calculated. Assuming that the measured height of the ladder is 600 pixels and the width is 200 pixels, the ratio of height to width is 0.333 (200 / 600). At this point, we need to set a second threshold to determine whether it is large enough to confirm whether the ladder is in the unfolded use state under normal circumstances. Assuming that we set the second threshold to 0.3, then because 0.333 is greater than 0.3, the ratio exceeds the second threshold. When the height of the ladder is greater than the first threshold and the ratio of its width to height is greater than the second threshold, we can determine that the ladder is in use. Identifying the use status of the ladder in this way greatly improves the recognition efficiency and avoids the waste of resources and computing power caused by the subsequent process of determining the safety rope when many images of the ladder are not in use.

[0116] like Figure 2 As shown, an embodiment of the present application further provides an electronic device 200, including a processor 201 and a memory 202, wherein the memory 202 stores computer instructions, and when the computer instructions are executed by the processor 201, the steps of any one of the methods in the embodiment of the method for detecting the safety of using a stepladder are implemented.

[0117] An embodiment of the present application also provides a storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, any one of the above-mentioned detection embodiments for the safe use of a stepladder is implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0118] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0119] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0120] The embodiments of the present disclosure may be systems, methods and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the embodiments of the present disclosure.

[0121] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0122] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0123] The computer program instructions for performing the operation of the embodiments of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the embodiments of the present disclosure.

[0124] Various aspects of the embodiments of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0125] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0126] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0127] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and a part of a module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0128] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting the safety of using a stepladder, characterized in that: include: Detecting whether a herringbone ladder is included in the target image by a first target detection model; In a case where the target image includes the stepladder, determining whether the target image includes a safety rope based on the second target detection model; In a case where the safety rope is included in the target image, it is determined whether the stepladder is equipped with a safety rope based on a relative positional relationship between the safety rope and the stepladder.

2. The method according to claim 1, characterized in that Before determining whether the target image includes a safety rope based on the second target detection model, the method further includes: In the case that the target image includes the herringbone ladder, the image of the herringbone ladder portion in the target image is cropped to remove the image of the non-herringbone ladder portion in the target image.

3. The method according to claim 2, characterized in that The step of determining whether the stepladder is equipped with a safety rope based on the relative position relationship between the safety rope and the stepladder comprises: Determine the center coordinates of the safety rope detection frame in the height direction of the stepladder; When the center coordinates are within the first preset interval, determining that the stepladder in the target image is equipped with a safety rope; When the center coordinate is outside the first preset interval, it is determined that the stepladder in the target image is not equipped with a safety rope.

4. The method according to claim 3, characterized in that The upper limit of the first preset interval is the product of the first parameter and the height value of the stepladder, and the lower limit of the first preset interval is the product of the second parameter and the height value of the stepladder.

5. The method according to claim 2, characterized in that: After the image of the herringbone ladder portion in the target image is cropped, the method further includes: Determining the use status of the stepladder according to the target image; When the stepladder is in use, a step of determining whether a safety rope is included in the target image based on the second target detection model is performed.

6. The method according to claim 2, characterized in that Determining the use status of the stepladder according to the target image includes: Determine whether the height value of the stepladder is greater than a first threshold, wherein the first threshold is the product of a third parameter and the height value of the target image; In the case that the height value of the herringbone ladder is greater than the first threshold, determining whether the ratio of the width value of the herringbone ladder to the height value of the herringbone ladder is greater than a second threshold; When the ratio is greater than the second threshold, it is determined that the stepladder is in use.

7. The method according to any one of claims 1 to 6, characterized in that: The first target detection model and the second target detection model are pre-trained YOLOv5 detection models, and the first target detection model and the second target detection model are pre-trained based on a first total loss function, wherein the first total loss function L is: L=L box +λ cls ·L cls +λ obj ·L obj Among them, λ cls is the weight of the target category classification loss, λ obj is the weight of target confidence loss, L box is the target box regression loss, L cls is the target category classification loss, L obj is the target confidence loss; The target box regression loss L box for: Among them, N obj is the number of grid cells with targets, λ coord is the coefficient used to balance the position regression loss, x i ,y i 、w i 、h i are the center coordinates and width and height of the prediction box; is the center coordinate of the actual box; The target category classification loss L cls for: Among them, classes is a collection of categories, y i,c is the one-hot encoding of the actual category, is the probability of the predicted category; The target confidence loss L obj for: in, is the predicted probability of the target existing.

8. The method according to any one of claims 1 to 6, characterized in that: The first target detection model and the second target detection model are pre-trained YOLOv5 detection models, and the first target detection model is pre-trained based on a first total loss function, wherein the first total loss function L is: L=L box +λ cls ·L cls +λ obj ·L obj Among them, λ cls is the weight of the target category classification loss, λ obj is the weight of target confidence loss, L box is the target box regression loss, L cls is the target category classification loss, L obj is the target confidence loss; The target box regression loss L box for: Among them, N obj is the number of grid cells with targets, λ coord is the coefficient used to balance the position regression loss, x i ,y i 、w i 、h i are the center coordinates and width and height of the prediction box; is the center coordinate of the actual box; The target category classification loss L cls for: Among them, classes is a collection of categories, y i,c is the one-hot encoding of the actual category, is the probability of the predicted category; The target confidence loss L obj for: in, is the predicted probability of the target existing; The second target detection model is trained based on a second total loss function, and the second total loss function L2 is: L2=L box +λ cls ·L cls2 +λ obj ·L obj2 The second target category classification loss L cls2 for: Among them, N pos is the number of grids without targets, α is the weight of positive and negative samples, is the probability predicted by the model, γ is the focusing parameter; The second target confidence loss L obj2 for: Where N is the number of detection boxes, y i The label of the test result.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of the method described in any one of claims 1 to 8 are implemented.

10. A storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images