Lifting operation detection method, device and equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a lifting operation detection method and device, equipment and a storage medium. Performing hanging object detection on the current operation image through a hanging object detection model to obtain a hanging object detection result; if the hanging object detection result indicates that the hanging object exists in the current operation image, performing human body detection on the current operation image through a human body detection model to obtain a human body detection result; if the lifting object detection result and the human body detection result meet the lifting person identification condition, it is determined that the lifting device lifts a person illegally, and compared with the prior art, the lifting operation detection efficiency and accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a hoisting operation detection method, a hoisting operation detection device, a computer device and a computer readable storage medium. BACKGROUND

[0002] The hoisting equipment such as the crawler crane, the bridge crane and the tower crane is a carrying equipment for lifting, lowering and horizontally moving materials, and is an important equipment for realizing the mechanization and automation of production processes, reducing heavy physical labor and improving labor productivity in industrial, transportation and construction enterprises. During the operation of the hoisting equipment, there are cases of directly hoisting people or indirectly hoisting people by hoisting objects, and such illegal operation mode is extremely easy to cause personal injury. In the related art, the safety officer usually detects whether the hoisting equipment is illegally hoisting people on site by manual detection, but the manual detection is greatly affected by the personal experience, attention and other factors of the safety officer, and the missed detection and false detection are easy to occur, so that the accuracy of the detection result is difficult to guarantee. In addition, the manual detection requires the safety officer to supervise on site for a long time, which not only increases the labor cost, but also has a large labor intensity, and is easy to cause the safety officer to be tired, further reducing the detection efficiency. SUMMARY

[0003] The hoisting operation detection method, the hoisting operation detection device, the computer device and the computer readable storage medium provided by the embodiments of the present application can improve the efficiency and accuracy of hoisting operation detection.

[0004] In one aspect, the hoisting operation detection method provided by the present application comprises: obtaining a current operation image of a hoisting equipment; performing object hoisting detection on the current operation image by an object hoisting detection model to obtain an object hoisting detection result; if the object hoisting detection result indicates that there is an object hoisted in the current operation image, performing human body detection on the current operation image by a human body detection model to obtain a human body detection result; if the object hoisting detection result and the human body detection result satisfy hoisting person identification conditions, determining that the hoisting equipment illegally hoists people.

[0005] Optionally, in one embodiment, the object hoisting detection result comprises an object hoisting detection frame, the human body detection result comprises a human body detection frame, and if the object hoisting detection result and the human body detection result satisfy the hoisting person identification conditions, determining that the hoisting equipment illegally hoists people comprises: determining a reference detection point corresponding to the human body detection frame; if the reference detection point is located within the object hoisting detection frame, determining that the hoisting equipment illegally hoists people.

[0006] Optionally, in an embodiment, if the reference detection point is located within the load detection frame, it is determined that the hoisting equipment is in violation of lifting people, comprising: cropping a sub-image corresponding to the load detection frame from the current work image; performing image segmentation on the sub-image through an image segmentation model to obtain a plurality of image regions; determining a load region from the plurality of image regions, and if the reference detection point is located within the load region, it is determined that the hoisting equipment is in violation of lifting people.

[0007] Optionally, in an embodiment, if the load detection result and the human body detection result satisfy the lifting people identification condition, after determining that the hoisting equipment is in violation of lifting people, the method further comprises: obtaining a historical work image corresponding to the current work image; if it is determined that the load has been displaced according to the current work image and the historical work image, and it is determined that the hoisting equipment is lifting people according to the historical work image, performing an alarm operation.

[0008] Optionally, in an embodiment, performing the alarm operation comprises: determining an alarm image from the current work image and the historical work image, and performing the alarm operation based on the alarm image.

[0009] Optionally, in an embodiment, after performing the alarm operation, the method further comprises: controlling the hoisting equipment to lower the load to the ground.

[0010] Optionally, in an embodiment, before controlling the hoisting equipment to lower the load to the ground, the method further comprises: evaluating a first probability of causing the hoisted person to be injured by lowering the load to the ground, and evaluating a second probability of causing the hoisted person to be injured by lifting the load to a target position; if the first probability is less than or equal to the second probability, controlling the hoisting equipment to lower the load to the ground.

[0011] In a second aspect, the present application provides a hoisting work detection device, comprising: an image acquisition module for acquiring a current work image of a hoisting equipment; an algorithm detection module for performing load detection on the current work image through a load detection model to obtain a load detection result, and if the load detection result indicates that there is a load in the current work image, performing human body detection on the current work image through a human body detection model to obtain a human body detection result; a logic processing module for determining that the hoisting equipment is in violation of lifting people if the load detection result and the human body detection result satisfy a lifting people identification condition.

[0012] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the lifting operation detection method.

[0013] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program executable on a processor to implement the lifting operation detection method.

[0014] The present application provides a lifting operation detection scheme, which obtains a current operation image of a lifting device, detects a hoisted object in the current operation image through a hoisted object detection model to obtain a hoisted object detection result, detects a human body in the current operation image through a human body detection model to obtain a human body detection result if the hoisted object detection result indicates that there is a hoisted object in the current operation image, and determines that the lifting device is lifting a person if the hoisted object detection result and the human body detection result meet hoisted person identification conditions. Thus, the present application realizes automatic detection of lifting operation of the lifting device, reduces labor cost and greatly improves detection efficiency compared with the traditional manual inspection method, and avoids problems such as manual missed detection, false detection, and visual fatigue, and ensures the accuracy of the detection result. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0016] Figure 1 is a flowchart of the lifting operation detection method provided by the embodiments of the present application; Figure 2 is an example of the current operation image in the embodiments of the present application; Figure 3 is Figure 2 is an example of the human body detection frame and the hoisted object detection frame of the current operation image shown in Figure 4 is a structural schematic diagram of the lifting operation detection device provided by the embodiments of the present application; Figure 5 is a structural schematic diagram of the computer device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0017] In order to make the technical problems solved by the present application, technical solutions and beneficial effects clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application, and not to limit the present application.

[0018] It should be understood that the term "comprising" as used in the specification and the appended claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0019] It should also be understood that the term "and / or" as used in the specification and the appended claims, means any combination of one or more of the associated listed items and all possible combinations of the items.

[0020] As used in the specification and the appended claims, the term "if' can be interpreted as meaning "when" or "once" or "in response to a determination" or "in response to a detection" depending on the context. Similarly, the phrase "if determined" or "if detected [the recited condition or event]" can be interpreted as meaning "once determined" or "in response to a determination" or "once detected [the recited condition or event]" or "in response to a detection [the recited condition or event]" depending on the context.

[0021] In addition, in the description of the specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0022] The reference "one embodiment" or "some embodiments" and the like described in the present specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in yet some embodiments" and the like appearing in various places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "including but not limited to", unless otherwise specifically emphasized.

[0023] The application provides a hoisting operation detection method, a hoisting operation detection device, a computer device and a storage medium, wherein the hoisting operation detection method can be executed by the hoisting operation detection device or a computer device integrated with the hoisting operation detection device, wherein a current operation image of a hoisting device is acquired; a lifting object detection model is used to detect the lifting object in the current operation image to obtain a lifting object detection result; if the lifting object detection result indicates that there is a lifting object in the current operation image, a human body detection model is used to detect the human body in the current operation image to obtain a human body detection result; and if the lifting object detection result and the human body detection result meet lifting person identification conditions, it is determined that the hoisting device is lifting a person in violation of regulations.

[0024] Please refer to Figure 1 , a flowchart of a hoisting operation detection method disclosed in an embodiment of the application is shown in Figure 1 , and the flowchart of the hoisting operation detection method can be as follows: In S110, a current operation image of a hoisting device is acquired.

[0025] The hoisting device is an important mechanical device indispensable in the fields of industrial production, construction and logistics transportation, which realizes the vertical lifting and horizontal movement of heavy objects through mechanical transmission or hydraulic systems. There are various types of such devices, including bridge cranes, gantry cranes, tower cranes, truck cranes and other types, which are widely used in port terminals, construction sites, factory workshops and other places. Hoisting devices not only can significantly improve work efficiency and reduce labor intensity, but also can complete heavy object handling tasks that cannot be done by manpower, and are important basic equipment for realizing mechanization and automation in modern industrial production.

[0026] The following embodiments take a computer device executing a hoisting operation detection method as an example for description.

[0027] In the embodiment of the application, the computer device first acquires the operation image of the hoisting device at the current time through an image acquisition device such as a camera, which is denoted as a current operation image, wherein the current operation image needs to clearly show the hoisting assembly (such as a hook) of the hoisting device and the possible lifting object. For example, please refer to Figure 2 , Figure 2 for an example diagram of the current operation image.

[0028] The lifting object refers to the object being hoisted by the hoisting device, such as normal lifting objects such as goods and construction materials, and may also be a person or other illegal lifting object.

[0029] In addition, after acquiring the current operation image of the hoisting operation device, the computer device can also preprocess the current operation image, including but not limited to image enhancement, noise suppression, edge sharpening and other operations, to improve the accuracy of subsequent detection.

[0030] In S120, the current work image is subjected to the lifting object detection model for lifting object detection, and a lifting object detection result is obtained.

[0031] It should be noted that in the embodiments of the present application, the lifting object detection model is pre-trained, which is configured to take an image possibly including a lifting object as input and output a lifting object detection result indicating an image region in which the lifting object may exist in the image. The model architecture and training method of the lifting object detection model are not specifically limited here and can be selected by those skilled in the art according to actual needs.

[0032] For example, the lifting object detection model can be trained in the following manner: A target detection model based on YOLOv8 is used as a basic model for training the lifting object detection model, and the model structure thereof includes the following components: Backbone network: The backbone network refers to a network used to extract image features, and its main function is to convert the original input image into a multi-layer feature map for subsequent detection tasks.

[0033] Neck network: The neck network is used to combine feature maps of different levels to generate a feature map with multi-scale information to improve the detection accuracy.

[0034] Detection head: composed of 3 different output layers, respectively responsible for detecting large, medium and small scale targets.

[0035] In the training phase, sample work images covering different work situations of the lifting equipment are collected, including diversified work images of different lifting object types (normal lifting object off the ground, normal lifting object not off the ground, lifting equipment directly lifting people, lifting equipment not lifting objects), lighting conditions, and background environments. Each sample work image is labeled to obtain a training label including the real bounding box of the lifting object in the sample work image. Each sample work image and its corresponding training label are input into the target detection model as a group of samples for training until a preset stopping condition is met, and a lifting object detection model suitable for lifting object detection is obtained. The configuration of the preset stopping condition is not specifically limited here, and for example, the preset stopping condition can be configured as loss convergence or the number of parameter updates of the target detection model reaching a preset number, such as 200 rounds, etc. Through the above training process, the lifting object detection model can efficiently identify various lifting objects, which can be normal lifting objects such as goods and building materials, or illegal lifting objects such as personnel.

[0036] In addition, the sample work images can also be enhanced using a configured data enhancement strategy before being used in the training process. The configuration of the data enhancement strategy is not specifically limited here, and for example, the data enhancement strategy includes random flipping, color transformation, scale scaling, brightness / saturation transformation, mosaic splicing, adding noise, etc.

[0037] Correspondingly, in the embodiment of the present application, after the computer device obtains the current working image of the hoisting device through the image acquisition device, the current working image is further input to the hoisting object detection model, and hoisting object detection is performed through the hoisting object detection model to obtain the hoisting object detection result output by the hoisting object detection model, which indicates the image area in the current working image that may contain hoisting objects.

[0038] It can be understood that, according to actual needs, the hoisting object detection model can also be obtained by training a basic model such as SSD, darknet, or Transformer.

[0039] In S130, if the hoisting object detection result indicates that there is a hoisting object in the current working image, human body detection is performed on the current working image through the human body detection model to obtain a human body detection result.

[0040] It should be noted that in the embodiment of the present application, a human body detection model is also pre-trained, which is configured to take an image that may contain a human body as input and output a human body detection result indicating the image area in the image that may contain a human body. The model architecture and training method of the human body detection model are not specifically limited here and can be selected by those skilled in the art according to actual needs.

[0041] For example, in the embodiment of the present application, the human body detection model is trained in a similar manner to the training of the hoisting object detection model in the above embodiment, with the difference being the training samples. Among them, sample human body images covering different scenes and different angles are collected, and each sample human body image is labeled to obtain a training label containing a human body real bounding box, and each sample human body image and its corresponding training label are input as a group of samples.

[0042] In the embodiment of the present application, after the computer device performs hoisting object detection through the hoisting object detection model and obtains the hoisting object detection result, it further determines whether there is a hoisting object in the current working image according to the hoisting object detection result. If the hoisting object detection result indicates that there is a hoisting object in the current working image, the current working image is input to the human body detection model, and human body detection is performed on the current working image through the human body detection model to obtain a human body detection result. If the hoisting object detection result indicates that there is no hoisting object in the current working image, the computer device determines that the hoisting device has not yet hoisted the hoisting object at this time, and continues to obtain subsequent working images of the hoisting device to monitor the hoisting process in real time.

[0043] In S140, if the hoisting object detection result and the human body detection result satisfy the hoisting person identification condition, it is determined that the hoisting device has violated the rules.

[0044] As above, after obtaining the current working image of the hoisting device, the computer device comprehensively determines whether the hoisting-person identification condition is met in combination with the hoist object detection result and the human body detection result. If it is identified that the hoist object detection result and the human body detection result meet the hoisting-person identification condition, it is determined that the hoisting device is in violation of hoisting people, otherwise, it is determined that the hoisting device is in a normal working state.

[0045] For example, the hoisting-person identification condition can include the overlap degree, relative position, etc. of the image area where the hoist object exists and the image area where the human body exists, and is determined by setting a threshold. For example, if the overlap degree of the image area where the hoist object exists and the image area where the human body exists exceeds a preset threshold, it is determined that the hoisting-person identification condition is met.

[0046] Optionally, in an embodiment, the hoist object detection result includes a hoist object detection box, the human body detection result includes a human body detection box, and if the hoist object detection result and the human body detection result meet the hoisting-person identification condition, it is determined that the hoisting device is in violation of hoisting people, including: determining a reference detection point corresponding to the human body detection box; if the reference detection point is located within the hoist object detection box, it is determined that the hoisting device is in violation of hoisting people.

[0047] In the embodiment of the present application, the hoist object detection model outputs the hoist object detection result in the form of a detection box, which is denoted as a hoist object detection box, and the human body detection model outputs the human body detection result in the form of a detection box, which is denoted as a human body detection box.

[0048] The hoisting-person identification condition can be configured as that the reference detection point of the human body detection box is located within the hoist object detection box. The reference detection point is used to represent the human body and can be selected by those skilled in the art according to actual needs.

[0049] For example, a sub-image corresponding to the human body detection box can be cut out from the current working image, and key point recognition is performed on the sub-image. The reference detection point can be selected from the human body key points identified, for example, the left and right hip key points or the left and right ankle key points can be selected as the reference detection point. Here, no specific limitation is made on the key point recognition algorithm, which can be selected by those skilled in the art according to actual needs.

[0050] As above, after determining the reference detection point used to represent the human body, the computer device further identifies whether the reference detection point is located within the hoist object detection box. If the reference detection point is located within the hoist object detection box, it is determined that the hoisting device is in violation of hoisting people. In addition, if it is identified that the reference detection point is located outside the hoist object detection box, the computer device determines that the hoisting device is not in violation of hoisting people but in a normal working state.

[0051] Optionally, in an embodiment, the reference detection point corresponding to the human body detection box is determined, including: A midpoint of a lower edge line of the human body detection frame is determined as a reference detection point corresponding to the human body detection frame.

[0052] In the embodiments of the present application, in the case of ensuring a certain detection accuracy, in order to improve the detection efficiency, the midpoint of the lower edge line of the human body detection frame can be directly used as the reference detection point.

[0053] For example, referring to Figure 3 , point A in the figure is the midpoint of the lower edge line of the human body detection frame, which is located in the hoist detection frame B, and the computer device will determine that the hoisting equipment is illegally hoisting a person.

[0054] Optionally, in an embodiment, in order to further improve the accuracy of hoisting operation detection, N current operation images are obtained, and the collection angles of the current operation images are different, N is a positive integer greater than 2, and if the hoist detection result and the human body detection result meet the hoisting person identification condition, it is determined that the hoisting equipment is illegally hoisting a person, including: if there are ⌈N / 2⌉ current operation images corresponding to the hoist detection result and the human body detection result meet the hoisting person identification condition, it is determined that the hoisting equipment is illegally hoisting a person.

[0055] Optionally, in an embodiment, in order to further improve the accuracy of hoisting operation detection, if the reference detection point is located within the hoist detection frame, it is determined that the hoisting equipment is illegally hoisting a person, including: cutting a sub-image corresponding to the hoist detection frame from the current operation image; performing image segmentation on the sub-image through an image segmentation model to obtain a plurality of image regions; determining a hoist region from the plurality of image regions, and if the reference detection point is located within the hoist region, it is determined that the hoisting equipment is illegally hoisting a person.

[0056] In the embodiments of the present application, when the computer device determines whether the hoisting equipment is illegally hoisting a person according to the reference detection point, the computer device first cuts a sub-image corresponding to the hoist detection frame from the current operation image, and then performs image segmentation on the sub-image through an image segmentation model to divide the sub-image into a plurality of image regions. For example, the computer device can use a SAM (segment Anything Model) general image segmentation model to segment the sub-image. After segmentation by the SAM model, the computer device further analyzes each image region to determine a hoist region from the plurality of image regions. The computer device can determine the largest image region as the hoist region, and if the reference detection point is located within the hoist region, it is determined that the hoisting equipment is illegally hoisting a person. In addition, if it is identified that the reference detection point is located outside the hoist region, the computer device determines that the hoisting equipment is not illegally hoisting a person but is in a normal operation state.

[0057] Through accurate image segmentation and region analysis, the background information interference can be excluded, the human body near the hoisted object can be avoided to cause misjudgment, and the accuracy of the detection result is ensured.

[0058] Optionally, in an embodiment, if the hoisted object detection result and the human body detection result satisfy the hoisted person identification condition, after determining that the hoisting equipment violates the hoisted person, the method further comprises: obtaining a historical work image corresponding to the current work image; if it is determined that the hoisted object is displaced according to the current work image and the historical work image, and it is determined that the hoisting equipment violates the hoisted person according to the historical work image, performing an alarm operation.

[0059] It should be noted that in the embodiment of the present application, each time a work image of the hoisting equipment is collected, the hoisting work detection scheme provided by the present application is used for detection, and the detection result is recorded.

[0060] In order to avoid the possible misjudgment risk, the computer device further obtains a historical work image corresponding to the current work image, compares and analyzes the position change of the hoisted object in the current work image and the historical work image, if it is found that the hoisted object is displaced and it is determined that the hoisting equipment violates the hoisted person according to the historical work image, an alarm operation is performed again, so as to ensure the safety of work. The type of the alarm operation is not limited here, and can be selected by a person skilled in the art according to actual needs.

[0061] It should be noted that the number of historical work images obtained in the embodiment of the present application is not strictly limited, and can be flexibly adjusted according to actual needs. For example, the computer device can obtain two work images collected at an interval of a preset time length (such as 10 seconds) before the collection time corresponding to the current work image as the historical work images of the current work image. For the current work image and the two historical work images before it, the computer device detects the distance between the center points of the hoisted object detection boxes corresponding to the adjacent two work images. If the distance between the center points corresponding to the two historical work images reaches a preset threshold, and the distance between the center points of the current work image and its adjacent historical work image also exceeds the preset threshold, it is determined that the hoisted object is displaced, As described above, by comparing and analyzing the position change of the hoisted object, misjudgment caused by the personnel behind the hoisted object can be avoided, and the reliability of the detection result is ensured.

[0062] Optionally, in an embodiment, the alarm operation comprises: determining an alarm image from the current work image and the historical work image, and performing an alarm operation based on the alarm image.

[0063] In the embodiment of the present application, when performing the alarm operation, the computer device can determine the alarm image from the current work image and the historical work image, and perform the alarm operation based on the determined alarm image. For example, the computer device can transmit the alarm image to the monitoring center and synchronously notify the on-site safety management personnel, so as to ensure rapid response and processing of potential risks and protect the safety of the work environment.

[0064] For example, the computer device can directly determine the current work image as the alarm image, and highlight mark the human body region in the alarm image, so as to facilitate rapid identification by the monitoring center.

[0065] Optionally, in an embodiment, after performing the alarm operation, the method further comprises: controlling the hoisting device to lower the hoisted object to the ground.

[0066] In the embodiment of the present application, after performing the alarm operation, in order to ensure the safety of the hoisted personnel, the computer device can also control the hoisting device to lower the hoisted object to the ground, so as to lower the hoisted personnel to the ground and avoid the hoisted personnel from falling and being injured.

[0067] Optionally, in an embodiment, before controlling the hoisting device to lower the hoisted object to the ground, the method further comprises: evaluating a first probability of the hoisted personnel being injured due to lowering the hoisted object to the ground, and evaluating a second probability of the hoisted personnel being injured due to lifting the hoisted object to the target position; if the first probability is less than or equal to the second probability, controlling the hoisting device to lower the hoisted object to the ground.

[0068] In the embodiment of the present application, after performing the alarm operation, the computer device does not directly control the hoisting device to lower the hoisted object to the ground, but first performs risk evaluation, compares the probabilities of the hoisted personnel being injured due to lowering the hoisted object and due to lifting the hoisted object, and selects the operation with lower risk, so as to further protect the safety of the hoisted personnel.

[0069] In the embodiment of the present application, the computer device evaluates the probability of the hoisted personnel being injured due to lowering the hoisted object to the ground by the hoisting device, which is recorded as a first probability, and evaluates the probability of the hoisted personnel being injured due to lifting the hoisted object to the target position by the hoisting device, which is recorded as a second probability. If the first probability is less than the second probability, the computer device controls the hoisting device to lower the hoisted object to the ground, so as to ensure the safety of the hoisted personnel. It should be noted that the configuration of the risk evaluation strategy in the embodiment of the present application is not specifically limited.

[0070] For example, the risk evaluation strategy can be configured as: By means of the pre-constructed digital twin model of the hoisting equipment, and by configuring the digital twin model to be in the same hoisting state as the current hoisting state of the hoisting equipment, the hoisting process of lowering the hoisted object to the ground and lifting the hoisted object to the target position is simulated, and the probabilities of the hoisted personnel being injured in the two hoisting processes are calculated respectively. For example, the hoisting process of lowering the hoisted object to the ground by the hoisting equipment can be simulated 100 times, and the injury of the hoisted personnel in each simulated hoisting process is recorded to obtain the first probability; similarly, the hoisting process of lifting the hoisted object to the target position by the hoisting equipment can be simulated 100 times, and the injury of the hoisted personnel in each simulated hoisting process is recorded to obtain the second probability.

[0071] As can be seen from the above, the hoisting operation detection scheme provided by the present application obtains the current operation image of the hoisting equipment, detects the hoisted object in the current operation image by means of the hoisted object detection model to obtain the hoisted object detection result, detects the human body in the current operation image by means of the human body detection model to obtain the human body detection result if the hoisted object detection result indicates that there is a hoisted object in the current operation image, and determines that the hoisting equipment is lifting a person in violation of regulations if the hoisted object detection result and the human body detection result meet the hoisted person identification condition. In this way, the present application realizes the automatic detection of the hoisting operation of the hoisting equipment, which not only reduces the labor cost compared with the traditional manual inspection method, but also greatly improves the detection efficiency. In addition, since the entire hoisting operation detection process does not require human intervention, the problems of human error, false detection and visual fatigue are avoided, and the accuracy of the detection result can be ensured.

[0072] In order to better implement the above hoisting operation detection method, the present application also provides a corresponding hoisting operation detection device. The meanings of the terms are the same as those in the above hoisting operation detection method, and the specific implementation details are described with reference to the above method embodiment.

[0073] Please refer to Figure 4 The hoisting operation detection device can include an image acquisition module 210, an algorithm detection module 220 and a logic processing module 230. The functions of each module are described in detail as follows: The image acquisition module 210 is configured to obtain the current operation image of the hoisting equipment. The algorithm detection module 220 is configured to detect the hoisted object in the current operation image by means of the hoisted object detection model to obtain the hoisted object detection result, and detect the human body in the current operation image by means of the human body detection model to obtain the human body detection result if the hoisted object detection result indicates that there is a hoisted object in the current operation image. The logic processing module 230 is configured to determine that the hoisting equipment is lifting a person in violation of regulations if the hoisted object detection result and the human body detection result meet the hoisted person identification condition.

[0074] Optionally, in an embodiment, the hoisted object detection result comprises a hoisted object detection frame, the human body detection result comprises a human body detection frame, and the logic processing module 230 is configured to: determine a reference detection point corresponding to the human body detection frame; and determine that the hoisting device is in violation of the regulation of hoisting people if the reference detection point is located within the hoisted object detection frame.

[0075] Optionally, in an embodiment, the logic processing module 230 is configured to: cut a sub-image corresponding to the hoisted object detection frame from the current work image; perform image segmentation on the sub-image by using an image segmentation model to obtain a plurality of image regions; determine a hoisted object region from the plurality of image regions; and determine that the hoisting device is in violation of the regulation of hoisting people if the reference detection point is located within the hoisted object region.

[0076] Optionally, in an embodiment, the logic processing module 230 is further configured to: obtain a historical work image corresponding to the current work image; and perform an alarm operation if it is determined that the hoisted object has been displaced according to the current work image and the historical work image, and it is determined that the hoisting device is hoisting people according to the historical work image.

[0077] Optionally, in an embodiment, the logic processing module 230 is configured to: determine an alarm image from the current work image and the historical work image, and perform an alarm operation based on the alarm image.

[0078] Optionally, in an embodiment, the logic processing module 230 is further configured to control the hoisting device to lower the hoisted object to the ground.

[0079] Optionally, in an embodiment, the logic processing module 230 is configured to: evaluate a first probability of causing the hoisted person to be injured by lowering the hoisted object to the ground, and evaluate a second probability of causing the hoisted person to be injured by lifting the hoisted object to the target position; and control the hoisting device to lower the hoisted object to the ground if the first probability is less than or equal to the second probability.

[0080] The specific limitations of the hoisting operation detection device can be referred to the limitations of the hoisting operation detection method in the foregoing, which will not be described herein. Each module in the hoisting operation detection device described above can be realized by software, hardware, and combinations thereof, in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0081] In one embodiment, a computer device is provided, and an internal structure diagram of the computer device can be as shown in FIG. 8. Figure 5As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to connect with external wireless clients and provide wireless network access services for connected wireless clients. The computer program is executed by the processor to implement the crane operation detection method provided by the application.

[0082] In an embodiment, a computer device is provided, including a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the crane operation detection method in the above-mentioned embodiments, such as: obtaining a current operation image of the crane equipment; performing object detection on the current operation image through the object detection model to obtain an object detection result; if the object detection result indicates that there is an object in the current operation image, performing human body detection on the current operation image through the human body detection model to obtain a human body detection result; if the object detection result and the human body detection result meet the person lifting identification condition, it is determined that the crane equipment violates the regulation of lifting people.

[0083] In an embodiment, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the crane operation detection method in the above-mentioned embodiments, such as: obtaining a current operation image of the crane equipment; performing object detection on the current operation image through the object detection model to obtain an object detection result; if the object detection result indicates that there is an object in the current operation image, performing human body detection on the current operation image through the human body detection model to obtain a human body detection result; if the object detection result and the human body detection result meet the person lifting identification condition, it is determined that the crane equipment violates the regulation of lifting people.

[0084] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0085] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0086] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

[0087] It should be noted that when the above embodiments of the present application are applied to specific products or technologies, relevant data of users are involved, and the user's permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A method for detecting lifting operations, characterized in that, include: Acquire the current operating image of the lifting equipment; The suspended object detection model is used to detect suspended objects in the current operation image, and the detection results are obtained. If the object detection result indicates that there is an object in the current operation image, then the human body detection model is used to perform human body detection on the current operation image to obtain the human body detection result; If the results of the load detection and the results of the human body detection meet the conditions for identifying a person being lifted, then it is determined that the lifting equipment is illegally lifting a person.

2. The lifting operation detection method according to claim 1, characterized in that, The load detection result includes a load detection frame, and the human body detection result includes a human body detection frame. If the load detection result and the human body detection result meet the conditions for identifying a person being lifted, then it is determined that the lifting equipment is illegally lifting a person, including: Determine the reference detection point corresponding to the human body detection frame; If the reference detection point is located within the hoisting detection frame, it is determined that the lifting equipment is illegally hoisting a person.

3. The lifting operation detection method according to claim 2, characterized in that, If the reference detection point is located within the load detection frame, then determining that the lifting equipment is illegally lifting a person includes: Extract the sub-image corresponding to the suspended object detection frame from the current working image; The sub-image is segmented using an image segmentation model to obtain multiple image regions; The hoisting area is determined from the multiple image areas. If the reference detection point is located within the hoisting area, it is determined that the lifting equipment is illegally hoisting a person.

4. The lifting operation detection method according to any one of claims 1-3, characterized in that, If the results of the load detection and the human body detection meet the criteria for identifying a person being lifted, then after determining that the lifting equipment is illegally lifting a person, the process further includes: Obtain the historical operation image corresponding to the current operation image; If it is determined that the load has shifted based on the current operation image and the historical operation image, and it is determined that the lifting equipment is lifting a person based on the historical operation image, then an alarm operation is executed.

5. The lifting operation detection method according to claim 4, characterized in that, The alarm execution operation includes: An alarm image is determined from the current operation image and the historical operation image, and an alarm operation is performed based on the alarm image.

6. The lifting operation detection method according to claim 4, characterized in that, After the alarm operation is performed, the following is also included: Control the lifting equipment to lower the suspended object to the ground.

7. The lifting operation detection method according to claim 6, characterized in that, Before controlling the lifting equipment to lower the suspended object to the ground, the method further includes: Assess the first probability that lowering the load to the ground will cause injury to the personnel being hoisted, and assess the second probability that raising the load to the target position will cause injury to the personnel being hoisted; If the first probability is less than or equal to the second probability, then control the lifting equipment to lower the suspended object to the ground.

8. A crane operation detection device, characterized in that, The lifting operation detection device includes: The image acquisition module is used to acquire the current working image of the lifting equipment; The algorithm detection module is used to perform suspended object detection on the current operation image using a suspended object detection model to obtain a suspended object detection result; and if the suspended object detection result indicates that there is a suspended object in the current operation image, then perform human body detection on the current operation image using a human body detection model to obtain a human body detection result. The logic processing module is used to determine that the lifting equipment is illegally lifting a person if the detection results of the suspended object and the detection results of the human body meet the conditions for identifying a person being lifted.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the lifting operation detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the lifting operation detection method according to any one of claims 1 to 7.

Images