Methods and devices for determining the risk of packages falling off

By detecting the positional offset information between packages and conveyor belts in sorting scenarios, the risk of packages falling off can be determined, solving the problem of low efficiency in existing technologies and achieving more efficient risk warning and reduced damage.

CN118279927BActive Publication Date: 2026-11-17SF TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202211715785.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-29
Publication Date
2026-11-17
Estimated Expiration
2042-12-29

AI Technical Summary

Technical Problem

Existing methods for determining the risk of packages falling off are inefficient, leading to frequent damage to goods.

Method used

By acquiring images of the target sorting scene, package detection is performed, the positional offset information between the package detection box and the conveyor belt area is calculated, the probability of package falling is determined by the number of overlapping detection boxes and the offset distance, and a prompt is issued when a high risk is detected.

Benefits of technology

It improves the efficiency of determining the risk of packages falling out of bounds and reduces the occurrence of cargo damage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118279927B_ABST
    Figure CN118279927B_ABST
Patent Text Reader

Abstract

This application provides a method and apparatus for determining the risk of package falling. The method includes: acquiring a target scene image within a target sorting scenario; performing package detection on the target scene image to obtain multiple package detection boxes; acquiring positional offset information between each package detection box and a preset conveyor belt area; and determining the probability of package falling risk for each package in each detection box based on the positional offset information between each package detection box and the preset conveyor belt area. This application creatively proposes a method for determining package falling risk that improves efficiency by detecting the offset information of packages on the conveyor belt, addressing the inefficiency of existing methods for determining package falling risk.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application mainly relates to the field of image recognition technology, specifically to a method and apparatus for determining the risk of a package falling off. Background Technology

[0002] In logistics scenarios, goods arriving at sorting points are transported via conveyor belts. Many conveyor belt systems are designed without side barriers for easy handling by staff. However, irregular stacking or other improper handling by employees can easily cause goods to fall off the conveyor belt during transport. This can easily lead to damage to the packaging or the goods themselves. Current technology relies mainly on manual monitoring, which cannot promptly identify packages at risk of falling, resulting in losses.

[0003] In other words, existing methods for determining the risk of packages falling are not very efficient. Summary of the Invention

[0004] This application provides a method and apparatus for determining the risk of package loss, aiming to solve the problem of low efficiency in existing methods for determining the risk of package loss.

[0005] Firstly, this application provides a method for determining the risk of a package falling out of bounds, the method comprising:

[0006] Acquire target scene images within the target sorting scenario;

[0007] Package detection is performed on the target scene image to obtain multiple package detection boxes;

[0008] Obtain the positional offset information between each package detection box and the preset conveyor belt area;

[0009] The probability of a package falling off each package in each detection frame is determined based on the positional offset information between each package detection frame and the preset conveyor belt area.

[0010] Optionally, determining the probability of a package falling off each package in each package detection frame based on the positional offset information between each package detection frame and a preset conveyor belt area includes:

[0011] Package detection boxes that overlap with the target package detection box are identified as overlapping detection boxes.

[0012] Get the number of overlapping detection boxes of the target package detection box;

[0013] The probability of a package falling out of the target package detection frame is determined based on the number of overlapping detection frames and the positional offset information of the target package detection frame, thus obtaining the probability of a package falling out of the package in each package detection frame.

[0014] Optionally, obtaining the positional offset information between each package detection frame and the preset conveyor belt area includes:

[0015] Obtain the centroid of the package detection frame;

[0016] Obtain the centerline of the conveyor belt in the aforementioned conveyor belt area;

[0017] The vertical offset distance between the centroid of the package detection frame and the central axis of the conveyor belt is determined as the position offset information.

[0018] Optionally, the step of determining the probability of a package falling out of a parcel within a target package detection frame based on the number of overlapping detection frames and the positional offset information of the target package detection frame, and obtaining the probability of a package falling out of each parcel detection frame, includes:

[0019] Obtain the preset weight coefficients;

[0020] The number of overlapping detection boxes and the vertical offset distance of the target package detection box are weighted according to the preset weight coefficients to obtain the probability level of the package falling out of the target package detection box.

[0021] Optionally, acquiring the target scene image within the target sorting scene includes:

[0022] Acquire multiple sequential scene images taken in chronological order within the target sorting scene;

[0023] Human detection is performed on multiple temporal scene images to obtain multiple human detection boxes;

[0024] Multiple temporal scene images located within human detection boxes are input into a video classification model to obtain target video classification results, wherein the target video classification results include picking operation type and putting operation type;

[0025] If the target video classification result is a delivery operation type, then one of the multiple time-series scene images will be identified as the target scene image within the target sorting scene.

[0026] Optionally, acquiring multiple temporal scene images taken sequentially within the target sorting scene includes:

[0027] Acquire initial scene images by taking pictures of the target sorting scene at a preset frequency;

[0028] Perform human detection on the initial scene image and determine whether a human detection box has been obtained.

[0029] If a human detection box is obtained, then multiple sequential scene images taken in the target sorting scene, with the shooting time located after the initial scene image, are obtained.

[0030] Optionally, the method for determining the risk of the package falling includes:

[0031] Determine whether there are any risky package detection boxes in each package detection box that have a higher probability of package falling than a preset probability;

[0032] If a risk package detection box is found with a probability higher than the preset probability of the package falling, a warning message will be issued.

[0033] Secondly, this application provides a device for determining the risk of a package falling off the parcel, the device comprising:

[0034] The first acquisition unit is used to acquire target scene images within the target sorting scene.

[0035] A package detection unit is used to perform package detection on the target scene image to obtain multiple package detection boxes;

[0036] The second acquisition unit is used to acquire the position offset information between each package detection box and the preset conveyor belt area;

[0037] The risk determination unit is used to determine the probability of a package falling off each package in each package detection frame based on the positional offset information between each package detection frame and the preset conveyor belt area.

[0038] Optionally, the risk determination unit is used for:

[0039] Package detection boxes that overlap with the target package detection box are identified as overlapping detection boxes.

[0040] Get the number of overlapping detection boxes of the target package detection box;

[0041] The probability of a package falling out of the target package detection frame is determined based on the number of overlapping detection frames and the positional offset information of the target package detection frame, thus obtaining the probability of a package falling out of the package in each package detection frame.

[0042] Optionally, the second acquisition unit is configured to:

[0043] Obtain the centroid of the package detection frame;

[0044] Obtain the centerline of the conveyor belt in the aforementioned conveyor belt area;

[0045] The vertical offset distance between the centroid of the package detection frame and the central axis of the conveyor belt is determined as the position offset information.

[0046] Optionally, the risk determination unit is used for:

[0047] Obtain the preset weight coefficients;

[0048] The number of overlapping detection boxes and the vertical offset distance of the target package detection box are weighted according to the preset weight coefficients to obtain the probability level of the package falling out of the target package detection box.

[0049] Optionally, the first acquisition unit is configured to:

[0050] Acquire multiple sequential scene images taken in chronological order within the target sorting scene;

[0051] Human detection is performed on multiple temporal scene images to obtain multiple human detection boxes;

[0052] Multiple temporal scene images located within human detection boxes are input into a video classification model to obtain target video classification results, wherein the target video classification results include picking operation type and putting operation type;

[0053] If the target video classification result is a delivery operation type, then one of the multiple time-series scene images will be identified as the target scene image within the target sorting scene.

[0054] Optionally, the first acquisition unit is configured to:

[0055] Acquire initial scene images by taking pictures of the target sorting scene at a preset frequency;

[0056] Perform human detection on the initial scene image and determine whether a human detection box has been obtained.

[0057] If a human detection box is obtained, then multiple sequential scene images taken in the target sorting scene, with the shooting time located after the initial scene image, are obtained.

[0058] Optionally, the risk determination unit is used for:

[0059] Determine whether there are any risky package detection boxes in each package detection box that have a higher probability of package falling than a preset probability;

[0060] If a risk package detection box is found with a probability higher than the preset probability of the package falling, a warning message will be issued.

[0061] Thirdly, this application provides a computer device, the computer device comprising:

[0062] One or more processors;

[0063] Memory; and

[0064] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for determining the risk of package drop as described in any of the first aspects.

[0065] Fourthly, this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps in the method for determining the risk of package drop as described in any one of the first aspects.

[0066] This application provides a method and apparatus for determining the risk of package falling. The method includes: acquiring a target scene image within a target sorting scenario; performing package detection on the target scene image to obtain multiple package detection boxes; acquiring positional offset information between each package detection box and a preset conveyor belt area; and determining the probability of package falling risk for each package in each detection box based on the positional offset information between each package detection box and the preset conveyor belt area. This application creatively proposes a method for determining package falling risk that improves efficiency by detecting the offset information of packages on the conveyor belt, addressing the inefficiency of existing methods for determining package falling risk. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A schematic diagram of a scenario for the package drop risk determination system provided in this application embodiment;

[0069] Figure 2 This is a schematic flowchart of an embodiment of the method for determining the risk of package falling provided in this application.

[0070] Figure 3 This is a schematic diagram of an embodiment of the package drop risk determination device provided in this application;

[0071] Figure 4 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0074] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0075] This application provides a method and apparatus for determining the risk of a package falling, which will be described in detail below.

[0076] Please see Figure 1 , Figure 1 This is a schematic diagram of a system for determining the risk of package drop provided in an embodiment of this application. The system for determining the risk of package drop may include a computer device 100, which integrates a device for determining the risk of package drop.

[0077] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0078] In this embodiment, the computer device 100 described above can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device 100 can be a desktop computer, a portable computer, a network server, a handheld computer (Personal Digital Assistant, PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. This embodiment does not limit the image category of the computer device 100.

[0079] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include more than one application scenario. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the document. It is understood that the system for determining the risk of package falling may also include one or more other computer devices capable of processing data, which are not specifically limited here.

[0080] In addition, such as Figure 1 As shown, the system for determining the risk of a package falling may also include a memory 200 for storing data.

[0081] It should be noted that, Figure 1 The schematic diagram of the package drop risk determination system shown is merely an example. The package drop risk determination system and scenario described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of the package drop risk determination system and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0082] First, this application provides a method for determining the risk of package falling, which includes: acquiring a target scene image within a target sorting scene; performing package detection on the target scene image to obtain multiple package detection boxes; acquiring positional offset information between each package detection box and a preset conveyor belt area; and determining the probability of package falling risk for each package in each package detection box based on the positional offset information between each package detection box and the preset conveyor belt area.

[0083] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the method for determining the risk of package falling provided in this application. The method for determining the risk of package falling includes the following steps S201 to S204:

[0084] S201. Obtain the target scene image within the target sorting scene.

[0085] In this embodiment, the target sorting scenario can be various sorting sites. The sorting site is equipped with sorting equipment for sorting goods, including a conveyor belt that transports packages as it moves. Sorting personnel place packages onto the conveyor belt from either side, or remove packages from the conveyor belt through a retrieval operation.

[0086] The target scene image can be an image captured by an image acquisition device set up in the target sorting scene. For example, the image acquisition device could be a camera, radar, etc. For instance, a camera might take multiple initial images of the target sorting scene at a preset frequency, such as 20Hz. The target scene image can be a real-time image captured by the camera, an image captured and stored historically by the camera, or an image extracted from historical video footage captured by the camera.

[0087] In one specific embodiment, acquiring a target scene image within the target sorting scene may include:

[0088] (1) Acquire multiple sequential scene images taken in chronological order within the target sorting scene.

[0089] In this embodiment of the application, multiple time-series scene images are obtained by taking pictures of the target sorting scene at a preset frequency. The multiple time-series scene images can be each frame of the captured image.

[0090] (2) Perform human detection on multiple time-series scene images to obtain multiple human detection boxes.

[0091] In this embodiment of the application, multiple time-series scene images are input into a preset human detection model for human detection to obtain multiple human detection boxes.

[0092] The default human detection model is a trained YOLOv3, YOLOv4, YOLOv5, etc.

[0093] (3) Input the region images of multiple time-series scene images located within the human body detection box into the video classification model to obtain the target video classification result, which includes the picking operation type and the putting operation type.

[0094] In this embodiment, the video classification model can be a TSN (Temporal Segment Network). TSNs are based on long-range temporal structure modeling, combining a sparse temporal sampling strategy and video-level supervision to ensure effective and efficient learning when using entire video segments. Two-stream convolutional networks are ineffective at modeling long-range temporal structures, primarily because they operate on only a single frame (spatial network) or a single stack of frames within a short segment (temporal network), thus limiting their access to the temporal context. The video-level framework TSN can model actions from the entire video. Like two-stream networks, TSN consists of spatial and temporal convolutional networks. However, unlike two-stream networks which use single frames or single stacks of frames, TSN uses a series of sparsely sampled short segments from the entire video. Each segment provides its own preliminary prediction of the behavior category, and the video-level prediction result is obtained from the "consensus" of these segments. During the learning process, the model parameters are iteratively updated to optimize the loss value for video-level prediction.

[0095] The video classification model can be pre-trained using a video classification training set. This training set includes multiple video classification training samples and their corresponding labels. The labels for each training sample can be either a pickup or drop-off operation type, and these labels are assigned after the videos have been manually viewed.

[0096] Furthermore, if each temporal scene image contains multiple human detection boxes, then all human detection boxes in the multiple temporal scene images are clustered according to their positions to obtain multiple detection box clusters. Each human detection box within the same detection box cluster represents the location of the same person in different temporal scene images. The regions within multiple human detection boxes in the same detection box cluster from multiple temporal scene images are input into the video classification model to obtain the video classification results for the detection box clusters. This yields the video classification results for each detection box cluster, and the video classification result that appears most frequently among the video classification results for each detection box cluster is determined as the target video classification result.

[0097] (4) If the target video classification result is the cargo release operation type, then one of the multiple time-series scene images will be identified as the target scene image in the target sorting scene.

[0098] If the target video is classified as a delivery operation, then one of the multiple time-series scene images will be identified as the target scene image within the target sorting scene. Specifically, if the target video is classified as a delivery operation, then the last of the multiple time-series scene images will be identified as the target scene image within the target sorting scene.

[0099] Furthermore, to avoid selecting time-series scene images without people in them, in a specific embodiment, acquiring multiple time-series scene images taken sequentially within the target sorting scene may include:

[0100] (1) Obtain the initial scene image obtained by taking pictures of the target sorting scene at a preset frequency.

[0101] (2) Perform human detection on the initial scene image and determine whether a human detection box has been obtained.

[0102] (3) If a human body detection box is obtained, multiple sequential scene images taken in the target sorting scene after the initial scene image are obtained.

[0103] If a human body detection box is obtained, it means that someone has started sorting. At this time, video classification can be started to avoid the video classification model being unable to identify people when there are no people in the video, and also to avoid the computational load caused by frequently starting the video classification model.

[0104] S202. Perform package detection on the target scene image to obtain multiple package detection boxes.

[0105] In this embodiment, the target scene image is input into a package detection model for package detection, resulting in multiple package detection boxes. The package detection model can be a pre-trained HRnet, HourglassNet, or CenterNet.

[0106] S203. Obtain the position offset information between each package detection box and the preset conveyor belt area.

[0107] In this embodiment, a test image captured by a camera is acquired beforehand, and a conveyor belt area manually marked on the test image is obtained. The target scene image is then aligned with the test image to obtain the conveyor belt area in the target scene image. In other embodiments, conveyor belt detection is performed on the target scene image to obtain the conveyor belt area.

[0108] In one specific embodiment, obtaining the positional offset information between each package detection frame and a preset conveyor belt area may include:

[0109] (1) Obtain the centroid of the package detection box.

[0110] (2) Obtain the center line of the conveyor belt in the conveyor belt area.

[0111] In the target scene image, the conveyor belt area is a rectangular area or a parallelogram area, and the central axis of the conveyor belt is a straight line passing through the centroid of the conveyor belt area and parallel to the long side.

[0112] (3) The vertical offset distance between the centroid of the package detection frame and the central axis of the conveyor belt is determined as the position offset information.

[0113] The greater the vertical offset between the centroid of the package detection frame and the central axis of the conveyor belt, the more likely the package is to fall off the conveyor belt.

[0114] S204. Determine the probability of a package falling off each package in each package detection frame based on the positional offset information between each package detection frame and the preset conveyor belt area.

[0115] In one specific embodiment, each package detection frame is determined as a target package detection frame, the positional offset information between the target package detection frame and the preset conveyor belt area is calculated, and the probability of the package falling off the package in each package detection frame is determined based on the positional offset information between the target package detection frame and the preset conveyor belt area.

[0116] Specifically, this establishes a mapping relationship between the preset vertical offset distance and the probability of the package falling. The larger the vertical offset distance, the higher the probability of the package falling. For example, a vertical offset distance of 20cm corresponds to a 60% probability of the package falling; a vertical offset distance of 30cm corresponds to an 80% probability of the package falling. The setting can be adjusted according to specific circumstances.

[0117] Furthermore, determining the probability of a package falling out of each detection frame based on the positional offset information between each package detection frame and the preset conveyor belt area can include:

[0118] (1) The package detection boxes that overlap with the target package detection box in each package detection box are identified as overlapping detection boxes.

[0119] Furthermore, overlapping package detection boxes that overlap with the target package detection box and have an intersection-union ratio (LOU) greater than a preset value are identified as overlapping detection boxes. The LOU function calculates the ratio of the intersection to the union of two bounding boxes. The preset ratio can be 50%, etc., depending on the specific situation.

[0120] (2) Obtain the number of overlapping detection boxes of the target package detection box.

[0121] (3) Determine the probability of the package falling out of the target package detection box based on the number of overlapping detection boxes of the target package detection box and the position offset information of the target package detection box, and obtain the probability of the package falling out of the package in each package detection box.

[0122] In a specific embodiment, the position offset information is the vertical offset distance. The number of overlapping detection boxes of the target package detection box and the vertical offset distance of the target package detection box are normalized to obtain the normalized quantity and distance values. The average value of the normalized quantity and distance values ​​is determined as the probability of the package falling out of the target package detection box, thus obtaining the probability of the package falling out of each package detection box.

[0123] Furthermore, a preset weighting coefficient is obtained. Based on this preset weighting coefficient, the number of overlapping detection frames and the vertical offset distance of the target package detection frame are weighted to obtain the probability level of the package falling out of the target package detection frame. The preset weighting coefficient includes quantity weight and distance weight, which can be different and set according to specific circumstances. Specifically, the number of overlapping detection frames and the vertical offset distance of the target package detection frame are normalized to obtain normalized quantity and normalized distance values. These values ​​are then weighted and summed based on the preset weighting coefficient to obtain the probability level of the package falling out of the target package detection frame.

[0124] Furthermore, it checks whether any of the package detection frames contain risky packages with a higher probability of falling than a preset probability. If such a risky package is found, a warning message is issued. The preset probability can be 70%, which can be set according to specific circumstances.

[0125] Furthermore, if there is a risk package detection box with a probability of package falling higher than a preset value, then the system acquires various human body detection boxes in the target scene image, obtains the human body detection box to be prompted that is closest to the centroid of the risk package detection box, acquires the latitude and longitude coordinates of the human body to be prompted in the target sorting scene, acquires the latitude and longitude coordinates of each preset voice prompt device in the target sorting scene, and controls the voice prompt device corresponding to the latitude and longitude coordinates of the prompt device with the smallest distance from the human body's latitude and longitude coordinates to issue a prompt message. The voice prompt device can be a speaker.

[0126] This application provides a method and apparatus for determining the risk of package falling. The method includes: acquiring a target scene image within a target sorting scenario; performing package detection on the target scene image to obtain multiple package detection boxes; acquiring positional offset information between each package detection box and a preset conveyor belt area; and determining the probability of package falling risk for each package in each detection box based on the positional offset information between each package detection box and the preset conveyor belt area. This application creatively proposes a method for determining package falling risk that improves efficiency by detecting the offset information of packages on the conveyor belt, addressing the inefficiency of existing methods for determining package falling risk.

[0127] To better implement the method for determining the risk of package falling in the embodiments of this application, based on the method for determining the risk of package falling, the embodiments of this application also provide a device for determining the risk of package falling, such as... Figure 3 As shown, the package drop risk determination device 300 includes:

[0128] The first acquisition unit 301 is used to acquire target scene images within the target sorting scene.

[0129] Package detection unit 302 is used to perform package detection on the target scene image to obtain multiple package detection boxes;

[0130] The second acquisition unit 303 is used to acquire the position offset information between each package detection box and the preset conveyor belt area;

[0131] The risk determination unit 304 is used to determine the probability of a package falling off each package in each package detection frame based on the positional offset information between each package detection frame and the preset conveyor belt area.

[0132] Optionally, the risk determination unit is used for:

[0133] Package detection boxes that overlap with the target package detection box are identified as overlapping detection boxes.

[0134] Get the number of overlapping detection boxes of the target package detection box;

[0135] The probability of a package falling out of the target package detection frame is determined based on the number of overlapping detection frames and the positional offset information of the target package detection frame, thus obtaining the probability of a package falling out of the package in each package detection frame.

[0136] Optionally, the second acquisition unit is configured to:

[0137] Obtain the centroid of the package detection frame;

[0138] Obtain the centerline of the conveyor belt in the aforementioned conveyor belt area;

[0139] The vertical offset distance between the centroid of the package detection frame and the central axis of the conveyor belt is determined as the position offset information.

[0140] Optionally, the risk determination unit is used for:

[0141] Obtain the preset weight coefficients;

[0142] The number of overlapping detection boxes and the vertical offset distance of the target package detection box are weighted according to the preset weight coefficients to obtain the probability level of the package falling out of the target package detection box.

[0143] Optionally, the first acquisition unit is configured to:

[0144] Acquire multiple sequential scene images taken in chronological order within the target sorting scene;

[0145] Human detection is performed on multiple temporal scene images to obtain multiple human detection boxes;

[0146] Multiple temporal scene images located within human detection boxes are input into a video classification model to obtain target video classification results, wherein the target video classification results include picking operation type and putting operation type;

[0147] If the target video classification result is a delivery operation type, then one of the multiple time-series scene images will be identified as the target scene image within the target sorting scene.

[0148] Optionally, the first acquisition unit is configured to:

[0149] Acquire initial scene images by taking pictures of the target sorting scene at a preset frequency;

[0150] Perform human detection on the initial scene image and determine whether a human detection box has been obtained.

[0151] If a human detection box is obtained, then multiple sequential scene images taken in the target sorting scene, with the shooting time located after the initial scene image, are obtained.

[0152] Optionally, the risk determination unit is used for:

[0153] Determine whether there are any risky package detection boxes in each package detection box that have a higher probability of package falling than a preset probability;

[0154] If a risk package detection box is found with a probability higher than the preset probability of the package falling, a warning message will be issued.

[0155] This application also provides a computer device that integrates any of the package drop risk determination devices provided in this application. The computer device includes:

[0156] One or more processors;

[0157] Memory; and

[0158] One or more applications, wherein the applications are stored in memory and configured to be executed by a processor as steps of the method for determining the risk of package drop in any of the embodiments described above.

[0159] like Figure 4 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0160] The computer device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that the computer device structure shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0161] Processor 601 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in memory 602, and by calling data stored in memory 602, thereby providing overall monitoring of the computer device. Optionally, processor 601 may include one or more processing cores; processor 601 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into processor 601.

[0162] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0163] The computer device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0164] The computer device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0165] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 runs the application programs stored in the memory 602 to realize various functions, as follows:

[0166] Acquire a target scene image within the target sorting scenario; perform package detection on the target scene image to obtain multiple package detection boxes; obtain the positional offset information between each package detection box and the preset conveyor belt area; determine the probability of package falling off each package in each package detection box based on the positional offset information between each package detection box and the preset conveyor belt area.

[0167] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0168] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the methods for determining the risk of package loss provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0169] Acquire a target scene image within the target sorting scenario; perform package detection on the target scene image to obtain multiple package detection boxes; obtain the positional offset information between each package detection box and the preset conveyor belt area; determine the probability of package falling off each package in each package detection box based on the positional offset information between each package detection box and the preset conveyor belt area.

[0170] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0171] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0172] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0173] The above provides a detailed description of a method and apparatus for determining the risk of package falling, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for determining the risk of a package falling out, characterized in that, The methods for determining the risk of the package falling include: Acquire target scene images within the target sorting scenario; Package detection is performed on the target scene image to obtain multiple package detection boxes; Obtain the positional offset information between each package detection box and the preset conveyor belt area; The probability of a package falling off each package in each detection frame is determined based on the positional offset information between each package detection frame and the preset conveyor belt area. The step of determining the probability of a package falling off each package in each detection frame based on the positional offset information between each package detection frame and the preset conveyor belt area includes: Package detection boxes that overlap with the target package detection box are identified as overlapping detection boxes. Get the number of overlapping detection boxes of the target package detection box; The probability of a package falling out of the target package detection frame is determined based on the number of overlapping detection frames and the positional offset information of the target package detection frame, thus obtaining the probability of a package falling out of the package in each package detection frame.

2. The method for determining the risk of a package falling as described in claim 1, characterized in that, The step of obtaining the positional offset information between each package detection frame and the preset conveyor belt area includes: Obtain the centroid of the package detection frame; Obtain the centerline of the conveyor belt in the aforementioned conveyor belt area; The vertical offset distance between the centroid of the package detection frame and the central axis of the conveyor belt is determined as the position offset information.

3. The method for determining the risk of a package falling as described in claim 2, characterized in that, The step of determining the probability of a package falling out of the target package detection frame based on the number of overlapping detection frames and the positional offset information of the target package detection frame, and obtaining the probability of a package falling out of the package in each package detection frame, includes: Obtain the preset weight coefficients; The number of overlapping detection boxes and the vertical offset distance of the target package detection box are weighted according to the preset weight coefficients to obtain the probability level of the package falling out of the target package detection box.

4. The method for determining the risk of a package falling as described in claim 1, characterized in that, The acquisition of the target scene image within the target sorting scene includes: Acquire multiple sequential scene images taken in chronological order within the target sorting scene; Human detection is performed on multiple temporal scene images to obtain multiple human detection boxes; Multiple temporal scene images located within human detection boxes are input into a video classification model to obtain target video classification results, wherein the target video classification results include picking operation type and putting operation type; If the target video classification result is a delivery operation type, then one of the multiple time-series scene images will be identified as the target scene image within the target sorting scene.

5. The method for determining the risk of a package falling as described in claim 4, characterized in that, The acquisition of multiple time-series scene images taken sequentially within the target sorting scene includes: Acquire initial scene images by taking pictures of the target sorting scene at a preset frequency; Perform human detection on the initial scene image and determine whether a human detection box has been obtained. If a human detection box is obtained, then multiple sequential scene images taken in the target sorting scene, with the shooting time located after the initial scene image, are obtained.

6. The method for determining the risk of a package falling as described in claim 1, characterized in that, The method for determining the risk of the package falling includes: Determine whether there are any risky package detection boxes in each package detection box that have a higher probability of package falling than a preset probability; If a risk package detection box is found with a probability higher than the preset probability of the package falling, a warning message will be issued.

7. A device for determining the risk of a package falling off, characterized in that, The device for determining the risk of package falling includes: The first acquisition unit is used to acquire target scene images within the target sorting scene. A package detection unit is used to perform package detection on the target scene image to obtain multiple package detection boxes; The second acquisition unit is used to acquire the position offset information between each package detection box and the preset conveyor belt area; The risk determination unit is used to determine the probability of a package falling off each package in each package detection frame based on the positional offset information between each package detection frame and the preset conveyor belt area. The risk determination unit is specifically used to identify overlapping package detection frames in each package detection frame as overlapping detection frames; obtain the number of overlapping detection frames in the target package detection frame; determine the probability of package falling in the target package detection frame based on the number of overlapping detection frames in the target package detection frame and the position offset information of the target package detection frame, and obtain the probability of package falling in each package detection frame.

8. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for determining the risk of package drop as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It contains a computer program that is loaded by a processor to perform the steps of the method for determining the risk of package falling as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Conveying material falling monitoring method, device and system based on image recognition

    CN112446896A