Sorting anomaly detection method, device and system

Abstract

The invention discloses a sorting anomaly detection method, device and system, relates to the technical field of machine vision, and is used for accurately identifying whether sorting anomaly exists in a package or not and preventing the package from being distributed to a wrong destination. The method comprises the steps of obtaining a pre-distributed first sorting port of a to-be-detected parcel and a first sliding-in moment when the to-be-detected parcel is predicted to slide into the first sorting port; and obtaining at least one parcel slip-in record. The parcel sliding-in record generation method comprises the steps that the form and the movement line of an object recognized in an image collection area are obtained, and under the condition that it is determined that the object is an actual parcel according to the form and the movement line and the actual parcel slides into a sorting opening, the parcel sliding-in record corresponding to the actual parcel is generated. The parcel sliding-in record comprises the second sorting port where the actual parcel slides in and the actual sliding-in time. And under the condition that the target parcel sliding-in record does not exist in the at least one parcel sliding-in record, it is confirmed that sorting of the parcels to be detected is abnormal.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a sorting anomaly detection method, device and system. Background Art

[0002] With the rapid development of industrial automation, the coverage of automated sorting equipment is getting higher and higher. Automated sorting equipment can automatically control the sorting vehicle to transport the parcel to the corresponding sorting port. However, in the process of the sorting vehicle transporting the parcel to the corresponding sorting port, there may be problems such as sorting vehicle transport errors, which may easily cause the parcel to be unable to enter the corresponding sorting port, and then cause the parcel to be delivered to the wrong destination. Summary of the invention

[0003] The embodiments of the present application provide a sorting anomaly detection method, device and system for accurately identifying whether a package has a sorting anomaly, thereby preventing the package from being delivered to the wrong destination.

[0004] To achieve the above objectives, the present application provides the following technical solutions:

[0005] In a first aspect, a sorting anomaly detection method is provided, comprising: obtaining a pre-assigned first sorting port and a first sliding-in time expected to slide into the first sorting port of a package to be detected. Obtain at least one package sliding-in record. The method for generating a package sliding-in record comprises obtaining the shape and movement path of an object identified in an image acquisition area, and generating a package sliding-in record corresponding to the actual package when the object is determined to be an actual package according to the shape and movement path and the actual package slides into the sorting port. The package sliding-in record includes a second sorting port into which the actual package actually slides, and the actual sliding-in time. In the case where there is no target package sliding-in record in at least one package sliding-in record, it is confirmed that the sorting of the package to be detected is abnormal. The second sorting port included in the target package sliding-in record is the same as the first sorting port, and the time difference between the actual sliding-in time and the first sliding-in time does not exceed a threshold.

[0006] In this technical solution, when a frame extraction method or a single-frame image is used to identify whether there is a package at the sorting port, it is easy to identify a package or other object stationary on the slope as a package that slides into the sorting port, causing a false detection problem and increasing the false alarm rate of the sorting system. In order to avoid such situations, the present application can accurately determine that the object is an actual package and that the actual package slides into the sorting port based on the shape and movement route of the object, and generate a corresponding package sliding-in record. Furthermore, by determining whether there is a target package sliding-in record corresponding to the package to be detected in at least one package sliding-in record, the computer device can accurately identify whether there is a sorting anomaly in the package to be detected, so that the staff can handle it in a timely manner when there is a sorting anomaly in the package to be detected, so as to avoid the package being delivered to the wrong destination.

[0007] Furthermore, considering that when a parcel slides on the parcel channel of the sorting port, the image of the parcel captured by the smart camera may not include information such as the barcode or QR code on the parcel, making it difficult to identify specific information such as the parcel number and delivery destination, and it is impossible to determine whether the parcel corresponds to the parcel assigned to the sorting port. In this case, the present application can support the detection of parcel sorting anomalies by identifying the morphological movement path of the object in the image, determining whether there is an actual parcel, and the parcel sliding record of the actual parcel, without the need to specifically identify information such as the barcode or QR code on the parcel, and has high practicality.

[0008] In a possible embodiment, the first sliding-in time refers to an estimated time of entering the first sorting port calculated based on the time when the package to be inspected leaves the sorting vehicle.

[0009] In a possible embodiment, obtaining a pre-assigned first sorting port and a first sliding-in time of the package to be detected that is expected to slide into the first sorting port includes: obtaining sorting information of the package to be detected. The sorting information is used to indicate the first sorting port and the sorting vehicle that transports the package to be detected. Determine the time when the sorting vehicle performs a sliding-out action, and the sliding-out action is used to make the package to be detected leave the sorting vehicle. Obtain a first sliding duration corresponding to the first sorting port. The first sliding duration is the sliding duration of the package sliding from the sorting vehicle at the first sorting port to the first sorting port. Based on the time when the sorting vehicle performs the sliding-out action and the first sliding duration, determine the first sliding-in time.

[0010] In a possible embodiment, determining that an object is an actual package according to its shape and motion path and that the actual package slides into a sorting port includes: in response to identifying at least two objects having different shapes and conforming to a preset shape in a same group of continuous multi-frame images, and the at least two objects respectively corresponding to at least two different motion paths, determining that the at least two objects are at least two different actual packages, and that the at least two actual packages both slide into the sorting port.

[0011] In a possible embodiment, the image acquisition area covers a plurality of sorting ports. The sorting anomaly detection method further includes: in response to identifying that the movement route of the object does not change in a preset area of ​​any sorting port, determining that there is a blocked bag at any sorting port.

[0012] In a possible embodiment, determining that the object is an actual package and the actual package slides into the sorting port according to the shape and the shape movement path includes: in response to the shape of the object conforming to a preset shape and the movement end point indicated by the movement path being a preset position, determining that the object is an actual package and the actual package slides into the sorting port. Alternatively, in response to the shape of the object conforming to a preset shape and the movement direction indicated by the movement path being a preset direction, determining that the object is an actual package and the actual package slides into the sorting port. Alternatively, based on the package recognition model, the recognition result of the shape and the movement path is that the package recognition is successful, determining that the object is an actual package and the actual package slides into the sorting port.

[0013] In a possible embodiment, when there is no target parcel sliding-in record in at least one parcel sliding-in record, confirming that the parcel to be detected has a sorting anomaly includes: determining a parcel sliding-in record in which the second sorting port and the first sorting port are the same as each other in the at least one parcel sliding-in record as a sliding-in record to be matched. When the time difference between the actual sliding-in time and the first sliding-in time included in all the sliding-in records to be matched exceeds a threshold, it is determined that the parcel to be detected has a sorting anomaly.

[0014] In a possible embodiment, after confirming that the parcel to be detected is sorted abnormally, the method further includes: obtaining a second sliding-in time at which the parcel to be detected is expected to slide into a third sorting port, where the third sorting port is the sorting port into which the parcel to be detected is expected to slide abnormally. In the case where there is an abnormal sliding-in record in the remaining sliding-in records, confirming that the parcel to be detected slides abnormally into the third sorting port. The second sorting port included in the abnormal sliding-in record is the same as the third sorting port, and the time difference between the actual sliding-in time included and the second sliding-in time does not exceed a threshold. The remaining sliding-in records include parcel sliding-in records excluding the target parcel sliding-in records corresponding to other parcels to be detected.

[0015] In a possible embodiment, after confirming that the sorting of the package to be detected is abnormal, the method further includes: outputting alarm information, where the alarm information is used to indicate that the sorting of the package to be detected is abnormal, and / or that the package to be detected slides abnormally into the third sorting port.

[0016] In a second aspect, a sorting anomaly detection device is provided, comprising: an acquisition unit and a processing unit.

[0017] The acquisition unit is used to acquire a pre-assigned first sorting port and a first sliding-in time estimated to slide into the first sorting port of the package to be detected.

[0018] The acquisition unit is further used to acquire at least one parcel sliding-in record. The acquisition unit is also specifically used for a method for generating a parcel sliding-in record, including acquiring the shape and movement path of an object identified in an image acquisition area, and generating a parcel sliding-in record corresponding to the actual parcel when the object is determined to be an actual parcel according to the shape and movement path and the actual parcel slides into the sorting port. The parcel sliding-in record includes the second sorting port into which the actual parcel actually slides, and the actual sliding-in time.

[0019] The processing unit is used to confirm that the parcel to be detected is abnormally sorted when there is no target parcel sliding-in record in at least one parcel sliding-in record. The second sorting port included in the target parcel sliding-in record is the same as the first sorting port, and the time difference between the actual sliding-in time and the first sliding-in time does not exceed a threshold.

[0020] In a possible embodiment, the first sliding-in time refers to an estimated time of entering the first sorting port calculated based on the time when the package to be inspected leaves the sorting vehicle.

[0021] In a possible embodiment, the acquisition unit is specifically used to: acquire sorting information of the package to be detected. The sorting information is used to indicate the first sorting port and the sorting vehicle that transports the package to be detected. Determine the moment when the sorting vehicle performs a slide-out action, and the slide-out action is used to make the package to be detected leave the sorting vehicle. Acquire a first sliding duration corresponding to the first sorting port. The first sliding duration is the sliding duration of the package from the sorting vehicle at the first sorting port to the first sorting port. Based on the moment when the sorting vehicle performs the slide-out action and the first sliding duration, determine the first slide-in moment.

[0022] In a possible embodiment, the acquisition unit is specifically used to: in response to identifying at least two objects with different shapes and conforming to preset shapes in the same group of continuous multi-frame images, and the at least two objects correspond to at least two different motion routes, determine that the at least two objects are at least two different actual packages, and the at least two actual packages both slide into the sorting port.

[0023] In a possible embodiment, the image acquisition area covers a plurality of sorting ports. The processing unit is further configured to determine that there is a bag jam at any sorting port in response to identifying that the movement route of the object does not change within a preset area of ​​any sorting port.

[0024] In a possible embodiment, the acquisition unit is specifically configured to: in response to the shape of the object conforming to a preset shape and the end point of the movement indicated by the movement line being a preset position, determine that the object is an actual package and the actual package slides into the sorting port. Alternatively, in response to the shape of the object conforming to a preset shape and the movement direction indicated by the movement line being a preset direction, determine that the object is an actual package and the actual package slides into the sorting port. Alternatively, based on the package recognition model, if the recognition result of the shape and the movement line is that the package is successfully recognized, determine that the object is an actual package and the actual package slides into the sorting port.

[0025] In a possible embodiment, the processing unit is specifically configured to: determine a parcel sliding-in record in which the second sorting port and the first sorting port are the same, included in at least one parcel sliding-in record, as a sliding-in record to be matched. When the time difference between the actual sliding-in time included in all the sliding-in records to be matched and the first sliding-in time exceeds a threshold, determine that the parcel to be detected has a sorting abnormality.

[0026] In a possible embodiment, the acquisition unit is further used to acquire the second sliding-in time when the package to be detected is expected to slide into the third sorting port, and the third sorting port is the sorting port where the package to be detected is expected to slide abnormally. The processing unit is further used to confirm that the package to be detected slides abnormally into the third sorting port when there is an abnormal sliding-in record in the remaining sliding-in records. The second sorting port included in the abnormal sliding-in record is the same as the third sorting port, and the time difference between the actual sliding-in time included and the second sliding-in time does not exceed a threshold. The remaining sliding-in records include the sliding-in records of the packages except the sliding-in records of the target packages corresponding to other packages to be detected.

[0027] In a possible embodiment, the device further includes: an output unit. The output unit is used to output alarm information, where the alarm information is used to indicate that the to-be-detected parcel is abnormally sorted and / or that the to-be-detected parcel slides abnormally into the third sorting port.

[0028] In a third aspect, a computer device is provided, comprising: a processor and a memory. The processor is connected to the memory, the memory is used to store computer-executable instructions, and the processor executes the computer-executable instructions stored in the memory, thereby implementing any one of the methods provided in the first aspect.

[0029] In a fourth aspect, a readable storage medium is provided, comprising computer execution instructions, which, when executed on a computer device, causes the computer device to execute any one of the methods provided in the first aspect.

[0030] In a fifth aspect, a computer program product is provided, comprising computer execution instructions, which, when executed on a computer device, cause the computer device to execute any one of the methods provided in the first aspect.

[0031] In a sixth aspect, a sorting anomaly detection system is provided, comprising: a computer device, and an intelligent camera. The intelligent camera obtains the shape and movement path of the object identified in the image acquisition area, determines that the object is an actual package according to the shape and movement path, and generates a package sliding-in record corresponding to the actual package when the actual package slides into the sorting port, and outputs the package sliding-in record to a processing unit. The package sliding-in record includes the second sorting port into which the actual package actually slides, and the actual sliding-in time. The processing unit executes any one of the sorting anomaly detection methods provided in the first aspect.

[0032] The technical effects brought about by any implementation method in the second to sixth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a parcel sorting scenario provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0035] Figure 3 A flowchart of a sorting anomaly detection method provided in an embodiment of the present application;

[0036] Figure 4 A flowchart of another sorting anomaly detection method provided in an embodiment of the present application;

[0037] Figure 5 A schematic diagram of the structure of a sorting anomaly detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "plurality" means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different.

[0039] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0040] First of all, in order to facilitate the understanding of the present application, the relevant elements involved in the present application are described.

[0041] A package is a package that is formed by encapsulating one or more items. It has the functions of storing, protecting and facilitating transportation. In addition, a barcode or QR code can be set on the surface of the package to facilitate quick identification and processing during transportation.

[0042] Package tracking refers to two or more packages entering the same sorting port at the same time, or entering the same sorting port one after the other with a very short interval. When package tracking occurs, packages may be stacked or close to each other, which may easily lead to incorrect counting of the number of packages entering the sorting port, resulting in problems such as missed detection or false detection.

[0043] Secondly, the application scenarios involved in this application are briefly introduced.

[0044] With the rapid development of industrial automation, the coverage of automated sorting equipment is getting higher and higher. Automated sorting equipment can automatically control the sorting vehicle to transport the parcel to the corresponding sorting port. However, in the process of the sorting vehicle transporting the parcel to the corresponding sorting port, there may be problems such as sorting vehicle transport errors, which may easily cause the parcel to be unable to enter the corresponding sorting port, and then cause the parcel to be delivered to the wrong destination.

[0045] Next, the implementation environment (implementation architecture) involved in this application is briefly introduced.

[0046] The embodiment of the present application provides a sorting anomaly detection method and system. The sorting anomaly detection system may include a smart camera and a processing unit.

[0047] The smart camera is used to obtain the shape and movement path of the object identified in the image acquisition area, and when it is determined that the object is an actual package according to the shape and movement path and the actual package slides into the sorting port, a package sliding-in record corresponding to the actual package is generated, and the package sliding-in record is output to the processing unit. The package sliding-in record includes the second sorting port where the actual package actually slides into, and the actual sliding-in time. Optionally, the smart camera can be a camera, a ball camera or other type of equipment with data acquisition and data processing capabilities.

[0048] The processing unit is used to execute the sorting anomaly detection method provided in the embodiment of the present application. Optionally, the processing unit can be deployed in the smart camera, or in a computer device different from the smart camera.

[0049] The embodiments of the present application do not impose any restrictions on the specific form of the computer device. For example, the computer device may specifically be an industrial control computer with a sorting anomaly detection function, or other equipment configured with a sorting anomaly detection function. Among them, the industrial control computer can be connected to various external devices on the sorting machine. Such as a barcode reader or RFID reader / writer on a workbench, an actuator that performs a slide-out action on a sorting vehicle, a smart camera, a display, etc. Other devices may be servers, etc. The server may be a physical or logical server, or it may be two or more physical or logical servers that share different responsibilities and work together to implement the various functions of the server.

[0050] For example, Figure 1 , which is a schematic diagram of a parcel sorting scenario provided in an embodiment of the present application. Figure 1 The automated sorting equipment (or sorting machine) shown in the figure includes a workbench 101, a sorting vehicle 102, a sorting port 103 and a smart camera 104. The workbench 101 may be equipped with a code reader or a radio frequency identification (RFID) reader / writer. After the staff puts the package on the workbench 101, the industrial control computer can read the barcode or QR code on the package through the code reader or RFID reader / writer to identify the destination and other information of the package, thereby allocating the sorting vehicle 102 and the sorting port 103 for the package. In this way, the sorting vehicle 102 can transport the package to the corresponding assigned sorting port 103, and perform a slide-out action to slide the package into the corresponding assigned sorting port 103. In addition, the actuator on the sorting vehicle 102 can send the time when the sorting vehicle 102 performs the slide-out action to the industrial control computer. In addition, the smart camera 104 can collect image or video information of each sorting port 103, and generate a package slide-in record corresponding to the actual package based on the collected image or video information. The parcel sliding-in record includes the second sorting port into which the actual parcel slides, and the actual sliding-in time.

[0051] Furthermore, when the processing unit in the sorting anomaly detection system is deployed in an industrial control computer, the industrial control computer can obtain the package sliding-in record corresponding to the actual package from the smart camera 104, so as to perform sorting anomaly detection in combination with the time when each sorting vehicle 102 performs the sliding-out action.

[0052] Alternatively, when the processing unit in the sorting anomaly detection system is deployed in the smart camera 104, the smart camera 104 can obtain the time when the sorting vehicle 102 performs the sliding-out action from the industrial control computer, so as to perform sorting anomaly detection in combination with the package sliding-in record corresponding to the actual package.

[0053] It should be understood that Figure 1 What is shown is an example of an equipment form of the automated sorting equipment, and does not impose any restrictions on the actual form of the automated sorting equipment. For example, the automated sorting equipment may also be a multi-layer structure, a U-shaped structure, or a circular structure.

[0054] In terms of hardware implementation, the above-mentioned smart camera or computer device can be implemented as follows: Figure 2 The electronic device 20 shown is implemented. Figure 2 FIG. 2 is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 can be used to implement the functions of the above-mentioned smart camera or computer device.

[0055] Figure 2 The electronic device 20 shown may include: a processor 201 , a memory 202 , a communication interface 203 , and a bus 204 . The processor 201 , the memory 202 , and the communication interface 203 may be connected via the bus 204 .

[0056] The processor 201 is the control center of the electronic device 20, and may be a general-purpose central processing unit (CPU) or other general-purpose processors, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0057] As an example, the processor 201 may include one or more CPUs, such as Figure 2 CPU0 and CPU1 are shown in the figure.

[0058] The memory 202 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0059] In a possible implementation, the memory 202 may exist independently of the processor 201. The memory 202 may be connected to the processor 201 via a bus 204 and used to store data, instructions or program codes. When the processor 201 calls and executes the instructions or program codes stored in the memory 202, the sorting anomaly detection method provided in the embodiment of the present application can be implemented.

[0060] In another possible implementation, the memory 202 may also be integrated with the processor 201 .

[0061] The communication interface 203 is used for connecting the electronic device 20 to other devices through a communication network, which may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 203 may include a receiving unit for receiving data and a sending unit for sending data.

[0062] The bus 204 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0063] It should be pointed out that Figure 2 The structure shown in the figure does not constitute a limitation on the electronic device 20, except Figure 2 In addition to the components shown, the electronic device 20 may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0064] For ease of understanding, the sorting anomaly detection method provided by the present application is specifically introduced below with reference to the accompanying drawings.

[0065] like Figure 3 FIG. 1 is a flow chart of a sorting anomaly detection method provided by the present application. The method includes: S301-S303.

[0066] S301: Acquire a pre-assigned first sorting port and a first sliding-in time estimated to slide into the first sorting port of the package to be inspected.

[0067] The package to be inspected can be a container that stores materials, such as a material box, a storage box, or other names. The materials can be food, daily necessities, and other items, or they can be parts processed during the production process of an industrial production line.

[0068] The first sorting port is a sorting port assigned to the package to be detected. For example, a plurality of sorting ports may be provided on the sorting machine. Multiple packages delivered to the same sorting port may be delivered to the same destination. That is, one sorting port corresponds to one delivery destination. Moreover, different sorting ports may correspond to different delivery destinations. The processing unit may assign a first sorting port corresponding to the package to be detected based on the delivery destination of the package to be detected before the package to be detected is placed on the sorting machine. Furthermore, the processing unit may assign a corresponding sorting vehicle to the package to be detected, and cause the sorting vehicle assigned to the package to be detected to slide out at the first sorting port. In this way, the package to be detected may leave the correspondingly assigned sorting vehicle at the first sorting port, and slide into the first sorting port through the package channel (such as a ramp) at the first sorting port.

[0069] The first sliding-in time when the parcel to be inspected is expected to slide into the first sorting port is the time when the parcel to be inspected is expected to slide into the first sorting port after the parcel to be inspected is assigned to the sorting vehicle and the sorting port. It should be understood that the first sliding-in time is not the time when the parcel to be inspected actually slides into the first sorting port. It should be understood that the time can also be replaced by time, time period or other names without limitation.

[0070] Optionally, the processing unit may determine the first sliding-in time when the package to be inspected is expected to slide into the first sorting port based on the time when the package to be inspected is placed on the corresponding assigned sorting vehicle, the time the package to be inspected is transported with the sorting vehicle, and the sliding time when the package to be inspected leaves the sorting vehicle at the first sorting port and slides into the first sorting port.

[0071] Alternatively, the processing unit may also be based on the time when the sorting vehicle corresponding to the package to be detected performs the sliding action and the sliding time of the package to be detected from leaving the sorting vehicle at the first sorting port to sliding into the first sorting port.

[0072] S302: Obtain at least one package sliding-in record.

[0073] The parcel sliding-in record includes the second sorting port into which the actual parcel actually slides, and the actual sliding-in time. The second sorting port is any sorting port on the sorting machine. That is, the parcel sliding-in record can be used to record the actual sliding-in event of the actual parcel on any sorting port. The actual sliding-in time is the time when the actual parcel slides out of the parcel channel at the second sorting port and falls into the second sorting port.

[0074] Considering that there are many packages that need to be sorted in actual application scenarios, and the number of sorting ports on the sorting machine is also large, the number of smart cameras deployed is large. If a single processing unit processes a large amount of image or video data collected by the smart camera to generate a record of the package sliding in, it is easy to reduce the detection speed of the processing unit for sorting anomalies, and a large amount of image or video data needs to be transmitted between the processing unit and a large number of smart cameras, which is easy to lead to higher communication bandwidth and processing performance deployment costs. In order to avoid these problems, in the embodiment of the present application, the smart camera generates a record of the package sliding in based on the collected image or video data. In this way, not only can the detection speed of the processing unit for sorting anomalies be improved, but also the deployment cost can be reduced, effectively reducing the communication pressure in the sorting system and the computing pressure of the processing unit.

[0075] The method for generating a parcel sliding-in record by a smart camera may include: obtaining the shape and movement path of an object identified in an image acquisition area, and generating a parcel sliding-in record corresponding to the actual parcel when the object is determined to be an actual parcel according to the shape and movement path of the object and the actual parcel slides into a sorting port. The image acquisition area is the sorting port area where the smart camera is deployed, and there may be one or more sorting ports. The movement path of the object may be determined based on the position of the object in each frame of a plurality of frames of images acquired continuously.

[0076] For example, the smart camera can identify whether the shape of the object conforms to the shape of the package, and whether the object's motion path is a path that slides along the package channel to the sorting port. If the smart camera determines that the shape of the object conforms to the shape of the package, and the object's motion path is a path that slides along the package channel to the sorting port, then it can be determined that the object is an actual package and that the actual package slides into the sorting port. Furthermore, the smart camera can determine that the sorting port corresponding to the image acquisition area where the actual package is located is the second sorting port that the actual package actually slides into, and the end time of the actual package's motion path is the actual sliding-in time of the actual package.

[0077] S303: When there is no target package sliding-in record in at least one package sliding-in record, confirming that the sorting of the package to be detected is abnormal.

[0078] The second sorting port included in the target package sliding-in record is the same as the first sorting port, and the time difference between the actual sliding-in time and the first sliding-in time does not exceed the threshold. The time difference between the actual sliding-in time and the first sliding-in time does not exceed the threshold, that is, the interval time length of the actual sliding-in time relative to the first sliding-in time is less than or equal to the threshold. Optionally, the threshold can be zero or 1 second, or other values ​​that meet actual needs.

[0079] The processing unit may match at least one parcel sliding-in record according to the pre-assigned first sorting port of the parcel to be detected and the first sliding-in time when the parcel is expected to slide into the first sorting port.

[0080] If a target package sliding-in record is matched, in which the second sorting port is the same as the first sorting port, and the time difference between the actual sliding-in time and the first sliding-in time does not exceed the threshold, it can be indicated that the event of the package to be detected sliding into the first sorting port occurred at the first sliding-in time. Then the processing unit determines that the sorting of the package to be detected is normal.

[0081] If the target package sliding-in record is not matched, in which the second sorting port is the same as the first sorting port and the time difference between the actual sliding-in time and the first sliding-in time does not exceed the threshold, it can be indicated that the event of the to-be-detected package sliding into the first sorting port did not occur at the first sliding-in time. Then the processing unit determines that the to-be-detected package is sorted abnormally.

[0082] Exemplarily, it is assumed that there are two parcel slide-in records, namely, parcel slide-in record 1 including sorting port 1 and moment 1, and parcel slide-in record 2 including sorting port 2 and moment 2. Moreover, the first sorting port of the parcel to be detected is sorting port 1, and the first slide-in moment is moment 3. Among them, the time difference between moment 1 and moment 3 exceeds the threshold, and the time difference between moment 2 and moment 3 does not exceed the threshold. In this case, for parcel slide-in record 1, the processing unit can determine that the sorting port 1 included therein is the same as the first sorting port, but the time difference between the included moment 1 and the first slide-in moment exceeds the threshold, and it is not the target parcel slide-in record. For parcel slide-in record 2, the processing unit can determine that the sorting port 2 included therein is not the same as the first sorting port, and the time difference between the included moment 2 and the first slide-in moment does not exceed the threshold, and it is not the target parcel slide-in record. Then the processing unit can confirm that the parcel to be detected is abnormal in the case where there is no target parcel slide-in record in at least one parcel slide-in record.

[0083] Furthermore, it is assumed that there is a parcel sliding-in record 3 including the sorting port 1 and the time 4, and the time difference between the time 3 and the time 4 does not exceed the threshold. Then for the parcel sliding-in record 3, the processing unit can determine that the sorting port 1 included therein is the same as the first sorting port, and the time difference between the included time 3 and the first sliding-in time does not exceed the threshold, and it is the target parcel sliding-in record. Furthermore, the processing unit can confirm that the sorting of the parcel to be detected is normal when there is a target parcel sliding-in record in at least one parcel sliding-in record.

[0084] In this way, the processing unit can accurately determine whether there is a sorting abnormality in the package to be detected by determining whether there is a target package sliding-in record in at least one package sliding-in record, so that the staff can handle it in time when there is a sorting abnormality in the package to be detected, avoiding the problem of the package being delivered to the wrong destination.

[0085] Furthermore, considering that when a parcel slides on the parcel channel of the sorting port, the image of the parcel captured by the smart camera may not include information such as the barcode or QR code on the parcel, making it difficult to identify specific information such as the parcel number and delivery destination, and it is impossible to determine whether the parcel corresponds to the parcel assigned to the sorting port. In this case, the present application can support the smart camera to only identify whether there is a parcel in the image and the actual sliding time of the parcel, without specifically identifying information such as the barcode or QR code on the parcel, so as to complete the detection of parcel sorting anomalies, which has high practicality.

[0086] In one embodiment, when a frame extraction method or a single-frame image is used to identify whether there is a package at the sorting port, it is easy to identify a package or other object stationary on a slope as a package that slides into the sorting port, causing a false detection problem and increasing the false alarm rate of the sorting system. In order to avoid such situations, when the smart camera determines that the object is an actual package and the actual package slides into the sorting port based on the shape and movement path of the object, it can respond to the shape of the object being a preset shape and the end point of the movement indicated by the movement path being a preset position, and determine that the object is an actual package and the actual package slides into the sorting port, that is, determine that the object is an actual package that slides into the sorting port. The preset position is the position where the end of the package channel at the sorting port falls into the sorting port.

[0087] For example, the smart camera may be configured with a recognition model and a position detection algorithm. The recognition model may be used to recognize the morphology of an object in an image. Based on this, when the smart camera recognizes through the recognition model that the morphology of an object on the sorting port conforms to a preset morphology, it may determine that a package exists, and continuously capture multiple frames of images, so as to further identify the position information of the package in each frame of the image based on the position detection algorithm, and determine the movement route of the package based on the position information of the package in the continuous multiple frames of images. Furthermore, the smart camera may determine that the object is an actual package and that the actual package slides into the sorting port based on the morphological information and movement route of the object, and determine the moment when the package slides out of the preset position and falls into the sorting port as the actual sliding-in moment when the actual package actually slides into the sorting port. Furthermore, the smart camera may generate a package sliding-in record corresponding to the actual package.

[0088] For another example, the smart camera may determine that the object is an actual package and the actual package slides into the sorting port in response to the shape of the object conforming to a preset shape and the motion direction indicated by the motion path being a preset direction. The preset direction refers to the direction in which the package slides into the sorting port from top to bottom on the package channel of the sorting port.

[0089] For another example, the smart camera may be configured with a package recognition model. The package recognition model may be obtained by training based on a plurality of sample information. The sample information may be image information of an actual package sliding into the sorting port according to a motion path. In this way, the smart camera may identify the package successfully based on the recognition result of the package recognition model on the shape and motion path, and determine that the object is an actual package and the actual package slides into the sorting port.

[0090] Based on this, the smart camera can determine that the object is an actual package and the actual package slides into the sorting port according to the shape and movement route of the object, and determine whether the object is an actual package that slides in, avoiding identifying other items different from the package as the actual package that slides in, effectively avoiding false detection problems caused by misidentification, and improving the accuracy of sorting anomaly detection.

[0091] In one embodiment, when using the frame extraction method or based on a single-frame image to identify whether there is a package at the sorting port, it is impossible to detect the continuity of the package movement. If a package follows, multiple packages may slide into the sorting port at the same time, and the number of packages cannot be identified, resulting in an increase in the system's false alarm rate. In order to avoid such problems, when the smart camera determines that the object is an actual package based on the shape and movement path and the actual package slides into the sorting port, it can respond to identifying at least two objects with different shapes and meeting the preset shape in the same set of continuous multi-frame images, and the at least two objects correspond to at least two different movement paths, and determine that the at least two objects are at least two different actual packages, and the at least two actual packages both slide into the sorting port.

[0092] For example, if there are N movement routes of different package shapes in the continuous multi-frame images, it can be indicated that there is a package following situation in the continuous multi-frame images. That is to say, there are N actual packages that slide into the sorting port at the same time in the continuous multi-frame images, or there are N actual packages that slide into the sorting port one after another with a very short interval. The reason for this situation may be that a single sorting vehicle transports N packages, or it may be that the package on the sorting vehicle slides into a non-pre-allocated sorting port by mistake. Among them, N is a positive integer greater than 1. In this case, the smart camera can determine that the N movement routes in the continuous multi-frame images correspond to different actual packages, and generate N package sliding records.

[0093] Furthermore, the actual sliding-in moments in the N parcel sliding-in records may be the same, and the included second sorting ports may be the same. That is, the N parcels in the continuous multi-frame images slide into the same sorting port at the same actual sliding-in moment. Subsequently, the processing unit may determine whether there are N parcels whose pre-assigned sorting ports are the same sorting port at the same actual sliding-in moment, so as to accurately determine whether there is a parcel that mistakenly slides into the same sorting port. Based on this, the embodiment of the present application can support the effective identification of different parcels in the same set of continuous multi-frame images by introducing the movement line of the parcel, so as to accurately identify sorting anomalies. Compared with the method of identifying whether a parcel slides into a sorting port based on a photoelectric sensor, the embodiment of the present application can support the identification of simultaneous sliding-in events of at least two parcels, avoiding the problem that the photoelectric sensor can only identify one parcel sliding in when at least two parcels slide in at the same time, thereby improving the accuracy of parcel identification.

[0094] In one embodiment, the first sliding-in time refers to the estimated time of entering the first sorting port calculated based on the time when the package to be inspected leaves the sorting vehicle.

[0095] In the above S301, that is, when the processing unit obtains the pre-assigned first sorting port and the first sliding-in time estimated to slide into the first sorting port of the package to be detected, the embodiment of the present application provides an optional implementation method, including: S3011-S3014.

[0096] S3011: Obtain sorting information of the package to be inspected.

[0097] The sorting information is used to indicate the first sorting port and the sorting vehicle that transports the package to be inspected.

[0098] S3012: Determine the time when the sorting vehicle performs the sliding-out action.

[0099] The sliding action is used to make the package to be inspected leave the sorting vehicle. For example, the sliding action may be an action in which the actuator of the sorting vehicle controls the rotating mechanism to rotate. After the rotating mechanism rotates, the platform on the sorting vehicle carrying the package to be inspected may tilt, so that the package to be inspected leaves the sorting vehicle and slides toward the sorting port.

[0100] For example, the actuator on the sorting vehicle can send clock information to the processing unit while performing the sliding action. Then, the processing unit can determine the time when the sorting vehicle that transports the package to be inspected performs the sliding action. For another example, the processing unit can send an execution instruction to the actuator on the sorting vehicle to make the actuator on the sorting vehicle perform the sliding action, and determine the sending time of the execution instruction as the time when the actuator on the sorting vehicle performs the sliding action. Then, the processing unit can determine the time when the sorting vehicle that transports the package to be inspected performs the sliding action.

[0101] S3013: Obtain a first sliding time duration corresponding to the first sorting port.

[0102] The first sliding time is the sliding time of the parcel from the sorting vehicle at the first sorting port to the first sorting port. When the length and slope of the parcel channel at the first sorting port are fixed, the time for parcels of different masses to slide from the upper end to the lower end of the parcel channel at the first sorting port is the same. That is, different parcels slide on the parcel channel at the first sorting port for the same length of time, which is the first sliding time.

[0103] In one possible implementation, the sliding time corresponding to each sorting port can be pre-configured in the processing unit. In this way, after the processing unit determines that the package to be inspected is assigned to the first sorting port, it can further determine the first sliding time corresponding to the first sorting port.

[0104] S3014: Determine a first sliding-in time based on the time when the sorting vehicle performs the sliding-out action and the first sliding duration.

[0105] For example, the processing unit may add the time for the sorting vehicle to perform the sliding-out action to the first sliding duration corresponding to the first sorting port to obtain the first sliding-in time of the package to be detected.

[0106] For another example, after the processing unit adds the time when the sorting vehicle performs the sliding-out action to the first sliding duration corresponding to the first sorting port to obtain a result, the processing unit can subtract the first duration from the result obtained by the addition to obtain the minimum value of the interval, and add the first duration to the result obtained by the addition to obtain the maximum value of the interval. Furthermore, the processing unit can determine any time within the period consisting of the minimum value of the interval and the maximum value of the interval as the first sliding-in time of the package to be detected.

[0107] Since the sorting vehicle performs the sliding-out action only when it moves to a position close to the sorting port, that is, the package to be detected slides out from the sorting vehicle at a position close to the sorting port to slide into the sorting port, and the slope at the sorting port is generally shorter. Therefore, the time required for the entire sliding process of the package to be detected on the slope is shorter. In this case, the first sliding-in time of the package to be detected determined by the processing unit based on the time when the sorting vehicle performs the sliding-out action has higher accuracy, and can improve the method of predicting the sliding-in time based on the time when the package is placed on the sorting vehicle, and the problem of low accuracy caused by the longer conveyor belt of the sorting machine, thereby improving the accuracy of sorting anomaly detection.

[0108] In an optional embodiment, in the above S303, that is, when the processing unit confirms that the sorting of the package to be detected is abnormal when there is no target package sliding-in record in at least one package sliding-in record, the embodiment of the present application provides an optional implementation method, including: S3031-S3032.

[0109] S3031: Determine the parcel sliding-in record in which the second sorting port and the first sorting port are the same as each other in at least one parcel sliding-in record as the sliding-in record to be matched.

[0110] Considering that a large number of sorting ports can be deployed on the sorting machine, a large number of parcel slide-in records can be generated in a short period of time. In order to improve the detection efficiency of sorting anomalies, the processing unit can first filter the at least one parcel slide-in record when matching it in at least one parcel slide-in record based on the estimated information of the parcel to be detected. For example, the processing unit can determine the parcel slide-in record in which the second sorting port included in the at least one parcel slide-in record is the same as the first sorting port as the slide-in record to be matched. In this way, the processing unit can quickly eliminate the slide-in records of other sorting ports that do not need to be matched, thereby improving the matching efficiency.

[0111] S3032: When the time difference between the actual slide-in time included in all slide-in records to be matched and the first slide-in time exceeds a threshold, it is determined that there is a sorting abnormality in the package to be detected.

[0112] If the time difference between the actual slide-in time included in all slide-in records to be matched and the first slide-in time exceeds the threshold, that is, there is no actual slide-in time in all slide-in records to be matched whose time difference with the first slide-in time does not exceed the threshold, it can indicate that the package to be sorted has not slid into the first sorting port. In this case, the processing unit can determine that the detection result is used to indicate that there is a sorting anomaly in the package to be detected. In addition, the processing unit can also record a missed detection event corresponding to the first sorting port, so that the staff can handle it in time to avoid the problem of the package to be detected being delivered to the wrong destination.

[0113] If the time difference between the actual sliding-in time and the first sliding-in time included in the sliding-in record to be matched does not exceed the threshold, it can indicate that the package to be sorted has slid into the first sorting port. In this case, the processing unit can determine that the detection result is used to indicate that the sorting of the package to be detected is normal.

[0114] In a possible embodiment, the number of packages to be detected may be one or more. In other words, the same sorting vehicle can transport one or more packages. In this case, if the number of actual sliding-in moments whose time difference with the first sliding-in moment does not exceed the threshold is not equal to the number of packages to be detected, it may be that in addition to the package to be detected sliding into the first sorting port at the actual sliding-in moment, other packages mistakenly slide into the first sorting port, or there may be a false detection event in which a stationary package or other object on the slope is mistakenly identified as a package sliding into the sorting port. Then the processing unit can determine that there is an abnormal package in the first sorting port, so that the staff can handle it in time and avoid the problem of the package being delivered to the wrong destination. In addition, the processing unit can generate a normal matching event and multiple false detection events. The normal matching event is used to record that the package to be detected slides into the first sorting port correctly. The false detection event is used to record that there is a false detection package in the first sorting port. The number of multiple false detection events is the difference between the number of packages corresponding to the actual sliding-in moment and the number of packages to be detected.

[0115] In one embodiment, the sorting anomaly detection method provided in the embodiment of the present application further includes: S401.

[0116] S401: In response to identifying that a movement path of an object does not change in a preset area of ​​any sorting port, determining that there is a bag jam at any sorting port.

[0117] If it is identified that the movement path of the object does not change in the preset area of ​​any sorting port, it can be indicated that there are stationary packages in the preset area of ​​the arbitrary sorting port, forming a package blockage. In this case, the processing unit can determine that there is a package blockage in the arbitrary sorting port. Or further, the processing unit can output a prompt message to prompt that there is a package blockage in the arbitrary sorting port.

[0118] For example, the processing unit may determine that the movement path of the object identified in the preset area of ​​any sorting port does not change when there is a continuous detection time of a package in any sorting port that is greater than or equal to a preset time. The continuous detection time of the package is used to indicate the time length for which the package is continuously detected in the preset area of ​​any sorting port. The preset area may be an area at the lower end of the slope of the sorting port, or an area at the upper end of the slope of the sorting port, or other areas on the slope of the sorting port, without limitation. It should be understood that the package is continuously detected in the preset area of ​​any sorting port, that is, the presence of the package in the preset area of ​​any sorting port is recognized in multiple consecutive frames of images.

[0119] Based on this, the present application can realize the identification of non-moving objects in the sorting port area, and can accurately identify the blockage of the sorting port caused by a single or multiple packages staying, so as to facilitate the staff to handle it in time and avoid the blockage of packages affecting the operation of the entire sorting system.

[0120] In one embodiment, after the above S303, that is, after the processing unit confirms that the sorting of the package to be detected is abnormal, the sorting abnormality detection method provided in the embodiment of the present application further includes: S501-S502.

[0121] S501: Obtaining a second sliding-in time at which the package to be inspected is expected to slide into the third sorting port.

[0122] The third sorting port is a sorting port into which the package to be detected is expected to slide abnormally. The third sorting port may also be any one of the multiple sorting ports on the sorting machine. For example, considering that the package to be detected may fall into a sorting port adjacent to the pre-assigned first sorting port, the third sorting port may be a sorting port adjacent to the first sorting port.

[0123] It should be understood that the implementation method of S501 can refer to the specific description in the above S3011-S3014 and will not be repeated here.

[0124] S502: When there is an abnormal slide-in record in the remaining slide-in records, confirm that the package to be detected slides abnormally into the third sorting port.

[0125] The second sorting port and the third sorting port included in the abnormal sliding-in record are the same, and the time difference between the actual sliding-in time and the second sliding-in time does not exceed a threshold.

[0126] The remaining slide-in records include the slide-in records of packages except the slide-in records of the target packages corresponding to other packages to be detected. That is, the remaining slide-in records are the slide-in records of packages that are not successfully matched to the slide-in records of the target packages of other packages to be detected in at least one slide-in record.

[0127] If there is an abnormal sliding-in record in the remaining sliding-in records, it can be indicated that the package to be detected did not slide into the first sorting port, and mistakenly slid into the third sorting port. In this case, the processing unit can confirm that the package to be detected slides into the third sorting port abnormally.

[0128] If there is no abnormal slide-in record in the remaining slide-in records, it may indicate that the package to be detected may have actually slid into the first sorting port but the package drop was not successfully identified. Alternatively, the package to be detected may have abnormally slid to the ground or other locations outside the sorting port during transportation by the sorting vehicle, resulting in the package not being identified as falling at any sorting port. In this case, the processing unit may determine that there is an identification abnormality or a transportation abnormality in the package to be detected.

[0129] Furthermore, if there is an identification anomaly in the package to be detected, the package to be detected will still be correctly delivered to the destination. In this case, the processing unit can automatically eliminate the identification anomaly by comparing the delivery information of the package to be detected. If there is a delivery anomaly in the package to be detected, the staff can re-assign the package to be detected when they find the package to be detected outside the sorting port, and manually operate to eliminate the identification anomaly. In this case, the processing unit can eliminate the identification anomaly in response to the staff's operation.

[0130] In one embodiment, after the processing unit confirms that the sorting of the package to be detected is abnormal, the sorting abnormality detection method provided in the embodiment of the present application further includes: S601.

[0131] S601: Output alarm information.

[0132] After the processing unit confirms that the parcel to be detected is sorted abnormally, it can also output an alarm message, so that the staff can handle it in time and avoid the parcel to be detected from being delivered to the wrong destination. The alarm message can be used to indicate that the parcel to be detected is sorted abnormally. Or further, if it is confirmed that the parcel to be detected slides abnormally into the third sorting port, the alarm message can also be used to indicate that the parcel to be detected slides abnormally into the third sorting port.

[0133] In one embodiment, Figure 4 FIG. 1 is a flow chart of another sorting anomaly detection method provided in an embodiment of the present application. The method includes: S701-S705.

[0134] S701: Obtain sorting information of the package to be inspected.

[0135] The sorting information is used to indicate the sorting vehicle that transports the package to be inspected, and the first sorting port that is allocated to the package to be inspected.

[0136] For example, in combination Figure 1 , the processing unit can establish a connection with the barcode reader or RFID reader / writer configured on the workbench of the sorting machine. After the staff puts the package to be tested on the workbench, the barcode reader or RFID reader / writer configured on the workbench can identify the barcode or QR code on the package to be tested to determine the number and delivery destination of the package to be tested, and send the number and delivery destination of the package to be tested to the processing unit. In this way, the processing unit can determine the number and delivery destination of the package to be tested, and determine the sorting vehicle used to transport the package to be tested, as well as the first sorting port corresponding to the package to be tested.

[0137] S702: Determine a first sliding-in time based on the time when the sorting vehicle performs the sliding-out action and the first sliding duration.

[0138] It should be understood that the specific implementation method of S702 can refer to the description in the above S3012-S3014, which will not be repeated here.

[0139] S703: Obtain a parcel sliding-in record to be matched corresponding to the first sorting port in at least one parcel sliding-in record.

[0140] S704: When the time difference between the actual slide-in time and the first slide-in time included in all slide-in records to be matched exceeds a threshold, it is determined that there is a sorting abnormality in the package to be detected.

[0141] It should be understood that the specific implementation of S703-S704 can refer to the description in the above S3031-S3032, which will not be repeated here.

[0142] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the method. It is understandable that in order to realize the above functions, the processing unit in the sorting anomaly detection system includes at least one of the hardware structure and software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0143] The embodiment of the present application can divide the processing unit into functional units according to the above method example. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0144] For example, Figure 5 The schematic diagram of the structure of a sorting anomaly detection device is shown. The sorting anomaly detection device 80 can be used to execute the method involved in the above embodiment. The sorting anomaly detection device 80 includes: an acquisition unit 801 and a processing unit 802.

[0145] The acquisition unit 801 is used to acquire a pre-assigned first sorting port and a first sliding-in time estimated to slide into the first sorting port of the package to be detected.

[0146] The acquisition unit 801 is further used to acquire at least one parcel sliding-in record. The acquisition unit 801 is further specifically used in a method for generating a parcel sliding-in record, including acquiring the shape and motion path of an object identified in an image acquisition area, and generating a parcel sliding-in record corresponding to the actual parcel when the object is determined to be an actual parcel according to the shape and motion path and the actual parcel slides into the sorting port. The parcel sliding-in record includes the second sorting port into which the actual parcel actually slides, and the actual sliding-in time.

[0147] The processing unit 802 is used to confirm that the parcel to be detected is abnormal in sorting when there is no target parcel sliding-in record in at least one parcel sliding-in record. The second sorting port included in the target parcel sliding-in record is the same as the first sorting port, and the time difference between the actual sliding-in time and the first sliding-in time does not exceed a threshold.

[0148] In a possible embodiment, the first sliding-in time refers to an estimated time of entering the first sorting port calculated based on the time when the package to be inspected leaves the sorting vehicle.

[0149] In a possible embodiment, the acquisition unit 801 is specifically used to: acquire sorting information of the package to be detected. The sorting information is used to indicate the first sorting port and the sorting vehicle that transports the package to be detected. Determine the moment when the sorting vehicle performs a slide-out action, and the slide-out action is used to make the package to be detected leave the sorting vehicle. Acquire a first sliding duration corresponding to the first sorting port. The first sliding duration is the sliding duration of the package from the sorting vehicle at the first sorting port to the first sorting port. Based on the moment when the sorting vehicle performs the slide-out action and the first sliding duration, determine the first slide-in time.

[0150] In a possible embodiment, the acquisition unit 801 is specifically used to: in response to identifying at least two objects with different shapes and conforming to preset shapes in the same group of continuous multi-frame images, and the at least two objects correspond to at least two different motion routes, determine that the at least two objects are at least two different actual packages, and the at least two actual packages both slide into the sorting port.

[0151] In a possible embodiment, the image acquisition area covers a plurality of sorting ports. The processing unit 802 is further configured to determine that there is a packet blockage at any sorting port in response to identifying that the movement route of the object does not change within a preset area of ​​any sorting port.

[0152] In a possible embodiment, the acquisition unit 801 is specifically used to: in response to the shape of the object conforming to a preset shape and the motion end point indicated by the motion line being a preset position, determine that the object is an actual package and the actual package slides into the sorting port. Alternatively, in response to the shape of the object conforming to a preset shape and the motion direction indicated by the motion line being a preset direction, determine that the object is an actual package and the actual package slides into the sorting port. Alternatively, based on the package recognition model, the recognition result of the shape and the motion line is that the package recognition is successful, determine that the object is an actual package and the actual package slides into the sorting port.

[0153] In a possible embodiment, the processing unit 802 is specifically configured to: determine a parcel sliding-in record in which the second sorting port and the first sorting port are the same, included in at least one parcel sliding-in record, as a sliding-in record to be matched. When the time difference between the actual sliding-in time included in all the sliding-in records to be matched and the first sliding-in time exceeds a threshold, it is determined that the parcel to be detected has a sorting abnormality.

[0154] In a possible embodiment, the acquisition unit 801 is further used to acquire the second sliding-in time when the package to be detected is expected to slide into the third sorting port, and the third sorting port is the sorting port where the package to be detected is expected to slide abnormally. The processing unit 802 is further used to confirm that the package to be detected slides abnormally into the third sorting port when there is an abnormal sliding-in record in the remaining sliding-in records. The second sorting port included in the abnormal sliding-in record is the same as the third sorting port, and the time difference between the actual sliding-in time included and the second sliding-in time does not exceed a threshold. The remaining sliding-in records include the sliding-in records of the packages except the sliding-in records of the target packages corresponding to other packages to be detected.

[0155] In a possible embodiment, the device further includes: an output unit 803. The output unit 803 is used to output alarm information, where the alarm information is used to indicate that the to-be-detected package is abnormally sorted and / or that the to-be-detected package slides abnormally into the third sorting port.

[0156] For the detailed description of the above optional methods, please refer to the above method embodiments, which will not be repeated here. In addition, any of the above explanations of the processing unit in the sorting anomaly detection system and the description of the beneficial effects can refer to the above corresponding method embodiments, which will not be repeated here.

[0157] As an example, combining Figure 2 The functions implemented in part or in whole in the acquisition unit 801, the processing unit 802 and the output unit 803 in the sorting abnormality detection device 80 can be realized by Figure 2 Processor 201 in the Figure 2 The program code in the memory 202 is implemented.

[0158] An embodiment of the present application further provides a readable storage medium having a computer program stored thereon. When the computer program is executed on a computer device, the computer device executes any of the methods executed by the computer devices provided above.

[0159] For the explanation of the relevant contents and description of the beneficial effects of any of the readable storage media provided above, reference may be made to the corresponding embodiments above, which will not be repeated here.

[0160] The embodiment of the present application also provides a computer program product including instructions, when the instructions are run on a computer device, the computer device executes any one of the methods in the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer device, the process or function according to the embodiment of the present application is generated in whole or in part. The computer device may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a readable storage medium, or transmitted from one readable storage medium to another readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a computer device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means to another website site, computer, computer device, or data center. The readable storage medium may be any available medium that the computer device can access or a data storage device such as a computer device, a data center, etc. that includes one or more computer devices that can be integrated with the medium. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), etc.

[0161] It should be noted that the above-mentioned devices for storing computer instructions or computer programs provided in the embodiments of the present application, such as but not limited to the above-mentioned memories, readable storage media, etc., are all non-transitory.

[0162] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer device, a process or function according to an embodiment of the present application is generated in whole or in part. The computer device may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a readable storage medium, or transmitted from one readable storage medium to another readable storage medium, for example, a computer instruction may be transmitted from a website site, a computer, a computer device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, computer device or data center. The readable storage medium may be any available medium that a computer device can access or a data storage device such as a computer device, a data center, etc. that contains one or more computer devices that can be integrated with a medium. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), etc.

[0163] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other changes to the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0164] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A sorting anomaly detection method, characterized in that: include: Acquire a pre-assigned first sorting port of the package to be inspected and a first sliding-in time estimated to slide into the first sorting port; Get at least one package sliding record; The method for generating the parcel sliding-in record comprises: acquiring the shape and movement path of an object recognized in an image acquisition area, and generating a parcel sliding-in record corresponding to the actual parcel when the object is determined to be an actual parcel according to the shape and the movement path and the actual parcel slides into a sorting port; the parcel sliding-in record comprises the second sorting port into which the actual parcel actually slides, and the actual sliding-in time; In the case that there is no target package sliding-in record in the at least one package sliding-in record, it is confirmed that the sorting of the package to be detected is abnormal; the second sorting port included in the target package sliding-in record is the same as the first sorting port, and the time difference between the actual sliding-in time and the first sliding-in time does not exceed a threshold.

2. The sorting anomaly detection method according to claim 1, characterized in that: The first sliding-in time refers to the estimated time of entering the first sorting port calculated according to the time when the package to be inspected leaves the sorting vehicle.

3. The sorting anomaly detection method according to claim 2, characterized in that: The step of obtaining a pre-allocated first sorting port of the package to be detected and a first sliding-in time point estimated to slide into the first sorting port comprises: Acquire sorting information of the package to be detected; the sorting information is used to indicate the first sorting port and a sorting vehicle for transporting the package to be detected; Determining a moment when the sorting vehicle performs a sliding-out action, wherein the sliding-out action is used to make the package to be inspected leave the sorting vehicle; Obtaining a first sliding time corresponding to the first sorting port; the first sliding time is the sliding time for a parcel to slide from the sorting vehicle at the first sorting port to the first sorting port; The first sliding-in time is determined based on the time when the sorting vehicle performs the sliding-out action and the first sliding duration.

4. The sorting anomaly detection method according to claim 1, characterized in that: Determining that the object is an actual package according to the shape and the movement route and that the actual package slides into the sorting port includes: In response to identifying at least two objects with different shapes and conforming to preset shapes in the same group of continuous multi-frame images, and the at least two objects respectively correspond to at least two different motion routes, it is determined that the at least two objects are at least two different actual packages, and the at least two actual packages both slide into the sorting port.

5. The sorting anomaly detection method according to claim 1, characterized in that: The image acquisition area covers a plurality of sorting ports; The sorting anomaly detection method further includes: In response to identifying that the movement route of the object does not change in a preset area of ​​any sorting port, it is determined that there is a bag jam at the any sorting port.

6. The sorting anomaly detection method according to claim 1, characterized in that: The step of determining that the object is an actual package according to the form and the movement route and that the actual package slides into the sorting port includes: In response to the shape of the object conforming to a preset shape and the movement end point indicated by the movement route being a preset position, it is determined that the object is the actual package and the actual package slides into the sorting port; or, In response to the shape of the object conforming to the preset shape and the motion direction indicated by the motion route being the preset direction, determining that the object is the actual package and the actual package slides into the sorting port; or, The recognition result of the shape and the movement route based on the package recognition model is that the package recognition is successful, and it is determined that the object is the actual package and the actual package slides into the sorting port.

7. The sorting anomaly detection method according to claim 1, characterized in that: In the case that there is no target package sliding-in record in the at least one package sliding-in record, confirming that the sorting of the package to be detected is abnormal includes: Determine the parcel sliding-in record in which the second sorting port and the first sorting port are the same, included in the at least one parcel sliding-in record, as the sliding-in record to be matched; When the time difference between the actual slide-in time included in all the slide-in records to be matched and the first slide-in time exceeds a threshold, it is determined that there is a sorting abnormality in the package to be detected.

8. The sorting anomaly detection method according to claim 1, characterized in that: After confirming that the parcel to be detected is abnormal in sorting, the method further includes: Acquire a second sliding-in time at which the package to be detected is expected to slide into a third sorting port, where the third sorting port is the sorting port into which the package to be detected is expected to slide abnormally; In the case that there is an abnormal slide-in record in the remaining slide-in records, it is confirmed that the package to be detected slides into the third sorting port abnormally; the second sorting port included in the abnormal slide-in record is the same as the third sorting port, and the time difference between the actual slide-in time and the second slide-in time does not exceed a threshold; the remaining slide-in records include the package slide-in records excluding the target package slide-in records corresponding to other packages to be detected.

9. The sorting anomaly detection method according to any one of claims 1 to 8, characterized in that: After confirming that the parcel to be detected is abnormal in sorting, the method further includes: Outputting warning information, wherein the warning information is used to indicate that the parcel to be detected is sorted abnormally, and / or that the parcel to be detected slides abnormally into the third sorting port.

10. A sorting anomaly detection device, characterized in that: include: Acquisition unit and processing unit; The acquisition unit is used to acquire a pre-assigned first sorting port of the package to be detected and a first sliding-in time estimated to slide into the first sorting port; The acquisition unit is further used to acquire at least one parcel sliding-in record; the method for generating the parcel sliding-in record comprises acquiring the shape and movement path of the object identified in the image acquisition area, and generating a parcel sliding-in record corresponding to the actual parcel when the object is determined to be an actual parcel according to the shape and the movement path and the actual parcel slides into the sorting port; the parcel sliding-in record comprises the second sorting port into which the actual parcel actually slides, and the actual sliding-in time; The processing unit is used to confirm that the sorting of the to-be-detected package is abnormal when there is no target package sliding-in record in the at least one package sliding-in record; the second sorting port included in the target package sliding-in record is the same as the first sorting port, and the time difference between the actual sliding-in time and the first sliding-in time does not exceed a threshold.

11. A sorting anomaly detection system, characterized in that: include: Smart camera,processing unit; The smart camera acquires the shape and movement path of the object recognized in the image acquisition area, determines that the object is an actual package according to the shape and the movement path, and generates a package sliding-in record corresponding to the actual package when the actual package slides into the sorting port, and outputs the package sliding-in record to the processing unit; the package sliding-in record includes the second sorting port into which the actual package actually slides, and the actual sliding-in time; The processing unit executes the sorting anomaly detection method according to any one of claims 1 to 9.

Images