Analytical method, device, equipment and storage medium for item sorting

By acquiring and analyzing grid-based target images, identifying the item path to determine sorting accuracy, the problem of low positioning accuracy in traditional photoelectric sensors in large-area sorting fields is solved, and efficient and low-cost item sorting control is achieved.

CN113129367BActive Publication Date: 2024-12-06SF TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010039773.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-01-15
Publication Date
2024-12-06
Estimated Expiration
2040-01-15

AI Technical Summary

Technical Problem

Traditional photoelectric sensors have problems with reduced positioning accuracy during the item sorting process, especially when dust or dirt is attached, and a large number of sensors are required to cover a large-area sorting yard, resulting in high costs and complex maintenance.

Method used

By acquiring multiple gridded target images, identifying the actual path of the target item and comparing it with the preset path, determining whether the item is sorted abnormally, and using deep learning models to extract image features to identify the item location and path, reducing the Number of sensors and maintenance requirements.

Benefits of technology

It realizes high-precision positioning of items in large-area sorting yards, reduces the number of sensors and maintenance costs, and improves the accuracy and efficiency of the sorting process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113129367B_ABST
    Figure CN113129367B_ABST
Patent Text Reader

Abstract

The embodiments of the present application disclose an analysis method, device, equipment and storage medium for item sorting. The analysis method for item sorting includes: acquiring multiple gridded target images, the target images include images of target items, the target images include multiple grids, each of the grids corresponds to a sorting area; determining the actual path of the target item according to the respective target images, the actual path covers multiple sorting areas; determining whether the target item is abnormally sorted according to the actual path and the preset path of the target item. The embodiments of the present application achieve precise positioning of the position of the item, determine whether the item is abnormally sorted, and provide a reference for further transmission or sorting control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of machine vision technology, and specifically to an analysis method, device, equipment and storage medium for item sorting. Background Art

[0002] With the continuous development of sensor technology, sensors are widely used in various fields. Among them, photoelectric sensors are often used to locate objects. The principle of locating objects is as follows: after light passes through the part of the object to be measured and is absorbed, the transmitted light is projected onto the photoelectric element, or light is projected onto the object to be measured and reflected from the surface of the object to be measured and then projected onto the photoelectric element. The photoelectric element converts the received light signal into an electrical signal. The photoelectric sensor determines the position of the object to be measured based on the change of the electrical signal converted by the photoelectric element.

[0003] For example, in the field of logistics technology, photoelectric sensors are often used to detect whether there is an object (such as a package) in the detection area of ​​the photoelectric sensor, so as to determine whether the object has reached a designated sorting area, thereby completing the sorting of the object.

[0004] Photoelectric sensors have high requirements for the operating environment. If dust, dirt, etc. are attached to the photoelectric sensor, it will affect the change in the amount of light and cause the detection accuracy of the photoelectric sensor to decrease, and even affect the photoelectric sensor's ability to locate the object. Summary of the invention

[0005] The embodiments of the present application provide an analysis method, device, equipment and storage medium for item sorting, which can accurately locate the position of items, determine whether items are sorted abnormally, and provide a reference basis for the control process of item sorting.

[0006] On the one hand, an embodiment of the present application provides an analysis method for item sorting, the method comprising:

[0007] Acquire a plurality of gridded target images, wherein the target images include images of target objects, and the target images include a plurality of grids, each of the grids corresponding to a sorting area;

[0008] Determine the actual path of the target object according to the respective target images, wherein the actual path covers a plurality of sorting areas;

[0009] According to the actual path and the preset path of the target object, it is determined whether the target object is abnormally sorted.

[0010] In some embodiments of the present application, the step of acquiring a plurality of gridded target images includes:

[0011] Get the item size of the target item;

[0012] Determine the number of target grids according to the size of the object, where the number of target grids is used to indicate the number of grids of each of the target images;

[0013] A plurality of gridded target images are acquired according to the number of target grids.

[0014] In some embodiments of the present application, the step of acquiring a plurality of gridded target images according to the number of target grids includes:

[0015] Acquire a plurality of gridded images according to the number of target grids, and extract a first image feature of each of the gridded images, wherein the number of target grids is inversely related to the number of gridded images, the gridded image includes a plurality of grids, each of the grids corresponds to a sorting area, and the first image feature is used to identify whether the gridded image contains a target object;

[0016] Determining a similarity between the first image feature and a second image feature, wherein the second image feature is an image feature of a reference image;

[0017] According to the similarity, each of the target images is acquired from each of the gridded images, wherein the similarity between the first image feature and the second image feature of the target image is greater than a preset threshold.

[0018] In some embodiments of the present application, determining the actual path of the target object according to each target image includes:

[0019] Identify each of the target images to obtain respective target grids, wherein the target grids refer to the grids in which the target objects exist among the grids of the target images;

[0020] Determining a plurality of target sorting areas through which the target object passes according to the target grids and the identifiers of the target grids, wherein the identifiers of the target grids are used to indicate the sorting areas corresponding to the target grids;

[0021] The actual path of the target object is determined according to the respective target sorting areas.

[0022] In some embodiments of the present application, determining the actual path of the target object according to each target sorting area includes:

[0023] According to the shooting time of each of the target images, obtaining the target time for the target object to pass through each of the target sorting areas;

[0024] The actual path of the target object is determined according to the target times and the target sorting areas.

[0025] In some embodiments of the present application, determining whether the target object is abnormally sorted according to the actual path and the preset path of the target object includes:

[0026] Acquire a preset sorting area and an actual sorting area, wherein the preset sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path, and the actual sorting area refers to a plurality of sorting areas corresponding to and covered by the actual path;

[0027] Obtaining a matching degree between the preset sorting area and the actual sorting area;

[0028] Determine whether the target object is abnormally sorted according to the matching degree.

[0029] In some embodiments of the present application, determining whether the target object is abnormally sorted according to the actual path and the preset path of the target object includes:

[0030] Acquire a reference sorting area, where the reference sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path;

[0031] When a transmission failure is detected in the reference sorting area, a new path of the target object is determined according to the target position of the target object, where the target position refers to the placement area of ​​the target object after being sorted;

[0032] According to the new path and the actual path, it is determined whether the target item is abnormally sorted.

[0033] On the other hand, an embodiment of the present application provides an analysis device for item sorting, the analysis device for item sorting comprising:

[0034] An acquisition unit, configured to acquire a plurality of gridded target images, wherein the target images include images of target objects, and the target images include a plurality of grids, each of which corresponds to a sorting area;

[0035] A determination unit is used to determine an actual path of the target object according to the target images acquired by the acquisition unit, wherein the actual path covers multiple sorting areas; and determine whether the target object is abnormally sorted according to the actual path and a preset path of the target object.

[0036] In some embodiments of the present application, the acquisition unit is specifically used to:

[0037] Get the item size of the target item;

[0038] Determine the number of target grids according to the size of the object, where the number of target grids is used to indicate the number of grids of each of the target images;

[0039] A plurality of gridded target images are acquired according to the number of target grids.

[0040] In some embodiments of the present application, the acquisition unit is specifically used to:

[0041] Acquire a plurality of gridded images according to the number of target grids, and extract a first image feature of each of the gridded images, wherein the number of target grids is inversely related to the number of gridded images, the gridded image includes a plurality of grids, each of the grids corresponds to a sorting area, and the first image feature is used to identify whether the gridded image contains a target object;

[0042] Determining a similarity between the first image feature and a second image feature, wherein the second image feature is an image feature of a reference image;

[0043] According to the similarity, each of the target images is acquired from each of the gridded images, wherein the similarity between the first image feature and the second image feature of the target image is greater than a preset threshold.

[0044] In some embodiments of the present application, the determining unit is specifically configured to:

[0045] Identify each of the target images to obtain target grids, where the target grids refer to the grids in which the target objects exist among the grids in the target images;

[0046] Determining a plurality of target sorting areas through which the target object passes according to the target grids and the identifiers of the target grids, wherein the identifiers of the target grids are used to indicate the sorting areas corresponding to the target grids;

[0047] The actual path of the target object is determined according to the respective target sorting areas.

[0048] In some embodiments of the present application, the determining unit is specifically configured to:

[0049] According to the shooting time of each of the target images, obtaining the target time for the target object to pass through each of the target sorting areas;

[0050] The actual path of the target object is determined according to the target times and the target sorting areas.

[0051] In some embodiments of the present application, the determining unit is specifically configured to:

[0052] Acquire a preset sorting area and an actual sorting area, wherein the preset sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path, and the actual sorting area refers to a plurality of sorting areas corresponding to and covered by the actual path;

[0053] Obtaining a matching degree between the preset sorting area and the actual sorting area;

[0054] Determine whether the target object is abnormally sorted according to the matching degree.

[0055] In some embodiments of the present application, the determining unit is specifically configured to:

[0056] Acquire a reference sorting area, where the reference sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path;

[0057] When a transmission failure is detected in the reference sorting area, a new path of the target object is determined according to the target position of the target object, where the target position refers to the placement area of ​​the target object after being sorted;

[0058] According to the new path and the actual path, it is determined whether the target item is abnormally sorted.

[0059] On the other hand, an embodiment of the present application further provides an analysis device for item sorting, the analysis device for item sorting comprising a processor and a memory, the memory storing a computer program, and when the processor calls the computer program in the memory, executing the steps of any one of the analysis methods for item sorting provided in the embodiments of the present application.

[0060] On the other hand, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the steps in the object sorting analysis method.

[0061] The embodiment of the present application obtains multiple target images containing target objects, wherein the target images are gridded according to the distribution of sorting areas; then each gridded target image is identified to determine the actual path of the target object, thereby locating the position of the target object; and determines whether the target object is abnormally sorted based on the preset path and the actual path of the target object, thereby providing a reference basis for further sorting control and effectively helping various fields to transport or sort objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 It is a schematic diagram of an application scenario of the analysis method for sorting items provided in an embodiment of the present application;

[0064] Figure 2 is a scene schematic diagram of a gridded image provided in an embodiment of the present application;

[0065] Figure 3 It is a schematic diagram of an embodiment of the process flow of the analysis method for sorting items provided in the embodiments of the present application;

[0066] Figure 4 is a schematic flow chart of an embodiment of step 20 provided in the embodiments of the present application;

[0067] Figure 5 Schematic diagram of the correspondence between each grid of the target image and each sorting area in the embodiment of the present application;

[0068] Figure 6 is a schematic diagram of the actual path of the target object provided in step 203 of the embodiment of the present application;

[0069] Figure 7 is a schematic diagram of the actual path comparison of the target object in the embodiment of the present application;

[0070] Figure 8 It is a schematic diagram of the structure of an embodiment of an analysis device for sorting items provided in an embodiment of the present application;

[0071] Fig. 9 It is a schematic diagram of the structure of an embodiment of the analysis equipment for sorting items provided in the embodiments of the present application. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0073] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0074] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0075] First of all, before introducing the embodiments of the present application, the relevant content about the application background of the embodiments of the present application is introduced.

[0076] In the field of logistics, it is often necessary to sort items (such as parcels); in order to sort the items correctly, the items need to be positioned.

[0077] The traditional photoelectric sensor positioning method can locate the position of the items to be sorted, but the traditional sensor positioning method has many disadvantages. For example, if the photoelectric sensor is attached with dust, pollutants, etc., it will affect the photoelectric sensor's accurate positioning of the item. For another example, the detection range of a photoelectric sensor is limited, and the detected object must be within the detection area of ​​the photoelectric sensor to be detected.

[0078] Therefore, if you want to locate items in a large sorting yard, you need to install a large number of photoelectric sensors. In addition to the high cost, installing a large number of photoelectric sensors also requires a lot of manpower for maintenance.

[0079] Based on the above-mentioned defects existing in the existing related technologies, the embodiments of the present application provide an analysis method for item sorting, which at least overcomes the defects existing in the existing related technologies to a certain extent.

[0080] The present application embodiment provides an analysis method for sorting items, referring to Figure 1 , Figure 1 Schematic diagram of an application scenario of the object sorting analysis method provided in the embodiment of the present application. In the embodiment of the present application, the object sorting analysis method can be applied to Figure 1 (a) shows the hardware deployment scenario of the sorting field. Figure 1 The scene shown in (a) shows that a large-scale sorting yard includes multiple sorting areas (where each sorting area has a conveyor so that items can be conveyed between two adjacent sorting areas). The field of view of a camera can cover all sorting areas of the entire sorting yard; that is, the information of the entire sorting yard can be obtained by acquiring an image taken by a camera. When the camera takes an image, it will form a grid image based on the multiple sorting areas covered by the camera's field of view, such as Figure 2 As shown; through Figure 1 The camera shown in (a) can form the gridded target image mentioned in the embodiments of the present application.

[0081] The analytical method for item sorting can also be applied to Figure 1 (b) shows the hardware deployment scenario of the sorting field. Figure 1 The scene shown in (b) shows that a large-scale sorting yard includes multiple sorting areas (where each sorting area has a conveying device so that items can be conveyed between two adjacent sorting areas). The sorting yard is deployed with multiple cameras, and the field of view of each camera can cover multiple sorting areas. The field of view of multiple cameras can cover each sorting area of ​​the entire sorting yard; that is, by acquiring images taken by multiple cameras, the information of the entire sorting yard can be obtained. When the camera takes an image, it will form a grid image based on the multiple sorting areas covered by the camera's field of view, such as Figure 2 As shown; through Figure 1 The cameras shown in (b) can form the gridded target image mentioned in the embodiment of the present application.

[0082] The executor of the analysis method for item sorting in the embodiment of the present application may be the analysis device for item sorting provided in the embodiment of the present application, or different types of analysis devices for item sorting such as a server device, a physical host or a user equipment (UE) that integrates the analysis device for item sorting, wherein the analysis device for item sorting may be implemented in hardware or software, and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a laptop computer, a PDA, a desktop computer or a personal digital assistant (PDA).

[0083] The item sorting analysis device can adopt a single-operation working mode or a device cluster working mode. By applying the item sorting analysis method provided in the embodiment of the present application, the position of the item can be accurately located, and it can be determined whether the item is sorted abnormally, providing a reference basis for the item sorting control process.

[0084] Next, an analysis method for item sorting provided in an embodiment of the present application is introduced. The analysis method for item sorting includes: acquiring a plurality of gridded target images; determining an actual path of the target item based on the respective target images; and determining whether the target item is abnormally sorted based on the actual path and a preset path of the target item.

[0085] Reference Figure 3 , Figure 3A flow chart of an analysis method for sorting items provided in an embodiment of the present application. It should be noted that although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here. The analysis method for sorting items includes:

[0086] 10. Obtain multiple gridded target images.

[0087] In practical applications, the item sorting analysis device provided by the embodiment of the present application may directly include one or more cameras of the sorting field in hardware, and store the images captured by the one or more cameras locally, which can be directly read inside the device; alternatively, the item sorting analysis device may also establish a network connection with each camera of the sorting field, and obtain the images captured by the cameras online from the cameras according to the network connection; alternatively, the processing device may also read the images captured by the cameras from relevant storage media storing the images captured by the cameras of the sorting field, and the specific acquisition method is not limited here.

[0088] The camera can capture images according to a preset shooting method, such as setting the shooting height, shooting direction or shooting distance. The specific shooting method can be adjusted according to the camera itself, and is not limited here. The image captured by the camera is composed of multiple frames of pictures, and can be composed into a video through a timeline.

[0089] In the embodiment of the present application, there may be multiple items to be sorted in the entire sorting field. For ease of understanding, the present application embodiment takes a sorting area as an example in which at most one item is placed. The target image includes an image of the target item, and the target image includes multiple grids, each grid corresponding to a sorting area, such as Figure 2 The target item refers to the item to be sorted that needs to be located.

[0090] In order to reduce the cost of hardware equipment, the hardware deployment scenario of the embodiment of the present application can be as follows: Figure 1 As shown in (a), when acquiring the target image, the following Figure 1 (a) The gridded image formed by the camera shooting shown in the scene is used as the gridded target image. Figure 1The number of sorting areas in (a) is only for illustration and is not used to limit the embodiments of the present application. The number of sorting areas can be determined according to specific circumstances. For example, there are 10 sorting areas in a sorting field. When configuring the hardware conditions, only one camera is configured for a sorting field. By adjusting the hardware configuration (such as adjusting the height and focal length of the camera), all 10 sorting areas fall within the field of view of the camera, so that 10 sorting areas can be photographed by one camera to form a gridded target image. Among them, the gridded image refers to the image formed by each camera being divided into multiple grid areas according to the multiple sorting areas covered by the field of view of each camera while the camera is imaging normally, so that each grid area of ​​the image corresponds to a sorting area, so as to facilitate the subsequent direct identification or positioning of the object.

[0091] It is understandable that, considering the convenience and simplicity of description, the description of the relevant actions in the following content will hide the analysis equipment for item sorting as the execution subject and will not be directly described.

[0092] When a camera is used to capture a grid image containing multiple sorting areas, in order to make each sorting area fall within the camera's field of view, the distance between the camera and the sorting area needs to be relatively far (or the focal length is far); if the objects in the sorting area are small, the image of the objects in the grid image will be small and unclear, which is not convenient for subsequent identification. To solve this problem, you can obtain Figure 1 (b) A gridded image formed by the camera shown in the scene is used as a gridded target image; that is, the number of sorting areas covered by the camera's field of view is reduced to make the image of the target object clearer in the image for subsequent identification. To this end, in some embodiments of the present application, the step of obtaining multiple gridded target images specifically includes:

[0093] (1) Get the object size of the target object.

[0094] As an implementation method, the size of the target object is determined based on the manually input information such as the size of the target object. As an implementation method, the size of the target object is determined based on the depth image of the target object.

[0095] (2) Determine the target number of grids according to the size of the object.

[0096] Among them, the target grid number is used to indicate the number of grids for each target image. On the one hand, the more grids a gridded image has (that is, the more sorting areas a gridded image corresponds to), the larger the field of view that the camera that captures the image needs to cover, and the smaller the image of the object in the gridded image. On the other hand, if the image of the object in the image is too small (or the object is not clear), it will affect the subsequent identification of the object from the image. Therefore, in an embodiment of the present application, the size of the object is positively correlated with the number of target grids. Specifically, a preset mapping table can be preset in which a one-to-one mapping relationship exists between the size of the object and the number of grids; when determining the target number of grids, the number of grids that has a mapping relationship with the size of the object is directly obtained in the preset mapping table as the target number of grids.

[0097] (3) Acquire multiple gridded target images according to the number of target grids.

[0098] Specifically, for ease of understanding, see Figure 1 (b) First, determine which sorting areas of the sorting field should be covered by the field of view of each camera according to the number of target grids; then, obtain images containing images of target items as gridded target images from one or more gridded images formed by each camera.

[0099] For example, in actual logistics sorting at a sorting yard, a grid image with 3x3=9 grids is formed when large packages are sorted; and a grid image with 2x2=4 grids is formed when small packages are sorted.

[0100] In an embodiment of the present application, the number of grids in each target image is obtained and determined according to the size of the target object, and the object size is positively correlated with the number of target grids. When the target object is small, the number of grids in the acquired target image is small, so that the image of the target object in the gridded image formed by the camera is increased as much as possible, thereby avoiding the problem of the target object being difficult to identify due to the small image of the target object.

[0101] Since there are multiple sorting areas in a large sorting field, there may be multiple objects to be sorted at the same time. It is necessary to determine and locate the position of the target object accurately based on the characteristics of the target object. Therefore, in some embodiments of the present application, multiple gridded target images are obtained according to the number of target grids, including:

[0102] (1) Acquire multiple gridded images according to the target grid quantity, and extract the first image feature of each gridded image respectively.

[0103] Among them, the number of target grids is inversely related to the number of gridded images, the gridded image includes multiple grids, each grid corresponds to a sorting area, and the first image feature is used to identify whether the gridded image contains the target item. In order to facilitate understanding that the number of grids is inversely related to the number of gridded images, a specific example is used for explanation. For example, in a large sorting field, including multiple sorting areas (such as 20), the field of view of a camera (such as the field of view of a camera can cover 4 sorting areas) cannot cover all sorting areas, and multiple cameras (20 / 5=4) are required to cover all sorting areas. Since a camera shoots and generates a gridded image, a sorting area corresponds to a grid; therefore, the more sorting areas are covered by the field of view of a camera (that is, the larger the number of grids in a gridded image), the fewer cameras are required for the entire sorting field (that is, the fewer the number of gridded images in the entire sorting field).

[0104] Specifically, for ease of understanding, please continue to refer to Figure 1 (b), since the location of the target object is uncertain, it is necessary to simultaneously capture multiple grid images through multiple cameras, and then filter out the image containing the target object from the grid images captured by each camera as the target image. For example, the sorting field includes 5 cameras (each camera covers 4 sorting areas, and the 5 cameras cover a total of 20 sorting areas in the entire sorting field). At t1, t2, and t3, 5 grid images are obtained through the 5 cameras, and a total of 15 grid images are obtained.

[0105] After acquiring multiple gridded images, a deep learning model can be used to filter out a target image containing a target object from the multiple gridded images. Specifically, first, a large number of gridded images are collected to train the deep learning model, and finally, a deep learning model is formed that can distinguish whether any gridded image contains a target object. Each gridded image is input into the deep learning model so that the deep learning model extracts the first image feature of the gridded image.

[0106] (2) Determine the similarity between the first image feature and the second image feature.

[0107] Based on the deep learning model, the similarity between the first image feature and the second image feature can be determined, wherein the second image feature is an image feature of a reference image, the reference image refers to a gridded image used to train the deep learning model, and the second image feature is used to indicate a feature of a gridded image containing a target object.

[0108] (3) Acquire the target images from the gridded images based on the similarity.

[0109] Based on the deep learning model, it can be determined whether the similarity between the first image feature and the second image feature of each gridded image is greater than a preset threshold; if the similarity between the first image feature and the second image feature of the gridded image is greater than the preset threshold, the gridded image is obtained as the target image. The similarity between the first image feature and the second image feature of the target image is greater than the preset threshold.

[0110] In the embodiment of the present application, the first image feature of the gridded image is extracted to identify whether the gridded image is an image containing the target object, so that the sorting area of ​​the target object can be directly located based on the target image containing the target object, without the need to analyze the gridded image that does not contain the target object, thereby reducing the amount of image processing and improving processing efficiency.

[0111] 20. Determine the actual path of the target object according to the respective target images.

[0112] The actual path of the target object covers multiple sorting areas. Specifically, each grid of the target image is identified and all grids containing the target object are found. Since each grid corresponds to a sorting area, by determining the grids where the target object has appeared, it is possible to determine which sorting areas the target object has passed through; and then, based on the multiple sorting areas that the target object has passed through, the actual path of the target object can be determined.

[0113] To accurately determine the actual path of the target object, see Figure 4 In some embodiments of the present application, determining the actual path of the target object according to each target image includes:

[0114] 201. Identify each of the target images to obtain target grids.

[0115] The target grid refers to the grid in which the target object exists among the grids of the target image. Specifically, through the image recognition algorithm, each grid of the target image is traversed until the grid containing the target object is found, and the grid containing the target object is used as the target grid; similarly, each target image is traversed to determine each target grid, and the information of each target grid (such as the identification of the target grid, the identification of the sorting area corresponding to the target grid, etc.) is obtained.

[0116] 202. Determine a plurality of target sorting areas through which the target object passes according to the target grids and identifiers of the target grids.

[0117] The target grid identifier is used to indicate the sorting area corresponding to the target grid. For ease of understanding, a specific embodiment is used for explanation. Figure 5For example, the grids of the target image are labeled 1, 2, 3, and 4 respectively. Figure 5 (a) As shown; the sorting areas in the actual sorting field are A, B, C, and D respectively. Figure 5 (b) is shown; wherein, grid 1 corresponds to sorting area A in the actual sorting field, grid 2 corresponds to sorting area B in the actual sorting field, grid 3 corresponds to sorting area C in the actual sorting field, and grid 4 corresponds to sorting area D in the actual sorting field. It can be seen that, when the target grid and the identifier of the target grid are determined, multiple target sorting areas that the target item passes through can be determined.

[0118] 203. Determine an actual path of the target object according to each target sorting area.

[0119] For ease of understanding, a specific example is used for illustration. Figure 6 , Figure 6 Each grid in the figure represents a sorting area. When the target sorting areas are determined to be sorting area 1, sorting area 6, sorting area 11, sorting area 12, sorting area 13, sorting area 18, and sorting area 19, it can be determined that the target items are sorted in the following manner: Figure 6 The actual path shown.

[0120] Even if the target item passes through the same multiple sorting areas, if the target item passes through each sorting area at different times, the actual path of the target item may be different. For example, the target item may pass through sorting areas 1, 2, 3, 4, 5, 6, or 1, 2, 3, 4, 3, 2, 3, 4, 5, 6. Although the sorting areas passed by the target item are exactly the same, the actual path of the target item is different. Figure 7 In order to improve the accuracy of the actual path of the target object, in some embodiments of the present application, the step of determining the actual path of the target object according to each target sorting area specifically includes:

[0121] (1) According to the shooting time of each of the target images, the target time for the target object to pass through each of the target sorting areas is obtained.

[0122] The target time refers to the time when the target item passes through the target sorting area during the actual sorting process. Specifically, the shooting time of the target image corresponding to each target grid is obtained, and the shooting time of the target image corresponding to the target grid is used as the target time when the target item passes through the target sorting area corresponding to the target grid.

[0123] For example, if the target grid 1 determined in step 201 is a grid in the target image A, and the target sorting area corresponding to the target grid 1 is sorting area a, then the shooting time (e.g., 12:02) when the camera shoots the target image A is the target time (12:02) when the target item passes through sorting area 1. If the target grid 2 determined in step 201 is a grid in the target image B, and the target sorting area corresponding to the target grid 2 is sorting area b, then the shooting time (e.g., 13:00) when the camera shoots the target image B is the target time (13:00) when the target item passes through sorting area 2.

[0124] (2) Determine the actual path of the target item according to the target time and the target sorting area.

[0125] For ease of understanding, a specific example is used for illustration. For example, if the target time for the target item to pass through the target sorting areas 1, 2, 3, 4, 5, and 6 is 12:00, 12:01, 12:02, 12:03, 12:04, and 12:05, respectively, then the actual path that the target item passes through during the sorting process is as follows: Figure 7 (a) As shown. If the target item passes through the target sorting areas 1, 2, 3, 4, 3, 2, 3, 4, 5, 6 at the target time 12:00, 12:01, 12:02, 12:03, 12:04, 12:05, 12:06, 12:07, 12:08, 12:09, respectively, then the actual path of the target item during the sorting process is as follows: Figure 7 (b) as shown.

[0126] In the embodiment of the present application, the target time for the target object to pass through each target sorting area is determined by acquiring the shooting time of the target image, so that the time sequence in which the target object passes through each target sorting area is determined, so that the actual path of the target object determined thereby is more in line with reality, thereby improving the accuracy of the actual path.

[0127] 30. Determine whether the target object is abnormally sorted according to the actual path and the preset path of the target object.

[0128] The preset path refers to the sorting path pre-planned for the target item before sorting the target item, based on the location where the target item should be placed after sorting; the preset path of the target item can be one or more. Specifically, when it is detected that the actual path matches any of the preset paths, it is determined that the target item has not been sorted abnormally; when it is detected that the actual path does not match any of the preset paths, it is determined that the target item has been sorted abnormally.

[0129] In an embodiment of the present application, multiple target images containing target objects are acquired, wherein the target images are gridded according to the distribution of the sorting areas; each gridded target image is then identified to determine the actual path that the target object has passed, thereby locating the position of the target object; and based on the preset path and the actual path of the target object, it is determined whether the target object has been abnormally sorted, thereby providing a reference for further sorting control and effectively helping various fields to transport or sort objects.

[0130] In order to accurately detect whether the target object is abnormally sorted, in some embodiments of the present application, the step of determining whether the target object is abnormally sorted according to the actual path and the preset path of the target object specifically includes:

[0131] (1) Obtain the preset sorting area and the actual sorting area.

[0132] For the convenience of description, the following will take the case where there is only one preset path as an example to illustrate whether the target item is abnormally sorted. Among them, the preset sorting area refers to the multiple sorting areas covered by the preset path, and the actual sorting area refers to the multiple sorting areas covered by the actual path. For example, if the actual path corresponds to covering multiple sorting areas, namely sorting area 1, sorting area 2, sorting area 5, and sorting area 6, then the preset sorting area includes sorting area 1, sorting area 2, sorting area 5, and sorting area 6. If the actual path corresponds to covering multiple sorting areas, namely sorting area 3, sorting area 4, sorting area 7, and sorting area 8, then the actual sorting area includes sorting area 3, sorting area 4, sorting area 7, and sorting area 8.

[0133] (2) Obtaining a matching degree between the preset sorting area and the actual sorting area.

[0134] Specifically, as an implementation method, the ratio of the number of sorting areas in the preset sorting area that correspond to the same sorting area in the actual sorting area to the number of sorting areas covered by the preset path is used as the matching degree between the preset sorting area and the actual sorting area. For example, the preset sorting area includes sorting area 1, sorting area 2, sorting area 5, and sorting area 6, and the actual sorting area includes sorting area 2, sorting area 3, sorting area 5, and sorting area 6. The number of sorting areas in the preset sorting area that correspond to the same sorting area in the preset sorting area (sorting area 2, sorting area 5, and sorting area 6) is 3, and the matching degree between the preset sorting area and the actual sorting area is 3 / 4.

[0135] As another implementation, the ratio of the number of sorting areas in the preset sorting area that correspond to the same sorting area in the actual sorting area to the number of sorting areas covered by the actual path is used as the matching degree between the preset sorting area and the actual sorting area. The preset sorting area includes sorting area 1, sorting area 2, and sorting area 3, and the actual sorting area includes sorting area 1, sorting area 2, sorting area 3, sorting area 5, and sorting area 6. The number of sorting areas (sorting area 1, sorting area 2, and sorting area 3) in the preset sorting area that correspond to the same sorting area in the actual sorting area is 3, and the matching degree between the preset sorting area and the actual sorting area is 3 / 5.

[0136] (3) Determine whether the target item is abnormally sorted according to the matching degree.

[0137] Since the higher the matching degree between the actual sorting area of ​​the target object and the preset sorting area, the greater the possibility that the actual path through which the target object is sorted is the pre-planned preset path. Therefore, in the embodiment of the present application, when the matching degree between the preset sorting area and the actual sorting area is greater than the preset value, it is determined that the target object has not been abnormally sorted; when the matching degree between the preset sorting area and the actual sorting area is less than or equal to the preset value, it is determined that the target object has been abnormally sorted.

[0138] In the embodiment of the present application, by obtaining the matching degree between the preset sorting area and the actual sorting area, it is accurately determined whether the target item is abnormally sorted, which provides a reference basis for further sorting control and effectively helps various fields to transport or sort items.

[0139] Since in the actual sorting process, the originally planned preset path may be changed to a new planned path for some reason, if the actual path of the target item matches the new planned path, it should be considered that the target item has not been abnormally sorted. In order to improve the accuracy of detecting whether an item has been abnormally sorted, in some embodiments of the present application, determining whether the target item has been abnormally sorted based on the actual path and the preset path of the target item includes:

[0140] (1) Obtain the reference sorting area.

[0141] The reference sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path.

[0142] (2) When a transport failure is detected in the reference sorting area, a new path of the target object is determined according to the target position of the target object.

[0143] The target location refers to the placement area after the target item is sorted. When it is detected that there is a transmission failure in any one or more sorting areas in the reference area (such as other items in the sorting area, or the sorting area cannot normally transmit items due to a failure, etc.), the available status of each sorting area in the actual sorting field (such as whether there are other items, whether there is a hardware failure, etc.) and the target location of the target item are obtained and the walking path of the sorting process of the target item is replanned to obtain a new path for the target item.

[0144] (3) Determine whether the target item is abnormally sorted based on the new path and the actual path.

[0145] Detect whether the new path matches the actual path. When it is detected that the new path matches the actual path, it is determined that the target item is not sorted abnormally; when it is detected that the new path does not match the actual path, it is determined that the target item is sorted abnormally.

[0146] Specifically, the specific implementation method of determining whether the target item is abnormally sorted according to the new path and the actual path can refer to the aforementioned step embodiment of determining whether the target item is abnormally sorted according to the actual path and the preset path, and will not be repeated here.

[0147] In an embodiment of the present application, by obtaining a new path for the re-planned target object, determining whether the target object has been abnormally sorted based on the new path and the actual path, thereby avoiding misjudging the target object as being abnormally sorted due to sorting the target object through the new path, and improving the accuracy of detecting whether the target object has been abnormally sorted.

[0148] In order to better implement the object sorting analysis method in the embodiment of the present application, based on the object sorting analysis method, the embodiment of the present application also provides an object sorting analysis device, such as Figure 8 FIG. 8 is a schematic diagram of a structure of an embodiment of an analysis device for item sorting in an embodiment of the present application. The analysis device 800 for item sorting includes:

[0149] An acquisition unit 801 is used to acquire a plurality of gridded target images, wherein the target images include images of target objects, and the target images include a plurality of grids, each of which corresponds to a sorting area;

[0150] The determination unit 802 is used to determine the actual path of the target object according to the target images acquired by the acquisition unit 801, where the actual path covers multiple sorting areas; and determine whether the target object is abnormally sorted according to the actual path and the preset path of the target object.

[0151] In some embodiments of the present application, the acquisition unit 801 is specifically used to:

[0152] Get the item size of the target item;

[0153] Determine the number of target grids according to the size of the object, where the number of target grids is used to indicate the number of grids of each of the target images;

[0154] A plurality of gridded target images are acquired according to the number of target grids.

[0155] In some embodiments of the present application, the acquisition unit 801 is specifically used to:

[0156] Acquire a plurality of gridded images according to the number of target grids, and extract a first image feature of each of the gridded images, wherein the number of target grids is inversely related to the number of gridded images, the gridded image includes a plurality of grids, each of the grids corresponds to a sorting area, and the first image feature is used to identify whether the gridded image contains a target object;

[0157] Determining a similarity between the first image feature and a second image feature, wherein the second image feature is an image feature of a reference image;

[0158] According to the similarity, each of the target images is acquired from each of the gridded images, wherein the similarity between the first image feature and the second image feature of the target image is greater than a preset threshold.

[0159] In some embodiments of the present application, the determining unit 802 is specifically configured to:

[0160] Identify each of the target images to obtain respective target grids, wherein the target grids refer to the grids in which the target objects exist among the grids of the target images;

[0161] Determining a plurality of target sorting areas through which the target object passes according to the target grids and the identifiers of the target grids, wherein the identifiers of the target grids are used to indicate the sorting areas corresponding to the target grids;

[0162] The actual path of the target object is determined according to the respective target sorting areas.

[0163] In some embodiments of the present application, the determining unit 802 is specifically configured to:

[0164] According to the shooting time of each of the target images, obtaining the target time for the target object to pass through each of the target sorting areas;

[0165] The actual path of the target object is determined according to the target times and the target sorting areas.

[0166] In some embodiments of the present application, the determining unit 802 is specifically configured to:

[0167] Acquire a preset sorting area and an actual sorting area, wherein the preset sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path, and the actual sorting area refers to a plurality of sorting areas corresponding to and covered by the actual path;

[0168] Obtaining a matching degree between the preset sorting area and the actual sorting area;

[0169] Determine whether the target object is abnormally sorted according to the matching degree.

[0170] In some embodiments of the present application, the determining unit 802 is specifically configured to:

[0171] Acquire a reference sorting area, where the reference sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path;

[0172] When a transmission failure is detected in the reference sorting area, a new path of the target object is determined according to the target position of the target object, where the target position refers to the placement area of ​​the target object after being sorted;

[0173] According to the new path and the actual path, it is determined whether the target item is abnormally sorted.

[0174] In addition, in order to better implement the object sorting analysis method in the embodiment of the present application, based on the object sorting analysis method, the embodiment of the present application also provides an object sorting analysis device, see Fig. 9 , Fig. 9 A schematic diagram of the structure of an analysis device for sorting items in an embodiment of the present application is shown. Specifically, the analysis device for sorting items provided in an embodiment of the present application includes a processor 901, and the processor 901 is used to execute a computer program stored in a memory 902 to implement the following Figures 1 to 7 Corresponding to each step of the method for analyzing item sorting in any embodiment; or, the processor 901 is used to execute the computer program stored in the memory 902 to implement the following Figure 8 The functions of each unit in the corresponding embodiment.

[0175] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 902, and executed by the processor 901 to complete the embodiment of the present application. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0176] The analysis device for item sorting may include, but is not limited to, a processor 901 and a memory 902. Those skilled in the art may understand that the illustration is only an example of the analysis device for item sorting, and does not constitute a limitation on the analysis device for item sorting, and may include more or fewer components than shown in the illustration, or a combination of certain components, or different components, for example, the analysis device for item sorting may also include an input / output device, a network access device, a bus, etc., and the processor 901, the memory 902, the input / output device, and the network access device are connected via a bus.

[0177] The processor 901 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the item sorting analysis device, and uses various interfaces and lines to connect various parts of the entire item sorting analysis device.

[0178] The memory 902 can be used to store computer programs and / or modules. The processor 901 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 902 and calling the data stored in the memory 902. The memory 902 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created by using an analysis device for item sorting (such as audio data, video data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0179] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described object sorting analysis device, equipment and corresponding units can refer to the following. Figures 1 to 7The description of the analysis method for sorting items in any embodiment will not be repeated here.

[0180] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0181] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the present application as follows: Figures 1 to 7 For the steps in the analysis method for sorting items in any embodiment, the specific operations can be referred to as follows: Figures 1 to 7 The description of the analysis method for sorting items in any embodiment will not be repeated here.

[0182] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0183] Due to the instructions stored in the computer-readable storage medium, the present application can be executed. Figures 1 to 7 Corresponding to the steps in the analysis method for sorting items in any embodiment, the present application can be implemented as follows Figures 1 to 7 The beneficial effects that can be achieved by the analysis method for sorting items in any embodiment are detailed in the previous description and will not be repeated here.

[0184] The above is a detailed introduction to the analysis method, device, equipment and storage medium for item sorting provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An analytical method for sorting items, characterized in that: The method comprises: Acquire multiple gridded target images, including: The sorting yard includes a plurality of sorting areas. A plurality of cameras are deployed in the sorting yard. A grid image is formed according to the plurality of sorting areas covered by the field of view of the camera. The grid image includes a plurality of grids, each of which corresponds to a sorting area. Get the item size of the target item; Determine the number of target grids according to the size of the object, where the number of target grids is used to indicate the number of grids of each target image; Acquire a plurality of gridded images according to the number of target grids, and extract a first image feature of each of the gridded images respectively, wherein the number of target grids is inversely related to the number of gridded images, and the first image feature is used to identify whether the gridded image contains a target object; Determining a similarity between the first image feature and a second image feature, wherein the second image feature is an image feature of a reference image; According to the similarity, acquiring each target image from each gridded image, wherein the similarity between the first image feature and the second image feature of the target image is greater than a preset threshold, the target image includes an image of a target object, and the target image includes a plurality of grids, each of which corresponds to a sorting area; Determine the sorting area through which the target object passes according to the grids in which the target object appears in each target image, and determine the actual path of the target object according to the sorting area through which the target object passes, wherein the actual path covers multiple sorting areas; According to the actual path and the preset path of the target object, it is determined whether the target object is abnormally sorted.

2. The method for analyzing object sorting according to claim 1, characterized in that: The step of determining the sorting area through which the target object passes according to the grids in which the target object appears in each target image, and determining the actual path of the target object according to the sorting area through which the target object passes includes: Identify each of the target images to obtain respective target grids, wherein the target grids refer to the grids in which the target objects exist among the grids of the target images; Determining a plurality of target sorting areas through which the target object passes according to the target grids and the identifiers of the target grids, wherein the identifiers of the target grids are used to indicate the sorting areas corresponding to the target grids; The actual path of the target object is determined according to the respective target sorting areas.

3. The method for analyzing object sorting according to claim 2, characterized in that: The determining the actual path of the target object according to each target sorting area includes: According to the shooting time of each of the target images, obtaining the target time for the target object to pass through each of the target sorting areas; The actual path of the target object is determined according to each of the target times and each of the target sorting areas.

4. The method for analyzing object sorting according to any one of claims 1 to 3, characterized in that: The determining whether the target object is abnormally sorted according to the actual path and the preset path of the target object includes: Acquire a preset sorting area and an actual sorting area, wherein the preset sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path, and the actual sorting area refers to a plurality of sorting areas corresponding to and covered by the actual path; Obtaining a matching degree between the preset sorting area and the actual sorting area; Determine whether the target object is abnormally sorted according to the matching degree.

5. The method for analyzing object sorting according to any one of claims 1 to 3, characterized in that: The determining whether the target object is abnormally sorted according to the actual path and the preset path of the target object includes: Acquire a reference sorting area, where the reference sorting area refers to a plurality of sorting areas corresponding to and covered by the preset path; When a transmission failure is detected in the reference sorting area, a new path of the target object is determined according to the target position of the target object, where the target position refers to the placement area of ​​the target object after being sorted; According to the new path and the actual path, it is determined whether the target item is abnormally sorted.

6. An analysis device for sorting items, characterized in that: The article sorting analysis device comprises: The acquisition unit is used to acquire a plurality of gridded target images, including: The sorting yard includes a plurality of sorting areas. A plurality of cameras are deployed in the sorting yard. A grid image is formed according to the plurality of sorting areas covered by the field of view of the camera. The grid image includes a plurality of grids, each of which corresponds to a sorting area. Get the item size of the target item; Determine the number of target grids according to the size of the object, where the number of target grids is used to indicate the number of grids of each target image; Acquire a plurality of gridded images according to the number of target grids, and extract a first image feature of each of the gridded images respectively, wherein the number of target grids is inversely related to the number of gridded images, and the first image feature is used to identify whether the gridded image contains a target object; Determining a similarity between the first image feature and a second image feature, wherein the second image feature is an image feature of a reference image; According to the similarity, acquiring each target image from each gridded image, wherein the similarity between the first image feature and the second image feature of the target image is greater than a preset threshold, the target image includes an image of a target object, and the target image includes a plurality of grids, each of which corresponds to a sorting area; a determination unit, configured to determine the sorting area through which the target object passes according to the grids in which the target object appears in each target image acquired by the acquisition unit, determine an actual path of the target object according to the sorting area through which the target object passes, wherein the actual path covers multiple sorting areas; and determine whether the target object is abnormally sorted according to the actual path and a preset path of the target object.

7. An analytical device for sorting items, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method for analyzing item sorting according to any one of claims 1 to 5 is executed.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the object sorting analysis method according to any one of claims 1 to 5.