A visual inspection simulation method, device and system for logistics sorting

Abstract

The present invention relates to the field of mechanical vision, and in particular to a visual detection simulation method, device and system for logistics sorting, wherein the method first establishes three-dimensional point cloud models for a sorting system and a package to be sorted, respectively, and then simulates sorting by mobilizing the point cloud model to determine a sorting method for collision-free sorting, and sends sorting information including the sorting method to a sorting device, so that the sorting device sorts the package according to the sorting information; in the present application, targeted simulation can be achieved for packages of any shape and size, and then collision-free sorting can be achieved, thereby ensuring the safety of each package during sorting.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and in particular to a visual detection simulation method, device and system for logistics sorting. Background Art

[0002] In current logistics sorting centers, in order to achieve efficient sorting of packages, an automated sorting system is often used to sort packages;

[0003] The sorting system often includes a conveying device, a sorting device and a sorting frame. The conveying device is used to convey the parcels, and the sorting device sorts the parcels on the conveying device into the sorting frame. The prior art often presets sorting parameters for the sorting device so that the parcels are transferred along the same path each time the parcels are sorted. However, since the volumes of different parcels vary greatly, if the parcels are sorted along the same path, small parcels may be sorted smoothly, but large parcels may collide with other structures of the sorting system (such as maintenance structures such as brackets and partitions), making it difficult to ensure the safety of the parcels. Summary of the invention

[0004] Based on this, it is necessary to provide a visual inspection simulation method, device and system for logistics sorting to address the above problems.

[0005] In one embodiment, the present invention provides a visual inspection simulation method for logistics sorting, the method comprising:

[0006] S1: Collecting a structural image of a logistics sorting system, and generating a first point cloud model based on the collected structural image, wherein the sorting system includes a conveying device, a sorting device, and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device, and a sorting frame model corresponding to each sorting frame;

[0007] S2: When each package is placed on the conveyor, determine the target sorting frame to which the package needs to be sorted, and then determine the target sorting frame model;

[0008] S3: collecting an appearance image of the package, and generating a second point cloud model according to the collected appearance image;

[0009] S4: mobilizing the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model;

[0010] S5: determining a grabbing position and a moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path;

[0011] S6: Integrate the determined grabbing position and moving path into sorting information corresponding to the package;

[0012] S7: Sending the sorting information to the sorting device, so that the sorting device sorts the parcel according to the sorting information when the parcel moves to the sorting position.

[0013] In one embodiment, the present invention provides a visual inspection simulation device for logistics sorting, the device comprising:

[0014] A first modeling module is used to obtain a structural image of a logistics sorting system, and generate a first point cloud model according to the obtained structural image, wherein the sorting system includes a conveying device, a sorting device and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device and a sorting frame model corresponding to each sorting frame;

[0015] A first processing module is used to determine a target sorting frame to which each parcel needs to be sorted when the parcel is placed on the conveying device, and then determine a target sorting frame model;

[0016] A second modeling module is used to collect the appearance image of the package and generate a second point cloud model according to the collected appearance image;

[0017] The second processing module is used to mobilize the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model;

[0018] The third processing module is used to determine the grabbing position and moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path;

[0019] A fourth processing module, for integrating the determined grabbing position and moving path into sorting information corresponding to the package;

[0020] A sending module, used for sending the sorting information to the sorting device, so that the sorting device sorts the parcel according to the sorting information when the parcel moves to the sorting position;

[0021] In one embodiment, the present invention provides a visual inspection simulation system for logistics sorting, the system comprising:

[0022] A sorting system, including a conveying device, a sorting device and a plurality of sorting frames, for sorting packages;

[0023] A visual device for collecting images;

[0024] The computer equipment is connected with the sorting system and the visual device, and is used to execute the visual inspection simulation method for logistics sorting.

[0025] The present invention provides a visual inspection simulation method, device and system for logistics sorting, wherein the method comprises collecting a structural image of a logistics sorting system, and generating a first point cloud model based on the collected structural image; when each package is placed on a conveying device, determining a target sorting frame to which the package needs to be sorted, and then determining a target sorting frame model; collecting an appearance image of the package, and generating a second point cloud model based on the collected appearance image; mobilizing a sorting model to simulate sorting of the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined; determining a grabbing position and a moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path; integrating the determined grabbing position and moving path into sorting information corresponding to the package; sending the sorting information to a sorting device, so that the sorting device sorts the package according to the sorting information when the package moves to the sorting position; in the present application, targeted simulation can be achieved for packages of any shape and size, and then collision-free sorting can be achieved, thereby ensuring the safety of each package during sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A first flow chart of a visual inspection simulation method for logistics sorting provided in one embodiment;

[0027] Figure 2 A second flow chart of a visual inspection simulation method for logistics sorting provided in one embodiment;

[0028] Figure 3 A schematic diagram of a point cloud model of a visual inspection simulation method for logistics sorting provided in one embodiment;

[0029] Figure 4 A schematic diagram of a module flow of a visual inspection simulation device for logistics sorting provided in one embodiment;

[0030] Figure 5 A schematic diagram of the composition of a visual inspection simulation system for logistics sorting provided in one embodiment;

[0031] Figure 6 FIG. 4 is a block diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script without departing from the scope of the present invention.

[0034] like Figure 1-Figure 2 As shown, in one embodiment, a visual inspection simulation method for logistics sorting is proposed, and the method includes:

[0035] S1: Collecting a structural image of a logistics sorting system, and generating a first point cloud model based on the collected structural image, wherein the sorting system includes a conveying device, a sorting device, and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device, and a sorting frame model corresponding to each sorting frame;

[0036] S2: When each package is placed on the conveyor, determine the target sorting frame to which the package needs to be sorted, and then determine the target sorting frame model;

[0037] S3: collecting an appearance image of the package, and generating a second point cloud model according to the collected appearance image;

[0038] S4: mobilizing the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model;

[0039] S5: determining a grabbing position and a moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path;

[0040] S6: Integrate the determined grabbing position and moving path into sorting information corresponding to the package;

[0041] S7: Sending the sorting information to the sorting device, so that the sorting device sorts the parcel according to the sorting information when the parcel moves to the sorting position.

[0042] In this embodiment, the method is executed in a computer device, which may be an independent physical server or terminal, or a server cluster composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN; the sorting system is a system used by a logistics center to sort packages, including a conveying device (such as a conveyor belt), a sorting device (such as a robot), and several sorting frames, and the computer device communicates with the sorting system; the visual device includes several cameras, some of which are used to obtain structural images of the sorting system, and some of which are used to obtain appearance images of packages on the conveyor belt.

[0043] In this embodiment, if Figure 3 As shown, images of multiple perspectives of an object (sorting system, package, etc.) are acquired through corresponding cameras. The camera used is a depth-of-field camera, and the acquired images include corresponding depth information. Then, feature points in each image are extracted through an image processing algorithm built into the computer device, such as the accelerated robust feature algorithm (SURF algorithm), and the same feature points in different images are matched. Then, a point cloud is generated based on the matched feature points through built-in 3D processing software, such as MeshLab, and then a corresponding 3D point cloud model is reconstructed, such as the first point cloud model and the second point cloud model mentioned above. A label with classification information related to the package (such as address, product category) is affixed to the package. The camera can recognize the information on the label and then determine the corresponding target sorting frame.

[0044] In this embodiment, the sorting system also includes other structures, such as maintenance structures such as brackets and partitions, and the generated first point cloud model is a model including all structures in the sorting system; collision-free sorting, that is, the second point cloud model only contacts the clamping part of the sorting model that clamps it, and does not conflict with any other part of the first point cloud model during the entire simulated sorting; if the simulated sorting is collision-free sorting, the sorting device performs actual sorting in the sorting mode of the simulated sorting, and can also prevent the package from colliding with the sorting system to ensure the safety of the package; and the determined collision-free sorting is the collision-free sorting with the shortest moving path, which ensures the safety of the package while taking into account the sorting efficiency.

[0045] In the present application, three-dimensional point cloud models are first established for the sorting system and the parcels to be sorted, and then sorting is simulated by mobilizing the point cloud model to determine a sorting method for collision-free sorting. The sorting information including the sorting method is sent to the sorting device, so that the sorting device sorts the parcels according to the sorting information. In the present application, targeted simulation can be achieved for parcels of any shape and size, and then collision-free sorting can be achieved, ensuring the safety of each parcel during sorting.

[0046] As a preferred embodiment, mobilizing the sorting model to perform simulated sorting on the second point cloud model several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined includes:

[0047] S41: determining a plurality of groups of grabbing positions according to the shape of the second point cloud model;

[0048] S42: taking a group of grasping positions, mobilizing the sorting model to grab the second point cloud model from the group of grasping positions, performing simulated sorting of different moving paths for several times, so as to determine the candidate simulated sorting corresponding to the shortest moving path of the group of grasping positions, and repeating this step until the candidate simulated sorting corresponding to each group of grasping positions is determined;

[0049] S43: Compare the moving path lengths of the candidate simulation sortings, and use the candidate simulation sorting with the shortest moving path as the final collision-free sorting.

[0050] In this embodiment, different grabbing positions will result in different hanging postures of the package after being grabbed, which will further result in different spatial ranges passed through during the transfer process; in this embodiment, for each group of grabbing positions, simulated sorting with different moving paths is performed several times to ensure the comprehensiveness of the simulation; furthermore, an area is preset on the conveying device in the sorting system as a sorting position, and a camera is also provided on the sorting device. When the camera recognizes that the package moves to the sorting position, the sorting system identifies the determined grabbing position through the camera, and mobilizes the sorting device to grab the package at the grabbing position for sorting; accordingly, the second point cloud model moves on the conveying model, and when it moves to the corresponding position of the sorting position, the sorting device is mobilized to perform corresponding simulated sorting.

[0051] As a preferred embodiment, determining a plurality of groups of grabbing positions according to the shape of the second point cloud model includes:

[0052] S411: Identify the shape of the second point cloud model, and determine a plurality of groups of clamping points according to the shape and a preset shape-clamping point correspondence table;

[0053] S412: taking a group of clamping points, and identifying a corresponding position point of each clamping point in the group of clamping points on the second point cloud model;

[0054] S413: Determine whether the distance between each two corresponding position points is less than the maximum contact distance of the clamping part of the sorting model. If so, the corresponding position points of the group of clamping points are a group of grasping positions. Otherwise, the corresponding position points of the group of clamping points are not grasping positions.

[0055] S414: Repeat steps S412 to S413 until it is determined whether the corresponding position point of each group of clamping points is a grasping position.

[0056] In this embodiment, different forms of sorting devices have different clamping parts, such as a two-point gripper and a three-point gripper. Taking the two-point gripper as an example, the two-point gripper has two contact points with the grasped object during grasping, and the two points constitute a group of grasping positions. In order to ensure the stability of grasping, corresponding shape-clamping point correspondence tables are pre-determined for different types of clamping parts. The table includes several groups of grasping positions that can stably grasp packages of different shapes. For example, for a rectangular package, the midpoints of the two side surfaces where the long sides are located are a group of grasping positions, and the midpoints of the two side surfaces where the short sides are located are another group of grasping positions; further, if the distance between the two corresponding position points is greater than the maximum contact distance of the clamping part, it indicates that the clamping part cannot touch the two points at the same time, that is, the corresponding grasping position is unavailable and is excluded.

[0057] As a preferred embodiment, a group of grabbing positions is taken, and the sorting model is mobilized to grab the second point cloud model from the group of grabbing positions, and simulated sorting of different moving paths is performed several times to determine the alternative simulated sorting of the shortest moving path corresponding to the group of grabbing positions, including:

[0058] S421: when the second point cloud model is grabbed by the sorting model, all candidate paths for moving the second point cloud model to the target sorting frame model are generated;

[0059] S422: Arrange the candidate paths in order from short to long to obtain a path sequence;

[0060] S423: taking the first selected path from the path sequence, and mobilizing the gripping device to sort the second point cloud model into the target sorting frame model according to the selected path;

[0061] S424: determining whether the second point cloud model collides with the first point cloud model during the movement process, and if not, determining the simulated sorting corresponding to the to-be-selected path as the candidate simulated sorting;

[0062] S425: If yes, remove the candidate path from the path sequence, and execute steps S423 to S425 until the candidate simulation sorting is determined.

[0063] In this embodiment, the moving range of the sorting device is limited by its size and structural form. The candidate paths generated in this embodiment are all moving paths that the corresponding sorting device can achieve. Furthermore, simulated sorting is performed one by one according to the path sequence. Since the candidate sequences in the path sequence are arranged from short to long in length, the determined alternative simulated sorting is the sorting method with the shortest path that can achieve collision-free sorting under the conditions of this group of grasping positions.

[0064] As a preferred embodiment, after sending the sorting information to the sorting device, the method further includes:

[0065] S8: When the sorting device picks up the package according to the sorting position of the sorting information, it is determined whether the picked up package swings when moving;

[0066] S9: If not, the set speed is sent to the sorting device, so that the sorting device sorts the package to the target sorting frame at the set speed according to the moving path of the sorting information;

[0067] S10: If so, determine the moving speed at which the swinging package just does not collide with the sorting system, divide the moving speed by a preset safety factor to obtain an upper limit moving speed, and send the upper limit moving speed to the sorting device, so that the sorting device sorts the package to the target sorting frame at the upper limit moving speed according to the moving path of the sorting information.

[0068] In this embodiment, the set speed may be 1 m / s, and the safety factor may be 1.05; the sorting device often sorts parcels according to the set moving speed. When the sorting device grabs the parcel and moves it, the parcel may swing. When swinging, the parcel may also collide with other parts of the sorting system, and the swing is mainly related to the moving speed of the sorting device. The greater the set moving speed, the greater the acceleration during starting and braking, which leads to the swing of the parcel; when the moving speed is very low, the kinetic energy generated by starting and braking is small, and the parcel often does not swing, but the low moving speed will lead to a decrease in sorting efficiency; in this embodiment, when the parcel swings, the fastest moving speed under the condition that it swings without colliding can be determined, which ensures both the safety of the parcel and the efficiency of sorting.

[0069] As a preferred embodiment, determining whether the picked-up package will swing when moving includes:

[0070] S81: Detecting the weight of the package through a weight sensor on the sorting device;

[0071] S82: mobilizing the sorting model to grab the second point cloud model at the grabbing position of the sorting information, and assigning the weight of the package to the second point cloud model, so that the second point cloud model generates inertia when moving;

[0072] S83: mobilizing the sorting model to move the second point cloud model at a set speed, and identifying whether the second point cloud model swings during the movement.

[0073] Determining the moving speed that allows the swinging package to just avoid colliding with the sorting system includes:

[0074] S101: Use the set speed as the simulation speed;

[0075] S102: mobilizing the sorting model to pick up the second point cloud model at the picking position of the sorting information, and sorting the second point cloud model to the target sorting frame model at the simulation speed according to the moving path of the sorting information, and determining whether the second point cloud model collides with the first point cloud model during the sorting process. If not, the simulation speed is a moving speed that makes the swinging package just not collide with the sorting system;

[0076] S103: If yes, the simulation speed is adjusted down to a set value, and steps S102 to S103 are executed until a moving speed is determined at which the swinging package just does not collide with the sorting system.

[0077] In this embodiment, the physical engine of the three-dimensional processing software is started. At this time, after the weight of the package is assigned to the second point cloud model, the second point cloud model will generate inertia when moving and swing accordingly. In this case, multiple simulations are performed at different moving speeds in the determined sorting method until the second point cloud model does not collide. Since the simulation order is from high to low in terms of moving speed, the fastest speed at which the second point cloud model does not collide can be obtained. The set value can be 0.1m / s. The smaller the set value, the higher the simulation density, and the easier it is to obtain the fastest speed at which the second point cloud model does not collide.

[0078] like Figure 4 As shown, in one embodiment, a visual inspection simulation device for logistics sorting is provided, the device comprising:

[0079] A first modeling module is used to obtain a structural image of a logistics sorting system, and generate a first point cloud model according to the obtained structural image, wherein the sorting system includes a conveying device, a sorting device and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device and a sorting frame model corresponding to each sorting frame;

[0080] A first processing module is used to determine a target sorting frame to which each parcel needs to be sorted when the parcel is placed on the conveying device, and then determine a target sorting frame model;

[0081] A second modeling module is used to collect the appearance image of the package and generate a second point cloud model according to the collected appearance image;

[0082] The second processing module is used to mobilize the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model;

[0083] The third processing module is used to determine the grabbing position and moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path;

[0084] A fourth processing module, for integrating the determined grabbing position and moving path into sorting information corresponding to the package;

[0085] The sending module is used to send the sorting information to the sorting device, so that the sorting device sorts the package according to the sorting information when the package moves to the sorting position.

[0086] The process of each module realizing its own function in the visual inspection simulation device for logistics sorting provided in the embodiment of the present application can be specifically referred to the aforementioned Figure 1 The description of the illustrated embodiment will not be repeated here.

[0087] like Figure 5 As shown, in one embodiment, a visual inspection simulation system for logistics sorting is provided, the system comprising:

[0088] A sorting system, including a conveying device, a sorting device and a plurality of sorting frames, for sorting packages;

[0089] A visual device for collecting images;

[0090] The computer equipment is connected with the sorting system and the visual device, and is used to execute the visual inspection simulation method for logistics sorting.

[0091] In this embodiment, the computer equipment cooperates with the visual device and the sorting system to execute the visual detection simulation method for logistics sorting. First, three-dimensional point cloud models are established for the sorting system and the packages to be sorted, and then the point cloud models are mobilized to simulate the sorting to determine the sorting method of collision-free sorting. The sorting information including the sorting method is sent to the sorting device, so that the sorting device sorts the packages according to the sorting information. In the present application, targeted simulation can be achieved for packages of any shape and size, and then collision-free sorting can be achieved, ensuring the safety of each package during sorting.

[0092] Figure 6 FIG. 2 shows an internal structure diagram of a computer device in one embodiment. Figure 6As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the visual detection simulation method for logistics sorting provided by an embodiment of the present invention. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the visual detection simulation method for logistics sorting provided by an embodiment of the present invention. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0093] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0094] In one embodiment, the visual inspection simulation device for logistics sorting provided by the embodiment of the present invention can be implemented in the form of a computer program. The computer program can be used in Figure 6 The computer device shown in the figure is run on the computer device. The memory of the computer device can store various program modules that constitute the visual inspection simulation device for logistics sorting, for example, Figure 4 The computer program composed of the first modeling module, the first processing module, the second modeling module, the second processing module, the third processing module, the fourth processing module, and the sending module shown in the figure enables the processor to execute the steps of the visual inspection simulation method for logistics sorting in various embodiments of the present invention described in this specification.

[0095] For example, Figure 6 The computer device shown can be Figure 4 The first modeling module in the visual inspection simulation device for logistics sorting shown executes step S1; the computer device can execute step S2 through the first processing module; the computer device can execute step S3 through the second modeling module; the computer device can execute step S4 through the second processing module; the computer device can execute step S5 through the third processing module; the computer device can execute step S6 through the fourth processing module; the computer device can execute step S7 through the sending module.

[0096] In one embodiment, a computer device is provided, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0097] S1: Collecting a structural image of a logistics sorting system, and generating a first point cloud model based on the collected structural image, wherein the sorting system includes a conveying device, a sorting device, and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device, and a sorting frame model corresponding to each sorting frame;

[0098] S2: When each package is placed on the conveyor, determine the target sorting frame to which the package needs to be sorted, and then determine the target sorting frame model;

[0099] S3: collecting an appearance image of the package, and generating a second point cloud model according to the collected appearance image;

[0100] S4: mobilizing the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model;

[0101] S5: determining a grabbing position and a moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path;

[0102] S6: Integrate the determined grabbing position and moving path into sorting information corresponding to the package;

[0103] S7: Sending the sorting information to the sorting device, so that the sorting device sorts the parcel according to the sorting information when the parcel moves to the sorting position.

[0104] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor performs the following steps:

[0105] S1: Collecting a structural image of a logistics sorting system, and generating a first point cloud model based on the collected structural image, wherein the sorting system includes a conveying device, a sorting device, and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device, and a sorting frame model corresponding to each sorting frame;

[0106] S2: When each package is placed on the conveyor, determine the target sorting frame to which the package needs to be sorted, and then determine the target sorting frame model;

[0107] S3: collecting an appearance image of the package, and generating a second point cloud model according to the collected appearance image;

[0108] S4: mobilizing the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model;

[0109] S5: determining a grabbing position and a moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path;

[0110] S6: Integrate the determined grabbing position and moving path into sorting information corresponding to the package;

[0111] S7: Sending the sorting information to the sorting device, so that the sorting device sorts the parcel according to the sorting information when the parcel moves to the sorting position.

[0112] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0113] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0114] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A visual inspection simulation method for logistics sorting, characterized in that: The method comprises: S1: Collecting a structural image of a logistics sorting system, and generating a first point cloud model based on the collected structural image, wherein the sorting system includes a conveying device, a sorting device, and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device, and a sorting frame model corresponding to each sorting frame; S2: When each package is placed on the conveyor, determine the target sorting frame to which the package needs to be sorted, and then determine the target sorting frame model; S3: collecting an appearance image of the package, and generating a second point cloud model according to the collected appearance image; S4: mobilizing the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model; S5: determining a grabbing position and a moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path; S6: Integrate the determined grabbing position and moving path into sorting information corresponding to the package; S7: sending the sorting information to the sorting device, so that the sorting device sorts the parcel according to the sorting information when the parcel moves to the sorting position; The sorting model is mobilized to simulate sorting of the second point cloud model several times until the collision-free sorting with the shortest moving path for the second point cloud model is determined to include: S41: determining a plurality of groups of grabbing positions according to the shape of the second point cloud model; S42: taking a group of grasping positions, mobilizing the sorting model to grab the second point cloud model from the group of grasping positions, performing simulated sorting of different moving paths for several times, so as to determine the candidate simulated sorting corresponding to the shortest moving path of the group of grasping positions, and repeating this step until the candidate simulated sorting corresponding to each group of grasping positions is determined; S43: comparing the moving path lengths of the candidate simulated sortings, and taking the candidate simulated sorting with the shortest moving path as the final collision-free sorting; Determining a plurality of groups of grasping positions according to the shape of the second point cloud model includes: S411: Identify the shape of the second point cloud model, and determine a plurality of groups of clamping points according to the shape and a preset shape-clamping point correspondence table; S412: taking a group of clamping points, and identifying a corresponding position point of each clamping point in the group of clamping points on the second point cloud model; S413: Determine whether the distance between each two corresponding position points is less than the maximum contact distance of the clamping part of the sorting model. If so, the corresponding position points of the group of clamping points are a group of grasping positions. Otherwise, the corresponding position points of the group of clamping points are not grasping positions. S414: Repeat steps S412 to S413 until it is determined whether the corresponding position point of each group of clamping points is a grasping position.

2. The method according to claim 1, characterized in that Take a set of grasping positions, mobilize the sorting model to grab the second point cloud model from the set of grasping positions, and perform simulated sorting of several different moving paths to determine the alternative simulated sorting of the shortest moving path corresponding to the set of grasping positions, including: S421: when the second point cloud model is grabbed by the sorting model, all candidate paths for moving the second point cloud model to the target sorting frame model are generated; S422: Arrange the candidate paths in order from short to long to obtain a path sequence; S423: taking the first selected path from the path sequence, and mobilizing the gripping device to sort the second point cloud model into the target sorting frame model according to the selected path; S424: determining whether the second point cloud model collides with the first point cloud model during the movement process, and if not, determining the simulated sorting corresponding to the to-be-selected path as the candidate simulated sorting; S425: If yes, remove the candidate path from the path sequence, and execute steps S423 to S425 until the candidate simulation sorting is determined.

3. The method according to claim 1, characterized in that After sending the sorting information to the sorting device, the method further includes: S8: When the sorting device picks up the package according to the sorting position of the sorting information, it is determined whether the picked up package swings when moving; S9: If not, the set speed is sent to the sorting device, so that the sorting device sorts the package to the target sorting frame at the set speed according to the moving path of the sorting information; S10: If so, determine the moving speed at which the swinging package just does not collide with the sorting system, divide the moving speed by a preset safety factor to obtain an upper limit moving speed, and send the upper limit moving speed to the sorting device, so that the sorting device sorts the package to the target sorting frame at the upper limit moving speed according to the moving path of the sorting information.

4. The method according to claim 3, characterized in that Determining whether the picked up package will swing when moving includes: S81: Detecting the weight of the package through a weight sensor on the sorting device; S82: mobilizing the sorting model to grab the second point cloud model at the grabbing position of the sorting information, and assigning the weight of the package to the second point cloud model, so that the second point cloud model generates inertia when moving; S83: mobilizing the sorting model to move the second point cloud model at a set speed, and identifying whether the second point cloud model swings during the movement.

5. The method according to claim 4, characterized in that Determining the moving speed that allows the swinging package to just avoid colliding with the sorting system includes: S101: Use the set speed as the simulation speed; S102: mobilizing the sorting model to pick up the second point cloud model at the picking position of the sorting information, and sorting the second point cloud model to the target sorting frame model at the simulation speed according to the moving path of the sorting information, and determining whether the second point cloud model collides with the first point cloud model during the sorting process. If not, the simulation speed is a moving speed that makes the swinging package just not collide with the sorting system; S103: If yes, the simulation speed is adjusted down to a set value, and steps S102 to S103 are executed until a moving speed is determined at which the swinging package just does not collide with the sorting system.

6. A visual inspection simulation device for logistics sorting, characterized in that: The device comprises: A first modeling module is used to obtain a structural image of a logistics sorting system, and generate a first point cloud model according to the obtained structural image, wherein the sorting system includes a conveying device, a sorting device and a plurality of sorting frames, and the first point cloud model includes a conveying model corresponding to the conveying device, a sorting model corresponding to the sorting device and a sorting frame model corresponding to each sorting frame; A first processing module is used to determine a target sorting frame to which each parcel needs to be sorted when the parcel is placed on the conveying device, and then determine a target sorting frame model; A second modeling module is used to collect the appearance image of the package and generate a second point cloud model according to the collected appearance image; The second processing module is used to mobilize the sorting model to perform simulated sorting on the second point cloud model for several times until a collision-free sorting with the shortest moving path for the second point cloud model is determined, wherein the collision-free sorting means that the second point cloud model does not collide with any other part of the first point cloud model during the process of being sorted by the sorting model to the target sorting frame model; The third processing module is used to determine the grabbing position and moving path of the sorting model for the second point cloud model during collision-free sorting with the shortest moving path; A fourth processing module, for integrating the determined grabbing position and moving path into sorting information corresponding to the package; A sending module, used for sending the sorting information to the sorting device, so that the sorting device sorts the parcel according to the sorting information when the parcel moves to the sorting position; The sorting model is mobilized to simulate sorting of the second point cloud model several times until the collision-free sorting with the shortest moving path for the second point cloud model is determined to include: S41: determining a plurality of groups of grabbing positions according to the shape of the second point cloud model; S42: taking a group of grasping positions, mobilizing the sorting model to grab the second point cloud model from the group of grasping positions, performing simulated sorting of different moving paths for several times, so as to determine the candidate simulated sorting corresponding to the shortest moving path of the group of grasping positions, and repeating this step until the candidate simulated sorting corresponding to each group of grasping positions is determined; S43: comparing the moving path lengths of the candidate simulated sortings, and taking the candidate simulated sorting with the shortest moving path as the final collision-free sorting; Determining a plurality of groups of grasping positions according to the shape of the second point cloud model includes: S411: Identify the shape of the second point cloud model, and determine a plurality of groups of clamping points according to the shape and a preset shape-clamping point correspondence table; S412: taking a group of clamping points, and identifying a corresponding position point of each clamping point in the group of clamping points on the second point cloud model; S413: Determine whether the distance between each two corresponding position points is less than the maximum contact distance of the clamping part of the sorting model. If so, the corresponding position points of the group of clamping points are a group of grasping positions. Otherwise, the corresponding position points of the group of clamping points are not grasping positions. S414: Repeat steps S412 to S413 until it is determined whether the corresponding position point of each group of clamping points is a grasping position.

7. A visual inspection simulation system for logistics sorting, characterized in that: The system comprises: A sorting system, including a conveying device, a sorting device and a plurality of sorting frames, for sorting packages; A visual device for collecting images; A computer device is connected to a sorting system and a visual device, and is used to execute the visual inspection simulation method for logistics sorting as described in any one of claims 1 to 5.

Images