Transfer method, transfer apparatus and electronic device

By identifying and adjusting the movement speed of the transport device according to the attribute information of the target object, the problem of low transport efficiency in the prior art is solved, and more efficient object transport is achieved.

WO2024230654A1PCT designated stage expired Publication Date: 2024-11-14MECH MIND ROBOTICS TECH LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/091291
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-05-06
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

In the prior art, the transfer device adopts the same movement speed for different express parcels, resulting in low transport efficiency, easy to damage large objects or extend the transfer time of small objects.

Method used

By collecting the target image, identifying the attribute information of the target object, determining the movement speed based on the attribute information and the preset relationship, and controlling the transport device to transport the target object at the corresponding speed.

Benefits of technology

Improve the transport efficiency of target objects with different attribute information, avoid object damage and extended transport time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024091291_14112024_PF_FP_ABST
    Figure CN2024091291_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present disclosure provides a transfer method, a transfer apparatus and an electronic device. The transfer method comprises: collecting a target image, the target image comprising a plurality of target objects; identifying the target image to obtain attribute information of each target object; according to the attribute information, determining the moving speed of a transfer apparatus transferring a corresponding target object, the attribute information and the moving speed having a preset correspondence; and controlling the transfer apparatus to transfer the corresponding target object according to the moving speed. According to the present disclosure, a transfer apparatus is controlled to transfer a target object with different attribute information at a corresponding moving speed, thereby improving the transfer efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Transfer method, transfer device, and electronic equipment

[0001] This disclosure claims priority to Chinese patent applications filed with the Chinese Patent Office on May 11, 2023, with application number 202310533520.8, application name “Transfer method, transfer equipment and electronic equipment”, application number 202310533017.2, application name “Transfer method, transfer equipment and electronic equipment”, and application number 202310531970.3, application name “Classification method, transfer method, device and electronic equipment”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of computer technology, and in particular to a transport method, a transport device, and an electronic device. Background Art

[0003] In many scenarios, objects need to be transferred. For example, in logistics scenarios, the transfer device can automatically grab express parcels and then transfer the express parcels to the corresponding destination to realize the sorting of express parcels.

[0004] In the related art, the transfer device uses the same transfer method for different express parcels, which results in low transfer efficiency. Technical Solutions

[0005] Various aspects of the present disclosure provide a transport method, a transport device, and an electronic device to improve the transport efficiency of a target object.

[0006] A first aspect of an embodiment of the present disclosure provides a transfer method, which is applied to a transfer device, including: acquiring a target image, the target image including multiple target objects; identifying the target image to obtain attribute information of each target object; determining a moving speed of the transfer device for transferring the corresponding target object based on the attribute information, the attribute information and the moving speed having a preset corresponding relationship; and controlling the transfer device to transfer the corresponding target object according to the moving speed.

[0007] A second aspect of the present disclosure provides a transfer device, comprising:

[0008] An acquisition module is used to acquire a target image, where the target image includes multiple target objects;

[0009] Recognition module, used to identify the target image and obtain the attribute information of each target object;

[0010] a determination module, configured to determine a moving speed of the transfer device for transferring a corresponding target object based on the attribute information, wherein the attribute information and the moving speed have a preset corresponding relationship;

[0011] The control module is used to control the transfer device to transfer the corresponding target object according to the moving speed.

[0012] A third aspect of an embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the transport method of the first aspect when executing the computer program.

[0013] A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the transport method of the first aspect.

[0014] A fifth aspect of an embodiment of the present disclosure provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the transport method of the first aspect.

[0015] The disclosed embodiments are applied in the transportation scenario of objects, by collecting target images, which include multiple target objects; identifying the target images to obtain attribute information of each target object; determining the moving speed of the transportation device for transporting the corresponding target object based on the attribute information, and the attribute information and the moving speed have a preset corresponding relationship; controlling the transportation device to transport the corresponding target object according to the moving speed, which can achieve the goal of transporting target objects with different attribute information by controlling the transportation device to adopt the corresponding moving speed, thereby improving the efficiency of transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0017] FIG1 is an application scenario diagram of a transshipment method provided by an exemplary embodiment of the present disclosure;

[0018] FIG2 is a flow chart of steps of a transport method provided by an exemplary embodiment of the present disclosure;

[0019] FIG3 is a schematic diagram of a target image provided by an exemplary embodiment of the present disclosure;

[0020] FIG4 is a schematic diagram of a transfer device provided by an exemplary embodiment of the present disclosure;

[0021] FIG5 is a flowchart of another method for transporting provided by an exemplary embodiment of the present disclosure;

[0022] FIG6 is a structural block diagram of a recognition model provided by an exemplary embodiment of the present disclosure;

[0023] FIG7 is a structural block diagram of a Focus provided by an exemplary embodiment of the present disclosure;

[0024] FIG8 is a structural block diagram of an SPP provided by an exemplary embodiment of the present disclosure;

[0025] FIG9 is a structural block diagram of a CSP1 provided by an exemplary embodiment of the present disclosure;

[0026] FIG10 is a structural block diagram of a CSP2 provided by an exemplary embodiment of the present disclosure;

[0027] FIG11 is a schematic diagram of a mask image provided by an exemplary embodiment of the present disclosure;

[0028] FIG12 is a structural block diagram of a transfer device provided by an exemplary embodiment of the present disclosure;

[0029] FIG13 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Modes for Carrying Out the Invention

[0030] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present disclosure.

[0031] When a transfer device is transferring a target object from one location to a designated location, the related art uses the same movement speed for different target objects during the transfer process. For example, if target object A is large and target object B is small, using a fast movement speed for target object A could easily cause the target object to be thrown out during transfer, preventing the transfer of target object A. Using a slow movement speed for target object B would extend the transfer time of target object B. In summary, using the same movement speed for transferring different target objects reduces the transfer efficiency of the target objects.

[0032] Based on the above problems, the present invention collects target images, which include multiple target objects; identifies the target images to obtain attribute information of each target object; determines the moving speed of the transfer device for transferring the corresponding target object based on the attribute information, and the attribute information and the moving speed have a preset corresponding relationship; controls the transfer device to transfer the corresponding target object according to the moving speed, thereby achieving corresponding moving speeds for target objects with different attribute information, thereby improving the transfer efficiency of the target objects.

[0033] In addition, an application scenario of an embodiment of the present disclosure is shown in Figure 1, which includes: a carrying platform P1, a carrying platform P2, and a target object D. In particular, the transfer device can grab the target object D on the carrying platform P1 and transfer the target object D to the carrying platform P2 according to transfer route a or transfer route b. The present disclosure is used to control the movement speed of the transfer device during the transfer process to improve the transfer efficiency of the target object.

[0034] 1 is only an exemplary application scenario, and the embodiments of the present disclosure can be applied to any object transport scenario. The embodiments of the present disclosure are not limited to specific application scenarios.

[0035] FIG2 is a flow chart of the steps of a transfer method provided by an exemplary embodiment of the present disclosure. The transfer method is applied to a transfer device and specifically includes the following steps:

[0036] S201, collecting target images.

[0037] Wherein, the target image includes multiple target objects. In the present disclosure, the target image may include one target object.

[0038] In the present disclosure, the transfer device carries a camera, which can capture target images of the target object.

[0039] Exemplarily, referring to FIG3 , an image of a target object placed on a carrying platform is captured to obtain a target image P. The target image P includes multiple target objects (h, z, q, b, d1, d2, d3, d4, y, u, and d5), where "FY" on target object z is the mark on the outer packaging of target object z.

[0040] In addition, the target object is an express parcel; the categories of the target object include: a box-shaped parcel, a cylindrical parcel, a spherical parcel, a bag-shaped parcel, a sheet-shaped parcel or an irregular-shaped parcel.

[0041] In the present disclosure, box-shaped packages are express parcels such as express parcels loaded in paper boxes, such as target objects h and u in Figure 3. Cylindrical packages refer to express parcels with cylindrical or quasi-cylindrical shapes, such as target object b in Figure 3. Spherical packages refer to express parcels with spherical or quasi-spherical shapes, such as target object q in Figure 3. Bag-shaped packages refer to express parcels loaded in plastic bags, such as target objects d1, d2, d3, and d4 in Figure 3. Sheet-shaped packages are such as envelopes, such as target object z in Figure 3. Special-shaped packages refer to express parcels of other types set in addition to box-shaped packages, cylindrical packages, spherical packages, bag-shaped packages, and sheet-shaped packages, such as target objects y and d5 in Figure 3. In addition, users can set special-shaped packages according to actual needs. For example, users can classify cylindrical packages and spherical packages as special-shaped packages. The category of the target object includes: one of box-shaped packages, bag-shaped packages, sheet packages, or special-shaped packages.

[0042] In the present disclosure, multiple categories may be pre-set, and then the category to which the target object belongs may be determined.

[0043] S202: Identify the target image and obtain attribute information of each target object.

[0044] In the embodiment of the present disclosure, the attribute information includes: basic attributes, probability values ​​of mask objects and target objects in different categories, wherein the basic attributes include at least one of category, completeness, posture state, color features, and texture features.

[0045] Among them, the degree of completeness can be divided into complete, semi-overlapping, highly overlapping and truncated. Complete means that the target object is not overlapped by other target objects and can be completely displayed in the target image, such as target object q, target object d3, target object h, target object y, target object u and target object d5 in Figure 3. Semi-overlapping means that a small part of the target object is overlapped by other target objects, such as target object b, target object d1 and target object z in Figure 3. Highly overlapping means that a large part of the target object is overlapped by other target objects, such as target object d4 in Figure 3. Truncated means that part of the target object is not displayed in the target image, such as target object d2 in Figure 3. In the present disclosure, the demarcation threshold between semi-overlapping and highly overlapping can be set as needed and is not limited here. For example, it can be set that the overlapping ratio is less than or equal to 50% is semi-overlapping, and the overlapping ratio is greater than 50% is highly overlapping.

[0046] Furthermore, the posture states can be divided into normal posture, vertical posture and inclined posture. The normal posture means that the object to be transported is placed on the carrying platform in a normal state, such as target object d1, target object d4, target object q, etc. in Figure 3. The vertical posture means that the object to be transported is placed vertically on the carrying platform. The inclined posture means that the object to be transported is placed on the carrying platform at an angle, such as the object to be transported h in Figure 3, where part of the object to be transported h is on the carrying platform and the other part is stacked on other objects to be transported, and is inclined relative to the carrying platform. Color features can be divided into normal color, reflection and other colors. For example, for box-shaped packages, the brown color of the paper box is the normal color. If the box-shaped package is loaded in a foam box, the color feature is white. If it is packaged with a black bag, the color feature is black. If it is wrapped with plastic tape, the color feature includes reflection. Texture features can be divided into normal, wrinkled, foam, damaged, etc.

[0047] In the present disclosure, the attribute information of the target object can be obtained by recognizing the target image.

[0048] S203: Determine the moving speed of the transfer device for transferring the corresponding target object according to the attribute information.

[0049] The attribute information and the moving speed have a preset corresponding relationship. In the present disclosure, the corresponding relationship between the attribute information and the moving speed can be preset. If the attribute information is a category, the corresponding relationship between the category and the moving speed can be preset, as shown in Table 1.

[0050] Table 1

[0051]

[0052] If the attribute information is category and posture state, the correspondence between (category, posture state) and movement speed can be preset, as shown in Table 2.

[0053] Table 2

[0054]

[0055] In the present disclosure, corresponding moving speeds can be set for target objects given different attribute information according to actual needs to improve the transport efficiency of the target objects.

[0056] Furthermore, referring to Figure 1 , during the transfer process, the target object can be transferred according to transfer route A or transfer route B. Transfer route A is divided into multiple segments (e.g., segment X, segment Y, and segment Z). In the present disclosure, the correspondence between attribute information and each segment can be preset. For example, if the category is a sheet package, the moving speed of segment X is 90% V, the moving speed of segment Y is V, and the moving speed of segment Z is 85%.

[0057] In the present disclosure, the correspondence between attribute information and movement speed can be preset as needed, so that during the transportation process, the movement speed that the target object should adopt can be determined based on the attribute information of the identified target object.

[0058] S204, controlling the transfer device to transfer the corresponding target object according to the moving speed.

[0059] In the present disclosure, the transfer device can grasp the target object and then transfer the grasped target object to the target location. During the transfer process, the target object moves according to the above-determined moving speed.

[0060] For example, referring to FIG4 , a transfer device 40 is shown, comprising a base 41, a motion segment 42, and an end effector 43. Base 41 is stationary, while end effector 43, as motion segment 42 moves, can move to platform P1 to grab a target object. For example, in FIG3 , if the target objects are q, h, and y, end effector 43 can transfer package q to platform P2 at the speed of package q, transfer package h to platform P2 at the speed of package h, and transfer package y to platform P2 at the speed of package y.

[0061] In addition, the transfer device 40 further includes a camera (not shown) for capturing target images.

[0062] Furthermore, if the transfer device is a suction cup device, the transfer device is controlled to transfer the corresponding target object according to the moving speed, including: determining the air flow rate of the suction cup device based on the attribute information, the attribute information and the air flow rate have a preset corresponding relationship; controlling the suction cup device to absorb the target object according to the air flow rate, and transferring the absorbed target object according to the moving speed.

[0063] Among them, the air flow corresponding to the sheet-shaped package is lower than the air flow corresponding to the box-shaped package.

[0064] In the embodiment of the present disclosure, referring to FIG4 , the end effector 43 may be a suction cup, and the transfer device may use the suction cup to suck up the target object. The suction cup moves the target object sucked from the carrying platform P1 to the carrying platform P2 at a moving speed.

[0065] Furthermore, in the present disclosure, the correspondence between the target object's attribute information and the airflow rate can be pre-set. The greater the airflow rate, the stronger the suction force of the suction cup. Since sheet-shaped packages (such as envelopes) are easily damaged by strong suction, the corresponding airflow rate can be set to be lower for sheet-shaped packages. Since box-shaped packages are typically rigid boxes, the corresponding airflow rate is higher.

[0066] Furthermore, the correspondence between attribute information and airflow can include a correspondence between category and airflow, or a correspondence between "category, other attribute information" and airflow. Users can pre-set this according to actual needs. For example, if the default airflow is Q, the airflow corresponding to box-shaped packages can be set to 100%Q, the airflow corresponding to bag-shaped packages to 80%Q, and the airflow corresponding to sheet-shaped packages to 60%Q.

[0067] In summary, the present invention collects target images, which include multiple target objects; identifies the target images to obtain attribute information of each target object; determines the moving speed of the transfer device for transporting the corresponding target object based on the attribute information, and the attribute information and the moving speed have a preset corresponding relationship; controls the transfer device to transport the corresponding target object according to the moving speed, and can realize the transfer of target objects with different attribute information by controlling the transfer device to adopt the corresponding moving speed, thereby improving the efficiency of transportation.

[0068] FIG5 is a flowchart of another method for transporting goods provided by an exemplary embodiment of the present disclosure. Specifically, the method includes the following steps:

[0069] S501, acquiring a target image.

[0070] The specific implementation method of this step is referred to S201 and will not be repeated here.

[0071] S502: Perform feature extraction processing on the target image through a backbone feature extraction module to obtain a first feature image.

[0072] 6 , identifying a target image includes identifying the target image using a recognition model 60 ; the recognition model 60 includes a backbone feature extraction module 61 , an enhanced feature extraction module 62 , and an attribute determination module 63 . The recognition model may be a YOLOX (a neural network) model.

[0073] Specifically, the backbone feature extraction module 61 includes Focus (concentration layer), (CBA / DWConv) 1 (first depth-wise separable convolution layer), CSP1 1 (first residual network layer 1), (CBA / DWConv) 2 (second depth-wise separable convolution layer), CSP1 2 (second residual network layer 1), (CBA / DWConv) 3 (third depth-wise separable convolution layer), CSP1 3 (third residual network layer 1), SPP (pooling layer) and CSP1 4 (fourth residual network layer 1).

[0074] As shown in Figure 7, Focus includes four Slices (program slices), Concat (concatenation unit), and CBA (attention mechanism feedback layer). Slice extracts a value for every other pixel in the target image (3×W×H, representing 3 channels, with length W and width H) to obtain four independent feature images. Concat and CBA then stack these four independent feature images, expanding the target image from 3 channels by four times. Focus processes and extracts features from the target image to obtain a feature image of {12×(W / 2)×(H / 2)}.

[0075] Furthermore, as shown in Figure 7, CBA includes Conv (convolutional layer), BN (batch normalization), and Act (activation layer). As shown in Figure 8, SPP includes three Maxpool (maximum pooling) layers, Concat, and CBA. As shown in Figure 9, CSP1 includes: a CBA branch and n Bottleneck layers, and a CBA branch. The outputs of the two branches are concatenated via Concat and then processed by a CBA. The Bottleneck layer includes a CBA and a (CBA / DWConv). The input features of the Bottleneck pass through the CBA and a (CBA / DWConv) layer, and then are superimposed with the input features for the output.

[0076] Furthermore, the target image can be processed by the backbone feature extraction module 61 to obtain a first feature image or multiple first feature images. As shown in Figure 6, the backbone feature extraction module 61 can output a first feature image a1, a first feature image a2, and a first feature image a3, or it can output only one of the three first feature images, for example, only one first feature image a3. Specifically, the first feature image a1 is a feature image of size {192 × (W / 8) × (H / 8)} output by CSP1 2, the first feature image a2 is a feature image of size {768 × (W / 16) × (H / 16)} output by CSP1 3, and the first feature image a3 is a feature image of size {3072 × (W / 32) × (H / 32)} output by CSP1 4.

[0077] S503: Perform feature enhancement processing on the first feature image by using an enhanced feature extraction module to obtain a second feature image.

[0078] In the present disclosure, referring to FIG6 , the enhanced feature extraction module 62 includes CBA 1 (first attention mechanism feedback layer), FPN (Feature Pyramid Network) and PAN (Perceptual adversarial network).

[0079] Furthermore, FPN includes: Upsample 1 (first upsampling layer), Concat 1 (first concatenation unit), CSP2 1 (first residual network layer 2), CBA 2 (second attention mechanism feedback layer), Upsample 2 (second upsampling layer), and Concat 2 (second concatenation unit). PAN includes: CSP2 2 (second residual network layer 2), (CBA / DWConv) 4 (fourth depthwise separable convolution layer), Concat 3 (third concatenation unit), CSP2 3 (third residual network layer 2), (CBA / DWConv) 5 (fifth depthwise separable convolution layer), Concat 4 (fourth concatenation unit), and CSP2 4 (fourth residual network layer 2).

[0080] 10 , CSP2 includes: a CBA and n CBAs of one branch, and a CBA of another branch. The outputs of the two branches are connected through Concat and then processed through a CBA.

[0081] In the present disclosure, the second feature image may be one or more. Referring to FIG. 6 , three second feature images may be included: a second feature image b1, a second feature image b2, and a second feature image b3. Alternatively, a single second feature image may be included, for example, where the enhanced feature extraction module 62 only outputs the second feature image b3. Furthermore, the second feature image b1 is output by CSP2 2, and its dimensions are {192 × (W / 8) × (H / 8)}. The second feature image b2 is output by CSP2 3, and its dimensions are {768 × (W / 16) × (H / 16)}. The second feature image b3 is output by CSP2 4, and its dimensions are {3072 × (W / 32) × (H / 32)}.

[0082] S504: Perform attribute analysis on the second feature image through an attribute determination module to obtain basic attributes of the target object and probability values ​​of the target object in different categories.

[0083] In the present disclosure, referring to FIG6 , the attribute determination module 63 includes: CBA 3 (third attention mechanism feedback layer, (CBA / DWConv) 6 (sixth depth-wise separable convolution layer), (CBA / DWConv) 7 (seventh depth-wise separable convolution layer), Conv 1 to Conv 5 (first to fifth convolution layers).

[0084] In the present disclosure, if there are multiple second feature images, the multiple second feature images can be input into the attribute determination module for processing. The attribute determination module can output the probability of belonging to different levels of completeness, the probability of being in different posture states, the probability of being in different categories, the probability of being in different color features, and the probability of being in different texture features for each second feature image. The multiple second feature images are then averaged to obtain the final completeness, posture state, color features, and texture features. The basic attributes include: completeness, posture state, color features, and texture features.

[0085] For example, referring to Table 3, if there is only one second feature image b3, then based on the probabilities corresponding to second feature image b3, the target object's attribute information can be determined to include: "Completeness: Semi-Overlapped, Posture: Normal, Color: Black, Texture: Wrinkled," and the probability values ​​of the target object belonging to different categories. Furthermore, if there are multiple second feature images, such as second feature image b1, second feature image b2, and second feature image b3, the attribute determination module processes each second feature image separately and calculates the average of the probabilities to determine the target object's attribute information as: Completeness: Semi-Overlapped, Posture: Normal, Color: Black, Texture: Wrinkled, and the probability values ​​of the target object belonging to different categories, such as: Box-shaped package: 60%; Column-shaped package: 2.3%; Spherical package: 1%; Bag-shaped package: 18%; Sheet-shaped package: 15%; and Odd-shaped package: 3.7%.

[0086] Table 3

[0087]

[0088] S505 , performing mask analysis processing on the second feature image by a mask determination module to obtain a mask image.

[0089] In one embodiment, when the enhanced feature extraction module performs feature enhancement processing on the first feature image, a second feature image is obtained; the mask determination module includes: a first convolution unit and a first decoder, and the mask determination module performs mask analysis processing on the second feature image to obtain a mask image, including: convolution processing on the second feature image by the first convolution unit to obtain a third feature image; decoding processing on the third feature image by the first decoder to obtain a mask image.

[0090] In another embodiment, when the enhanced feature extraction module performs feature enhancement processing on the first feature image, multiple second feature images of different sizes are obtained; the mask determination module also includes: multiple second convolution units, splicing units and second decoders, the convolution kernel sizes of the multiple second convolution units are different, and the mask determination module performs mask analysis processing on the second feature image to obtain a mask image, including: performing convolution processing on the second feature image one by one through the second convolution unit to obtain multiple fourth feature images of the same size; performing splicing processing on multiple fourth feature images of the same size through the splicing unit to obtain a fifth feature image; and decoding processing on the fifth feature image through the second decoder to obtain a mask image.

[0091] 6 , the attribute information further includes: a mask object of the target object in a mask image corresponding to the target image, and the recognition model further includes a mask determination module 64 .

[0092] In an optional embodiment, when the enhanced feature extraction module performs feature enhancement processing on the first feature image, a second feature image is obtained; the mask determination module includes: a first convolution unit and a first decoder, and the mask determination module performs mask analysis processing on the second feature image to obtain a mask image, including: convolution processing on the second feature image through the first convolution unit to obtain a third feature image; decoding processing on the third feature image through the first decoder to obtain a mask image.

[0093] Specifically, if the enhanced feature extraction module outputs only a second feature image, the mask determination module includes only a first convolution unit (Conv) and a first decoder. The size of the second feature image corresponds to the convolution kernel size of the first convolution unit. For example, if the size of the second feature image is {192×(W / 8)×(H / 8), the convolution kernel size of the first convolution unit is 1×1. If the size of the second feature image is {768×(W / 16)×(H / 16)}, the convolution kernel size of the first convolution unit is 2×2. If the size of the second feature image is {3072×(W / 32)×(H / 32)}, the convolution kernel size of the first convolution unit is 4×4. The mask image is then decoded by the first decoder to obtain a 3×W×H mask image.

[0094] In another optional embodiment, when the enhanced feature extraction module performs feature enhancement processing on the first feature image, multiple second feature images of different sizes are obtained; the mask determination module also includes: multiple second convolution units, splicing units and second decoders, and the convolution kernel sizes of the multiple second convolution units are different. The mask determination module performs mask analysis processing on the second feature image to obtain a mask image, including: performing convolution processing on the second feature image one by one through the second convolution unit to obtain multiple fourth feature images of the same size; performing splicing processing on multiple fourth feature images of the same size through the splicing unit to obtain a fifth feature image; and decoding processing on the fifth feature image through the second decoder to obtain a mask image.

[0095] For example, referring to Figure 6 , the multiple second feature images of different sizes are second feature image b1 {192 × (W / 8) × (H / 8), second feature image b2 {768 × (W / 16) × (H / 16)}, and second feature image b3 {3072 × (W / 32) × (H / 32)}. These correspond to three second convolutional units: the second convolutional unit (Conv 8), the second convolutional unit (Conv 7), and the second convolutional unit (Conv 6) in Figure 6 . The convolution kernel size of Conv 8 is 1×1. The convolution kernel size of Conv 7 is 2×2. The convolution kernel size of Conv 6 is 4×4.

[0096] Furthermore, after the second feature image is convolved by the second convolution unit, it is spliced ​​by the splicing unit (Concat 5). The spliced ​​features are decoded by the second decoder (Decoder) to obtain a mask image. The size of the mask image is 3×W×H.

[0097] The mask object can represent the shape of the target object. In the present disclosure, the mask object of the target object is determined based on the mask image. The recognition model also outputs the mask object of the target object in the mask image corresponding to the target image.

[0098] 11 , which shows a mask image Y corresponding to the target image Pa in FIG3 .

[0099] S506: Determine a mask object of the target object according to the mask image.

[0100] 3 and 11 , the target objects (h, d4, z, q, d1, b, d2, y, and d3) correspond to mask objects (y1, y2, y3, y4, y5, y6, y7, y8, and y9), respectively.

[0101] S507: Determine the category corresponding to the maximum probability value among the probability values ​​of different categories as the preliminary category.

[0102] The preliminary categories of the multiple target objects included in the target image may be the same or different. In the present disclosure, the preliminary categories may be determined based on multiple factors such as the shape, color features, and texture features of the target objects. The mask object may represent the shape of the target object.

[0103] Among them, referring to Table 3, the probability values ​​of the target object in different categories are obtained, such as box-shaped package: 60%; cylindrical package: 2.3%; spherical package: 1%; bag-shaped package: 18%; sheet package: 15%; special-shaped package: 3.7%. It can be determined that the preliminary category of the target object is boxed package.

[0104] For example, if the target object is a box-shaped package, its shape is roughly rectangular, its color is brown, and its texture is paper or foam. A bag-shaped package is irregular in shape, and its color can be gray, black, or gray. Its texture is plastic.

[0105] In the present disclosure, multiple categories may be pre-set, and then the category to which the target object belongs is determined by identifying the target image, and this category is the preliminary category.

[0106] In the present disclosure, referring to FIG3 , a box-shaped package is an express package loaded in a paper box, and the preliminary category of the target object h in FIG3 is a box-shaped package. A cylindrical package refers to an express package with a cylindrical or quasi-cylindrical shape, such as the preliminary category of the target object b in FIG3 is a cylindrical package. A spherical package refers to an express package with a spherical or quasi-spherical shape, such as the preliminary category of the target object q in FIG3 is a spherical package. A bag-shaped package refers to an express package loaded in a plastic bag, such as the preliminary category of the target objects d1, d2, d3, d4, and d5 in FIG3 is a bag-shaped package. A sheet-shaped package is such as an envelope, such as the preliminary category of the target object z in FIG3 is a sheet-shaped package. An irregular-shaped package refers to an express package of other types set in addition to box-shaped packages, cylindrical packages, spherical packages, bag-shaped packages, and sheet packages, such as the preliminary category of the target object y in FIG3 is an irregular-shaped package. In addition, users can set the special-shaped package according to actual needs. For example, users can classify cylindrical packages and spherical packages as special-shaped packages.

[0107] S508: If the preliminary category does not belong to the preset category, determine the final category of the target object according to the mask object.

[0108] The attribute information also includes the final category of the target object, which has a preset corresponding relationship with the moving speed.

[0109] In the present disclosure, the preset category can be pre-set by the user. For example, when forwarding express parcels, the user pre-sets that express parcels belonging to the preset category will not be forwarded, such as special-shaped parcels.

[0110] In an alternative embodiment, if the preset category is irregular-shaped packages, the final category may be the initial category or the preset category. For example, if the initial category is box-shaped packages, the final category may be box-shaped packages or irregular-shaped packages. If the initial category is bag-shaped packages, the final category may be bag-shaped packages or irregular-shaped packages. If the initial category is sheet-shaped packages, the final category may be sheet-shaped packages or irregular-shaped packages.

[0111] For example, referring to FIG3 , if the preliminary category of target object h is a box-shaped package, and this preliminary category (box-shaped package) is not the preset category (irregular-shaped package), then after determining, based on the mask object, that the three-dimensional structure of target object h does not belong to the preset three-dimensional structure corresponding to the preset category, the final category of the target object can be determined to be a box-shaped package. If the preliminary category of target object u is a box-shaped package, and this preliminary category (box-shaped package) is not the preset category (irregular-shaped package), then after determining, based on the mask object, that the three-dimensional structure of target object u belongs to the preset three-dimensional structure corresponding to the preset category, the final category of the target object can be determined to be an irregular-shaped package. Based on the above, it can be understood that the present disclosure classifies all target objects that do not fall into the standard box, bag, or sheet shape as irregular-shaped packages, thereby improving the detection rate of irregular-shaped packages.

[0112] In another optional embodiment, the final category is a standard preliminary category or a non-standard preliminary category. For example, if the preliminary category is box-shaped packages, the final category is a standard box-shaped package or a non-standard box-shaped package. If the preliminary category is bag-shaped packages, the final category is a standard bag-shaped package or a non-standard bag-shaped package. If the preliminary category is sheet-shaped packages, the final category is a standard sheet-shaped package or a non-standard sheet-shaped package.

[0113] In the disclosed embodiment, the final category of the target object can be one of: standard box-shaped package, non-standard box-shaped package, standard columnar package, non-standard columnar package, standard spherical package, non-standard spherical package, standard bag-shaped package, non-standard bag-shaped package, standard sheet-shaped package, or non-standard sheet-shaped package. Different processing can be performed on target objects of different final categories, thereby improving the classification accuracy of the target objects.

[0114] For example, for a box-shaped package, if the ratio of its length, width, and height falls within a corresponding preset range, it is considered a standard box-shaped package. For example, the target object h in Figure 3 is a standard box-shaped package. If the ratio of its length, width, and height falls outside of the preset range, it is considered a non-standard box-shaped package. For example, if the target object u in Figure 3 is initially identified as a box-shaped package, but its length is significantly greater than its width and height, it can be determined to be a non-standard box-shaped package. For another example, the target object d3 in Figure 3 is initially classified as a bag-shaped package and ultimately classified as a standard bag-shaped package. The bag target object d5 in Figure 3 is initially classified as a bag-shaped package and ultimately classified as a non-standard bag-shaped package.

[0115] Furthermore, if the preliminary category does not belong to the preset category, the final category of the target object is determined based on the mask object, including: if the preliminary category does not belong to the preset category and the probability value under the preliminary category is less than the first threshold, the final category of the target object is determined based on the mask object; if the preliminary category does not belong to the preset category and the probability value under the preliminary category is greater than or equal to the first threshold, the final category of the target object is determined to be the preliminary category.

[0116] Specifically, in the disclosed embodiment, a first threshold may be pre-set, such as 80%. If the preliminary category of the object to be classified is a box-shaped package, and the probability value of the target object in the preliminary category (box-shaped package) is 60%, which is less than 80%, then if these two conditions are met, the final category of the target object is determined based on the mask object.

[0117] If the probability value under the preliminary category is greater than or equal to the first threshold, it can be understood that the recognition model recognizes the target object more accurately, and the final category of the target object can be directly determined to be the preliminary category.

[0118] For example, referring to Figure 3 , if the target object h is initially identified as a box-shaped package and the probability value of target object h being a box-shaped package is greater than 80%, the final category of target object h can be determined to be a box-shaped package. If the target object u is initially identified as a box-shaped package and the probability value of target object u being a box-shaped package is less than 80%, the final category of target object u can be determined to be an irregular-shaped package using the three-dimensional structure.

[0119] The present disclosure sets a condition for determining whether the probability value under the preliminary category is less than a first threshold value, so that the three-dimensional structure of the target object greater than the first threshold value does not need to be determined again, thereby improving the efficiency of classification.

[0120] In the embodiment of the present disclosure, the preliminary category and mask object of the target object can be first obtained through recognition model identification, and then based on the preliminary category and the probability value under the preliminary category, it is determined whether to use the mask object to determine the final category. In addition, by using the mask object to determine the final category, the classification accuracy can be improved.

[0121] Among them, determining the final category of the target object according to the mask object includes: determining the three-dimensional structure of the target object according to the mask object, the mask object including depth information of the target object; if the three-dimensional structure belongs to a preset three-dimensional structure of a preset category, determining the final category of the target object as the preset category; otherwise, determining the final category of the target object as the preliminary category.

[0122] The mask object may include pixel position information and depth information of the mask object. A three-dimensional structure of the target object may be constructed based on this pixel position information and depth information. If this three-dimensional structure indicates that the target object is a pre-set three-dimensional structure of a special-shaped package, the final category of the target object is a special-shaped package. Otherwise, the final category is determined to be a preliminary category (i.e., a box-shaped package).

[0123] In the present disclosure, a preset category may pre-set a corresponding plurality of preset three-dimensional structures, or pre-set the shape of a preset three-dimensional structure.

[0124] Among them, the preset category is special-shaped package.

[0125] In the embodiment of the present disclosure, it also includes: not transferring target objects belonging to a preset category.

[0126] The disclosed embodiments are applied in the classification of express parcels by collecting a target image, which includes a target object; identifying the target image to obtain a preliminary category and a mask object of the target object; if the preliminary category does not belong to a preset category, determining the final category of the target object based on the mask object, thereby achieving accurate classification of the target object.

[0127] S509: Remove the first object from the multiple target objects according to the completeness of the multiple target objects to obtain multiple second objects.

[0128] The first object has an incomplete degree of completeness, wherein the basic attributes include: degree of completeness. A target object with an incomplete degree of completeness refers to a degree of completeness other than complete, such as overlapping, semi-overlapping, or truncated, all of which are incomplete.

[0129] Furthermore, the removed first object is an incomplete target object. For example, in Figure 3, target object d4, target object z, target object d1, target object b, and target object d2 are all first objects. After removing the first objects, the multiple second objects obtained are target object h, target object q, target object u, target object d3, target object d5, and target object y.

[0130] S510: Determine at least part of the target objects based on the plurality of second objects.

[0131] In an optional embodiment, a plurality of second objects may be determined as at least part of the target object.

[0132] Alternatively, the basic attributes also include: posture state; determining at least part of the target objects based on multiple second objects, including: if the posture state of a third object among the multiple second objects is a preset non-transport posture state, then removing the third object from the multiple second objects to obtain at least part of the target objects.

[0133] For example, the preset non-transportation posture states are a vertical posture and an inclined posture. As shown in Figure 3, if the target object h is in a vertical posture, the target object h is the third object, and the target object h is removed from the multiple second images. The remaining target object q, target object u, target object d3, target object d5 and target object y are at least partial target objects.

[0134] Alternatively, if the plurality of second objects can be divided into at least two categories, then at least some of the target objects are determined based on the plurality of second objects, including: if the category of the fourth object among the plurality of second objects is a preset non-transfer category, then the fourth object is removed from the plurality of second objects to obtain at least some of the target objects.

[0135] Exemplarily, the irregular-shaped parts are pre-set as a preset non-transfer category, and the irregular-shaped parts can be removed from the plurality of second objects (such as the target object y in FIG. 3 ).

[0136] In addition, the preset non-transportation posture states and preset non-transportation categories can also be removed from multiple second objects. For example, the target object h with an inclined posture and the target object y with a special-shaped part can be removed, and the remaining target object q, target object u, target object d3, and target object d5 are at least partial target objects.

[0137] S511: Determine a transfer order of at least some of the target objects according to the attribute information of the target objects.

[0138] At least some of the target objects are transportable among the multiple target objects. In the present disclosure, at least some of the target objects can be pre-determined as transportable. For example, target objects with a complete integrity level can be transportable, or target objects with a complete or truncated integrity level can be transportable, or target objects with a normal posture can be transportable. In embodiments of the present disclosure, a user can pre-determine preset attribute information. Then, during actual operation, target objects with attribute information that matches the preset attribute information can be determined as at least some of the target objects that can be transported.

[0139] Furthermore, transfer priorities corresponding to attribute information can be preset. For example, sheet-shaped packages have a higher transfer priority than bag-shaped packages, normal-position packages have a higher priority than tilted packages, and higher rectangularity and integrity have higher priorities. In actual operation, the transfer order of at least some of the target objects can be determined based on the attribute information of at least some of the target objects and the transfer priorities corresponding to the attribute information. Target objects ranked higher in the order have higher attribute information corresponding to their priorities and are therefore transferred first.

[0140] For example, referring to FIG3 , if at least some of the target objects are target object h, target object q, target object d3, target object u, and target object d5, and if the preset transfer priorities are set by category, and the preset transfer priorities are, from highest to lowest, sheet packages, bagged packages, box packages, and spherical packages, then the transfer order of target object h, target object u, target object q, target object d3, and target object d5 may be, in order, "target object d3, target object d5, target object h, target object u, target object q." Furthermore, if the preset transfer priorities are such that higher rectangularity indicates higher transfer priority, and if the rectangularity of target object h > the rectangularity of target object d3 > the rectangularity of target object u > the rectangularity of target object d5 > the rectangularity of target object q, then the transfer order may be, in order, "target object h, target object d3, target object u, target object d5, target object q."

[0141] In an optional embodiment, the attribute information also includes: the rectangularity corresponding to the target object, and based on the attribute information, determining the transfer order of at least some of the target objects among the multiple target objects, including: determining the order of the rectangularity of at least some of the target objects from high to low, as the transfer order of at least some of the target objects, wherein the higher the rectangularity of the target object, the higher the corresponding transfer priority.

[0142] In the embodiment of the present disclosure, the higher the rectangularity, the more regular the target object is, and the easier it is to transport. Therefore, the present disclosure prioritizes the transport of target objects with high rectangularity. This method is applicable when at least some target objects are of the same or different categories. Referring to Figure 3, for at least some target objects of different categories (target object d3, target object d5, target object h, target object u, target object q), the order of transport obtained according to the rectangularity is "target object h, target object d3, target object u, target object d5, target object q". Referring to Figure 4, for at least some target objects of the same category (target object h1, target object h3, target object h4), the order of transport obtained according to the rectangularity is "target object h1, target object d3, target object h4".

[0143] In another optional embodiment, for a case where at least part of the target objects are of multiple categories, the transfer order of at least part of the multiple target objects is determined based on the attribute information, including: grouping at least part of the target objects according to category to obtain multiple groups; determining a first transfer priority for each group based on a preset transfer priority corresponding to the category of the group, wherein target objects belonging to the same group have the same first transfer priority; for the target objects in each group, determining a second transfer priority of the target object in the group to which it belongs based on a preset transfer priority corresponding to the rectangularity of the target object, wherein the higher the rectangularity of the target object, the higher the corresponding transfer priority; determining the transfer order of at least part of the target objects based on the first transfer priority and the second transfer priority.

[0144] Among them, referring to Table 4, the preset transfer priority corresponding to the category can be pre-set. For example, in Table 4, the preset transfer priority corresponding to sheet packages is level 1 (highest), the preset transfer priority corresponding to bag packages is level 2, the preset transfer priority corresponding to box packages is level 3, and the preset transfer priority corresponding to spherical packages is level 4 (lowest).

[0145] Table 4

[0146]

[0147] Furthermore, the preset transfer priority corresponding to the rectangularity is that the higher the rectangularity, the higher the preset transfer priority.

[0148] For example, referring to Figure 3 , if at least some of the target objects include target object h, target object d3, target object u, target object d5, and target object q, then at least some of the objects are first grouped. This results in group z1 (target objects h and u), group z2 (target objects d3 and d5), and group z3 (target object q). Then, based on the preset transfer priorities corresponding to the group categories, the first transfer priority of group z2 is determined to be higher than the first transfer priority of group z1, and the first transfer priority of group z1 is higher than the first transfer priority of group z3. Furthermore, in group z1, target objects h and target objects u have the same first transfer priority, and in group z2, target objects d3 and target objects d5 have the same first transfer priority. If, in group z1, the rectangularity of target object h is greater than that of target object u, then in group z1, the second transfer priority of target object h is higher than that of target object u. Similarly, in group z2, the second transfer priority of target object d3 is higher than that of target object d5. Then, according to the first transfer priority and the second transfer priority, the transfer order of at least some of the target objects may be determined to be target object d3, target object d5, target object h, target object u, and target object q.

[0149] The present disclosure can also set the correspondence between attribute information and transfer priority according to needs, which is not limited here.

[0150] The present disclosure determines the transfer order of at least some target objects among multiple target objects through attribute information, so that the transfer device can transfer the target objects on the carrying platform in an orderly manner, thereby improving the transfer order of the target objects.

[0151] In the present disclosure, no transfer is performed on target objects whose preliminary or final categories are preset categories.

[0152] During the target object transshipment process, it can be determined whether to transship the target object based on its category. For example, special-shaped parcels will not be transshipped, while other parcels except special-shaped parcels will be transshipped.

[0153] The present disclosure adopts the above classification method to accurately identify special-shaped parts, so that special-shaped parts are not transported during the transportation process, thereby improving the transportation efficiency.

[0154] S512: Determine the pixel area of ​​the mask object in the mask image.

[0155] In the embodiment of the present disclosure, the pixel area of ​​each mask object in the mask image can be determined based on the mask image. The pixel area can represent the actual area size of the target object. The pixel area and the actual area of ​​the target object are positively correlated.

[0156] S513: Obtain the moving speed of the transfer device for transferring the corresponding target object according to the final category and pixel area.

[0157] In the present disclosure, the moving speed of at least part of the target object is determined, wherein the final category, the pixel area and the moving speed have a preset corresponding relationship.

[0158] Exemplarily, referring to Table 5, there is a preset correspondence between the final category, the pixel area, and the moving speed.

[0159] Table 5

[0160]

[0161] In summary, the present disclosure can pre-set the corresponding relationship between the final category, pixel area and movement speed. Then, after determining the final category and pixel area of ​​the target object, the movement speed can be determined based on the corresponding relationship.

[0162] S514: Determine the placement height of the target object.

[0163] Among them, determining the placement height of the target object includes: obtaining a first posture of the carrying platform, and the transfer device is used to transfer the target object placed on the carrying platform to the target placement position; obtaining a second posture of the target object when it is on the carrying platform; and determining the placement height according to the first posture and the second posture.

[0164] In the present disclosure, the position of the supporting platform is pre-set, so the first pose of the supporting platform, shown as P1 in Figure 1, can be directly obtained. If the gripping device is as shown in Figure 4, the first pose is the pose of the supporting platform in the coordinate system of base 41. Furthermore, the pose of the target object in the coordinate system of base 41 can be determined as the second pose. The height H of the target object can then be determined based on the first and second poses (see Figure 1). This height H is the placement height.

[0165] S515: Determine a position that is a height above the reference placement position as the target placement position.

[0166] In this disclosure, the reference placement position is pre-set, and its position is known. Referring to Figure 1 , position F1 on platform P2 is the reference placement position. The position at a height H above reference placement position F1 is then determined as target placement position F2.

[0167] S516: Control the transfer device to transfer the corresponding target object to the target placement position according to the moving speed.

[0168] Specifically, the transfer device is controlled to transfer at least part of the target objects according to the transfer sequence, and when transferring the target objects among at least part of the target objects, the target objects are transferred according to the moving speed so that the target objects are transferred to the target placement position.

[0169] For example, in Figure 3, at least some of the target objects are target object d3, target object d5, target object h, target object u, and target object q. Furthermore, the determined transfer order is target object d3, target object d5, target object h, target object u, and target object q. Furthermore, if the speed of target object d3 is determined to be V1, the speed of target object d5 is determined to be V2, the speed of target object h is determined to be V3, the speed of target object u is determined to be V4, and the speed of target object q is determined to be V5, then during the transfer process, target object d3 is first transferred at speed V1, then target object d5 is transferred at speed V2, then target object h is transferred at speed V3, then target object u is transferred at speed V5, and finally target object q is transferred at speed V5.

[0170] The transfer device transfers the target object to the target placement position, and can directly release the target object to allow it to freely fall to the reference placement position F1, or can slowly drop the target object to place it at the reference placement position F1.

[0171] It is understood that determining the target placement position can prevent the target object from falling too high to the reference placement position during transportation, thereby damaging the target object. In addition, the target object can also be accurately transported to the reference placement position, thereby improving transportation efficiency.

[0172] In the present disclosure, a recognition model can be used to accurately identify the attribute information of the target object, and then accurately determine at least part of the target objects that can be transported among multiple target objects, and then the transport order of the at least part of the target objects can be determined. After determining the transport order, the target object can be transported according to the transport order and moving speed of the target object, so as to avoid damage to the target object during the transport process and further improve the transport efficiency of the target object.

[0173] 12 , in addition to providing a transfer method, the present disclosure also provides a transfer device 120 , which includes:

[0174] An acquisition module 121 is configured to acquire a target image, where the target image includes a plurality of target objects;

[0175] Recognition module 122, used to recognize the target image and obtain attribute information of each target object;

[0176] A determination module 123 is used to determine the moving speed of the transfer device for transferring the corresponding target object based on the attribute information, where the attribute information and the moving speed have a preset corresponding relationship;

[0177] The control module 124 is used to control the transfer device to transfer the corresponding target object according to the moving speed.

[0178] In an optional embodiment, the transfer device is a suction cup device, and the control module 124 is specifically used to: determine the air flow rate of the suction cup device based on the attribute information, and the attribute information and the air flow rate have a preset correspondence; control the suction cup device to absorb the target object according to the air flow rate, and transfer the absorbed target object according to the moving speed.

[0179] In an optional embodiment, identifying a target image includes identifying the target image through a recognition model; the recognition model includes: a backbone feature extraction module, an enhanced feature extraction module and an attribute determination module; the attribute information includes basic attributes, and the recognition module 122 is specifically used to perform feature extraction processing on the target image through the backbone feature extraction module to obtain a first feature image; perform feature enhancement processing on the first feature image through the enhanced feature extraction module to obtain a second feature image; perform attribute analysis processing on the second feature image through the attribute determination module to obtain basic attributes of the target object and probability values ​​of the target object in different categories.

[0180] In an optional embodiment, the attribute information also includes the final category of the target object, and the recognition module 122 is further used to: determine the category corresponding to the maximum probability value among the probability values ​​of different categories as the preliminary category; if the preliminary category does not belong to the preset category, determine the final category of the target object based on the mask object, and the final category has a preset correspondence with the moving speed.

[0181] In an optional embodiment, if the preliminary category does not belong to the preset category, the recognition module 122, when determining the final category of the target object based on the mask object, is specifically used to: if the preliminary category does not belong to the preset category and the probability value under the preliminary category is less than the first threshold, determine the final category of the target object based on the mask object; if the preliminary category does not belong to the preset category and the probability value under the preliminary category is greater than or equal to the first threshold, determine that the final category of the target object is the preliminary category.

[0182] In an optional embodiment, when the recognition module 122 determines the final category of the target object based on the mask object, it is specifically used to: determine the three-dimensional structure of the target object based on the mask object, where the mask object includes depth information of the target object; if the three-dimensional structure belongs to a preset three-dimensional structure of a preset category, then determine the final category of the target object as the preset category; otherwise, determine the final category of the target object as the preliminary category.

[0183] In an optional embodiment, the preset category is special-shaped parts packaging.

[0184] In an optional embodiment, the transfer device is further configured to: not transfer target objects belonging to a preset category.

[0185] In an optional embodiment, the control module 124 is specifically used to: determine the transfer order of at least some of the target objects among multiple target objects based on the attribute information of the target objects; control the transfer device to transfer at least some of the target objects according to the transfer order, and when transferring the target objects among at least some of the target objects, transfer the target objects according to the moving speed.

[0186] In an optional embodiment, the basic attributes include: degree of completeness, the determination module 123 is also used to, before determining the transfer order of at least some of the target objects among the multiple target objects based on the attribute information of the target objects, further used to: remove the first object from the multiple target objects based on the degree of completeness of the multiple target objects to obtain multiple second objects, and the degree of completeness of the first object is incomplete; determine at least some of the target objects based on the multiple second objects.

[0187] In an optional embodiment, the basic attributes also include: posture state; when the determination module 123 determines at least part of the target objects based on multiple second objects, it is specifically used to: if the posture state of the third object among the multiple second objects is a preset non-transport posture state, then remove the third object from the multiple second objects to obtain at least part of the target objects.

[0188] In an optional embodiment, the attribute information also includes: a final category. If the multiple second objects can be divided into at least two final categories, the determination module 123 is specifically used to determine at least part of the target objects based on the multiple second objects: if the final category of the fourth object among the multiple second objects is a preset non-transfer category, then the fourth object is removed from the multiple second objects to obtain at least part of the target objects.

[0189] In an optional embodiment, the attribute information further includes: rectangularity corresponding to the target object. When determining the transfer order of at least some of the target objects according to the attribute information of the target object, the determination module 123 is specifically configured to:

[0190] The order of the rectangularity of at least some of the target objects is determined from high to low as the transfer order of at least some of the target objects, wherein the higher the rectangularity of the target object, the higher the corresponding transfer priority.

[0191] In an optional embodiment, when determining the transfer order of at least some of the target objects according to the attribute information of the target objects, the determination module 123 is specifically configured to:

[0192] Grouping at least some of the target objects according to the final categories to obtain a plurality of groups;

[0193] Determine a first transfer priority for each group based on the preset transfer priority corresponding to the final category of the group, wherein target objects belonging to the same group have the same first transfer priority;

[0194] For the target object in each group, the second transfer priority of the target object in the group to which it belongs is determined according to the preset transfer priority corresponding to the rectangularity of the target object, wherein the higher the rectangularity of the target object, the higher the corresponding transfer priority;

[0195] A transfer order of at least part of the target objects is determined according to the first transfer priority and the second transfer priority.

[0196] In an optional embodiment, the attribute information further includes: a mask object of the target object in a mask image corresponding to the target image; the recognition model further includes a mask determination module; and the recognition module 122, after performing feature enhancement processing on the first feature image by the enhanced feature extraction module to obtain a second feature image, is further configured to:

[0197] Performing mask analysis processing on the second feature image by a mask determination module to obtain a mask image;

[0198] A mask object of the target object is determined according to the mask image.

[0199] In an optional embodiment, when the enhanced feature extraction module performs feature enhancement processing on the first feature image, a second feature image is obtained; the mask determination module includes: a first convolution unit and a first decoder, and the recognition module 122 performs mask analysis processing on the second feature image through the mask determination module to obtain a mask image, specifically for:

[0200] performing convolution processing on the second feature image through the first convolution unit to obtain a third feature image;

[0201] The third feature image is decoded by the first decoder to obtain a mask image.

[0202] In an optional embodiment, when the enhanced feature extraction module performs feature enhancement processing on the first feature image, multiple second feature images of different sizes are obtained; the mask determination module further includes: multiple second convolution units, a splicing unit, and a second decoder, and the convolution kernels of the multiple second convolution units have different sizes. The recognition module 122 performs mask analysis processing on the second feature image through the mask determination module to obtain the mask image, which is specifically used to:

[0203] Performing convolution processing on the second feature images one by one through the second convolution unit to obtain a plurality of fourth feature images of the same size;

[0204] performing a splicing process on the plurality of fourth feature images having the same size by a splicing unit to obtain a fifth feature image;

[0205] The fifth feature image is decoded by the second decoder to obtain a mask image.

[0206] In an optional embodiment, the determining module 123 is specifically configured to:

[0207] determining the pixel area of ​​the mask object in the mask image;

[0208] According to the final category and pixel area, the moving speed of the transfer device in transferring the corresponding target object is obtained. The final category, pixel area and moving speed have a preset corresponding relationship.

[0209] In an optional embodiment, the target object includes: an express package; the category of the target object includes: a box-shaped package, a cylindrical package, a spherical package, a bag-shaped package, a sheet-shaped package or an irregular-shaped package.

[0210] In an optional embodiment, the air flow rate corresponding to the sheet-shaped package is lower than the air flow rate corresponding to the box-shaped package.

[0211] In an optional embodiment, the basic attributes further include: at least one of: degree of completeness, posture state, color feature, and texture feature.

[0212] In an optional embodiment, before controlling the transfer device to transfer the corresponding target object according to the moving speed, the determination module 123 is further configured to: determine the placement height of the target object; obtain a preset reference placement position; determine a position that is the placement height above the reference placement position as the target placement position;

[0213] The control module 124 is specifically used to control the transfer device to transfer the corresponding target object to the target placement position according to the moving speed.

[0214] In an optional embodiment, the determination module, when determining the placement height of the target object, is specifically used to: obtain the first posture of the carrying platform, and the transfer device is used to transfer the target object placed on the carrying platform to the target placement position; obtain the second posture of the target object when it is on the carrying platform; and determine the placement height based on the first posture and the second posture.

[0215] The transfer method provided by the present disclosure can control the transfer device to transfer target objects with different attribute information at a corresponding moving speed, thereby improving the efficiency of transfer.

[0216] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or in parallel. They are only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0217] FIG13 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. As shown in FIG13 , the electronic device 130 includes: a processor 131 and a memory 132 communicatively connected to the processor 131 , wherein the memory 132 stores computer-executable instructions.

[0218] Among them, the processor executes the computer-executable instructions stored in the memory to implement the transfer method provided by any of the above-mentioned method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0219] An embodiment of the present disclosure further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the transport method provided by any of the above method embodiments.

[0220] An embodiment of the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the transfer method provided by any of the above method embodiments.

[0221] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0222] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0223] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0224] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present disclosure. The aforementioned storage medium includes: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0225] Those skilled in the art will clearly understand that for the sake of convenience and brevity, the division of the above-mentioned functional modules is only used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0226] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0227] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A transport method, applied to a transport device, characterized in that: include: Acquiring a target image, wherein the target image includes a plurality of target objects; Identify the target image and obtain attribute information of each target object; Determine, based on the attribute information, a moving speed of the transfer device for transferring the corresponding target object, wherein the attribute information and the moving speed have a preset corresponding relationship; The transfer device is controlled to transfer the corresponding target object according to the moving speed.

2. The transport method according to claim 1, characterized in that: The transfer device is a suction cup device, and the controlling the transfer device to transfer the corresponding target object according to the moving speed includes: Determine the air flow rate of the suction cup device according to the attribute information, wherein the attribute information and the air flow rate have a preset corresponding relationship; The suction cup device is controlled to suck the target object according to the air flow rate, and the sucked target object is transported according to the moving speed.

3. The transport method according to claim 1 or 2, characterized in that: The identifying the target image includes identifying the target image through a recognition model; the recognition model includes: a backbone feature extraction module, an enhanced feature extraction module and an attribute determination module; the attribute information includes basic attributes, and the identifying the target image to obtain the attribute information of each target object includes: Performing feature extraction processing on the target image by the backbone feature extraction module to obtain a first feature image; Performing feature enhancement processing on the first feature image by the enhanced feature extraction module to obtain a second feature image; The attribute determination module performs attribute analysis on the second feature image to obtain basic attributes of the target object and probability values ​​of the target object in different categories.

4. The transport method according to claim 3, characterized in that: The attribute information also includes the final category of the target object, and the identifying the target image to obtain the attribute information of each target object also includes: Determine the category corresponding to the maximum probability value among the probability values ​​of different categories as the preliminary category; If the preliminary category does not belong to the preset category, the final category of the target object is determined according to the mask object, and the final category has a preset corresponding relationship with the moving speed.

5. The transport method according to claim 4, characterized in that: If the preliminary category does not belong to the preset category, determining the final category of the target object according to the mask object includes: If the preliminary category does not belong to the preset category and the probability value under the preliminary category is less than a first threshold, determining the final category of the target object according to the mask object; If the preliminary category does not belong to the preset category, and the probability value under the preliminary category is greater than or equal to the first threshold, the final category of the target object is determined to be the preliminary category.

6. The transport method according to claim 5, characterized in that: The determining a final category of the target object according to the mask object comprises: determining a three-dimensional structure of the target object according to the mask object, wherein the mask object includes depth information of the target object; If the three-dimensional structure belongs to a preset three-dimensional structure of the preset category, determining that the final category of the target object is the preset category; Otherwise, the final category of the target object is determined to be the preliminary category.

7. The transport method according to claim 6, characterized in that: The preset category is special-shaped parts packaging.

8. The transport method according to claim 7, characterized in that: Also includes: No transfer will be implemented for target objects belonging to the preset categories.

9. The method for transporting according to claim 4, characterized in that: The controlling the transfer device to transfer the corresponding target object according to the moving speed includes: Determining a transfer order of at least some of the target objects among the multiple target objects according to the attribute information of the target objects; The transfer device is controlled to transfer at least part of the target objects according to the transfer sequence, and when transferring the target objects among the at least part of the target objects, the target objects are transferred according to the moving speed.

10. The transport method according to claim 9, characterized in that: The basic attributes include: completeness. Before determining the transfer order of at least some of the target objects among the multiple target objects according to the attribute information of the target objects, the method further includes: According to the completeness of the multiple target objects, removing a first object from the multiple target objects to obtain multiple second objects, wherein the completeness of the first object is incomplete; The at least part of the target object is determined according to the multiple second objects.

11. The method for transporting according to claim 10, characterized in that: The basic attributes further include: posture state; and determining at least part of the target object according to the plurality of second objects includes: If the posture state of a third object among the multiple second objects is a preset non-transport posture state, the third object is removed from the multiple second objects to obtain the at least partial target object.

12. The method for transporting according to claim 10, characterized in that: The attribute information further includes: a final category. If the plurality of second objects can be divided into at least two final categories, then determining the at least part of the target objects according to the plurality of second objects includes: If the final category of the fourth object among the plurality of second objects is the preset non-transfer category, the fourth object is removed from the plurality of second objects to obtain the at least part of the target object.

13. The transport method according to any one of claims 10 to 12, characterized in that: The attribute information further includes: a rectangular degree corresponding to the target object; and determining the transfer order of at least some of the target objects among the multiple target objects according to the attribute information of the target object includes: The order of the rectangularity of at least part of the target objects from high to low is determined as the transfer order of at least part of the target objects, wherein the higher the rectangularity of the target object, the higher the corresponding transfer priority.

14. The method for transporting according to claim 12, characterized in that: The step of determining the transfer order of at least some of the target objects according to the attribute information of the target objects comprises: Grouping at least part of the target objects according to the final categories to obtain a plurality of groups; Determine the first transfer priority of each group according to the preset transfer priority corresponding to the final category of the group, wherein target objects belonging to the same group have the same first transfer priority; For the target object in each group, according to the preset transfer priority corresponding to the rectangularity of the target object, determine the second transfer priority of the target object in the group to which it belongs, wherein the higher the rectangularity of the target object, the higher the corresponding transfer priority; The transfer order of at least part of the target objects is determined according to the first transfer priority and the second transfer priority.

15. The method for transporting according to claim 4, characterized in that: The attribute information further includes: a mask object of the target object in a mask image corresponding to the target image; the recognition model further includes a mask determination module; and after the enhanced feature extraction module performs feature enhancement processing on the first feature image to obtain a second feature image, the method further includes: Performing mask analysis processing on the second feature image by the mask determination module to obtain the mask image; A mask object of the target object is determined according to the mask image.

16. The method for transporting according to claim 15, characterized in that: In the enhanced feature extraction module performing feature enhancement processing on the first feature image, a second feature image is obtained; The mask determination module includes: a first convolution unit and a first decoder, and the mask determination module performs mask analysis processing on the second feature image to obtain the mask image, including: Performing convolution processing on the second feature image by the first convolution unit to obtain a third feature image; The third feature image is decoded by the first decoder to obtain the mask image.

17. The method for transporting according to claim 16, characterized in that: In the enhanced feature extraction module performing feature enhancement processing on the first feature image, a plurality of second feature images of different sizes are obtained; The mask determination module further includes: a plurality of second convolution units, a splicing unit, and a second decoder, wherein the convolution kernels of the plurality of second convolution units have different sizes, and the mask determination module performs mask analysis processing on the second feature image to obtain the mask image, including: Performing convolution processing on the second feature images one by one through the second convolution unit to obtain a plurality of fourth feature images of the same size; Performing a splicing process on the plurality of fourth feature images having the same size by the splicing unit to obtain a fifth feature image; The fifth feature image is decoded by the second decoder to obtain the mask image.

18. The method for transporting according to claim 15, characterized in that: Determining the moving speed of the transfer device for transferring the corresponding target object according to the attribute information includes: determining a pixel area of ​​the mask object in the mask image; The moving speed of the transport device for transporting the corresponding target object is obtained according to the final category and the pixel area, and the final category, the pixel area and the moving speed have a preset corresponding relationship.

19. The method for transporting according to claim 3, characterized in that: The target object includes: express parcels; the category of the target object includes: one of: box-shaped parcels, columnar parcels, spherical parcels, bag-shaped parcels, sheet-shaped parcels or special-shaped parcels.

20. The method for transporting according to claim 19, characterized in that: The air flow rate corresponding to the sheet-shaped package is lower than that corresponding to the box-shaped package.

21. The method for transporting according to claim 3, characterized in that: The basic attributes also include: at least one of completeness, posture status, color characteristics and texture characteristics.

22. The transport method according to claim 1 or 2, characterized in that: Before controlling the transfer device to transfer the corresponding target object according to the moving speed, the method further includes: Determining a placement height of the target object; Get the preset reference placement position; Determine the position of the reference placement position upward by the placement height as the target placement position; The controlling the transfer device to transfer the corresponding target object according to the moving speed includes: The transfer device is controlled to transfer the corresponding target object to the target placement position according to the moving speed.

23. The method for transporting according to claim 22, characterized in that: The step of determining the placement height of the target object includes: Acquire the first posture of the carrying platform, the transfer device is used to transfer the target object placed on the carrying platform to the target placement position; Acquire a second posture of the target object on the carrying platform; The placement height is determined according to the first posture and the second posture.

24. A transfer device, characterized in that: include: An acquisition module, used for acquiring a target image, wherein the target image includes a plurality of target objects; A recognition module, used to recognize the target image and obtain attribute information of each target object; A determination module, used to determine the moving speed of the transfer device for transferring the corresponding target object according to the attribute information, wherein the attribute information and the moving speed have a preset corresponding relationship; The control module is used to control the transfer device to transfer the corresponding target object according to the moving speed.

25. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the transport method according to any one of claims 1 to 23 is implemented.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the transport method according to any one of claims 1 to 23.

Citation Information

Patent Citations

  • Parcel sorting method and device and computer readable storage medium

    CN109513629A

  • Robot multi-gripper assembly and method for gripping and holding objects

    CN111993448A

  • Mask R-CNN mineral particle identification and particle size detection method based on improved mask

    CN114897816A

  • Product sorting method and system

    CN115345556A

  • Classification method and device, transfer method and device and electronic equipment

    CN116563834A