Tray collecting method and apparatus, electronic device, storage medium, and program

Abstract

To relate to the field of computer and automatic control, and in particular to a tray collecting method and apparatus, an electronic device, a storage medium, and a program.SOLUTION: A method includes: after a robot grabbing M yarn spindles to be packaged from a yarn spindle trolley through N grippers, controlling an image collection device to capture images towards the N grippers to obtain an image that represents a grabbing result; and after the robot placing the M yarn spindles to be packaged one by one on M yarn spindle trays included in a target tray group arranged on a main line of an assembly line, and when determining that there is a tray to be collected in the target tray group based on the image that represents the grabbing result, controlling a tray diversion device to divert the tray to be collected from the main line of the assembly line to a bypass line of the assembly line to execute collection of the tray to be collected.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to the field of computers and automation control, and particularly to a tray recovery method, apparatus, electronic device, storage medium, and program.

Background Art

[0002] The packaging of winding packages is an important part of the yarn manufacturing process. During the packaging process of winding packages, due to reasons such as manual picking and forgetting, there may be some trays to be recovered among a plurality of winding package trays on the production line. Currently, in order to avoid these trays to be recovered from affecting the progress of winding package packaging, usually, a recovery process is performed by a manual recovery method.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, the manual recovery method not only wastes a large amount of human resources, but also affects the recovery efficiency of the trays to be recovered.

[0004] The present disclosure provides a tray recovery method, apparatus, electronic device, storage medium, and program to solve or alleviate one or more technical problems in the prior art.

Means for Solving the Problems

[0005] According to a first aspect of the present disclosure, there is provided a tray recovery method applicable to an electronic device included in a winding package packaging system, where the electronic device communicates with an image acquisition device and a tray diversion device respectively, The tray recovery method includes After the robot grips M packaged winding packages from the winding package cart by N hand grips, the image acquisition device is controlled to capture an image in the direction where the N hand grips are located, and an image representing the gripping result is obtained. Here, each of the N hand grips can grip one packaged winding package, N ≥ 2, and N is an integer, 0 ≤ M ≤ N, and M is an integer. After the robot places the M packaged winding packages one-to-one on the M winding package trays included in the target tray group installed on the main line of the production line, when it is determined based on the image representing the gripping result that there is a tray to be recovered in the target tray group, the tray diversion device is controlled to divert the tray to be recovered from the main line of the production line to the bypass line of the production line, thereby performing the recovery process of the tray to be recovered. Here, the tray to be recovered is an empty first recovery target tray or a second recovery target tray initially evaluated that the placed packaged winding package is a predetermined grade winding package.

[0006] According to the second aspect of the present disclosure, a tray recovery device applied to an electronic device included in a winding package packaging system is provided, and the electronic device communicates with an image acquisition device and a tray diversion device respectively. The tray recovery device includes an image acquisition unit for obtaining an image representing a gripping result by controlling the image acquisition device to capture an image in the direction where the N hand grips are located after the robot grips M packaged winding packages from the winding package cart by N hand grips. Here, each of the N hand grips can grip one packaged winding package, N ≥ 2, and N is an integer, 0 ≤ M ≤ N, and M is an integer. A recovery control unit for executing a recovery process of a tray to be recovered by controlling a tray diversion device to divert the tray to be recovered from the main line of the production line to the bypass line of the production line, after a robot places M packaged winding packages one-to-one on M winding package trays included in a target tray group installed on the main line of the production line, and when it is determined based on an image representing the gripping result that there is a tray to be recovered in the target tray group, where the tray to be recovered is an empty first tray to be recovered or a second tray to be recovered that is preliminarily evaluated as having the packaged winding package placed thereon being a predetermined grade winding package, and the recovery control unit is provided.

[0007] According to a third aspect of the present disclosure, an electronic device is provided, and the device includes at least one processor; a memory communicatively connected to the at least one processor, and the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, they cause any one of the methods in the embodiments of the present disclosure to be executed. The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, they cause any one of the methods in the embodiments of the present disclosure to be executed.

[0008] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute any one of the methods in the embodiments of the present disclosure is provided.

[0009] According to a fifth aspect of the present disclosure, a program is provided, and when the program is executed by a processor, it realizes any one of the methods in the embodiments of the present disclosure.

[0010] In the technical solution provided by the present disclosure, after the robot grips M winding packages to be packaged from a winding package trolley by N hand grips, the image acquisition device is controlled to capture an image in the direction where the N hand grips are located, so as to obtain an image representing the gripping result. After the robot places the M winding packages to be packaged one-to-one on M winding package trays included in a target tray group installed on the main line of the production line, if it is determined based on the image representing the gripping result that there is a tray to be recovered in the target tray group, the tray diversion device is controlled to divert the tray to be recovered from the main line of the production line to the bypass line of the production line, thereby performing the recovery process of the tray to be recovered. Thereby, automatic recovery of the tray to be recovered can be realized, a large amount of human resources can be saved compared with the current manual recovery method, and the recovery efficiency of the tray to be recovered can be improved.

[0011] It should be understood that the content described herein is not intended to describe the key points or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. For other features of the present disclosure, understanding is promoted through the following specification.

Brief Description of the Drawings

[0012] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent by referring to the following detailed description in connection with the drawings. In the drawings, the same or similar reference numerals represent the same or similar elements.

[0013]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 2F

Figure 2G

Figure 2H

Figure 2I

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 3E

Figure 3F

Figure 3G

Figure 3H

Figure 4A

Figure 4B

Figure 4C

Figure 4D

Figure 4E

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, the present disclosure will be described in more detail with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar elements. Also, in the accompanying drawings, various aspects of the embodiments are shown, but these accompanying drawings are not necessarily drawn to scale unless otherwise specified.

[0015] Furthermore, in order to better explain the present disclosure, many specific details are described in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be implemented similarly even without some details. In some examples, methods, means, components, circuits, etc. well known to those skilled in the art are not described in detail so that the gist of the present disclosure can be made clear.

[0016] As described above, the packaging of the wound yarn package is an important part of the yarn manufacturing process. During the packaging process of the wound yarn package, due to reasons such as manual picking and forgetting, there may be several trays to be recycled, such as empty wound yarn package trays or other wound yarn package trays that need to be recycled, among multiple wound yarn package trays on the production line. Currently, in order to avoid these trays to be recycled from affecting the progress of the wound yarn package packaging, usually, the recycling process is carried out by a manual recycling method. However, the manual recycling method not only wastes a large amount of human resources but also affects the recycling efficiency of the trays to be recycled.

[0017] In order to save a large amount of human resources and improve the recycling efficiency of the trays to be recycled, embodiments of the present disclosure provide a tray recycling method applied to an electronic device included in a wound yarn package packaging system, and the electronic device communicates with an image collection device and a tray diversion device respectively. Here, the electronic device may be a computer, a programmable logic controller (PLC), etc., the image collection device may be an industrial camera, and the tray diversion device may be an aerodynamic diversion device, an electric diversion device, a drum type diversion device, etc., and the embodiments of the present disclosure are not limited thereto.

[0018] Furthermore, in the embodiments of the present disclosure, the main types of wound package products can include one or more of partially oriented yarns (POY), fully drawn yarns (FDY), draw textured yarns (DTY) (or low-elastic filaments), etc. For example, the types of wound package products specifically include polyester partially oriented yarns, polyester fully drawn yarns, polyester drawn yarns, polyester draw textured yarns, etc.

[0019] FIG. 1 is a flowchart of a tray recovery method according to an embodiment of the present disclosure. Hereinafter, the tray recovery method provided by the embodiment of the present disclosure will be described with reference to FIG. 1. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order.

[0020] In step S101, after the robot grips M wound package to be packaged from the wound package trolley by N hand grips, the image acquisition device is controlled to take an image in the direction where the N hand grips are located, and an image representing the gripping result is obtained.

[0021] Here, the robot may be an N-axis industrial robot, that is, the robot includes N hand grips, and each hand grip among the N hand grips can grip one wound package to be packaged. Here, N≥2 and N is an integer.

[0022] In an embodiment of the present disclosure, when a robot grips a winding package to be packaged from a winding package cart by N hand grips, it can be understood that some hand grips may not be able to successfully grip the winding package to be packaged. Therefore, in the embodiment of the present disclosure, 0≤M≤N, and M is an integer. Specifically, when a robot grips a winding package to be packaged from a winding package cart by N hand grips, if all hand grips succeed in gripping the winding package to be packaged, M = N. On the other hand, if some hand grips cannot successfully grip the winding package to be packaged, 0≤M<N, and M is an integer.

[0023] In step S102, after the robot places M winding packages to be packaged one-to-one on M winding package trays included in a target tray group installed on the main line of the production line, if it is determined based on an image representing the gripping result that there is a tray to be recovered in the target tray group, the tray diversion device is controlled to divert the tray to be recovered from the main line of the production line to the bypass line of the production line, thereby performing the recovery process of the tray to be recovered.

[0024] Here, the tray to be recovered is an empty first tray to be recovered or a second tray to be recovered in which the placed winding package to be packaged is initially evaluated as a winding package of a predetermined grade product. Here, the winding package of the predetermined grade product may be a winding package product of a non-AA grade, that is, a winding package product downgraded from the default AA grade to another grade (for example, A grade, B grade, C grade) for quality reasons. The quality reasons may be the existence of causes such as thread breakage, oil stain, thread hanging, poor forming, and paper tube breakage.

[0025] In the embodiments of the present disclosure, the target tray group usually includes a fixed number of winding package trays. For example, the target tray group includes a total of N winding package trays, and the N winding package trays may correspond one-to-one with N hand grips. Therefore, when the robot grips the winding package to be packaged from the winding package carriage by N hand grips, if some hand grips cannot successfully grip the winding package to be packaged (in this case, 0 ≦ M < N), when the robot places M winding packages to be packaged one-to-one on M winding package pallets included in the target tray group installed on the main line of the production line, there will be some empty first recovery pallets in the target tray group. Also, in the embodiments of the present disclosure, among the M winding packages to be packaged gripped by the robot from the winding package cart by N hand grips, there may also be a second recovery target tray initially evaluated as a winding package of a predetermined grade.

[0026] According to the tray recovery method provided by the embodiments of the present disclosure, after the robot grips M winding packages to be packaged from the winding package carriage by N hand grips, the image acquisition device is controlled to capture an image in the direction where the N hand grips are located to obtain an image representing the gripping result. After the robot places the M winding packages to be packaged one-to-one on M winding package trays included in the target tray group installed on the main line of the production line, if it is determined based on the image representing the gripping result that there is a recovery target tray in the target tray group, the tray diversion device is controlled to divert the recovery target tray from the main line of the production line to the bypass line of the production line, thereby performing the recovery process of the recovery target tray. Thereby, the automatic recovery of the recovery target tray can be realized, a large amount of human resources can be saved compared with the current manual recovery method, and the recovery efficiency of the recovery target tray can be improved.

[0027] In some optional embodiments, the electronic device further communicates with a robot, and "controlling the image acquisition device to capture images in the direction where the N hand grips are located and obtaining an image representing the gripping result" in step S101 may include the following steps.

[0028] In step S1011, control the N hand grips to rotate with respect to the image acquisition device.

[0029] In one example, the N hand grips of the robot can be connected to the main body structure of the robot via the same mechanical axis. Based on this, in the embodiments of the present disclosure, the mechanical axis can be rotated so that the N hand grips rotate as a whole with respect to the image acquisition device.

[0030] In step S1012, during the process that the N hand grips rotate with respect to the image acquisition device, control the image acquisition device to capture images in the direction where the N hand grips are located and obtain Z target images to be processed.

[0031] Here, Z≥2 and Z is an integer.

[0032] The image acquisition device captures images in a certain direction of the N hand grips during the process that the N hand grips rotate with respect to the image acquisition device and obtains Z target images to be processed. Therefore, the Z target images correspond to different viewing angles of the N hand grips.

[0033] In step S1013, based on the Z target images to be processed, obtain an image representing the gripping result.

[0034] In one example, the Z target images to be processed can be processed by operations such as decomposition and stitching to obtain an image representing the gripping result.

[0035] Referring to FIG. 2A, illustratively, the robot 201 can include five hand grips (the first hand grip 2021, the second hand grip 2022, the third hand grip 2023, the fourth hand grip 2024, and the fifth hand grip 2025 (i.e., N = 5 in this example)). Due to reasons such as manual picking and forgetting, the winding package trolley 203 can only include a group of winding packages to be packaged with four winding packages to be packaged (the first winding package to be packaged 2041, the third winding package to be packaged 2043, the fourth winding package to be packaged 2044, and the fifth winding package to be packaged 2045). Further referring to FIG. 2B, when the robot 201 grips the winding packages to be packaged from the winding package trolley 203 with five hand grips, the first hand grip 2021 successfully grips the first winding package to be packaged 2041, the second hand grip 2022 successfully grips the winding package to be packaged, the third hand grip 2023 fails to grip the third winding package to be packaged 2043, the fourth hand grip 2024 successfully grips the fourth winding package to be packaged 2044, and the fifth hand grip 2025 successfully grips the fifth winding package to be packaged 2045. After the robot 201 grips four winding packages to be packaged from the winding package trolley 203 with five hand grips, in order to rotate the five hand grips as a whole with respect to the image acquisition device 206, the mechanical shaft 205 for connecting the five hand grips to the main body structure of the robot 201 can be rotated. Specifically, the mechanical shaft 205 can be rotated along the direction of "A1→A2", and the five hand grips can be rotated as a whole with respect to the image acquisition device 206. During the process of the five hand grips rotating with respect to the image acquisition device 206, the image acquisition device 206 is controlled to image in the direction where the five hand grips are located, Z processed target images are obtained, and it can be ensured that the Z processed target images correspond to different viewing angles of the N hand grips.

[0036] Taking Z = 2 as an example, when the five handgrips are rotated to the first angle as shown in FIG. 2B with respect to the image acquisition device 206, the image acquisition device 206 is controlled to image in the direction where the five handgrips are located, and a first processing target image can be obtained. The first processing target image is mainly used to represent the image features on the first side surface of the four winding packages to be packaged. That is, as shown in FIG. 2C, the first processing target image includes a single-angle image SD11 for representing the image features on the first side surface of the first winding package 2041 to be packaged, a single-angle image SD31 for representing the image features on the first side surface of the third winding package 2043 to be packaged, a single-angle image SD41 for representing the image features on the first side surface of the fourth winding package 2044 to be packaged, and a single-angle image SD51 for representing the image features on the first side surface of the fifth winding package 2045 to be packaged. In addition, since the second handgrip 2022 has not succeeded in gripping the winding package to be packaged, the second handgrip 2022 is not shielded by the winding package to be packaged. Thus, the first processing target image can include a single-angle image SZ21 for representing the image features on the first side surface of the second handgrip 2022.

[0037] Similarly, when the five handgrips are rotated to a second angle with respect to the image acquisition device 206 as shown in FIG. 2D, the image acquisition device 206 is controlled to image in the direction where the five handgrips are located, and a second processing target image is obtained. The second processing target image is mainly used to represent the image features on the second side of the four winding packages to be packaged, which is opposite to the first side. That is, specifically as shown in FIG. 2E, the second processing target image includes a single image angle image SD12 for representing the image features on the second side of the first winding package 2041 to be packaged, a single image angle image SD32 for representing the image features on the second side of the third winding package 2043 to be packaged, a single image angle image SD42 for representing the image features on the second side of the fourth winding package 2044 to be packaged, and a single image angle image SD52 for representing the image features on the second side of the fifth winding package 2045 to be packaged. Also, since the second handgrip 2022 has not succeeded in gripping the winding package to be packaged, the second handgrip 2022 is not shielded by the winding package to be packaged. Thus, the second processing target image can include a single image angle image SZ22 for representing the image features on the second side of the second handgrip 2022.

[0038] Also, in the embodiments of the present disclosure, it is necessary to explain that both the first processing target image shown in FIG. 2B and the second processing target image shown in FIG. 2D are preprocessed. Here, the preprocessing can include trimming processing, noise removal processing, feature enhancement, etc., and the embodiments of the present disclosure are not limited thereto.

[0039] After obtaining the first processing target image and the second processing target image, by operations such as decomposition and splicing, the Z processing target images can be processed to obtain an image representing the gripping result.

[0040] Through the steps included in step S101, in the embodiments of the present disclosure, after the robot grasps M winding packages to be packaged from the winding package cart by N hand grips, the N hand grips are controlled to rotate with respect to the image acquisition device. During the process of the N hand grips rotating with respect to the image acquisition device, the image acquisition device is controlled to capture images in the direction where the N hand grips are located, and Z images to be processed (the Z images to be processed correspond to different viewing angles of the N hand grips) are obtained. Based on the Z images to be processed, a feature image representing the grasping result can be obtained. In this way, the image representing the grasping result comprehensively represents the M winding packages to be packaged grasped by the N hand grips, avoids omission of features, thereby reducing the misjudgment rate of the tray to be recovered, and improving the accuracy of tray recovery.

[0041] Furthermore, in some optional embodiments, step S1013 may include the following steps.

[0042] In step S10131, N single-view-angle images are cut out from each image to be processed among the Z images to be processed, and N×Z single-view-angle images are obtained.

[0043] Here, the N single-view-angle images correspond one-to-one to the N hand grips.

[0044] In step S10132, the Z single-view-angle images corresponding to the same hand grip among the N×Z single-view-angle images are combined to obtain N multi-view-angle images.

[0045] In step S10133, based on the N multi-view-angle images, an image representing the grasping result is obtained.

[0046] In one example, the N multi-view-angle images can be arranged in a tile pattern according to the ordering method on the robot of the N hand grips to obtain an image representing the grasping result.

[0047] Continuing with the foregoing example (where N = 5 and Z = 2), after obtaining the first image to be processed shown in FIG. 2C and the second image to be processed shown in FIG. 2E, five single-viewpoint images are cropped from the first image to be processed, namely, single-viewpoint image SD11, single-viewpoint image SZ21, single-viewpoint image SD31, single-viewpoint image SD41, and single-viewpoint image SD51. Similarly, five single-viewpoint images are cropped from the second image to be processed, namely, single-viewpoint image SD21, single-viewpoint image SZ22, single-viewpoint image SD32, single-viewpoint image SD42, and single-viewpoint image SD52. Further, two single-viewpoint images corresponding to the same hand grip among these 5×2 = 10 single-viewpoint images are joined together to obtain five multi-viewpoint images, namely, multi-viewpoint image SD11&SD12 corresponding to the first hand grip 2021, multi-viewpoint image SZ21&SZ22 corresponding to the second hand grip 2022, multi-viewpoint image SD31&SD32 corresponding to the third hand grip 2023, multi-viewpoint image SD41&SD42 corresponding to the fourth hand grip 2024, and multi-viewpoint image SD51&SD52 corresponding to the fifth hand grip 2025. Further, according to the sorting method on the robot for the five hand grips, these five multi-viewpoint images are arranged in a tile-like manner, specifically, as shown in FIG. 2F, an image representing the gripping result is obtained.

[0048] Through the above steps included in step S1013, in the embodiments of the present disclosure, a specific method for obtaining an image representing the gripping result is provided, where the process is simple and the execution efficiency of the tray recovery method can be improved.

[0049] After the execution of step S101, the robot may be controlled to be placed one-to-one corresponding to M winding package trays included in M target tray groups installed on the main line of the production line. Continuing with the above example (in this example, M = 4), referring to FIGS. 2G and 2H, it is assumed that the target tray group installed on the main line 207 of the production line includes five winding package trays, that is, the first winding package tray 2081 corresponding to the first hand grip 2021, the second winding package tray 2082 corresponding to the second hand grip 2022, the third winding package tray 2083 corresponding to the third hand grip 2023, the fourth winding package tray 2084 corresponding to the fourth hand grip 2024, and the fifth winding package tray 2085 corresponding to the fifth hand grip 2025. Then, the robot 201 can be controlled to place the first winding package to be packaged 2041 on the first winding package tray 2081, the third winding package to be packaged 2043 on the third winding package tray 2083, the fourth winding package to be packaged 2044 on the fourth winding package tray 2084, and the fifth winding package to be packaged 2045 on the second winding package tray 2085.

[0050] Also, as described above, in the embodiments of the present disclosure, the image representing the gripping result may include N multi-angle images, and the N multi-angle images may correspond one-to-one to N hand grips. The target tray group may include N winding package trays, and the N winding package trays may correspond one-to-one to N hand grips. Therefore, a one-to-one correspondence relationship between the N winding package trays and the N multi-angle images can be constructed.

[0051] Continuing with the previous example, the images representing the gripping results include a plurality of five multi-angle images, namely, the multi-angle images SD11&SD12 corresponding to the first hand grip 2021, the multi-angle images SZ21&SZ22 corresponding to the second hand grip 2022, the multi-angle images SD31&SD32 corresponding to the third hand grip 2023, the multi-angle images SD41&SD42 corresponding to the fourth hand grip 2024, and the multi-angle images SD51&SD52 corresponding to the fifth hand grip 2025. The target tray group includes five winding package trays, namely, the first winding package tray 2081 corresponding to the first hand grip 2021, the second winding package tray 2082 corresponding to the second hand grip 2022, the third winding package tray 2083 corresponding to the third hand grip 2023, the fourth winding package tray 2084 corresponding to the fourth hand grip 2024, and the fifth winding package tray 2085 corresponding to the fifth hand grip 2025, specifically shown in FIGS. 2F, 2G, and 2H. Thus, a one-to-one correspondence between N winding package trays and N multi-angle images can be constructed.

[0052] Multi-angle images SD11&SD12 ←→ First winding package tray 2081, Multi-angle images SZ21&SZ22 ←→ Second winding package tray 2082, Multi-angle images SD31&SD32 ←→ Third winding package tray 2083, Multi-angle images SD41&SD42 ←→ Fourth winding package tray 2084, Multi-angle images SD51&SD52 ←→ Fifth winding package tray 2085.

[0053] The above correspondence can be specifically shown as in FIG. 2I.

[0054] Based on the above, in some optional embodiments, "determining that there is a tray to be recovered in the target tray group based on the image representing the gripping result" in step S102 can include the following steps.

[0055] In step S1021, for each of the N multi-angle images among the plurality of multi-angle images, if it is determined that there is no winding package image for representing the winding package to be packaged in the plurality of multi-angle images, it is determined that there is a tray to be collected in the target tray group, and among the N winding package trays, the winding package tray corresponding to the multi-angle image is set as the first tray to be collected.

[0056] In one example, it is possible to determine whether there is a winding package image for representing the winding package to be packaged in the plurality of multi-angle images by means of a connected region analysis method (i.e., also called a Blob analysis method, BlobAnalysis), a template matching method, a deep learning method, or the like.

[0057] Also, in the embodiment of the present disclosure, after determining the first tray to be collected from the N winding package trays, the tray label of the first tray to be collected may be recorded, and the tray label of the first tray to be collected can be transmitted to the tray diversion device. Here, the tray label may be a tray code.

[0058] In step S1022, if it is determined that there is a winding package image for representing the winding package to be packaged in the plurality of multi-angle images, and it is determined that there are defect features in the winding package image by using a defect detection network, it is determined that there is a tray to be collected in the target tray group, and among the N winding package trays, the winding package tray corresponding to the multi-angle image is set as the second tray to be collected.

[0059] Here, examples of the defect features can include at least one of a thread breakage feature, an oil stain feature, a thread hanging feature, a molding defect, a paper tube breakage, and the like.

[0060] In one example, the defect detection network may be implemented by combining a U-net network and a residual network (i.e., ResNet), or may be implemented based on a Convolutional Neural Networks (CNN) or other suitable neural network, and the embodiments of the present disclosure are not limited thereto. Here, the U-net network is a neural network model with a "U" shape structure. The left side of the U-net network is used to realize image feature extraction, and the right side is used to realize the upsampling process of image features.

[0061] Also, in the embodiments of the present disclosure, after determining the second collection target tray from N winding package trays, the tray label of the second collection target tray may be recorded, and the tray label of the second collection target tray may be sent to the tray diversion device.

[0062] Continuing with the previous example, the image representing the gripping result includes a plurality of five angular images, the target tray group includes five winding package trays, and the plurality of five angular images and the five winding package trays have a one-to-one correspondence as shown in FIG. 2I. Then, when executing step S1021, since it can be determined that there is no winding package image for representing the winding package to be packaged in the plurality of angular images SZ21&SZ22, it is determined that there is a tray to be recovered in the target tray group. Among the five winding package trays, the second winding package tray 2082 corresponding to the plurality of angular images SZ21&SZ22 is taken as the first tray to be recovered, the tray identifier of the first tray to be recovered is recorded, and the tray identifier of the first tray to be recovered can be sent to the tray diversion device. When step S1022 is executed, since it can be determined that there are defective features in both the plurality of angular images SD41&SD42 and the plurality of angular images SD51&SD52, it is determined that there is a tray to be recovered in the target tray group. Among the five winding package trays, the fourth winding package tray 2084 corresponding to the plurality of angular images SD41&SD42 and the fifth winding package tray 2085 corresponding to the plurality of angular images SD51&SD52 are taken as the second trays to be recovered, the tray identifiers of the second trays to be recovered are recorded, and the tray identifiers of the second trays to be recovered can be sent to the tray diversion device.

[0063] Through the above steps included in step S102, in the embodiments of the present disclosure, for each of the plurality of angular images, it is determined whether the corresponding tray to be recovered is the first tray to be recovered. When it is determined that the corresponding tray to be recovered is not the first tray to be recovered, by determining whether the corresponding tray to be recovered is the second tray to be recovered, through a cascaded two-stage determination process, the misjudgment rate of the tray to be recovered can be reduced, and the accuracy of tray recovery can be improved. On the other hand, in the embodiments of the present disclosure, since the machine vision method is used to identify the tray to be recovered, not only can the execution efficiency of the tray recovery method be improved, but also the accuracy of tray recovery can be improved.

[0064] Referring to FIG. 3A, in one specific example, the defect detection network can include a first network layer, a second network layer, and a third network layer.

[0065] The first network layer combines a channel attention mechanism and a spatial attention mechanism to perform feature extraction on the winding package image and obtain K feature representation diagrams with different scales, where K≥2 and K is an integer. The second network layer performs feature fusion processing based on the K feature representation diagrams to obtain K fused feature diagrams, where the K fused feature diagrams correspond one-to-one to the K feature representation diagrams. The third network layer is for obtaining a defect detection result based on the K fused feature diagrams, where the defect detection result is used to indicate whether there are defect features in the winding package image.

[0066] Referring to FIG. 3B, in one specific example, the first network layer can include a first feature extraction module, a second feature extraction module, and a third feature extraction module connected in cascade (i.e., K = 3 in this example).

[0067] Here, the first feature extraction module processes the input image features (here, the winding package image, for example, the winding package image in the plurality of view angle images SD41&SD42 shown in FIGS. 2F and 2I) to obtain a first feature representation diagram. Further, the first feature extraction module can include a first convolution processing unit and a first attention unit connected in cascade. Here, the first convolution processing unit is for performing convolution processing on the input image features, and the first attention unit combines a channel attention mechanism and a spatial attention mechanism to extract features from the input image features.

[0068] The second feature extraction module is for processing the input image features (the first feature representation diagram) to obtain the second feature representation diagram. Further, the second feature extraction module can include a second convolutional processing unit and a second attention unit connected in cascade. Here, the second convolutional processing unit is for performing convolutional processing on the input image features, and the second attention unit combines a channel attention mechanism and a spatial attention mechanism to extract features from the input image features.

[0069] The third feature extraction module is for processing the input image features (the second feature representation diagram) to obtain the third feature representation diagram. Further, the third feature extraction module can include a third convolutional processing unit, a spatial pyramid pooling (Spatial Pyramid Pooling Fast, SPPF) unit, and a third attention unit connected in cascade. Here, the convolutional processing unit is for performing convolutional processing on the input image features, the SPPF unit is for fusing image features with different receptive fields, and the third attention unit combines a channel attention mechanism and a spatial attention mechanism to extract features from the input image features.

[0070] Referring to FIG. 3C, in this example, the first attention unit, the second attention unit, and the third attention unit can have the same structure. For example, it can include a first sub-unit, a second sub-unit, and a third sub-unit in parallel, and a fusion sub-unit for fusing the image features output by the first sub-unit, the second sub-unit, and the third sub-unit, and an activation sub-unit for normalizing the image features output by the fusion sub-unit (for example, using a Sigmoid function to normalize the image features output by the fusion sub-unit). Here, the third sub-unit can include a first internal unit and a second internal unit connected in cascade, the first internal unit and the first sub-unit can have the same structure, the second internal unit and the second sub-unit can have the same structure, the first internal unit and the first sub-unit are for extracting features of the input image features based on a channel attention mechanism, and the second internal unit and the second sub-unit are for extracting features of the input image features based on a spatial attention mechanism.

[0071] More specifically, referring to FIG. 3D, the first internal unit and the first sub-unit include a first internal pooling unit and a second internal pooling unit connected in parallel, a first internal convolutional unit connected in series to the first internal pooling unit and the second internal pooling unit having a parallel structure, a third internal pooling unit and a fourth internal pooling unit connected in cascade to the first internal convolutional unit and having a parallel structure, an internal fusion unit for fusing the image features output by the third internal pooling unit and the fourth internal pooling unit, and a first internal activation unit for normalizing the image features output by the internal fusion unit (for example, using a Sigmoid function to normalize the image features output by the internal fusion unit).

[0072] Here, the first internal pooling unit is for performing average pooling processing on the input image features. The second internal pooling unit is for performing max pooling processing on the input image features. The first internal convolutional unit may be a convolutional unit realized based on a depthwise convolution (DW) layer, or may be a convolutional unit realized based on a depthwise separable convolution (DSC) layer. The third internal pooling unit is for performing average pooling processing on the input image features. The fourth internal pooling unit is for performing max pooling processing on the input image features. Here, the DSC layer can be constructed by combining the DW layer and a pointwise convolution (PW) layer. In the actual application process, the first internal convolutional unit can perform deep channel feature extraction on the input image features, thereby enhancing the channel attention ability and feature representation ability. Moreover, since the first internal convolutional unit has a weight sharing characteristic, some unnecessary parameter amounts in the neural network calculation process can be reduced.

[0073] Specifically, referring to FIG. 3E, the second internal unit and the second sub-unit can include a fifth internal pooling unit and a sixth internal pooling unit connected in cascade, and a channel combination unit, a second internal convolutional unit, and a second internal activation unit that are connected in cascade to the sixth internal pooling unit and form a cascade structure.

[0074] Here, the fifth internal pooling unit is for performing average pooling processing on the input image features. The sixth internal pooling unit is for performing max pooling processing on the input image features. The channel combination unit is for splicing the input image features according to channels. The second internal convolutional unit may be a convolutional unit realized based on a standard convolutional network, or may be realized directly using a CNN. The second internal activation unit is for normalizing the image features output by the second internal convolutional unit (for example, using the Sigmoid function to normalize the image features output by the second internal convolutional unit).

[0075] Referring to FIG. 3F, illustratively, the second network layer can include an upsampling module and a downsampling module.

[0076] Here, the upsampling module can include a cascaded first Ghost Shuffle Convolution (GSC) unit, a first Tensor Concat unit, a first bottleneck unit, a second GSC unit, and a second Concat unit. Here, the first GSC unit is for performing convolution processing on the third feature representation diagram. The first Concat unit performs tensor combination on the second feature representation diagram and the image features output by the first GSC unit after upsampling processing to obtain the output image features, and after performing upsampling processing, sequentially inputs them to the first bottleneck unit and the second GSC unit for processing. The second Concat unit performs tensor combination on the first feature representation diagram and the image features output by the second GSC unit after upsampling processing to obtain the output image features.

[0077] The downsampling module may include a cascaded second bottleneck unit, a third GSC unit, a third Concat unit, a third bottleneck unit, a fourth GSC unit, a fourth Concat unit, and a fourth bottleneck unit. Here, the second bottleneck unit is for processing the image features output by the second GSC unit to obtain a first fused feature map. The third GSC unit is for performing convolution processing on the first fused feature map. The third Concat unit performs tensor concatenation on the image features output by the second GSC unit and the image features output by the downsampled third GSC unit to obtain the output image features, and after performing downsampling processing, sequentially inputs them to the third bottleneck unit and the fourth GSC unit for processing. The fourth Concat unit performs tensor concatenation on the image features output by the first GSC unit and the image features output by the downsampled fourth GSC unit to obtain the output image features. The fourth bottleneck unit is for processing the image features output by the downsampled fourth Concat unit to obtain a third fused feature map.

[0078] In this example, the first bottleneck unit, the second bottleneck unit, the third bottleneck unit, and the fourth bottleneck unit can have the same structure. More specifically, referring to FIG. 3G, the first bottleneck unit, the second bottleneck unit, the third bottleneck unit, and the fourth bottleneck unit include a cascaded first convolution sub-unit and a bottleneck sub-unit composed of a bottleneck layer (e.g., Bottleneck layer) and a channel split and concatenate (CSP) layer, a second convolution sub-unit connected in parallel with the first convolution sub-unit and the bottleneck sub-unit having a cascade structure, a Concat sub-unit that takes as input the image features output by the bottleneck sub-unit and the second convolution sub-unit, It can include a third convolutional subunit cascade-connected to the Concat subunit.

[0079] Here, the first convolutional subunit, the second convolutional subunit, and the third convolutional subunit may be convolutional subunits realized based on a standard convolutional network. The Bottleneck layer is for reducing the amount of unnecessary parameters during neural network calculation. The CSP layer is for performing operations such as Split, Concat, and Shuffle on the input image features.

[0080] Referring to FIG. 3H, illustratively, the third network layer can include a first detection unit for connecting to the second bottleneck unit, a second detection unit for connecting to the third bottleneck unit, and a third detection unit for connecting to the fourth bottleneck unit.

[0081] Here, the first detection unit, the second detection unit, and the third detection unit may be convolutional subunits realized based on a standard convolutional network.

[0082] Based on the image features output by the first detection unit, the second detection unit, and the third detection unit, a defect detection result can be obtained.

[0083] In an embodiment of the present disclosure, the defect detection network includes a first network layer, a second network layer, and a third network layer. The first network layer combines a channel attention mechanism and a spatial attention mechanism to extract features from the winding package image and obtain K feature representation diagrams with different scales, so as to ensure the feature representation ability of the K feature representation diagrams. In this way, after performing feature fusion processing on the K feature representation diagrams using the second network layer to obtain K fused feature diagrams, and obtaining a defect detection result based on the K fused feature diagrams using the third network layer, the reliability of the defect detection result can be improved, and the accuracy of tray recovery can be further improved.

[0084] In addition, in an embodiment of the present disclosure, the defect detection network can be trained according to the following flow.

[0085] (1) Obtain a winding package image sample and a data label corresponding to the winding package image sample. Here, the data label is used to represent whether there are yarn breakage, oil stain, yarn snagging, molding defect, paper tube breakage, etc. in the winding package image.

[0086] (2) Input the winding package image sample into the initial detection model to obtain the image features output by the initial detection model.

[0087] (3) Obtain a defect prediction result based on the image features output by the initial detection model.

[0088] (4) Use a preset loss function to calculate the loss value between the defect prediction result and the data label. If the loss value does not meet the preset convergence condition, adjust the model parameters of the initial detection model. If the loss value meets the preset convergence condition, use the initial detection model as the defect detection model.

[0089] Here, the preset loss function may be a Shape and Intersection over Union Loss (SIoU).

[0090] Furthermore, in the embodiments of the present disclosure, the tray diversion device includes a first diversion device and a second diversion device installed on the main line of the production line, and the bypass line of the production line includes a first bypass line and a second bypass line. Based on this, in some optional embodiments, "controlling the tray diversion device to divert the tray to be recycled from the main line of the production line to the bypass line of the production line" in step S102 can include the following.

[0091] In step S1021, control the first diversion device to divert the first tray to be recycled from the main line of the production line to the first bypass line of the production line.

[0092] Here, the first diversion device can receive, as the first target label, the tray label of the first tray to be recycled transmitted by the electronic device.

[0093] When N winding package trays included in the target tray group pass through the first diversion device, the first diversion device determines whether there is a tray to be recycled among the N winding package trays whose tray label matches the first target label. If there is a tray to be recycled among the N winding package trays whose tray label matches the first target label, confirm that tray as the first tray to be recycled, and the first tray to be recycled can be diverted from the main line of the production line to the first bypass line of the production line.

[0094] In step S1022, control the second diversion device to divert the second tray to be recycled from the main line of the production line to the second bypass line of the production line.

[0095] Here, the second diversion device can receive, as the second target label, the tray label of the second tray to be recycled transmitted by the electronic device.

[0096] Of the N winding package trays included in the target tray group, when the other winding package trays excluding the first recovery target tray pass through the second diversion device, the second diversion device determines whether there is a recovery target tray in the other winding package trays whose tray label matches the second target label. If there is a recovery target tray in the other winding package trays whose tray label matches the second target label, the recovery target tray is confirmed as the second recovery target tray, and the second recovery target tray can be diverted from the main line of the production line to the second bypass line of the production line.

[0097] Referring to FIG. 4A, illustratively, the tray diversion device includes a first diversion device 4021 and a second diversion device 4022 installed on the main line 401 of the production line, and the bypass line of the production line includes a first bypass line 4031 and a second bypass line 4032. The target tray group includes five winding package trays (i.e., N = 5), namely the first winding package tray 4041, the second winding package tray 4042, the third winding package tray 4043, the fourth winding package tray 4044, and the fifth winding package tray 4045. Here, the second winding package tray 4042 is the first recovery target tray, and the fourth winding package tray 4044 and the fifth winding package tray 4045 are the second recovery target trays.

[0098] Now, when the five winding package trays included in the target tray group pass through the first diversion device 4021, the first diversion device 4021 reads the tray identifier of the second winding package tray 4042 using Radio Frequency Identification (RFID). After that, if it determines that the tray identifier of the second winding package tray 4042 matches the first target identifier, it confirms the second winding package tray 4042 as the first tray to be recovered, and the first tray to be recovered can be diverted from the main line 401 of the production line to the first bypass line 4031 of the production line. Among the five winding package trays included in the target tray group, when other winding package trays (i.e., the first winding package tray 4041, the third winding package tray 4043, the fourth winding package tray 4044, and the fifth winding package tray 4045) except the first tray to be recovered (i.e., the second winding package tray 4042) pass through the second diversion device 4022, the second diversion device 4022 reads the tray identifiers of the other winding package trays using RFID. After that, it can determine that the tray identifiers of the fourth winding package tray 4044 and the fifth winding package tray 4045 in the other winding package trays match the second target identifier, confirm the fourth winding package tray 4044 and the fifth winding package tray 4045 as the second trays to be recovered, and divert the second trays to be recovered from the main line 401 of the production line to the second bypass line 4032 of the production line.

[0099] Through the above steps included in step S102, the first tray to be recycled and the second tray to be recycled are diverted onto different bypass lines to perform different processes, ensuring that the accuracy of tray recycling can be improved. For example, the first tray to be recycled diverted onto the first bypass line is finally diverted to the tray recycling location, can form a new target tray group, and can be used to enter the source of the main line of the production line again. Further, for example, the second tray to be recycled diverted onto the second bypass line can proceed to a re-inspection process for determining whether the packaged winding package placed on the second tray to be recycled is indeed a predetermined grade winding package, thereby avoiding misjudgment.

[0100] As described above, in the embodiments of the present disclosure, the second tray to be recycled diverted onto the second bypass line can determine whether the packaged winding package placed on the second tray to be recycled is indeed a predetermined grade winding package, and can proceed to a re-inspection process for avoiding misjudgment. In this case, in the embodiments of the present disclosure, the tray diversion device can further include a third diversion device placed on the second bypass line, and the bypass line further includes a third bypass line. Based on this, in some optional embodiments, after executing step S1022, step S102 can also include any of the following steps.

[0101] In step S1023, when controlling the third diversion device and the result of the re-inspection of the second tray to be recycled indicates that the packaged winding package placed on the second tray to be recycled is a predetermined grade winding package, divert the second tray to be recycled from the second bypass line to the tray recycling location.

[0102] Here, the purpose of reinspecting the second collection target tray is to determine whether the packaged winding package placed on the second collection target tray is indeed a predetermined grade winding package, and to avoid misjudgment. For example, there may be cases where cleaning - possible yarns are attached to the surface of the packaged winding package placed on the winding package tray. However, when it is determined that "there is a collection target tray in the target tray group based on the image representing the gripping result", there is a possibility of misjudging these winding package trays as the second collection target trays. Also, in the embodiments of the present disclosure, the reinspection process may be realized using a machine - vision method or may be performed manually, and the embodiments of the present disclosure are not limited thereto.

[0103] In one example, the reinspection result can be recorded by scanning the tray label. Specifically, when reinspecting the second collection target tray and determining that the packaged winding package placed on the second collection target tray is a predetermined grade winding package, scan the tray label of the second collection target tray as the third target label, transmit the third target label to an electronic device for recording, and then the electronic device can transmit the third target label to the third diversion device. In this way, when the second collection target tray passes through the third diversion device, the third diversion device determines whether the tray label of the second collection target tray matches the third target label. When the tray label of the second collection target tray matches the third target label information, it is determined that the reinspection result of the second collection target tray indicates that the packaged winding package placed on the second collection target tray is a predetermined grade winding package, and the second collection target tray can be diverted from the second bypass line to the tray collection location.

[0104] In another example, the re-inspection results can be indirectly recorded by removing the winding package to be packaged. Specifically, when the second collection target tray is reinspected and it is determined that the winding package to be packaged placed on the second collection target tray is a predetermined grade product winding package, the winding package to be packaged placed on the second collection target tray can be removed from the second collection target tray, and the second collection target tray can be made into an empty second collection target tray. Thereafter, when the empty second collection target tray passes through the empty tray detection device on the second bypass line and the empty tray detection device detects that the second collection target tray is empty, the tray label of the empty second collection target tray is recorded as the fourth target label, the fourth target label is transmitted to and recorded by an electronic device, and the fourth target label can be transmitted by the electronic device to the third diversion device. In this way, when the second collection target tray passes through the third diversion device, the third diversion device determines whether the tray label of the second collection target tray matches the fourth target label. If the tray label of the second collection target tray matches the fourth target label, it is determined that the re-inspection result of the second collection target tray indicates that the winding package to be packaged placed on the second collection target tray is a predetermined grade product winding package, and the second collection target tray can be diverted from the second bypass line to the tray collection location. Here, the empty tray detection device can detect whether the second collection target tray is empty using a mechanical vision method.

[0105] In step S1024, when controlling the third diversion device, if the re-inspection result of the second collection target tray indicates that the winding package to be packaged placed on the second collection target tray is a predetermined grade product winding package, the second collection target tray is diverted from the second bypass line to the tray collection location.

[0106] In one example, the re-inspection results can be recorded by scanning the tray label. Then, when the second collection target tray passes through the third diversion device, the third diversion device determines whether the tray label of the second collection target tray matches the third target label. If the tray label of the second collection target tray does not match the third target label, it is determined that the re-inspection result of the second collection target tray indicates that the packaged winding package placed on the second collection target tray is not a predetermined grade winding package (for example, a winding package product of AA grade). The second collection target tray is diverted from the second bypass line to the third bypass line and then refluxed from the third bypass line to the main line of the production line, and proceeds to the next process, such as weighing, appearance inspection, bagging, palletizing, banding device, film packaging, and marking.

[0107] In another example, the re-inspection results are indirectly recorded by removing the packaged winding package. Then, when the second collection target tray passes through the third diversion device, the third diversion device determines whether the tray label of the second collection target tray matches the fourth target label. If the tray label of the second collection target tray does not match the fourth target label, it is determined that the re-inspection result of the second collection target tray indicates that the packaged winding package placed on the second collection target tray is not a predetermined grade winding package (for example, a winding package product of AA grade). The second collection target tray is diverted from the second bypass line to the third bypass line and then refluxed from the third bypass line to the main line of the production line, and proceeds to the next process, such as weighing, appearance inspection, bagging, palletizing, banding device, film packaging, and marking.

[0108] Continuing with the example shown in FIG. 4A and referring to FIG. 4B, in this example, assuming that the reinspection results are recorded by scanning the tray label, the reinspection result of the second collection target tray 4044 indicates that the winding package to be packaged placed on the second collection target tray 4044 is a predetermined grade winding package, and the reinspection result of the second collection target tray 4045 indicates that the winding package to be packaged placed on the second collection target tray 4045 is not a predetermined grade winding package. Also, when reinspecting the second collection target tray 4044 and it is determined that the winding package to be packaged placed on the second collection target tray 4044 is a predetermined grade winding package, the tray label of the second collection target tray 4044 has already been scanned, set as the third target label, the third target label is transmitted to and recorded by an electronic device, and then the third target label is transmitted by the electronic device to the third diversion device 4023.

[0109] Then, referring to FIG. 4C, when the second tray to be collected 4044 passes through the third flow-dividing device 4023, after the third flow-dividing device 4023 reads the tray label of the second tray to be collected 4044 using RFID, it can be determined that the tray label of the second tray to be collected 4044 matches the third target label, and it is confirmed that the re-inspection result of the second tray to be collected 4044 indicates that the winding package to be packaged placed on the second tray to be collected 4044 is a predetermined grade winding package. Then, the second tray to be collected 4044 is diverted from the second bypass line 4032 to the tray collection location. When the second tray to be collected 4045 passes through the third flow-dividing device 4023, after the third flow-dividing device 4023 reads the tray label of the second tray to be collected 4045 using RFID, it can be determined that the tray label of the second tray to be collected 4045 does not match the third target label, and it is confirmed that the re-inspection result of the second tray to be collected 4045 indicates that the winding package to be packaged placed on the second tray to be collected 4045 is not a predetermined grade winding package. Then, the second tray to be collected 4045 is diverted from the second bypass line 4032 to the third bypass line 4033, and is refluxed from the third bypass line 4033 to the main line 401 of the production line and proceeds to the next process. For example, it proceeds to processes such as weighing, appearance inspection, bagging, palletizing, banding device, film packaging, and marking.

[0110] Continuing with the example shown in FIG. 4A, referring to FIG. 4D, in this example, the re-inspection results are indirectly recorded in a manner of removing the winding package to be packaged. The re-inspection result of the second collection target tray 4044 indicates that the winding package to be packaged placed on the second collection target tray 4044 is a predetermined grade winding package, and the re-inspection result of the second collection target tray 4045 indicates that the winding package to be packaged placed on the second collection target tray 4045 is not a predetermined grade winding package. Assume that the second collection target tray 4044 is re-inspected, and when it is determined that the winding package to be packaged placed on the second collection target tray 4044 is a predetermined grade winding package, the winding package to be packaged placed on the second collection target tray 4044 has already been removed from the second collection target tray 4044, and the second collection target tray 4044 has become an empty second collection target tray 4044. Thereafter, the empty second collection target tray 4044 passes through the empty tray detection device 405 on the second bypass line 4032. When the empty tray detection device 405 detects that the second collection target tray 4044 is empty, the tray label of the empty second collection target tray 4044 is recorded as the fourth target label, the fourth target label is transmitted to and recorded by an electronic device, and the fourth target label can be transmitted by the electronic device to the third diversion device 4023.

[0111] Then, referring to FIG. 4E, when the second tray to be recycled 4044 passes through the third flow diversion device 4023, after the third flow diversion device 4023 reads the tray label of the second tray to be recycled 4044 using RFID, it can be determined that the tray label of the second tray to be recycled 4044 matches the fourth target label. It is confirmed that the re-inspection result of the second tray to be recycled 4044 indicates that the winding package to be packaged placed on the second tray to be recycled 4044 is a predetermined grade winding package. Then, the second tray to be recycled 4044 is diverted from the second bypass line 4032 to the tray collection location. When the second tray to be recycled 4045 passes through the third flow diversion device 4023, after the third flow diversion device 4023 reads the tray label of the second tray to be recycled 4045 using RFID, it can be determined that the tray label of the second tray to be recycled 4045 does not match the fourth target label. It is confirmed that the re-inspection result of the second tray to be recycled 4045 indicates that the winding package to be packaged placed on the second tray to be recycled 4045 is not a predetermined grade winding package. Then, the second tray to be recycled 4045 is diverted from the second bypass line 4032 to the third bypass line 4033, and is refluxed from the third bypass line 4033 to the main line 401 of the production line, and then proceeds to the next process. For example, it proceeds to processes such as weighing, appearance inspection, bagging, palletizing, banding device, film packaging, and marking.

[0112] Through the above steps included in step S102, in the embodiment of the present disclosure, after further determining whether the winding package to be packaged placed on the second tray to be recycled is indeed a predetermined grade winding package, the second tray to be recycled is re-diverted, and the accuracy of tray collection can be further improved.

[0113] To better implement the above tray recovery method, embodiments of the present disclosure are also applied to an electronic device included in a winding package packaging system. The electronic device provides a tray recovery device that communicates with an image collection device and a tray diversion device respectively. The electronic device may be a computer, a PLC, etc. The image collection device may be an industrial camera, and the tray diversion device may be an aerodynamic diversion device, an electric diversion device, a drum-type diversion device, etc. Embodiments of the present disclosure are not limited thereto.

[0114] Hereinafter, the tray recovery device 500 provided by the disclosed embodiments will be described with reference to the block diagram showing the configuration shown in FIG. 5.

[0115] The tray recovery device 500 includes an image acquisition unit 501 for obtaining an image representing the gripping result by controlling the image collection device to capture an image in the direction where N hand grips are located after the robot grips M winding packages to be packaged from the winding package trolley by N hand grips. Here, each of the N hand grips can grip one winding package to be packaged, N≥2, and N is an integer, 0≤M≤N, and M is an integer. a recovery control unit 502 for executing the recovery process of the tray to be recovered by controlling the tray diversion device to divert the tray to be recovered from the main line of the production line to the bypass line of the production line when it is determined that there is a tray to be recovered in the target tray group based on the image representing the gripping result after the robot places the M winding packages to be packaged one-to-one on the M winding package trays included in the target tray group installed on the main line of the production line. Here, the tray to be recovered is an empty first tray to be recovered or a second tray to be recovered that is initially evaluated that the placed winding package to be packaged is a winding package of a predetermined grade. The tray recovery device 500 includes the image acquisition unit 501 and the recovery control unit 502.

[0116] In some optional embodiments, the electronic device further communicates with the robot. Here, the image acquisition unit 501 controls the N handgrips to rotate with respect to the image acquisition device, and during the process of the N handgrips rotating with respect to the image acquisition device, controls the image acquisition device to capture images in the direction where the N handgrips are located, so as to obtain Z target images to be processed. Here, Z ≥ 2 and Z is an integer, and the Z target images to be processed correspond to different viewing angles of the N handgrips, and is used to obtain an image representing the gripping result based on the Z target images to be processed.

[0117] In some optional embodiments, the image acquisition unit 501 cuts out N single-view-angle images from each of the Z target images to be processed to obtain N×Z single-view-angle images, where the N single-view-angle images correspond one-to-one to the N handgrips, joins together the Z single-view-angle images corresponding to the same handgrip among the N×Z single-view-angle images to obtain N multi-view-angle images, and is used to obtain an image representing the gripping result based on the N multi-view-angle images.

[0118] In some optional embodiments, the image representing the gripping result includes the N multi-view-angle images, the N multi-view-angle images correspond one-to-one to the N handgrips, the target tray group includes N winding package trays, and in order to establish a one-to-one correspondence between the N winding package trays and the N multi-view-angle images, the N winding package trays correspond one-to-one to the N handgrips, Here, the recovery control unit 502 For each of the N multi-view-angle images, if it is determined that there is no winding package image representing the packaged winding package in the multi-view-angle image, and it is determined that there is a recovery target tray in the target tray group, the winding package tray corresponding to the multi-view-angle image among the N winding package trays is set as the first recovery target tray, When it is determined that there is a winding package image for representing the winding package to be packaged in the plurality of multi-angle images, and it is determined that there are defect features in the winding package image using the defect detection network, if it is determined that there is a tray to be recovered in the target tray group, and among the N winding package trays, the winding package tray corresponding to the plurality of multi-angle images is used as the second tray to be recovered.

[0119] In some optional embodiments, the defect detection network includes a first network layer, a second network layer, and a third network layer. The first network layer combines a channel attention mechanism and a spatial attention mechanism to perform feature extraction on the winding package image to obtain K feature representation diagrams with different scales, where K≥2 and K is an integer. The second network layer performs feature fusion processing based on the K feature representation diagrams to obtain K fused feature diagrams, where the K fused feature diagrams correspond one-to-one to the K feature representation diagrams. The third network layer is used to obtain a defect detection result for indicating whether there are defect features in the winding package image based on the K fused feature diagrams.

[0120] In some optional embodiments, the tray diversion device includes a first diversion device and a second diversion device installed on the main line of the production line, and the bypass line of the production line includes a first bypass line and a second bypass line. Here, the recovery control unit 502 is used to control the first diversion device to divert the first tray to be recovered from the main line of the production line to the first bypass line of the production line, and control the second diversion device to divert the second tray to be recovered from the main line of the production line to the second bypass line of the production line.

[0121] In some optional embodiments, the tray diversion device further includes a third diversion device installed on the second bypass line, and the bypass line of the production line further includes a third bypass line. Here, the recovery control unit 502 controls the third diversion device to divert the second tray to be recovered from the second bypass line to the tray recovery location when the re-inspection result of the second tray to be recovered indicates that the winding package to be packaged placed on the second tray to be recovered is a predetermined grade winding package. Alternatively, it is used to control the third diversion device to divert the second tray to be recovered from the second bypass line to the third bypass line and then reflux it from the third bypass line to the main line of the production line when the re-inspection result of the second tray to be recovered indicates that the winding package to be packaged placed on the second tray to be recovered is a non-predetermined grade winding package.

[0122] For the specific functions and examples of each unit in the tray recovery device provided by the embodiments of the present disclosure, reference can be made to the relevant descriptions of the corresponding steps in the embodiments of the above method, and the descriptions are omitted here.

[0123] In the technical solution of the present disclosure, the acquisition, storage, and application of such personal information of users all comply with relevant laws and regulations and do not violate public order and good customs.

[0124] FIG. 6 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 6, the electronic device includes a memory 601 and a processor 602, and a computer program executable by the processor 602 is stored in the memory 601. The number of the memory 601 and the processor 602 can be one or more. The memory 601 can store one or more computer programs, and when the one or more computer programs are executed by the electronic device, the electronic device is caused to execute the method provided by the embodiment of the above method. The electronic device can further include the following. A communication interface 603 is used for communicating with an external device and performing data interaction and transmission.

[0125] When the memory 601, the processor 602, and the communication interface 603 are independently implemented, the memory 601, the processor 602, and the communication interface 603 are connected to each other via a bus and can communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be classified into an address bus, a data bus, a control bus, etc. For ease of explanation, only a single thick line is shown in FIG. 6, but it does not represent only a single bus or a single type of bus.

[0126] Optionally, in a specific implementation form, when the memory 601, the processor 602, and the communication interface 603 are integrated on one chip, the memory 601, the processor 602, and the communication interface 603 can communicate with each other via an internal interface.

[0127] The above-mentioned processor may be a Central Processing Unit (CPU), and it should be understood that it may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware assemblies, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. Note that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0128] Furthermore, optionally, the memory may include a read-only memory and a random access memory, and may further include a non-volatile random access memory. The memory can be either a volatile memory or a non-volatile memory, or can include both a volatile memory and a non-volatile memory. Here, the non-volatile memory can include ROM (Read-Only Memory), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically EPROM), or flash memory. The volatile memory can include a random access memory (Random Access Memory, RAM) that functions as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), synchronous DRAM (Synchronous DRAM, SDRAM), double data rate SDRAM (Double Data Rate SDRAM, DDR SDRAM), enhanced SDRAM (Enhanced SDRAM, ESDRAM), synchlink DRAM (Synchlink DRAM, SLDRAM), and direct RAMBUS RAM (Direct RAMBUS RAM, DR RAM).

[0129] In the above embodiments, they may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of it may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present disclosure are generated in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, Bluetooth (registered trademark), microwave, etc.). The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device including a server, data center, etc. integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy (registered trademark) disk, hard disk, magnetic tape), an optical medium (such as a Digital Versatile Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)), etc. It should be noted that the computer-readable storage medium mentioned in the present disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0130] Those skilled in the art can understand that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above storage medium may be a read-only memory, a magnetic disk, an optical disk, or the like.

[0131] In the description of the embodiments of the present disclosure, the description of reference terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or features described in relation to the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or features described can be combined in any one or more embodiments or examples in an appropriate manner. Furthermore, those skilled in the art may combine different embodiments or examples described in the present disclosure and the features of different embodiments or examples as long as they do not conflict with each other.

[0132] In the description of the embodiments of the present disclosure, " / " represents the meaning of "or" unless otherwise explained. For example, A / B can represent either A or B. "And / or" in the present disclosure only explains the relationship of related objects and indicates that there may be three types of relationships. For example, A and / or B can represent the following. There are three situations where A exists alone, A and B exist simultaneously, and B exists alone.

[0133] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used only for the purpose of description and should not be construed as indicating or implying relative importance, nor should they be construed as implying the number of technical features shown. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "a plurality" means two or more unless otherwise explained.

[0134] The above are only exemplary embodiments of the present disclosure and do not limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure shall all be included within the protection scope of the present disclosure.

Claims

1. A tray recovery method applied to an electronic device included in a winding package packaging system, wherein the electronic device communicates with an image collection device and a tray diversion device respectively, The tray recovery method includes: After a robot grips M winding packages to be packaged from a winding package cart by N hand grips, controlling the image collection device to take an image in the direction where the N hand grips are located, so as to obtain an image representing the gripping result. Here, each of the N hand grips can grip one winding package to be packaged, N≥2, and N is an integer, 0≤M≤N, and M is an integer. And After the robot places the M winding packages to be packaged one-to-one on M winding package trays included in a target tray group installed on the main line of the production line, based on the image representing the gripping result, when it is determined that there is a tray to be recovered in the target tray group, controlling the tray diversion device to divert the tray to be recovered from the main line of the production line onto the bypass line of the production line, thereby performing the recovery process of the tray to be recovered. Here, the tray to be recovered is an empty first tray to be recovered or a second tray to be recovered initially evaluated that the placed winding package to be packaged is a winding package of a predetermined grade. And A tray recovery method.

2. The electronic device further communicates with the robot, Controlling the image collection device to take an image in the direction where the N hand grips are located, so as to obtain an image representing the gripping result includes: Controlling the N hand grips to rotate with respect to the image collection device, During the process that the N hand grips rotate with respect to the image collection device, controlling the image collection device to take an image in the direction where the N hand grips are located, so as to obtain Z images to be processed. Here, Z≥2, and Z is an integer, and the Z images to be processed correspond to different viewing angles of the N hand grips. And Obtaining an image representing the gripping result based on the Z images to be processed. And The tray recovery method according to Claim 1.

3. Obtaining an image representing the gripping result based on the Z target images to be processed includes: Cutting out N single-viewpoint images from each of the Z target images to be processed in the Z target images to be processed, to obtain N×Z single-viewpoint images, where the N single-viewpoint images correspond one-to-one to the N hand grips; Joining together the Z single-viewpoint images corresponding to the same hand grip among the N×Z single-viewpoint images to obtain N multi-viewpoint images; Obtaining an image representing the gripping result based on the N multi-viewpoint images; The tray recovery method according to claim 2.

4. The image representing the gripping result includes N multi-viewpoint images, the N multi-viewpoint images correspond one-to-one to the N hand grips, the target tray group includes N winding package trays, and in order to establish a one-to-one correspondence between the N winding package trays and the N multi-viewpoint images, the N winding package trays correspond one-to-one to the N hand grips; Determining that there is a tray to be recovered in the target tray group based on the image representing the gripping result includes: When it is determined that there is no winding package image for representing the winding package to be packaged in a multi-viewpoint image among the N multi-viewpoint images, it is determined that there is a tray to be recovered in the target tray group, and the winding package tray corresponding to the multi-viewpoint image among the N winding package trays is set as the first tray to be recovered; When it is determined that there is a winding package image for representing the winding package to be packaged in the multi-viewpoint image and it is determined that there are defect features in the winding package image using a defect detection network, it is determined that there is a tray to be recovered in the target tray group, and the winding package tray corresponding to the multi-viewpoint image among the N winding package trays is set as the second tray to be recovered; The tray recovery method according to claim 1.

5. The defect detection network includes a first network layer, a second network layer, and a third network layer; The first network layer combines a channel attention mechanism and a spatial attention mechanism to perform feature extraction on the winding package image to obtain K feature representation diagrams with different scales, where K≥2 and K is an integer; The second network layer is for performing feature fusion processing based on the K feature representation diagrams to obtain K fused feature diagrams. Here, the K fused feature diagrams correspond one-to-one with the K feature representation diagrams. The third network layer is for obtaining a defect detection result indicating whether there are defect features in the winding package image based on the K fused feature diagrams. The tray recovery method according to claim 4.

6. The tray diversion device includes a first diversion device and a second diversion device installed on the main line of the production line. The bypass line of the production line includes a first bypass line and a second bypass line. Controlling the tray diversion device to divert the tray to be recovered from the main line of the production line to the bypass line of the production line is Controlling the first diversion device to divert the first tray to be recovered from the main line of the production line to the first bypass line of the production line, and Controlling the second diversion device to divert the second tray to be recovered from the main line of the production line to the second bypass line of the production line, including. The tray recovery method according to claim 1.

7. The tray diversion device further includes a third diversion device installed on the second bypass line. The bypass line of the production line further includes a third bypass line. Controlling the tray diversion device to divert the tray to be recovered from the main line of the production line to the bypass line of the production line is Controlling the third diversion device to divert the second tray to be recovered from the second bypass line to the tray recovery location when the re-inspection result of the second tray to be recovered indicates that the winding package to be packaged placed on the second tray to be recovered is a predetermined grade winding package. Or, controlling the third diversion device to divert the second tray to be recovered from the second bypass line to the third bypass line and then reflux it from the third bypass line to the main line of the production line when the re-inspection result of the second tray to be recovered indicates that the winding package to be packaged placed on the second tray to be recovered is a non-predetermined grade winding package, further including. The tray recovery method according to claim 6.

8. A tray recovery device applied to an electronic device included in a winding package packaging system, wherein the electronic device communicates with an image collection device and a tray diversion device respectively, The tray recovery device is An image acquisition unit for obtaining an image representing a gripping result, which is controlled such that after a robot grips M winding packages to be packaged from a winding package cart by N hand grips, the image collection device images in the direction where the N hand grips are located. Here, each of the N hand grips can grip one winding package to be packaged, N≥2, and N is an integer, 0≤M≤N, and M is an integer. A recovery control unit for performing a recovery process of the recovery target tray by controlling the tray diversion device to divert the recovery target tray from the main line of the production line to the bypass line of the production line when it is determined that there is a recovery target tray in the target tray group based on the image representing the gripping result after the robot places the M winding packages to be packaged one-to-one on M winding package trays included in a target tray group installed on the main line of the production line. Here, the recovery target tray is an empty first recovery target tray or a second recovery target tray initially evaluated as having a packaged winding package to be a predetermined grade winding package. The tray recovery device includes the recovery control unit. Tray recovery device.

9. The electronic device further communicates with the robot. The image acquisition unit is Controlling the N hand grips to rotate with respect to the image collection device, During the process of the N hand grips rotating with respect to the image collection device, controlling the image collection device to image in the direction where the N hand grips are located, and obtaining Z images to be processed. Here, Z≥2, and Z is an integer, and the Z images to be processed correspond to different viewing angles of the N hand grips. Used to obtain an image representing the gripping result based on the Z images to be processed. The tray recovery device according to claim 8.

10. The image acquisition unit is Cutting out N single-viewpoint images from each of the Z images to be processed, and obtaining N×Z single-viewpoint images, wherein the N single-viewpoint images correspond one-to-one to the N hand grips, Joining together the Z single-viewpoint images corresponding to the same hand grip among the N×Z single-viewpoint images to obtain N multi-viewpoint images, Used to obtain an image representing the gripping result based on the N multi-viewpoint images, The tray recovery device according to claim 9.

11. The image representing the gripping result includes N multi-viewpoint images, the N multi-viewpoint images correspond one-to-one to the N hand grips, the target tray group includes N winding package trays, and in order to establish a one-to-one correspondence between the N winding package trays and the N multi-viewpoint images, the N winding package trays correspond one-to-one to the N hand grips, The recovery control unit, For each of the N multi-viewpoint images among the N multi-viewpoint images, when it is determined that there is no winding package image for representing the winding package to be packaged in the multi-viewpoint image, and it is determined that there is a recovery target tray in the target tray group, the winding package tray corresponding to the multi-viewpoint image among the N winding package trays is used as the first recovery target tray, When it is determined that there is a winding package image for representing the winding package to be packaged in the multi-viewpoint image, and it is determined that there are defect features in the winding package image using the defect detection network, and it is determined that there is a recovery target tray in the target tray group, the winding package tray corresponding to the multi-viewpoint image among the N winding package trays is used as the second recovery target tray, The tray recovery device according to claim 8.

12. The defect detection network includes a first network layer, a second network layer, and a third network layer, The first network layer combines a channel attention mechanism and a spatial attention mechanism to perform feature extraction on the winding package image to obtain K feature representation diagrams with different scales, where K≧2 and K is an integer, The second network layer is for performing feature fusion processing based on the K feature representation diagrams to obtain K fused feature diagrams. Here, the K fused feature diagrams correspond one-to-one with the K feature representation diagrams. The third network layer is for obtaining a defect detection result for indicating whether there are defect features in the winding package image based on the K fused feature diagrams. The tray recovery device according to claim 11.

13. The tray diversion device includes a first diversion device and a second diversion device installed on the main line of the production line. The bypass line of the production line includes a first bypass line and a second bypass line. The recovery control unit is used to control the first diversion device to divert the first tray to be recovered from the main line of the production line to the first bypass line of the production line. is used to control the second diversion device to divert the second tray to be recovered from the main line of the production line to the second bypass line of the production line. The tray recovery device according to claim 8.

14. The tray diversion device further includes a third diversion device installed on the second bypass line. The bypass line of the production line further includes a third bypass line. The recovery control unit is used to control the third diversion device to divert the second tray to be recovered from the second bypass line to the tray recovery location when the re-inspection result of the second tray to be recovered indicates that the winding package to be packaged placed on the second tray to be recovered is a predetermined grade winding package. Or, it is used to control the third diversion device to divert the second tray to be recovered from the second bypass line to the third bypass line and then reflux it from the third bypass line to the main line of the production line when the re-inspection result of the second tray to be recovered indicates that the winding package to be packaged placed on the second tray to be recovered is a non-predetermined grade winding package. The tray recovery device according to claim 13.

15. at least one processor; a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is caused to execute the method according to any one of claims 1 to 7. An electronic device. **Claim 16** A non-transitory computer-readable storage medium for storing instructions that cause a computer to execute the method according to any one of claims 1 to 7. **Claim 17** A program that, when executed by a processor in a computer, implements the method according to any one of claims 1 to 7.

Images