TRAY COLLECTION METHOD, APPARATUS, ELECTRONIC DEVICE, STORAGE MEDIUM, AND PROGRAM

Abstract

The present disclosure relates to the field of computers and automation control, and in particular to a tray retrieval method, apparatus, electronic device, storage medium and program. [Solution] The method involves a robot gripping M wound thread packages to be packaged from a wound thread package cart with N hand grips, and then controlling the image collection device to take an image in the direction of the N hand grips to obtain an image representing the gripping result; after the robot places the M wound thread packages to be packaged one-to-one on M wound thread package trays included in a target tray group installed on the main line of the production line, if it is determined based on the image representing the gripping result that a tray to be collected is present in the target tray group, a tray diversion device is controlled to divert the tray to be collected from the main line of the production line to a bypass line of the production line, thereby performing a collection process for the tray to be collected.

Description

[Technical field]

[0001] The present disclosure relates to the field of computers and automation control, and in particular to a tray retrieval method, apparatus, electronic device, storage medium and program. [Background technology]

[0002] The wrapping of yarn packages is an important part of the yarn manufacturing process. During the wrapping process of yarn packages, there may be some trays to be collected among the multiple yarn package trays on the production line due to manual picking, misplacement, etc. Currently, in order to prevent these trays to be collected from affecting the wrapping of yarn packages, they are usually collected by manual collection. Summary of the Invention [Problem to be solved by the invention]

[0003] However, the manual collection method not only consumes a large amount of human resources, but also affects the collection efficiency of the trays to be collected.

[0004] The present disclosure provides a tray recovery method, apparatus, electronic device, storage medium, and program to solve or alleviate one or more technical problems in the prior art. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a tray collection method adapted for an electronic device included in a wound yarn package packaging system, the electronic device being in communication with an image collection device and a tray diversion device, respectively: The tray collection method is as follows: After the robot grasps M wound yarn packages to be packaged from the yarn package carrier with N hand grips, the image acquisition device is controlled to capture images in the direction of the N hand grips to obtain an image representing the grasping result, where each of the N hand grips can grasp one wound yarn package to be packaged, N≧2 and N is an integer, and 0≦M≦N and M is an integer; The method includes: after the robot places M wound thread packages to be packaged one-to-one on M wound thread package trays included in a group of target trays installed on the main line of the production line, and if it is determined that a tray to be collected is present in the group of target trays based on an image representing the gripping result, controlling a tray diversion device to divert the tray to be collected from the main line of the production line to a bypass line of the production line, thereby performing a collection process for the tray to be collected, wherein the tray to be collected is an empty first target tray or a second target tray in which the placed wound thread package to be packaged has been initially evaluated to be a wound thread package of a specified grade.

[0006] According to a second aspect of the present disclosure, there is provided a tray collection device adapted for an electronic device included in a wound yarn package packaging system, the electronic device being in communication with an image collection device and a tray diversion device, respectively; The tray recovery device is an image capturing unit for controlling the image collecting device to capture images in the direction of the N hand grips after the robot grasps M wound yarn packages to be packaged from the yarn package carrier with the N hand grips, to obtain an image representing the grasping result, where each of the N hand grips can grasp one wound yarn package to be packaged, N≧2 and N is an integer, and 0≦M≦N and M is an integer; The robot places M wound yarn packages to be packaged one-to-one on M wound yarn package trays included in a group of target trays installed on the main line of the production line, and when it determines that a tray to be collected is present in the group of target trays based on an image representing the gripping result, the recovery control unit controls the tray diversion device to divert the tray to be collected from the main line of the production line to a bypass line of the production line, thereby performing a recovery process of the tray to be collected, wherein the tray to be collected is an empty first target tray or a second target tray in which the placed wound yarn package to be packaged has been initially evaluated to be a wound yarn package of a specified grade.

[0007] According to a third aspect of the present disclosure, there is provided an electronic device, the device comprising: At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the implementation of any one of the methods in the embodiments of the present disclosure.

[0008] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform any one of the methods according to the embodiments of the present disclosure.

[0009] According to a fifth aspect of the present disclosure, there is provided a program, which, when executed by a processor, realizes any one of the methods in the embodiments of the present disclosure.

[0010] In the technical solution provided by the present disclosure, after the robot grasps M wound yarn packages to be packaged from the yarn package carrier with N hand grips, the image collecting device is controlled to capture images in the direction of the N hand grips to obtain an image representing the grasping result, and after the robot places the M wound yarn packages to be packaged one-to-one on M wound yarn package trays included in a target tray group installed on the main line of the production line, if it is determined based on the image representing the grasping result that a tray to be collected exists in the target tray group, the tray diversion device is controlled to divert the tray to be collected from the main line of the production line to the bypass line of the production line, thereby performing a collection process of the tray to be collected, which can realize automatic collection of the tray to be collected, which can save a large amount of manpower and improve the collection efficiency of the tray to be collected compared with the current manual collection method.

[0011] It should be understood that the contents described herein are not intended to describe key points or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure, other features of the present disclosure are facilitated by the following specification. [Brief description of the drawings]

[0012] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the drawings in which the same or similar reference numerals represent the same or similar elements.

[0013] [Figure 1] 1 is a flowchart of a tray recovery method according to an embodiment of the present disclosure. [Figure 2A] 11A to 11C are diagrams illustrating an example of a flow for acquiring an image showing a gripping result in a tray collection method according to an embodiment of the present disclosure. [Figure 2B] 11A to 11C are diagrams illustrating an example of a flow for acquiring an image showing a gripping result in a tray collection method according to an embodiment of the present disclosure. [Figure 2C]11A to 11C are diagrams illustrating an example of a flow for acquiring an image showing a gripping result in a tray collection method according to an embodiment of the present disclosure. [Figure 2D] 11A to 11C are diagrams illustrating an example of a flow for acquiring an image showing a gripping result in a tray collection method according to an embodiment of the present disclosure. [Figure 2E] 11A to 11C are diagrams illustrating an example of a flow for acquiring an image showing a gripping result in a tray collection method according to an embodiment of the present disclosure. [Figure 2F] 11A to 11C are diagrams illustrating an example of a flow for acquiring an image showing a gripping result in a tray collection method according to an embodiment of the present disclosure. [Figure 2G] 11A to 11C are diagrams illustrating an example of a process of establishing a correspondence relationship between N wound yarn package trays and N images with multiple viewing angles in a tray collection method according to an embodiment of the present disclosure. [Figure 2H] 11A to 11C are diagrams illustrating an example of a process of establishing a correspondence relationship between N wound yarn package trays and N images with multiple viewing angles in a tray collection method according to an embodiment of the present disclosure. [Figure 2I] 11A to 11C are diagrams illustrating an example of a process of establishing a correspondence relationship between N wound yarn package trays and N images with multiple viewing angles in a tray collection method according to an embodiment of the present disclosure. [Figure 3A] FIG. 2 is a schematic diagram illustrating a configuration of a defect detection network according to an embodiment of the present disclosure. [Figure 3B] FIG. 2 is a schematic diagram illustrating a configuration of a first network layer according to an embodiment of the present disclosure. [Figure 3C] 2 is a schematic diagram showing the configuration of a first attention unit, a second attention unit, and a third attention unit according to an embodiment of the present disclosure. FIG. [Figure 3D] FIG. 2 is a schematic diagram illustrating a configuration of a first internal unit and a first subunit according to an embodiment of the present disclosure. [Figure 3E] FIG. 13 is a schematic diagram illustrating a configuration of a second internal unit and a second subunit according to an embodiment of the present disclosure. [Figure 3F] FIG. 2 is a schematic diagram illustrating a configuration of a second network layer according to an embodiment of the present disclosure. [Figure 3G]1 is a schematic diagram illustrating the configuration of a first bottleneck unit, a second bottleneck unit, a third bottleneck unit and a fourth bottleneck unit according to an embodiment of the present disclosure. [Figure 3H] FIG. 2 is a schematic diagram illustrating a configuration of a third network layer according to an embodiment of the present disclosure. [Figure 4A] 11A and 11B are diagrams illustrating an example of a flow division process in a tray recovery method according to an embodiment of the present disclosure. [Figure 4B] 11A and 11B are diagrams illustrating an example of a flow division process in a tray recovery method according to an embodiment of the present disclosure. [Figure 4C] 11A and 11B are diagrams illustrating an example of a flow division process in a tray recovery method according to an embodiment of the present disclosure. [Figure 4D] 11A and 11B are diagrams illustrating an example of a flow division process in a tray recovery method according to an embodiment of the present disclosure. [Figure 4E] 11A and 11B are diagrams illustrating an example of a flow division process in a tray recovery method according to an embodiment of the present disclosure. [Diagram 5] 1 is a block diagram showing a configuration of a tray recovery device according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The present disclosure will now be described in more detail with reference to the accompanying drawings, in which like reference numbers represent the same or similar elements, and in which various aspects of the embodiments are illustrated, and which, unless otherwise noted, are not necessarily drawn to scale.

[0015] Furthermore, in order to better explain the present disclosure, many specific details are described in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be similarly implemented without some details. In some embodiments, methods, means, components, circuits, etc. that are well known to those skilled in the art are not described in detail so that the gist of the present disclosure is clear.

[0016] As mentioned above, wrapping of yarn packages is an important part of the yarn manufacturing process. During the wrapping process of yarn packages, due to manual picking, misplacing, etc., there may be some trays to be collected among the multiple yarn package trays on the production line, such as empty yarn package trays and other yarn package trays that need to be collected. Currently, in order to prevent these trays to be collected from affecting the progress of wrapping of yarn packages, they are usually collected by manual collection. However, the manual collection method not only wastes a large amount of human resources, but also affects the collection efficiency of the trays to be collected.

[0017] In order to save a large amount of manpower and improve the efficiency of collecting trays to be collected, an embodiment of the present disclosure provides a tray collecting method applied to an electronic device included in a wound yarn package packaging system, the electronic device communicates with an image collecting device and a tray diverting device respectively, where the electronic device can be a computer, a programmable logic controller (PLC), etc., the image collecting device can be an industrial camera, and the tray diverting device can be an aerodynamic type diverting device, an electric type diverting device, a drum type diverting device, etc., and the embodiment of the present disclosure is not limited thereto.

[0018] Further, in the embodiments of the present disclosure, the main types of wound yarn package products may include one or more of partially oriented yarns (POY), fully drawn yarns (FDY), drawn textured yarns (DTY) (or low modulus filaments), etc. For example, the types of wound yarn package products may specifically include polyester partially oriented yarns, polyester fully drawn yarns, polyester drawn yarns, polyester drawn textured yarns, etc.

[0019] Fig. 1 is a flowchart of a tray collection method according to an embodiment of the present disclosure. Hereinafter, the tray collection method provided by the embodiment of the present disclosure will be described with reference to Fig. 1. It should be noted that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in a different order.

[0020] In step S101, the robot grasps M wound yarn packages to be packaged from the wound yarn package cart with N hand grips, and then controls the image collection device to capture images in the direction in which the N hand grips are located, thereby obtaining an image showing the grasping result.

[0021] Here, the robot may be an N-axis industrial robot, i.e., the robot includes N handgrips, and each handgrip in the N handgrips can hold one wound yarn package to be wrapped, where N≧2 and N is an integer.

[0022] In an embodiment of the present disclosure, when a robot grips a winding package to be packaged from a winding package carriage by N hand grips, it can be understood that some of the hand grips may not be able to successfully grip the winding package to be packaged. Therefore, in the embodiment of the present disclosure, 0≤M≤N, and M is an integer. Specifically, when a robot grips a winding package to be packaged from a winding package carriage by N hand grips, if all the hand grips succeed in gripping the winding package to be packaged, M = N. On the other hand, if some of the hand grips cannot successfully grip the winding package to be packaged, 0≤M<N, and M is an integer.

[0023] In step S102, after the robot places M winding packages to be packaged one-to-one on M winding package trays included in a target tray group installed on the main line of the production line, if it is determined based on an image representing the gripping result that there is a tray to be recovered in the target tray group, the tray diversion device is controlled to divert the tray to be recovered from the main line of the production line to the bypass line of the production line, thereby performing the recovery process of the tray to be recovered.

[0024] Here, the tray to be recovered is an empty first tray to be recovered or a second tray to be recovered in which the placed winding package to be packaged is preliminarily evaluated as a winding package of a predetermined grade. Here, the winding package of the predetermined grade may be a winding package product of a non-AA grade, that is, a winding package product downgraded from the default AA grade to another grade (for example, A grade, B grade, C grade) for quality reasons. The quality reasons may include the existence of causes such as yarn breakage, oil stain, yarn entanglement, poor forming, and paper tube breakage.

[0025] In the embodiments of the present disclosure, the target tray group usually includes a fixed number of winding package trays. For example, the target tray group includes a total of N winding package trays, and the N winding package trays may correspond one-to-one to N hand grips. Therefore, when the robot grips the winding package to be packaged from the winding package trolley by N hand grips, if some hand grips cannot successfully grip the winding package to be packaged (in this case, 0 ≦ M < N), when the robot places M winding packages to be packaged one-to-one on M winding package pallets included in the target tray group installed on the main line of the production line, there will be some empty first recovery pallets in the target tray group. Also, in the embodiments of the present disclosure, among the M winding packages to be packaged gripped by the robot from the winding package vehicle by N hand grips, there may also be a second recovery target tray initially evaluated as a winding package of a predetermined grade.

[0026] According to the tray recovery method provided by the embodiments of the present disclosure, after the robot grips M winding packages to be packaged from the winding package trolley by N hand grips, the image acquisition device is controlled to capture an image in the direction where the N hand grips are located, and an image representing the gripping result is obtained. After the robot places the M winding packages to be packaged one-to-one on M winding package trays included in the target tray group installed on the main line of the production line, if it is determined based on the image representing the gripping result that there is a recovery target tray in the target tray group, the tray diversion device is controlled to divert the recovery target tray from the main line of the production line to the bypass line of the production line, thereby performing the recovery process of the recovery target tray. Thereby, automatic recovery of the recovery target tray can be realized, a large amount of human resources can be saved compared with the current manual recovery method, and the recovery efficiency of the recovery target tray can be improved.

[0027] In some optional embodiments, the electronic device further communicates with the robot, and step S101 of “controlling the image collection device to capture images in the direction in which the N hand grips are located and obtaining an image representing the grasping result” can include the following steps:

[0028] In step S1011, the N handgrips are controlled to rotate relative to the image acquisition device.

[0029] In one example, the N handgrips of the robot can be coupled to the body structure of the robot via the same mechanical axis, based on which the embodiments of the present disclosure can rotate such that the N handgrips rotate as a whole relative to the image acquisition device.

[0030] In step S1012, the image acquisition device is controlled to capture images in the direction in which the N hand grips are located while the N hand grips are rotating relative to the image acquisition device, thereby obtaining Z images to be processed.

[0031] Here, Z≧2 and Z is an integer.

[0032] The image collection device captures images of the N handgrips in a certain direction as the N handgrips rotate relative to the image collection device, thereby obtaining Z images to be processed, where the Z images to be processed correspond to different angles of view of the N handgrips.

[0033] In step S1013, an image showing the gripping result is obtained based on the Z processing target images.

[0034] In one example, Z target images can be processed by operations such as decomposing and stitching to obtain an image showing the gripping result.

[0035] 2A, for example, the robot 201 may include five hand grips (a first hand grip 2021, a second hand grip 2022, a third hand grip 2023, a fourth hand grip 2024, and a fifth hand grip 2025 (i.e., N=5 in this example)). Due to manual picking, misplacement, etc., the wound yarn package cart 203 may have four wound yarn packages to be packaged (a first wound yarn package 2041 to be packaged, a third wound yarn package 2042 to be packaged, and a fourth wound yarn package 2043 to be packaged). 2B , when the robot 201 grasps the wound yarn packages to be packaged from the wound yarn package carriage 203 with the five hand grips, the first hand grip 2021 successfully grasps the first wound yarn package 2041 to be packaged, the second hand grip 2022 successfully grasps the wound yarn package to be packaged, and the third hand grip 2023 successfully grasps the third wound yarn package 2042 to be packaged. 43, the fourth hand grip 2024 succeeds in gripping the fourth wound yarn package 2044 to be packaged, and the fifth hand grip 2025 succeeds in gripping the fifth wound yarn package 2045 to be packaged. After the robot 201 grips the four wound yarn packages to be packaged from the wound yarn package cart 203 with the five hand grips, the mechanical shaft 205 for connecting the five hand grips to the main body structure of the robot 201 can be rotated to rotate the five hand grips as a whole relative to the image collecting device 206. Specifically, the mechanical shaft 205 can be rotated along the direction of "A1 -> A2" to rotate the five hand grips as a whole relative to the image collecting device 206. In the process of the five hand grips rotating relative to the image collecting device 206, the image collecting device 206 can be controlled to capture images in the direction in which the five hand grips are located, to obtain Z images to be processed, and it can be ensured that the Z images to be processed correspond to different angles of view of the N hand grips.

[0036] Taking Z=2 as an example, when the five hand grips rotate at a first angle as shown in FIG. 2B relative to the image collecting device 206, the image collecting device 206 can be controlled to capture an image in the direction in which the five hand grips are located, thereby obtaining a first processing target image. The first processing target image is mainly used to represent image features on the first side surfaces of the four wound yarn packages to be packaged. That is, as shown in FIG. 2C, the first processing target image can include a single-angle image SD11 for representing image features on the first side surface of the first wound yarn package to be packaged 2041, a single-angle image SD31 for representing image features on the first side surface of the third wound yarn package to be packaged 2043, a single-angle image SD41 for representing image features on the first side surface of the fourth wound yarn package to be packaged 2044, and a single-angle image SD51 for representing image features on the first side surface of the fifth wound yarn package to be packaged 2045. Furthermore, since the second hand grip 2022 does not succeed in gripping the wound yarn package to be packaged, the second hand grip 2022 is not occluded by the wound yarn package to be packaged, so that the first image to be processed can include a single angle of view image SZ21 to represent image features at the first side of the second hand grip 2022.

[0037] Similarly, when the five hand grips rotate at a second angle as shown in FIG. 2D relative to the image collecting device 206, the image collecting device 206 is controlled to capture an image in the direction in which the five hand grips are located, thereby obtaining a second image to be processed. The second image to be processed is mainly used to represent image features on the second side opposite to the first side of the four wound yarn packages to be packaged. That is, the second image to be processed can include, specifically, a single-angle image SD12 for representing image features on the second side of the first wound yarn package to be packaged 2041, a single-angle image SD32 for representing image features on the second side of the third wound yarn package to be packaged 2043, a single-angle image SD42 for representing image features on the second side of the fourth wound yarn package to be packaged 2044, and a single-angle image SD52 for representing image features on the second side of the fifth wound yarn package to be packaged 2045, as shown in FIG. 2E. Furthermore, since the second hand grip 2022 does not succeed in gripping the wound yarn package to be packaged, the second hand grip 2022 is not occluded by the wound yarn package to be packaged, thereby allowing the second image to be processed to include a single angle of view image SZ22 for representing image features at the second side of the second hand grip 2022.

[0038] It should also be noted that in the embodiment of the present disclosure, both the first processing target image shown in Fig. 2B and the second processing target image shown in Fig. 2D are pre-processed, where the pre-processing can include cropping, noise removal, feature enhancement, etc., and the embodiment of the present disclosure is not limited thereto.

[0039] After obtaining the first processing target image and the second processing target image, the Z processing target images can be processed by operations such as decomposition and joining to obtain an image showing the gripping result.

[0040] According to the above steps included in step S101, in the embodiment of the present disclosure, after the robot grasps M wound yarn packages to be packaged from the yarn package cart with the N hand grips, the N hand grips are controlled to rotate relative to the image collecting device, and in the process of the N hand grips rotating relative to the image collecting device, the image collecting device is controlled to capture images in the direction in which the N hand grips are located, to obtain Z images to be processed (the Z images to be processed correspond to different angles of view of the N hand grips), and a feature image representing the grasping result can be obtained based on the Z images to be processed. In this way, the image representing the grasping result represents the M wound yarn packages to be packaged grasped by the N hand grips in all directions, and avoids missing features, thereby reducing the rate of misjudgment of the tray to be collected and improving the accuracy of tray collection.

[0041] Further, in some optional embodiments, step S1013 may include the following steps.

[0042] In step S10131, N single-angle images are cut out from each of the Z processing target images to obtain N×Z single-angle images.

[0043] Here, the N single-angle images correspond one-to-one to the N hand grips.

[0044] In step S10132, Z single-angle images of N×Z single-angle images that correspond to the same hand grip are stitched together to obtain N multiple-angle images.

[0045] In step S10133, an image showing the gripping result is obtained based on N multiple angle images.

[0046] In one example, N multiple view images can be tiled according to a robotic ordering scheme for the N handgrips to obtain an image representative of the grasping result.

[0047] Following the above example (in this example, N=5 and Z=2), after obtaining the first processing target image shown in FIG. 2C and the second processing target image shown in FIG. 2E, five single-angle images, i.e., single-angle image SD11, single-angle image SZ21, single-angle image SD31, single-angle image SD41, and single-angle image SD51, are cut out from the first processing target image, and similarly, five single-angle images, i.e., single-angle image SD21, single-angle image SZ22, single-angle image SD32, single-angle image SD42, and single-angle image SD52, are cut out from the second processing target image, and further, two single-angle images corresponding to the same hand grip among the 5×2=10 single-angle images are cut out. The images are stitched together to obtain five multiple-angle images, namely, multiple-angle images SD11 & SD12 corresponding to the first hand grip 2021, multiple-angle images SZ21 & SZ22 corresponding to the second hand grip 2022, multiple-angle images SD31 & SD32 corresponding to the third hand grip 2023, multiple-angle images SD41 & SD42 corresponding to the fourth hand grip 2024, and multiple-angle images SD51 & SD52 corresponding to the fifth hand grip 2025. These five multiple-angle images are then arranged in a tiled pattern in accordance with the robot's ordering scheme for the five hand grips to obtain an image showing the grasping result, as shown specifically in Figure 2F.

[0048] By the above steps included in step S1013, the embodiment of the present disclosure provides a specific method for acquiring an image representing the gripping result, which has a simple process and can improve the execution efficiency of the tray collection method.

[0049] After performing step S101, the robot may be controlled to place the M wound yarn packages to be packaged on the M wound yarn package trays included in the target tray group installed on the main line of the production line in one-to-one correspondence. Continuing with the above example (in this example, M=4), referring to Figures 2G and 2H, it is assumed that the target tray group installed on the main line 207 of the production line includes five wound yarn package trays, namely, a first wound yarn package tray 2081 corresponding to the first hand grip 2021, a second wound yarn package tray 2082 corresponding to the second hand grip 2022, a third wound yarn package tray 2083 corresponding to the third hand grip 2023, a fourth wound yarn package tray 2084 corresponding to the fourth hand grip 2024, and a fifth wound yarn package tray 2085 corresponding to the fifth hand grip 2025. The robot 201 can then be controlled to place the first wound yarn package 2041 to be packaged on the first wound yarn package tray 2081, the third wound yarn package 2043 to be packaged on the third wound yarn package tray 2083, the fourth wound yarn package 2044 to be packaged on the fourth wound yarn package tray 2084, and the fifth wound yarn package 2045 to be packaged on the second wound yarn package tray 2085.

[0050] Also, as described above, in an embodiment of the present disclosure, the image representing the gripping result includes N multiple-angle images, and the N multiple-angle images may correspond one-to-one to the N hand grips, and the target tray group includes N wound thread package trays, and the N wound thread package trays may correspond one-to-one to the N hand grips, so that a one-to-one correspondence can be established between the N wound thread package trays and the N multiple-angle images.

[0051] Continuing from the previous example, the images representing the gripping results include five multiple-angle images, namely, multiple-angle images SD11 & SD12 corresponding to the first hand grip 2021, multiple-angle images SZ21 & SZ22 corresponding to the second hand grip 2022, multiple-angle images SD31 & SD32 corresponding to the third hand grip 2023, multiple-angle images SD41 & SD42 corresponding to the fourth hand grip 2024, and multiple-angle images SD51 & SD52 corresponding to the fifth hand grip 2025. The target tray group is five. 2F, 2G, and 2H, the system includes three wound yarn package trays, a first wound yarn package tray 2081 corresponding to the first hand grip 2021, a second wound yarn package tray 2082 corresponding to the second hand grip 2022, a third wound yarn package tray 2083 corresponding to the third hand grip 2023, a fourth wound yarn package tray 2084 corresponding to the fourth hand grip 2024, and a fifth wound yarn package tray 2085 corresponding to the fifth hand grip 2025. As a result, a one-to-one correspondence between the N wound yarn package trays and the N multiple angle images can be established.

[0052] Multiple angle images SD11 & SD12←→1st yarn package tray 2081, Multiple angle images SZ21 & SZ22 ←→ 2nd yarn packaging tray 2082, Multiple angle images SD31 & SD32←→ 3rd yarn package tray 2083, Multiple angle images SD41 & SD42←→ 4th winding yarn package tray 2084, Multiple angle images SD51 & SD52 ←→ 5th roll yarn package tray 2085.

[0053] The above correspondence can be specifically shown in FIG. 2I.

[0054] Based on the above, in some optional embodiments, step S102 "determining that a tray to be collected is present in the target tray group based on an image representing the gripping result" may include the following steps.

[0055] In step S1021, if it is determined that no wound thread package image representing the wound thread package to be packaged is present in each of the N multiple angle images, it is determined that a collection target tray is present in the target tray group, and the wound thread package tray among the N wound thread package trays corresponding to the multiple angle image is designated as the first collection target tray.

[0056] In one example, it can be determined whether a wound yarn package image representing the wound yarn package to be packaged is present in the multiple angle images by using a connected region analysis method (i.e., Blob analysis method, also called BlobAnalysis), a template matching method, a deep learning method, etc.

[0057] In addition, in the embodiment of the present disclosure, after determining the first tray to be collected from the N wound yarn package trays, a tray identification of the first tray to be collected may be recorded, and the tray identification of the first tray to be collected may be sent to the tray diversion device, where the tray identification may be a tray code.

[0058] In step S1022, if it is determined that a wound yarn package image representing a wound yarn package to be packaged is present in the multiple angle image, and if it is determined using a defect detection network that the wound yarn package image has defect features, it is determined that a tray to be collected is present in the target tray group, and the wound yarn package tray corresponding to the multiple angle image among the N wound yarn package trays is designated as the second target tray to be collected.

[0059] Here, the defect feature may include at least one of a thread breakage feature, an oil stain feature, a thread hanging feature, a molding defect, a paper core breakage, and the like.

[0060] In one example, the defect detection network may be implemented by combining a U-net network and a residual network (i.e., ResNet), or may be implemented based on a Convolutional Neural Network (CNN) or other suitable neural network, and the embodiments of the present disclosure are not limited thereto. Here, the U-net network is a neural network model with a "U" shape structure, and the left side of the U-net network is used to realize image feature extraction, and the right side is used to realize image feature upsampling processing.

[0061] In addition, in an embodiment of the present disclosure, after the second tray to be collected is determined from the N wound yarn package trays, the tray identification of the second tray to be collected may be recorded, and the tray identification of the second tray to be collected may be transmitted to the tray diversion device.

[0062] Continuing from the previous example, the image showing the gripping result includes five multiple-angle images, the target tray group includes five wound yarn package trays, and the five multiple-angle images and the five wound yarn package trays have a one-to-one correspondence as shown in Fig. 2I. In this case, when performing step S1021, it can be determined that no wound yarn package image representing the wound yarn package to be packed is present in the multiple-angle images SZ21 & SZ22, so it can be determined that a collection target tray is present in the target tray group, and the second wound yarn package tray 2082 corresponding to the multiple-angle images SZ21 & SZ22 among the five wound yarn package trays can be set as the first collection target tray, the tray identification of the first collection target tray can be recorded, and the tray identification of the first collection target tray can be transmitted to the tray diversion device. When step S1022 is executed, it can be determined that there are defect features in both the multiple angle of view images SD41 & SD42 and the multiple angle of view images SD51 & SD52, so it is determined that a tray to be collected is present in the target tray group, and the fourth wound yarn package tray 2084 corresponding to the multiple angle of view images SD41 & SD42 and the fifth wound yarn package tray 2085 corresponding to the multiple angle of view images SD51 & SD52 among the five wound yarn package trays are designated as second trays to be collected, the tray marker of the second tray to be collected is recorded, and the tray marker of the second tray to be collected can be sent to the tray diversion device.

[0063] In the embodiment of the present disclosure, by the above steps included in step S102, for each of the multiple angle of view images, it is determined whether the corresponding tray to be collected is a first tray to be collected, and if it is determined that the corresponding tray to be collected is not the first tray to be collected, it is determined whether the corresponding tray to be collected is a second tray to be collected, thereby reducing the rate of misjudgment of the tray to be collected by the two-time cascaded judgment process, and improving the accuracy of tray collection. Meanwhile, in the embodiment of the present disclosure, the tray to be collected is identified using a machine vision method, so that not only can the execution efficiency of the tray collection method be improved, but also the accuracy of tray collection can be improved.

[0064] Referring to FIG. 3A, in one embodiment, the defect detection network can include a first network layer, a second network layer, and a third network layer.

[0065] The first network layer combines a channel attention mechanism and a spatial attention mechanism to perform feature extraction on the yarn package image to obtain K feature representation maps with different scales, where K≧2 and K is an integer. The second network layer performs feature fusion processing based on the K feature representation maps to obtain K fused feature maps, where the K fused feature maps have a one-to-one correspondence with the K feature representation maps. The third network layer obtains a defect detection result based on the K fused feature maps, where the defect detection result is used to represent whether the yarn package image has a defect feature.

[0066] Referring to FIG. 3B, in one embodiment, the first network layer may include a first feature extraction module, a second feature extraction module, and a third feature extraction module cascaded together (i.e., K=3 in this example).

[0067] Here, the first feature extraction module is for processing the input image features (here, the wound yarn package images, for example, the wound yarn package images in the multiple viewing angle images SD41 & SD42 shown in FIG. 2F and FIG. 2I) to obtain a first feature representation diagram. Furthermore, the first feature extraction module can include a first convolution processing unit and a first attention unit that are cascaded. Here, the first convolution processing unit is for convolution processing the input image features, and the first attention unit is for extracting the input image features by combining the channel attention mechanism and the spatial attention mechanism.

[0068] The second feature extraction module is for processing the input image features (first feature representation diagram) to obtain a second feature representation diagram. Furthermore, the second feature extraction module can include a second convolution processing unit and a second attention unit cascaded together, where the second convolution processing unit is for convolution processing the input image features, and the second attention unit is for combining the channel attention mechanism and the spatial attention mechanism to extract features from the input image features.

[0069] The third feature extraction module is for processing the input image features (second feature representation diagram) and obtaining a third feature representation diagram. Furthermore, the third feature extraction module may include a cascaded third convolution processing unit, a Spatial Pyramid Pooling Fast (SPPF) unit, and a third attention unit. Here, the convolution processing unit is for convolution processing the input image features, the SPPF unit is for fusing image features of different receptive fields, and the third attention unit is for combining the channel attention mechanism and the spatial attention mechanism to extract features from the input image features.

[0070] Referring to FIG. 3C, in this example, the first attention unit, the second attention unit, and the third attention unit may have the same structure. For example, the third subunit may include a first subunit, a second subunit, and a third subunit in parallel, and may include a fusion subunit for fusing image features output by the first subunit, the second subunit, and the third subunit, and an activation subunit for normalizing the image features output by the fusion subunit (for example, for normalizing the image features output by the fusion subunit using a Sigmoid function). Here, the third subunit may include a first internal unit and a second internal unit cascaded together, the first internal unit and the first subunit may have the same structure, the second internal unit and the second subunit may have the same structure, the first internal unit and the first subunit are for extracting features from input image features based on a channel attention mechanism, and the second internal unit and the second subunit are for extracting features from input image features based on a spatial attention mechanism.

[0071] More specifically, referring to FIG. 3D, the first internal unit and the first subunit are: A first internal pooling unit and a second internal pooling unit connected in parallel; a first internal convolution unit connected in series to a first internal pooling unit and a second internal pooling unit in a parallel structure; a third inner pooling unit and a fourth inner pooling unit cascaded to the first inner convolution unit to form a parallel structure; an internal fusion unit for fusion processing of the image features output by the third internal pooling unit and the fourth internal pooling unit; and a first internal activation unit for normalizing the image features output by the internal fusion unit (e.g., normalizing the image features output by the internal fusion unit using a Sigmoid function).

[0072] Here, the first internal pooling unit is for performing average pooling processing on the input image features. The second internal pooling unit is for performing maximum pooling processing on the input image features. The first internal convolution unit may be a convolution unit realized based on a Depthwise Convolution (DW) layer, or may be a convolution unit realized based on a Depthwise Separable Convolution (DSC) layer. The third internal pooling unit is for performing average pooling processing on the input image features. The fourth internal pooling unit is for performing maximum pooling processing on the input image features. Here, the DSC layer can be constructed by combining a DW layer and a Pointwise Convolution (PW) layer. In practical application, the first inner convolution unit can perform deep channel feature extraction on the input image features, thereby improving the channel attention ability and feature representation ability, and because the first inner convolution unit has weight sharing characteristics, it can also reduce some unnecessary parameter amounts in the neural network calculation process.

[0073] Specifically, referring to FIG. 3E, the second internal unit and the second subunit include: a fifth internal pooling unit and a sixth internal pooling unit connected in cascade; The sixth inner pooling unit may include a channel combining unit, a second inner convolution unit, and a second inner activation unit, which are cascaded with the sixth inner pooling unit and form a cascaded structure.

[0074] Here, the fifth inner pooling unit is for average pooling the input image features. The sixth inner pooling unit is for max pooling the input image features. The channel combining unit is for splicing the input image features according to the channels. The second inner convolution unit may be a convolution unit realized based on a standard convolution network, or may be realized directly using a CNN. The second inner activation unit is for normalizing the image features output by the second inner convolution unit (for example, using a Sigmoid function to normalize the image features output by the second inner convolution unit).

[0075] Referring to FIG. 3F, illustratively, the second network layer may include an upsampling module and a downsampling module.

[0076] Here, the upsampling module may include a first Ghost Shuffle Convolution (GSC) unit, a first tensor concat (Concat) unit, a first bottleneck unit, a second GSC unit, and a second Concat unit, which are cascaded together. Here, the first GSC unit is for convolution processing of the third feature representation diagram. The first Concat unit is for tensor concatenation of the second feature representation diagram and the upsampled image features output by the first GSC unit to obtain output image features, and after upsampling, input the output image features to the first bottleneck unit and the second GSC unit for processing. The second Concat unit is for tensor concatenation of the first feature representation diagram and the upsampled image features output by the second GSC unit to obtain output image features.

[0077] The downsampling module may include a second bottleneck unit, a third GSC unit, a third Concat unit, a third bottleneck unit, a fourth GSC unit, a fourth Concat unit, and a fourth bottleneck unit, which are cascaded together. Here, the second bottleneck unit processes the image features output by the second GSC unit to obtain a first fusion feature map. The third GSC unit convolves the first fusion feature map. The third Concat unit performs tensor combination on the image features output by the second GSC unit and the downsampled image features output by the third GSC unit to obtain output image features, which are then downsampled and input to the third bottleneck unit and the fourth GSC unit for processing. The fourth Concat unit performs tensor combination on the image features output by the first GSC unit and the downsampled image features output by the fourth GSC unit to obtain output image features. The fourth bottleneck unit processes the downsampled image features output by the fourth Concat unit to obtain a third fused feature map.

[0078] In this example, the first bottleneck unit, the second bottleneck unit, the third bottleneck unit, and the fourth bottleneck unit may have the same structure. More specifically, referring to FIG. 3G, the first bottleneck unit, the second bottleneck unit, the third bottleneck unit, and the fourth bottleneck unit may have the same structure. A bottleneck subunit consisting of a cascaded first convolution subunit, a bottleneck layer (e.g., a Bottleneck layer) and a Channel Split and Concatenate (CSP) layer; a second convolution subunit connected in parallel to the first convolution subunit and the bottleneck subunit forming a cascade structure; a Concat subunit that receives the image features output by the bottleneck subunit and the second convolution subunit; a third convolution subunit cascaded to the Concat subunit.

[0079] Here, the first convolution subunit, the second convolution subunit, and the third convolution subunit may be convolution units realized based on a standard convolution network. The Bottleneck layer is for reducing the amount of unnecessary parameters during neural network calculation. The CSP layer is for performing operations such as split, concat, and shuffle on the input image features.

[0080] Referring to Figure 3H, illustratively, the third network layer may include a first detection unit for connecting to the second bottleneck unit, a second detection unit for connecting to the third bottleneck unit, and a third detection unit for connecting to the fourth bottleneck unit.

[0081] Here, the first detection unit, the second detection unit and the third detection unit may be convolution units realized based on a standard convolutional network.

[0082] A defect detection result can be obtained based on the image features output by the first detection unit, the second detection unit and the third detection unit.

[0083] In the embodiment of the present disclosure, the defect detection network includes a first network layer, a second network layer, and a third network layer. The first network layer combines a channel attention mechanism and a spatial attention mechanism to extract features from a wound yarn package image and obtain K feature representation maps with different scales, so as to ensure the feature representation ability of the K feature representation maps. In this way, the second network layer is used to perform feature fusion processing based on the K feature representation maps to obtain K fused feature maps, and the third network layer is used to obtain a defect detection result based on the K fused feature maps, so that the reliability of the defect detection result can be improved and the accuracy of tray collection can be further improved.

[0084] In addition, in an embodiment of the present disclosure, the defect detection network can be trained according to the following flow.

[0085] (1) Obtaining a yarn package image sample and a data label corresponding to the yarn package image sample, the data label being used to indicate whether or not there is yarn breakage, oil stains, yarn hanging, molding defects, paper core breakage, etc. in the yarn package image.

[0086] (2) Input a sample image of the wound yarn package into the early detection model, and obtain the image features output by the early detection model.

[0087] (3) Obtain defect prediction results based on the image features output by the initial detection model.

[0088] (4) Using a preset loss function, a loss value between the defect prediction result and the data label is calculated; if the loss value does not satisfy the preset convergence condition, the model parameters of the initial detection model are adjusted; if the loss value satisfies the preset convergence condition, the initial detection model is the defect detection model.

[0089] Here, the preset loss function may be a Shape and Intersection over Union Loss (SIoU).

[0090] Further, in an embodiment of the present disclosure, the tray diversion device includes a first diversion device and a second diversion device installed on the main line of the production line, and the bypass line of the production line includes a first bypass line and a second bypass line. Based on this, in some optional embodiments, "controlling the tray diversion device to divert the tray to be collected from the main line of the production line onto the bypass line of the production line" in step S102 may include the following.

[0091] In step S1021, a first diversion device is controlled to divert the first tray to be collected from the main line of the production line onto a first bypass line of the production line.

[0092] Here, the first diversion device can receive the tray identifier of the first tray to be collected transmitted by the electronic device as the first target identifier.

[0093] When the N wound yarn package trays included in the target tray group pass through the first diversion device, the first diversion device determines whether there is a collection target tray among the N wound yarn package trays whose tray mark matches the first target mark, and if there is a collection target tray among the N wound yarn package trays whose tray mark matches the first target mark, it can identify the collection target tray as a first collection target tray and divert the first collection target tray from the main line of the production line to a first bypass line of the production line.

[0094] In step S1022, the second diversion device is controlled to divert the second collection target tray from the main line of the production line onto a second bypass line of the production line.

[0095] Here, the second diversion device can receive the tray identifier of the second tray to be collected transmitted by the electronic device as the second target identifier.

[0096] When the other wound yarn package trays, excluding the first collection target tray, among the N wound yarn package trays included in the target tray group pass through the second diversion device, the second diversion device determines whether there is a collection target tray among the other wound yarn package trays whose tray mark matches the second target mark, and if there is a collection target tray among the other wound yarn package trays whose tray mark matches the second target mark, it can identify the collection target tray as a second collection target tray and divert the second collection target tray from the main line of the production line to a second bypass line of the production line.

[0097] 4A, exemplarily, the tray diversion device includes a first diversion device 4021 and a second diversion device 4022 installed on the main line 401 of the production line, and the bypass line of the production line includes a first bypass line 4031 and a second bypass line 4032. The target tray group includes five wound yarn package trays (i.e., N=5), namely, a first wound yarn package tray 4041, a second wound yarn package tray 4042, a third wound yarn package tray 4043, a fourth wound yarn package tray 4044, and a fifth wound yarn package tray 4045. Here, the second wound yarn package tray 4042 is the first target collection tray, and the fourth wound yarn package tray 4044 and the fifth wound yarn package tray 4045 are the second target collection trays.

[0098] Now, when the five wound yarn package trays included in the target tray group pass through the first diversion device 4021, the first diversion device 4021 reads the tray mark of the second wound yarn package tray 4042 using radio frequency identification (RFID), and then determines that the tray mark of the second wound yarn package tray 4042 matches the first target mark, identifies the second wound yarn package tray 4042 as the first collection target tray, and can divert the first collection target tray from the main line 401 of the production line to the first bypass line 4031 of the production line. When the other wound yarn package trays (i.e., the first wound yarn package tray 4041, the third wound yarn package tray 4043, the fourth wound yarn package tray 4044, and the fifth wound yarn package tray 4045) among the five wound yarn package trays included in the target tray group, excluding the first collection target tray (i.e., the second wound yarn package tray 4042), pass through the second diversion device 4022, the second diversion device 4022 reads the tray marks of the other wound yarn package trays using RFID, and then determines that the tray marks of the fourth wound yarn package tray 4044 and the fifth wound yarn package tray 4045 among the other wound yarn package trays match the second target mark, confirms that the fourth wound yarn package tray 4044 and the fifth wound yarn package tray 4045 are the second collection target trays, and diverts the second collection target trays from the main line 401 of the production line to the second bypass line 4032 of the production line.

[0099] Through the above steps included in step S102, the first and second trays to be collected can be diverted onto different bypass lines for different processing, and the accuracy of tray collection can be improved. For example, the first trays to be collected diverted onto the first bypass line can finally be diverted to a tray collection location to form a new target tray group and can be used to enter the main line source again. Furthermore, for example, the second trays to be collected diverted onto the second bypass line can proceed to a re-inspection process to determine whether the wound yarn package to be packaged placed on the second tray to be collected is indeed a wound yarn package of a predetermined grade, and misjudgment can be avoided.

[0100] As described above, in the embodiment of the present disclosure, the second collection target tray diverted onto the second bypass line can proceed to a re-inspection process to determine whether the wound yarn package to be packaged placed on the second collection target tray is indeed a wound yarn package of a specified grade, and to avoid erroneous determination. In this case, in the embodiment of the present disclosure, the tray diverting device can further include a third diverting device placed on the second bypass line, and the bypass line further includes a third bypass line. Based on this, in some optional embodiments, after performing step S1022, step S102 can also include any of the following steps.

[0101] In step S1023, the third diversion device is controlled to divert the second tray to be collected from the second bypass line to a tray collection location if the result of re-inspection of the second tray to be collected indicates that the wound yarn package to be packaged placed on the second tray to be collected is a wound yarn package of a specified grade.

[0102] Here, the purpose of re-inspecting the second tray to be collected is to determine whether the wound yarn package to be packed placed on the second tray to be collected is indeed a wound yarn package of a specified grade, and to avoid misjudgment. For example, the wound yarn package to be packed placed on the wound yarn package tray may have a cleanable yarn on its surface. If it is determined that the target tray group contains a tray to be collected based on the image showing the gripping result, the wound yarn package tray may be misjudged as the second tray to be collected. In addition, in the embodiment of the present disclosure, the re-inspection process may be realized using a machine visual method or may be performed manually, and the embodiment of the present disclosure is not limited thereto.

[0103] In one example, the re-inspection result can be recorded by scanning the tray mark. Specifically, when the second tray to be collected is re-inspected and it is determined that the wound yarn package to be packed placed on the second tray to be collected is a wound yarn package of a predetermined grade, the tray mark of the second tray to be collected can be scanned as a third target mark, the third target mark can be sent to the electronic device for recording, and the third target mark can be sent to the third diversion device by the electronic device. In this way, when the second tray to be collected passes through the third diversion device, the third diversion device determines whether the tray mark of the second tray to be collected matches the third target mark, and if the tray mark of the second tray to be collected matches the third target mark information, it is determined that the re-inspection result of the second tray to be collected represents that the wound yarn package to be packed placed on the second tray to be collected is a wound yarn package of a predetermined grade, and the second tray to be collected can be diverted from the second bypass line to the tray collection location.

[0104] In another example, the re-inspection result can be indirectly recorded by removing the wound yarn package to be packed. Specifically, when the second collection tray is re-inspected and it is determined that the wound yarn package to be packed placed on the second collection tray is a wound yarn package of a predetermined grade, the wound yarn package to be packed placed on the second collection tray can be removed from the second collection tray, and the second collection tray can be an empty second collection tray. Then, when the empty second collection tray passes through the empty tray detection device on the second bypass line and the empty tray detection device detects that the second collection tray is empty, the tray mark of the empty second collection tray can be recorded as a fourth target mark, the fourth target mark can be transmitted to the electronic device for recording, and the fourth target mark can be transmitted to the third diversion device by the electronic device. In this way, when the second tray to be collected passes through the third diverting device, the third diverting device determines whether the tray mark of the second tray to be collected matches the fourth target mark, and if the tray mark of the second tray to be collected matches the fourth target mark, it determines that the re-inspection result of the second tray to be collected represents that the wound yarn package to be packaged placed on the second tray to be collected is a wound yarn package of a predetermined grade, and the second tray to be collected can be diverted from the second bypass line to a tray collection location. Here, the empty tray detection device can detect whether the second tray to be collected is empty using a mechanical visual method.

[0105] In step S1024, the third diversion device is controlled to divert the second tray to be collected from the second bypass line to a tray collection location if the result of re-inspection of the second tray to be collected indicates that the wound yarn package to be packaged placed on the second tray to be collected is a wound yarn package of a specified grade.

[0106] In one example, the re-inspection result can be recorded by scanning the tray mark, and when the second tray to be collected passes through the third diversion device, the third diversion device determines whether the tray mark of the second tray to be collected matches the third target mark, and if the tray mark of the second tray to be collected does not match the third target mark, it determines that the re-inspection result of the second tray to be collected indicates that the wound yarn package to be packaged placed on the second tray to be collected is not a wound yarn package of a certain grade (e.g., AA grade wound yarn package product), and diverts the second tray to be collected from the second bypass line to the third bypass line, and returns from the third bypass line to the main line of the production line to proceed to the next process, such as weighing, appearance inspection, bagging, palletizing, strapping device, film wrapping, and marking.

[0107] In another example, the re-inspection result is indirectly recorded by removing the wound yarn package to be packed, and when the second tray to be collected passes through the third diversion device, the third diversion device determines whether the tray mark of the second tray to be collected matches the fourth target mark, and if the tray mark of the second tray to be collected does not match the fourth target mark, it determines that the re-inspection result of the second tray to be collected indicates that the wound yarn package to be packed placed on the second tray to be collected is not a wound yarn package of a certain grade (e.g., AA grade wound yarn package product), and diverts the second tray to be collected from the second bypass line to the third bypass line, and then returns from the third bypass line to the main line of the production line to proceed to the next process, such as weighing, appearance inspection, bagging, palletizing, strapping device, film wrapping, and marking.

[0108] Continuing from the example shown in FIG. 4A, referring to FIG. 4B, in this example, it is assumed that the re-inspection result is recorded by scanning the tray mark, and the re-inspection result of the second target tray 4044 indicates that the wound yarn package to be packed placed on the second target tray 4044 is a wound yarn package of a predetermined grade, and the re-inspection result of the second target tray 4045 indicates that the wound yarn package to be packed placed on the second target tray 4045 is not a wound yarn package of a predetermined grade. In addition, when the second target tray 4044 is re-inspected and it is determined that the wound yarn package to be packed placed on the second target tray 4044 is a wound yarn package of a predetermined grade, the tray mark of the second target tray 4044 has already been scanned and is taken as a third target mark, and the third target mark is sent to the electronic device for recording, and the third target mark is sent to the third diversion device 4023 by the electronic device.

[0109] Then, referring to FIG. 4C, when the second tray 4044 to be collected passes through the third diversion device 4023, the third diversion device 4023 can use RFID to read the tray mark of the second tray 4044 to be collected, and then determine that the tray mark of the second tray 4044 to be collected matches the third target mark, and determine that the re-inspection result of the second tray 4044 to be collected indicates that the wound yarn package to be packaged placed on the second tray 4044 to be collected is a wound yarn package of a specified grade, and diverts the second tray 4044 to be collected from the second bypass line 4032 to the tray collection location. When the second tray 4045 to be collected passes through the third diversion device 4023, the third diversion device 4023 can read the tray mark of the second tray 4045 to be collected by RFID and determine that the tray mark of the second tray 4045 to be collected does not match the third target mark, and determine that the re-inspection result of the second tray 4045 to be collected indicates that the wound yarn package to be packed placed on the second tray 4045 to be collected is not a wound yarn package of a predetermined grade, and divert the second tray 4045 to be collected from the second bypass line 4032 to the third bypass line 4033, and return from the third bypass line 4033 to the main line 401 of the production line, and then proceed to the next process, such as weighing, appearance inspection, bagging, palletizing, strapping device, film wrapping, and marking.

[0110] Continuing from the example shown in Fig. 4A, referring to Fig. 4D, in this example, it is assumed that the reinspection result is indirectly recorded by removing the wound yarn package to be packed, and the reinspection result of the second collection tray 4044 indicates that the wound yarn package to be packed placed on the second collection tray 4044 is a wound yarn package of a predetermined grade, and the reinspection result of the second collection tray 4045 indicates that the wound yarn package to be packed placed on the second collection tray 4045 is not a wound yarn package of a predetermined grade. In addition, when the second collection tray 4044 is reinspected and it is determined that the wound yarn package to be packed placed on the second collection tray 4044 is a wound yarn package of a predetermined grade, the wound yarn package to be packed placed on the second collection tray 4044 has already been removed from the second collection tray 4044, and the second collection tray 4044 is now an empty second collection tray 4044. The empty second tray 4044 to be collected then passes through the empty tray detection device 405 on the second bypass line 4032, and if the empty tray detection device 405 detects that the second tray 4044 to be collected is empty, it can record the tray indicator of the empty second tray 4044 to be collected as a fourth target indicator, transmit the fourth target indicator to an electronic device for recording, and then transmit the fourth target indicator to the third diversion device 4023 by the electronic device.

[0111] Then, referring to FIG. 4E, when the second tray 4044 to be collected passes through the third diversion device 4023, the third diversion device 4023 can use RFID to read the tray mark of the second tray 4044 to be collected, and then determine that the tray mark of the second tray 4044 to be collected matches the fourth target mark, and determine that the re-inspection result of the second tray 4044 to be collected indicates that the wound yarn package to be packaged placed on the second tray 4044 to be collected is a wound yarn package of a specified grade, and diverts the second tray 4044 to be collected from the second bypass line 4032 to the tray collection location. When the second tray 4045 to be collected passes through the third diversion device 4023, the third diversion device 4023 can use RFID to read the tray mark of the second tray 4045 to be collected and then determine that the tray mark of the second tray 4045 to be collected does not match the fourth target mark, and determine that the re-inspection result of the second tray 4045 to be collected indicates that the wound yarn package to be packed placed on the second tray 4045 to be collected is not a wound yarn package of a predetermined grade, and divert the second tray 4045 to be collected from the second bypass line 4032 to the third bypass line 4033, and return from the third bypass line 4033 to the main line 401 of the production line, and then proceed to the next process, such as weighing, appearance inspection, bagging, palletizing, strapping device, film packaging, and marking.

[0112] By the above steps included in step S102, in an embodiment of the present disclosure, after further determining whether the wound yarn package to be packaged placed on the second collection target tray is indeed a wound yarn package of a specified grade, the second collection target tray can be re-diverted to further improve the accuracy of tray collection.

[0113] In order to better implement the above tray collecting method, the embodiment of the present disclosure also applies to an electronic device included in a wound yarn package packaging system, and the electronic device provides a tray collecting device that communicates with an image collecting device and a tray diverting device, respectively. The electronic device can be a computer, a PLC, etc., the image collecting device can be an industrial camera, and the tray diverting device can be an aerodynamic type diverting device, an electric type diverting device, a drum type diverting device, etc., and the embodiment of the present disclosure is not limited thereto.

[0114] A tray collection device 500 provided by the disclosed embodiment will be described below with reference to the block diagram of the configuration shown in FIG.

[0115] The tray recovery device 500 is an image capturing unit 501 for controlling an image collecting device to capture images in the direction of the N hand grips after the robot has captured M wound yarn packages to be packaged from a yarn package carrier using N hand grips, to obtain an image representing the capturing result, where each of the N hand grips can capture one wound yarn package to be packaged, N≧2, N being an integer, and 0≦M≦N, M being an integer; The recovery control unit 502 is provided with a recovery control device for controlling a tray diversion device to divert the tray to be collected from the main line of the production line to a bypass line of the production line when the robot places M wound yarn packages to be packaged one-to-one on M wound yarn package trays included in a group of target trays installed on the main line of the production line and, based on an image representing the gripping result, determines that a tray to be collected is present in the group of target trays, thereby performing a recovery process for the tray to be collected, wherein the tray to be collected is an empty first target tray or a second target tray in which the placed wound yarn package to be packaged has been initially evaluated to be a wound yarn package of a specified grade.

[0116] In some optional embodiments, the electronic device further communicates with the robot; Here, the image acquisition unit 501 is Controlling the N handgrips to rotate relative to the image acquisition device; In the process of the N hand grips rotating relative to the image acquisition device, the image acquisition device is controlled to capture images in the directions in which the N hand grips are located, and Z images to be processed are obtained, where Z≧2 and Z is an integer, and the Z images to be processed correspond to different angles of view of the N hand grips; It is used to obtain an image representing the grasping result based on the Z pieces of processing target images.

[0117] In some optional embodiments, the image acquisition unit 501 includes: N single-angle images are cut out from each of the Z processing target images to obtain N×Z single-angle images, the N single-angle images corresponding one-to-one to the N hand grips; Among the N×Z single-angle images, Z single-angle images corresponding to the same hand grip are stitched together to obtain N multiple-angle images; It is used for obtaining an image showing the grasping result based on N multiple angle images.

[0118] In some optional embodiments, the image representing the gripping result includes N multiple-angle images, the N multiple-angle images corresponding one-to-one to the N hand grips, the target tray group includes N wound thread package trays, and in order to establish a one-to-one correspondence between the N wound thread package trays and the N multiple-angle images, the N wound thread package trays correspond one-to-one to the N hand grips, Here, the collection control unit 502: when it is determined that there is no wound yarn package image representing a wound yarn package to be packed in each of the N multiple angle images, it is determined that a collection target tray exists in the target tray group, and the wound yarn package tray corresponding to the multiple angle image among the N wound yarn package trays is set as a first collection target tray; If it is determined that a wound yarn package image representing a wound yarn package to be packaged is present in the multiple angle image, and if it is determined using a defect detection network that the wound yarn package image has a defect feature, it is determined that a tray to be collected is present in the target tray group, and the wound yarn package tray corresponding to the multiple angle image among the N wound yarn package trays is designated as the second target tray to be collected.

[0119] In some optional embodiments, the defect detection network includes a first network layer, a second network layer, and a third network layer; The first network layer combines the channel attention mechanism and the spatial attention mechanism to perform feature extraction on the yarn package image, and obtain K feature representation maps with different scales, where K≧2 and K is an integer; The second network layer is for performing feature fusion processing according to the K feature representation maps to obtain K fused feature maps, where the K fused feature maps have a one-to-one correspondence with the K feature representation maps; The third network layer is for obtaining a defect detection result to indicate whether there are defective features in the wound yarn package image based on the K fused feature maps.

[0120] In some optional embodiments, the tray diversion device includes a first diversion device and a second diversion device installed on a main line of the production line, and the bypass line of the production line includes a first bypass line and a second bypass line; Here, the collection control unit 502: Controlling a first diversion device to divert a first collection target tray from a main line of the production line onto a first bypass line of the production line; and controlling the second diversion device to divert the second collection target trays from the main line of the production line onto a second bypass line of the production line.

[0121] In some optional embodiments, the tray diversion device further includes a third diversion device installed on the second bypass line, and the bypass line of the production line further includes a third bypass line; Here, the collection control unit 502: controlling a third diversion device to divert the second collection tray from the second bypass line to a tray collection location when a result of re-inspection of the second collection tray indicates that the wound yarn package to be packed placed on the second collection tray is a wound yarn package of a specified grade; Alternatively, the third diversion device is used to divert the second tray to be collected from the second bypass line to the third bypass line, and then return the second tray to the main line of the production line from the third bypass line when the result of re-inspection of the second tray to be collected indicates that the wound yarn package to be packaged placed on the second tray to be collected is a wound yarn package of a non-specified grade.

[0122] For a description of the specific functions and examples of each unit in the tray collection device provided by the embodiments of the present disclosure, please refer to the relevant descriptions of the corresponding steps in the above-mentioned method embodiments, and the description will be omitted here.

[0123] In the technical solution disclosed herein, the acquisition, storage and application of the relevant users' personal information shall all comply with relevant laws and regulations and shall not violate public order and morals.

[0124] FIG. 6 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 6, the electronic device includes a memory 601 and a processor 602, and the memory 601 stores a computer program executable by the processor 602. The number of the memory 601 and the processor 602 may be one or more. The memory 601 may store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the method provided by the above method embodiment. The electronic device may further include: A communication interface 603 is used for communicating with an external device and performing data interaction and transmission.

[0125] When the memory 601, the processor 602, and the communication interface 603 are implemented independently, the memory 601, the processor 602, and the communication interface 603 are connected to each other via a bus and can communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be classified into an address bus, a data bus, a control bus, and the like. For ease of explanation, only one thick line is shown in FIG. 6, but this does not indicate only one bus or only one type of bus.

[0126] Optionally, in a specific implementation, when the memory 601, the processor 602, and the communication interface 603 are integrated on one chip, the memory 601, the processor 602, and the communication interface 603 can communicate with each other via an internal interface.

[0127] It should be understood that the processor may be a Central Processing Unit (CPU) or may be other general purpose processors, Digital Signal Processing (DSP), Application Specific Integrated Circuits (ASIC), Field Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware assemblies, etc. The general purpose processor may be a microprocessor or any conventional processor, etc. Additionally, the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0128] Additionally, optionally, the memory may include read-only memory and random access memory, and may further include non-volatile random access memory. The memory may be either volatile or non-volatile memory, or may include both volatile and non-volatile memory. Here, non-volatile memory may include Read-Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically EPROM (EEPROM), or flash memory. Volatile memory may include Random Access Memory (RAM), which acts as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (Enhanced SDRAM, ESDRAM), synchlink DRAM (Synchlink DRAM, SLDRAM), and direct RAMBUS RAM (DR RAM).

[0129] The above-mentioned embodiments may be implemented in whole or in part in software, hardware, firmware, or any combination thereof. When implemented in software, the whole or in part may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, a process or function according to the embodiments of the present disclosure is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, Bluetooth, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device including a server, a data center, etc. integrated with one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Versatile Disc (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)). Note that the computer-readable storage medium referred to in this disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0130] Those skilled in the art can understand that all or part of the steps for realizing the above embodiments may be implemented by hardware, or may be implemented by instructing relevant hardware by a program, and the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0131] In the description of the embodiments of the present disclosure, the reference words "one embodiment", "several embodiments", "examples", "specific examples", or "several examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. And, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. Furthermore, a person skilled in the art may combine different embodiments or examples described in the present disclosure and features of different embodiments or examples to the extent that they are not mutually inconsistent.

[0132] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean either A or B. "And / or" in the present disclosure is merely for describing the related relationship of related objects, and indicates that three kinds of relationships may exist, for example, A and / or B can indicate the following three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0133] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying a relative importance, nor should they be construed as implying a number of technical features shown. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "plurality" means two or more, unless otherwise specified.

[0134] The above are merely illustrative examples of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. A tray collection method for an electronic device included in a yarn package packaging system, the electronic device being in communication with an image collection device and a tray diversion device, The tray recovery method includes: After the robot grasps M wound yarn packages to be packaged from a yarn package carriage with N hand grips, the image acquisition device is controlled to capture images in the direction of the N hand grips to obtain an image representing the grasping result, where each of the N hand grips can grasp one wound yarn package to be packaged, N≧2 and N is an integer, and 0≦M≦N and M is an integer; and when the robot places the M wound yarn packages to be packaged one-to-one on M wound yarn package trays included in a target tray group installed on a main line of a production line and, based on an image representing the gripping result, determines that a tray to be collected is present in the target tray group, the tray diversion device is controlled to divert the tray to be collected from the main line of the production line onto a bypass line of the production line, thereby performing a collection process for the tray to be collected, wherein the tray to be collected is an empty first target tray or a second target tray in which the placed wound yarn package to be packaged has been initially evaluated as being a wound yarn package of a predetermined grade. Tray collection method.

2. The electronic device is further in communication with the robot; Controlling the image acquisition device to capture images in the direction in which the N hand grips are located and obtaining an image representing a gripping result includes: Controlling the N hand grips to rotate relative to the image acquisition device; In the process of the N hand grips rotating relative to the image acquisition device, the image acquisition device is controlled to capture images in the direction of the N hand grips, thereby obtaining Z processing target images, where Z≧2 and Z is an integer, and the Z processing target images correspond to different angles of view of the N hand grips; Obtaining an image representing the gripping result based on the Z processing target images. The tray recovery method according to claim 1 .

3. Obtaining an image representing the gripping result based on the Z processing target images, N single-angle-of-view images are cut out from each of the Z processing target images to obtain N×Z single-angle-of-view images, and the N single-angle-of-view images correspond one-to-one to the N hand grips; Among the N×Z single-angle-of-view images, Z single-angle-of-view images corresponding to the same hand grip are stitched together to obtain N multiple-angle-of-view images; Obtaining an image representing the gripping result based on the N multiple angle of view images. The tray recovery method according to claim 2 .

4. the image representing the gripping result includes N multiple-angle images, the N multiple-angle images corresponding one-to-one to the N hand grips, the target tray group includes N wound yarn package trays, and in order to establish a one-to-one correspondence between the N wound yarn package trays and the N multiple-angle images, the N wound yarn package trays correspond one-to-one to the N hand grips, Determining that a tray to be collected is present in the target tray group based on an image representing the gripping result, when it is determined that there is no yarn package image representing a wound yarn package to be packed in each of the N multiple angle images, it is determined that a collection target tray exists in the target tray group, and a yarn package tray corresponding to the multiple angle image among the N number of yarn package trays is set as the first collection target tray; and when it is determined that a yarn package image representing a wound yarn package to be packed is present in the multiple angle image and when it is determined that the yarn package image has a defect feature using a defect detection network, it is determined that a tray to be collected is present in the target tray group, and a yarn package tray corresponding to the multiple angle image among the N number of yarn package trays is set as the second target tray. The tray recovery method according to claim 1 .

5. the defect detection network includes a first network layer, a second network layer, and a third network layer; The first network layer combines a channel attention mechanism and a spatial attention mechanism to perform feature extraction on the yarn package image to obtain K feature representation images with different scales, where K≧2 and K is an integer; The second network layer is for performing a feature fusion process according to the K feature representation maps to obtain K fused feature maps, where the K fused feature maps correspond one-to-one to the K feature representation maps; The third network layer is for obtaining a defect detection result indicating whether or not there is a defect feature in the wound yarn package image based on the K fused feature maps. The tray recovery method according to claim 4.

6. The tray diversion device includes a first diversion device and a second diversion device installed on a main line of the production line, and the bypass line of the production line includes a first bypass line and a second bypass line; Controlling the tray diversion device to divert the tray to be collected from the main line of the production line to the bypass line of the production line includes: Controlling the first diversion device to divert the first collection target tray from a main line of the production line onto a first bypass line of the production line; and controlling the second diversion device to divert the second collection target tray from the main line of the production line onto a second bypass line of the production line. The tray recovery method according to claim 1 .

7. the tray diversion device further includes a third diversion device installed on the second bypass line, and the production line bypass line further includes a third bypass line; Controlling the tray diversion device to divert the tray to be collected from the main line of the production line to the bypass line of the production line includes: controlling the third diversion device to divert the second collection tray from the second bypass line to a tray collection location when a result of re-inspection of the second collection tray indicates that the wound yarn package to be packed placed on the second collection tray is a wound yarn package of a predetermined grade; Alternatively, the method further includes controlling the third diversion device to divert the second tray to be collected from the second bypass line to the third bypass line and to return the second tray to the main line of the production line from the third bypass line when a result of re-inspection of the second tray to be collected indicates that the wound yarn package to be packaged placed on the second tray to be collected is a wound yarn package of a non-predetermined grade. The tray recovery method according to claim 6.

8. A tray collection device for an electronic device included in a yarn package packaging system, the electronic device being in communication with an image collection device and a tray diversion device, The tray recovery device is an image acquisition unit for controlling the image collection device to capture images in the direction of the N handgrips after the robot grasps M wound yarn packages to be packaged from the yarn package carrier with N handgrips, to obtain an image representing the grasping result, where each of the N handgrips can grasp one wound yarn package to be packaged, N≧2, N being an integer, 0≦M≦N, M being an integer; a recovery control unit for controlling the tray diversion device to divert the tray to be collected from the main line of the production line onto a bypass line of the production line when the robot places the M wound yarn packages to be packaged one-to-one on M wound yarn package trays included in a target tray group installed on a main line of the production line and it is determined that a tray to be collected is present in the target tray group based on an image representing the gripping result, and thereby executing a recovery process for the tray to be collected, wherein the tray to be collected is an empty first target tray or a second target tray in which the placed wound yarn package to be packaged has been initially evaluated as being a wound yarn package of a predetermined grade. Tray collection device.

9. The electronic device is further in communication with the robot; The image acquisition unit includes: Controlling the N hand grips to rotate relative to the image acquisition device; In the process of the N hand grips rotating relative to the image acquisition device, the image acquisition device is controlled to capture images in the direction of the N hand grips, thereby obtaining Z processing target images, where Z≧2 and Z is an integer, and the Z processing target images correspond to different angles of view of the N hand grips; Obtaining an image representing the gripping result based on the Z processing target images; The tray recovery device according to claim 8.

10. The image acquisition unit includes: N single-angle-of-view images are cut out from each of the Z processing target images to obtain N×Z single-angle-of-view images, and the N single-angle-of-view images correspond one-to-one to the N hand grips; Among the N×Z single-angle-of-view images, Z single-angle-of-view images corresponding to the same hand grip are stitched together to obtain N multiple-angle-of-view images; Obtaining an image representing the gripping result based on the N multiple angle images; The tray recovery device according to claim 9.

11. the image representing the gripping result includes N multiple-angle images, the N multiple-angle images corresponding one-to-one to the N hand grips, the target tray group includes N wound yarn package trays, and in order to establish a one-to-one correspondence between the N wound yarn package trays and the N multiple-angle images, the N wound yarn package trays correspond one-to-one to the N hand grips, The recovery control unit when it is determined that there is no yarn package image representing a wound yarn package to be packed in each of the N multiple angle images, it is determined that a collection target tray exists in the target tray group, and a yarn package tray corresponding to the multiple angle image among the N number of yarn package trays is set as the first collection target tray; when it is determined that a yarn package image representing a wound yarn package to be packed is present in the multiple view angle image and when it is determined that the yarn package image has a defect feature using a defect detection network, it is determined that a recovery target tray is present in the target tray group, and the yarn package tray corresponding to the multiple view angle image among the N number of yarn package trays is set as the second recovery target tray. The tray recovery device according to claim 8.

12. the defect detection network includes a first network layer, a second network layer, and a third network layer; The first network layer combines a channel attention mechanism and a spatial attention mechanism to perform feature extraction on the yarn package image to obtain K feature representation images with different scales, where K≧2 and K is an integer; The second network layer is for performing a feature fusion process according to the K feature representation maps to obtain K fused feature maps, where the K fused feature maps correspond one-to-one to the K feature representation maps; The third network layer is for obtaining a defect detection result indicating whether or not there is a defect feature in the wound yarn package image based on the K fused feature maps. The tray recovery device according to claim 11.

13. The tray diversion device includes a first diversion device and a second diversion device installed on a main line of the production line, and the bypass line of the production line includes a first bypass line and a second bypass line; The recovery control unit Controlling the first diversion device to divert the first collection target tray from a main line of the production line onto a first bypass line of the production line; and controlling the second diversion device to divert the second collection target tray from the main line of the production line onto a second bypass line of the production line. The tray recovery device according to claim 8.

14. the tray diversion device further includes a third diversion device installed on the second bypass line, and the production line bypass line further includes a third bypass line; The recovery control unit controlling the third diversion device to divert the second collection tray from the second bypass line to a tray collection location when a result of re-inspection of the second collection tray indicates that the wound yarn package to be packed placed on the second collection tray is a wound yarn package of a predetermined grade; Alternatively, the third diversion device is used to divert the second collection tray from the second bypass line to the third bypass line and return the second collection tray from the third bypass line to the main line of the production line when the result of re-inspection of the second collection tray indicates that the wound yarn package to be packed placed on the second collection tray is a wound yarn package of a non-predetermined grade. The tray recovery device according to claim 13.

15. At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 7. Electronic devices.

16. A non-transitory computer readable storage medium for storing instructions that cause a computer to perform the method of any one of claims 1 to 7.

17. A program for implementing the method according to any one of claims 1 to 7 when executed by a processor in a computer.

Images