Tray recycling method, device, electronic equipment and storage medium

Abstract

The present disclosure relates to the field of computer and automatic control, and particularly relates to a tray recycling method and device, electronic equipment and storage medium. The method comprises the following steps: after a robot grasps M to-be-packaged silk spools from a silk spool trolley through N hand grabs, controlling an image acquisition device to shoot in the direction where the N hand grabs are located to obtain a grasping result representation image; after the robot places the M to-be-packaged silk spools one by one in M silk spool trays included in a target tray group arranged on a main line of a flow line, in the case that it is determined based on the grasping result representation image that there is a to-be-recycled tray in the target tray group, the to-be-recycled tray is diverted from the main line of the flow line to a branch line of the flow line by controlling a tray diversion device, so as to realize recycling processing of the to-be-recycled tray. By adopting the present disclosure, a large amount of human resources can be saved, and the recycling efficiency of the to-be-recycled tray is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer and automatic control, and particularly relates to a tray recycling method and device, an electronic device and a storage medium. BACKGROUND

[0002] Silk spool packaging is an important link in the silk production process. In the silk spool packaging process, due to manual picking, missing and other reasons, there may be some to-be-recycled trays in the multiple silk spool trays on the assembly line. At present, in order to avoid the influence of these to-be-recycled trays on the silk spool packaging progress, an artificial recycling method is usually used for recycling processing. However, the artificial recycling method not only wastes a large amount of human resources, but also affects the recycling efficiency of the to-be-recycled trays. SUMMARY

[0003] The present disclosure provides a tray recycling method and device, an electronic device and a storage medium to solve or alleviate one or more technical problems in the prior art.

[0004] In a first aspect, the present disclosure provides a tray recycling method, applied to an electronic device included in a silk spool packaging system, the electronic device being in communication with an image acquisition device and a tray shunting device respectively;

[0005] The method comprises the following steps.

[0006] After the robot picks up M to-be-packaged silk spools from the silk spool car through N hand grabs, the image acquisition device is controlled to shoot in the direction where the N hand grabs are located, to obtain a picking result representation image; each hand grab in the N hand grabs can pick up one to-be-packaged silk spool, N is an integer greater than or equal to 2, and M is an integer greater than or equal to 0 and less than or equal to N;

[0007] After the robot places the M to-be-packaged silk spools one by one in the M silk spool trays included in the target tray group arranged on the main line of the assembly line, in the case that there is a to-be-recycled tray in the target tray group based on the picking result representation image, the to-be-recycled tray is shunted from the main line of the assembly line to the branch line of the assembly line by controlling the tray shunting device, to realize recycling processing of the to-be-recycled tray; the to-be-recycled tray is a first to-be-recycled tray which is empty or a second to-be-recycled tray in which the placed to-be-packaged silk spool is preliminarily evaluated as a grade silk spool.

[0008] In a second aspect, the present disclosure provides a tray recycling device, comprising:

[0009] The electronic device is applied to an electronic device included in a silk spool packaging system, and the electronic device is in communication with an image acquisition device and a tray shunting device respectively;

[0010] The device comprises the following.

[0011] The image acquisition unit is configured to control the image acquisition device to capture an image in a direction where the N hand grabs are located after the robot picks up M to-be-packaged spools from the spool car by the N hand grabs, so as to obtain a picking result representation image; each hand grab of the N hand grabs can pick up one to-be-packaged spool, N is an integer greater than or equal to 2, and M is an integer greater than or equal to 0 and less than or equal to N;

[0012] The recycling control unit is configured to, after the robot places the M to-be-packaged spools one by one in M spool trays included in a target tray group arranged on the main line of the flow line, in a case where it is determined based on the picking result representation image that there is a to-be-recycled tray in the target tray group, control the tray shunting device to shunt the to-be-recycled tray from the main line of the flow line to the branch line of the flow line, so as to realize recycling processing of the to-be-recycled tray; the to-be-recycled tray is a first to-be-recycled tray that is empty or a second to-be-recycled tray in which the placed to-be-packaged spool is preliminarily evaluated as a grade spool.

[0013] In a third aspect, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein

[0016] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any of the embodiments of the present disclosure.

[0017] In a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform the method according to any of the embodiments of the present disclosure.

[0018] In a fifth aspect, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method according to any of the embodiments of the present disclosure.

[0019] In the technical solutions provided by the present disclosure, after the robot grasps M to-be-packaged spools from the spool car by N hand grabs, the image acquisition device is controlled to shoot in the direction where the N hand grabs are located to obtain a grasping result representation image. After the robot places the M to-be-packaged spools one by one in M spool trays included in a target tray group arranged on the main line of the assembly line, in the case that it is determined based on the grasping result representation image that there is a to-be-recycled tray in the target tray group, the to-be-recycled tray is diverted from the main line of the assembly line to the branch line of the assembly line by controlling the tray diversion device, so as to realize the recycling processing of the to-be-recycled tray. In this way, the automatic recycling of the to-be-recycled tray can be realized, a large amount of human resources can be saved compared with the current manual recycling mode, and the recycling efficiency of the to-be-recycled tray is improved.

[0020] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0021] In the drawings, like reference numerals designate like or similar parts throughout the several views, and the reference numerals are the same as or similar to those in the embodiments of the present disclosure. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments in accordance with the present disclosure and should not be considered limiting the scope of the present disclosure.

[0022] Figure 1 A flowchart of a tray recycling method provided by an embodiment of the present disclosure;

[0023] Figures 2A-2F An example diagram of a process of obtaining a grasping result representation image in a tray recycling method provided by an embodiment of the present disclosure;

[0024] Figure 2G 、 Figure 2H and Figure 2I An example diagram of a process of establishing a correspondence between N spool trays and N multi-view images in a tray recycling method provided by an embodiment of the present disclosure;

[0025] Figure 3A A structural diagram of a defect detection network provided by an embodiment of the present disclosure;

[0026] Figure 3B A structural diagram of a first network layer provided by an embodiment of the present disclosure;

[0027] Figure 3C Structural diagrams of a first attention unit, a second attention unit and a third attention unit provided by an embodiment of the present disclosure;

[0028] Figure 3D A structural diagram of a first internal unit and a first sub-unit provided by an embodiment of the present disclosure is provided.

[0029] Figure 3E A structural diagram of a second internal unit and a second sub-unit provided by an embodiment of the present disclosure is provided.

[0030] Figure 3F A structural diagram of a second network layer provided by an embodiment of the present disclosure is provided.

[0031] Figure 3G A structural diagram of a first bottleneck unit, a second bottleneck unit, a third bottleneck unit and a fourth bottleneck unit passed by an embodiment of the present disclosure is provided.

[0032] Figure 3H A structural diagram of a third network layer provided by an embodiment of the present disclosure is provided.

[0033] Figures 4A-4E An example diagram of a shunting process in a tray recycling method provided by an embodiment of the present disclosure is provided.

[0034] Figure 5 A schematic structural block diagram of a tray recycling device provided by an embodiment of the present disclosure is provided.

[0035] Figure 6 A schematic structural block diagram of an electronic device provided by an embodiment of the present disclosure is provided. DETAILED DESCRIPTION

[0036] The present disclosure will be described in further detail below with reference to the drawings. The same reference numerals in the drawings denote elements or components having the same function or similar functions. Although various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0037] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail in order to highlight the main idea of the present disclosure.

[0038] As described previously, the silk spool packaging is an important link in the silk production process. In the silk spool packaging process, due to manual picking, missing and other reasons, there may be some to-be-recycled trays in the plurality of silk spool trays on the assembly line, such as empty silk spool trays or other silk spool trays that need to be recycled. At present, in order to avoid the influence of these to-be-recycled trays on the silk spool packaging progress, manual recycling is usually used for recycling processing. However, the manual recycling method not only wastes a large amount of human resources, but also affects the recycling efficiency of the to-be-recycled trays.

[0039] In order to save a large amount of human resources and improve the recycling efficiency of the to-be-recycled trays, the embodiment of the present disclosure provides a tray recycling method applied to an electronic device included in a silk spool packaging system. The electronic device is in communication with an image acquisition device and a tray shunting device respectively. The electronic device can be a computer, a programmable logic controller (PLC), etc.; the image acquisition device can be an industrial camera; and the tray shunting device can be a pneumatic shunt, an electric shunt, a roller shunt, etc., which are not limited in the embodiment of the present disclosure.

[0040] In addition, it should be noted that in the embodiment of the present disclosure, the main types of silk spool products can include at least one of partially oriented yarns (POY), fully drawn yarns (FDY), draw textured yarns (DTY) (or low stretch yarns), etc. For example, the types of silk spool products can specifically include polyester partially oriented yarns (Polyester Partially Oriented Yarns), polyester fully drawn yarns (Polyester Fully Drawn Yarns), polyester drawn yarns (Polyester Drawn Yarns), polyester draw textured yarns (Polyester Draw Textured Yarns), etc.

[0041] Figure 1 is a flowchart of a tray recycling method provided by the embodiment of the present disclosure. In the following, the tray recycling method provided by the embodiment of the present disclosure will be described in combination with Figure 1 The tray recycling method provided by the embodiment of the present disclosure will be described. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can also be executed in other orders.

[0042] In step S101, after the robot picks up M to-be-packaged silk spools from the silk spool car through N hand grabs, the image acquisition device is controlled to shoot in the direction where the N hand grabs are located, and a grabbing result representation image is obtained.

[0043] wherein the robot can be an N-axis industrial robot, i.e., the robot can include N hand grabs, and each of the N hand grabs can grab one to-be-packaged spool. Here, N≥2, and N is an integer.

[0044] It can be understood that, in the embodiment of the present disclosure, when the robot grabs the to-be-packaged spools from the spool car through the N hand grabs, some hand grabs can fail to successfully grab the to-be-packaged spools. Therefore, in the embodiment of the present disclosure, 0≤M≤N, and M is an integer. Specifically, when the robot grabs the to-be-packaged spools from the spool car through the N hand grabs, if all the hand grabs successfully grab the to-be-packaged spools, M=N; if some hand grabs fail to successfully grab the to-be-packaged spools, 0≤M

[0045] After the robot places the M to-be-packaged spools one-to-one in the M spool trays included in the target tray group arranged on the main line of the flow line, in the case where it is determined based on the grabbing result characterization image that there is a to-be-recycled tray in the target tray group, the to-be-recycled tray is diverted from the main line of the flow line to the branch line of the flow line by controlling the tray diversion device, so as to realize recycling processing of the to-be-recycled tray.

[0046] wherein the to-be-recycled tray is a first to-be-recycled tray that is empty or a second to-be-recycled tray in which the placed to-be-packaged spool is preliminarily evaluated as a grade product spool. Here, the grade product spool can be a spool product that is not of AA grade, i.e., can be a spool product that is reduced from the default AA grade to other grades (such as A grade, B grade, C grade) due to quality reasons, and the quality reasons can be due to the presence of loose silk, oil stains, silk entanglement, poor molding, or paper tube damage, etc.

[0047] It should be noted that, in the embodiment of the present disclosure, the target tray group generally includes a fixed number of spool trays, for example, the target tray group can include N spool trays, and the N spool trays are one-to-one corresponding to the N hand grabs. Therefore, when the robot grabs the to-be-packaged spools from the spool car through the N hand grabs, if some hand grabs fail to successfully grab the to-be-packaged spools (at this time, 0≤M

[0048] The tray recycling method provided in the embodiment of the present disclosure can be used to control the image acquisition device to shoot in the direction where the N hand grabs are located after the robot picks up M to-be-packaged spools from the spool car by the N hand grabs, to obtain a grabbing result representation image, and to control the tray shunting device to shunt the to-be-recycled tray from the main line of the flow line to the branch line of the flow line after the robot places the M to-be-packaged spools one by one in the M spool trays included in the target tray group arranged on the main line of the flow line, in the case where it is determined based on the grabbing result representation image that there is a to-be-recycled tray in the target tray group, so as to realize the recycling processing of the to-be-recycled tray. In this way, the automatic recycling of the to-be-recycled tray can be realized, a large amount of human resources can be saved compared with the current manual recycling method, and the recycling efficiency of the to-be-recycled tray is improved.

[0049] In some optional embodiments, the electronic device is also in communication with the robot, and the step S101 of "controlling the image acquisition device to shoot in the direction where the N hand grabs are located, to obtain a grabbing result representation image" can include the following steps:

[0050] In step S1011, the N hand grabs are controlled to rotate relative to the image acquisition device.

[0051] In an example, the N hand grabs of the robot can be connected to the main body structure of the robot through the same mechanical shaft. Based on this, in the embodiment of the present disclosure, the mechanical shaft can be rotated so that the N hand grabs rotate as a whole relative to the image acquisition device.

[0052] In step S1012, the image acquisition device is controlled to shoot in the direction where the N hand grabs are located during the rotation of the N hand grabs relative to the image acquisition device, to obtain Z to-be-processed images.

[0053] Wherein, Z≥2, and Z is an integer.

[0054] Since the image acquisition device shoots in the direction where the N hand grabs are located during the rotation of the N hand grabs relative to the image acquisition device, to obtain Z to-be-processed images, the Z to-be-processed images correspond to different angles of view of the N hand grabs.

[0055] In step S1013, a grabbing result representation image is obtained based on the Z to-be-processed images.

[0056] In an example, the Z to-be-processed images can be processed by decomposition, splicing, etc. to obtain the grabbing result representation image.

[0057] Please refer to Figure 2A, exemplarily, the robot 201 can include 5 hand grabs, i.e., a first hand grab 2021, a second hand grab 2022, a third hand grab 2023, a fourth hand grab 2024, and a fifth hand grab 2025 (i.e., in this example, N = 5), and due to manual taking, missing, etc., there are only 4 to-be-packaged spools on the spool carrier 203, i.e., a first to-be-packaged spool 2041, a third to-be-packaged spool 2043, a fourth to-be-packaged spool 2044, and a fifth to-be-packaged spool 2045. In combination with Figure 2B , when the robot 201 grabs the to-be-packaged spools from the spool carrier 203 by the 5 hand grabs, the first hand grab 2021 successfully grabs the first to-be-packaged spool 2041, the second hand grab 2022 fails to grab a to-be-packaged spool, the third hand grab 2023 successfully grabs the third to-be-packaged spool 2043, the fourth hand grab 2024 successfully grabs the fourth to-be-packaged spool 2044, and the fifth hand grab 2025 successfully grabs the fifth to-be-packaged spool 2045. After the robot 201 grabs the 4 to-be-packaged spools from the spool carrier 203 by the 5 hand grabs, the mechanical shaft 205 used to connect the 5 hand grabs to the main body structure of the robot 201 can be rotated, so that the 5 hand grabs are rotated as a whole relative to the image acquisition device 206. Specifically, the mechanical shaft 205 can be rotated in the direction of “A1→A2”, so that the 5 hand grabs are rotated as a whole relative to the image acquisition device 206. During the rotation of the 5 hand grabs relative to the image acquisition device 206, the image acquisition device 206 can be controlled to shoot in the direction of the 5 hand grabs, and Z to-be-processed images are obtained, so as to ensure that the Z to-be-processed images correspond to different viewing angles of the N hand grabs.

[0058] Taking Z = 2 as an example, when the 5 hand grabs are rotated to a first angle as shown in FIG. 8A, the image acquisition device 206 can be controlled to shoot in the direction of the 5 hand grabs, and a first to-be-processed image is obtained. Figure 2B The first to-be-processed image is mainly used to represent the image features of the 4 to-be-packaged spools on the first side, i.e., the first to-be-processed image can include a single-view image SD11 representing the image features of the first to-be-packaged spool 2041 on the first side, a single-view image SD31 representing the image features of the third to-be-packaged spool 2043 on the first side, a single-view image SD41 representing the image features of the fourth to-be-packaged spool 2044 on the first side, and a single-view image SD51 representing the image features of the fifth to-be-packaged spool 2045 on the first side, as shown in FIG. 8B. Figure 2C In addition, since the second hand grab 2022 fails to grab a to-be-packaged spool, the second hand grab 2022 is not blocked by a to-be-packaged spool, which makes the first to-be-processed image further include a single-view image SZ21 representing the image features of the second hand grab 2022 on the first side.

[0059] Similarly, when the five grippers rotate relative to the image acquisition device 206, as... Figure 2D At the second angle shown, the image acquisition device 206 is controlled to capture images in the direction of the five hand grips, obtaining a second image to be processed. The second image to be processed is mainly used to characterize the image features of the four yarn spindles to be packaged on the second side opposite to the first side. That is, the second image to be processed may include a single-view image SD12 characterizing the image features of the first yarn spindle 2041 to be packaged on the second side, a single-view image SD32 characterizing the image features of the third yarn spindle 2043 to be packaged on the second side, a single-view image SD42 characterizing the image features of the fourth yarn spindle 2044 to be packaged on the second side, and a single-view image SD52 characterizing the image features of the fifth yarn spindle 2045 to be packaged on the second side, as detailed below. Figure 2E As shown. In addition, since the second gripper 2022 failed to grab the yarn spindle to be packaged, the second gripper 2022 was not obscured by the yarn spindle to be packaged, so the second image to be processed can also include a single-view image SZ22 for characterizing the image features of the second gripper 2022 on the second side.

[0060] Furthermore, it should be noted that in the embodiments disclosed herein, Figure 2B The first image to be processed shown and Figure 2D The second images to be processed shown have all undergone preprocessing. Preprocessing may include cropping, denoising, feature enhancement, etc., and this embodiment does not limit the scope of the present disclosure.

[0061] After obtaining the first and second images to be processed, the Z images can be processed through operations such as decomposition and splicing to obtain the image representing the capture result.

[0062] Through the steps included in step S101, in this embodiment of the present disclosure, after the robot picks up M silk spindles to be packaged from the silk spindle cart using N grippers, the N grippers are controlled to rotate relative to the image acquisition device. During the rotation of the N grippers relative to the image acquisition device, the image acquisition device is controlled to take pictures in the direction of the N grippers, obtaining Z images to be processed (the Z images correspond to different perspectives of the N grippers). Based on the Z images to be processed, a grasping result representation image is obtained. In this way, the grasping result representation image can comprehensively represent the M silk spindles to be packaged picked up by the N grippers, avoiding feature omissions, thereby reducing the misjudgment rate of the pallet to be recycled and improving the accuracy of pallet recycling.

[0063] Furthermore, in some alternative implementations, step S1013 may include the following steps:

[0064] Step S10131, N single-view images are cropped from each of the Z to-be-processed images, to obtain N×Z single-view images.

[0065] The N single-view images correspond to the N hand grabs one by one.

[0066] Step S10132, the Z single-view images corresponding to the same hand grab in the N×Z single-view images are spliced to obtain N multi-view images.

[0067] Step S10133, a grab result representation image is obtained based on the N multi-view images.

[0068] In an example, the N multi-view images can be tiled according to the order of the N hand grabs on the robot to obtain the grab result representation image.

[0069] Continuing the foregoing example (in which N=5 and Z=2), after obtaining the first to-be-processed image as shown in Figure 2C and the second to-be-processed image as shown in Figure 2E , 5 single-view images, single-view image SD11, single-view image SZ21, single-view image SD31, single-view image SD41, and single-view image SD51, can be cropped from the first to-be-processed image, and similarly, 5 single-view images, single-view image SD21, single-view image SZ22, single-view image SD32, single-view image SD42, and single-view image SD52, can be cropped from the second to-be-processed image, and the 2 single-view images corresponding to the same hand grab in the 5×2=10 single-view images are spliced to obtain 5 multi-view images, multi-view image SD11&SD12 corresponding to the first hand grab 2021, multi-view image SZ21&SZ22 corresponding to the second hand grab 2022, multi-view image SD31&SD32 corresponding to the third hand grab 2023, multi-view image SD41&SD42 corresponding to the fourth hand grab 2024, and multi-view image SD51&SD52 corresponding to the fifth hand grab 2025, and the 5 multi-view images are tiled according to the order of the 5 hand grabs on the robot to obtain the grab result representation image, which can be specifically as shown in Figure 2F .

[0070] Through the above steps included in step S1013, in the embodiments of the present disclosure, a specific acquisition manner of the grab result representation image is provided, which has a simple flow and can improve the execution efficiency of the tray recycling method.

[0071] After step S101 is executed, the robot can be controlled to place the M to-be-packaged spools one by one in the M spool trays included in the target tray group provided on the main line of the assembly line. Continuing the foregoing example (in which M=4), combined withFigure 2G and Figure 2H Assuming that the target tray group arranged on the main line 207 of the pipeline includes 5 spool trays in total, corresponding to the first spool tray 2081 of the first hand grab 2021, the second spool tray 2082 of the second hand grab 2022, the third spool tray 2083 of the third hand grab 2023, the fourth spool tray 2084 of the fourth hand grab 2024, and the fifth spool tray 2085 of the fifth hand grab 2025, then the robot 201 can be controlled to place the first to-be-packaged spool 2041 on the first spool tray 2081, the third to-be-packaged spool 2043 on the third spool tray 2083, the fourth to-be-packaged spool 2044 on the fourth spool tray 2084, and the fifth to-be-packaged spool 2045 on the second spool tray 2085.

[0072] In addition, as mentioned above, in the embodiments of the present disclosure, the image of the grabbing result representation can include N multi-view images, and the N multi-view images correspond to the N hand grabs one by one, and the target tray group can include N spool trays in total, and the N spool trays correspond to the N hand grabs one by one, so that a one-to-one correspondence relationship between the N spool trays and the N multi-view images can be established.

[0073] Continuing the foregoing example, the image of the grabbing result representation includes 5 multi-view images, corresponding to the multi-view images SD11&SD12 of the first hand grab 2021, the multi-view images SZ21&SZ22 of the second hand grab 2022, the multi-view images SD31&SD32 of the third hand grab 2023, the multi-view images SD41&SD42 of the fourth hand grab 2024, and the multi-view images SD51&SD52 of the fifth hand grab 2025, and the target tray group includes 5 spool trays in total, corresponding to the first spool tray 2081 of the first hand grab 2021, the second spool tray 2082 of the second hand grab 2022, the third spool tray 2083 of the third hand grab 2023, the fourth spool tray 2084 of the fourth hand grab 2024, and the fifth spool tray 2085 of the fifth hand grab 2025, as shown in Figure 2F 、 Figure 2G and Figure 2H So that a one-to-one correspondence relationship between the N spool trays and the N multi-view images can be established.

[0074]

[0075]

[0076]

[0077]

[0078]

[0079] The above correspondence can be specifically as shown in Figure 2I

[0080] Based on the above, in some optional embodiments, the “determining that there is a to-be-recycled tray in the target tray group based on the image characterized by the crawling result” in step S102 can include the following steps:

[0081] Step S1021, for each of the N multi-view images, in a case where it is determined that there is no spool image for characterizing a to-be-packaged spool in the multi-view image, it is determined that there is a to-be-recycled tray in the target tray group, and the spool tray corresponding to the multi-view image among the N spool trays is taken as a first to-be-recycled tray.

[0082] In an example, whether there is a spool image for characterizing a to-be-packaged spool in the multi-view image can be determined by a connected region analysis method (also known as Blob analysis), a template matching method, a deep learning method, etc.

[0083] In addition, in the embodiments of the present disclosure, after the first to-be-recycled tray is determined from the N spool trays, the tray identification of the first to-be-recycled tray can be recorded, and the tray identification of the first to-be-recycled tray is sent to the tray shunting device. Wherein, the tray identification can be a tray code.

[0084] Step S1022, in a case where it is determined that there is a spool image for characterizing a to-be-packaged spool in the multi-view image, and it is determined that the multi-view image has a defect feature by using the defect detection network, it is determined that there is a to-be-recycled tray in the target tray group, and the spool tray corresponding to the multi-view image among the N spool trays is taken as a second to-be-recycled tray.

[0085] Wherein, the defect feature can include at least one of a lint feature, a grease feature, a tangling feature, a poor forming, and a paper tube damage, etc.

[0086] In an example, the defect detection network can be implemented in combination with a U-net network and a residual network (also known as ResNet), and can also be implemented based on a convolutional neural network (CNN) or other suitable neural network, and the embodiments of the present disclosure do not limit this. Wherein, the U-net network is a neural network model with a “U” shape structure, the left side of the U-net network is used to implement image feature extraction, and the right side is used to implement up-sampling processing of image features.

[0087] ​In addition, in the embodiment of the present disclosure, after the second to-be-recycled tray is determined from the N spool trays, the tray identifier of the second to-be-recycled tray can also be recorded and sent to the tray shunting device.

[0088] Continuing the foregoing example, the image representing the result of the grabbing includes 5 multi-view images, the target tray group includes 5 spool trays in total, and the 5 multi-view images and the 5 spool trays have a one-to-one correspondence as shown in the following table. Figure 2I Then, when step S1021 is performed, it can be determined that there is no spool image for representing the to-be-packaged spool in the multi-view images SZ21&SZ22, and thus it can be determined that there is a to-be-recycled tray in the target tray group, and the second spool tray 2082 corresponding to the multi-view images SZ21&SZ22 among the 5 spool trays is taken as the first to-be-recycled tray, the tray identifier of the first to-be-recycled tray is recorded, and the tray identifier of the first to-be-recycled tray is sent to the tray shunting device; when step S1022 is performed, it can be determined that both the multi-view images SD41&SD42 and the multi-view images SD51&SD52 have defect features, and thus it can be determined that there is a to-be-recycled tray in the target tray group, and the fourth spool tray 2084 corresponding to the multi-view images SD41&SD42 and the fifth spool tray 2085 corresponding to the multi-view images SD51&SD52 among the 5 spool trays are both taken as the second to-be-recycled tray, the tray identifier of the second to-be-recycled tray is recorded, and the tray identifier of the second to-be-recycled tray is sent to the tray shunting device.

[0089] Through the above steps included in step S102, in the embodiment of the present disclosure, on the one hand, for each multi-view image, it is first determined whether the to-be-recycled tray corresponding thereto is the first to-be-recycled tray, and then in the case where the to-be-recycled tray corresponding thereto is determined to be not the first to-be-recycled tray, it is determined whether the to-be-recycled tray corresponding thereto is the second to-be-recycled tray, so that through the two-round judgment process in cascade, the misjudgment rate of the to-be-recycled tray is reduced, and the accuracy of tray recycling is improved; on the other hand, in the embodiment of the present disclosure, since the machine vision method is used to determine the to-be-recycled tray, not only the execution efficiency of the tray recycling method can be improved, but also the accuracy of tray recycling can be improved.

[0090] Please refer to Figure 3A In a specific example, the defect detection network can include a first network layer, a second network layer, and a third network layer.

[0091] The first network layer is configured to perform feature extraction on the spool image by combining the channel attention mechanism and the spatial attention mechanism to obtain K feature representation maps with different scales, where K≥2 and K is an integer; the second network layer is configured to perform feature fusion processing based on the K feature representation maps to obtain K fusion feature maps, where the K fusion feature maps and the K feature representation maps have a one-to-one correspondence; and the third network layer is configured to perform defect detection on the K fusion feature maps to obtain the defect detection result.Figure 1 a pair of corresponding; the third network layer is configured to obtain a defect detection result based on the K fused feature maps, wherein the defect detection result is used to represent whether the filament image has a defect feature.

[0092] Please refer to Figure 3B , for example, the first network layer can include a cascaded first feature extraction module, a second feature extraction module and a third feature extraction module (i.e., in this example, K = 3).

[0093] The first feature extraction module is configured to process the input image feature (here, it can be a filament image, such as Figure 2F and Figure 2I the filament image in the multi-view image SD41&SD42) to obtain a first feature representation map. Further, the first feature extraction module can include a cascaded first convolution processing unit and a first attention unit. Here, the first convolution processing unit is configured to perform convolution processing on the input image feature, and the first attention unit is configured to combine the channel attention mechanism and the spatial attention mechanism to extract features from the input image feature.

[0094] The second feature extraction module is configured to process the input image feature (the first feature representation map) to obtain a second feature representation map. Further, the second feature extraction module can include a cascaded second convolution processing unit and a second attention unit. Here, the second convolution processing unit is configured to perform convolution processing on the input image feature, and the second attention unit is configured to combine the channel attention mechanism and the spatial attention mechanism to extract features from the input image feature.

[0095] The third feature extraction module is configured to process the input image feature (the second feature representation map) to obtain a third feature representation map. Further, the third feature extraction module can include a cascaded third convolution processing unit, a spatial pyramid pooling (SPPF) unit and a third attention unit. Here, the convolution processing unit is configured to perform convolution processing on the input image feature, the SPPF unit is configured to fuse image features of different receptive fields, and the third attention unit is configured to combine the channel attention mechanism and the spatial attention mechanism to extract features from the input image feature.

[0096] Please refer to Figure 3CIn this example, the first attention unit, the second attention unit, and the third attention unit can have the same structure. For example, the first attention unit, the second attention unit, and the third attention unit can include a first subunit, a second subunit, and a third subunit in parallel, and further include a fusion subunit configured to perform a fusion process on image features output by the first subunit, the second subunit, and the third subunit, and an activation subunit configured to perform a normalization process on image features output by the fusion subunit (e.g., using a Sigmoid function to perform the normalization process on the image features output by the fusion subunit). The third subunit can include a first internal unit and a second internal unit in cascade, and the first internal unit can have the same structure as the first subunit, and the second internal unit can have the same structure as the second subunit. The first internal unit and the first subunit are configured to perform feature extraction on input image features based on a channel attention mechanism, and the second internal unit and the second subunit are configured to perform feature extraction on the input image features based on a spatial attention mechanism.

[0097] More specifically, referring to FIG. 6, Figure 3D The first internal unit and the first subunit can include:

[0098] a first internal pooling unit and a second internal pooling unit in parallel;

[0099] a first internal convolution unit in cascade with the first internal pooling unit and the second internal pooling unit in parallel;

[0100] a third internal pooling unit and a fourth internal pooling unit in parallel in cascade with the first internal convolution unit;

[0101] an internal fusion unit configured to perform a fusion process on image features output by the third internal pooling unit and the fourth internal pooling unit;

[0102] a first internal activation unit configured to perform a normalization process on image features output by the internal fusion unit (e.g., using a Sigmoid function to perform the normalization process on the image features output by the internal fusion unit).

[0103] The first internal pooling unit is configured to perform average pooling on the input image features; the second internal pooling unit is configured to perform maximum pooling on the input image features; the first internal convolution unit can be a convolution unit implemented based on a depthwise convolution (DW) layer or a depthwise separable convolution (DSC) layer; the third internal pooling unit is configured to perform average pooling on the input image features; and the fourth internal pooling unit is configured to perform maximum pooling on the input image features. Here, the DSC layer can be constructed by combining the DW layer and a pointwise convolution (PW) layer. In actual application, the first internal convolution unit can perform in-depth channel feature extraction on the input image features, thereby improving the channel attention capability and feature expression capability, and meanwhile, the first internal convolution unit can reduce unnecessary parameter quantity in neural network calculation due to the weight sharing characteristic.

[0104] More specifically, please refer to Figure 3E The second internal unit and the second sub-unit can include:

[0105] a fifth internal pooling unit and a sixth internal pooling unit in cascade;

[0106] a channel concatenation unit, a second internal convolution unit and a second internal activation unit in cascade with the sixth internal pooling unit and in a cascade structure.

[0107] The fifth internal pooling unit is configured to perform average pooling on the input image features; the sixth internal pooling unit is configured to perform maximum pooling on the input image features; the channel concatenation unit is configured to concatenate the input image features according to channels; the second internal convolution unit can be a convolution unit implemented based on a standard convolution network or directly implemented by a CNN; and the second internal activation unit is configured to perform normalization on the image features output by the second internal convolution unit (for example, by using a Sigmoid function to perform normalization on the image features output by the second internal convolution unit).

[0108] Please refer to Figure 3F Exemplarily, the second network layer can include an up-sampling module and a down-sampling module.

[0109] The up-sampling module can include a first Ghost Shuffle Convolution (GSC) unit, a first tensor concatenation (Concat) unit, a first bottleneck unit, a second GSC unit, and a second Concat unit. Here, the first GSC unit is configured to perform convolution processing on the third feature representation map; the first Concat unit is configured to perform tensor concatenation on the second feature representation map and the image feature output by the first GSC unit after up-sampling processing, and sequentially input the first bottleneck unit and the second GSC unit for processing after up-sampling processing; and the second Concat unit is configured to perform tensor concatenation on the first feature representation map and the image feature output by the second GSC unit after up-sampling processing, to obtain the output image feature.

[0110] The down-sampling module can include a second bottleneck unit, a third GSC unit, a third Concat unit, a third bottleneck unit, a fourth GSC unit, a fourth Concat unit, and a fourth bottleneck unit. Here, the second bottleneck unit is configured to process the image feature output by the second GSC unit to obtain a first fusion feature map; the third GSC unit is configured to perform convolution processing on the first fusion feature map; the third Concat unit is configured to perform tensor concatenation on the image feature output by the second GSC unit and the image feature output by the third GSC unit after down-sampling processing, to obtain the output image feature, and sequentially input the third bottleneck unit and the fourth GSC unit for processing after down-sampling processing, wherein the image feature output by the third bottleneck unit is the second fusion feature map; the fourth Concat unit is configured to perform tensor concatenation on the image feature output by the first GSC unit and the image feature output by the fourth GSC unit after down-sampling processing, to obtain the output image feature; and the fourth bottleneck unit is configured to process the image feature output by the fourth Concat unit after down-sampling processing to obtain a third fusion feature map.

[0111] In this example, the first bottleneck unit, the second bottleneck unit, the third bottleneck unit, and the fourth bottleneck unit can have the same structure. More specifically, please refer to Figure 3G , the first bottleneck unit, the second bottleneck unit, the third bottleneck unit, and the fourth bottleneck unit can include:

[0112] a first convolution subunit in cascade and a bottleneck subunit composed of a bottleneck layer (such as a Bottleneck layer) and a channel split and concatenate (CSP) layer;

[0113] a second convolution subunit in parallel with the first convolution subunit and the bottleneck subunit in cascade;

[0114] a Concat subunit taking the image features output by the bottleneck subunit and the second convolution subunit as input;

[0115] a third convolution subunit concatenated with the Concat subunit.

[0116] The first convolution subunit, the second convolution subunit, and the third convolution subunit can be convolution units implemented based on a standard convolution network; the bottleneck layer is used to reduce unnecessary parameter quantity in the neural network calculation process; and the CSP layer is used to perform split, concatenation, shuffle, and the like on the input image features.

[0117] Please refer to Figure 3H For example, the third network layer can include a first detection unit for connecting with the second bottleneck unit, a second detection unit for connecting with the third bottleneck unit, and a third detection unit for connecting with the fourth bottleneck unit.

[0118] The first detection unit, the second detection unit, and the third detection unit can be convolution units implemented based on a standard convolution network.

[0119] Based on the image features output by the first detection unit, the second detection unit, and the third detection unit, a defect detection result can be obtained.

[0120] In the embodiments of the present disclosure, the defect detection network includes a first network layer, a second network layer, and a third network layer. Since the first network layer is used to combine the channel attention mechanism and the spatial attention mechanism to perform feature extraction on the filament image to obtain K feature representation maps with different scales, the feature representation capability of the K feature representation maps can be ensured. In this way, after the second network layer is used to perform feature fusion processing based on the K feature representation maps to obtain K fused feature maps, and the third network layer is used to obtain a defect detection result based on the K fused feature maps, the reliability of the defect detection result can be improved, thereby further improving the accuracy of the tray recycling.

[0121] In addition, it should be noted that in the embodiments of the present disclosure, the defect detection network can be trained through the following process:

[0122] (1) Obtain a filament image sample and a data label corresponding to the filament image sample; wherein the data label is used to represent whether the filament image has a filament, oil stain, filament jam, poor molding, or paper tube damage, and the like.

[0123] (2) input the filament image sample into an initial detection model to obtain image features output by the initial detection model.

[0124] (3) obtaining a defect prediction result based on the image features output by the initial detection model.

[0125] (4) calculating a loss value between the defect prediction result and the data label by using a preset loss function, adjusting the model parameters of the initial detection model in a case where the loss value does not satisfy a preset convergence condition, and taking the initial detection model as the defect detection model in a case where the loss value satisfies the preset convergence condition.

[0126] The preset loss function can be a shape and intersection over union loss function (SIoU).

[0127] Further, in the embodiments of the present disclosure, the tray diversion device includes a first diversion device and a second diversion device arranged on the main line of the assembly line, and the branch lines of the assembly line include a first branch line and a second branch line. Based on this, in some optional embodiments, the step of "diverting the to-be-recycled tray from the main line of the assembly line to the branch line of the assembly line by controlling the tray diversion device" in step S102 can include the following steps:

[0128] In step S1021, the first to-be-recycled tray is diverted from the main line of the assembly line to the first branch line of the assembly line by controlling the first diversion device.

[0129] The first diversion device can receive the tray identifier of the first to-be-recycled tray sent by the electronic device as a first target identifier.

[0130] When the N spool trays included in the target tray group pass through the first diversion device, the first diversion device can determine whether there is a to-be-recycled tray with a tray identifier matching the first target identifier in the N spool trays. In a case where there is a to-be-recycled tray with a tray identifier matching the first target identifier in the N spool trays, the to-be-recycled tray is confirmed as the first to-be-recycled tray, and the first to-be-recycled tray is diverted from the main line of the assembly line to the first branch line of the assembly line.

[0131] In step S1022, the second to-be-recycled tray is diverted from the main line of the assembly line to the second branch line of the assembly line by controlling the second diversion device.

[0132] The second diversion device can receive the tray identifier of the second to-be-recycled tray sent by the electronic device as a second target identifier.

[0133] When the other spool trays except the first to-be-recycled tray in the N spool trays included in the target tray group pass through the second shunting device, the second shunting device can determine whether there is a to-be-recycled tray with a tray identification matching the second target identification in the other spool trays, and in the case that there is a to-be-recycled tray with a tray identification matching the second target identification in the other spool trays, the to-be-recycled tray is confirmed as the second to-be-recycled tray, and the second to-be-recycled tray is shunted from the main line of the assembly line to the second branch line of the assembly line.

[0134] Please refer to Figure 4A , for example, the tray shunting device includes a first shunting device 4021 and a second shunting device 4022 arranged on the main line 401 of the assembly line, and the branch line of the assembly line includes a first branch line 4031 and a second branch line 4032. The target tray group includes five spool trays (i.e., N = 5) - a first spool tray 4041, a second spool tray 4042, a third spool tray 4043, a fourth spool tray 4044, and a fifth spool tray 4045. Among them, the second spool tray 4042 is the first to-be-recycled tray, and the fourth spool tray 4044 and the fifth spool tray 4045 are the second to-be-recycled tray.

[0135] Then, when the five spool trays included in the target tray group pass through the first shunting device 4021, the first shunting device 4021 can determine that the tray identification of the second spool tray 4042 matches the first target identification after reading the tray identification of the second spool tray 4042 by radio frequency identification (RFID), and confirm the second spool tray 4042 as the first to-be-recycled tray, and then shunt the first to-be-recycled tray from the main line 401 of the assembly line to the first branch line 4031 of the assembly line; when the other spool trays (i.e., the first spool tray 4041, the third spool tray 4043, the fourth spool tray 4044, and the fifth spool tray 4045) except the first to-be-recycled tray (i.e., the second spool tray 4042) in the five spool trays included in the target tray group pass through the second shunting device 4022, the second shunting device 4022 can determine that the fourth spool tray 4044 and the fifth spool tray 4045 have tray identifications matching the second target identification in the other spool trays after reading the tray identifications of the other spool trays by RFID, and confirm the fourth spool tray 4044 and the fifth spool tray 4045 as the second to-be-recycled tray, and then shunt the second to-be-recycled tray from the main line 401 of the assembly line to the second branch line 4032 of the assembly line.

[0136] Through the above steps included in step S102, the first to-be-recycled tray and the second to-be-recycled tray are shunted to different branches for different processing, so as to ensure the accuracy of tray recycling. For example, the first to-be-recycled tray shunted to the first branch can finally be shunted to the tray recycling place for forming a new target tray group and entering the main line source of the assembly line again; for example, the second to-be-recycled tray shunted to the second branch can enter the re-inspection process to determine whether the to-be-packaged spool placed on the second to-be-recycled tray is indeed a grade spool, so as to avoid misjudgment.

[0137] As described above, in the embodiment of the present disclosure, the second to-be-recycled tray shunted to the second branch can enter the re-inspection process to determine whether the to-be-packaged spool placed on the second to-be-recycled tray is indeed a grade spool, so as to avoid misjudgment. In this case, in the embodiment of the present disclosure, the tray shunting device can further include a third shunting device arranged on the second branch, and the branch of the assembly line further includes a third branch. Based on this, in some optional embodiments, after step S1022 is performed, step S102 can further include one of the following steps:

[0138] Step S1023, by controlling the third shunting device, in the case that the re-inspection result of the second to-be-recycled tray indicates that the to-be-packaged spool placed on the second to-be-recycled tray is a grade spool, the second to-be-recycled tray is shunted from the second branch to the tray recycling place.

[0139] Wherein, the purpose of re-inspecting the second to-be-recycled tray is to determine whether the to-be-packaged spool placed on the second to-be-recycled tray is indeed a grade spool, so as to avoid misjudgment. For example, in some cases, the surface of the to-be-packaged spool placed on the spool tray can be attached with cleanable lines, but when the image is determined to exist the to-be-recycled tray in the target tray group based on the grabbing result, these spool trays can be misjudged as the second to-be-recycled tray. In addition, in the embodiment of the present disclosure, the re-inspection process can be realized by machine vision method, or can be performed by manual, and the present disclosure does not limit this.

[0140] In an example, the re-inspection result can be recorded in a manner of scanning the tray identification. Specifically, in a case that the second to-be-recycled tray is re-inspected and it is determined that the to-be-packaged silk spool placed on the second to-be-recycled tray is a grade silk spool, the tray identification of the second to-be-recycled tray can be scanned as a third target identification, and the third target identification is sent to the electronic device for recording, and then the electronic device sends the third target identification to the third shunting device. In this way, when the second to-be-recycled tray passes through the third shunting device, the third shunting device can determine whether the tray identification of the second to-be-recycled tray matches the third target identification, and in a case that the tray identification of the second to-be-recycled tray matches the third target identification, it is determined that the re-inspection result of the second to-be-recycled tray indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray is a grade silk spool, and the second to-be-recycled tray is shunted from the second branch line to the tray recycling position.

[0141] In another example, the re-inspection result can be recorded indirectly in a manner of taking off the to-be-packaged silk spool. Specifically, in a case that the second to-be-recycled tray is re-inspected and it is determined that the to-be-packaged silk spool placed on the second to-be-recycled tray is a grade silk spool, the to-be-packaged silk spool placed on the second to-be-recycled tray can be taken off from the second to-be-recycled tray, so that the second to-be-recycled tray becomes an empty second to-be-recycled tray. Thereafter, the empty second to-be-recycled tray will pass through the empty tray detection device on the second branch line, and in a case that the empty tray detection device detects that the second to-be-recycled tray is empty, the tray identification of the empty second to-be-recycled tray can be recorded as a fourth target identification, and the fourth target identification is sent to the electronic device for recording, and then the electronic device sends the fourth target identification to the third shunting device. In this way, when the second to-be-recycled tray passes through the third shunting device, the third shunting device can determine whether the tray identification of the second to-be-recycled tray matches the fourth target identification, and in a case that the tray identification of the second to-be-recycled tray matches the fourth target identification, it is determined that the re-inspection result of the second to-be-recycled tray indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray is a grade silk spool, and the second to-be-recycled tray is shunted from the second branch line to the tray recycling position. The empty tray detection device can detect whether the second to-be-recycled tray is empty by using a machine vision method.

[0142] In a case that the re-inspection result of the second to-be-recycled tray indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray is not a grade silk spool, the second to-be-recycled tray is shunted from the second branch line to the third branch line by the third shunting device, so as to be flowed back to the main line of the assembly line from the third branch line.

[0143] In an example, the re-inspection result is recorded in a manner of scanning the tray identification. Then, when the second recycling tray passes through the third shunting device, the third shunting device can determine whether the tray identification of the second recycling tray matches the third target identification. In the case that the tray identification of the second recycling tray does not match the third target identification, it is determined that the re-inspection result of the second recycling tray represents that the to-be-packaged spool on the second recycling tray is a non-grade spool (for example, an AA grade spool product), and the second recycling tray is shunted from the second branch line to the third branch line to flow back to the main line of the assembly line from the third branch line, and then enters the next process, such as the weighing, external inspection, bagging, stacking, banding, film wrapping, and label pasting processes.

[0144] In another example, the re-inspection result is recorded in a manner of removing the to-be-packaged spool. Then, when the second recycling tray passes through the third shunting device, the third shunting device can determine whether the tray identification of the second recycling tray matches the fourth target identification. In the case that the tray identification of the second recycling tray does not match the fourth target identification, it is determined that the re-inspection result of the second recycling tray represents that the to-be-packaged spool on the second recycling tray is a non-grade spool (for example, an AA grade spool product), and the second recycling tray is shunted from the second branch line to the third branch line to flow back to the main line of the assembly line from the third branch line, and then enters the next process, such as the weighing, external inspection, bagging, stacking, banding, film wrapping, and label pasting processes.

[0145] Continue Figure 4A The provided example, in combination with Figure 4B Assuming that, in the example, the re-inspection result is recorded in a manner of scanning the tray identification, and the re-inspection result of the second recycling tray 4044 represents that the to-be-packaged spool on the second recycling tray 4044 is a grade spool, and the re-inspection result of the second recycling tray 4045 represents that the to-be-packaged spool on the second recycling tray 4045 is a non-grade spool. In addition, when the second recycling tray 4044 is re-inspected and it is determined that the to-be-packaged spool on the second recycling tray 4044 is a grade spool, the tray identification of the second recycling tray 4044 has been scanned as the third target identification, and the third target identification is sent to the electronic device for recording, and then the third target identification is sent to the third shunting device 4023 by the electronic device.

[0146] Then, please refer to Figure 4CWhen the second to-be-recycled tray 4044 passes through the third shunting device 4023, the third shunting device 4023 can determine, after reading the tray identifier of the second to-be-recycled tray 4044 by using the RFID, that the tray identifier of the second to-be-recycled tray 4044 matches the third target identifier, to determine that the re-inspection result of the second to-be-recycled tray 4044 indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray 4044 is a grade silk spool, and shunt the second to-be-recycled tray 4044 from the second branch line 4032 to the tray recycling position; when the second to-be-recycled tray 4045 passes through the third shunting device 4023, the third shunting device 4023 can determine, after reading the tray identifier of the second to-be-recycled tray 4045 by using the RFID, that the tray identifier of the second to-be-recycled tray 4045 does not match the third target identifier, to determine that the re-inspection result of the second to-be-recycled tray 4045 indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray 4045 is not a grade silk spool, and shunt the second to-be-recycled tray 4045 from the second branch line 4032 to the third branch line 4033, to flow back from the third branch line 4033 to the main line 401 of the assembly line, and then enter the next process, such as weighing, external inspection, bagging, stacking, banding, film wrapping, and label pasting.

[0147] Continue Figure 4A The provided example, in combination with Figure 4D , it is assumed that, in this example, the re-inspection result is indirectly recorded by removing the to-be-packaged silk spool, and the re-inspection result of the second to-be-recycled tray 4044 indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray 4044 is a grade silk spool, and the re-inspection result of the second to-be-recycled tray 4045 indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray 4045 is not a grade silk spool. In addition, when the second to-be-recycled tray 4044 is re-inspected and determined to be a grade silk spool, the to-be-packaged silk spool placed on the second to-be-recycled tray 4044 has been removed from the second to-be-recycled tray 4044, so that the second to-be-recycled tray 4044 becomes an empty second to-be-recycled tray 4044. Thereafter, the empty second to-be-recycled tray 4044 will pass through the empty tray detection device 405 on the second branch line 4032, and when the empty tray detection device 405 detects that the second to-be-recycled tray 4044 is empty, the tray identifier of the empty second to-be-recycled tray 4044 can be recorded as a fourth target identifier, and the fourth target identifier can be sent to the electronic device for recording, and then the fourth target identifier can be sent to the third shunting device 4023 by the electronic device.

[0148] Then, please combine Figure 4EWhen the second to-be-recycled tray 4044 passes through the third shunting device 4023, the third shunting device 4023 can determine, after reading the tray identifier of the second to-be-recycled tray 4044 by using the RFID, that the tray identifier of the second to-be-recycled tray 4044 matches the fourth target identifier, to determine that the re-inspection result of the second to-be-recycled tray 4044 indicates that the to-be-packaged spool on the second to-be-recycled tray 4044 is a grade spool, and to shunt the second to-be-recycled tray 4044 from the second branch line 4032 to the tray recycling position; when the second to-be-recycled tray 4045 passes through the third shunting device 4023, the third shunting device 4023 can determine, after reading the tray identifier of the second to-be-recycled tray 4045 by using the RFID, that the tray identifier of the second to-be-recycled tray 4045 does not match the fourth target identifier, to determine that the re-inspection result of the second to-be-recycled tray 4045 indicates that the to-be-packaged spool on the second to-be-recycled tray 4045 is not a grade spool, and to shunt the second to-be-recycled tray 4045 from the second branch line 4032 to the third branch line 4033, so as to flow back to the main line 401 of the assembly line from the third branch line 4033 and enter the next process, such as the weighing process, the external inspection process, the bagging process, the stacking process, the banding process, the film wrapping process, and the label pasting process.

[0149] Through the above steps included in step S102, in the embodiment of the present disclosure, the second to-be-recycled tray can be re-shunted after it is further determined whether the to-be-packaged spool placed on the second to-be-recycled tray is indeed a grade spool, so as to further ensure the accuracy of the tray recycling.

[0150] In order to better implement the above tray recycling method, the embodiment of the present disclosure further provides a tray recycling device applied to an electronic device included in a spool packaging system, and the electronic device is in communication with an image acquisition device and a tray shunting device. The electronic device can be a computer, a PLC, etc.; the image acquisition device can be an industrial camera; and the tray shunting device can be a pneumatic shunting device, an electric shunting device, a roller shunting device, etc., which are not limited in the embodiment of the present disclosure.

[0151] In the following, a tray recycling device 500 provided by the embodiment of the present disclosure will be described in combination with the structural schematic diagram shown in FIG. 5. Figure 5

[0152] The tray recycling device 500 includes:

[0153] The image acquisition unit 501 is configured to control the image acquisition device to shoot in a direction where the N hand grabs are located after the robot grabs M to-be-packaged spools from the spool trolley by using the N hand grabs, to obtain a grabbing result representation image; wherein each hand grab in the N hand grabs can grab one to-be-packaged spool, N≥2 and N is an integer, 0≤M≤N and M is an integer.

[0154] ​The recovery control unit 502 is configured to, after the robot places the M to-be-packaged spools in the M spool trays included in the target tray group on the main line of the flow line one by one, based on the image representing the grabbing result, determine that the target tray group includes a to-be-recovered tray, and control the tray shunting device to shunt the to-be-recovered tray from the main line of the flow line to the branch line of the flow line, so as to realize recovery processing of the to-be-recovered tray; wherein the to-be-recovered tray is the first to-be-recovered tray or the second to-be-recovered tray.

[0155] In some optional embodiments, the electronic device is also in communication with the robot;

[0156] The image acquisition unit 501 is configured to:

[0157] Control the N hand grabs to rotate relative to the image acquisition device;

[0158] During the rotation of the N hand grabs relative to the image acquisition device, control the image acquisition device to capture images in the direction of the N hand grabs to obtain Z to-be-processed images; wherein Z is an integer greater than or equal to 2, and the Z to-be-processed images correspond to different perspectives of the N hand grabs;

[0159] Based on the Z to-be-processed images, obtain an image representing the grabbing result.

[0160] In some optional embodiments, the image acquisition unit 501 is configured to:

[0161] From each of the Z to-be-processed images, extract N single-perspective images to obtain N×Z single-perspective images; wherein the N single-perspective images correspond one-to-one to the N hand grabs;

[0162] Splice the Z single-perspective images corresponding to the same hand grab in the N×Z single-perspective images to obtain N multi-perspective images;

[0163] Based on the N multi-perspective images, obtain an image representing the grabbing result.

[0164] In some optional embodiments, the image representing the grabbing result includes the N multi-perspective images, and the N multi-perspective images correspond one-to-one to the N hand grabs, and the target tray group includes N spool trays, and the N spool trays correspond one-to-one to the N hand grabs, so as to establish a one-to-one correspondence between the N spool trays and the N multi-perspective images;

[0165] The recovery control unit 502 is configured to:

[0166] For each of the N multi-view images, in a case where it is determined that there is no spool image in the multi-view image for representing a spool to be packaged, it is determined that there is a to-be-recycled tray in the target tray group, and a spool tray corresponding to the multi-view image in the N spool trays is taken as a first to-be-recycled tray;

[0167] In a case where it is determined that there is a spool image in the multi-view image for representing a spool to be packaged, and it is determined by the defect detection network that the spool image has a defect feature, it is determined that there is a to-be-recycled tray in the target tray group, and a spool tray corresponding to the multi-view image in the N spool trays is taken as a second to-be-recycled tray.

[0168] In some optional embodiments, the defect detection network includes a first network layer, a second network layer, and a third network layer.

[0169] The first network layer is configured to perform feature extraction on the spool image by combining a channel attention mechanism and a spatial attention mechanism, to obtain K feature representation maps of different scales; K is an integer and K≥2.

[0170] The second network layer is configured to perform feature fusion processing based on the K feature representation maps, to obtain K fusion feature maps; the K fusion feature maps correspond to the K feature representation maps. Figure 1

[0171] The third network layer is configured to obtain a defect detection result based on the K fusion feature maps; the defect detection result is used to represent whether the spool image has a defect feature.

[0172] In some optional embodiments, the tray shunting device includes a first shunting device and a second shunting device arranged on a main line of the assembly line, and the branch lines of the assembly line include a first branch line and a second branch line.

[0173] The recycling control unit 502 is configured to:

[0174] The first to-be-recycled tray is shunted from the main line of the assembly line to the first branch line of the assembly line by controlling the first shunting device.

[0175] The second to-be-recycled tray is shunted from the main line of the assembly line to the second branch line of the assembly line by controlling the second shunting device.

[0176] In some optional embodiments, the tray shunting device further includes a third shunting device arranged on the second branch line, and the branch lines of the assembly line further include a third branch line.

[0177] The recycling control unit 502 is configured to:

[0178] ​By controlling the third shunting device, the second to-be-recycled tray is shunted from the second branch line to the tray recycling position in a case that the re-inspection result of the second to-be-recycled tray indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray is a grade product silk spool;

[0179] Alternatively, by the third shunting device, the second to-be-recycled tray is shunted from the second branch line to the third branch line in a case that the re-inspection result of the second to-be-recycled tray indicates that the to-be-packaged silk spool placed on the second to-be-recycled tray is a non-grade product silk spool, so as to be shunted from the third branch line to the main line of the assembly line.

[0180] The specific functions and examples of the units in the tray recycling device provided by the embodiments of the present disclosure can be referred to the related descriptions of the corresponding steps in the method embodiments, which will not be repeated here.

[0181] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0182] Figure 6 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in the figure, the electronic device includes a memory 601 and a processor 602, and the memory 601 stores a computer program that can run on the processor 602. The number of memories 601 and processors 602 can be one or more. The memory 601 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the method provided in the above method embodiments. The electronic device can also include a communication interface 603 for communicating with external devices and transmitting data. Figure 6

[0183] If the memory 601, the processor 602 and the communication interface 603 are independently implemented, the memory 601, the processor 602 and the communication interface 603 can be connected to each other through a bus and complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0184] ​Optionally, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can complete the communication among each other through an internal interface.

[0185] It should be understood that the processor described above can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It should be noted that the processor can be a processor supporting an advanced RISC machine (ARM) architecture.

[0186] Further, the aforementioned memory can include a read-only memory and a random access memory, and can also include a non-volatile random access memory. The memory can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can include a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used. For example, a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced SDRAM (ESDRAM), a Synchlink DRAM (SLDRAM), and a direct Rambus RAM (DR RAM) can be used.

[0187] In the above embodiments, all or part of the steps can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the steps can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present disclosure are generated. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (for example: coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example: infrared, Bluetooth, microwave, etc.) mode. The computer readable storage medium can be any available medium accessible by a computer, or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example: floppy disk, hard disk, magnetic tape), an optical medium (for example: digital versatile disc (DVD)) or a semiconductor medium (for example: solid state disk (SSD)) and the like. It is worth noting that the computer readable storage medium mentioned in the present disclosure can be a non-volatile storage medium, in other words, it can be a non-transitory storage medium.

[0188] A person of ordinary skill in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or the program can instruct the related hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0189] In the description of the embodiments of the present disclosure, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0190] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or, for example, A / B can mean A or B. "And / or" in this document only describes the relationship between associated objects, which means that there can be three relationships, for example, A and / or B, which can mean: A exists alone, A and B exist together, and B exists alone.

[0191] In the description of the embodiments of the present disclosure, the terms "first", "second", "third" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", "third" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more.

[0192] The above only describes exemplary embodiments of the present disclosure, and is not intended to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A pallet recycling method, applied to electronic equipment included in a spindle packaging system, wherein the electronic equipment communicates with an image acquisition device and a pallet diversion device respectively; in, The pallet recycling method includes: After the robot uses N grippers to grab M spindles to be packaged from the spindle cart, the image acquisition device is controlled to take pictures in the direction of the N grippers to obtain a characterization image of the grabbing result; wherein, each of the N grippers can grab one spindle to be packaged, N≥2 and N is an integer, 0≤M≤N and M is an integer; After the robot places the M spools to be packaged one by one onto the M spool trays included in the target tray group set on the main line of the production line, and if it is determined based on the image representing the grasping result that there are trays to be recycled in the target tray group, the tray diversion device is controlled to divert the trays to be recycled from the main line of the production line to the branch line of the production line, so as to realize the recycling of the trays to be recycled; wherein, the trays to be recycled are either empty first trays to be recycled or second trays containing spools to be packaged that have been preliminarily assessed as graded spools to be recycled.

2. The pallet recycling method according to claim 1, wherein, The electronic device also communicates with the robot; The method of controlling the image acquisition device to capture images in the direction of the N grasping hands to obtain a grasping result characterization image includes: Control the N grippers to rotate relative to the image acquisition device; During the rotation of the N grippers relative to the image acquisition device, the image acquisition device is controlled to take pictures in the direction of the N grippers, resulting in Z images to be processed; where Z≥2 and Z is an integer, the Z images to be processed correspond to different viewpoints of the N grippers; Based on the Z images to be processed, the characterization image of the capture result is obtained.

3. The pallet recycling method according to claim 2, wherein, The process of obtaining the characterization image of the capture result based on the Z images to be processed includes: N single-view images are extracted from each of the Z images to be processed, resulting in N×Z single-view images; wherein, the N single-view images correspond one-to-one with the N hand grasps; The Z single-view images corresponding to the same hand grasping from the N×Z single-view images are stitched together to obtain N multi-view images; Based on the N multi-view images, the characterization image of the capture result is obtained.

4. The pallet recycling method according to claim 1, wherein, The image representing the grasping result includes N multi-view images, and the N multi-view images correspond one-to-one with the N hand grippers. The target tray group includes N silk spindle trays, and the N silk spindle trays correspond one-to-one with the N hand grippers, so as to establish a one-to-one correspondence between the N silk spindle trays and the N multi-view images. The step of determining whether there is a tray to be recycled in the target tray group based on the image representing the capture result includes: For each of the N multi-view images, if it is determined that there is no image of a spindle to be packaged in the multi-view images, it is determined that there is a tray to be recycled in the target tray group, and the spindle tray corresponding to the multi-view image among the N spindle trays is taken as the first tray to be recycled. If it is determined that there is a spindle image in the multi-view image that represents the spindle to be packaged, and the defect detection network determines that there are defect features in the spindle image, then it is determined that there is a tray to be recycled in the target tray group, and the spindle tray corresponding to the multi-view image among the N spindle trays is taken as the second tray to be recycled.

5. The pallet recycling method according to claim 4, wherein, The defect detection network includes a first network layer, a second network layer, and a third network layer; The first network layer is used to combine channel attention mechanism and spatial attention mechanism to extract features from the spindle image and obtain K feature representation maps of different scales; where K≥2 and K is an integer; The second network layer is used to perform feature fusion processing based on the K feature representation maps to obtain K fused feature maps; wherein, the K fused feature maps correspond one-to-one with the K feature representation maps; The third network layer is used to obtain defect detection results based on the K fused feature maps; wherein, the defect detection results are used to characterize whether there are defect features in the spindle image.

6. The pallet recycling method according to claim 1, wherein, The pallet diversion device includes a first diversion device and a second diversion device disposed on the main line of the assembly line, and the branch line of the assembly line includes a first branch line and a second branch line; The step of controlling the pallet diversion device to divert the pallets to be recycled from the main line of the production line to a branch line of the production line includes: By controlling the first diversion device, the first tray to be recycled is diverted from the main line of the production line to the first branch line of the production line; By controlling the second diversion device, the second tray to be recycled is diverted from the main line of the production line to the second branch line of the production line.

7. The pallet recycling method according to claim 6, wherein, The pallet diversion device also includes a third diversion device disposed on the second branch line, and the branch line of the production line also includes a third branch line; The method of controlling the pallet diversion device to divert the pallets to be recycled from the main line of the production line to the branch line of the production line further includes: By controlling the third diversion device, if the re-inspection result of the second tray to be recycled indicates that the yarn spindle to be packaged placed on the second tray to be recycled is a graded yarn spindle, the second tray to be recycled is diverted from the second branch line to the tray recycling point. Alternatively, if the re-inspection result of the second recycling tray indicates that the silk spindle to be packaged on the second recycling tray is a non-grade silk spindle, the second recycling tray is diverted from the second branch line to the third branch line by the third branch line, so that it can be returned from the third branch line to the main line of the production line.

8. A pallet recycling device, applied to electronic equipment included in a spindle packaging system, wherein the electronic equipment communicates with an image acquisition device and a pallet diversion device respectively; in, The pallet recycling device includes: The image acquisition unit is used to control the image acquisition device to take pictures in the direction of the N grippers after the robot has picked up M silk spindles to be packaged from the silk spindle cart by N grippers, so as to obtain a characterization image of the picking result; wherein, each of the N grippers can pick up one silk spindle to be packaged, N≥2 and N is an integer, 0≤M≤N and M is an integer; The recycling control unit is configured to, after the robot places the M spools to be packaged one-to-one onto the M spool trays included in the target tray group set on the main line of the production line, and if it is determined based on the grasping result characterization image that there are trays to be recycled in the target tray group, control the tray diversion device to divert the trays to be recycled from the main line of the production line to the branch line of the production line, so as to realize the recycling processing of the trays to be recycled; wherein, the trays to be recycled are either an empty first tray to be recycled or a second tray to be recycled containing spools to be packaged that have been preliminarily assessed as graded spools.

9. The pallet recycling device according to claim 8, wherein, The electronic device also communicates with the robot; The image acquisition unit is used for: Control the N grippers to rotate relative to the image acquisition device; During the rotation of the N grippers relative to the image acquisition device, the image acquisition device is controlled to take pictures in the direction of the N grippers, resulting in Z images to be processed; where Z≥2 and Z is an integer, the Z images to be processed correspond to different viewpoints of the N grippers; Based on the Z images to be processed, the characterization image of the capture result is obtained.

10. The pallet recycling device according to claim 9, wherein, The image acquisition unit is used for: N single-view images are extracted from each of the Z images to be processed, resulting in N×Z single-view images; wherein, the N single-view images correspond one-to-one with the N hand grasps; The Z single-view images corresponding to the same hand grasping from the N×Z single-view images are stitched together to obtain N multi-view images; Based on the N multi-view images, the characterization image of the capture result is obtained.

11. The pallet recycling device according to claim 8, wherein, The image representing the grasping result includes N multi-view images, and the N multi-view images correspond one-to-one with the N hand grippers. The target tray group includes N silk spindle trays, and the N silk spindle trays correspond one-to-one with the N hand grippers, so as to establish a one-to-one correspondence between the N silk spindle trays and the N multi-view images. The recycling control unit is used for: For each of the N multi-view images, if it is determined that there is no image of a spindle to be packaged in the multi-view images, it is determined that there is a tray to be recycled in the target tray group, and the spindle tray corresponding to the multi-view image among the N spindle trays is taken as the first tray to be recycled. If it is determined that there is a spindle image in the multi-view image that represents the spindle to be packaged, and the defect detection network determines that there are defect features in the spindle image, then it is determined that there is a tray to be recycled in the target tray group, and the spindle tray corresponding to the multi-view image among the N spindle trays is taken as the second tray to be recycled.

12. The pallet recycling device according to claim 11, wherein, The defect detection network includes a first network layer, a second network layer, and a third network layer; The first network layer is used to combine channel attention mechanism and spatial attention mechanism to extract features from the spindle image and obtain K feature representation maps of different scales; where K≥2 and K is an integer; The second network layer is used to perform feature fusion processing based on the K feature representation maps to obtain K fused feature maps; wherein, the K fused feature maps correspond one-to-one with the K feature representation maps; The third network layer is used to obtain defect detection results based on the K fused feature maps; wherein, the defect detection results are used to characterize whether there are defect features in the spindle image.

13. The pallet recycling device according to claim 8, wherein, The pallet diversion device includes a first diversion device and a second diversion device disposed on the main line of the assembly line, and the branch line of the assembly line includes a first branch line and a second branch line; The recycling control unit is used for: By controlling the first diversion device, the first tray to be recycled is diverted from the main line of the production line to the first branch line of the production line; By controlling the second diversion device, the second tray to be recycled is diverted from the main line of the production line to the second branch line of the production line.

14. The pallet recycling device according to claim 13, wherein, The pallet diversion device also includes a third diversion device disposed on the second branch line, and the branch line of the production line also includes a third branch line; The recycling control unit is used for: By controlling the third diversion device, if the re-inspection result of the second tray to be recycled indicates that the yarn spindle to be packaged placed on the second tray to be recycled is a graded yarn spindle, the second tray to be recycled is diverted from the second branch line to the tray recycling point. Alternatively, if the re-inspection result of the second recycling tray indicates that the silk spindle to be packaged on the second recycling tray is a non-grade silk spindle, the second recycling tray is diverted from the second branch line to the third branch line by the third branch line, so that it can be returned from the third branch line to the main line of the production line.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the tray recycling method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the pallet recycling method according to any one of claims 1 to 7.

Images