Control method, apparatus, device, and storage medium

Abstract

The present disclosure provides a control method, device, equipment and storage medium. The method comprises: in the case of conveying a first silk spool with a first identifier on a conveying channel, and in the case of determining that the number of the first silk spool located in a preset area needs to be detected, detecting a first number of the first silk spool located in the preset area; the conveying channel is a channel for conveying silk spools in an automatic packaging workshop, and is used for conveying the conveyed silk spool to the preset area; the preset area is a working area that can be irradiated by a first mechanical arm; in the case that the first number of the first silk spool located in the preset area is less than the total number of silk spools that can be grasped by the first mechanical arm, determining a target number of the first silk spool that needs to be supplemented, and generating a first control instruction; the first control instruction is used to instruct a second mechanical arm to grasp the target number of the first silk spool from a target reel associated with the first identifier of the first silk spool, and place the target number of the first silk spool in the conveying channel or the preset area.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a control method, apparatus, device and storage medium. Background Technology

[0002] During the palletizing process in the automated packaging workshop, if the number of spools in the working area of ​​the robotic arm is less than the number of spools that the robotic arm can grasp, normal palletizing will not be possible. In addition, if a batch change is required, two batches of spools will appear in the working area of ​​the robotic arm. If the original method is followed, a mixed batch production accident may occur, that is, the batches are mixed, and spools of different batches are stacked together, which will cause significant losses to the manufacturer. Summary of the Invention

[0003] This disclosure provides a control method, apparatus, device, and storage medium to solve or alleviate one or more technical problems in the prior art.

[0004] Firstly, this disclosure provides a control method, including:

[0005] When a first filament spindle with a first identifier is conveyed on a conveying channel, and when it is determined that the quantity of the first filament spindles located within a preset area needs to be detected, a first quantity of the first filament spindles located within the preset area is detected; the conveying channel is a channel for conveying filament spindles in an automated packaging workshop, used to convey the conveyed filament spindles to the preset area; the preset area is a working area that can be reached by a first robotic arm; the first robotic arm is used to grasp multiple filament spindles within the working area that can be reached, and transfer the grasped multiple filament spindles to a target position;

[0006] If the first number of first spindles located in the preset area is less than the total number of spindles that the first robotic arm can grasp, a target number of first spindles to be supplemented is determined, and a first control command is generated; the first control command is used to instruct the second robotic arm to grasp the target number of first spindles from the target spindle cart associated with the first identifier of the first spindle, and place them in the conveying channel or the preset area.

[0007] Secondly, this disclosure provides a control device, comprising:

[0008] A detection unit is used to detect a first quantity of first filament spindles located within a preset area when a first filament spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that the quantity of first filament spindles located within a preset area needs to be detected; the conveying channel is a channel for conveying filament spindles in an automated packaging workshop, used to convey the conveyed filament spindles to the preset area; the preset area is a working area that can be reached by a first robotic arm; the first robotic arm is used to grasp multiple filament spindles within the working area it can reach, and transfer the grasped multiple filament spindles to a target position;

[0009] The processing unit is configured to determine the target number of first spindles to be supplemented when the first number of first spindles located in the preset area is less than the total number of spindles that the first robotic arm can grasp, and generate a first control command; the first control command is configured to instruct the second robotic arm to grasp the target number of first spindles from the target spindle cart associated with the first identifier of the first spindle, and place them in the conveying channel or the preset area.

[0010] Thirdly, an electronic device is provided, comprising:

[0011] At least one processor; and

[0012] The memory is communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0014] Fourthly, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of the present disclosure.

[0015] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of the present disclosure.

[0016] In this way, the present invention can generate control commands when the number of first filament spindles in the preset area is less than the total number of filament spindles that the first robotic arm can grasp, so as to automatically replenish the first filament spindle with the first identifier. This provides support for the smooth automated palletizing and avoids palletizing filament spindles of different batches together. Compared with the existing manual intervention method, the present invention can realize a complete automated process without human intervention, laying the foundation for significantly improving packaging efficiency.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0018] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments provided according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0019] Figure 1 This is a schematic flowchart of a control method according to an embodiment of this application;

[0020] Figures 2(a) and 2(b) are schematic diagrams illustrating an application scenario of the control method according to an embodiment of this application in a specific example;

[0021] Figures 3(a) and 3(b) are schematic diagrams of a control method according to an embodiment of the present application in a specific scenario;

[0022] Figure 4 This is a schematic diagram of the processing flow of the first identification model in the control method according to an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the processing flow of the second identification model in the control method according to an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of the structure of a control device according to an embodiment of this application;

[0025] Figure 7 This is a block diagram of an electronic device used to implement the control method of the embodiments of this disclosure. Detailed Implementation

[0026] The present disclosure will now be described in further detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0027] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0028] In the automated packaging workshop, the yarn spindles in the yarn carts or boxes that have been put into operation are placed on the automated packaging conveyor line. After weighing, sorting and bagging, they are collected in the working area of ​​the robotic arm. The robotic arm picks up the yarn spindles from the working area and places them in the stacking position for stacking. After stacking, the top cover is added, the strap is attached, the film is wrapped, and the label is affixed. Finally, the yarn spindles are taken off the line and transported into the warehouse. Here, during the stacking process, yarn spindles of the same batch number need to be stacked together.

[0029] During the automated packaging of yarn spindles, once the packaging task for the current batch meets the business requirements, the packaging task for the current batch is completed, and the packaging of the next batch of yarn spindles begins—this is known as batch change. However, during batch change, the following situations may occur: During palletizing, the number of yarn spindles in the working area of ​​the robotic arm is less than the number of yarn spindles the robotic arm can grasp. In this case, normal palletizing is not possible; that is, the number of yarn spindles in the current batch is insufficient to form a complete stack, and the robotic arm will be in a waiting state. Alternatively, two batches of yarn spindles may appear in the working area of ​​the robotic arm. If palletizing is performed in the original manner, a mixed batch production accident will occur, meaning that yarn spindles from different batches will be palletized together, affecting subsequent processing. For these scenarios, manual intervention is required to avoid these problems.

[0030] Based on this, the present disclosure proposes a control method to achieve automated processes.

[0031] Specifically, Figure 1 This is a schematic flowchart of a control method according to an embodiment of this application. The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. The method includes at least a portion of the following: Figure 1 As shown, it includes:

[0032] Step S101: When a first spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that the number of first spindles located within a preset area needs to be detected, the first number of first spindles located within the preset area is detected.

[0033] Here, the conveying channel is the channel for conveying yarn spindles in the automated packaging workshop, used to transport the conveyed yarn spindles to the preset area; the preset area is the working area that the first robotic arm can reach. The first robotic arm is used to grasp multiple yarn spindles within its working area and transfer the grasped yarn spindles to a target location. For example, the first robotic arm can grasp multiple yarn spindles from the preset area and transfer them to a stacking position to facilitate subsequent packaging and warehousing of the grasped yarn spindles.

[0034] In a specific example, the first robotic arm may be a gantry robot.

[0035] Step S102: If the first number of first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp, determine the target number of first spindles to be supplemented and generate a first control command.

[0036] Here, the first control command is used to instruct the second robotic arm to grab the target number of first spindles from the target yarn cart associated with the first identifier of the first spindle and place them in the conveying channel or the preset area.

[0037] It should be noted that the target silk carriage associated with the first identifier of the first silk spindle can be specifically a preset silk carriage, and the silk spindle on the preset silk carriage has the first identifier; further, in one example, the preset silk carriage is a specific silk carriage used to replenish silk spindles.

[0038] In one example, the target number of the first spindles to be replenished is based on the first number of first spindles located within the preset area and the total number of spindles that the first robotic arm can grasp; for example, the target number of the first spindles to be replenished = the total number of spindles that the first robotic arm can grasp - the first number of first spindles located within the preset area.

[0039] It is understood that the number of spindles that the preset area can accommodate is related to the total number of spindles that the first robotic arm can grasp. For example, the number of spindles that the preset area can accommodate is equal to the total number of spindles that the first robotic arm can grasp.

[0040] In one example, the identifier (also known as the batch number) of a spindle can specifically include spindle information, such as the spindle's production specifications and product type. It is understood that spindles produced in the same production batch will have the same identifier.

[0041] In this way, the present invention can automatically generate control commands when the first number of first filament spindles with the first identifier is less than the total number of filament spindles that the first robotic arm can grasp, so as to automatically replenish the first filament spindles with the first identifier. This provides support for the smooth automated palletizing. Compared with the existing manual intervention method, the present invention can realize a complete automated process without human intervention, laying the foundation for significantly improving packaging efficiency.

[0042] In addition, since the automatically replenished spindles are spindles associated with the first identifier, such as the first spindle with the first identifier, it effectively avoids the occurrence of two different identifiers in the preset area, and thus effectively avoids stacking spindles with different identifiers together.

[0043] In a specific example of the disclosed solution, the execution conditions for detecting the first quantity of the first spindle located within a preset area are given below; specifically, in the case of conveying the first spindle with the first identifier on the conveying channel, and in the case of determining that it is necessary to detect the quantity of the first spindle located within the preset area, detecting the first quantity of the first spindle located within the preset area (i.e., step S101 described above) specifically includes one of the following:

[0044] Condition 1: When a first spindle with a first identifier is being conveyed on the conveying channel, and when it is determined that a second spindle with a second identifier needs to be conveyed continuously, a first number of first spindles located within a preset area is detected; the second identifier is different from the first identifier.

[0045] In other words, when the first spindle with the first mark is being conveyed on the conveying channel, if the next spindle being conveyed has a different mark, for example, if a batch change is required, the first quantity of the first spindle located in the preset area needs to be detected. This is to avoid the appearance of two batches of spindles in the working area where the first robotic arm is located during the batch change process, and thus avoid stacking spindles of different batches together.

[0046] In a specific example, when condition 1 is used as the condition for performing the detection, the method may specifically include:

[0047] Step 1-1: When conveying a first spindle with a first identifier on the conveying channel, and when it is determined that a second spindle with a second identifier needs to be conveyed continuously, detect the first number of the first spindles located in the preset area; the second identifier is different from the first identifier.

[0048] Step 1-2: Determine whether the first number of the first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp; if yes, proceed to step 1-3; otherwise, proceed to step 1-4.

[0049] Steps 1-3: If the first number of first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp, determine the target number of first spindles to be supplemented and generate a first control command.

[0050] Step 1-4: If the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, determine whether the conveying channel is still conveying other first spindles; if yes, proceed to step 1-5; otherwise, proceed to step 1-6.

[0051] Steps 1-5: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and the conveying channel is also conveying other first spindles, generate a second control command and return to step 1-2.

[0052] Here, the second control command is used to instruct the first number of first spindles located within the preset area to be re-detected after the first robotic arm has completed a grasping task.

[0053] Steps 1-6: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and there are no first spindles in the conveying channel, a third control command is generated, which is used to instruct the first robotic arm to stop operating.

[0054] Condition 2: When a first spindle with a first identifier is being conveyed on the conveying channel, and when it is determined that the first spindle currently placed on the conveying channel is the last first spindle, the first number of first spindles located in the preset area is detected.

[0055] In other words, when a first spindle with a first identifier is being conveyed on the conveying channel, and it is determined that the first spindle with the current identifier is about to be conveyed, the actual number of first spindles located in the preset area may be less than the total number of spindles that the first robotic arm can grasp, which may result in the inability to stack normally. Therefore, in order to avoid the above problem, it is necessary to detect the first number of first spindles located in the preset area.

[0056] In a specific example, when condition 2 is used as the condition for performing the detection, the method may specifically include:

[0057] Step 2-1: When conveying a first spindle with a first identifier on the conveying channel, and when determining that the first spindle currently placed on the conveying channel is the last first spindle, detect the first number of first spindles located in the preset area.

[0058] Step 2-2: Determine whether the first number of the first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp; if yes, proceed to step 2-3; otherwise, proceed to step 2-4.

[0059] Step 2-3: If the first number of first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp, determine the target number of first spindles that need to be supplemented and generate a first control command.

[0060] Step 2-4: If the number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, determine whether the conveying channel is still conveying other first spindles; if yes, proceed to step 2-5; otherwise, proceed to step 2-6.

[0061] Step 2-5: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and the conveying channel is also conveying other first spindles, generate a second control command and return to step 2-2.

[0062] Here, the second control command is used to instruct the first number of first spindles located within the preset area to be re-detected after the first robotic arm has completed a grasping task.

[0063] Steps 2-6: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and there are no first spindles in the conveying channel, a third control command is generated, which is used to instruct the first robotic arm to stop operating.

[0064] Condition 3: When a first spindle with a first mark is being conveyed on the conveying channel, and when it is determined that the total number of first spindles being conveyed on the conveying channel does not reach the preset maximum number that the conveying channel can carry, the first number of first spindles located within the preset area is detected.

[0065] In other words, when the first spindle with the first mark is being conveyed on the conveying channel, and it is determined that the first spindle with the current mark is about to be conveyed, the actual number of the first spindles located in the preset area may be less than the total number of spindles that the first robotic arm can grasp, which may result in the inability to stack normally. Therefore, in order to avoid the above problem, it is necessary to detect the first spindles located in the preset area.

[0066] In a specific example, when condition 3 is used as the condition for performing the detection, the method may specifically include:

[0067] Step 3-1: When conveying the first spindle with the first mark on the conveying channel, and when it is determined that the total number of the first spindles conveyed by the conveying channel does not reach the preset maximum number that the conveying channel can carry, detect the first number of the first spindles located in the preset area.

[0068] Step 3-2: Determine whether the first number of the first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp; if yes, proceed to step 3-3; otherwise, proceed to step 3-4.

[0069] Step 3-3: If the first number of first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp, determine the target number of first spindles that need to be supplemented and generate a first control command.

[0070] Step 3-4: If the number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, determine whether the conveying channel is still conveying other first spindles; if yes, proceed to step 3-5; otherwise, proceed to step 3-6.

[0071] Step 3-5: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and the conveying channel is also conveying other first spindles, generate a second control command and return to step 3-2.

[0072] Here, the second control command is used to instruct the first number of first spindles located within the preset area to be re-detected after the first robotic arm has completed a grasping task.

[0073] Steps 3-6: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and there are no first spindles in the conveying channel, a third control command is generated, which is used to instruct the first robotic arm to stop operating.

[0074] It should be noted that in practical applications, one of the above conditions needs to be selected based on the requirements of the scenario, and this public solution does not impose any restrictions on this.

[0075] In a specific example of the disclosed solution, when a first spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that the number of first spindles located within a preset area needs to be detected, the detection can be performed in the following manner to obtain a first number of first spindles located within the preset area; specifically, the detection of the first number of first spindles located within the preset area as described above includes:

[0076] Acquire the target image containing the preset region;

[0077] The target image is input into the target fusion model to obtain the first number of the first spindles located within the preset area; the target fusion model is used to detect the number of spindles located within the preset area in the target image.

[0078] In other words, when detecting the number of first spindles located within a preset area, a target image containing the preset area is first obtained. Then, the target image is input into a target blending model to determine the first number of first spindles located within the preset area using the target blending model.

[0079] In one example, the target image can be acquired by an image acquisition component, which may specifically include a camera. For example, the target image is obtained by acquiring an image of the preset area using the camera, such as by taking a picture of the preset area to obtain an image, or by acquiring a video of the preset area for a preset duration to obtain multiple consecutive video frames, and then selecting an image from the consecutive video frames as the target image.

[0080] Furthermore, in one example, the target image may also be obtained by preprocessing the image or video data acquired by the image acquisition component; the preprocessing may include, but is not limited to, one or more of the following: scaling, cropping, resizing, flipping, color dithering, grayscale processing, and noise reduction. In practical applications, appropriate preprocessing methods can be selected to process the acquired image according to actual needs.

[0081] It should be noted that, in one example, the preset area is at least a portion of the conveying channel. For example, as shown in Figure 2(a), the second robotic arm can grab the first yarn spindle from the target yarn cart associated with the first identifier of the first yarn spindle and place it on an empty yarn tray in the yarn spindle placement area; the conveying channel transports the yarn tray containing the first yarn spindle from the yarn spindle placement area to the preset area, which is a portion of the conveying channel. The first robotic arm is used to grab the first yarn spindle located in the preset area and transfer the grabbed first yarn spindle to a designated area for stacking.

[0082] It should be noted that after the first robotic arm transfers the first yarn spindle it has grasped to the designated area, there will be an empty yarn tray left in the preset area. At this time, the empty yarn tray can be continued to be transported to the conveying channel to continue to be transported to the yarn spindle placement area, which is convenient for subsequent automated processing.

[0083] Alternatively, in one example, the preset area is an independent area outside the conveying channel. For example, as shown in Figure 2(b), the second robotic arm picks up the first yarn spindle from the target yarn cart associated with the first identifier of the first yarn spindle and places it on an empty yarn tray in the yarn spindle placement area; the conveying channel 1 is used to convey the yarn tray containing the yarn spindle from the yarn spindle placement area to the preset area; the first robotic arm is used to pick up the first yarn spindle located in the preset area and transfer the picked up first yarn spindle to a designated area for stacking.

[0084] It should be noted that after the first robotic arm transfers the first yarn spindle it has grasped to the designated area, there will be an empty yarn tray remaining in the preset area. At this time, the empty yarn tray can be transferred to the conveyor channel 2 and then sent back to the yarn spindle placement area by the conveyor channel 2 to continue transmission to the yarn spindle placement area, facilitating subsequent automated processing. In this scenario, the preset area is independently set between the end of the conveyor channel 1 and the beginning of the conveyor channel 2.

[0085] In a specific example of the disclosed solution, the first number of first spindles located within a preset area can be obtained further by the following method; specifically, the above-mentioned inputting the target image into the target blending model to obtain the first number of first spindles located within the preset area specifically includes:

[0086] Step 1: Input the target image into the first recognition model in the target fusion model to obtain at least one sub-image.

[0087] Here, the first recognition model is used to locate the regions where each target object is located in the target image, and when it is determined that the region where the target object is located is within a preset region, it obtains sub-images corresponding to the regions where each target object is located within the preset region. Furthermore, different sub-images contain different target objects, and the number of sub-images is related to the number of target objects; for example, the number of sub-images may be the same as the number of target objects.

[0088] In other words, in step one, the first recognition model can be used to locate the region where each target object is located in the target image, then determine whether the region where the target object is located is within a preset region, and finally obtain the sub-image corresponding to the region where each target object is located within the preset region.

[0089] In one example, the target body includes, but is not limited to, one of the following: a spindle, or a yarn tray for placing the spindle.

[0090] For example, as shown in Figure 3(a), the target image contains target objects 1 to 10. Target objects 1 to 4, and target object 7 are empty yarn trays, while target objects 5, 6, and 8 are yarn trays containing silk spindles. Target objects 9 and 10 are other objects besides silk spindles and yarn trays. In this scenario, the first recognition model can be used to perform the following processing: locate the regions where each target object is located in the target image (i.e., the dashed areas containing the target objects in Figure 3(a)), that is, locate the regions where target object 1, target object 2, ..., target object 10 are located, and obtain the regions where the target objects are located within the preset regions, such as the regions where target object 1 is located to the regions where target object 6 is located. Finally, based on the regions where target object 1 is located to the regions where target object 6 is located, obtain the sub-images of each region in the regions where target object 1 is located to the regions where target object 6 is located, a total of 6, which are the sub-images corresponding to the regions where target object 1 is located. Figure 1 Subgraph 2 corresponding to the region where target 2 is located, subgraph 3 corresponding to the region where target 3 is located, and subgraph 4 corresponding to the region where target 4 is located. Figure 4 The sub-area corresponding to the target body 5 Figure 5 And the sub-area corresponding to the location of target 6 Figure 6 .

[0091] Furthermore, in a specific example, the first recognition model includes at least a first network layer, a second network layer, and a third network layer.

[0092] Furthermore, such as Figure 4 As shown, the first network layer is used to extract image features of the target image, and based on the image features of the target image, to select the region where the target body is located, thus obtaining a candidate box region; here, the candidate box region is the region selected by the candidate box, which contains the target body.

[0093] In one example, the first network layer mentioned above can be a "feature extractor + region proposal network (RPN)". The feature extractor is used to extract image features of the target image. The feature extractor can include, but is not limited to, recurrent neural networks (RNN), convolutional neural networks (CNN), and feature extraction networks based on transforms. The region proposal network is used to select the region where the target object is located based on the image features of the target image and obtain candidate bounding box regions.

[0094] Furthermore, such as Figure 4As shown, the second network layer is used to filter the candidate box regions obtained in the first network layer. For example, it removes candidate boxes that do not meet the requirements (such as removing candidate box regions whose aspect ratio does not meet the preset aspect ratio requirements) to obtain at least one target box region. In this way, the amount of computation is reduced, laying the foundation for the rapid recognition of the subsequent model.

[0095] Here, the filtering method may include, but is not limited to, determining the filtering conditions based on at least one of the following: score, aspect ratio, area, confidence level, and non-maximum suppression (NMS) to achieve filtering of candidate box regions.

[0096] For example, using the first network layer, candidate box regions of each target body from target body 1 to target body 10 are obtained. Using the second network layer, the first aspect ratio corresponding to the candidate box region of each target body is first determined. Then, candidate box regions whose first aspect ratio is not within the preset aspect ratio range are removed. For example, as shown in Figure 3(b), the candidate box regions of target body 9 and target body 10 are removed, and the remaining candidate box regions are used as target box regions.

[0097] Furthermore, such as Figure 4 As shown, the third network layer is used to determine the first center position of the target box region in the at least one target box region, and select target box regions from the at least one target box region whose positional relationship with the preset center position of the preset region meets preset requirements, thereby obtaining a sub-graph corresponding to the target box region that meets the preset requirements. For example, firstly, the center point coordinates of the target box region are determined. If the distance between the center point coordinates of the target box region and the center point coordinates of the preset region is less than a preset threshold, the target box region with a distance less than the preset threshold is determined as the target box region located in the preset region, thereby obtaining the sub-graph corresponding to the target box region with a distance less than the preset threshold. Here, the preset threshold is an empirical value and can be set according to actual needs. This disclosure does not limit it in this regard.

[0098] Step 2: Input each sub-graph in the at least one sub-graph into the second recognition model in the target hybrid model to obtain the first number of the first spindle located in the preset area.

[0099] Here, the second recognition model is used to identify whether the target body contained in the subgraph is a silk spindle, and also to count the number of subgraphs whose target body is a silk spindle.

[0100] In other words, in step two, the second recognition model can be used to identify whether the target body contained in the sub-graph is a silk spindle, and the number of sub-graphs with the target body being a silk spindle can be counted.

[0101] Continuing with Figure 3(a) as an example, the obtained sub- Figure 1 To the Son Figure 6 Each sub-image in the model is input into the second recognition model, and the target objects contained in each sub-image are identified to obtain the recognition results of each sub-image. That is, target objects 1 to 4 are not silk spindles, target objects 5 and 6 are all silk spindles, and the number of sub-images with target objects being silk spindles is counted as 2.

[0102] Furthermore, in a specific example, the second recognition model includes at least a fourth network layer, a fifth network layer, and a sixth network layer.

[0103] Here, as Figure 5 As shown, the fourth network layer is used to extract features from the sub-image to obtain at least one feature map corresponding to the sub-image. The number of feature maps in the at least one feature map is negatively correlated with the feature block size of the feature map. Further, in one example, during the feature extraction process of the sub-image, the fourth network layer can reduce the size of the feature blocks of the extracted feature map, for example, to a preset value, in order to increase the number of feature maps. In this way, rich feature information is obtained while ensuring the model complexity, thereby improving the accuracy of the model recognition results.

[0104] Furthermore, such as Figure 5 As shown, the fifth network layer is used to predict whether the target body contained in the subgraph is a silk spindle based on at least one feature map corresponding to the subgraph, and obtain a probability value; for example, to obtain a specific value greater than 0 and less than 1.

[0105] Furthermore, such as Figure 5 As shown, the sixth network layer is used to process the probability values ​​to obtain a first value or a second value; then, the number of first values ​​is counted. Here, the first value is used to characterize the target body contained in the subgraph as a silk spindle; the second value is used to characterize the target body contained in the subgraph as not being a silk spindle.

[0106] For example, in one example, the fourth network layer described above can be a Deep Residual Network (ResNet) layer; further, the fifth network layer described above can be a MultiLayer Perceptron (MLP) layer; and even further, the sixth network layer can be a non-linear activation layer, such as a sigmoid function layer. Figure 5As shown, firstly, at least one sub-image in the sub-image is input into the ResNet layer in the second recognition model to obtain at least one feature map corresponding to the sub-image; secondly, at least one feature map corresponding to the sub-image is input into the MLP layer to predict the probability value of the target body in the sub-image being a silk spindle; finally, the predicted probability value is input into the Sigmoid layer to obtain the output result, such as a first value or a second value, and the number of recognition results with the first value is counted.

[0107] In summary, the control method provided by this disclosure has the following advantages compared to the prior art, including:

[0108] First, it improves the packaging efficiency of filament spindles. This disclosed solution can use a target mixing model to quickly detect the number of filament spindles located within a preset area, and can promptly replenish filament spindles that match the currently being transported based on the detection results. This facilitates automated palletizing, thereby reducing labor costs, saving time, and further improving packaging efficiency.

[0109] Secondly, it achieves a completely automated process without human intervention. When the number of spools detected within the preset area is less than the number of spools that the first robotic arm can grasp, this disclosed solution can generate control commands to automatically replenish the corresponding spools, thereby completing the spool packaging task. Compared with existing technologies, this process requires no human intervention and can achieve a complete automated process without human intervention.

[0110] Third, it avoids mixing and stacking spindles from different batches. The spindles automatically replenished by this disclosed solution can be spindles with the same markings as the currently conveyed spindles. Therefore, it effectively avoids the situation where spindles with different markings appear in the preset area where the first robotic arm is located, thereby avoiding stacking spindles from different batches together and further improving packaging efficiency.

[0111] Fourth, it enhances the safety of workshop operations. In automated packaging workshops, this disclosed solution achieves a complete and automated process without human intervention, thus providing technical support for reducing the incidence of safety accidents and laying the foundation for effectively avoiding personnel injuries. This greatly ensures the normal operation of the workshop and improves its overall safety.

[0112] This disclosure also provides a control device, such as... Figure 6 As shown, it includes:

[0113] The detection unit 601 is used to detect a first quantity of first filament spindles located within a preset area when a first filament spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that the quantity of first filament spindles located within a preset area needs to be detected; the conveying channel is a channel for conveying filament spindles in an automated packaging workshop, used to convey the conveyed filament spindles to the preset area; the preset area is a working area that can be reached by a first robotic arm; the first robotic arm is used to grasp multiple filament spindles within the working area it can reach, and transfer the grasped multiple filament spindles to a target position;

[0114] The processing unit 602 is configured to determine the target number of first spindles to be supplemented when the first number of first spindles located in the preset area is less than the total number of spindles that the first robotic arm can grasp, and generate a first control command; the first control command is configured to instruct the second robotic arm to grasp the target number of first spindles from the target spindle cart associated with the first identifier of the first spindle, and place them in the conveying channel or the preset area.

[0115] In a specific example of the scheme disclosed herein, the detection unit is specifically used for one of the following:

[0116] When conveying a first spindle with a first identifier on a conveying channel, and when it is determined that a second spindle with a second identifier needs to be conveyed subsequently, a first number of first spindles located within a preset area is detected; the second identifier is different from the first identifier.

[0117] In the case of conveying a first spindle with a first identifier on the conveying channel, and in the case of determining that the first spindle currently placed on the conveying channel is the last first spindle, the first number of first spindles located in the preset area is detected;

[0118] In the case of conveying a first spindle with a first mark on the conveying channel, and in the case of determining that the total number of first spindles conveyed by the conveying channel does not reach the preset maximum number that the conveying channel can carry, the first number of first spindles located in the preset area is detected.

[0119] In a specific example of the scheme disclosed herein, the detection unit is specifically used for:

[0120] Acquire the target image containing the preset region;

[0121] The target image is input into the target fusion model to obtain the first number of the first spindles located within the preset area; the target fusion model is used to detect the number of spindles located within the preset area in the target image.

[0122] In a specific example of the scheme disclosed herein, the detection unit is specifically used for:

[0123] The target image is input into the first recognition model in the target fusion model to obtain at least one sub-image; wherein, the first recognition model is used to locate the region where each target object is located in the target image, and when it is determined that the region where the target object is located is located within a preset region, the sub-images corresponding to the regions where each target object is located within the preset region are obtained, wherein different sub-images contain different target objects, and the number of sub-images is related to the number of target objects;

[0124] Each subgraph in the at least one subgraph is input into the second recognition model to obtain the first number of first spindles located within a preset area; the second recognition model is used to identify whether the target body contained in the subgraph is a spindle, and is also used to count the number of subgraphs whose target body is a spindle.

[0125] In a specific example of the scheme disclosed herein, the first recognition model includes at least a first network layer, a second network layer, and a third network layer;

[0126] The first network layer is used to extract image features of the target image and, based on the image features of the target image, to select the region where the target object is located to obtain a candidate box region.

[0127] The second network layer is used to filter the candidate box regions obtained by the first network layer to obtain at least one target box region;

[0128] The third network layer is used to determine the first center position of the target box region in the at least one target box region, and select the target box region whose positional relationship with the preset center position of the preset region meets the preset requirements from the at least one target box region, so as to obtain the sub-graph corresponding to the target box region that meets the preset requirements.

[0129] In a specific example of the scheme disclosed herein, the second identification model includes at least a fourth network layer, a fifth network layer, and a sixth network layer;

[0130] The fourth network layer is used to extract features from the subgraph to obtain at least one feature map corresponding to the subgraph. The number of feature maps in the at least one feature map is negatively correlated with the feature block size of the feature map.

[0131] The fifth network layer is used to predict whether the target body contained in the subgraph is a silk spindle based on at least one feature map corresponding to the subgraph, and to obtain a probability value;

[0132] The sixth network layer is used to process the probability values ​​to obtain a first value or a second value, and to count the number of first values; the first value is used to characterize that the target body contained in the subgraph is a silk spindle; the second value is used to characterize that the target body contained in the subgraph is not a silk spindle.

[0133] In a specific example of the scheme disclosed herein, the processing unit is further configured to:

[0134] When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and the conveying channel also conveys other first spindles, a second control command is generated, wherein the second control command is used to instruct the first number of first spindles located within the preset area to be re-detected after the first robotic arm has completed a grasping task.

[0135] or,

[0136] When the number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and there are no first spindles in the conveying channel, a third control command is generated, which is used to instruct the first robotic arm to stop operating.

[0137] For a description of the specific functions and examples of each unit of the apparatus in this disclosure embodiment, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, which will not be repeated here.

[0138] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0139] Figure 7 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 7 As shown, the electronic device includes a memory 710 and a processor 720. The memory 710 stores a computer program that can run on the processor 720. The number of memories 710 and processors 720 can be one or more. The memory 710 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the methods provided in the above-described method embodiments. The electronic device may also include a communication interface 730 for communicating with external devices and performing data exchange and transmission.

[0140] If the memory 710, processor 720, and communication interface 730 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0141] Optionally, in a specific implementation, if the memory 710, processor 720, and communication interface 730 are integrated on a single chip, then the memory 710, processor 720, and communication interface 730 can communicate with each other through an internal interface.

[0142] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0143] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).

[0144] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, Bluetooth, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)). It is worth noting that the computer-readable storage media mentioned in this disclosure can be non-volatile storage media; in other words, it can be non-transient storage media.

[0145] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0146] In the description of the embodiments of this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0147] In the description of the embodiments disclosed herein, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0148] In the description of embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0149] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A control method, comprising: When a first spindle with a first identifier is conveyed on a conveying channel, and when it is determined that the number of first spindles located within a preset area needs to be detected, a first number of first spindles located within the preset area is detected; wherein, the first number is obtained by calling a target blending model to detect the number of spindles located within the preset area in a target image containing the preset area; the conveying channel is a channel for conveying spindles in an automated packaging workshop, used to convey the conveyed spindles to the preset area; the preset area is the working area that the first robotic arm can reach, and the number of spindles that the preset area can accommodate is related to the total number of spindles that the first robotic arm can grasp in one grasping operation; the first robotic arm is used to grasp multiple spindles within the working area it can reach, and transfer the grasped multiple spindles to the target position; If the number of first spindles located within the preset area is less than the total number of spindles that the first robotic arm can grasp in one grasping operation, a target number of first spindles to be supplemented is determined, and a first control command is generated. The first control command is used to instruct the second robotic arm to grasp the target number of first spindles from the target spindle cart associated with the first identifier of the first spindle, and place them in the conveying channel or the preset area, so that the number of first spindles in the preset area is not less than the total number that the first robotic arm can grasp in one grasping operation.

2. The method according to claim 1, wherein, When a first spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that the number of first spindles located within a preset area needs to be detected, detecting the first number of first spindles located within the preset area includes one of the following: When a first spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that a second spindle with a second identifier needs to be conveyed subsequently, a first number of first spindles located within a preset area is detected; the second identifier is different from the first identifier. In the case of conveying a first spindle with a first identifier on the conveying channel, and in the case of determining that the first spindle currently placed on the conveying channel is the last first spindle, the first number of first spindles located in the preset area is detected; In the case of conveying a first spindle with a first mark on the conveying channel, and in the case of determining that the total number of first spindles conveyed by the conveying channel does not reach the preset maximum number that the conveying channel can carry, the first number of first spindles located in the preset area is detected.

3. The method according to claim 1, wherein, The detection of the first number of first spindles located within the preset area includes: Acquire the target image containing the preset region; The target image is input into the target blending model to obtain the first number of the first spindles located within the preset area; the target blending model is used to detect the number of spindles located within the preset area in the target image.

4. The method according to claim 3, wherein, The step of inputting the target image into the target blending model to obtain the first number of the first spindles located within the preset region includes: The target image is input into the first recognition model in the target fusion model to obtain at least one sub-image; wherein, the first recognition model is used to locate the region where each target object is located in the target image, and when it is determined that the region where the target object is located is located within a preset region, the sub-images corresponding to the regions where each target object is located within the preset region are obtained, wherein different sub-images contain different target objects, and the number of sub-images is related to the number of target objects; Each subgraph in the at least one subgraph is input into the second recognition model in the target hybrid model to obtain the first number of first spindles located in the preset area; the second recognition model is used to identify whether the target body contained in the subgraph is a spindle, and is also used to count the number of subgraphs whose target body is a spindle.

5. The method according to claim 4, wherein, The first recognition model includes at least a first network layer, a second network layer, and a third network layer; The first network layer is used to extract image features of the target image and, based on the image features of the target image, to select the region where the target object is located to obtain a candidate box region. The second network layer is used to filter the candidate box regions obtained by the first network layer to obtain at least one target box region; The third network layer is used to determine the first center position of the target box region in the at least one target box region, and select the target box region whose positional relationship with the preset center position of the preset region meets the preset requirements from the at least one target box region, so as to obtain the sub-graph corresponding to the target box region that meets the preset requirements.

6. The method according to claim 4, wherein, The second recognition model includes at least a fourth network layer, a fifth network layer, and a sixth network layer; The fourth network layer is used to extract features from the subgraph to obtain at least one feature map corresponding to the subgraph. The number of feature maps in the at least one feature map is negatively correlated with the feature block size of the feature map. The fifth network layer is used to predict whether the target body contained in the subgraph is a silk spindle based on at least one feature map corresponding to the subgraph, and to obtain a probability value; The sixth network layer is used to process the probability values ​​to obtain a first value or a second value, and to count the number of first values; the first value is used to characterize that the target body contained in the subgraph is a silk spindle; the second value is used to characterize that the target body contained in the subgraph is not a silk spindle.

7. The method according to any one of claims 1-6, further comprising: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and the conveying channel also conveys other first spindles, a second control command is generated, wherein the second control command is used to instruct the first number of first spindles located within the preset area to be re-detected after the first robotic arm has completed a grasping task. or, When the number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and there are no first spindles in the conveying channel, a third control command is generated, which is used to instruct the first robotic arm to stop operating.

8. A control device, comprising: A detection unit is used to detect a first number of first spindles located within a preset area when a first spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that the number of first spindles located within a preset area needs to be detected. The first number is obtained by calling a target blending model to detect the number of spindles located within the preset area in a target image containing the preset area. The conveying channel is a channel for conveying spindles in an automated packaging workshop, used to convey the conveyed spindles to the preset area. The preset area is a working area that can be reached by a first robotic arm, and the number of spindles that the preset area can accommodate is related to the total number of spindles that the first robotic arm can grasp in one grasping operation. The first robotic arm is used to grasp multiple spindles within its working area and transfer the grasped spindles to a target location. The processing unit is configured to determine the target number of first spindles to be supplemented when the first number of first spindles located in the preset area is less than the total number of spindles that the first robotic arm can grasp in one grasping operation, and generate a first control command; the first control command is configured to instruct the second robotic arm to grasp the target number of first spindles from the target yarn cart associated with the first identifier of the first spindle, and place them in the conveying channel or the preset area, so that the number of first spindles in the preset area is not less than the total number that the first robotic arm can grasp in one grasping operation.

9. The apparatus according to claim 8, wherein, The detection unit is specifically used for one of the following: When a first spindle with a first identifier is being conveyed on a conveying channel, and when it is determined that a second spindle with a second identifier needs to be conveyed subsequently, a first number of first spindles located within a preset area is detected; the second identifier is different from the first identifier. In the case of conveying a first spindle with a first identifier on the conveying channel, and in the case of determining that the first spindle currently placed on the conveying channel is the last first spindle, the first number of first spindles located in the preset area is detected; In the case of conveying a first spindle with a first mark on the conveying channel, and in the case of determining that the total number of first spindles conveyed by the conveying channel does not reach the preset maximum number that the conveying channel can carry, the first number of first spindles located in the preset area is detected.

10. The apparatus according to claim 8, wherein, The detection unit is specifically used for: Acquire the target image containing the preset region; The target image is input into the target blending model to obtain the first number of the first spindles located within the preset area; The target fusion model is used to detect the number of spindles located within a preset area in the target image.

11. The apparatus according to claim 10, wherein, The detection unit is specifically used for: The target image is input into the first recognition model in the target fusion model to obtain at least one sub-image; wherein, the first recognition model is used to locate the region where each target object is located in the target image, and when it is determined that the region where the target object is located is located within a preset region, the sub-images corresponding to the regions where each target object is located within the preset region are obtained, wherein different sub-images contain different target objects, and the number of sub-images is related to the number of target objects; Each subgraph in the at least one subgraph is input into the second recognition model in the target hybrid model to obtain the first number of first spindles located in the preset area; the second recognition model is used to identify whether the target body contained in the subgraph is a spindle, and is also used to count the number of subgraphs whose target body is a spindle.

12. The apparatus according to claim 11, wherein, The first recognition model includes at least a first network layer, a second network layer, and a third network layer; The first network layer is used to extract image features of the target image and, based on the image features of the target image, to select the region where the target object is located to obtain a candidate box region. The second network layer is used to filter the candidate box regions obtained by the first network layer to obtain at least one target box region; The third network layer is used to determine the first center position of the target box region in the at least one target box region, and select the target box region whose positional relationship with the preset center position of the preset region meets the preset requirements from the at least one target box region, so as to obtain the sub-graph corresponding to the target box region that meets the preset requirements.

13. The apparatus according to claim 11, wherein, The second recognition model includes at least a fourth network layer, a fifth network layer, and a sixth network layer; The fourth network layer is used to extract features from the subgraph to obtain at least one feature map corresponding to the subgraph. The number of feature maps in the at least one feature map is negatively correlated with the feature block size of the feature map. The fifth network layer is used to predict whether the target body contained in the subgraph is a silk spindle based on at least one feature map corresponding to the subgraph, and to obtain a probability value; The sixth network layer is used to process the probability values ​​to obtain a first value or a second value, and to count the number of first values; the first value is used to characterize that the target body contained in the subgraph is a silk spindle; the second value is used to characterize that the target body contained in the subgraph is not a silk spindle.

14. The apparatus according to any one of claims 8-13, wherein, The processing unit is further configured to: When the first number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and the conveying channel also conveys other first spindles, a second control command is generated, wherein the second control command is used to instruct the first number of first spindles located within the preset area to be re-detected after the first robotic arm has completed a grasping task. or, When the number of first spindles located within the preset area is equal to the total number that the first robotic arm can grasp, and there are no first spindles in the conveying channel, a third control command is generated, which is used to instruct the first robotic arm to stop operating.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images