Control method, apparatus, device, and storage medium

Abstract

The present disclosure provides a control method, device, equipment and storage medium. The method comprises: in the case where it is determined that the feature information of a mechanical gripper meets a first preset requirement, acquiring a first target image collected by a first collection device and a second target image collected by a second collection device; wherein the first collection device is used for image collection of the mechanical gripper; the second collection device is used for image collection of a target spool on a wire car to be grabbed by the mechanical gripper; detecting a first center position of the mechanical gripper in the first target image and detecting a second center position of the target spool in the second target image; in the case where it is determined that a target positional relationship between the first center position and the second center position does not meet a second preset requirement, generating a calibration instruction, the calibration instruction being used for calibration of the center position of the mechanical gripper or for calibration of the center position of the target spool to be grabbed by the mechanical gripper.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a control method, apparatus, device, and storage medium. Background Technology

[0002] In the production workshop of silk spindles, when the robot picks up the silk spindles from the silk trolley, it may experience positioning problems due to mechanical wear and other factors. This can lead to the robot's mechanical gripper damaging the silk spindle or even failing to pick it up successfully. Therefore, there is an urgent need for a method to calibrate the robot's gripping position. Summary of the Invention

[0003] This disclosure provides a control method, apparatus, device, and storage medium to solve or alleviate one or more technical problems in the prior art.

[0004] Firstly, this disclosure provides a control method, including:

[0005] If the feature information of the mechanical gripper meets the first preset requirement, the first target image acquired by the first acquisition device and the second target image acquired by the second acquisition device are obtained; wherein, the first acquisition device is used to acquire images of the mechanical gripper; the second acquisition device is used to acquire images of the target spindle on the wire machine that the mechanical gripper needs to grasp.

[0006] Detect the first center position of the mechanical gripper in the first target image, and detect the second center position of the target spindle in the second target image;

[0007] If the target position relationship between the first center position and the second center position does not meet the second preset requirement, a calibration command is generated. The calibration command is used to calibrate the center position of the mechanical gripper, or to calibrate the center position of the target spindle that the mechanical gripper needs to grasp.

[0008] Secondly, this disclosure provides a control device, comprising:

[0009] The acquisition unit is used to acquire a first target image acquired by a first acquisition device and a second target image acquired by a second acquisition device when the feature information of the mechanical gripper meets a first preset requirement; wherein, the first acquisition device is used to acquire images of the mechanical gripper; and the second acquisition device is used to acquire images of the target spindle on the wire machine that the mechanical gripper needs to grasp.

[0010] The detection unit is used to detect the first center position of the mechanical gripper in the first target image and the second center position of the target spindle in the second target image;

[0011] The processing unit is configured to generate a calibration command when the target position relationship between the first center position and the second center position does not meet a second preset requirement. The calibration command is used to calibrate the center position of the mechanical gripper or to calibrate the center position of the target spindle to be gripped by the mechanical gripper.

[0012] Thirdly, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] The memory is communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0016] Fourthly, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of the present disclosure.

[0017] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of the present disclosure.

[0018] In this way, the present invention can detect the center position of the mechanical gripper and the center position of the target spindle based on the acquired first target image and second target image, so as to determine the target position relationship between the two. Then, calibration can be performed according to the target position relationship. In this way, the problem of inaccurate positioning and gripping caused by mechanical wear can be quickly dealt with, effectively improving the gripping accuracy of the mechanical gripper and providing technical support for the automatic calibration of the gripping position of the mechanical gripper.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments provided according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0021] Figure 1This is an illustrative flow diagram of a control method according to an embodiment of this application. Figure 1 ;

[0022] Figures 2(a) and 2(b) are schematic diagrams of a scene where a mechanical gripper grasps a wire spindle according to an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of the structure of a target detection model according to an embodiment of this application;

[0024] Figure 4 This is a schematic flowchart of a control method according to an embodiment of this application;

[0025] Figure 5 This is an illustrative flow diagram of a control method according to an embodiment of this application. Figure 3 ;

[0026] Figure 6 This is an illustrative flow diagram of a control method according to an embodiment of this application. Figure 4 ;

[0027] Figure 7 This is a schematic diagram of the structure of a control device according to an embodiment of this application;

[0028] Figure 8 This is a block diagram of an electronic device used to implement the control method of the embodiments of this disclosure. Detailed Implementation

[0029] The present disclosure will now be described in further detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0030] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0031] In the production workshop of silk spindles, when the robot picks up the silk spindles from the silk cart, it may experience positioning problems due to factors such as mechanical wear. This can lead to the robot's mechanical grippers damaging the silk spindles during the gripping process, or even failing to pick up the silk spindles successfully.

[0032] Based on this, the present disclosure proposes a control method for calibrating the robot's grasping position.

[0033] Specifically, Figure 1 This is an illustrative flow diagram of a control method according to an embodiment of this application. Figure 1This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0034] Furthermore, the method includes at least a portion of the following: For example... Figure 1 As shown, it includes:

[0035] Step S101: If the feature information of the mechanical gripper meets the first preset requirement, acquire the first target image acquired by the first acquisition device and the second target image acquired by the second acquisition device.

[0036] Here, the first acquisition device is used to acquire images of the mechanical gripper; the second acquisition device is used to acquire images of the target spindle on the wire carriage that the mechanical gripper needs to grasp.

[0037] In one example, the first acquisition device can be set at any position that can acquire the complete mechanical gripper, such as the first acquisition device being set on the yarn train; the second acquisition device can be set at any position that can acquire the complete target yarn on the yarn train, such as the second acquisition device being set on the robotic arm where the mechanical gripper is located, and this disclosure does not limit this.

[0038] Furthermore, the first acquisition device (or the second acquisition device) may specifically include a camera. For example, the first target image is obtained by capturing an image of the mechanical gripper using the camera; for instance, the first target image is obtained by taking a picture of the mechanical gripper using the camera, or by capturing video of the mechanical gripper for a preset duration to obtain multiple consecutive video frames, and then selecting one image from the consecutive video frames as the first target image. Here, the second acquisition device is similar to the first acquisition device, and will not be described in detail here.

[0039] It should be noted that the target spindle is the spindle that the mechanical gripper currently needs to pick up from the spinning wheel. In one example, as shown in Figure 2(a), a mechanical gripper is fixed to the end of the robotic arm. The mechanical gripper extends towards the spinning wheel in a first direction. At this time, when the mechanical gripper moves to a specific position and is about to pick up the target spindle, the first acquisition device set on the robotic arm can acquire the target spindle that the mechanical gripper is about to pick up, as shown in Figure 2(b), thus obtaining a first target image containing the complete target spindle. It can be understood that the position of the first acquisition device set on the robotic arm can be adjusted to control the detailed information of the target body contained in the acquired first target image. For example, by adjusting the position of the first acquisition device set on the robotic arm, the acquired first target image can contain a complete spindle, and this complete spindle is the target spindle. In this way, the interference of other spindles around the target spindle on the identification of the center position of the target spindle is effectively avoided, thereby laying the foundation for effectively improving the calibration accuracy.

[0040] Furthermore, the feature information of the mechanical gripper may specifically include, but is not limited to, at least one of the following: the running time of the mechanical gripper, the number of times the mechanical gripper grips the spindles on the yarn train, etc.

[0041] In one example, the running time of the mechanical gripper is determined, and when the running time of the mechanical gripper reaches a preset running time, a first target image acquired by a first acquisition device and a second target image acquired by a second acquisition device are obtained.

[0042] Alternatively, in another example, the number of times the mechanical gripper grasps the spindle on the yarn carriage is counted, and when the number of grasps reaches a preset number, a first target image acquired by a first acquisition device and a second target image acquired by a second acquisition device are obtained.

[0043] Step S102: Detect the first center position of the mechanical gripper in the first target image, and detect the second center position of the target spindle in the second target image.

[0044] In a specific example, the first center position and the second center position can be obtained in the following manner; specifically, the above-described detection of the first center position of the mechanical gripper in the first target image and the detection of the second center position of the target spindle in the second target image (i.e., the above-described step S102) specifically includes:

[0045] Step S102-1: Input the first target image into the target detection model to obtain the first center position of the mechanical gripper in the first target image.

[0046] Step S102-2: Input the second target image into the target detection model to obtain the second center position of the target spindle in the second target image.

[0047] Furthermore, in a specific example, the target detection model in step S102-1 (or step S102-2) above includes at least a first network layer, a second network layer, and a third network layer;

[0048] Here, the first network layer includes at least a first sub-network layer, a second sub-network layer, and a third sub-network layer. The first sub-network layer is used to perform a first convolutional processing on the input image to obtain a first convolutional feature map. The second sub-network layer is used to perform linear processing on the first convolutional feature map to obtain a linear feature map, and then connects the linear feature map with the first convolutional feature map to obtain a connection feature map. The third sub-network layer is used to perform a second convolutional processing on the connection feature map to obtain a second convolutional feature map. In this way, the richness of the extracted features is effectively improved, thereby improving the accuracy of subsequent recognition of mechanical grippers or target spindles.

[0049] For example, in one example, the first network layer above is a GhostNet layer. Further, the GhostNet layer includes a first sub-network layer that is a regular convolutional (Conv) network layer, a second sub-network layer that is a ghost convolutional (GhostConv) network layer, and a third sub-network layer that is a hybrid dilated convolutional (HDC) network layer. Figure 3 As shown, firstly, the image (e.g., the first target image and the second target image) is input into a regular convolutional network layer to obtain the regular convolutional feature map of the image; secondly, the regular convolutional feature map is input into a phantom convolutional network layer. For example, after linear processing by the phantom convolutional network layer, a linear feature map is obtained, and the linear feature map is connected with the regular convolutional feature map to obtain a connected feature map; finally, the connected feature map is input into a hybrid dilated convolutional network layer to obtain a hybrid dilated convolutional feature map.

[0050] Furthermore, the second network layer is used to perform first convolution processing on the connection feature map and the second convolution feature map respectively, and to perform feature fusion processing on the processed results to obtain a fused feature map. For example, the second network layer is a Feature Pyramid Network (FPN) layer, such as... Figure 3 As shown, the connection feature map and the hybrid dilated convolution feature map obtained from the phantom network layer are input into the FPN layer for feature fusion processing to obtain the fused feature map.

[0051] Furthermore, the third network layer includes a fourth sub-network layer and a fifth sub-network layer. The fourth sub-network layer is used to identify the mechanical gripper or target spindle in the image based on the fused feature map. The fifth sub-network layer is used to obtain the center position of the mechanical gripper or the center position of the target spindle in the image based on the fused feature map. For example, the fifth sub-network layer is used to obtain the center position of the mechanical gripper or the center position of the target spindle in the image based on the fused feature map and the output of the fourth sub-network layer.

[0052] For example, the fourth sub-network layer is a classification sub-network layer, and the fifth sub-network layer is a regression sub-network layer, such as... Figure 3 As shown, the output of the FPN layer, such as the fused feature map, is input into the classification sub-network layer to identify the mechanical gripper in the image, for example, by selecting the mechanical gripper in the image; and the fused feature map and the selected mechanical gripper in the image (for example, a portion of the image containing the mechanical gripper) are input into the regression sub-network layer to obtain the center position of the mechanical gripper in the image; or, the fused feature map is input into the classification sub-network layer to identify the target spindle in the image, for example, by selecting the target spindle in the image; and the fused feature map and the selected target spindle in the image (for example, a portion of the image containing the target spindle) are input into the regression sub-network layer to obtain the center position of the target spindle in the image.

[0053] In this way, the present invention can use the target detection model to obtain the center position of the mechanical gripper and the center position of the target spindle, thereby facilitating the determination of the target position relationship between the two, thus laying the foundation for subsequent calibration of the gripping position of the mechanical gripper.

[0054] Step S103: If the target position relationship between the first center position and the second center position does not meet the second preset requirement, a calibration command is generated. The calibration command is used to calibrate the center position of the mechanical gripper or to calibrate the center position of the target spindle that the mechanical gripper needs to grasp.

[0055] In this way, the present invention can detect the center position of the mechanical gripper and the center position of the target spindle based on the acquired first target image and second target image, so as to determine the target position relationship between the two. Then, calibration can be performed according to the target position relationship. In this way, the problem of inaccurate positioning and gripping caused by mechanical wear can be quickly dealt with, effectively improving the gripping accuracy of the mechanical gripper and providing technical support for the automatic calibration of the gripping position of the mechanical gripper.

[0056] Furthermore, in the production workshop of the silk spindle, especially in the automatic packaging process of the silk spindle, the present invention enables the mechanical gripper to accurately position and grasp the silk spindle, laying the foundation for avoiding damage during the subsequent grasping of the silk spindle, and also laying the foundation for ensuring the normal operation of the silk spindle packaging business and greatly improving packaging efficiency.

[0057] In a specific example, after the calibration command is executed, the calibration result can be verified. This ensures that the gripping position of the mechanical gripper is successfully calibrated, laying the foundation for the mechanical gripper to accurately grip the wire spindle in the future. Specifically, Figure 4 This is a schematic flowchart of a control method according to an embodiment of this application. This method can optionally be applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It is understood that the above... Figure 1 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0058] Furthermore, the method includes at least a portion of the following. Specifically, such as... Figure 4 As shown, it includes:

[0059] Step S401: Determine whether the feature information of the mechanical gripper meets the first preset requirement. If yes, proceed to step S402; otherwise, proceed to step S406.

[0060] Step S402: Acquire the first target image acquired by the first acquisition device and the second target image acquired by the second acquisition device. Proceed to step S403.

[0061] Step S403: Detect the first center position of the mechanical gripper in the first target image, and detect the second center position of the target spindle in the second target image. Proceed to step S404.

[0062] Step S404: Determine whether the target position relationship between the first center position and the second center position meets the second preset requirement. If yes, proceed to step S406; otherwise, proceed to step S405.

[0063] Step S405: If the target position relationship between the first center position and the second center position does not meet the second preset requirement, a calibration command is generated. The calibration command is used to calibrate the center position of the mechanical gripper, or to calibrate the center position of the target spindle that the mechanical gripper needs to grasp. Then proceed to step S407.

[0064] Step S406: Continue executing the crawling task.

[0065] Step S407: After the calibration command is executed, a first control command is generated and the process returns to step S402 to reacquire a new first target image and a new second target image.

[0066] Here, the first control command is used to instruct the first acquisition device to acquire a new first target image, and to instruct the second acquisition device to acquire a new second target image.

[0067] In other words, in the current calibration process, after the generated calibration command is executed, a new first target image and a new second target image can be acquired. Based on the new first target image and the new second target image, the first center position of the mechanical gripper in the new first target image and the second center position of the target spindle in the new second target image are detected, thereby obtaining a new target position relationship. At this time, it is determined whether the obtained new target position relationship meets the second preset requirement. If it does, that is, if the second preset requirement is met, the calibration is successful and the current calibration process ends; otherwise, that is, if the second preset requirement is not met, a new calibration command is generated until the new target position relationship meets the second preset requirement.

[0068] In a specific example, Figure 5 This is an illustrative flow diagram of a control method according to an embodiment of this application. Figure 3 This method can optionally be applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It is understood that the above... Figure 1 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0069] Furthermore, the method includes at least a portion of the following. Specifically, such as... Figure 5 As shown, it includes:

[0070] Step S501: If the feature information of the mechanical gripper meets the first preset requirement, acquire the first target image acquired by the first acquisition device and the second target image acquired by the second acquisition device.

[0071] Here, the first acquisition device is used to acquire images of the mechanical gripper; the second acquisition device is used to acquire images of the target spindle on the wire carriage that the mechanical gripper needs to grasp.

[0072] Step S502: Detect the first center position of the mechanical gripper in the first target image, and detect the second center position of the target spindle in the second target image.

[0073] Here, the detection of the first center position and the second center position can be referred to the above example, and will not be repeated here.

[0074] Step S503: Based on the first center position and the second center position, estimate the target distance; the target distance represents the relative distance between the center positions of the mechanical gripper and the target spindle in the same target coordinate system.

[0075] Here, the first center position is the position information in the first coordinate system corresponding to the first target image; the second center position is the position information in the second coordinate system corresponding to the second target image; the target coordinate system is one of the first coordinate system, the second coordinate system, and the world coordinate system.

[0076] In other words, after step S502, the target distance used to describe the target position relationship can be estimated based on the obtained first center position and second center position, so as to facilitate the determination of whether calibration is needed based on the target distance.

[0077] Step S504: If the target distance is determined to be greater than a preset threshold, a calibration command is generated.

[0078] Furthermore, in a specific example, the target distance can be obtained in the following manner; specifically, the target distance is estimated based on the first center position and the second center position (i.e., step S503 described above), which specifically includes:

[0079] Step S503-1: Based on the first center position, estimate the first center coordinates of the mechanical gripper in the target coordinate system.

[0080] For example, in one instance, when the target coordinate system is a first coordinate system, the first center coordinates can be obtained directly based on the first center position. For instance, when the first center position is a coordinate point, the coordinate point representing the first center position can be directly used as the first center coordinates in the first coordinate system; or, when the first center position is a central region (for example, a central region containing multiple coordinate points), a center coordinate can be obtained based on the multiple coordinate points in the central region, and the obtained center coordinate can be directly used as the first center coordinates in the first coordinate system.

[0081] Alternatively, in another example, when the target coordinate system is the second coordinate system, the first center coordinates of the target spindle in the second coordinate system can be estimated based on the first center position and the coordinate transformation relationship between the first and second coordinate systems. For example, if the first center position is a coordinate point, the coordinate point representing the first center position can be transformed to the second coordinate system based on the coordinate transformation relationship between the first and second coordinate systems to obtain the first center coordinates; or, if the first center position is a central region (e.g., a central region containing multiple coordinate points), the center coordinates corresponding to the first center position can be obtained based on the multiple coordinate points in the central region, and the obtained center coordinates corresponding to the first center position can be transformed to the second coordinate system to obtain the first center coordinates; or, if the first center position is a central region (e.g., a central region containing multiple coordinate points), the multiple coordinate points in the central region can be transformed to the second coordinate system based on the coordinate transformation relationship between the first and second coordinate systems to obtain multiple coordinate points in the second coordinate system, and then the first center coordinates can be obtained based on the multiple coordinate points in the second coordinate system.

[0082] Alternatively, in another example, when the target coordinate system is the world coordinate system, the first center coordinates of the target spindle in the world coordinate system can be estimated based on the first center position and the coordinate transformation relationship between the first coordinate system and the world coordinate system. For example, if the first center position is a coordinate point, the coordinate point representing the first center position can be transformed to the world coordinate system based on the coordinate transformation relationship between the first coordinate system and the world coordinate system to obtain the first center coordinates; or, if the first center position is a central region (e.g., a central region containing multiple coordinate points), the center coordinates corresponding to the first center position can be obtained based on the multiple coordinate points in the central region, and the obtained center coordinates corresponding to the first center position can be transformed to the world coordinate system to obtain the first center coordinates; or, if the first center position is a central region (e.g., a central region containing multiple coordinate points), the multiple coordinate points in the central region can be transformed to the world coordinate system based on the coordinate transformation relationship between the first coordinate system and the world coordinate system to obtain multiple coordinate points in the world coordinate system, and then the first center coordinates can be obtained based on the multiple coordinate points in the world coordinate system.

[0083] Step S503-2: Based on the second center position, estimate the second center coordinates of the target spindle in the target coordinate system.

[0084] For example, in one instance, when the target coordinate system is the first coordinate system, the second center coordinates of the target spindle in the first coordinate system can be estimated based on the second center position and the coordinate transformation relationship between the first and second coordinate systems. For instance, if the second center position is a single coordinate point, the coordinate point representing the second center position can be transformed to the first coordinate system based on the coordinate transformation relationship between the first and second coordinate systems to obtain the second center coordinates. Alternatively, if the second center position is a central region (e.g., a central region containing multiple coordinate points), the center coordinates corresponding to the second center position can be obtained based on the multiple coordinate points within the central region, and then the obtained center coordinates corresponding to the second center position can be transformed to the first coordinate system to obtain the second center coordinates. Or, if the second center position is a central region (e.g., a central region containing multiple coordinate points), the multiple coordinate points within the central region can be transformed to the first coordinate system based on the coordinate transformation relationship between the first and second coordinate systems to obtain multiple coordinate points in the first coordinate system, and then the second center coordinates can be obtained based on these multiple coordinate points in the first coordinate system.

[0085] Alternatively, in another example, when the target coordinate system is the second coordinate system, the second center coordinates can be obtained directly based on the second center position. For example, when the second center position is a coordinate point, the coordinate point representing the second center position can be directly used as the second center coordinates in the second coordinate system; or, when the second center position is a central region (for example, a central region containing multiple coordinate points), a center coordinate can be obtained based on multiple coordinate points within the central region, and the obtained center coordinate can be directly used as the second center coordinates in the second coordinate system.

[0086] Alternatively, in another example, when the target coordinate system is the world coordinate system, the second center coordinates of the target spindle in the world coordinate system can be estimated based on the second center position and the coordinate transformation relationship between the second coordinate system and the world coordinate system. For example, if the second center position is a single coordinate point, the coordinate point representing the second center position can be transformed to the world coordinate system based on the coordinate transformation relationship between the second coordinate system and the world coordinate system to obtain the second center coordinates; or, if the second center position is a central region (e.g., a central region containing multiple coordinate points), the center coordinates corresponding to the second center position can be obtained based on the multiple coordinate points within the central region, and the obtained center coordinates corresponding to the second center position can be transformed to the world coordinate system to obtain the second center coordinates; or, if the second center position is a central region (e.g., a central region containing multiple coordinate points), the multiple coordinate points within the central region can be transformed to the world coordinate system based on the coordinate transformation relationship between the second coordinate system and the world coordinate system to obtain multiple coordinate points in the world coordinate system, and then the second center coordinates can be obtained based on the multiple coordinate points in the world coordinate system.

[0087] Step S503-3: Based on the first center coordinates of the mechanical gripper in the target coordinate system and the second center coordinates of the target spindle in the target coordinate system, the target distance is obtained.

[0088] In this way, the present invention can estimate the target distance describing the target position relationship between the mechanical gripper and the target spindle based on the detected center position of the mechanical gripper and the center position of the target spindle. Then, the gripping position of the mechanical gripper can be calibrated according to the target distance. In this way, the problem of inaccurate positioning and gripping caused by mechanical wear can be quickly dealt with, effectively improving the gripping accuracy of the mechanical gripper. This lays the foundation for improving the gripping efficiency of the mechanical gripper on the spindle, and also lays the foundation for avoiding damage to the spindle during gripping and avoiding failure to grip the spindle.

[0089] Furthermore, in a specific example, the calibration command can also be generated in the following manner; specifically, the generation of the calibration command when the target distance is determined to be greater than a preset threshold (i.e., step S504 described above) specifically includes:

[0090] Method 1: When it is determined that the target distance is greater than the first-level critical value and the target distance is less than or equal to the second-level critical value, a calibration command is generated. In this case, the calibration command is used to instruct the control parameters of the mechanical gripper to be adjusted based on the target distance.

[0091] or,

[0092] Method 2: When it is determined that the target distance is greater than the first-level critical value and the target distance is greater than the second-level critical value, a calibration command is generated. In this case, the calibration command is used to instruct the position of the yarn carriage where the target yarn spindle is located to be adjusted based on the target distance.

[0093] Here, the first-level critical value is less than the second-level critical value, and both the first-level critical value and the second-level critical value are empirical values ​​that can be set according to actual needs. This disclosure does not limit this.

[0094] Specifically, Figure 6 This is an illustrative flow diagram of a control method according to an embodiment of this application. Figure 4 This method can optionally be applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It is understood that the above... Figure 1 and Figure 5 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0095] Furthermore, the method includes at least a portion of the following. Specifically, such as... Figure 6 As shown, it includes:

[0096] Step S601: If the feature information of the mechanical gripper meets the first preset requirement, acquire the first target image acquired by the first acquisition device and the second target image acquired by the second acquisition device.

[0097] Here, the first acquisition device is used to acquire images of the mechanical gripper; the second acquisition device is used to acquire images of the target spindle on the wire carriage that the mechanical gripper needs to grasp.

[0098] Step S602: Detect the first center position of the mechanical gripper in the first target image, and detect the second center position of the target spindle in the second target image.

[0099] Step S603: Based on the first center position and the second center position, estimate the target distance; the target distance represents the relative distance between the center positions of the mechanical gripper and the target spindle in the same target coordinate system.

[0100] Here, the first center position is the position information in the first coordinate system corresponding to the first target image; the second center position is the position information in the second coordinate system corresponding to the second target image; the target coordinate system is one of the first coordinate system, the second coordinate system, and the world coordinate system.

[0101] Step S604: Determine whether the target distance is greater than the first-level critical value. If yes, proceed to step S606; otherwise, proceed to step S605.

[0102] Step S605: Re-determine the feature information of the mechanical gripper and return to step S601.

[0103] For example, if it is determined that the running time of the mechanical gripper reaches a preset time (e.g., 1 hour) and the target distance is less than the first-level threshold, the running time of the mechanical gripper is initialized, the running time of the mechanical gripper is recounted, and it is determined whether the recounted running time reaches 1 hour; or, for example, if it is determined that the number of gripping operations of the mechanical gripper reaches a preset number (e.g., 80 times) and the target distance is less than the first-level threshold, the number of gripping operations of the mechanical gripper is initialized, the number of gripping operations of the mechanical gripper is recounted, and it is determined whether the recounted number of gripping operations reaches 80 times.

[0104] Step S606: Determine whether the target distance is less than or equal to the second-level critical value. If yes, proceed to step S607; otherwise, proceed to step S608.

[0105] Step S607: If it is determined that the target distance is greater than the first-level critical value and the target distance is less than or equal to the second-level critical value, a calibration instruction is generated, which is used to instruct the control parameters of the mechanical gripper to be adjusted based on the target distance.

[0106] Step S608: If it is determined that the target distance is greater than the first-level critical value and the target distance is greater than the second-level critical value, a calibration instruction is generated. The calibration instruction is used to instruct the position of the yarn carriage where the target yarn spindle is located to be adjusted based on the target distance.

[0107] Understandably, given the first and second center positions, the target distance in the same target coordinate system is estimated. It is then determined whether the obtained target distance is greater than the first-level critical value. If so, it is determined whether the obtained target distance is less than or equal to the second-level critical value. If so, it is generated as a calibration command to instruct the adjustment of the control parameters of the mechanical gripper based on the target distance, thereby adjusting the center position of the mechanical gripper. In this example, when the obtained target distance is greater than the first-level critical value and less than or equal to the second-level critical value, the deviation between the mechanical gripper and the center position of the target spindle is considered small. In this case, the control parameters of the mechanical gripper can be adjusted by fine-tuning to adjust the center position of the mechanical gripper, thereby reducing the distance between the mechanical gripper and the center position of the target spindle.

[0108] Furthermore, if the obtained target distance is greater than the first-level critical value, and also greater than the second-level critical value, a calibration command is generated to instruct the adjustment of the position of the yarn carriage containing the target spindle based on the target distance, thereby adjusting the center position of the target spindle. It can be understood that in this example, if the obtained target distance is greater than the second-level critical value, it can be considered that the deviation between the mechanical gripper and the center position of the target spindle is large. In this case, the gripping position of the mechanical gripper can be calibrated by adjusting the position of the yarn carriage, thereby reducing the distance between the mechanical gripper and the center position of the target spindle. This allows for rapid calibration, improving calibration efficiency and enhancing the accuracy of the mechanical gripper's gripping.

[0109] It should be noted that in real-world scenarios, the control parameters of the mechanical gripper can be adjusted by referring to the target distance, or the position of the wire carriage can be adjusted. For example, if the target distance is obtained in the world coordinate system, the control parameters of the mechanical gripper or the position of the wire carriage can be adjusted based on the real relative distance.

[0110] In summary, the control method provided by this disclosure has the following advantages over the prior art, specifically including:

[0111] First, it is more automated. Compared with existing technologies, the solution disclosed in this paper can quickly address the problem of inaccurate positioning and gripping caused by mechanical wear. Moreover, it provides technical support for the automated calibration of the gripping position of the mechanical grippers in the automatic packaging process of the filament. This process does not require human intervention, thus further improving the automation level of the automatic packaging process and saving a lot of labor and time costs.

[0112] Secondly, the gripping accuracy is improved. This disclosed solution can provide different calibration methods based on the obtained target distance. For example, when the deviation between the mechanical gripper and the center position of the target spindle is considered to be small, the control parameters of the mechanical gripper are adjusted; or, when the deviation between the mechanical gripper and the center position of the target spindle is considered to be large, the position of the yarn carriage is adjusted. In this way, the gripping accuracy of the mechanical gripper is effectively improved.

[0113] Third, automated packaging is more efficient. This disclosed solution effectively solves the problem of inaccurate positioning and gripping by mechanical grippers, laying the foundation for preventing damage to or failure to grip the spindles due to inaccurate gripping. It also lays the foundation for ensuring the normal operation of the spindle packaging business and significantly improving packaging efficiency.

[0114] This disclosure also provides a control device, such as... Figure 7 As shown, it includes:

[0115] The acquisition unit 701 is used to acquire a first target image acquired by a first acquisition device and a second target image acquired by a second acquisition device when the feature information of the mechanical gripper meets a first preset requirement; wherein, the first acquisition device is used to acquire images of the mechanical gripper; and the second acquisition device is used to acquire images of the target spindle on the wire machine that the mechanical gripper needs to grasp.

[0116] The detection unit 702 is used to detect the first center position of the mechanical gripper in the first target image and the second center position of the target spindle in the second target image;

[0117] The processing unit 703 is configured to generate a calibration command when it is determined that the target position relationship between the first center position and the second center position does not meet the second preset requirement. The calibration command is used to calibrate the center position of the mechanical gripper or to calibrate the center position of the target spindle to be gripped by the mechanical gripper.

[0118] In a specific example of the scheme disclosed herein, the detection unit is specifically used for:

[0119] The first target image is input into the target detection model to obtain the first center position of the mechanical gripper in the first target image;

[0120] The second target image is input into the target detection model to obtain the second center position of the target spindle in the second target image.

[0121] In a specific example of the scheme disclosed herein, the target detection model includes at least a first network layer, a second network layer, and a third network layer;

[0122] The first network layer includes at least a first sub-network layer, a second sub-network layer, and a third sub-network layer. The first sub-network layer performs a first convolutional process on the input image to obtain a first convolutional feature map. The second sub-network layer performs a linear process on the first convolutional feature map to obtain a linear feature map, and then connects the linear feature map with the first convolutional feature map to obtain a connection feature map. The third sub-network layer performs a second convolutional process on the connection feature map to obtain a second convolutional feature map.

[0123] The second network layer is used to perform first convolution processing on the connection feature map and the second convolution feature map respectively, and perform feature fusion processing on the processed results to obtain a fused feature map;

[0124] The third network layer includes a fourth sub-network layer and a fifth sub-network layer. The fourth sub-network layer is used to identify the mechanical gripper or target spindle in the image based on the fused feature map. The fifth sub-network layer is used to obtain the center position of the mechanical gripper or the center position of the target spindle in the image based on the fused feature map.

[0125] In a specific example of the scheme disclosed herein, the processing unit is further configured to:

[0126] Based on the first center position and the second center position, the target distance is estimated, whereby the target distance represents the relative distance between the center positions of the mechanical gripper and the target spindle in the same target coordinate system; the first center position is the position information in the first coordinate system corresponding to the first target image; the second center position is the position information in the second coordinate system corresponding to the second target image; the target coordinate system is one of the first coordinate system, the second coordinate system, and the world coordinate system.

[0127] If the target distance is determined to be greater than a preset threshold, a calibration command is generated.

[0128] In a specific example of the disclosed solution, the processing unit is specifically used for:

[0129] Based on the first center position, the first center coordinates of the mechanical gripper in the target coordinate system are estimated;

[0130] Based on the second center position, the second center coordinates of the target spindle in the target coordinate system are estimated;

[0131] The target distance is obtained based on the first center coordinates of the mechanical gripper in the target coordinate system and the second center coordinates of the target spindle in the target coordinate system.

[0132] In a specific example of the disclosed solution, the processing unit is specifically used for:

[0133] If it is determined that the target distance is greater than the first-level critical value and the target distance is less than or equal to the second-level critical value, a calibration command is generated, which is used to instruct the control parameters of the mechanical gripper to be adjusted based on the target distance;

[0134] or,

[0135] If it is determined that the target distance is greater than the first-level critical value and the target distance is greater than the second-level critical value, a calibration command is generated, which is used to instruct the position of the spinning wheel where the target spindle is located to be adjusted based on the target distance.

[0136] In a specific example of the scheme disclosed herein,

[0137] The processing unit is further configured to generate a first control command after the calibration command is executed; the first control command is configured to instruct the first acquisition device to acquire a new first target image and to instruct the second acquisition device to acquire a new second target image.

[0138] The acquisition unit is further configured to acquire a new first target image acquired by the first acquisition device, and to acquire a new second target image acquired by the second acquisition device;

[0139] The detection unit is also used to detect the first center position of the mechanical gripper in the new first target image and the second center position of the target spindle in the new second target image;

[0140] The processing unit is further configured to determine whether the target position relationship between the first center position in the new first target image and the second center position in the new second target image meets the second preset requirement.

[0141] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0142] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0143] Figure 8 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 8 As shown, the electronic device includes a memory 810 and a processor 820. The memory 810 stores a computer program that can run on the processor 820. The number of memories 810 and processors 820 can be one or more. The memory 810 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the methods provided in the above-described method embodiments. The electronic device may also include a communication interface 830 for communicating with external devices and performing data exchange and transmission.

[0144] If the memory 810, processor 820, and communication interface 830 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0145] Optionally, in a specific implementation, if the memory 810, processor 820, and communication interface 830 are integrated on a single chip, then the memory 810, processor 820, and communication interface 830 can communicate with each other through an internal interface.

[0146] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0147] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).

[0148] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, Bluetooth, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)). It is worth noting that the computer-readable storage media mentioned in this disclosure may be non-volatile storage media; in other words, they may be non-transient storage media.

[0149] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0150] In the description of the embodiments of this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0151] In the description of the embodiments disclosed herein, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0152] In the description of embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0153] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A control method, comprising: If the characteristic information of the mechanical gripper meets the first preset requirement, a first target image acquired by the first acquisition device and a second target image acquired by the second acquisition device are obtained; wherein, the characteristic information of the mechanical gripper includes at least one of the following: the running time of the mechanical gripper and the number of times the mechanical gripper grips the spindle on the yarn train; the first acquisition device is set on the yarn train for image acquisition of the mechanical gripper; the second acquisition device is set on the robotic arm where the mechanical gripper is located for image acquisition of the target spindle on the yarn train to be gripped by the mechanical gripper; Detect the first center position of the mechanical gripper in the first target image, and detect the second center position of the target spindle in the second target image; If the target position relationship between the first center position and the second center position does not meet the second preset requirement, a calibration command is generated. The calibration command is used to calibrate the center position of the mechanical gripper, or to calibrate the center position of the target spindle that the mechanical gripper needs to grasp. Wherein, generating a calibration command when the target positional relationship between the first center position and the second center position does not meet the second preset requirement includes: If the target distance is determined to be greater than a preset threshold, a calibration command is generated, wherein the target distance represents the relative distance between the center positions of the mechanical gripper and the target spindle in the same target coordinate system; The step of generating a calibration command when the target distance is determined to be greater than a preset threshold includes: If the target distance is determined to be greater than a first-level critical value and a second-level critical value, a calibration command is generated; wherein the calibration command is used to instruct the adjustment of the position of the yarn carriage where the target yarn spindle is located based on the target distance, and the first-level critical value is less than the second-level critical value.

2. The method according to claim 1, wherein, The step of detecting the first center position of the mechanical gripper in the first target image and detecting the second center position of the target spindle in the second target image includes: The first target image is input into the target detection model to obtain the first center position of the mechanical gripper in the first target image; The second target image is input into the target detection model to obtain the second center position of the target spindle in the second target image.

3. The method according to claim 2, wherein, The target detection model includes at least a first network layer, a second network layer, and a third network layer; The first network layer includes at least a first sub-network layer, a second sub-network layer, and a third sub-network layer. The first sub-network layer performs a first convolutional process on the input image to obtain a first convolutional feature map. The second sub-network layer performs a linear process on the first convolutional feature map to obtain a linear feature map, and then connects the linear feature map with the first convolutional feature map to obtain a connection feature map. The third sub-network layer performs a second convolutional process on the connection feature map to obtain a second convolutional feature map. The second network layer is used to perform first convolution processing on the connection feature map and the second convolution feature map respectively, and perform feature fusion processing on the processed results to obtain a fused feature map; The third network layer includes a fourth sub-network layer and a fifth sub-network layer. The fourth sub-network layer is used to identify the mechanical gripper or target spindle in the image based on the fused feature map. The fifth sub-network layer is used to obtain the center position of the mechanical gripper or the center position of the target spindle in the image based on the fused feature map.

4. The method according to any one of claims 1-3, wherein, Before determining that the target positional relationship between the first center position and the second center position does not meet the second preset requirement, the process also includes: The target distance is estimated based on the first center position and the second center position; the first center position is the position information in the first coordinate system corresponding to the first target image; the second center position is the position information in the second coordinate system corresponding to the second target image; the target coordinate system is one of the first coordinate system, the second coordinate system, and the world coordinate system.

5. The method according to claim 4, wherein, The step of estimating the target distance based on the first center position and the second center position includes: Based on the first center position, the first center coordinates of the mechanical gripper in the target coordinate system are estimated; Based on the second center position, the second center coordinates of the target spindle in the target coordinate system are estimated; The target distance is obtained based on the first center coordinates of the mechanical gripper in the target coordinate system and the second center coordinates of the target spindle in the target coordinate system.

6. The method according to claim 1, further comprising: If the target distance is determined to be greater than a first-level critical value and less than or equal to a second-level critical value, a calibration command is generated, which is used to instruct the control parameters of the mechanical gripper to be adjusted based on the target distance.

7. The method according to any one of claims 1-3, wherein, After generating the calibration instructions, the following is also included: After the calibration command is executed, a first control command is generated; the first control command is used to instruct the first acquisition device to acquire a new first target image and to instruct the second acquisition device to acquire a new second target image. Acquire a new first target image acquired by the first acquisition device, and acquire a new second target image acquired by the second acquisition device; The first center position of the mechanical gripper in the newly acquired first target image and the second center position of the target spindle in the newly acquired second target image are detected. Determine whether the target position relationship between the first center position in the new first target image and the second center position in the new second target image meets the second preset requirement.

8. A control device, comprising: The acquisition unit is configured to acquire a first target image acquired by a first acquisition device and a second target image acquired by a second acquisition device, provided that the feature information of the mechanical gripper meets a first preset requirement. The feature information of the mechanical gripper includes at least one of the following: the operating time of the mechanical gripper and the number of times the mechanical gripper grips a spindle on the yarn train. The first acquisition device is mounted on the yarn train for image acquisition of the mechanical gripper. The second acquisition device is mounted on the robotic arm where the mechanical gripper is located for image acquisition of the target spindle on the yarn train to be gripped by the mechanical gripper. The detection unit is used to detect the first center position of the mechanical gripper in the first target image and the second center position of the target spindle in the second target image; The processing unit is configured to generate a calibration command when the target position relationship between the first center position and the second center position does not meet the second preset requirement. The calibration command is used to calibrate the center position of the mechanical gripper or to calibrate the center position of the target spindle to be gripped by the mechanical gripper. Specifically, the processing unit is used for: If the target distance is determined to be greater than the first-level critical value and the target distance is greater than the second-level critical value, a calibration command is generated. The calibration command is used to instruct the adjustment of the position of the yarn carriage where the target yarn spindle is located based on the target distance. The first-level critical value is less than the second-level critical value. The target distance represents the relative distance between the center positions of the mechanical gripper and the target yarn spindle in the same target coordinate system.

9. The apparatus according to claim 8, wherein, The detection unit is specifically used for: The first target image is input into the target detection model to obtain the first center position of the mechanical gripper in the first target image; The second target image is input into the target detection model to obtain the second center position of the target spindle in the second target image.

10. The apparatus according to claim 9, wherein, The target detection model includes at least a first network layer, a second network layer, and a third network layer; The first network layer includes at least a first sub-network layer, a second sub-network layer, and a third sub-network layer. The first sub-network layer performs a first convolutional process on the input image to obtain a first convolutional feature map. The second sub-network layer performs a linear process on the first convolutional feature map to obtain a linear feature map, and then connects the linear feature map with the first convolutional feature map to obtain a connection feature map. The third sub-network layer performs a second convolutional process on the connection feature map to obtain a second convolutional feature map. The second network layer is used to perform first convolution processing on the connection feature map and the second convolution feature map respectively, and perform feature fusion processing on the processed results to obtain a fused feature map; The third network layer includes a fourth sub-network layer and a fifth sub-network layer. The fourth sub-network layer is used to identify the mechanical gripper or target spindle in the image based on the fused feature map. The fifth sub-network layer is used to obtain the center position of the mechanical gripper or the center position of the target spindle in the image based on the fused feature map.

11. The apparatus according to any one of claims 8-10, wherein, The processing unit is further configured to: The target distance is estimated based on the first center position and the second center position; the first center position is the position information in the first coordinate system corresponding to the first target image; the second center position is the position information in the second coordinate system corresponding to the second target image; the target coordinate system is one of the first coordinate system, the second coordinate system, and the world coordinate system.

12. The apparatus according to claim 11, wherein, The processing unit is specifically used for: Based on the first center position, the first center coordinates of the mechanical gripper in the target coordinate system are estimated; Based on the second center position, the second center coordinates of the target spindle in the target coordinate system are estimated; The target distance is obtained based on the first center coordinates of the mechanical gripper in the target coordinate system and the second center coordinates of the target spindle in the target coordinate system.

13. The apparatus according to claim 8, wherein, The processing unit is further configured to: If the target distance is determined to be greater than a first-level critical value and less than or equal to a second-level critical value, a calibration command is generated, which is used to instruct the control parameters of the mechanical gripper to be adjusted based on the target distance.

14. The apparatus according to any one of claims 8-10, wherein, The processing unit is further configured to generate a first control command after the calibration command is executed; the first control command is configured to instruct the first acquisition device to acquire a new first target image and to instruct the second acquisition device to acquire a new second target image. The acquisition unit is further configured to acquire a new first target image acquired by the first acquisition device, and to acquire a new second target image acquired by the second acquisition device; The detection unit is also used to detect the first center position of the mechanical gripper in the new first target image and the second center position of the target spindle in the new second target image; The processing unit is further configured to determine whether the target position relationship between the first center position in the new first target image and the second center position in the new second target image meets the second preset requirement.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images