A target medicine bottle grabbing method, device, equipment and storage medium

Abstract

The present application relates to target medicine bottle grabbing method, device, equipment and storage medium, belong to mechanical arm technical field.The present application includes: obtaining training sample picture and the current state of mechanical arm, target feature extraction is carried out to training sample picture, obtains the corresponding target position information, determines the current action vector;Mechanical arm executes according to the current action vector, and obtains current grabbing result and current grabbing state after completion;According to the preset reward function, the corresponding reward value is calculated, the action prediction model is optimized, the optimized action prediction model is obtained, and the current grabbing state is used as the next current state;According to the optimized action prediction model and the target position information of the target medicine bottle, the action vector of the mechanical arm is used to grab the target medicine bottle.Through the present application, it is helpful to solve the problem of high cost and low efficiency caused by manual picking and supporting in the existing grabbing pharmaceutical production line.

Description

Technical Field

[0001] This invention relates to the field of robotic arm technology, specifically to a method, apparatus, equipment, and storage medium for grasping target medicine bottles. Background Technology

[0002] In pharmaceutical production lines, medicine bottles can sometimes fall over. However, the entire pharmaceutical process must be carried out in a Class A cleanroom environment. Personnel handling these bottles on the production line must adhere to strict aseptic techniques, which is demanding, and the working environment is harsh. Furthermore, because bottle falls are relatively infrequent, the effective working time for these personnel is very short, resulting in significant costs. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, device and storage medium for picking up target medicine bottles, so as to solve the problems of high cost and low efficiency caused by manual picking up of fallen medicine bottles in the existing technology for picking up medicine bottles in pharmaceutical production lines.

[0004] According to a first aspect of the present invention, a method for grasping a target medicine bottle is provided, comprising:

[0005] Acquire training sample images and the current state of the robotic arm. The training sample images are pictures of medicine bottles that have fallen on the assembly line. The current state includes the current position information and posture information.

[0006] Target features are extracted from the training sample images to obtain the corresponding target location information;

[0007] Within a preset number of grasping steps, the current motion vector is determined based on the target position information and the current state of the robotic arm;

[0008] After the robotic arm completes the operation according to the current motion vector, it obtains the current grasping result and the current grasping state.

[0009] Based on the current capture result, the corresponding reward value is calculated according to the preset reward function;

[0010] Based on the reward value and the current grasping state, the action prediction model is optimized to obtain an optimized action prediction model, and the current grasping state is used as the current state for the next step.

[0011] Based on the optimized motion prediction model and the target position information of the target medicine bottle, the motion vector of the robotic arm is predicted to grasp the target medicine bottle.

[0012] Preferably, determining the current motion vector based on the target position information and the current state of the robotic arm includes:

[0013] If the current state of the robotic arm is the initial state, the offset value and offset angle of the robotic arm in multiple directions are randomly determined;

[0014] The current action vector is determined based on the offset values ​​and offset angles in the multiple directions.

[0015] Preferably, the method further includes:

[0016] The current state, the current action vector, the current grasping state, and the reward value are stored as an experience data in the experience pool.

[0017] Preferably, the method further includes:

[0018] Determine whether the experience pool is full. If it is full, randomly select multiple experience data from the experience pool, input them into the action prediction model, optimize the action prediction model, and delete the oldest experience data in the experience pool.

[0019] Preferably, the current capture result includes the number of collisions, whether the capture was successful, the distance to the target location information, and the current step count. The step of calculating the corresponding reward value based on the current capture result according to a preset reward function includes:

[0020] The collision reward is determined based on the number of collisions that occur.

[0021] Based on the current motion vector, determine whether the robotic arm exceeds the preset motion range and determine the motion range reward;

[0022] The result reward is determined based on whether the robotic arm's grasping result is successful or unsuccessful.

[0023] The distance reward is determined based on the distance to the target location information;

[0024] The step reward is determined based on the current number of steps.

[0025] An angle reward is determined based on the target position information and the posture information of the robotic arm;

[0026] The collision reward, range of motion reward, result reward, distance reward, step reward, and angle reward are weighted and summed to obtain the corresponding reward value.

[0027] According to a second aspect of the present invention, a target medicine bottle gripping device is provided, comprising:

[0028] The training image acquisition module is used to acquire training sample images and the current state of the robotic arm. The training sample images are images of medicine bottles that have fallen on the assembly line. The current state includes the current position information and posture information.

[0029] The target feature extraction module is used to extract target features from the training sample images to obtain the corresponding target location information;

[0030] The motion vector determination module is used to determine the current motion vector within a preset number of grasping steps, based on the target position information and the current state of the robotic arm.

[0031] The result acquisition module is used to obtain the current grasping result and the current grasping state after the robotic arm has completed the execution according to the current action vector;

[0032] The calculation module is used to calculate the corresponding reward value according to the current capture result and a preset reward function;

[0033] The model training module is used to optimize the action prediction model based on the reward value and the current grasping state to obtain the optimized action prediction model, and to use the current grasping state as the current state for the next step.

[0034] A target grasping module is used to predict the motion vector of the robotic arm based on the optimized motion prediction model and the target position information of the medicine bottle to be detected, so as to grasp the target medicine bottle. According to a third aspect of the present invention, a target medicine bottle grasping device is provided, comprising:

[0035] Memory, on which executable programs are stored;

[0036] A processor for executing the executable program in the memory to implement the steps of any of the methods described above;

[0037] A robotic arm for performing the current action according to the current action vector.

[0038] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a computer to perform the steps of any of the methods described above.

[0039] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0040] By acquiring training sample images and the current state of the robotic arm (since the training sample images are pictures of medicine bottles falling on an assembly line, the current state includes current position and posture information), target features are extracted from the training sample images to obtain the corresponding target position information. Within a preset number of grasping steps, the current motion vector is determined based on the target position information and the current state of the robotic arm. After the robotic arm executes the current motion vector, the current grasping result and current grasping state are obtained. Based on the current grasping result, the corresponding reward value is calculated according to a preset reward function. Based on the reward value and the current grasping state, the motion prediction model is optimized to obtain the optimized motion prediction model. The current grasping state is used as the current state for the next step. Based on the optimized motion prediction model and the target position information of the target medicine bottle, the motion vector of the robotic arm is predicted to grasp the target medicine bottle. Since the optimized motion prediction model is trained on images of fallen medicine bottles, it has experience and can predict the motion vector of the robotic arm based on the fallen target medicine bottles on the production line. The robotic arm can then grasp the fallen target medicine bottle based on the motion vector, which is more efficient and does not require human intervention, saving human resources and time costs. This effectively solves the problem of high cost and low efficiency caused by manual picking up of fallen medicine bottles in the pharmaceutical production line.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0043] Figure 1 This is a flowchart illustrating a method according to an exemplary embodiment;

[0044] Figure 2 This is a block diagram illustrating a system according to an exemplary embodiment. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0046] Currently, most robotic arm designs for grasping are based on the ability to accurately grasp target items. However, for medicine bottles in pharmaceutical production lines, which are often clustered together, such robotic arms that only focus on accurately identifying target items may disturb other normally positioned medicine bottles, causing them to tip over, increasing unnecessary work and costs. This does not effectively solve the problem of grasping fallen medicine bottles. Furthermore, due to limited training data, the accuracy of identifying target medicine bottles is low, the grasping process is time-consuming, and the efficiency is low.

[0047] This invention provides a method for grasping target medicine bottles, see [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart illustrating a target medicine bottle grasping method according to an exemplary embodiment, the method comprising:

[0048] Step S11: Obtain training sample images and the current state of the robotic arm. The training sample images are pictures of medicine bottles that have fallen on the assembly line. The current state includes the current position information and posture information.

[0049] Step S12: Extract target features from the training sample images to obtain the corresponding target location information;

[0050] Step S13: Within a preset number of grasping steps, determine the current motion vector based on the target position information and the current state of the robotic arm;

[0051] Step S14: After the robotic arm completes the operation according to the current motion vector, it obtains the current grasping result and the current grasping state.

[0052] Step S15: Based on the current capture result, calculate the corresponding reward value according to the preset reward function;

[0053] Step S16: Optimize the action prediction model based on the reward value and the current grasping state to obtain the optimized action prediction model, and use the current grasping state as the current state for the next step.

[0054] Step S17: Based on the optimized motion prediction model and the target position information of the target medicine bottle, predict the motion vector of the robotic arm to grasp the target medicine bottle.

[0055] Specifically, the process involves acquiring training sample images and the current state of the robotic arm. The training sample images are pictures of medicine bottles that have fallen on the assembly line, including the pose data of the fallen bottles. Multiple training samples can be obtained. The current state of the robotic arm includes its current position and orientation information. The position information includes the spatial position of the gripper's end effector relative to a reference coordinate system, which can be represented by coordinates (p...). x ,py ,p z The attitude information includes the RPY angle information of the gripper at the end of the robotic arm relative to the reference coordinate system, which can be represented by [α, β, γ]. The RPY angle information describes the attitude of the gripper relative to a certain reference coordinate system, referring to the attitude angle of the gripper. R represents the angle of rotation around the x-axis, also known as the roll angle; P represents the angle of rotation around the y-axis, also known as the pitch angle; and -Y represents the angle of rotation around the z-axis, also known as the yaw angle. The RPY angle information and position information of the gripper can represent the current state of the robotic arm, and the next action of the robotic arm can be controlled accordingly based on the RPY angle information and position information of the gripper.

[0056] For the training sample images obtained above, target features are extracted to obtain the corresponding target location information. This target location information includes the center coordinates and posture information of the fallen medicine bottle. As training sample images, the target location information can also be multiple key points extracted from the medicine bottle, such as vertices and edge points.

[0057] An initial model is established by combining Actor and Critic networks. Within a preset number of grasping steps, the target position information of the fallen medicine bottle and the current state of the robotic arm are input into the initial Actor network. The Actor network processes the data through an action selection mechanism (such as a Gaussian noise strategy) to obtain the current motion vector. This current motion vector is then sent as an action command to the robotic arm, enabling it to perform the corresponding action and grasp the fallen medicine bottle. The current motion vector includes the offset values ​​and offset angles of the robotic arm in multiple directions.

[0058] After the robotic arm executes the motion command according to the current motion vector, it obtains the current grasping result and the current grasping state. The current grasping state is the state of the robotic arm after executing the motion command, including the position and posture information of the robotic arm after execution. The current grasping result includes information such as whether the grasp was successful, the distance to the fallen medicine bottle, and whether a collision occurred.

[0059] The current capture result and current capture state are input into the Critic network. The Critic network calculates the reward value corresponding to the current action vector according to the preset reward function, which is used to evaluate the quality of the current action vector generated by the Actor network and feeds it back to the Actor network.

[0060] Based on the calculated reward value and the current capture state, the Actor network is optimized to continuously improve its generation capabilities and update its parameters. Simultaneously, the Critic network calculates the corresponding reward value based on the action vectors generated by the Actor network to update its parameters. After updating the parameters, the optimized action prediction model is obtained.

[0061] For a robotic arm that has executed the current action vector, the current grasping state is used as the current state for the next step. The updated Actor and Critic networks are used to predict action vectors and update parameters. In the continuous grasping task, the Actor and Critic networks continuously optimize the robotic arm's action vector prediction, enabling the robotic arm to grasp the fallen medicine bottle more accurately.

[0062] The system monitors medicine bottles on the production line in real time, captures photos, and identifies fallen medicine bottles as target bottles. After extracting the target location information of the target bottle, this information, along with the current state of the robotic arm, is input into the optimized motion prediction model. Based on this model, the system predicts the current motion vector of the robotic arm. By iteratively performing motion prediction, the robotic arm can grab the fallen target medicine bottle on the production line and lift it up without manual intervention.

[0063] It should be noted that each execution of a motion vector by the robotic arm is considered one step. The preset number of steps represents the number of times the robotic arm executes motion vectors when picking up a fallen medicine bottle. It can be 50, or it can be set as needed. This invention does not impose any specific limitations on this.

[0064] In a specific example, both the Actor and Critic networks consist of two fully connected layers. The Actor network uses ReLU and Tanh activation functions, while the Critic network uses the ReLU activation function. All online networks and the target network have identical structures (the number of neurons in each layer and the number of layers are the same). The Actor network has 128 neurons in layer 1 and 64 neurons in layer 2. Given the current state s, the Actor network processes the input through the fully connected layers into the neurons in layer 1, then through the ReLU function into the neurons in layer 2, and finally outputs the current action vector a through the Tanh function. The Critic network consists of two groups of 128 neurons each in layer 1 and 64 neurons each in layer 2. Given the current state s and the current action vector a, the Critic network processes the input through the fully connected layers into one group of neurons in layer 1. Then, the two groups of neurons in layer 1 are concatenated, processed through the ReLU function, and input into the neurons in layer 2, finally outputting the reward value Q.

[0065] It is understood that the technical solution provided in this embodiment obtains training sample images and the current state of the robotic arm. Since the training sample images are pictures of medicine bottles that have fallen on the assembly line, the current state includes current position information and posture information. Target features are extracted from the training sample images to obtain the corresponding target position information. Within a preset number of grasping steps, the current action vector is determined based on the target position information and the current state of the robotic arm. After the robotic arm completes the action according to the current action vector, the current grasping result and current grasping state are obtained. Based on the current grasping result, the corresponding reward value is calculated according to a preset reward function. Based on the reward value and the current grasping state, the action prediction model is optimized to obtain an optimal result. The optimized motion prediction model is used, and the current grasping state is taken as the current state for the next step. Based on the optimized motion prediction model and the target position information of the target medicine bottle, the motion vector of the robotic arm is predicted to grasp the target medicine bottle. Since the optimized motion prediction model is trained on images of fallen medicine bottles, it has experience and can predict the motion vector of the robotic arm based on the fallen target medicine bottles on the production line. The robotic arm can then grasp the fallen target medicine bottle based on the motion vector, which is more efficient and does not require human intervention, saving human resources and time costs. This effectively solves the problem of high cost and low efficiency caused by manual picking up of fallen medicine bottles in pharmaceutical production lines.

[0066] Preferably, in step S13, determining the current motion vector based on the target position information and the current state of the robotic arm includes:

[0067] If the current state of the robotic arm is the initial state, the offset value and offset angle of the robotic arm in multiple directions are randomly determined;

[0068] The current action vector is determined based on the offset values ​​and offset angles in the multiple directions.

[0069] Specifically, based on the target position information and the current state of the robotic arm, the current motion vector is determined. During training, when the robotic arm's current state is the initial state (i.e., before any grasping action), the offset values ​​and angles of the robotic arm in multiple directions are randomly determined based on the initial state, where the offset value is [d]. x d y d z ] represents the offset value of the gripping point of the jaw relative to the base coordinate system in the three directions [x, y, z], with an offset angle of [d]. α d β d γ [d] represents the offset angle of the gripper relative to the base coordinate system in the [x, y, z] directions. Based on the offset value [d]... x d y dz ] and offset angle is [d α d β d γ ], determine the current action vector as [d x d y d z d α d β d γ After the current motion vector is sent to the robotic arm, the robotic arm executes the motion, where the rotation sequence of the robotic arm is x—y—z.

[0070] It should be noted that the offset value of the gripper can be pre-defined in the range of [-200mm, 200mm] in the three directions [x, y, z] for each execution, and the rotation range of the offset angle of the gripper is [-20°, 20°].

[0071] When the current state of the robotic arm is not the initial state, that is, after the grasping action has been performed, the current action vector is determined based on the updated Actor and Critic networks.

[0072] Preferably, the method further includes:

[0073] The current state, the current action vector, the current grasping state, and the reward value are stored as an experience data in the experience pool.

[0074] Specifically, the data information before and after the robotic arm executes the current motion vector, including the current state, current motion vector, current grasping state, and calculated reward value, is stored as an experience data entry in the experience pool. The experience pool is used to store the experience data of the robotic arm grasping the fallen medicine bottle.

[0075] It should be noted that the experience pool can be pre-set in size to store a certain amount of experience data.

[0076] Preferably, the method further includes:

[0077] Determine whether the experience pool is full. If it is full, randomly select multiple experience data from the experience pool, input them into the action prediction model, optimize the action prediction model, and delete the oldest experience data in the experience pool.

[0078] Specifically, after the robotic arm executes each motion vector and calculates the corresponding reward value, it needs to determine whether the experience pool is full. If it is not full, the current state, current motion vector, current grasping state, and calculated reward value are stored as one piece of experience data in the experience pool. Then, it checks whether the experience pool is full again. If it is still not full, it continues to extract target position information from training sample images. Based on the target position information and the current state of the robotic arm, it determines the current motion vector, calculates the reward value, and stores the current state, current motion vector, current grasping state, and calculated reward value from this step as another piece of experience data in the experience pool.

[0079] If the pool is full, multiple empirical data points are randomly selected. The current state and target position information from each empirical data point are input into the Actor and Critic networks to obtain the corresponding current action vector, current grasping state, and reward value. The parameters of the action prediction model are then updated. The oldest empirical data point in the pool is deleted, and the current state, current action vector, current grasping state, and calculated reward value from this step are stored as a new empirical data point in the pool, thus updating the pool. Randomly selecting empirical data from the pool removes correlation and dependency among samples, making the action prediction model more likely to converge.

[0080] Preferably, the current capture result includes the number of collisions, whether the capture was successful, the distance to the target location information, and the current step count. The step of calculating the corresponding reward value based on the current capture result according to a preset reward function includes:

[0081] The collision reward is determined based on the number of collisions that occur.

[0082] Based on the current motion vector, determine whether the robotic arm exceeds the preset motion range and determine the motion range reward;

[0083] The result reward is determined based on whether the robotic arm's grasping result is successful or unsuccessful.

[0084] The distance reward is determined based on the distance to the target location information;

[0085] The step reward is determined based on the current number of steps.

[0086] An angle reward is determined based on the target position information and the posture information of the robotic arm;

[0087] The collision reward, range of motion reward, result reward, distance reward, step reward, and angle reward are weighted and summed to obtain the corresponding reward value.

[0088] It should be noted that reward sparsity is a common problem during the motion prediction training of robotic arms, which directly affects the convergence of the reward function and the training effect of the robotic arm. A well-designed reward function can, to some extent, mitigate the reward sparsity problem.

[0089] Specifically, the current grasping result of the robotic arm includes the number of collisions that occurred during the execution of the motion vector, whether the grasp was successful, the distance to the target position information, and the current step number. The current step number is the number of times the motion vector has been executed up to the current step number in this round of grasping the fallen medicine bottle.

[0090] Based on the current capture results, the corresponding reward value is calculated according to the preset reward function as follows:

[0091] Based on the current grasping results and the number of collisions, a collision reward R1 is determined. For example, if the mechanical wall or the gripper collides with other medicine bottles on the assembly line, the ground, or the robotic arm itself, then R1 = -100. If the mechanical wall or the gripper does not collide, then R1 = 0.

[0092] Based on the current motion vector, determine whether the robotic arm exceeds the preset motion range, and determine the motion range reward R2. The preset motion range can be set with a threshold value as needed. For example, if the current motion vector of the robotic arm exceeds the preset motion range, then R2 = -100. If the current motion vector does not exceed the preset motion range, then R1 = 0.

[0093] The reward R3 is determined based on whether the robotic arm's grasping result is successful or unsuccessful. For example, if the robotic arm's grasping result is successful, then R3 = 200.

[0094] The distance reward R4 is determined based on the distance to the target location information. For example: R4 = ln(d t -d t+1 -1), where d t Let d be the distance between the gripper's holding point and the target's position information at step t. t+1 This represents the distance to the target's location at step t+1.

[0095] The step reward R5 is determined based on the current number of steps. For example, in a round where the robotic arm needs to grab a fallen medicine bottle to complete the grabbing target with fewer steps, a negative reward of -1 is given for each execution of the action vector in each round. The maximum number of steps per round is limited to 50. If this maximum number of steps is exceeded, the grabbing task is immediately terminated. For example: R5 = number of grabbing steps × (-1).

[0096] Based on the target location information and the robotic arm's posture information, determine the angle reward R6. For example: Where Rt P t P t R, P, and Y represent the attitude information of the gripper at step t, respectively, and R, P, and Y represent the attitude information in the target position information of the fallen medicine bottle.

[0097] The collision reward R1, range of motion reward R2, result reward R3, distance reward R4, step reward R5, and angle reward R6 are weighted and summed to obtain the corresponding reward value L.

[0098] For example, the formula for calculating L is: L=(R1+R2+R3+R4+R5+R6)-1.

[0099] Preferably, the above method further includes:

[0100] The motion prediction model can also employ a Transformer model built using deep learning frameworks such as TensorFlow or PyTorch. When the experience pool is full, a certain amount of experience data can be selected from the pool and input into the Transformer model as training samples to update the Transformer model's parameters. The selection of experience data can prioritize those corresponding to higher reward values. The current state, current motion vector, current grasping state, and reward value from each experience data point are input into the Transformer model for iterative optimization and parameter updates until convergence, resulting in an optimized motion prediction model. Using this optimized model and the target position information of the medicine bottle, the robot arm's motion vector is predicted to grasp the target medicine bottle. The optimized motion prediction model, trained using experience data with higher reward values, enables the robot arm to grasp fallen medicine bottles on the assembly line more accurately and quickly.

[0101] It is understood that the technical solution provided in this embodiment obtains training sample images and the current state of the robotic arm. Since the training sample images are pictures of medicine bottles that have fallen on the assembly line, the current state includes current position information and posture information. Target features are extracted from the training sample images to obtain the corresponding target position information. Within a preset number of grasping steps, the current action vector is determined based on the target position information and the current state of the robotic arm. After the robotic arm completes the action according to the current action vector, the current grasping result and current grasping state are obtained. Based on the current grasping result, the corresponding reward value is calculated according to a preset reward function. Based on the reward value and the current grasping state, the action prediction model is optimized to obtain an optimal result. The optimized motion prediction model is used, and the current grasping state is taken as the current state for the next step. Based on the optimized motion prediction model and the target position information of the target medicine bottle, the motion vector of the robotic arm is predicted to grasp the target medicine bottle. Since the optimized motion prediction model is trained on images of fallen medicine bottles, it has experience and can predict the motion vector of the robotic arm based on the fallen target medicine bottles on the production line. The robotic arm can then grasp the fallen target medicine bottle based on the motion vector, which is more efficient and does not require human intervention, saving human resources and time costs. This effectively solves the problem of high cost and low efficiency caused by manual picking up of fallen medicine bottles in pharmaceutical production lines.

[0102] This invention provides a target medicine bottle gripping device, see [link to related document]. Figure 2 , Figure 2 This is a block diagram illustrating a target medicine bottle gripping device according to an exemplary embodiment, comprising:

[0103] The training image acquisition module 21 is used to acquire training sample images and the current state of the robotic arm. The training sample images are images of medicine bottles that have fallen on the assembly line. The current state includes the current position information and posture information.

[0104] The target feature extraction module 22 is used to extract target features from the training sample images to obtain the corresponding target location information;

[0105] The motion vector determination module 23 is used to determine the current motion vector within a preset number of grasping steps, based on the target position information and the current state of the robotic arm.

[0106] The result acquisition module 24 is used to obtain the current grasping result and the current grasping state after the robotic arm has completed the execution according to the current motion vector;

[0107] The calculation module 25 is used to calculate the corresponding reward value according to the current capture result and a preset reward function.

[0108] The model training module 26 is used to optimize the action prediction model based on the reward value and the current grasping state to obtain the optimized action prediction model, and to use the current grasping state as the current state for the next step.

[0109] The target grasping module 27 is used to predict the motion vector of the robotic arm based on the optimized motion prediction model and the target position information of the medicine bottle to be detected, so as to grasp the target medicine bottle.

[0110] It is understood that the technical solution provided in this embodiment, through the acquisition of training sample images and the current state of the robotic arm mentioned in the above embodiment, since the training sample images are images of medicine bottles falling on the assembly line, the current state includes current position information and posture information; target feature extraction is performed on the training sample images to obtain the corresponding target position information; within a preset grasping step count, the current action vector is determined based on the target position information and the current state of the robotic arm; after the robotic arm completes the execution according to the current action vector, the current grasping result and the current grasping state are obtained; based on the current grasping result, the corresponding reward value is calculated according to a preset reward function; based on the reward value and the current grasping state, the action prediction model is optimized. The process involves optimizing the motion prediction model and using the current grasping state as the next step. Based on the optimized motion prediction model and the target position information of the target medicine bottle, the motion vector of the robotic arm is predicted to grasp the target medicine bottle. Since the optimized motion prediction model is trained on images of fallen medicine bottles, it has experience and can predict the motion vector of the robotic arm based on the fallen target medicine bottles on the production line. This allows the robotic arm to grasp the fallen target medicine bottle based on the motion vector, resulting in higher efficiency. It eliminates the need for human intervention, saving human resources and time costs. This effectively solves the problem of high cost and low efficiency caused by manual picking up of fallen medicine bottles in pharmaceutical production lines.

[0111] The present invention also provides an apparatus comprising:

[0112] Memory, on which executable programs are stored;

[0113] A processor for executing the executable program in the memory to implement the steps of any of the methods described above;

[0114] A robotic arm for performing the current action according to the current action vector.

[0115] Furthermore, the present invention also provides a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of any of the methods described above. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0116] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0117] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0118] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0119] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0120] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0121] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0122] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0123] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0124] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for grasping a target medicine bottle, characterized in that, include: The training sample images and the current state of the robotic arm are obtained. The training sample images are pictures of medicine bottles that have fallen on the assembly line. The current state includes the current position information and posture information. Target features are extracted from the training sample images to obtain the corresponding target location information; Within a preset number of grasping steps, the current motion vector is determined based on the target position information and the current state of the robotic arm; After the robotic arm completes the operation according to the current motion vector, it obtains the current grasping result and the current grasping state. Based on the current capture result, the corresponding reward value is calculated according to the preset reward function; The current state, the current action vector, the current grasping state, and the reward value are stored as an experience data in the experience pool; When the experience pool is full, a preset number of experience data with larger reward values ​​are selected from the experience pool. The selected experience data is input into the action prediction model for iterative optimization until the action prediction model converges, and the optimized action prediction model is obtained. The current grasping state is then used as the current state for the next step. Based on the optimized motion prediction model and the target position information of the target medicine bottle, the motion vector of the robotic arm is predicted to grasp the target medicine bottle. The current capture result includes the number of collisions, whether the capture was successful, the distance to the target location information, and the current step count. Based on the current capture result, a corresponding reward value is calculated according to a preset reward function, including: The collision reward is determined based on the number of collisions that occur. Based on the current motion vector, determine whether the robotic arm exceeds the preset motion range and determine the motion range reward; The result reward is determined based on whether the robotic arm's grasping result is successful or unsuccessful. The distance reward is determined based on the distance to the target location information; The step reward is determined based on the current number of steps. An angle reward is determined based on the target position information and the posture information of the robotic arm; The collision reward, range of motion reward, result reward, distance reward, step reward, and angle reward are weighted and summed to obtain the corresponding reward value.

2. The method according to claim 1, characterized in that, Determining the current motion vector based on the target position information and the current state of the robotic arm includes: If the current state of the robotic arm is the initial state, the offset value and offset angle of the robotic arm in multiple directions are randomly determined; The current action vector is determined based on the offset values ​​and offset angles in the multiple directions.

3. The method according to claim 1, characterized in that, The method further includes: Determine whether the experience pool is full. If it is full, randomly select multiple experience data from the experience pool, input them into the action prediction model, optimize the action prediction model, and delete the oldest experience data in the experience pool.

4. A target medicine bottle gripping device, characterized in that, include: The training image acquisition module is used to acquire training sample images and the current state of the robotic arm. The training sample images are images of medicine bottles that have fallen on the assembly line. The current state includes the current position information and posture information. The target feature extraction module is used to extract target features from the training sample images to obtain the corresponding target location information; The motion vector determination module is used to determine the current motion vector within a preset number of grasping steps, based on the target position information and the current state of the robotic arm. The result acquisition module is used to obtain the current grasping result and the current grasping state after the robotic arm has completed the execution according to the current action vector; The calculation module is used to calculate the corresponding reward value according to the current grasping result and a preset reward function; and to store the current state, the current action vector, the current grasping state and the reward value as an experience data in the experience pool. The model training module is used to select a preset number of experience data with larger reward values ​​from the experience pool when the experience pool is full, input the selected experience data into the action prediction model for iterative optimization until the action prediction model converges, obtain the optimized action prediction model, and use the current grasping state as the current state for the next step. The target grasping module is used to predict the motion vector of the robotic arm based on the optimized motion prediction model and the target position information of the medicine bottle to be detected, so as to grasp the target medicine bottle; The current capture result includes the number of collisions, whether the capture was successful, the distance to the target location information, and the current step count. Based on the current capture result, a corresponding reward value is calculated according to a preset reward function, including: The collision reward is determined based on the number of collisions that occur. Based on the current motion vector, determine whether the robotic arm exceeds the preset motion range and determine the motion range reward; The result reward is determined based on whether the robotic arm's grasping result is successful or unsuccessful. The distance reward is determined based on the distance to the target location information; The step reward is determined based on the current number of steps. An angle reward is determined based on the target position information and the posture information of the robotic arm; The collision reward, range of motion reward, result reward, distance reward, step reward, and angle reward are weighted and summed to obtain the corresponding reward value.

5. A target medicine bottle gripping device, characterized in that, include: Memory, on which executable programs are stored; A processor for executing the executable program in the memory to implement the steps of the method according to any one of claims 1 to 3; A robotic arm for performing the current action according to the current action vector.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the steps of the method according to any one of claims 1 to 3.

Images