Experimental Robot Control Method, Device, Electronic Device and Storage Medium

Abstract

The present application relates to an experimental robot control method, device, electronic device, and storage medium. By acquiring an image collected by a reagent kit corresponding to the experimental robot, the image collected by the reagent kit is an image containing the experimental reagent kit within the target area corresponding to the experimental robot. Based on the image collected by the reagent kit, the reagent kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the reagent kit identifier is determined. Subsequently, robot control information is generated through the target teaching operation information. The robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information. Thus, the reagent kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and there is no need for experimental personnel to manually execute the relevant steps. That is, the problem of low experimental efficiency existing in the existing related technology where experiments are conducted by experimental personnel is solved, and the experimental efficiency can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular, to an experimental robot control method, device, electronic device, and storage medium. Background Art

[0002] With the development of society, the research and development in various fields are also increasing continuously, which makes the experiments carried out in the laboratory gradually increase. Since the execution steps and the data to be collected required for different experiments are different, at this time, professional experimental personnel are needed to carry out the experiments according to the situation by adopting appropriate steps and collect relevant data; however, with the increase in the types and frequencies of experiments, when the experiments are carried out by experimental personnel, there will be a problem of low experimental efficiency. Summary of the Invention

[0003] In order to solve the problem of low experimental efficiency existing in the experiments carried out by experimental personnel in the existing related technologies, the present application provides an experimental robot control method, device, electronic device, and storage medium.

[0004] In a first aspect, the present application provides an experimental robot control method, including:

[0005] Obtaining an image of a reagent kit collected by the experimental robot, where the image of the reagent kit collected is an image including the experimental reagent kit within a target area corresponding to the experimental robot;

[0006] Determining a reagent kit identifier corresponding to the experimental robot according to the image of the reagent kit collected;

[0007] Determining target teaching operation information corresponding to the reagent kit identifier;

[0008] Generating robot control information through the target teaching operation information, where the robot control information is used to control the experimental robot to perform a teaching operation corresponding to the target teaching operation information.

[0009] Optionally, after generating the robot control information through the target teaching operation information, it further includes:

[0010] Obtaining monitoring image information corresponding to the experimental robot;

[0011] Performing operation analysis on the monitoring image information to obtain execution operation information;

[0012] Performing operation matching between the execution operation information and target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot.

[0013] Optionally, the operation matching of the execution operation information with the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot includes:

[0014] Determine the execution operation identifier corresponding to the execution operation information;

[0015] Obtain at least one preset operation information corresponding to the target teaching operation information, and determine the target operation identifier corresponding to the preset operation information;

[0016] When the execution operation identifier matches the target operation identifier, determine the preset operation information corresponding to the target operation identifier as the target operation information;

[0017] Perform operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information;

[0018] Generate the control result corresponding to the experimental robot according to the operation deviation information.

[0019] Optionally, the performing operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information includes:

[0020] Perform execution operation analysis on the execution operation information to obtain execution operation position information and execution operation action information;

[0021] Perform target operation analysis on the target operation information to obtain target operation position information and target operation action information;

[0022] Perform operation position deviation analysis based on the execution operation position information and the target operation position information to obtain operation position deviation information;

[0023] Perform operation action deviation analysis based on the execution operation action information and the target operation action information to obtain operation action deviation information;

[0024] Generate the operation deviation information according to the operation position deviation information and the operation action deviation information.

[0025] Optionally, the generating the control result corresponding to the experimental robot according to the operation deviation information includes:

[0026] Determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information;

[0027] When the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range, determine whether the operation action deviation value corresponding to the operation action deviation information belongs to the preset operation action deviation range;

[0028] When the operation action deviation value belongs to the preset operation action deviation range, determine the normal execution operation result as the control result;

[0029] When the operation action deviation value does not belong to the preset operation action deviation range, determine the abnormal execution operation result as the control result.

[0030] Optionally, the determining the kit identifier corresponding to the experimental robot according to the image collected by the kit includes:

[0031] Perform identification recognition on the image collected by the kit to obtain the kit collection identifier;

[0032] Determine the preset identifier that matches the kit collection identifier as the kit identifier.

[0033] Optionally, the experimental robot control method further includes:

[0034] Obtain the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot;

[0035] Perform operation recording according to the recorded position operation information, recorded action operation information, and timing information to obtain the preset teaching operation information;

[0036] Establish a matching relationship between the preset teaching operation information and the preset operation identifier to generate the kit identifier and the target teaching operation information.

[0037] In a second aspect, the present application provides an experimental robot control device, including:

[0038] An acquisition module, configured to acquire an image of a kit collected by the experimental robot, where the image of the kit collected by the experimental robot is an image including an experimental kit in a target area corresponding to the experimental robot;

[0039] A first determination module, configured to determine the kit identifier corresponding to the experimental robot according to the image of the kit collected;

[0040] A second determination module, configured to determine the target teaching operation information corresponding to the kit identifier;

[0041] A control module, configured to generate robot control information based on the target teaching operation information, where the robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information.

[0042] In a third aspect, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0043] The memory is used to store a computer program;

[0044] The processor, when executing the program stored in the memory, implements the experimental robot control method according to any one of the first aspects.

[0045] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the experimental robot control method according to any one of the first aspects is implemented.

[0046] The experimental robot control method, device, electronic device, and storage medium provided by the embodiments of the present application obtain an image of a reagent kit collected by the experimental robot. The image of the reagent kit collected is an image containing the experimental reagent kit within the target area corresponding to the experimental robot. Based on the image of the reagent kit collected, the reagent kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the reagent kit identifier is determined. Subsequently, robot control information is generated through the target teaching operation information. The robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information. Thus, the robot can be controlled to complete its corresponding experimental steps by using the reagent kit identifier and the target teaching operation information, and then there is no need for experimental personnel to manually execute the relevant steps, that is, the problem of low experimental efficiency existing in the existing related technologies where experiments are carried out by experimental personnel is solved, and the experimental efficiency can be effectively improved. Description of the Drawings

[0047] The drawings here are incorporated into the specification and constitute a part of this specification, showing the embodiments that conform to the present invention, and are used together with the specification to explain the principles of the present invention.

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a schematic flowchart of an experimental robot control method provided by an embodiment of the present application;

[0050] Figure 2 The structural schematic diagram of an experimental robot control device provided by an embodiment of the present application;

[0051] Figure 3 The structural schematic diagram of an electronic device provided by an embodiment of the present application. Specific embodiments

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only a part rather than all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0053] In the traditional technology, when conducting various experiments, it is usually necessary to use a reagent kit to hold the materials for subsequent various experimental steps. With the continuous increase in the types and frequencies of experiments, and the increasing degree of laboratory automation, in order to improve the experimental efficiency, robots or robotic arms are used in the existing related technologies to assist in completing some physical operations of the experiment, such as moving the reagent kit, changing the temperature, etc. However, the existing robots or robotic arms can only perform some of the steps. Although this method reduces the operation steps of the experimental personnel, the experimental personnel still need to operate and monitor each experiment, that is, the improvement of the experimental efficiency is not obvious, and there is still the problem of low experimental efficiency existing in the experiments conducted by the experimental personnel.

[0054] To solve the problem of low experimental efficiency in the existing related technologies in the experiments conducted by the experimental personnel, the present application provides an experimental robot control method, device, electronic device, and storage medium. By acquiring an image of the reagent kit corresponding to the experimental robot, the image of the reagent kit is an image containing the experimental reagent kit in the target area corresponding to the experimental robot, and based on the image of the reagent kit, the reagent kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the reagent kit identifier is determined. Subsequently, robot control information is generated through the target teaching operation information, and the robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information; thus, the reagent kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and then there is no need for the experimental personnel to manually execute the relevant steps, that is, the problem of low experimental efficiency in the existing related technologies in the experiments conducted by the experimental personnel is solved, and the experimental efficiency can be effectively improved.

[0055] Figure 1 The flowchart of an experimental robot control method provided by an embodiment of the present application.

[0056] As shown Figure 1 in the figure, a method for controlling an experimental robot provided by an embodiment of the present application may specifically include the following steps:

[0057] Step S110: Obtain an image of a reagent kit collected by the experimental robot. The image of the reagent kit collected is an image including the reagent kit within the target area corresponding to the experimental robot.

[0058] Among them, the experimental robot may represent a machine device capable of performing experimental steps, such as a robot device, a robotic arm, etc.; and the image of the reagent kit collected by the experimental robot may represent an image including the reagent kit within the target area corresponding to the experimental robot. The target area may represent an area pre-configured for placing the reagent kit grabbed by the experimental robot, that is, the experimental robot can grab the reagent kit from the target area to perform experimental steps. The acquisition method of the image of the reagent kit collected may be camera acquisition, scanner scanning, etc.; this embodiment does not make specific limitations on this.

[0059] Step S120: Determine the reagent kit identifier corresponding to the experimental robot according to the image of the reagent kit collected.

[0060] In this embodiment, after obtaining the image of the reagent kit collected, the reagent kit identifier corresponding to the experimental robot can be determined according to the image of the reagent kit collected. Among them, the reagent kit identifier may represent the identifier corresponding to the reagent kit for which the experimental robot currently needs to perform experimental steps.

[0061] Specifically, this embodiment may include one or more experimental robots, and obtaining the reagent kit identifier corresponding to the experimental robot can be identified by an identifier recognition device. The identifier recognition device may be an image scanning device, a camera, an infrared sensing device, a radio frequency identification device, etc. Of course, the identifier recognition device may be configured on the experimental robot or at a position where the reagent kit placement area corresponding to the experimental robot can be recognized. This embodiment does not make specific limitations on this; among them, the reagent kit placement area corresponding to the experimental robot may represent an area for placing the reagent kit for experimental operations by the experimental robot.

[0062] In addition, the reagent kit identifier may also be used to represent the identifier of the experimental steps to be performed on the reagent kit. For example, the reagent kit identifier A can be used to mark that the experimental steps to be performed on the reagent kit are the steps of extracting deoxyribonucleic acid, and the reagent kit identifier B can be used to mark that the experimental steps to be performed on the reagent kit are the steps of extracting ribonucleic acid. Of course, the above is only an example for illustration, and this embodiment does not make specific limitations on this.

[0063] In specific implementation, the kit identifier can be an identifier obtained from a corresponding preset area of the kit, such as an area on the top of the kit. For example, identifiers such as QR codes and barcodes can be configured in the area on the top of the kit, so as to identify or scan the area on the top of the kit, and then the kit identifier corresponding to the QR code or barcode can be obtained. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0064] It can be seen that in this embodiment, the kit identifier can be used to accurately and quickly identify the experimental steps that the current kit needs to execute. When different experimental steps need to be executed, different kits can be used for loading, thereby improving the experimental efficiency.

[0065] Step S130: Determine the target teaching operation information corresponding to the kit identifier.

[0066] Specifically, after obtaining the kit identifier, the target teaching operation information corresponding to the kit identifier can be determined. The target teaching operation information can represent the operation information of the experimental steps corresponding to the kit identifier. Since the experimental steps corresponding to the kit identifier can be one or more steps, that is, the target teaching operation information can also include one or more operation information, and different operation information can be used to represent different operation steps for the kit.

[0067] In an example, when the target teaching operation information includes moving operation information, dropping experimental solution operation information, and collecting pH value operation information, it means that the operation steps of moving the kit corresponding to the kit identifier, dropping the experimental solution, and collecting the pH value are required. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0068] Step S140: Generate robot control information through the target teaching operation information. The robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information.

[0069] Specifically, after obtaining the target teaching operation information, the robot control information can be generated through the target teaching operation information. The robot control information can be used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information, that is, the robot control information can be a robot control instruction, and the robot control information can include one or more robot control instructions and timing instructions. By the robot control instructions and timing instructions, the robot can be controlled to execute each teaching operation according to the preset timing.

[0070] It can be seen that by acquiring the image collected by the kit corresponding to the experimental robot, where the image collected by the kit is an image containing the experimental kit within the corresponding target area of the experimental robot, based on the image collected by the kit, the kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the kit identifier is determined. Subsequently, based on the target teaching operation information, robot control information is generated, and the robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information. Thus, the kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and thus there is no need for the experimental personnel to manually execute the relevant steps, that is, the problem of low experimental efficiency existing in the existing related technologies where the experiment is carried out by the experimental personnel is solved, and the experimental efficiency can be effectively improved.

[0071] In an alternative embodiment of the present application, after generating the robot control information through the target teaching operation information in step S140, the following steps may specifically be further included:

[0072] Acquire the monitoring image information corresponding to the experimental robot;

[0073] Perform operation analysis on the monitoring image information to obtain the execution operation information;

[0074] Perform operation matching between the execution operation information and the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot.

[0075] After generating the robot control information through the target teaching operation information in this embodiment, the monitoring image information corresponding to the experimental robot can be obtained. The monitoring image information can represent the image of the robot during the teaching operation. Thus, the operation analysis can be performed on the monitoring image information to obtain the execution operation information. The operation analysis can represent the specific operation performed by the current robot determined through the analysis of the monitoring image information, and the execution operation information can represent the specific operation performed by the robot shown in the current monitoring image information, such as operation position and operation action information. Furthermore, the execution operation information can be operationally matched with the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot. Among them, the target operation information corresponding to the target teaching operation information can represent the operation information corresponding to each specific teaching operation included in the target teaching operation information, such as operation position and operation action information. The process of operationally matching the execution operation information with the target operation information can be to determine whether the operation action and operation position between the execution operation information and the target operation information match, and then generate the control result corresponding to the experimental robot. That is, the control result can represent the result of whether the operation performed by the current controlled robot matches the preset target operation. Therefore, the control result can be a normal execution operation result or an abnormal execution operation result. The normal execution operation result can represent the result that the operation performed by the current controlled robot matches the preset target operation, and the abnormal execution operation result can represent the result that the operation performed by the current controlled robot does not match the preset target operation and there is an abnormality.

[0076] In one example, since the target teaching operation information can be one or more pieces of operation information. For example, when the target teaching operation information includes teaching operation A, teaching operation B, and teaching operation C, the target operation information corresponding to the target teaching operation information can include target operation information A, target operation information B, and target operation information C. Target operation information A can represent the moving position and execution action corresponding to teaching operation A, target operation information B can represent the moving position and execution action corresponding to teaching operation B, and target operation information C can represent the moving position and execution action corresponding to teaching operation B. Therefore, various different target teaching operation information can be configured in advance according to different experimental requirements, so that when it is necessary to perform different experimental operations through the robot, the robot can be directly controlled to perform related operations according to the corresponding target teaching operation information, thereby improving the experimental efficiency.

[0077] In specific implementation, the following steps can be referred to:

[0078] 1) The artificial intelligence monitor monitors the robot's experiment execution process in real time, collects experimental data through a camera, that is, obtains the monitoring image information corresponding to the experimental robot. Among them, the camera is relatively fixed with respect to the robot base and does not move with the robot arm.

[0079] 2) Analyze the data using a deep learning model, compare it with the preset standard operation, and detect the deviation from the standard operation, that is, perform operation analysis on the monitoring image information to obtain the execution operation information. The long short-term memory network LSTM is a variant of the RNN, which solves the problems of gradient vanishing and gradient explosion in the RNN when dealing with long sequences. LSTM controls the flow of information by introducing a gating mechanism (input gate, forget gate, output gate), which enables LSTM to learn long-term dependencies. The basic structure of LSTM: Forget gate: Determines which information should be discarded from the cell state. Input gate: Determines which new information will be stored in the cell state. Cell state: Stores long-term memory. Output gate: Determines the output value based on the cell state and the current input. The forward propagation of LSTM: At each time step t, the forward propagation of LSTM can be expressed as: ft = σ(Wf ⋅ [ht−1, xt] + bf); it = σ(Wi ⋅ [ht−1, xt] + bi); t = tanh(W ⋅ [ht−1, xt] + b); Ct = ft * Ct−1 + it * ; ot = σ(Wo ⋅ [ht−1, xt] + bo); ht = ot * tanh(Ct). Where: ft is the activation vector of the forget gate. it is the activation vector of the input gate. is the new candidate cell state. Ct is the cell state. ot is the activation vector of the output gate. ht is the hidden state. W and b are the weight and bias terms. σ is the sigmoid activation function. * represents element-wise multiplication.

[0080] 3) Build a three-level visual perception architecture: The first level: Rapid detection of utensils modified from YOLOv5s (100ms-level response). The second level: 3D reconstruction of the operation scene modified from PointNet++. The third level: Dynamic monitoring of liquid surface fluctuations using the optical flow method

[0081] 4) Develop an error compensation decision tree: Trigger trajectory replanning when the positioning deviation δ > 3mm. Trigger an emergency stop when the force feedback anomaly lasts for 200ms. Start multi-sensor fusion positioning when the visual confidence < 0.7.

[0082] 5) According to the robot's action deviation, position deviation, and grasping deviation, the system may automatically adjust the robot or immediately stop the experiment and issue an alarm to the operator; that is, perform operation matching on the execution operation information and the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot.

[0083] 6) Record the experimental operations and deviation events, generate an experimental report, and improve subsequent analysis and improvement.

[0084] In an optional embodiment of the present application, the operation information is executed, and the target operation information corresponding to the target teaching operation information is subjected to operation analysis to generate a control result corresponding to the experimental robot, which may specifically include the following sub-steps:

[0085] Determine the execution operation identifier corresponding to the execution operation information;

[0086] Obtain at least one preset operation information corresponding to the target teaching operation information, and determine the target operation identifier corresponding to the preset operation information;

[0087] When the execution operation identifier matches the target operation identifier, determine the preset operation information corresponding to the target operation identifier as the target operation information;

[0088] Perform operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information;

[0089] Generate a control result corresponding to the experimental robot according to the operation deviation information.

[0090] In the process of this embodiment analyzing the operation of the execution operation information and the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot, the execution operation identifier corresponding to the execution operation information can be determined. The execution operation identifier can be used to represent the identifier of the operation corresponding to the marked execution operation information. For example, the operation identifier for moving the reagent kit to position A, the operation identifier for shaking the reagent kit X times, etc.; thus, at least one preset operation information corresponding to the target teaching operation information can be obtained, and the target operation identifier corresponding to the preset operation information can be determined. Among them, the preset operation information can represent the operation information corresponding to the teaching operation included in the target teaching operation information, and the target operation identifier can represent the identifier used to mark the specific teaching operation. For example, the preset operation information includes the preset operation information A for moving the reagent kit to position A and the preset operation information B for adding solution Y to the reagent kit. At this time, the target operation identifier can include the target operation identifier A for marking the movement of the reagent kit to position A and the target operation identifier B for marking the movement of the reagent kit to position A; then it can be judged whether the execution operation identifier matches the target operation identifier, which serves to judge whether the operation executed by the current robot matches the preset target operation. That is, when the execution operation identifier matches the target operation identifier, it indicates that the operation executed by the current robot matches the preset target operation. At this time, the preset operation information corresponding to the target operation identifier can be determined as the target operation information; furthermore, the operation deviation analysis can be performed based on the execution operation information and the target operation information to obtain the operation deviation information. The operation deviation information can represent the deviation situation between the execution operation information and the target operation information, such as the position deviation value, action deviation, etc.; subsequently, the control result corresponding to the experimental robot can be generated according to the operation deviation information. At this time, the deviation degree analysis of the operation deviation information can be performed. When the deviation situation represented by the operation deviation information belongs to the abnormal deviation situation, the abnormal execution operation result can be determined as the control result, and when the deviation situation represented by the operation deviation information belongs to the normal deviation situation, the normal execution operation result can be determined as the control result.

[0091] In an alternative embodiment of the present application, performing operation deviation analysis based on the execution operation information and the target operation information to obtain the operation deviation information may specifically include the following sub-steps:

[0092] Performing execution operation analysis on the execution operation information to obtain the execution operation position information and the execution operation action information;

[0093] Performing target operation analysis on the target operation information to obtain the target operation position information and the target operation action information;

[0094] Performing operation position deviation analysis based on the execution operation position information and the target operation position information to obtain the operation position deviation information;

[0095] Perform operation action deviation analysis based on the executed operation action information and the target operation action information to obtain operation action deviation information;

[0096] Generate operation deviation information based on the operation position deviation information and the operation action deviation information.

[0097] In this embodiment, during the process of performing operation deviation analysis based on the executed operation information and the target operation information to obtain operation deviation information, the executed operation information can be analyzed for the executed operation to obtain the executed operation position information and the executed operation action information. The executed operation position information can represent the position movement of the current robot during the operation on the reagent kit, and the executed operation action information can represent the action to which the current robot's operation on the reagent kit belongs. And the target operation information can be analyzed for the target operation to obtain the target operation position information and the target operation action information. The target operation position information can represent the position movement of the preset operation on the reagent kit, and the executed operation action information can represent the action of the preset operation on the reagent kit. Thus, operation position deviation analysis can be performed based on the executed operation position information and the target operation position information to obtain operation position deviation information, where the operation position deviation information can represent the position deviation between the position represented by the executed operation position information and the position represented by the target operation position information. And operation action deviation analysis can be performed based on the executed operation action information and the target operation action information to obtain operation action deviation information. The operation action deviation information can represent the action deviation between the action represented by the executed operation action information and the action represented by the target operation action information. Furthermore, the operation position deviation information and the operation action deviation information can be integrated to generate operation deviation information, that is, taking the preset target operation information as the standard to perform deviation analysis on the executed operation information, where the deviation is analyzed by combining two main factors of position and action, which plays a role in improving the monitoring of the robot and analyzing the operation deviation.

[0098] In one example, when the execution operation information indicates that the robot moves the kit from A1 to A2, where A1 and A2 can be the same or different positions, the execution operation position information at this time includes the specific coordinate positions of A1 and A2 and the operation timing sequence, and the execution operation action information is the moving operation; at this time, if the target operation information is to move the kit from B1 to B2, the target operation position information includes the specific coordinate positions of B1 and B2 and the operation timing sequence, and the target operation action information is the moving operation; then, by performing operation position deviation analysis based on the execution operation position information and the target operation position information, it is possible to analyze the position deviation between A1 and A2 in the execution operation position information and B1 and B2 in the target operation position information, as well as the timing deviation between the operation timing sequence in the execution operation position information and the operation timing sequence in the target operation position information, thereby obtaining the operation position deviation information. For example, calculating the position deviation between A1 and B1 to obtain the starting position deviation, calculating the position deviation between B1 and B2 to obtain the ending position deviation, and calculating the deviation between the operation timing sequence in the execution operation position information and the operation timing sequence in the target operation position information to obtain the timing deviation, and integrating the starting position deviation, the ending position deviation, and the timing deviation into the operation position deviation information; subsequently, by performing operation action deviation analysis based on the execution operation action information and the target operation action information, it is possible to analyze the operation deviation between the moving operation in the execution operation position information and the moving operation in the target operation position information, thereby obtaining the operation action deviation information. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0099] In an alternative embodiment of the present application, according to the operation deviation information, a control result corresponding to the experimental robot is generated, which may specifically include the following sub-steps:

[0100] Determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information;

[0101] When the operation position deviation value corresponding to the operation position deviation information falls within the preset operation position deviation range, determine whether the operation action deviation value corresponding to the operation action deviation information falls within the preset operation action deviation range;

[0102] When the operation action deviation value falls within the preset operation action deviation range, determine the normal execution operation result as the control result;

[0103] When the operation action deviation value does not fall within the preset operation action deviation range, determine the abnormal execution operation result as the control result.

[0104] In the process of generating the control result corresponding to the experimental robot based on the operation deviation information, the present embodiment can first determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information, and judge whether the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range. The operation position deviation value can represent the position deviation value between the position represented by the execution operation position information and the position represented by the target operation position information, and the preset operation position deviation range can represent the pre-configured range value belonging to the normal position deviation. That is, when the operation position deviation value corresponding to the operation position deviation information does not belong to the preset operation position deviation range, it means that the position of the current robot operating the test kit does not belong to the normal position range. At this time, there is no need to execute subsequent steps, and the abnormal execution operation result is determined as the control result; and when the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range , indicating that the position of the current robot's operation on the reagent box belongs to the normal position range. At this time, it can be determined whether the operation action deviation value corresponding to the operation action deviation information belongs to the preset operation action deviation range; wherein, the operation action deviation value can represent the action deviation value between the action represented by the execution operation position information and the action represented by the target operation position information, such as the action type difference, the action amplitude difference, etc., and the preset operation action deviation range can represent the pre-configured range belonging to the normal action deviation; that is, when the operation action deviation value belongs to the preset operation action deviation range, it means that the action performed by the current robot operating the reagent box is within the normal range, and the normal execution operation result can be determined as the control result; and when the operation action deviation value does not belong to the preset operation action deviation range, it means that the action performed by the current robot operating the reagent box is not within the normal range, and the abnormal execution operation result can be determined as the control result.

[0105] It should be noted that in the process of determining whether the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range in this embodiment, it is possible to make judgments for two positions, namely the starting position and the ending position. That is, it is possible to judge whether the starting position deviation value and the ending position deviation value corresponding to the operation position deviation value respectively belong to the preset operation position deviation range. Only when both the starting position deviation value and the ending position deviation value belong to the preset operation position deviation range, will it be determined whether the operation action deviation value corresponding to the operation action deviation information belongs to the preset operation action deviation range. Of course, in the process of determining whether the operation action deviation value corresponding to the operation action deviation information belongs to the preset operation action deviation range, it is also possible to make separate judgments on the action type deviation and the action amplitude deviation. That is, it is possible to separately judge whether the action amplitude deviation belongs to the preset action amplitude deviation range and whether the action type deviation value belongs to the preset action type deviation range. Specifically, the action type can be numerically replaced. The smaller the action type deviation value, the closer the action type is, and the larger the action type deviation value, the greater the type gap. For example, the moving operation is 10, the shaking operation is 20, the left - right shaking operation is 21, and the front - back shaking operation is 22. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0106] It can be seen that in the process of generating robot control information through the target teaching operation information to control the robot to execute the experimental steps in this embodiment, it is also possible to check the deviation between the specific execution operation of the robot and the preset target operation, and generate an operation result according to the deviation situation. And in the case where the operation result is an abnormal execution operation result, a robot stop instruction and an abnormal prompt information can be generated. The robot stop instruction is used to control the robot to stop when there is a deviation in the robot's execution operation; the abnormal prompt information is used to prompt the user that the current robot has an abnormality, which can effectively improve the experimental efficiency.

[0107] In an alternative embodiment of the present application, based on the image collected by the reagent kit, determining the reagent kit identifier corresponding to the experimental robot specifically may include the following sub - steps:

[0108] Perform identifier recognition on the image collected by the reagent kit to obtain the reagent kit collection identifier;

[0109] Determine the preset identifier that matches the reagent kit collection identifier as the reagent kit identifier.

[0110] After obtaining the image collected by the kit in this embodiment, the image collected by the kit can be subjected to identification recognition to obtain the identification collected by the kit, and the identification collected by the kit can represent the identification recognized in the image collected by the kit. Thus, it can be determined whether there is a preset identification that matches the identification collected by the kit, where the preset identification can represent an identification pre-configured for marking one or more kits, that is, the preset identification and the kit type can be in a one-to-one pairing relationship. That is, when there is a preset identification that matches the identification collected by the kit, it indicates that the current kit belongs to the kit pre-configured with an identification. At this time, the preset identification that matches the identification collected by the kit can be determined as the kit identification.

[0111] Specifically, performing identification recognition on the image collected by the kit to obtain the identification collected by the kit can be to extract a preset position area from the image collected by the kit to obtain a preset position area image, and perform identification recognition on the preset position area image to obtain the identification collected by the kit, where the preset position area can represent an area pre-configured on the kit for recording the identification, such as the top of the kit lid, the side of the kit body, etc. Thus, by pre-classifying the kits and configuring the preset identification or kit identification, as well as the target teaching operation information corresponding to the kit identification, when a certain experiment needs to be carried out, the kit for this experiment can be directly used to hold the materials, and the robot can adaptively complete the corresponding experiment. In this process, there is no need for the participation of experimental personnel, thereby effectively improving the experimental efficiency.

[0112] In an alternative embodiment of the present application, it may specifically further include the following steps:

[0113] Obtain the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot;

[0114] Perform operation recording based on the recorded position operation information, recorded action operation information, and timing information to obtain the preset teaching operation information;

[0115] Establish a matching relationship between the preset teaching operation information and the preset operation identifier to generate the kit identification and the target teaching operation information.

[0116] In this embodiment, the preset method for the kit identification and the target teaching operation information may be as follows: First, obtain the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot. The recorded position operation information may represent the position information where the current robot is guided to perform an operation, the recorded action operation information may represent the action information where the current robot is guided to perform an operation, and the timing information may represent the timing when the current robot is guided to perform an operation. Then, an operation record may be made based on the recorded position operation information, recorded action operation information, and timing information to obtain the preset teaching operation information, and a matching relationship may be established between the preset teaching operation information and the preset operation identifier to generate the kit identification and the target teaching operation information, where the preset operation identifier may be the identifier corresponding to the kit input by the user. Thus, the robot can be guided to operate according to requirements in advance, and the record matching can be performed to generate the kit identification and the target teaching operation information.

[0117] In one example, the robot can be manually guided to move to a specific position and posture through the robot teaching handle, and then these paths and actions can be recorded, and the preset operation identifier can be input to generate the kit identification and the target teaching operation information. The specific steps can be referred to as follows

[0118] 1) Prepare the reagents and consumables required for the experiment, start the robot system and input the preset operation identifier.

[0119] 2) Use the robot teaching handle to guide the robot to perform the experiment. The robot records the operation actions and the timing process through the camera and sensors, that is, the recorded position operation information, recorded action operation information, and timing information are obtained.

[0120] Among them, the robot records the operation actions and timing processes through cameras and sensors. The way to obtain the recorded position operation information, recorded action operation information, and timing information can be to convert the recorded operation steps into an executable instruction sequence, learn and store these steps, and establish an automated operation module. For example, use Cartesian space to record and read the encoder values of each joint to obtain the current joint angles. Use the forward kinematics formula to calculate the Cartesian coordinates (x, y, z) and orientation (usually represented by Euler angles or quaternions) of the end effector. Store these coordinates and orientation as a vector P = [x, y, z, α, β, γ]. Associate the vector P with a timestamp or path point index for use during playback. Use the forward kinematics formula to convert joint angles to Cartesian coordinates. Linear interpolation formula for path point playback: P(t) = P0 + t(P1 - P0). Where P0 and P1 are two adjacent path points, t is a parameter, and 0 ≤ t ≤ 1. Spline interpolation formula: P(t) = ∑i = 0n ci Bi(t) where Bi(t) is the B-spline basis function and ci is the control point. Specific implementation method: Configure the robot control system as the input device of the teaching handle. Control the position, pose, and button status of the handle through the API. Collect the parameter data of the handle to generate the motion trajectory of the robot end effector. Use the inverse kinematics algorithm to calculate the angles of each joint of the robot based on the motion trajectory of the end effector. Generate teaching code according to the parameter data of the robot, download the code to the robot controller, and the robot executes the teaching operation.

[0121] 3) Verify and adjust the actions of the robot, and feedback and optimize the robot operation process. Specific implementation method: Verify the accuracy according to the path recorded and played back by the robot. If errors are found, the path points can be modified or re-taught.

[0122] 4) Synchronously collect multi-modal data. During the teaching phase, synchronously record six-dimensional force perception data (sampling rate 1 kHz) to construct a spatio-temporal alignment matrix: Msync = [Pt Ft Vt]. Where Pt is the pose vector, Ft is the end force torque, and Vt is the visual feature vector.

[0123] 5) Trajectory optimization module. Adopt constrained minimum energy optimization:

[0124] min∫tft0∥τ(t)∥2dt

[0125] s.t. qmin ≤ q(t) ≤ qmax

[0126] ∥q˙(t)∥ ≤ vmax

[0127] 6) Save the information corresponding to the correct operation steps, that is, perform operation recording according to the operation information of the recorded position, the operation information of the recorded action, and the timing information, to obtain the preset teaching operation information.

[0128] As Figure 2 shown, the present application also discloses an embodiment, which provides an experimental robot control device applied to a television, and may include the following modules:

[0129] An acquisition module 210, configured to acquire a kit acquisition image corresponding to the experimental robot, where the kit acquisition image is an image including an experimental kit in a target area corresponding to the experimental robot;

[0130] A first determination module 220, configured to determine a kit identifier corresponding to the experimental robot according to the kit acquisition image;

[0131] A second determination module 230, configured to determine target teaching operation information corresponding to the kit identifier;

[0132] A control module 240, configured to generate robot control information through the target teaching operation information, where the robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information.

[0133] In an alternative embodiment of the present application, the experimental robot control device may further include:

[0134] A monitoring module, configured to acquire monitoring image information corresponding to the experimental robot;

[0135] An operation analysis module, configured to perform operation analysis on the monitoring image information to obtain execution operation information;

[0136] An operation matching module, configured to perform operation matching between the execution operation information and the target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot.

[0137] In an alternative embodiment of the present application, the operation matching module may include:

[0138] A first determination unit, configured to determine an execution operation identifier corresponding to the execution operation information;

[0139] A first acquisition unit, configured to acquire at least one preset operation information corresponding to the target teaching operation information and determine a target operation identifier corresponding to the preset operation information;

[0140] A second determination unit, configured to, when the execution operation identifier matches the target operation identifier, determine the preset operation information corresponding to the target operation identifier as the target operation information;

[0141] An operation deviation analysis unit, configured to perform operation deviation analysis based on the executed operation information and the target operation information to obtain operation deviation information;

[0142] A first generation unit, configured to generate a control result corresponding to the experimental robot according to the operation deviation information.

[0143] In an optional embodiment of the present application, the operation deviation analysis unit may include:

[0144] An operation analysis subunit, configured to perform executed operation analysis on the executed operation information to obtain executed operation position information and executed operation action information;

[0145] A target operation analysis subunit, configured to perform target operation analysis on the target operation information to obtain target operation position information and target operation action information;

[0146] An operation position deviation analysis subunit, configured to perform operation position deviation analysis based on the executed operation position information and the target operation position information to obtain operation position deviation information;

[0147] An operation action deviation analysis subunit, configured to perform operation action deviation analysis based on the executed operation action information and the target operation action information to obtain operation action deviation information;

[0148] A generation subunit, configured to generate the operation deviation information according to the operation position deviation information and the operation action deviation information.

[0149] In an optional embodiment of the present application, the first generation unit may include:

[0150] A first determination subunit, configured to determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information;

[0151] A second determination subunit, configured to determine whether the operation action deviation value corresponding to the operation action deviation information belongs to a preset operation action deviation range when the operation position deviation value corresponding to the operation position deviation information belongs to a preset operation position deviation range;

[0152] A third determination subunit, configured to determine the normal execution operation result as the control result when the operation action deviation value belongs to the preset operation action deviation range;

[0153] A fourth determination subunit, configured to determine the abnormal execution operation result as the control result when the operation action deviation value does not belong to the preset operation action deviation range.

[0154] In an alternative embodiment of the present application, the acquisition module 210 may include:

[0155] An identification recognition unit, configured to perform identification recognition on the image collected by the kit to obtain a kit collection identification;

[0156] A third determination unit, configured to determine a preset identification that matches the kit collection identification as the kit identification.

[0157] In an alternative embodiment of the present application, the experimental robot control device may further include:

[0158] An acquisition record module, configured to acquire the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot;

[0159] An operation record module, configured to perform operation recording according to the recorded position operation information, recorded action operation information, and timing information to obtain preset teaching operation information;

[0160] A matching establishment module, configured to establish a matching relationship between the preset teaching operation information and a preset operation identification, and generate the kit identification and the target teaching operation information.

[0161] The implementation processes of the functions and roles of each module in the above device are specifically described in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0162] As Figure 3 shown, an embodiment of the present application provides an electronic device, including a processor 310, a communication interface 320, a memory 330, and a communication bus 340. Among them, the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340;

[0163] The memory 330 is used to store a computer program;

[0164] In an embodiment of the present application, when the processor 310 executes the program stored on the memory 330, it implements the experimental robot control method provided in any of the foregoing method embodiments. By acquiring an image of a reagent kit corresponding to the experimental robot, the image of the reagent kit is an image containing the experimental reagent kit within the corresponding target area of the experimental robot. Based on the image of the reagent kit, the reagent kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the reagent kit identifier is determined. Subsequently, through the target teaching operation information, robot control information is generated. The robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information. Thus, the reagent kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and further, there is no need for an experimental operator to manually execute the relevant steps, that is, the problem of low experimental efficiency existing in the prior related art where experiments are carried out by experimental operators is solved, and the experimental efficiency can be effectively improved.

[0165] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the experimental robot control method provided in any of the foregoing method embodiments. By acquiring an image of a reagent kit corresponding to the experimental robot, the image of the reagent kit is an image containing the experimental reagent kit within the corresponding target area of the experimental robot. Based on the image of the reagent kit, the reagent kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the reagent kit identifier is determined. Subsequently, through the target teaching operation information, robot control information is generated. The robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information. Thus, the reagent kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and further, there is no need for an experimental operator to manually execute the relevant steps, that is, the problem of low experimental efficiency existing in the prior related art where experiments are carried out by experimental operators is solved, and the experimental efficiency can be effectively improved.

[0166] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0167] The specific embodiments of the present specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0168] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. An experimental robot control method, characterized in that, Including: Obtaining a kit acquisition image corresponding to the experimental robot, where the kit acquisition image is an image containing an experimental kit within a target area corresponding to the experimental robot; Determining a kit identifier corresponding to the experimental robot according to the kit acquisition image; Determining target teaching operation information corresponding to the kit identifier; Generating robot control information through the target teaching operation information, where the robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information; After generating the robot control information through the target teaching operation information, it further includes: Obtaining monitoring image information corresponding to the experimental robot, Performing operation analysis on the monitoring image information to obtain execution operation information, Performing operation matching on the execution operation information and target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot; The performing operation matching on the execution operation information and target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot includes: Determining an execution operation identifier corresponding to the execution operation information; Obtaining at least one preset operation information corresponding to the target teaching operation information and determining a target operation identifier corresponding to the preset operation information; When the execution operation identifier matches the target operation identifier, determining the preset operation information corresponding to the target operation identifier as the target operation information; Performing operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information; Generating a control result corresponding to the experimental robot according to the operation deviation information.

2. The experimental robot control method according to claim 1, characterized in that The performing operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information includes: Performing execution operation analysis on the execution operation information to obtain execution operation position information and execution operation action information; Performing target operation analysis on the target operation information to obtain target operation position information and target operation action information; Performing operation position deviation analysis based on the execution operation position information and the target operation position information to obtain operation position deviation information; Performing operation action deviation analysis based on the execution operation action information and the target operation action information to obtain operation action deviation information; Generating the operation deviation information according to the operation position deviation information and the operation action deviation information.

3. The experimental robot control method according to claim 2, characterized in that The generating a control result corresponding to the experimental robot according to the operation deviation information includes: Determining the operation position deviation information and the operation action deviation information corresponding to the operation deviation information; When the operation position deviation value corresponding to the operation position deviation information is within a preset operation position deviation range, determining whether the operation action deviation value corresponding to the operation action deviation information is within a preset operation action deviation range; When the operation action deviation value is within the preset operation action deviation range, determining the normal execution operation result as the control result; In the case where the operation action deviation value does not belong to the preset operation action deviation range, determine the abnormal execution operation result as the control result.

4. The experimental robot control method according to claim 1, characterized in that, The determining the kit identifier corresponding to the experimental robot according to the image collected by the kit includes: Perform identification recognition on the image collected by the kit to obtain the kit collection identifier; Determine the preset identifier that matches the kit collection identifier as the kit identifier.

5. The experimental robot control method according to claim 1, characterized in that, It further includes: Obtain the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot; Perform an operation record according to the recorded position operation information, recorded action operation information, and timing information to obtain the preset teaching operation information; Establish a matching relationship between the preset teaching operation information and the preset operation identifier to generate the kit identifier and the target teaching operation information.

6. An experimental robot control device, characterized in that, It includes: An acquisition module, configured to acquire the kit collection image corresponding to the experimental robot, where the kit collection image is an image including an experimental kit within the target area corresponding to the experimental robot; A first determination module, configured to determine the kit identifier corresponding to the experimental robot according to the kit collection image; A second determination module, configured to determine the target teaching operation information corresponding to the kit identifier; A control module, configured to generate robot control information through the target teaching operation information, where the robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information; The experimental robot control device further includes: A monitoring module, configured to acquire the monitoring image information corresponding to the experimental robot, An operation analysis module, configured to perform operation analysis on the monitoring image information to obtain the execution operation information, An operation matching module, configured to perform operation matching between the execution operation information and the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot; The operation matching module includes: A first determination unit, configured to determine the execution operation identifier corresponding to the execution operation information; A first acquisition unit, configured to acquire at least one preset operation information corresponding to the target teaching operation information and determine the target operation identifier corresponding to the preset operation information; A second determination unit, configured to, in the case where the execution operation identifier matches the target operation identifier, determine the preset operation information corresponding to the target operation identifier as the target operation information; An operation deviation analysis unit, configured to perform operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information; A first generation unit, configured to generate the control result corresponding to the experimental robot according to the operation deviation information.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store a computer program; The processor, when executing the program stored on the memory, implements the experimental robot control method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the experimental robot control method according to any one of claims 1-5.

Images