Experimental robot control method and device, electronic equipment and storage medium

Abstract

The invention relates to an experimental robot control method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a kit collection image corresponding to an experimental robot, and enabling the kit collection image to be an image containing an experimental kit in a target region corresponding to the experimental robot, so as to achieve the control of the experimental robot according to the kit collection image; a kit identifier corresponding to the experimental robot is determined, target teaching operation information corresponding to the kit identifier is determined, then robot control information is generated through the target teaching operation information, and the robot control information is used for controlling the experimental robot and executing teaching operation corresponding to the target teaching operation information; therefore, the robot can be controlled to complete the corresponding experiment steps by using the kit identifier and the target teaching operation information, so that an experimenter does not need to manually execute the related steps, the problem that the experiment efficiency is low when the experimenter performs an experiment in the prior art is solved, and the experiment efficiency can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular, to an experimental robot control method, device, electronic device, and storage medium. Background Art

[0002] With the development of society, the research and development in various fields are also increasing continuously, which makes the experiments carried out in the laboratory gradually increase. Since the execution steps and the data to be collected required for different experiments are different, professional experimental personnel are needed to conduct experiments according to the situation using appropriate steps and collect relevant data at this time; however, with the increase in the types and frequencies of experiments, conducting experiments through experimental personnel will have the problem of low experimental efficiency at this time. Summary of the Invention

[0003] In order to solve the problem of low experimental efficiency in the existing related technologies when conducting experiments through experimental personnel, this application provides an experimental robot control method, device, electronic device, and storage medium.

[0004] In a first aspect, this application provides an experimental robot control method, including: Obtain a kit acquisition image corresponding to the experimental robot, where the kit acquisition image is an image including an experimental kit in a target area corresponding to the experimental robot; Determine a kit identifier corresponding to the experimental robot according to the kit acquisition image; Determine target teaching operation information corresponding to the kit identifier; Generate robot control information through the target teaching operation information, where the robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information.

[0005] Optionally, after generating the robot control information through the target teaching operation information, it further includes: Obtain monitoring image information corresponding to the experimental robot; Perform operation analysis on the monitoring image information to obtain execution operation information; Perform operation matching between the execution operation information and the target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot.

[0006] Optionally, the performing operation matching between the execution operation information and the target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot includes: Determine an execution operation identifier corresponding to the execution operation information; Obtain at least one preset operation information corresponding to the target teaching operation information, and determine a target operation identifier corresponding to the preset operation information; When the execution operation identifier matches the target operation identifier, determine the preset operation information corresponding to the target operation identifier as the target operation information; Perform operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information; Generate a control result corresponding to the experimental robot according to the operation deviation information.

[0007] Optionally, the performing operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information includes: Perform execution operation analysis on the execution operation information to obtain execution operation position information and execution operation action information; Perform target operation analysis on the target operation information to obtain target operation position information and target operation action information; Perform operation position deviation analysis based on the execution operation position information and the target operation position information to obtain operation position deviation information; Perform operation action deviation analysis based on the execution operation action information and the target operation action information to obtain operation action deviation information; Generate the operation deviation information according to the operation position deviation information and the operation action deviation information.

[0008] Optionally, the generating a control result corresponding to the experimental robot according to the operation deviation information includes: Determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information; When the operation position deviation value corresponding to the operation position deviation information is within a preset operation position deviation range, determine whether the operation action deviation value corresponding to the operation action deviation information is within a preset operation action deviation range; When the operation action deviation value is within the preset operation action deviation range, determine the normal execution operation result as the control result; When the operation action deviation value is not within the preset operation action deviation range, determine the abnormal execution operation result as the control result.

[0009] Optionally, the determining a kit identifier corresponding to the experimental robot according to the image collected by the kit includes: Perform identification recognition on the image collected by the kit to obtain a kit collection identifier; Determine the preset identifier that matches the collection identifier of the kit as the kit identifier.

[0010] Optionally, the experimental robot control method further includes: Obtain the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot; Perform an operation record based on the recorded position operation information, recorded action operation information, and timing information to obtain preset teaching operation information; Establish a matching relationship between the preset teaching operation information and a preset operation identifier to generate the kit identifier and the target teaching operation information.

[0011] In a second aspect, the present application provides an experimental robot control device, including: An acquisition module, configured to acquire a kit acquisition image corresponding to the experimental robot, where the kit acquisition image is an image including an experimental kit within a target area corresponding to the experimental robot; A first determination module, configured to determine a kit identifier corresponding to the experimental robot based on the kit acquisition image; A second determination module, configured to determine target teaching operation information corresponding to the kit identifier; A control module, configured to generate robot control information through the target teaching operation information, where the robot control information is used to control the experimental robot to execute a teaching operation corresponding to the target teaching operation information.

[0012] In a third aspect, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store a computer program; The processor, when executing the program stored on the memory, implements the experimental robot control method according to any one of the first aspects.

[0013] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the experimental robot control method according to any one of the first aspects is implemented.

[0014] The experimental robot control method, device, electronic device, and storage medium provided by the embodiments of the present application obtain an image collected by a reagent kit corresponding to the experimental robot. The image collected by the reagent kit is an image containing the experimental reagent kit within the target area corresponding to the experimental robot. Based on the image collected by the reagent kit, the reagent kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the reagent kit identifier is determined. Subsequently, robot control information is generated through the target teaching operation information. The robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information. Thus, the reagent kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and there is no need for experimental personnel to manually execute the relevant steps, that is, the problem of low experimental efficiency existing in the existing related technologies where experiments are carried out by experimental personnel is solved, and the experimental efficiency can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a schematic flowchart of an experimental robot control method provided by an embodiment of the present application; Figure 2 It is a schematic structural diagram of an experimental robot control device provided by an embodiment of the present application; Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0019] In the traditional technology, when conducting various experiments, it is usually necessary to use a kit to contain the materials, which is convenient for subsequent various experimental steps on the materials. With the continuous increase in the types and frequencies of experiments, and the increasing degree of laboratory automation, in order to improve the experimental efficiency, robots or robotic arms are used in the existing related technologies to assist in completing some physical operations of the experiment, such as moving the kit, changing the temperature, etc. However, the existing robots or robotic arms can only perform some steps. Although this method reduces the operation steps of the experimental personnel, the experimental personnel still need to operate and monitor each experiment. That is, the improvement of the experimental efficiency is not obvious, and there is still the problem of low experimental efficiency existing in the experiments carried out by the experimental personnel.

[0020] To solve the problem of low experimental efficiency in the existing related technologies where experiments are carried out by experimental personnel, this application provides an experimental robot control method, device, electronic device, and storage medium. By obtaining the acquisition image of the kit corresponding to the experimental robot, the acquisition image of the kit is an image containing the experimental kit in the target area corresponding to the experimental robot, and based on the acquisition image of the kit, the kit identifier corresponding to the experimental robot is determined, and the target teaching operation information corresponding to the kit identifier is determined. Subsequently, the robot control information is generated through the target teaching operation information, and the robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information. Thus, the kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and then there is no need for the experimental personnel to manually execute the relevant steps. That is, the problem of low experimental efficiency in the existing related technologies where experiments are carried out by experimental personnel is solved, and the experimental efficiency can be effectively improved.

[0021] Figure 1 It is a schematic flowchart of an experimental robot control method provided by an embodiment of this application.

[0022] As Figure 1 shown, an experimental robot control method provided by an embodiment of this application may specifically include the following steps: Step S110: Obtain the acquisition image of the kit corresponding to the experimental robot, and the acquisition image of the kit is an image containing the experimental kit in the target area corresponding to the experimental robot.

[0023] Among them, the experimental robot can represent a machine device capable of performing experimental steps, such as a robot device, a robotic arm, etc.; the image of the reagent kit collected by the experimental robot can represent an image containing the reagent kit within the corresponding target area of the experimental robot, and the target area can represent an area pre-configured for placing the reagent kit to be grabbed by the experimental robot, that is, the experimental robot can grab the reagent kit from the target area to perform experimental steps, and the acquisition method of the image of the reagent kit collected can be camera acquisition, scanner scanning, etc.; this embodiment does not make specific limitations on this.

[0024] Step S120: Determine the reagent kit identifier corresponding to the experimental robot according to the image of the reagent kit collected.

[0025] In this embodiment, after obtaining the image of the reagent kit collected, the reagent kit identifier corresponding to the experimental robot can be determined according to the image of the reagent kit collected, where the reagent kit identifier can represent the identifier corresponding to the reagent kit for which the experimental robot currently needs to perform experimental steps.

[0026] Specifically, this embodiment may include one or more experimental robots, and obtaining the reagent kit identifier corresponding to the experimental robot can be recognized by an identifier recognition device, and the identifier recognition device can be an image scanning device, a camera, an infrared sensing device, a radio frequency identification device, etc. Of course, the identifier recognition device can be configured on the experimental robot or at a position where the placement area of the reagent kit corresponding to the experimental robot can be recognized. This embodiment does not make specific limitations on this; among them, the placement area of the reagent kit corresponding to the experimental robot can represent the area for placing the reagent kit for which the experimental robot performs experimental operations.

[0027] In addition, the reagent kit identifier can also be used to represent the identifier of the experimental steps to be performed on the reagent kit. For example, the reagent kit identifier A can be used to mark that the experimental steps to be performed on the reagent kit are the steps of extracting deoxyribonucleic acid, and the reagent kit identifier B can be used to mark that the experimental steps to be performed on the reagent kit are the steps of extracting ribonucleic acid. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0028] During specific implementation, the reagent kit identifier can be an identifier obtained from a preset area corresponding to the reagent kit. For example, the area on the top of the reagent kit, such as a two-dimensional code, a bar code, etc. can be configured in the area on the top of the reagent kit, so as to identify or scan the area on the top of the reagent kit, and the reagent kit identifier corresponding to the two-dimensional code and the bar code can be obtained. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0029] It can be seen that in this embodiment, the kit identifier can be used to accurately and quickly identify the experimental steps that the current kit needs to perform, so that when different experimental steps need to be performed, different kits can be used for loading, thereby improving the experimental efficiency.

[0030] Step S130: Determine the target teaching operation information corresponding to the kit identifier.

[0031] Specifically, after obtaining the kit identifier, the target teaching operation information corresponding to the kit identifier can be determined. The target teaching operation information can represent the operation information of the experimental steps corresponding to the kit identifier. Since the experimental steps corresponding to the kit identifier can be one or more steps, that is, the target teaching operation information can also include one or more operation information, and different operation information can be used to represent different operation steps for the kit.

[0032] In one example, when the target teaching operation information includes movement operation information, dropping experimental solution operation information, and collecting pH value operation information, it means that the operation steps of moving the kit corresponding to the kit identifier, dropping the experimental solution, and collecting the pH value need to be performed. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations.

[0033] Step S140: Generate robot control information through the target teaching operation information. The robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information.

[0034] Specifically, after obtaining the target teaching operation information, robot control information can be generated through the target teaching operation information. The robot control information can be used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information, that is, the robot control information can be a robot control instruction, and the robot control information can include one or more robot control instructions and timing instructions. By means of the robot control instructions and timing instructions, the robot can be controlled to perform each teaching operation according to the preset timing.

[0035] It can be seen that by acquiring the image collected by the kit corresponding to the experimental robot, where the image collected by the kit is an image containing the experimental kit within the target area corresponding to the experimental robot, to determine the kit identifier corresponding to the experimental robot based on the image collected by the kit, and to determine the target teaching operation information corresponding to the kit identifier. Subsequently, robot control information is generated through the target teaching operation information, and the robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information. Thus, the kit identifier and the target teaching operation information can be utilized to control the robot to complete its corresponding experimental steps, and further, it is not necessary for the experimental personnel to manually execute the relevant steps, that is, the problem of low experimental efficiency existing in the existing related technologies where experiments are conducted by experimental personnel is solved, and the experimental efficiency can be effectively improved.

[0036] In an alternative embodiment of the present application, after generating the robot control information through the target teaching operation information in step S140, the following steps may specifically further be included: Acquire the monitoring image information corresponding to the experimental robot; Perform operation analysis on the monitoring image information to obtain the execution operation information; Perform operation matching between the execution operation information and the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot.

[0037] After generating the robot control information based on the target teaching operation information in this embodiment, the monitoring image information corresponding to the experimental robot can be obtained. The monitoring image information can represent the images of the robot during the teaching operation. Then, the operation analysis can be performed on the monitoring image information to obtain the execution operation information. The operation analysis can represent determining the specific operations performed by the current robot through the analysis of the monitoring image information, and the execution operation information can represent the specific operations performed by the robot shown in the current monitoring image information, such as operation position and operation action information. Furthermore, the execution operation information can be operationally matched with the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot. Among them, the target operation information corresponding to the target teaching operation information can represent the operation information corresponding to each specific teaching operation included in the target teaching operation information, such as operation position and operation action information. The process of operationally matching the execution operation information with the target operation information can be to determine whether the operation actions and operation positions between the execution operation information and the target operation information match, and then generate the control result corresponding to the experimental robot. That is, the control result can represent whether the operation performed by the current controlled robot matches the preset target operation. Therefore, the control result can be a normal execution operation result or an abnormal execution operation result. The normal execution operation result can represent that the operation performed by the current controlled robot matches the preset target operation, and the abnormal execution operation result can represent that the operation performed by the current controlled robot does not match the preset target operation and there is an abnormality.

[0038] In one example, since the target teaching operation information can be one or more operation information. For example, when the target teaching operation information includes teaching operation A, teaching operation B, and teaching operation C, the target operation information corresponding to the target teaching operation information can include target operation information A, target operation information B, and target operation information C. Target operation information A can represent the moving position and execution action corresponding to teaching operation A, target operation information B can represent the moving position and execution action corresponding to teaching operation B, and target operation information C can represent the moving position and execution action corresponding to teaching operation B. Therefore, various different target teaching operation information can be configured in advance according to different experimental requirements, so that when different experimental operations need to be performed by the robot, the robot can be directly controlled to perform relevant operations according to the corresponding target teaching operation information, thereby improving the experimental efficiency.

[0039] In specific implementation, the following steps can be referred to: 1) The artificial intelligence monitor monitors the robot's execution of the experiment in real time, collects experimental data through the camera, that is, obtains the monitoring image information corresponding to the experimental robot. Among them, the camera is relatively fixed with the robot base and does not move with the robot arm.

[0040] 2) Analyze the data using a deep learning model, compare it with the preset standard operations, and detect the deviation from the standard operations, that is, perform operation analysis on the monitored image information to obtain the execution operation information. The long short-term memory network LSTM is a variant of the RNN, which solves the problems of gradient vanishing and gradient explosion in the RNN when dealing with long sequences. LSTM controls the flow of information by introducing a gating mechanism (input gate, forget gate, output gate), which enables LSTM to learn long-term dependencies. The basic structure of LSTM: Forget gate: Determines which information should be discarded from the cell state. Input gate: Determines which new information will be stored in the cell state. Cell state: Stores long-term memory. Output gate: Determines the output value, based on the cell state and the current input. The forward propagation of LSTM: At each time step t, the forward propagation of LSTM can be expressed as: ft = σ(Wf ⋅ [ht−1, xt] + bf); it = σ(Wi ⋅ [ht−1, xt] + bi); t = tanh(W ⋅ [ht−1, xt] + b); Ct = ft * Ct−1 + it * ; ot = σ(Wo ⋅ [ht−1, xt] + bo); ht = ot * tanh(Ct). Where: ft is the activation vector of the forget gate. it is the activation vector of the input gate. is the new candidate cell state. Ct is the cell state. ot is the activation vector of the output gate. ht is the hidden state. W and b are the weight and bias terms. σ is the sigmoid activation function. * represents element-wise multiplication.

[0041] 3) Build a three-level visual perception architecture: The first level: Rapid detection of utensils modified from YOLOv5s (response at the 100ms level). The second level: 3D reconstruction of the operation scene modified from PointNet++. The third level: Dynamic monitoring of liquid surface fluctuations using the optical flow method 4) Develop an error compensation decision tree: Trigger trajectory replanning when the positioning deviation δ > 3mm. Trigger an emergency stop when the force feedback anomaly lasts for 200ms. Start multi-sensor fusion positioning when the visual confidence < 0.7.

[0042] 5) According to the action deviation, position deviation, and grasping deviation of the robot, the system may automatically adjust the robot or immediately stop the experiment and issue an alarm to the operator; that is, perform operation matching on the execution operation information and the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot.

[0043] 6) Record the experimental operations and deviation events, generate an experimental report, and improve subsequent analysis and improvement.

[0044] In an optional embodiment of the present application, operation analysis is performed on the execution operation information and the target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot, which may specifically include the following sub-steps: Determine the execution operation identifier corresponding to the execution operation information; Obtain at least one preset operation information corresponding to the target teaching operation information, and determine the target operation identifier corresponding to the preset operation information; When the execution operation identifier matches the target operation identifier, determine the preset operation information corresponding to the target operation identifier as the target operation information; Perform operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information; Generate a control result corresponding to the experimental robot according to the operation deviation information.

[0045] In the process of this embodiment analyzing the operation of the execution operation information and the target operation information corresponding to the target teaching operation information to generate the control result corresponding to the experimental robot, the execution operation identifier corresponding to the execution operation information can be determined. The execution operation identifier can be used to represent the identifier of the operation corresponding to the marked execution operation information. For example, the operation identifier for moving the reagent kit to position A, the operation identifier for shaking the reagent kit X times, etc. Thus, at least one preset operation information corresponding to the target teaching operation information can be obtained, and the target operation identifier corresponding to the preset operation information can be determined. Among them, the preset operation information can represent the operation information corresponding to the teaching operation included in the target teaching operation information, and the target operation identifier can represent the identifier used to mark the specific teaching operation. For example, the preset operation information A for moving the reagent kit to position A and the preset operation information B for adding solution Y into the reagent kit are included in the preset operation information. At this time, the target operation identifier can include the target operation identifier A for marking the movement of the reagent kit to position A and the target operation identifier B for marking the movement of the reagent kit to position A. Then, it can be determined whether the execution operation identifier matches the target operation identifier, which serves to determine whether the operation performed by the current robot matches the preset target operation. That is, when the execution operation identifier matches the target operation identifier, it indicates that the operation performed by the current robot matches the preset target operation. At this time, the preset operation information corresponding to the target operation identifier can be determined as the target operation information. Furthermore, operation deviation analysis can be performed based on the execution operation information and the target operation information to obtain operation deviation information. The operation deviation information can represent the deviation situation between the execution operation information and the target operation information. For example, the position deviation value, action deviation, etc. Subsequently, the control result corresponding to the experimental robot can be generated according to the operation deviation information. At this time, the deviation degree analysis of the operation deviation information can be performed. When the deviation situation represented by the operation deviation information belongs to the abnormal deviation situation, the abnormal execution operation result can be determined as the control result, and when the deviation situation represented by the operation deviation information belongs to the normal deviation situation, the normal execution operation result can be determined as the control result.

[0046] In an alternative embodiment of the present application, performing operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information may specifically include the following sub-steps: Performing execution operation analysis on the execution operation information to obtain the execution operation position information and the execution operation action information; Performing target operation analysis on the target operation information to obtain the target operation position information and the target operation action information; Performing operation position deviation analysis based on the execution operation position information and the target operation position information to obtain the operation position deviation information; Performing operation action deviation analysis based on the execution operation action information and the target operation action information to obtain the operation action deviation information; Generate operation deviation information based on the operation position deviation information and the operation action deviation information.

[0047] In this embodiment, during the process of performing operation deviation analysis based on the execution operation information and the target operation information to obtain the operation deviation information, the execution operation information can be analyzed to obtain the execution operation position information and the execution operation action information. The execution operation position information can represent the position movement of the current robot during the operation on the reagent kit, and the execution operation action information can represent the action to which the current robot's operation on the reagent kit belongs. And the target operation information can be analyzed to obtain the target operation position information and the target operation action information. The target operation position information can represent the position movement during the preset operation on the reagent kit, and the execution operation action information can represent the action of the preset operation on the reagent kit. Thus, the operation position deviation analysis can be performed based on the execution operation position information and the target operation position information to obtain the operation position deviation information, where the operation position deviation information can represent the position deviation between the position represented by the execution operation position information and the position represented by the target operation position information. And the operation action deviation analysis can be performed based on the execution operation action information and the target operation action information to obtain the operation action deviation information. The operation action deviation information can represent the action deviation between the action represented by the execution operation action information and the action represented by the target operation action information. Furthermore, the operation position deviation information and the operation action deviation information can be integrated to generate the operation deviation information, that is, taking the preset target operation information as the standard to perform deviation analysis on the execution operation information, where the deviation is combined with two main factors of position and action, which plays a role in improving the monitoring of the robot and analyzing the operation deviation.

[0048] In one example, when the execution operation information indicates that the robot moves the kit from A1 to A2, where A1 and A2 can be the same or different positions, the execution operation position information at this time includes the specific coordinate positions of A1 and A2 and the operation timing sequence, and the execution operation action information is the moving operation; at this time, if the target operation information is to move the kit from B1 to B2, the target operation position information includes the specific coordinate positions of B1 and B2 and the operation timing sequence, and the target operation action information is the moving operation; then, by performing operation position deviation analysis based on the execution operation position information and the target operation position information, it is possible to analyze the position deviation between A1 and A2 in the execution operation position information and B1 and B2 in the target operation position information, as well as the timing deviation between the operation timing sequence in the execution operation position information and the operation timing sequence in the target operation position information, thereby obtaining the operation position deviation information. For example, calculating the position deviation between A1 and B1 to obtain the starting position deviation, calculating the position deviation between B1 and B2 to obtain the ending position deviation, and calculating the deviation between the operation timing sequence in the execution operation position information and the operation timing sequence in the target operation position information to obtain the timing deviation, and integrating the starting position deviation, the ending position deviation, and the timing deviation into the operation position deviation information in this way; subsequently, by performing operation action deviation analysis based on the execution operation action information and the target operation action information, it is possible to analyze the operation deviation between the moving operation in the execution operation position information and the moving operation in the target operation position information, thereby obtaining the operation action deviation information. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0049] In an alternative embodiment of the present application, according to the operation deviation information, a control result corresponding to the experimental robot is generated, which may specifically include the following sub-steps: Determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information; When the operation position deviation value corresponding to the operation position deviation information falls within the preset operation position deviation range, determine whether the operation action deviation value corresponding to the operation action deviation information falls within the preset operation action deviation range; When the operation action deviation value falls within the preset operation action deviation range, determine the normal execution operation result as the control result; When the operation action deviation value does not fall within the preset operation action deviation range, determine the abnormal execution operation result as the control result.

[0050] In the process of generating the control result corresponding to the experimental robot based on the operation deviation information, the present embodiment can first determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information, and judge whether the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range. The operation position deviation value can represent the position deviation value between the position represented by the execution operation position information and the position represented by the target operation position information, and the preset operation position deviation range can represent the pre-configured range value belonging to the normal position deviation. That is, when the operation position deviation value corresponding to the operation position deviation information does not belong to the preset operation position deviation range, it means that the position of the current robot operating the test kit does not belong to the normal position range. At this time, there is no need to execute subsequent steps, and the abnormal execution operation result is determined as the control result; and when the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range , indicating that the position of the current robot's operation on the reagent box belongs to the normal position range. At this time, it can be determined whether the operation action deviation value corresponding to the operation action deviation information belongs to the preset operation action deviation range; wherein, the operation action deviation value can represent the action deviation value between the action represented by the execution operation position information and the action represented by the target operation position information, such as the action type difference, the action amplitude difference, etc., and the preset operation action deviation range can represent the pre-configured range belonging to the normal action deviation; that is, when the operation action deviation value belongs to the preset operation action deviation range, it means that the action performed by the current robot operating the reagent box is within the normal range, and the normal execution operation result can be determined as the control result; and when the operation action deviation value does not belong to the preset operation action deviation range, it means that the action performed by the current robot operating the reagent box is not within the normal range, and the abnormal execution operation result can be determined as the control result.

[0051] It should be noted that in the process of determining whether the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range in this embodiment, it is possible to make judgments for two positions, namely the starting position and the ending position. That is, it is possible to determine whether the starting position deviation value and the ending position deviation value corresponding to the operation position deviation value respectively belong to the preset operation position deviation range. Only when both the starting position deviation value and the ending position deviation value belong to the preset operation position deviation range, can it be determined whether the operation action deviation value corresponding to the operation action deviation information belongs to the preset operation action deviation range. Of course, in the process of determining whether the operation action deviation value corresponding to the operation action deviation information belongs to the preset operation action deviation range, it is also possible to make separate judgments on the action type deviation and the action amplitude deviation. That is, it is possible to separately determine whether the action amplitude deviation belongs to the preset action amplitude deviation range, and whether the action type deviation value belongs to the preset action type deviation range. Specifically, the action type can be numerically replaced. The smaller the action type deviation value, the closer the action type is. The larger the action type deviation value, the greater the type gap. For example, the moving operation is 10, the shaking operation is 20, the left - right shaking operation is 21, and the front - back shaking operation is 22. Of course, the above is only for illustrative purposes, and this embodiment does not make specific limitations on this.

[0052] It can be seen that in the process of controlling and executing the experimental steps of the robot by generating robot control information through the target teaching operation information in this embodiment, it is also possible to check the deviation between the specific execution operation of the robot and the preset target operation, and generate an operation result according to the deviation situation. And in the case where the operation result is an abnormal execution operation result, a robot stop instruction and an abnormal prompt information can be generated. The robot stop instruction is used to control the robot to stop when there is a deviation in the robot's execution operation; the abnormal prompt information is used to prompt the user that the current robot is abnormal, which can effectively improve the experimental efficiency.

[0053] In an alternative embodiment of the present application, based on the image collected by the reagent kit, determining the reagent kit identifier corresponding to the experimental robot specifically may include the following sub - steps: Perform identification recognition on the image collected by the reagent kit to obtain the reagent kit collection identifier; Determine the preset identifier that matches the reagent kit collection identifier as the reagent kit identifier.

[0054] After obtaining the image of the kit collected in this embodiment, the image of the kit collected can be subjected to identification recognition to obtain the kit collection identification, where the kit collection identification can represent the identification recognized in the image of the kit collected; thus, it can be determined whether there is a preset identification that matches the kit collection identification, where the preset identification can represent the identification pre-configured for marking one or more kits, that is, the preset identification and the kit type can be in a one-to-one pairing relationship; that is, when there is a preset identification that matches the kit collection identification, it indicates that the current kit belongs to the kit pre-configured with an identification. At this time, the preset identification that matches the kit collection identification can be determined as the kit identification.

[0055] Specifically, performing identification recognition on the image of the kit collected to obtain the kit collection identification may be to extract a preset position area from the image of the kit collected to obtain a preset position area image, and perform identification recognition on the preset position area image to obtain the kit collection identification, where the preset position area may represent the area pre-configured on the kit for recording the identification, such as the top of the lid of the kit, the side of the kit body, etc. Thus, by pre-classifying the kits and configuring the preset identification or kit identification, as well as the target teaching operation information corresponding to the kit identification, when a certain experiment needs to be carried out, the kit for this experiment can be directly used to hold the materials, and then the robot can adaptively complete the corresponding experiment. In this process, there is no need for the participation of experimental personnel, thereby effectively improving the experimental efficiency.

[0056] In an alternative embodiment of the present application, it may specifically further include the following steps: Obtain the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot; Perform operation recording according to the recorded position operation information, recorded action operation information, and timing information to obtain preset teaching operation information; Establish a matching relationship between the preset teaching operation information and the preset operation identification to generate a kit identification and target teaching operation information.

[0057] In this embodiment, the preset method for the kit identification and the target teaching operation information can be as follows: First, obtain the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot. The recorded position operation information can represent the position information where the current robot is guided to perform an operation, the recorded action operation information can represent the action information where the current robot is guided to perform an operation, and the timing information can represent the timing when the current robot is guided to perform an operation. Then, the operation can be recorded based on the recorded position operation information, recorded action operation information, and timing information to obtain the preset teaching operation information, and a matching relationship is established between the preset teaching operation information and the preset operation identifier to generate the kit identification and the target teaching operation information, where the preset operation identifier can be the identifier corresponding to the kit input by the user. Thus, the robot can be guided to operate according to requirements in advance, and the recording and matching are performed to generate the kit identification and the target teaching operation information.

[0058] In one example, the robot can be manually guided to move to a specific position and posture through a robot teaching handle, and then these paths and actions are recorded, and the preset operation identifier is input to generate the kit identification and the target teaching operation information. Specifically, the following steps can be referred to 1) Prepare the reagent consumables required for the experiment, start the robot system and input the preset operation identifier.

[0059] 2) Use the robot teaching handle to guide the robot to perform the experiment. The robot records the operation actions and the timing process through the camera and sensors, that is, the recorded position operation information, recorded action operation information, and timing information are obtained.

[0060] Among them, the robot records the operation actions and timing processes through cameras and sensors. The way to obtain the recorded position operation information, recorded action operation information, and timing information can be to convert the recorded operation steps into an executable instruction sequence, learn and store these steps, and establish an automated operation module. For example, use Cartesian space to record and read the encoder values of each joint to obtain the current joint angles. Use the forward kinematics formula to calculate the Cartesian coordinates (x, y, z) and orientation (usually represented by Euler angles or quaternions) of the end effector. Store these coordinates and orientation as a vector P = [x, y, z, α, β, γ]. Associate the vector P with a timestamp or path point index for use during playback. Use the forward kinematics formula to convert joint angles to Cartesian coordinates. Linear interpolation formula for path point playback: P(t) = P0 + t(P1 - P0). Where P0 and P1 are two adjacent path points, t is a parameter, and 0 ≤ t ≤ 1. Spline interpolation formula: P(t) = ∑i = 0n ci Bi(t) Where Bi(t) is the B-spline basis function and ci is the control point. Specific implementation method: Configure the robot control system as the input device of the teaching handle. Regulate the position, posture, and button status of the handle through the API. Collect the parameter data of the handle to generate the motion trajectory of the robot end effector. Use the inverse kinematics algorithm to calculate the angles of each joint of the robot based on the motion trajectory of the end effector. Generate teaching code according to the parameter data of the robot, download the code to the robot controller, and the robot executes the teaching operation.

[0061] 3) Verify and adjust the actions of the robot, and feedback and optimize the robot operation process. Specific implementation method: Verify the accuracy according to the path recorded and played back by the robot. If errors are found, the path points can be modified or re-taught.

[0062] 4) Synchronously collect multi-modal data. During the teaching phase, synchronously record six-dimensional force perception data (sampling rate 1 kHz) to construct a spatio-temporal alignment matrix: Msync = [Pt Ft Vt]. Where Pt is the pose vector, Ft is the end force torque, and Vt is the visual feature vector.

[0063] 5) Trajectory optimization module. Adopt constrained minimum energy optimization: min∫tft0∥τ(t)∥2dt s.t. qmin≤q(t)≤qmax ∥q˙(t)∥≤vmax 6) Save the information corresponding to the correct operation steps, that is, perform operation records based on the recorded position operation information, recorded action operation information, and timing information to obtain the preset teaching operation information.

[0064] Such as Figure 2As shown, the present application also discloses an embodiment, providing an experimental robot control device, which is applied to a television and may include the following modules: An acquisition module 210, configured to acquire an image of a reagent kit collected by the experimental robot, where the image of the reagent kit collected is an image including the experimental reagent kit within a target area corresponding to the experimental robot; A first determination module 220, configured to determine a reagent kit identifier corresponding to the experimental robot according to the image of the reagent kit collected; A second determination module 230, configured to determine target teaching operation information corresponding to the reagent kit identifier; A control module 240, configured to generate robot control information through the target teaching operation information, where the robot control information is used to control the experimental robot to perform a teaching operation corresponding to the target teaching operation information.

[0065] In an optional embodiment of the present application, the experimental robot control device may further include: A monitoring module, configured to acquire monitoring image information corresponding to the experimental robot; An operation analysis module, configured to perform operation analysis on the monitoring image information to obtain execution operation information; An operation matching module, configured to perform operation matching on the execution operation information and target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot.

[0066] In an optional embodiment of the present application, the operation matching module may include: A first determination unit, configured to determine an execution operation identifier corresponding to the execution operation information; A first acquisition unit, configured to acquire at least one preset operation information corresponding to the target teaching operation information and determine a target operation identifier corresponding to the preset operation information; A second determination unit, configured to, when the execution operation identifier matches the target operation identifier, determine the preset operation information corresponding to the target operation identifier as the target operation information; An operation deviation analysis unit, configured to perform operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information; A first generation unit, configured to generate a control result corresponding to the experimental robot according to the operation deviation information.

[0067] In an optional embodiment of the present application, the operation deviation analysis unit may include: An operation analysis subunit, configured to perform execution operation analysis on the execution operation information to obtain execution operation position information and execution operation action information; A target operation analysis subunit, configured to perform target operation analysis on the target operation information to obtain target operation position information and target operation action information; An operation position deviation analysis subunit, configured to perform operation position deviation analysis based on the executed operation position information and the target operation position information to obtain operation position deviation information; An operation action deviation analysis subunit, configured to perform operation action deviation analysis based on the executed operation action information and the target operation action information to obtain operation action deviation information; A generation subunit, configured to generate the operation deviation information according to the operation position deviation information and the operation action deviation information.

[0068] In an alternative embodiment of the present application, the first generation unit may include: A first determination subunit, configured to determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information; A second determination subunit, configured to determine whether the operation action deviation value corresponding to the operation action deviation information belongs to a preset operation action deviation range when the operation position deviation value corresponding to the operation position deviation information belongs to the preset operation position deviation range; A third determination subunit, configured to determine the normal execution operation result as the control result when the operation action deviation value belongs to the preset operation action deviation range; A fourth determination subunit, configured to determine the abnormal execution operation result as the control result when the operation action deviation value does not belong to the preset operation action deviation range.

[0069] In an alternative embodiment of the present application, the acquisition module 210 may include: An identification recognition unit, configured to perform identification recognition on the kit acquisition image to obtain a kit acquisition identification; A third determination unit, configured to determine the preset identification matching the kit acquisition identification as the kit identification.

[0070] In an alternative embodiment of the present application, the experimental robot control device may further include: An acquisition record module, configured to acquire the recorded position operation information, recorded action operation information, and timing information corresponding to the experimental robot; An operation record module, configured to perform operation recording according to the recorded position operation information, recorded action operation information, and timing information to obtain preset teaching operation information; A matching module is established to establish a matching relationship between the preset teaching operation information and the preset operation identifier, and generate the kit identifier and the target teaching operation information.

[0071] For the specific implementation process of the functions and roles of each module in the above device, please refer to the implementation process of the corresponding steps in the above method, which will not be elaborated here.

[0072] As Figure 3 shown, an embodiment of the present application provides an electronic device, including a processor 310, a communication interface 320, a memory 330, and a communication bus 340. Among them, the processor 310, the communication interface 320, and the memory 330 complete mutual communication through the communication bus 340; The memory 330 is used to store computer programs; In an embodiment of the present application, when the processor 310 executes the program stored on the memory 330, it implements the experimental robot control method provided by any one of the foregoing method embodiments. By acquiring an image of a kit corresponding to the experimental robot, the kit acquisition image is an image containing an experimental kit in the target area corresponding to the experimental robot, so as to determine the kit identifier corresponding to the experimental robot based on the kit acquisition image, and determine the target teaching operation information corresponding to the kit identifier. Subsequently, robot control information is generated through the target teaching operation information. The robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information; thus, the kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and then there is no need for experimental personnel to manually execute the relevant steps, that is, the problem of low experimental efficiency in the existing related technologies where experiments are carried out by experimental personnel is solved, and the experimental efficiency can be effectively improved.

[0073] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the experimental robot control method provided by any one of the foregoing method embodiments. By acquiring an image of a kit corresponding to the experimental robot, the kit acquisition image is an image containing an experimental kit in the target area corresponding to the experimental robot, so as to determine the kit identifier corresponding to the experimental robot based on the kit acquisition image, and determine the target teaching operation information corresponding to the kit identifier. Subsequently, robot control information is generated through the target teaching operation information. The robot control information is used to control the experimental robot to execute the teaching operation corresponding to the target teaching operation information; thus, the kit identifier and the target teaching operation information can be used to control the robot to complete its corresponding experimental steps, and then there is no need for experimental personnel to manually execute the relevant steps, that is, the problem of low experimental efficiency in the existing related technologies where experiments are carried out by experimental personnel is solved, and the experimental efficiency can be effectively improved.

[0074] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0075] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0076] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An experimental robot control method, characterized in that: include: Acquire a reagent kit acquisition image corresponding to the experimental robot, wherein the reagent kit acquisition image is an image containing the experimental reagent kit in a target area corresponding to the experimental robot; Determine the reagent kit identification corresponding to the experimental robot according to the reagent kit acquisition image; Determine the target teaching operation information corresponding to the reagent kit identification; The robot control information is generated through the target teaching operation information, and the robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information.

2. The experimental robot control method according to claim 1, characterized in that: After generating the robot control information through the target teaching operation information, the method further includes: Acquiring monitoring image information corresponding to the experimental robot; Performing operation analysis on the monitoring image information to obtain execution operation information; The execution operation information is matched with the target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot.

3. The experimental robot control method according to claim 2, characterized in that: The step of matching the execution operation information with the target operation information corresponding to the target teaching operation information to generate a control result corresponding to the experimental robot includes: Determine an execution operation identifier corresponding to the execution operation information; Acquire at least one preset operation information corresponding to the target teaching operation information, and determine the target operation identifier corresponding to the preset operation information; In a case where the execution operation identifier matches the target operation identifier, determining the preset operation information corresponding to the target operation identifier as the target operation information; Performing an operation deviation analysis based on the execution operation information and the target operation information to obtain operation deviation information; Based on the operation deviation information, a control result corresponding to the experimental robot is generated.

4. The experimental robot control method according to claim 3, characterized in that: The performing operation deviation analysis based on the execution operation information and the target operation information to obtain the operation deviation information includes: Performing an execution operation analysis on the execution operation information to obtain execution operation position information and execution operation action information; Performing target operation analysis on the target operation information to obtain target operation position information and target operation action information; Performing an operation position deviation analysis based on the execution operation position information and the target operation position information to obtain operation position deviation information; Performing an operation action deviation analysis based on the execution operation action information and the target operation action information to obtain operation action deviation information; The operation deviation information is generated according to the operation position deviation information and the operation action deviation information.

5. The experimental robot control method according to claim 4, characterized in that: Generating a control result corresponding to the experimental robot according to the operation deviation information includes: Determine the operation position deviation information and the operation action deviation information corresponding to the operation deviation information; In a case where the operation position deviation value corresponding to the operation position deviation information is within a preset operation position deviation range, determining whether the operation action deviation value corresponding to the operation action deviation information is within a preset operation action deviation range; When the operation action deviation value falls within the preset operation action deviation range, determining a normal execution operation result as the control result; When the operation action deviation value does not fall within the preset operation action deviation range, the abnormal execution operation result is determined as the control result.

6. The experimental robot control method according to claim 1, characterized in that: The step of collecting the image of the reagent kit and determining the reagent kit identifier corresponding to the experimental robot comprises: Performing identification on the kit collection image to obtain a kit collection identification; A preset identifier that matches the reagent kit collection identifier is determined as the reagent kit identifier.

7. The experimental robot control method according to claim 1, characterized in that: Also includes: Obtaining recorded position operation information, recorded action operation information and timing information corresponding to the experimental robot; Perform operation recording according to the recorded position operation information, recorded action operation information and timing information to obtain preset teaching operation information; A matching relationship is established between the preset teaching operation information and the preset operation identifier to generate the reagent kit identifier and the target teaching operation information.

8. An experimental robot control device, characterized in that: include: An acquisition module, used to acquire a reagent kit acquisition image corresponding to the experimental robot, wherein the reagent kit acquisition image is an image containing the experimental reagent kit in a target area corresponding to the experimental robot; A first determination module is used to determine the reagent kit identifier corresponding to the experimental robot according to the reagent kit acquisition image; A second determination module is used to determine the target teaching operation information corresponding to the reagent kit identification; The control module is used to generate robot control information through the target teaching operation information, and the robot control information is used to control the experimental robot to perform the teaching operation corresponding to the target teaching operation information.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the experimental robot control method described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the experimental robot control method according to any one of claims 1 to 7 is implemented.

Images