Testing method of robot system, electronic equipment and medium

Abstract

The invention provides a training sample determination method and electronic equipment, which are applied to the technical field of computers, and the test method of a robot system comprises the following steps: identifying a plurality of objects on a workbench; obtaining a plurality of tasks; according to the multiple objects and the multiple tasks, at least one target instruction is determined, and the target instruction indicates execution of the corresponding tasks for the objects; controlling the robot to execute at least one target instruction; according to the method and device for testing the robot system, automatic testing of the robot system can be achieved, and the testing efficiency of the robot system is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of testing technology, and in particular to a testing method, electronic equipment and medium for a robot system. Background Art

[0002] At present, the performance of robot systems used in grasping scenarios needs to be tested manually. For example, objects need to be placed manually and control instructions need to be input manually. After the robot system executes the control instructions, it is necessary to manually check whether the robot has successfully executed the instructions.

[0003] Among them, the manual testing robot system has the problem of low testing efficiency. Summary of the invention

[0004] Various aspects of the present disclosure provide a testing method, electronic device, and medium for a robot system to solve the technical problem of low testing efficiency in manual testing of a robot system.

[0005] A first aspect of an embodiment of the present disclosure provides a testing method for a robot system, the robot system comprising: a robot and a workbench, the testing method comprising cyclically executing the following testing steps:

[0006] Identify multiple objects on the workbench;

[0007] Get multiple tasks;

[0008] Determine at least one target instruction according to the plurality of objects and the plurality of tasks, the target instruction instructing to execute a corresponding task for the object;

[0009] Controlling the robot to execute at least one target instruction;

[0010] Determine the execution result of each target instruction executed by the robot.

[0011] A second aspect of the present disclosure provides a testing device for a robot system, the testing device comprising:

[0012] A recognition module for recognizing multiple objects on a workbench;

[0013] Acquisition module, used to acquire multiple tasks;

[0014] A first determination module is used to determine at least one target instruction according to the plurality of objects and the plurality of tasks, wherein the target instruction indicates to execute a corresponding task for the object;

[0015] A control module, used to control the robot to execute at least one target instruction;

[0016] The second determination module is used to determine the execution result of each target instruction executed by the robot.

[0017] A third aspect of an embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the testing method of the robot system of the first aspect when executing the computer program.

[0018] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the testing method of the robot system of the first aspect.

[0019] A fifth aspect of an embodiment of the present disclosure provides a computer program product, the program product comprising: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the testing method of the robot system of the first aspect.

[0020] The disclosed embodiments are applied in a test scenario of a robot system by identifying multiple objects on a workbench; acquiring multiple tasks; determining at least one target instruction based on the multiple objects and the multiple tasks, wherein the target instruction indicates that a corresponding task is to be performed on the object; controlling the robot to execute the at least one target instruction; and determining the execution result of each target instruction executed by the robot, thereby achieving automatic testing of the robot system and improving the testing efficiency of the robot system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0022] Figure 1 An application scenario diagram of a testing method for a robot system provided by an exemplary embodiment of the present disclosure;

[0023] Figure 2 A flowchart of a method for testing a robot system provided by an exemplary embodiment of the present disclosure;

[0024] Figure 3 A flowchart of another method for testing a robot system provided by an exemplary embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of a testing device for a robot system provided by an exemplary embodiment of the present disclosure;

[0026] Figure 5 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure will be clearly and completely described below in combination with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0028] In order to ensure the stable operation of the robot system, the stability test of the robot system is an important link. Due to the complexity of the robot system, in related technologies, manual testing of the robot system is usually carried out. Among them, manual testing includes: manual placement of objects, manual input of instructions, manual inspection of success or failure, manual recording of execution status, manual statistical data, etc. Manual testing consumes a lot of manpower and it is difficult to ensure continuous testing. In addition, manual instructions are limited by people's preference bias and mental state, and it is difficult to ensure that the instructions of the test process are fully covered. In summary, manual testing has the problems of low test efficiency, low test accuracy, and incomplete coverage of test instructions.

[0029] Based on the above problems, the present disclosure provides a testing method for a robot system, which can realize automatic placement of objects on a workbench, automatic input of instructions, automatic judgment and recording of execution results, thereby improving the testing efficiency, test accuracy and instruction coverage of the robot system.

[0030] In addition, an application scenario of the embodiment of the present disclosure is as follows Figure 1 , including a robot system 10, which includes a robot 11, a workbench 12 and a shuffling device 13, wherein the shuffling device 13 carries multiple objects, the shuffling device 13 shuffles the multiple objects and pours them on the workbench 12, and then controls the robot 11 to grab the objects in the workbench 12.

[0031] in, Figure 1 This is just an exemplary application scenario, and the embodiment of the present disclosure can be applied in any test scenario of a robot system. The embodiment of the present disclosure does not limit the specific application scenario.

[0032] Figure 2 A flowchart of a testing method for a robot system provided by an exemplary embodiment of the present disclosure, wherein the robot system includes: a robot and a workbench, and specifically includes the following steps:

[0033] S201, identifying multiple objects on a workbench.

[0034] In one embodiment, the robot system further includes: a camera, which can be controlled to capture images of multiple objects on the workbench to obtain a target image, and then identify the objects in the target image to obtain multiple objects on the workbench. Figure 1 The multiple objects on the workbench are object a1, object a2, object a3, object a4, object a5, object a6, object a7 and object a8. The present disclosure recognizes object a1 as an apple, object a2 as a carrot, object a3 as a battery, object a4 as a building block, object a5 as a carrot, object a6 as an apple, object a7 as a building block, and object a8 as a battery.

[0035] In addition, the present disclosure may also identify multiple objects on a workbench in other ways, which are not limited herein.

[0036] In the present disclosure, the multiple objects on the workbench can be placed according to the application scenario of the robot. For example, if the robot is used to sort fruits, the multiple objects can be different fruits. If the robot is used to sort daily necessities, the multiple objects can be different daily necessities. If the robot is used to sort various objects, the multiple objects can be different objects. The present disclosure does not limit the categories of the multiple objects.

[0037] S202, obtaining multiple tasks.

[0038] In the embodiment of the present disclosure, multiple tasks may be preset, and during the execution process, the preset multiple tasks may be directly acquired.

[0039] Specifically, there can be multiple categories of tasks that the robot can perform, such as "move to container", "move to position", "move to other objects", "build a fixed shape", "take objects out of container", and specific tasks can be pre-set based on these categories of tasks, such as tasks such as "move to basket", "move to the left", "move to the side of the box", "build a house" or "take objects out of the workbench".

[0040] In addition, multiple tasks can be preset according to the execution capability of the robot system. For example, if the robot system has the ability to move objects, the multiple tasks include "move to the basket", "move to the left" and "move next to the banana". If the robot system has the ability to build a fixed shape, the multiple tasks include "build a house", and if the robot has the ability to take objects out of a container, the multiple tasks include "take objects out of the workbench".

[0041] S203: Determine at least one target instruction according to the multiple objects and the multiple tasks.

[0042] The target instruction indicates to execute a corresponding task on the object.

[0043] In the present disclosure, target instructions are generated according to multiple objects and multiple tasks. One approach is to randomly generate target instructions according to multiple objects and multiple tasks, thereby improving the test coverage of the robot system.

[0044] For example, refer to Figure 1 There are 8 objects, and their corresponding names are apple, battery, building block, and carrot. Multiple tasks include: "move to the basket", "move to the left", "move to the side of the box", "build a house", or "take an object from the workbench". Multiple objects and multiple tasks can generate multiple instructions in Table 1. The target instruction in the present disclosure is at least one of the multiple instructions in Table 1.

[0045] Table 1

[0046]

[0047] In the disclosed embodiment, a corresponding target instruction may be randomly executed for each object. If different objects are the same object, the target instructions corresponding to the different objects indicate different tasks.

[0048] For example, refer to Figure 1 The target instruction corresponding to object a1 (apple) is “pick up the apple and put it in the basket”, the target instruction corresponding to object a2 (carrot) is “move the carrot to the left”, the target instruction corresponding to object a3 (battery) is “move the battery next to the box”, the target instruction corresponding to object a4 (building block) is “build a house with building blocks”, the target instruction corresponding to object a5 (carrot) is “pick up the carrot”, the target instruction corresponding to object a6 (apple) is “move the apple next to the box”, the target instruction corresponding to object a7 (building block) is “move the building block a little to the left”, and the target instruction corresponding to object a8 (battery) is “pick up the battery”.

[0049] S204, controlling the robot to execute at least one target instruction.

[0050] In the present disclosure, the robot executes each target instruction of at least one target instruction in sequence.

[0051] For example, execute target instruction b1 "grab the apples and put them in the basket" to grab the apples on the workbench and place them in the basket; execute target instruction b2 "move the carrots to the left" to grab the carrots on the workbench and place them next to the box; execute target instruction b3 "move the batteries to the box" to grab the carrots on the workbench and place them next to the box; execute target instruction b4 "build a house with building blocks" to grab the building blocks on the workbench to build a house; execute target instruction b5 "grab the carrots" to grab the carrots on the workbench; execute target instruction b6 "move the apples next to the box" to grab the apples on the workbench and place them next to the box; execute target instruction b7 "move the building blocks a little to the left" to move the building blocks on the workbench to the left; execute target instruction b8 "grab the batteries" to grab the batteries on the workbench.

[0052] In the present disclosure, there is no limitation on how the robot executes the target instruction.

[0053] S205, determining the execution result of each target instruction executed by the robot.

[0054] The execution result includes: execution failure or execution success. In the present disclosure, after the robot finishes executing one of the target instructions, the camera can be controlled to capture images, and then the images are analyzed to determine whether the target instruction is successfully executed. For example, after the robot executes the target instruction b1 "grab the apple and put it in the basket", the camera is controlled to capture images of the workbench and the basket, and then the images are identified to determine whether the apple is successfully grabbed into the basket.

[0055] In addition, the present disclosure also includes recording the test results of each target instruction. Specifically, the test results include: the mapping relationship between the object, the task, the target instruction, and the execution result. For example, the test results recorded for the target instructions b1 to b8 refer to Table 2:

[0056] Table 2

[0057]

[0058]

[0059] Furthermore, the test results may also record data such as the time and date of executing the target instruction, which is not limited here.

[0060] In addition, the present disclosure executes S201 to S205 in a loop to improve the coverage of different instructions executed by the test robot. The testing method of the present disclosure also includes: obtaining the execution count of the test step; if the execution count is a preset total count, stopping the execution of the test step.

[0061] It can be understood that the present disclosure can preset the total number of executions of the test steps. If the number of executions reaches the preset total number, the test is stopped, and the performance of the robot system can be analyzed based on the recorded test results.

[0062] In summary, the present invention can realize automatic identification of objects on the workbench, automatically generate target instructions to control the robot to execute the target instructions, and can automatically determine the execution results of the target instructions. The testing process of the robot system of the present invention does not require human participation, thereby improving the testing efficiency of the robot system.

[0063] Figure 3 The present invention provides a flowchart of a method for testing a robot system according to an exemplary embodiment of the present invention. The robot system includes a robot and a workbench. The method includes cyclically executing the following test steps:

[0064] S301, if the preset shuffling interval is reached, control the shuffling device to shuffle multiple objects.

[0065] The robot system further comprises a shuffling device, which is used to shuffle multiple objects in the shuffling device. It can be understood that shuffling is a process of evenly dispersing multiple objects.

[0066] In the initial stage, the shuffling device is first controlled to shuffle multiple objects, and then S302 is executed. After executing S302, S303 to S311 are executed. After executing S311, it is determined whether the shuffling interval is reached. If the shuffling interval is reached, S301 is executed. If the shuffling interval is not reached, S303 to S311 are executed again.

[0067] In the present disclosure, the shuffle interval is pre-set. If the current shuffle interval (the time interval between the current time and the last shuffle time) is equal to or greater than the preset shuffle interval, it is determined that the preset shuffle interval is reached. If the current shuffle interval is less than the preset shuffle interval, it is determined that the preset shuffle interval is not reached.

[0068] S302, controlling the shuffling device to pour the shuffled objects into the workbench.

[0069] The shuffling device is controlled to pour the shuffled objects onto the workbench surface for subsequent testing of the robot system.

[0070] S303, identifying multiple objects on the workbench.

[0071] The specific implementation process of this step refers to S201 and will not be repeated here.

[0072] S304, obtaining multiple tasks.

[0073] The specific implementation process of this step refers to S202 and will not be repeated here.

[0074] S305: Determine at least one target object among the multiple objects.

[0075] Among them, determining at least one target object among multiple objects includes: obtaining a first familiarity of each object among the multiple objects, wherein the first familiarity of each object represents a first success rate of the robot in historically executing instructions corresponding to the object, and the first familiarity is positively correlated with the first success rate; and determining at least one target object among the multiple objects based on the first familiarity.

[0076] It can be understood that the first success rate is the ratio of the number of successful executions of instructions on the object to the total number of times in history, wherein the higher the first success rate, the greater the first familiarity.

[0077] In the disclosed embodiment, the first familiarity of objects of the same object is the same, for example, Figure 1 In the equation, object a1 and object a6 are both apples, so the first familiarity of object a1 and object a6 is the same.

[0078] In addition, the initial first familiarity of each object is the same, and the first familiarity of the object can be updated later according to whether the instruction executed on the object is successful. For example, the initial first familiarity of each object is 1.

[0079] In the disclosed embodiment, the current first familiarity of each object in the multiple objects is obtained. If the current execution is the first time, the current first familiarity is 1. If the current execution is not the first time, the current first familiarity can be other values. For example, the first familiarity of apples is 1.5, the first familiarity of carrots is 1.5, the first familiarity of batteries is 2 / 3, and the first familiarity of building blocks is 2 / 3.

[0080] It can be understood that if the first familiarity of all objects is the same, such as 1, then it can be determined that all objects are target objects, or some objects are randomly selected as target objects.

[0081] Furthermore, based on the first familiarity, at least one target object is determined among multiple objects, including: determining a first total value of the first familiarities of the multiple objects; for each object among the multiple objects, determining a first ratio of the first familiarity of the object to the first total value; randomly generating a first threshold; if the first ratio is less than the first threshold, determining that the object is a target object.

[0082] Among them, refer to Figure 1 , if there are 8 objects, the corresponding first familiarity, first total value and first ratio refer to Table 3:

[0083] Table 3

[0084]

[0085] Referring to Table 3, it can be understood that the first familiarity and the first ratio of the objects of the same object are the same. Further, the first threshold is randomly generated and is a value between 0 and 1. For example, the first threshold is 5 / 52, and the target objects include: object a3, object a4, object a7 and object a8.

[0086] In the disclosed embodiment, the first familiarity threshold may be determined randomly, and an object whose first familiarity is less than the first familiarity threshold is determined as the target object.

[0087] In the embodiments of the present disclosure, the target object may also be determined in other ways, such as randomly selecting at least one object from multiple objects as the target object, or selecting multiple objects with the first familiarity and sorting them in ascending order, and the top N objects in the sorting are the target objects, where N is an integer greater than or equal to 1. The present disclosure does not limit the specific way of determining the target object.

[0088] In summary, the above-mentioned method of selecting target objects disclosed in the present invention can make it possible to perform multiple tests on objects that failed to be executed by the robot during historical testing, so as to more efficiently test the performance of the robot in processing the object.

[0089] S306: Determine at least one target task among the multiple tasks.

[0090] Among them, determining at least one target task among multiple tasks includes: obtaining the second familiarity of each task among the multiple tasks, the second familiarity of each task represents the second success rate of the robot in history for executing instructions corresponding to the task, and the second familiarity is positively correlated with the second success rate; according to the second familiarity, determining at least one target task among the multiple tasks.

[0091] It can be understood that the first success rate is the ratio of the number of successful execution instructions for the task to the total number of successes in history, wherein the higher the second success rate, the greater the second familiarity.

[0092] In addition, the initial second familiarity of each task is the same, and the second familiarity of the task can be updated later according to whether the instruction for executing the task is successful. For example, the initial second familiarity of each task is 1.

[0093] In the disclosed embodiment, the current second familiarity of each task in the multiple tasks is obtained. If it is the first time to execute, the current second familiarity is 1. If it is not the first time to execute, the current second familiarity can be other values. For example, the second familiarity of "move to the basket" is 1.5, the second familiarity of "move to the left" is 1.5, the second familiarity of "move to the side of the box" is 2 / 3, the second familiarity of "build a house" is 2 / 3, and the second familiarity of "take an object from the workbench" is 1.5.

[0094] If the first familiarity of all tasks is the same, such as 1, then it can be determined that all tasks are target tasks, or some tasks are randomly selected as target tasks.

[0095] Furthermore, based on the second familiarity, at least one target task is determined among the multiple tasks, including: determining a second total value of the second familiarity of the multiple tasks; for each task in the multiple tasks, determining a second ratio of the second familiarity of the task to the second total value; randomly generating a second threshold; if the second ratio is less than the second threshold, determining the task to be the target task.

[0096] The second familiarity, second total value and second ratio corresponding to different tasks refer to Table 4:

[0097] Table 4

[0098]

[0099]

[0100] In the present disclosure, the second threshold is randomly generated and is a value between 0 and 1. For example, if the second threshold is 5 / 35, the target tasks include: "move to the side of the box" and "build a house".

[0101] In the disclosed embodiment, the second familiarity threshold may be determined randomly, and a task whose second familiarity is less than the second familiarity threshold is determined as the target task.

[0102] In the embodiments of the present disclosure, the target task may also be determined in other ways, such as randomly selecting at least one task from multiple tasks as the target task, or selecting multiple tasks from the second familiarity level and sorting them in ascending order, and the top M tasks in the sorting are the target tasks, where M is an integer greater than or equal to 1. The present disclosure does not limit the specific way of determining the target task.

[0103] In summary, the above-mentioned method of selecting target tasks disclosed in the present invention can make the tasks that failed to be executed by the robot in the historical testing process be tested multiple times, so as to more efficiently test the performance of the robot in processing the task.

[0104] S307: Determine at least one target instruction according to at least one target object and at least one target task.

[0105] In one embodiment, a target instruction may be determined for each target object and each target task. Figure 1 , the target objects include: object a3, object a4, object a7 and object a8, and the corresponding objects are batteries and building blocks. The target tasks include: "move to the side of the box" and "build a house", and the target instructions obtained are shown in Table 5:

[0106] Table 5

[0107]

[0108] Among them, determining at least one target instruction based on at least one target object and at least one target task includes: obtaining a preset prompt word; inputting at least one target object, at least one target task and the preset prompt word into a pre-trained language model for processing to obtain at least one target instruction.

[0109] In the disclosed embodiment, the language model is used for target instructions, which are in natural language format and are used to control the robot. For example, the preset prompt is "Please generate a target instruction executable by the robot based on at least one target object and at least one target task", and the input language model is "At least one target object is a battery and a building block, and at least one target task is to move to the side of the box and build a house. Please generate a target instruction executable by the robot based on at least one target object and at least one target task.

[0110] In the embodiment of the present disclosure, the target instruction indicates to execute the target task for the target object. The present disclosure can also generate at least one target instruction according to other methods, which are not limited here.

[0111] S308, controlling the robot to execute at least one target instruction.

[0112] The specific implementation process of this step refers to S204 and will not be repeated here.

[0113] S309, determining the execution result of each target instruction executed by the robot.

[0114] The specific implementation process of this step refers to S205 and will not be repeated here.

[0115] S310: If the execution result indicates that the corresponding target instruction is executed successfully, then the first familiarity of the target object corresponding to the target instruction is increased and / or the second familiarity of the target task corresponding to the target instruction is increased.

[0116] In the present disclosure, a first coefficient greater than 1 (such as 1.5) is set. If the target instruction is executed successfully, the first coefficient is multiplied by the current first familiarity of the object corresponding to the target instruction to obtain the new first familiarity, and the first coefficient is multiplied by the current second familiarity of the task corresponding to the target instruction to obtain the new second familiarity.

[0117] For example, refer to Figure 5 , if the current first familiarity of "battery" is 2 / 3, the current first familiarity of "building block" is 2 / 3, the current second familiarity of "move to the side of the box" is 2 / 3, and the current second familiarity of "build a house" is 2 / 3, and the target instruction "move the battery to the side of the box" is executed successfully, then the current first familiarity of the corresponding object "battery" multiplied by the first coefficient is 1, and the current second familiarity of the corresponding task "move to the side of the box" multiplied by the first coefficient is 1.

[0118] S311 , if the execution result indicates that the corresponding target instruction fails to execute, then reducing the first familiarity of the target object corresponding to the target instruction and / or reducing the second familiarity of the target task corresponding to the target instruction.

[0119] In the present disclosure, if the target instruction fails to execute, the current first familiarity of the object corresponding to the target instruction is divided by the first coefficient as the new first familiarity, and the current second familiarity of the task corresponding to the target instruction is divided by the first coefficient as the new second familiarity.

[0120] For example, refer to Figure 5 , if the current first familiarity of "battery" is 2 / 3, the current first familiarity of "building block" is 2 / 3, the current second familiarity of "move to the side of the box" is 2 / 3, and the current second familiarity of "build a house" is 2 / 3, and the target instruction "move the battery to the side of the box" is executed successfully, then the current first familiarity of the corresponding object "battery" multiplied by the first coefficient is 1, and the current second familiarity of the corresponding task "move to the side of the box" multiplied by the first coefficient is 1.

[0121] At this time, the current first familiarity of "battery" is 1, the current first familiarity of "building block" is 2 / 3, the current second familiarity of "move to the side of the box" is 1, and the current second familiarity of "build a house" is 2 / 3. The execution of the target instruction "put the building block next to the box" fails, then the corresponding object "building block"'s current first familiarity (2 / 3) divided by the first coefficient (1.5) is 4 / 9, and the corresponding task "move to the side of the box"'s second familiarity (1) divided by the first coefficient (1.5) is 2 / 3.

[0122] Furthermore, the current first familiarity of "battery" is 1, the current first familiarity of "building block" is 4 / 9, the current second familiarity of "move to the side of the box" is 2 / 3, and the current second familiarity of "build a house" is 2 / 3. Then the execution of the target instruction "build a house with building blocks" fails, then the corresponding object "building block"'s current first familiarity (4 / 9) is divided by the first coefficient (1.5) to get 8 / 27, and the corresponding task "build a house"'s second familiarity (2 / 3) is divided by the first coefficient (1.5) to get 4 / 9.

[0123] In summary, the objects and tasks with lower execution success rates have lower corresponding familiarity. More tests should be conducted on objects and tasks with lower familiarity, thereby improving the test accuracy of the robot system and accurately testing objects and tasks that the robot cannot handle correctly.

[0124] The present invention can realize automatic control of the shuffling device to shuffle objects, the automatic control of the shuffling device pours the objects into the workbench, automatically identifies the objects on the workbench, automatically generates target instructions to control the robot to execute the target instructions, and can automatically determine the execution results of the target instructions. The test process of the robot system of the present invention does not require human participation, thereby improving the test efficiency of the robot system. In addition, determining the target object and target task based on familiarity can ensure the coverage of the test. The use of language models to generate target instructions ensures the diversity of target instructions and the naturalness of sentences. The recording of test results can ensure result analysis.

[0125] In addition, after adding new tasks and new objects, the present invention can re-execute the above steps to realize automatic testing of the robot system, which is simple and easy to expand.

[0126] Reference Figure 4 , is a structural block diagram of a test device 40 for a robot system provided by the present disclosure, and the test device 40 for the robot system includes:

[0127] A recognition module 41, used for recognizing multiple objects on the workbench;

[0128] An acquisition module 42, used for acquiring multiple tasks;

[0129] A first determination module 43 is used to determine at least one target instruction according to the multiple objects and the multiple tasks, where the target instruction indicates to execute a corresponding task for the object;

[0130] A control module 44, used to control the robot to execute at least one target instruction;

[0131] The second determination module 45 is used to determine the execution result of each target instruction executed by the robot.

[0132] In an optional embodiment, the first determination module 43 is specifically configured to determine at least one target object among the multiple objects;

[0133] determining at least one target task among the multiple tasks;

[0134] At least one target instruction is determined according to at least one target object and at least one target task.

[0135] In an optional embodiment, when the first determining module 43 determines at least one target object from a plurality of objects, it is specifically configured to:

[0136] Obtaining a first familiarity of each object from the plurality of objects, wherein the first familiarity of each object represents a first success rate of the robot in historically executing instructions corresponding to the object, and the first familiarity is positively correlated with the first success rate;

[0137] At least one target object is determined from among the plurality of objects according to the first familiarity.

[0138] In an optional embodiment, the first determination module 43, when determining at least one target object from a plurality of objects according to the first familiarity, is specifically configured to:

[0139] determining a first total value of first familiarity of the plurality of objects;

[0140] For each object in the plurality of objects, determining a first ratio of a first familiarity of the object to a first total value;

[0141] Randomly generate a first threshold;

[0142] If the first ratio is less than the first threshold, the object is determined to be a target object.

[0143] In an optional embodiment, when the first determining module 43 determines at least one target task from a plurality of tasks, it is specifically configured to:

[0144] Obtaining a second familiarity of each task in the plurality of tasks, wherein the second familiarity of each task represents a second success rate of the robot in the history of executing instructions corresponding to the task, and the second familiarity is positively correlated with the second success rate;

[0145] At least one target task is determined from among the plurality of tasks according to the second familiarity.

[0146] In an optional embodiment, when the first determination module 43 determines at least one target task from a plurality of tasks according to the second familiarity, it is specifically configured to:

[0147] determining a second total value of the second familiarity of the plurality of tasks;

[0148] For each task in the plurality of tasks, determining a second ratio of a second familiarity of the task to a second total value;

[0149] randomly generating a second threshold;

[0150] If the second ratio is less than the second threshold, the task is determined to be a target task.

[0151] In an optional embodiment, an update module (not shown) is further included, which is used to determine the execution result of the robot executing each target instruction. If the execution result indicates that the corresponding target instruction is executed successfully, the first familiarity of the target object corresponding to the target instruction and / or the second familiarity of the target task corresponding to the target instruction are increased; if the execution result indicates that the corresponding target instruction fails to execute, the first familiarity of the target object corresponding to the target instruction and / or the second familiarity of the target task corresponding to the target instruction are reduced.

[0152] In an optional embodiment, when the first determination module 43 determines at least one target instruction according to at least one target object and at least one target task, it is specifically configured to:

[0153] Get the preset prompt words;

[0154] At least one target object, at least one target task and a preset prompt word are input into a pre-trained language model for processing to obtain at least one target instruction.

[0155] In an optional embodiment, the robot system further includes a shuffling device, and before identifying the multiple objects on the workbench, the method further includes:

[0156] If the preset shuffling interval is reached, controlling the shuffling device to shuffle the multiple objects;

[0157] The shuffling device is controlled to pour the shuffled multiple objects into the workbench.

[0158] In an optional embodiment, the device further includes:

[0159] An acquisition module (not shown), used to acquire the execution times of the test steps;

[0160] The stop module (not shown) is used to stop executing the test step if the execution times reach the preset total times.

[0161] The testing device for the robot system provided in the present disclosure can implement the testing method for the robot system described above. Please refer to the above for details and will not be repeated here.

[0162] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and are only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "second", "first", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit "second" and "first" to be different types.

[0163] Figure 5 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 5 As shown, the electronic device 50 includes: a processor 51, and a memory 52 communicatively connected to the processor 51, and the memory 52 stores computer-executable instructions.

[0164] Among them, the processor executes the computer execution instructions stored in the memory to implement the testing method of the robot system provided by any of the above method embodiments, and the specific functions and technical effects that can be achieved are not repeated here.

[0165] The embodiment of the present disclosure further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any of the above methods.

[0166] The embodiments of the present disclosure also provide a computer program product, which includes: a computer program, which is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes any of the above methods.

[0167] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0168] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0170] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform some steps of the methods of various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0171] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0172] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0173] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for testing a robot system, characterized in that: The robot system comprises: a robot and a workbench, and the testing method comprises cyclically executing the following testing steps: identifying a plurality of objects on the workbench; Get multiple tasks; Determine at least one target instruction according to the plurality of objects and the plurality of tasks, the target instruction instructing to execute a corresponding task for the object; Controlling the robot to execute the at least one target instruction; Determine the execution result of each target instruction executed by the robot.

2. The testing method according to claim 1, characterized in that: The step of determining at least one target instruction according to the plurality of objects and the plurality of tasks comprises: determining at least one target object among the plurality of objects; determining at least one target task among the plurality of tasks; The at least one target instruction is determined according to the at least one target object and the at least one target task.

3. The testing method according to claim 2, characterized in that: The determining at least one target object among the multiple objects comprises: Acquire a first familiarity of each object in the plurality of objects, wherein the first familiarity of each object represents a first success rate of the robot in historically executing instructions corresponding to the object, and the first familiarity is positively correlated with the first success rate; At least one target object is determined from among the plurality of objects according to the first familiarity.

4. The testing method according to claim 3, characterized in that: Determining at least one target object from the plurality of objects according to the first familiarity includes: determining a first total value of first familiarities of the plurality of objects; For each object of the plurality of objects, determining a first ratio of a first familiarity of the object to the first total value; Randomly generate a first threshold; If the first ratio is less than the first threshold, the object is determined to be the target object.

5. The testing method according to claim 4, characterized in that: The step of determining at least one target task from among the plurality of tasks comprises: Acquire a second familiarity of each of the multiple tasks, where the second familiarity of each task represents a second success rate of the robot in historically executing instructions corresponding to the task, and the second familiarity is positively correlated with the second success rate; At least one target task is determined from among the plurality of tasks according to the second familiarity.

6. The testing method according to claim 5, characterized in that: Determining at least one target task from the plurality of tasks according to the second familiarity includes: determining a second total value of second familiarity of the plurality of tasks; determining, for each task in the plurality of tasks, a second ratio of a second familiarity of the task to the second total value; randomly generating a second threshold; If the second ratio is less than the second threshold, the task is determined to be the target task.

7. The testing method according to claim 6, characterized in that: After determining the execution result of each target instruction executed by the robot, the method further includes: If the execution result indicates that the corresponding target instruction is successfully executed, then increasing the first familiarity of the target object corresponding to the target instruction and / or increasing the second familiarity of the target task corresponding to the target instruction; If the execution result indicates that the corresponding target instruction fails to execute, then the first familiarity of the target object corresponding to the target instruction is reduced and / or the second familiarity of the target task corresponding to the target instruction is reduced.

8. The testing method according to any one of claims 2 to 7, characterized in that: The determining, according to the at least one target object and the at least one target task, the at least one target instruction comprises: Get the preset prompt words; The at least one target object, the at least one target task and the preset prompt word are input into a pre-trained language model for processing to obtain the at least one target instruction.

9. The testing method according to any one of claims 1 to 7, characterized in that: The robot system further includes a shuffling device, and before identifying the plurality of objects on the workbench, the robot system further includes: If the preset shuffling interval is reached, controlling the shuffling device to shuffle the multiple objects; The shuffling device is controlled to pour the shuffled multiple objects into the workbench.

10. The testing method according to any one of claims 1 to 7, characterized in that: The method further comprises: Obtain the number of executions of the test step; If the execution times are the preset total times, the test step is stopped.

11. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the robot system testing method according to any one of claims 1 to 10 when executing the computer program.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the testing method for the robot system according to any one of claims 1 to 10.

Images