A robot-oriented ontology perception generation method and device

Abstract

The present disclosure relates to a robot-oriented ontology perception generation method and device. The ontology perception generation method comprises: obtaining first perception data; inputting the first perception data into a gating network in a hybrid expert network for calculation to obtain a matching result; determining at least one target expert network from a plurality of expert networks according to the matching result; inputting a current target position into each target expert network for calculation to obtain a current feasible result output by each target expert network for each target position in a first environment, and determining a voting result according to each current feasible result; and generating an ontology perception result according to the voting result corresponding to all target positions. The robot-oriented ontology perception generation method and device can support a robot to autonomously generate an ontology perception result in different environments, so that the robot can quickly and clearly determine the functional range of the robot in the environment, thereby safely and efficiently completing various tasks.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of robots, and in particular to a robot-oriented body perception generation method and device. BACKGROUND

[0002] Robots are a landmark tool in the digital economy era and are profoundly changing human production and lifestyle. A large number of robot field enterprises and innovation institutions are constantly exploring cutting-edge technologies, which has promoted the vigorous development of the robot industry. With the rapid development of new technologies in the robot field, robots are increasingly widely used. Currently, new generation intelligent technologies and robot technologies are rapidly converging, and as the intelligence level of robots improves, the functions carried by robots are becoming richer and the intelligence level is becoming higher. Highly complex autonomous systems such as driverless vehicles, intelligent service robots, and intelligent personal assistants are gradually moving from the laboratory to people's lives.

[0003] However, different environments will bring different constraints to the functional range of robots. For example, the center of gravity of a robot is different on flat ground and on a slope, and the motion space that ensures the safety of the robot is also different, so there are different functional ranges. Due to the complexity and dynamic characteristics of real scenes, it is very necessary for robots to autonomously generate functional ranges in different environments, so as to safely and efficiently complete various tasks. SUMMARY

[0004] Therefore, the present disclosure proposes a robot-oriented body perception generation method and device, which can support robots to autonomously generate body perception results in different environments, so that the robots can quickly and clearly determine the functional range of the robots in the environment, thereby safely and efficiently completing various tasks.

[0005] According to an aspect of the present disclosure, a robot-oriented body perception generation method is provided, which includes: obtaining first perception data, the first perception data being data obtained by a robot performing perception detection in a first environment; inputting the first perception data into a gating network in a hybrid expert network for calculation to obtain a matching result, wherein the hybrid expert network includes the gating network and a plurality of expert networks, and the matching result indicates a matching probability of each expert network and the first environment; determining at least one target expert network from the plurality of expert networks according to the matching result; for each target position in the first environment, inputting a current target position into each target expert network for calculation to obtain a current feasible result output by each target expert network, and determining a voting result according to each current feasible result, the voting result indicating the feasibility of an action of the robot at the current target position; and generating a body perception result according to the voting result corresponding to all target positions, the body perception result indicating a feasible motion range of the robot in the first environment.

[0006] In a possible implementation, the obtaining the first perception data comprises: obtaining a plurality of first actual perception data detected by the robot when performing the same action in the first environment; selecting at least one target perception data from the plurality of first actual perception data according to a predetermined first selection rule; and processing the at least one target perception data according to a predetermined processing rule to obtain the first perception data.

[0007] In a possible implementation, the determining the at least one target expert network from the plurality of expert networks according to the matching result comprises: taking an expert network corresponding to a maximum value in the matching probabilities as the target expert network; and the determining the voting result according to the current feasible result comprises: determining the voting result based on a current feasible result output by the target expert network and a predetermined threshold.

[0008] In a possible implementation, the determining the at least one target expert network from the plurality of expert networks according to the matching result comprises: taking all the expert networks as the target expert network; and the determining the voting result according to the current feasible result comprises: determining a weight of each expert network according to a matching probability of the expert network and the first environment; multiplying the weight corresponding to the current expert network and the current feasible result to obtain a product result corresponding to each expert network; and determining the voting result based on a sum of the product results corresponding to all the expert networks and a predetermined threshold.

[0009] In a possible implementation, the method further comprises a training step of the gating network, and the training step of the gating network comprises: obtaining a plurality of first data groups, each of the first data groups comprising a plurality of second perception data, each of the second perception data in a same first data group being data obtained by the robot performing perception detection in a same second environment, and each of the first data groups corresponding to a different second environment; constructing a gating network training set based on the plurality of first data groups, and performing gating network training by using the gating network training set.

[0010] In a possible implementation, the obtaining the plurality of first data groups comprises: obtaining a plurality of second actual perception data corresponding to each second environment, the second actual perception data being obtained by the robot performing perception detection when performing the same action in each second environment; selecting at least one second actual perception data from the plurality of second actual perception data of each second environment according to a predetermined second selection rule, and constructing the first data group corresponding to each second environment based on the at least one second actual perception data.

[0011] In one possible implementation, the method also includes a training step for the multiple expert networks, and the training step for the multiple expert networks includes: obtaining multiple second data groups, the number of the second data groups is the same as the number of the expert networks, each second data group includes multiple range data, each range data in the same second data group is data obtained by motion range detection of the robot's robotic arm in the same second environment, and each second data group corresponds to a different second environment; assigning a corresponding second data group to each expert network, and using each second data group to perform corresponding expert network training.

[0012] According to another aspect of the present disclosure, a proprioception generation device for a robot is provided, comprising: an acquisition module for acquiring first perception data, wherein the first perception data is data obtained by the robot performing perception detection in a first environment; a first calculation module for inputting the first perception data into a gating network in a hybrid expert network for calculation to obtain a matching result, wherein the hybrid expert network includes the gating network and multiple expert networks, and the matching result indicates the matching probability of each of the expert networks with the first environment; a determination module for determining at least one target expert network from the multiple expert networks based on the matching result; a second calculation module for inputting the current target position into each of the target expert networks for calculation for each target position in the first environment to obtain a current feasible result output by each of the target expert networks, and determining a voting result based on each current feasible result, wherein the voting result indicates the feasibility of the robot performing an action at the current target position; a generation module for generating a proprioception result based on the voting results corresponding to all target positions, wherein the proprioception result indicates the feasible motion range of the robot in the first environment.

[0013] In one possible implementation, obtaining the first perception data includes: obtaining multiple first actual perception data obtained by perception detection when the robot performs the same action in the first environment; selecting at least one target perception data from the multiple first actual perception data according to a predetermined first selection rule; and processing the at least one target perception data according to a predetermined processing rule to obtain the first perception data.

[0014] In one possible implementation, determining at least one target expert network from the multiple expert networks based on the matching results includes: taking the expert network corresponding to the maximum value among the matching probabilities as the target expert network; determining the voting result based on the current feasible results includes: determining the voting result based on the current feasible result output by the target expert network and a predetermined threshold.

[0015] In one possible implementation, determining at least one target expert network from the multiple expert networks based on the matching results includes: taking all expert networks as the target expert network; determining the voting result based on each current feasible result includes: determining the weight of each expert network based on the matching probability of each expert network with the first environment; for each expert network, multiplying the weight corresponding to the current expert network and the current feasible result to obtain the product result corresponding to each expert network; and determining the voting result based on the sum of the product results corresponding to all expert networks and a predetermined threshold.

[0016] In one possible implementation, the device also includes a first training module for executing the training steps of the gated network, and the training steps of the gated network include: obtaining multiple first data groups, each first data group includes multiple second perception data, each second perception data in the same first data group is data obtained by the robot performing perception detection in the same second environment, and each first data group corresponds to a different second environment; constructing a gated network training set based on the multiple first data groups, and using the gated network training set to train the gated network.

[0017] In one possible implementation, obtaining multiple first data groups includes: obtaining multiple second actual perception data corresponding to each second environment obtained by perception detection when the robot performs the same action in each second environment; selecting at least one second actual perception data from the multiple second actual perception data of each second environment according to a predetermined second selection rule, and constructing the first data group corresponding to each second environment based on the at least one second actual perception data.

[0018] In one possible implementation, the device also includes a second training module for executing the training steps of the multiple expert networks, and the training steps of the multiple expert networks include: obtaining multiple second data groups, the number of the second data groups is the same as the number of the expert networks, each second data group includes multiple range data, each range data in the same second data group is data obtained by motion range detection of the robot's robotic arm in the same second environment, and each second data group corresponds to a different second environment; assigning a corresponding second data group to each expert network, and using each second data group to perform corresponding expert network training.

[0019] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0020] According to another aspect of the present disclosure, a non-volatile computer readable storage medium is provided, having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method described above.

[0021] The method and device for generating robot-oriented body perception provided by the embodiments of the present disclosure obtain first perception data, input the first perception data into a gating network in a hybrid expert network to obtain a matching result, determine at least one target expert network from the plurality of expert networks according to the matching result, input a current target position into each target expert network to obtain a current feasible result output by each target expert network for each target position in the first environment, determine a voting result according to each current feasible result, generate a body perception result according to the voting result corresponding to all target positions, obtain the first perception data of the robot in a new environment, obtain a matching probability of each expert network through inference of the gating network, determine the target expert network according to the matching probability, and then infer the feasible working space of the robot based on the target expert network. In this way, the accurate feasible working space of the robot can be autonomously generated in the new environment, efficient body perception generation is achieved, the hybrid expert network does not need to be trained in advance using data of the new environment, and the general capability of the robot is greatly improved.

[0022] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, rather than restrictive of the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of the exemplary embodiments, with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0024] Figures 1 to 4 A schematic diagram of a method for generating robot-oriented body perception is shown.

[0025] Figures 5 and 6 A block diagram of a device for generating robot-oriented body perception is shown. DETAILED DESCRIPTION

[0026] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the drawings indicate functionally similar or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0027] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0028] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0029] In order to facilitate those skilled in the art to understand the technical solution provided by the embodiments of the present disclosure, the technical environment in which the technical solution is implemented is described below.

[0030] Existing robotic systems are often composed of multiple components, such as the robot, its attached arm, and various sensors. While individual components, such as the arm, may have precise simulation models, the resulting robotic system lacks pre-modeling, making it impossible to accurately determine the system's functional range. This range indicates where the robot can travel and what actions it can perform within feasible locations. Building accurate simulation models incurs significant development costs. Furthermore, robotic system simulation models are only applicable to ideal, static environments. Real-world environments can cause variations in the robot's position and center of gravity, making the functional range calculated in the simulation model inappropriate for real-world environments and resulting in a decrease in the quality of the robot's task completion.

[0031] In existing robotic systems, establishing proprioception has been widely discussed in various forms. This is because autonomous proprioception can avoid safety issues and potential risks associated with robotic motion, thereby preventing catastrophic accidents. However, a practical method for generating proprioception is currently lacking, and there is little discussion of universal proprioception generation methods for multiple robotic systems. Furthermore, the more complex the robot's configuration and the greater the number of degrees of freedom of motion, the more important proprioception capabilities become, and the more challenging the method for establishing proprioception becomes.

[0032] The disclosed embodiments provide a method for generating proprioception for a robot, comprising: obtaining first perception data, which is data obtained by the robot performing perception detection in a first environment; inputting the first perception data into a gated network in a hybrid expert network for calculation to obtain a matching result, wherein the hybrid expert network includes a gated network and multiple expert networks, and the matching result indicates the matching probability of each expert network with the first environment; determining at least one target expert network from the multiple expert networks based on the matching result; inputting the current target position into each target expert network for calculation for each target position, obtaining a current feasible result output by each target expert network, determining a voting result based on each current feasible result, and indicating the feasibility of the robot performing an action at the current target position; and generating a proprioception result based on the voting results corresponding to all target positions, wherein the proprioception result indicates the feasible range of motion of the robot in the first environment. In this way, the robot can autonomously generate proprioception results in different environments, allowing the robot to quickly determine its functional scope in the environment, thereby completing various tasks safely and efficiently.

[0033] Figures 1 to 4 A schematic diagram of a robot-oriented proprioception generation method according to an embodiment of the present disclosure is shown. Figures 1 to 4 The robot-oriented proprioception generation method provided in the embodiment of the present disclosure is schematically illustrated.

[0034] like Figure 1 As shown, the proprioception generation method may include the following steps S101 to S105.

[0035] S101. Obtain first perception data.

[0036] The first perception data is the data obtained by the robot through perception detection in the first environment, such as speed, center of gravity, joint position, etc. The shape of the robot can be set according to actual needs. Robots of different shapes can achieve autonomous proprioception generation based on this proprioception generation method. For robots of different shapes, data can be obtained in an environment corresponding to actual needs. The data may include perception data, range data, and other data that help the robot to autonomously establish proprioception. The first environment is generally a new environment that the robot has not reached. This new environment is relative to the second environment used to train the hybrid expert network. For details, see the process of training the hybrid expert network later, which will not be expanded here.

[0037] S102: Input the first perception data into the gated network in the hybrid expert network for calculation to obtain a matching result.

[0038] The Mixture of Experts (MoE) network consists of a gating network and multiple expert networks. These expert networks are used to accurately determine the robot's operating range within its environment. Based on the sensory data acquired from the new environment (the first environment), the gating network accurately determines the matching probability (or similarity) between the new environment and each expert network. The matching results indicate the matching probability of each expert network with the first environment.

[0039] S103: Determine at least one target expert network from the multiple expert networks according to the matching results.

[0040] The matching results include multiple matching probabilities. The higher the matching probability, the more closely the expert network corresponding to that matching probability matches the new environment. The expert network with the higher matching probability can then be used to determine the working range, allowing for accurate calculation of the robot's workspace in the new environment. The number of target expert networks can be one or more, depending on actual needs.

[0041] S104: For each target position in the first environment, input the current target position into each target expert network for calculation, obtain the current feasible result output by each target expert network, and determine the voting result according to each current feasible result.

[0042] The voting result indicates the feasibility of the robot's action at the target location. The target location is a feasible point in the new environment to be confirmed. In other words, if the voting result for a target location indicates "pass," the robot can perform the action at that target location and the target location is considered a feasible point for the robot. The number of target locations in the new environment can be set based on actual needs.

[0043] S105: Generate proprioception results based on the voting results corresponding to all target positions.

[0044] The proprioception results indicate the robot's feasible range of motion in the first environment. By traversing all the set target positions, the robot can obtain a feasible workspace in the new environment, allowing the robot to quickly determine its functional scope in the new environment and complete various tasks safely and efficiently.

[0045] Through steps S101 to S105, the first perception data obtained by the robot in the new environment is used to obtain the matching probability of each expert network through gated network reasoning, and the target expert network is determined according to the matching probability. Then, the feasible workspace of the robot is inferred based on the target expert network. In this way, an accurate feasible workspace of the robot can be autonomously generated in the new environment, and efficient proprioception generation can be achieved. There is no need to train the hybrid expert network with data from the new environment in advance, and it is not restricted by the environment, which greatly improves the general capabilities of the robot.

[0046] In one possible implementation, S101 may include: obtaining a plurality of first actual perception data obtained by perception detection when the robot performs the same action in the first environment, the first actual perception data being data obtained by perception detection by the robot in the first environment, such as speed, center of gravity, joint position, etc. For example, the robot may repeatedly perform the same action in a new environment, such as running from point A to point B in the new environment, and then running from point B back to point A, and using sensors provided on the robot to collect a plurality of first actual perception data, each first actual perception data may correspond to an action performed once; selecting at least one target perception data from the plurality of first actual perception data according to a predetermined first selection rule, for example, selecting several first actual perception data with a larger difference from the plurality of first actual perception data obtained as target perception data, the larger difference can be understood as a larger degree of change between the selected first actual perception data; processing the at least one target perception data according to a predetermined processing rule to obtain first perception data, for example, by taking the average value of all target perception data as the first perception data to represent the perception data of the robot in the new environment, so as to input the gated network.

[0047] It should be noted that although the first selection rule is described using a large difference as an example and the processing rule is described using an average value as an example, those skilled in the art will appreciate that the present disclosure is not limited thereto. In fact, users can flexibly set the first selection rule and processing rule based on actual application scenarios.

[0048] In one possible implementation, the proprioceptive perception generation method may further include a gated network training step. The gated network training step may include: obtaining multiple first data sets, each first data set including multiple second perception data, each second perception data in the same first data set being data obtained by the robot performing perception detection in the same second environment, and each first data set corresponding to a different second environment; constructing a gated network training set based on the multiple first data sets, and training the gated network using the gated network training set.

[0049] Each second environment used to train the gating network is different from the first environment. Each second environment used to train the gating network also serves as the environment for training the expert network. The second environment can be selected based on the robot's form and function, specifically configured according to actual needs. The method for training the gating network can be selected appropriately based on actual needs and is not limited in this disclosure.

[0050] A corresponding first data set is obtained for each second environment. The type of action the robot performs when performing perception detection in the same second environment is set according to actual needs. The types of actions the robot performs when performing perception detection in different second environments can be the same or different, and the disclosed embodiments do not impose any restrictions on this. In this way, the proprioceptive perception generation method can achieve proprioceptive perception generation in a new environment when the robot is in a new environment, without being restricted by the environment. After generating a feasible workspace for the new environment, the robot can execute tasks within the corresponding functional scope.

[0051] The robot's perception of the environment is achieved by collecting various sensor data in the environment. Sensor data can be obtained through various underlying sensors installed on the robot. The specific underlying sensors are set according to actual needs. Since components such as the robotic arm may move during the data collection process, causing the center of gravity to change, perception data (or sensor data) needs to be collected continuously. Therefore, the acquisition of multiple first data groups in the training step of the gated network may include:

[0052] Acquire multiple second actual perception data corresponding to each second environment obtained by perception detection when the robot performs the same action in each second environment, that is, the second actual perception data is the data obtained by the robot through perception detection in the second environment, such as speed, center of gravity, joint position, etc. The acquisition process of the second actual perception data is the same as that of the first actual perception data. For details, please refer to the previous text and will not be repeated here; according to a predetermined second selection rule, select at least one second actual perception data from the multiple second actual perception data of each second environment. For example, to facilitate subsequent processing, several second actual perception data with a larger difference can be used as the representative data obtained in the current second environment, and a first data group corresponding to each second environment is constructed based on at least one second actual perception data.

[0053] The gating network is trained based on the first data set in each second environment. The output of the gating network is a one-hot encoding that matches the number of expert networks. Each encoded value represents the degree of similarity between the input first perception data and each expert network. The structure of the gating network can be determined based on the form of the perception data, for example, using a multilayer perceptron (MLP) model, which is not limited in the present embodiment.

[0054] In a possible implementation, S103 can be: taking the expert network corresponding to the maximum value in the matching probabilities as the target expert network. In this case, determining the voting result according to the current feasible results in S104 can include: determining the voting result according to the current feasible result output by the target expert network and the predetermined threshold value, for example, if the current feasible result output by the target expert network is greater than the predetermined threshold value, it is considered that the robot can act at the current target position, otherwise it cannot. The maximum matching probability indicates that the corresponding expert network is closest to the new environment, so that directly taking the expert network that matches the new environment most as the target expert network can more quickly realize the ontology perception generation.

[0055] In a possible implementation, S103 can be: taking all the expert networks as the target expert networks. In this case, determining the voting result according to the current feasible results in S104 can include: determining the weights of the expert networks according to the matching probabilities of the expert networks and the first environment, for example, the probability values with higher probabilities in the matching probabilities output by the gating networks can be selected for normalization to obtain the weights of the expert networks; for each expert network, multiplying the weight corresponding to the current expert network and the current feasible result to obtain a product result corresponding to the expert network, for example, if the number of expert networks is 3, the weights of the three expert networks are 0.8, 0.2 and 0.05 in turn, and the current feasible results of the three expert networks are T, F and F (T represents True, F represents False, and T and F are numerical values, the specific sizes of which are set according to actual requirements), then the product results of the three expert networks are 0.8*F, 0.2*T and 0.05*T in turn; determining the voting result based on the sum of the product results corresponding to all the expert networks and the predetermined threshold value, for example, comparing the sum of the product results of the above three expert networks, 0.8*F+0.2*T+0.05*T, with the predetermined threshold value, if the sum of the product results is greater than the predetermined threshold value, it is considered that the robot can act at the current target position, otherwise it cannot. In this way, weights are set for the expert networks, and a more accurate voting result can be obtained based on the output results and the weights of the expert networks to indicate the feasibility of the robot acting at the target position.

[0056] In one possible implementation, the proprioception generation method may further include a training step for multiple expert networks. The training step for multiple expert networks may include: obtaining multiple second data groups, the number of second data groups being the same as the number of expert networks, each second data group including multiple range data, such as the range (x, y, z) within which a robot's manipulator can move. Each range data in the same second data group is data obtained by detecting the range of motion of the robot's manipulator in the same second environment, and each second data group corresponds to a different second environment; assigning a corresponding second data group to each original expert network, and using each second data group to train the corresponding expert network.

[0057] A second data set is obtained for each second environment. The type of action performed by the robot when performing motion range detection in the same second environment is set according to actual needs. The types of actions performed by the robot when performing motion range detection in different second environments can be the same or different, and the embodiments of the present disclosure do not impose any restrictions on this. In the same second environment, the robot can perform motion range detection and perception detection simultaneously to obtain training data for training the entire hybrid expert network. The training steps of the gated network and the training steps of multiple expert networks can be performed simultaneously or sequentially, and the embodiments of the present disclosure do not impose any restrictions on this.

[0058] The method of training multiple expert networks can be selected in an appropriate manner according to actual needs, and the embodiments of the present disclosure do not limit this. For example, the training of the expert network includes two parts, one is the basic model training, and the other is the fine-tuning of the trained technical model to obtain multiple expert models. Basic model training requires a large amount of data training to ensure the reliability of the model, so that the subsequent fine-tuning can obtain better performance of the expert network. There are many ways to collect data, two of which are commonly used methods, namely using simulation data and using multiple second data groups. If the robot has a corresponding accurate simulation model, a large amount of basic data can be generated by simulation to obtain the functional range of the robot within a certain range, but this method requires modeling the environment in advance. When using the second data group in the actual environment, that is, the second environment, data can be collected through repeated experiments in common scenarios (flat ground, slopes, etc.) to obtain multiple second data groups.

[0059] The greater the difference between the expert networks, the more representative each expert network is, making the generated proprioception generation result more accurate. Since the data collection cost is high in many actual scenarios, some representative data can be collected to fine-tune the model. The number of expert networks can be determined according to the degree to which the robot is affected by the environment, such as 8, 16, etc., and the embodiments of the present disclosure do not impose any restrictions on this. The structure of the basic model can be determined according to actual needs such as the amount of data and the form of the data, for example, using MLP. In this way, an expert model with excellent performance can be obtained, while reducing the training cost and avoiding the problem of overfitting of the model due to the small amount of data collected in the second environment.

[0060] In some embodiments, as Figure 2 As shown, a pre-training process is first performed, which includes multiple scene data (i.e. Figure 3 After the pre-training is completed, the proprioception generation process is carried out, which includes the new scene data (i.e. Figure 4 The first perception data and each target position in the new environment) are collected, gated network reasoning is performed (to determine the matching result), expert network reasoning is performed (to determine the current feasible result), and after traversing all target positions in the new environment, the ontological perception generation is ended to obtain the ontological perception result. Among them, the gated network reasoning and expert network reasoning are used to calculate the feasible workspace of the robot in the new environment, so as to determine the functional range that the robot can achieve in the new environment. In other words, the gated network reasoning and expert network reasoning are used to calculate the functional range of the robot. Specifically, the gated network uses the sensor data collected by the robot in the current new environment to determine the matching probability of each expert network, and then uses the preferred expert network to calculate the feasibility of each position in the new environment space. The ontological perception is established according to the voting principle. This method is applicable to different types of robots and diverse environments and is universal.

[0061] The disclosed embodiments provide a robot-oriented proprioception generation method to address the challenges posed by unmodeled robots and unknown environments. The method may include an expert network training step, a gated network training step, and a proprioception generation process in a new environment to calculate the functional scope or feasible space of the robot in the new environment. This method can achieve efficient proprioception generation. The solution is universal and can support various robots to autonomously generate proprioception in different environments, allowing the robot system to quickly determine its own functional scope, thereby completing various tasks safely and efficiently.

[0062] The disclosed embodiment also provides a robot-oriented proprioception generation device. Figure 5 FIG. 1 is a block diagram of a robot-oriented proprioception generation device according to an embodiment of the present disclosure. Figure 5As shown, the proprioception generation device 500 may include the following acquisition module 501, a first calculation module 502, a determination module 503, a second calculation module 504, and a generation module 505:

[0063] An acquisition module 501 is configured to acquire first perception data, where the first perception data is data obtained by the robot performing perception detection in a first environment;

[0064] a first calculation module 502 configured to input the first perception data into a gating network in a hybrid expert network for calculation to obtain a matching result, wherein the hybrid expert network includes the gating network and multiple expert networks, and the matching result indicates a matching probability between each of the expert networks and the first environment;

[0065] A determination module 503 is configured to determine at least one target expert network from the multiple expert networks according to the matching result;

[0066] A second calculation module 504 is configured to input the current target position into each target expert network for calculation for each target position in the first environment, obtain a current feasible result output by each target expert network, and determine a voting result based on each current feasible result, wherein the voting result indicates the feasibility of the robot performing an action at the current target position;

[0067] The generating module 505 is configured to generate a proprioception result according to the voting results corresponding to all target positions, wherein the proprioception result indicates a feasible motion range of the robot in the first environment.

[0068] In one possible implementation, obtaining the first perception data includes: obtaining multiple first actual perception data obtained by perception detection when the robot performs the same action in the first environment; selecting at least one target perception data from the multiple first actual perception data according to a predetermined first selection rule; and processing the at least one target perception data according to a predetermined processing rule to obtain the first perception data.

[0069] In one possible implementation, determining at least one target expert network from the multiple expert networks based on the matching results includes: taking the expert network corresponding to the maximum value among the matching probabilities as the target expert network; determining the voting result based on the current feasible results includes: determining the voting result based on the current feasible result output by the target expert network and a predetermined threshold.

[0070] In one possible implementation, determining at least one target expert network from the multiple expert networks based on the matching results includes: taking all expert networks as the target expert network; determining the voting result based on each current feasible result includes: determining the weight of each expert network based on the matching probability of each expert network with the first environment; for each expert network, multiplying the weight corresponding to the current expert network and the current feasible result to obtain the product result corresponding to each expert network; and determining the voting result based on the sum of the product results corresponding to all expert networks and a predetermined threshold.

[0071] In one possible implementation, the device also includes a first training module for executing the training steps of the gated network, and the training steps of the gated network include: obtaining multiple first data groups, each first data group includes multiple second perception data, each second perception data in the same first data group is data obtained by the robot performing perception detection in the same second environment, and each first data group corresponds to a different second environment; constructing a gated network training set based on the multiple first data groups, and using the gated network training set to train the gated network.

[0072] In one possible implementation, obtaining multiple first data groups includes: obtaining multiple second actual perception data corresponding to each second environment obtained by perception detection when the robot performs the same action in each second environment; selecting at least one second actual perception data from the multiple second actual perception data of each second environment according to a predetermined second selection rule, and constructing the first data group corresponding to each second environment based on the at least one second actual perception data.

[0073] In one possible implementation, the device also includes a second training module for executing the training steps of the multiple expert networks, and the training steps of the multiple expert networks include: obtaining multiple second data groups, the number of the second data groups is the same as the number of the expert networks, each second data group includes multiple range data, each range data in the same second data group is data obtained by motion range detection of the robot's robotic arm in the same second environment, and each second data group corresponds to a different second environment; assigning a corresponding second data group to each expert network, and using each second data group to perform corresponding expert network training.

[0074] In some embodiments, the functions or modules included in the ontology perception generation device provided in the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above ontology perception generation method embodiment. For the sake of brevity, it will not be repeated here.

[0075] The present disclosure also provides an electronic device comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above-described body perception generation method when executing the instructions stored in the memory. In some embodiments, the functions or modules included in the electronic device provided by the present disclosure can be used to perform the method described in the above method embodiment. The specific implementation thereof can refer to the description of the above body perception generation method embodiment, and for the sake of brevity, it is not repeated here.

[0076] The embodiments of the present disclosure also provide a computer-readable storage medium having computer program instructions stored thereon, which implement the above-mentioned ontology perception generation method when the computer program instructions are executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium. In some embodiments, the functions or modules contained in the computer-readable storage medium provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above text body perception generation method embodiment. For the sake of brevity, it will not be repeated here.

[0077] The present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device executes the above-mentioned body perception generation method. In some embodiments, the functions or modules included in the computer program product provided by the present disclosure can be used to execute the method described in the above method embodiment. The specific implementation thereof can refer to the description of the above body perception generation method embodiment. For the sake of brevity, it is not repeated here.

[0078] Figure 6 1900 is a block diagram of a robot-oriented proprioception generation device according to an embodiment of the present disclosure. For example, the device 1900 can be provided as a server or a terminal device. Figure 6 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0079] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0080] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the apparatus 1900 to perform the above-described method.

[0081] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0082] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0083] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0084] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0085] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0086] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0087] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0088] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0089] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A robot-oriented proprioception generation method, characterized in that: include: Acquire first perception data, where the first perception data is data obtained by the robot performing perception detection in a first environment; Inputting the first perception data into a gating network in a hybrid expert network for calculation to obtain a matching result, wherein the hybrid expert network includes the gating network and multiple expert networks, and the matching result indicates a matching probability between each of the expert networks and the first environment; determining at least one target expert network from the multiple expert networks according to the matching results; For each target position in the first environment, input the current target position into each target expert network for calculation, obtain a current feasible result output by each target expert network, and determine a voting result based on each current feasible result, wherein the voting result indicates the feasibility of the robot performing an action at the current target position; generating a proprioception result according to voting results corresponding to all target positions, wherein the proprioception result indicates a feasible motion range of the robot in the first environment; The method further includes a training step for the gated network, the training step for the gated network comprising: obtaining a plurality of first data groups, each of the first data groups including a plurality of second perception data, each of the second perception data in the same first data group being data obtained by the robot performing perception detection in the same second environment, and each of the first data groups corresponding to a different second environment; constructing a gated network training set based on the plurality of first data groups, and performing gated network training using the gated network training set; Among them, the obtaining of multiple first data groups includes: obtaining multiple second actual perception data corresponding to each second environment obtained by perception detection when the robot performs the same action in each second environment; according to a predetermined second selection rule, selecting at least one second actual perception data from the multiple second actual perception data of each second environment, and constructing the first data group corresponding to each second environment based on the at least one second actual perception data.

2. The method according to claim 1, characterized in that The acquiring of the first perception data includes: Acquire a plurality of first actual perception data obtained by perception detection when the robot performs the same action in the first environment; selecting at least one target perception data from the plurality of first actual perception data according to a predetermined first selection rule; The at least one target perception data is processed according to a predetermined processing rule to obtain the first perception data.

3. The method according to claim 1 or 2, characterized in that Determining at least one target expert network from the plurality of expert networks according to the matching results includes: taking the expert network corresponding to the maximum value among the matching probabilities as the target expert network; Determining the voting result according to each current feasible result includes: determining the voting result based on the current feasible result output by the target expert network and a predetermined threshold.

4. The method according to claim 1 or 2, characterized in that Determining at least one target expert network from the multiple expert networks according to the matching results includes: taking all expert networks as the target expert network; Determining the voting result based on each current feasible result includes: determining the weight of each expert network based on the matching probability between each expert network and the first environment; for each expert network, multiplying the weight corresponding to the current expert network and the current feasible result to obtain the product result corresponding to each expert network; and determining the voting result based on the sum of the product results corresponding to all expert networks and a predetermined threshold.

5. The method according to claim 1 or 2, characterized in that The method further includes a step of training the multiple expert networks, wherein the step of training the multiple expert networks includes: Acquire multiple second data groups, the number of the second data groups being the same as the number of the expert networks, each second data group including multiple range data, each range data in the same second data group being data obtained by detecting the range of motion of the robot's manipulator arm in the same second environment, and each second data group corresponding to a different second environment; A corresponding second data set is allocated to each expert network, and each second data set is used to perform corresponding expert network training.

6. A proprioceptive perception generating device for a robot, characterized in that: include: An acquisition module is used to acquire first perception data, where the first perception data is data obtained by the robot performing perception detection in a first environment; a first computing module, configured to input the first perception data into a gating network in a hybrid expert network for computing to obtain a matching result, wherein the hybrid expert network includes the gating network and a plurality of expert networks, and the matching result indicates a matching probability between each of the expert networks and the first environment; a determination module, configured to determine at least one target expert network from the plurality of expert networks according to the matching result; a second calculation module, configured to input a current target position into each target expert network for calculation, for each target position in the first environment, obtain a current feasible result output by each target expert network, and determine a voting result based on each current feasible result, wherein the voting result indicates the feasibility of the robot performing an action at the current target position; a generating module, configured to generate a proprioception result according to voting results corresponding to all target positions, wherein the proprioception result indicates a feasible motion range of the robot in the first environment; The device also includes a first training module for executing a training step for the gated network, wherein the training step for the gated network includes: obtaining a plurality of first data groups, each of the first data groups including a plurality of second perception data, each of the second perception data in the same first data group is data obtained by the robot performing perception detection in the same second environment, and each of the first data groups corresponds to a different second environment; constructing a gated network training set based on the plurality of first data groups, and performing gated network training using the gated network training set; Among them, the obtaining of multiple first data groups includes: obtaining multiple second actual perception data corresponding to each second environment obtained by perception detection when the robot performs the same action in each second environment; according to a predetermined second selection rule, selecting at least one second actual perception data from the multiple second actual perception data of each second environment, and constructing the first data group corresponding to each second environment based on the at least one second actual perception data.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 5 when executing the instructions stored in the memory.

8. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images