Simulation method, device and equipment with intelligent body and storage medium

Abstract

The embodiment of the invention provides an intelligent simulation method and device, equipment and a storage medium, and relates to the technical field of sensor simulation, and the method comprises the steps: displaying a three-dimensional virtual scene; in response to a layout instruction input for the three-dimensional virtual scene, setting virtual geometries at a plurality of positions indicated by the layout instruction; controlling the carrier to perform simulation motion in the three-dimensional virtual scene, and controlling each virtual geometry to move along with the carrier; in the simulation motion process of the carrier, respectively determining the contact depth of each virtual geometry and an object in the three-dimensional virtual scene, and taking the contact depth as the contact depth corresponding to each virtual geometry; and for each array element, calculating the pressure of the array element according to the contact depth corresponding to the virtual geometry located at the array element, and determining a detection signal detected under the condition that the array element bears the pressure. By adopting the embodiment of the invention, the original detection signal can be obtained when the contact sensor is simulated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sensor simulation, in particular to a simulation method and device for embodied intelligence, equipment and storage medium. BACKGROUND

[0002] In the embodied intelligence scenario, a large amount of interaction data between the embodied intelligent carrier (such as a robot) and the environment needs to be obtained to optimize the decision model to adapt to variable scenarios. However, the interaction data collection of the embodied intelligent carrier in the real physical environment is difficult and costly, so in the related technology, a large amount of interaction data in a virtual physical scene is often obtained through a simulation process to train the embodied intelligent system. In this paper, the interaction data in a virtual physical scene obtained through simulation is referred to as synthetic data.

[0003] In the simulation process, the detection signal generated by simulating the contact sensor is a major component of the synthetic data. However, in the existing technology, the process of simulating the contact sensor is limited by the performance of the simulation software, and usually only mechanical simulation can be performed, that is, the mechanical signal of the sensor contacting the object in the virtual environment is simulated as synthetic data for the training of the embodied intelligent agent. This also leads to the fact that the embodied intelligence can only learn the mapping relationship from the mechanical signal to the processing method in the training process, that is, the embodied intelligent agent can only learn how to make decisions based on the mechanical signal.

[0004] However, the working principle of the real contact sensor is to convert the detection signal detected by the contact sensor when it contacts an object into a mechanical signal. Since the mechanical signal is obtained by converting and processing the detection signal detected by the sensor, there is a certain loss of information, so compared with the original detection signal detected by the sensor, the mechanical signal contains less information.

[0005] Therefore, when simulating the contact sensor, the original detection signal can be obtained, which can enable the embodied intelligent agent to learn how to make decisions based on the original detection signal, and thus enrich the information based on which the embodied intelligent agent makes decisions, thereby improving the accuracy of the decisions. SUMMARY

[0006] The purpose of the embodiments of the present application is to provide a simulation method, device, equipment and storage medium for embodied intelligence, which can obtain the original detection signal when simulating the contact sensor. The specific technical solutions are as follows:

[0007] In a first aspect, the embodiments of the present application provide a simulation method for embodied intelligence, which comprises:

[0008] displaying a three-dimensional virtual scene, wherein the three-dimensional virtual scene includes an embodied intelligent carrier;

[0009] in response to a layout instruction input for the three-dimensional virtual scene, setting virtual geometric bodies at positions indicated by the layout instruction respectively, wherein the layout instruction is used to indicate positions of each element of a multi-element sensor of the carrier, and each of the virtual geometric bodies does not have a collision volume;

[0010] controlling the carrier to simulate motion in the three-dimensional virtual scene, and controlling each of the virtual geometric bodies to follow the motion of the carrier;

[0011] during the simulation motion of the carrier, determining a contact depth of each of the virtual geometric bodies with an object in the three-dimensional virtual scene as a respective contact depth of each of the virtual geometric bodies;

[0012] for each of the elements, calculating a pressure of the element according to the respective contact depth of the virtual geometric body located at the element, and determining a detection signal detected when the element bears the pressure.

[0013] In an embodiment of the present application, the method further comprises:

[0014] displaying a preset object data package, wherein the object data package at least includes a shape of an object and a physical rule to be followed when the object moves;

[0015] in response to a simulation environment building instruction input for the displayed object data package, determining a target object data package indicated by the simulation environment building instruction;

[0016] generating an object having the shape in the target object data package and following the physical rule in the target object data package, to obtain a three-dimensional virtual scene.

[0017] In an embodiment of the present application, the shape of the three-dimensional rigid body is a convex hull, and the physical rule in the object data package of the three-dimensional rigid body is used to represent a rigidity of the three-dimensional rigid body and a friction force of the three-dimensional rigid body when simulating motion in the three-dimensional virtual scene; and / or

[0018] the shape of the three-dimensional soft body is a tetrahedron set, and the physical rule in the object data package of the three-dimensional soft body is used to represent a Poisson coefficient of the three-dimensional soft body and a friction force of the three-dimensional soft body when simulating motion in the three-dimensional virtual scene; and / or

[0019] the physical rule in the object data package of the three-dimensional fluid is used to represent a Reynolds number of the three-dimensional fluid; and / or

[0020] the physical rule in the object data package of the three-dimensional gas is used to represent a Reynolds number of the three-dimensional gas; and / or

[0021] The carrier is cylindrical in shape, and the physical rules in the object data package of the carrier are used to represent: the motor torque and motor gain of the carrier, and the friction force when the carrier moves in a simulated three-dimensional virtual scene.

[0022] In one embodiment of this application, the method further includes:

[0023] The system displays preset scene configuration data packages; wherein, the scene configuration data packages include: lighting data packages, and / or camera data packages, and / or media data packages; the lighting data packages include: the emission spectrum of the light, and / or color temperature, and / or spectral range, and / or mask; the camera data packages include: the camera's color matching function curve, and / or color, and / or depth; the media data packages include: the density field of the medium, and / or radiation field;

[0024] In response to the simulation environment configuration command input for each of the displayed configuration data packets, determine the target configuration data packet indicated by the simulation environment configuration command;

[0025] Generate the lights and / or cameras and / or media indicated by the target configuration data packet.

[0026] In one embodiment of this application, the method further includes:

[0027] Based on the signal strength of the detection signal of each array element, a bitmap comprising multiple image regions is generated. Each region in the bitmap corresponds to an array element, and different regions correspond to different array elements. The relative position between any two regions is the same as the relative position between the corresponding array elements. The pixel value of each region in the bitmap is related to the signal strength of the detection signal of the array element corresponding to each region.

[0028] In one embodiment of this application, the method further includes:

[0029] Based on the preset correspondence between sensor types and noise algorithms, the target noise algorithm corresponding to the type of the multi-element sensor is determined.

[0030] According to the target noise algorithm, noise is added to the detection signal of each array element.

[0031] Secondly, embodiments of this application provide a simulation device for embodied intelligence, the device comprising:

[0032] A 3D virtual scene display module is used to display 3D virtual scenes, which include carriers of embodied intelligence.

[0033] The virtual geometry setting module is used to set virtual geometries at multiple locations indicated by the layout instructions in response to the layout instructions input to the three-dimensional virtual scene. The layout instructions are used to indicate the position of each element in the multi-element sensor of the carrier, and each virtual geometry has no collision volume.

[0034] The carrier control module is used to control the carrier to simulate movement in the three-dimensional virtual scene, and to control each virtual geometry to follow the movement of the carrier;

[0035] The contact depth determination module is used to determine the contact depth between each virtual geometry and an object in the three-dimensional virtual scene during the simulated motion of the carrier, and to use the contact depth of each virtual geometry as its respective contact depth.

[0036] The detection signal determination module is used to calculate the pressure of each array element based on the contact depth corresponding to the virtual geometry located at the array element, and to determine the detection signal detected when the array element is subjected to the pressure.

[0037] In one embodiment of this application, the apparatus further includes:

[0038] The object data packet display module is used to display preset object data packets, which include at least: the shape of the object and the physical rules that the object must follow when it moves;

[0039] The target environment data packet determination module is used to determine the target object data packet indicated by the simulation environment setup instruction in response to the simulation environment setup instruction input for each displayed object data packet;

[0040] The 3D virtual scene generation module is used to generate objects that have the shape in the target object data packet and comply with the physical rules in the target object data packet, thereby obtaining a 3D virtual scene.

[0041] In one embodiment of this application, the shape of the three-dimensional rigid body is a convex hull, and the physical rules in the object data package of the three-dimensional rigid body are used to represent: the stiffness of the three-dimensional rigid body, and the frictional force of the three-dimensional rigid body during simulated motion in a three-dimensional virtual scene; and / or

[0042] The 3D software body is a set of tetrahedrons. The physical rules in the object data package of the 3D software body are used to represent: the Poisson coefficient of the 3D software body, and the frictional force when the 3D software body simulates motion in a 3D virtual scene; and / or

[0043] The physical rules in the object data package for three-dimensional fluids are used to represent: the Reynolds number of the three-dimensional fluid; and / or

[0044] The physical rules in the object data package for three-dimensional gases are used to represent: the Reynolds number of the three-dimensional gas; and / or

[0045] The carrier is cylindrical in shape, and the physical rules in the object data package of the carrier are used to represent: the motor torque and motor gain of the carrier, and the friction force when the carrier moves in a simulated three-dimensional virtual scene.

[0046] In one embodiment of this application, the apparatus further includes:

[0047] The scene configuration data package display module is used to display preset scene configuration data packages; wherein, the scene configuration data packages include: lighting data packages, and / or camera data packages, and / or media data packages; the lighting data packages include: the emission spectrum of the light, and / or color temperature, and / or spectral range, and / or mask; the camera data packages include: the camera's color matching function curve, and / or color, and / or depth; the media data packages include: the density field of the medium, and / or radiation field;

[0048] The target configuration data packet determination module is used to determine the target configuration data packet indicated by the simulation environment configuration command in response to the simulation environment configuration command input for each displayed configuration data packet;

[0049] Generate the lights and / or cameras and / or media indicated by the target configuration data packet.

[0050] In one embodiment of this application, the apparatus further includes:

[0051] The bitmap generation module is used to generate a bitmap including multiple image regions based on the signal strength of the detection signal of each array element. Each region in the bitmap corresponds to an array element, and different regions correspond to different array elements. The relative position between any two regions is the same as the relative position between the corresponding array elements. The pixel value of each region in the bitmap is related to the signal strength of the detection signal of the array element corresponding to each region.

[0052] In one embodiment of this application, the apparatus further includes:

[0053] The target noise algorithm determination module is used to determine the target noise algorithm corresponding to the type of the multi-element sensor based on a preset correspondence between sensor types and noise algorithms.

[0054] The noise addition module is used to add noise to the detection signal of each array element according to the target noise algorithm.

[0055] This application also provides an electronic device, including:

[0056] Memory, used to store computer programs;

[0057] A processor, when executing a program stored in memory, implements any of the above-described embody intelligence simulation methods.

[0058] This application also provides a computer-readable storage medium storing a computer program, which is executed by a processor using any of the embodied intelligence simulation methods described above.

[0059] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the embodied intelligence simulation methods described above.

[0060] Beneficial effects of the embodiments in this application:

[0061] In the technical solution provided in this application embodiment, the carrier of the embodied intelligence that constructs virtual geometry is controlled to perform simulated motion, thereby enabling the virtual geometry to come into contact with objects in a three-dimensional virtual scene, thus realizing the simulation of a multi-element sensor. During the simulation process, the contact depth between each virtual geometry and the objects in the three-dimensional virtual scene is determined as the corresponding contact depth for each virtual geometry. For each element in the multi-element sensor, the pressure of the element is calculated based on the contact depth corresponding to the virtual geometry located at the element, and the detection signal detected when the element bears the pressure is determined.

[0062] As can be seen, in this embodiment, the mechanical signal generated when the virtual geometry comes into contact with an object in the 3D virtual scene is not used as the detection signal. Instead, the pressure of the array element is calculated based on the contact depth of the virtual geometry located at the array element, and the detection signal detected when the array element is under pressure is determined. This can characterize the original detection signal detected by the sensor and has richer information. Based on this, the embodied agent can learn how to make decisions based on the original detection signal, thereby enriching the information on which the embodied agent makes decisions and improving the accuracy of the decisions. In addition, in this embodiment, only mechanical simulation is required in the process of simulating the original detection signal, without the need for other forms of simulation such as electrical simulation or optical simulation, which reduces the performance requirements of the simulation software.

[0063] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0065] Figure 1 A flowchart illustrating the first embodied intelligence simulation method provided in this application embodiment;

[0066] Figure 2 This is a schematic diagram illustrating the contact between a virtual geometry and an object in a three-dimensional virtual scene, as provided in an embodiment of this application.

[0067] Figure 3 A flowchart illustrating the second embodied intelligence simulation method provided in this application embodiment;

[0068] Figure 4 A schematic diagram of the configuration interface of a simulation software provided in an embodiment of this application;

[0069] Figure 5 This is a schematic diagram of the rendered three-dimensional virtual scene provided in the embodiments of this application;

[0070] Figure 6 A flowchart illustrating the third embodied intelligence simulation method provided in this application embodiment;

[0071] Figure 7 A schematic diagram of a bitmap provided in an embodiment of this application;

[0072] Figure 8 A flowchart illustrating the fourth embodied intelligence simulation method provided in this application embodiment;

[0073] Figure 9 A system architecture diagram of an embodied intelligent system provided in this application embodiment;

[0074] Figure 10 A schematic diagram of the structure of a simulation device for embodied intelligence provided in an embodiment of this application;

[0075] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0077] In order to obtain the original detection signal when simulating contact sensor simulation, this application provides a simulation method, apparatus, device and medium with embodied intelligence.

[0078] See Figure 1 This is a flowchart illustrating the first embodied intelligence simulation method provided in this application embodiment. This method can be applied to electronic devices equipped with simulation software. For ease of description, the following text uses electronic devices as the execution subject, which is not intended to be limiting. The above method includes steps S101-S105.

[0079] S101 showcases a three-dimensional virtual scene.

[0080] Among them, the three-dimensional virtual scene includes a carrier of embodied intelligence.

[0081] S102, in response to a layout instruction input for a 3D virtual scene, sets virtual geometry at multiple locations indicated by the layout instruction.

[0082] Among them, the layout instructions are used to indicate the position of each element in the multi-element sensor of the carrier, and there is no collision volume in each virtual assembly.

[0083] S103 controls the carrier to simulate movement in a three-dimensional virtual scene and controls each virtual geometry to follow the carrier's movement.

[0084] S104, during the simulated motion of the carrier, the contact depth between each virtual geometry and the objects in the three-dimensional virtual scene is determined, which is used as the corresponding contact depth of each virtual geometry.

[0085] S105, for each array element, calculate the pressure of the array element based on the contact depth corresponding to the virtual geometry located at the array element, and determine the detection signal detected when the array element is under pressure.

[0086] In the technical solution provided in this application embodiment, the carrier of the embodied intelligence that constructs virtual geometry is controlled to perform simulated motion, thereby enabling the virtual geometry to come into contact with objects in a three-dimensional virtual scene, thus realizing the simulation of a multi-element sensor. During the simulation process, the contact depth between each virtual geometry and the objects in the three-dimensional virtual scene is determined as the corresponding contact depth for each virtual geometry. For each element in the multi-element sensor, the pressure of the element is calculated based on the contact depth corresponding to the virtual geometry located at the element, and the detection signal detected when the element bears the pressure is determined.

[0087] As can be seen, in this embodiment, the detection signals detected by each array element can characterize the original detection signal and have richer information. Based on this, the embodied agent can learn how to make decisions based on the original detection signal, thereby enriching the information on which the embodied agent bases its decisions and improving the accuracy of the decisions. In addition, in this embodiment, only mechanical simulation is required in the process of simulating the original detection signal; other forms of simulation such as electrical simulation and optical simulation are not needed, which reduces the difficulty of implementing the solution.

[0088] The following is an exemplary description of steps S101-S105.

[0089] In step S101 above, the three-dimensional virtual scene can be understood as a virtual scene built in simulation software. Simulation software is used to simulate the behavior or physical processes of a real system in a virtual environment. For example, in this embodiment, the simulation focuses on the process of a robot's gripper grasping an object, sensing mechanical signals based on contact sensors on the gripper, and adjusting the gripper's tightness based on these mechanical signals. Therefore, the simulation software used is mechanical simulation software.

[0090] The carrier of embodied intelligence can be understood as an intelligent system that interacts with its environment in real time through a physical body. For example, the carrier of embodied intelligence can be a robot, a drone, or an unmanned vehicle. The carrier of embodied intelligence included in a 3D virtual scene is a simulation and is the same carrier as the embodied intelligence carrier in the real scene. For example, in this embodiment of the application, the process of a robot's gripper grasping an object, sensing mechanical signals based on contact sensors on the gripper, and adjusting the gripper's tightness based on these mechanical signals, is simulated. Therefore, the carrier of embodied intelligence in the 3D virtual scene is the robot.

[0091] In this embodiment, the electronic device can respond to a user's command to display a 3D virtual scene, thereby displaying the 3D virtual scene. The 3D virtual scene can be built by the user using simulation software according to their actual needs; how to build a 3D virtual scene will be described in detail later. Alternatively, the user can select from a variety of preset 3D virtual scenes provided by the simulation software provider. Multiple different 3D virtual scenes can be pre-stored in the electronic device, allowing the user to select the target 3D virtual scene to be displayed and issue a command to the electronic device to display it. The electronic device can then display the target 3D virtual scene to the user.

[0092] In step S102 above, the actual carrier of embodied intelligence is generally equipped with sensors, enabling the carrier to perceive the real environment based on the sensors. For example, a contact sensor is installed at the end of the robot's gripper. This contact sensor can detect mechanical signals and adjust the tightness of the gripper based on the mechanical signals, so that the robot's gripper can better grasp objects.

[0093] A multi-element sensor is a sensor composed of multiple independent sensing units (elements) arranged in a specific geometric structure, which generates multiple contact points when in contact with an object. For example, a multi-element sensor can be a capacitive array sensor, which consists of capacitors arranged in a preset array.

[0094] Based on this, in the embodiments of this application, virtual geometries are set at the positions of each element in the multi-element sensor of the carrier. Multiple virtual geometries are used to simulate the multi-element sensor. The virtual geometries do not have collision volume. It can be simply understood that the virtual geometries are regarded as non-physical entities in the simulation scene. Other objects can pass directly through the virtual geometries without triggering a collision response.

[0095] The number of virtual geometries is related to the number of elements in a multi-element sensor. More specifically, the number of virtual geometries can be a multiple of the number of elements in a multi-element sensor. A higher number of virtual geometries results in higher simulation accuracy, but also places higher demands on the performance of the electronic equipment. The number of virtual geometries can be set according to actual needs to maintain a balance between accuracy and performance. Specifically, a higher number of virtual geometries can be used when higher accuracy is required, while a lower number can be used when higher performance is required. For example, for a 6*10 capacitive array sensor, the corresponding number of virtual geometries can be 60. If the user-input layout instructions indicate the locations of the 60 capacitors in the capacitive array sensor on the carrier, virtual geometries can be set at these locations. Alternatively, for a 6*10 capacitive array sensor, the corresponding number of virtual geometries can also be 120. If the user-input layout instructions indicate the locations of the 60 capacitors in the capacitive array sensor on the carrier, two virtual geometries can be set at each location.

[0096] In this embodiment, the electronic device can display a configuration page of the simulation software to the user. The user can then input layout instructions for the 3D virtual scene based on the configuration page, specifying the positions of each element in the multi-element sensor array of the carrier. The electronic device can then respond to the input layout instructions by setting virtual geometric shapes at the multiple locations indicated by the instructions. This completes the simulation of building a carrier with a multi-element sensor array.

[0097] In step S103 above, virtual geometry is set at the position of each element in the multi-element sensor of the carrier. In this way, the carrier can be controlled to simulate motion in a three-dimensional virtual scene, and the virtual geometry can also follow the carrier to move.

[0098] In this embodiment, the motion trajectory of the carrier in the three-dimensional virtual scene can be preset, so that the electronic device can control the carrier to simulate motion in the three-dimensional virtual scene, and further, control each virtual geometry to follow the movement of the carrier.

[0099] In step S104 above, the 3D virtual scene includes not only the embodied intelligence carrier but also other objects. These other objects are simulated objects that exist in the real world. Based on their physical properties, these objects can be classified into four main categories: rigid bodies, soft bodies, gases, and fluids. Correspondingly, other objects in the 3D virtual scene can also be classified into four main categories: 3D rigid bodies, 3D soft bodies, 3D gases, and 3D fluids. For example, imagine placing an aluminum can on a table covered with a tablecloth. The gripper on the robot picks up the aluminum can from the table. Simulating this process, in the constructed 3D virtual scene, the table and the aluminum can are 3D rigid bodies, and the tablecloth is a 3D soft body.

[0100] In a 3D virtual scene, contact depth can characterize the vertical distance by which two objects penetrate or indent each other during contact. Since the virtual geometries do not have collision volumes, during the simulated movement of the carrier, the virtual geometries on the carrier come into contact with objects in the 3D virtual scene. The virtual geometries will directly penetrate the objects in the 3D virtual scene, and the positions of the virtual geometries and objects in the 3D virtual scene are known. Therefore, the contact depth between the virtual geometries and the objects in the 3D virtual scene can be determined based on the degree of overlap between their positions.

[0101] In this embodiment, the electronic device can determine the contact depth between a virtual geometry and an object in a 3D virtual scene based on a geometric collision detection method, or it can determine the contact depth based on a finite element analysis method. This embodiment does not limit the specific implementation method for determining the contact depth. When a virtual geometry does not make contact with an object in the 3D scene, the contact depth between the virtual geometry and the object can be considered zero. After determining the contact depth between each virtual geometry and an object in the 3D virtual scene, this depth is taken as the contact depth corresponding to each virtual geometry.

[0102] For ease of understanding, this application provides a schematic diagram illustrating the contact between virtual geometries and objects in a 3D virtual scene. See [link / reference]. Figure 2 As can be seen from the figure, the object in the 3D virtual scene is an aluminum can. Multiple virtual geometric bodies form a multi-element sensor (as shown in the box in the figure). The multi-element sensor composed of multiple virtual geometric bodies is set at the end of the robot gripper (carrier). The movement of the robot gripper drives the movement of the virtual geometric bodies, thereby realizing the contact between the virtual geometric bodies and the object. When the virtual geometric bodies come into contact with the object in the 3D virtual scene, they directly penetrate the object in the 3D virtual scene.

[0103] In step S105 above, after determining the contact depth corresponding to each virtual geometry, the electronic device calculates the pressure of the array element for each array element sensor based on the contact depth of the virtual geometry located at the array element. Based on this, it can be seen that the embodiments of this application are essentially performing mechanical simulation. However, in these embodiments, the calculated pressure is not used as a detection signal. Instead, the detection signal detected when the array element bears the pressure is determined based on the contact depth of the virtual geometry located at the array element. That is, the contact depth between the virtual geometry and the object in the three-dimensional virtual scene is mapped to the detection signal detected by the array element.

[0104] Since different types of sensors detect a variety of original detection signals, such as photoelectric sensors detecting optical signals and capacitive sensors detecting electrical signals, and simulation software for mechanical simulation is often limited by its performance, it is difficult to perform optical and electrical simulations. In this embodiment, in the process of simulating the detection signal detected by the array element, only mechanical simulation is required, without the need for electrical or optical simulations, to simulate the original detection signal detected by the sensor, thus reducing the performance requirements of the simulation software.

[0105] In one embodiment of this application, when only one virtual geometry is provided at the location of an array element, the contact depth corresponding to that virtual geometry can be used as the contact depth corresponding to the virtual geometry located at the array element. In another embodiment of this application, when multiple virtual geometries are provided at the location of an array element, the maximum, minimum, or average value of the contact depths corresponding to these multiple virtual geometries can be used as the contact depth corresponding to the virtual geometry located at the array element.

[0106] In this embodiment, for each element of a multi-element sensor, the electronic device can convert the contact depth corresponding to the virtual geometry located at the element into the pressure of the element based on a conversion formula. This conversion formula can be obtained based on the Hertzian contact equation.

[0107] It is easy to understand that there is a mapping relationship between the pressure exerted by an array element and the detection signal detected by the array element. For each array element in a multi-element sensor, after the electronic device calculates the pressure of each element, it can determine the detection signal detected when the array element is subjected to that pressure based on the above mapping relationship. In one embodiment, there is a positive correlation between the pressure exerted by an array element and the strength of the detection signal, that is, the greater the pressure exerted by the array element, the stronger the detection signal. The electronic device can determine the strength of the detection signal detected when the array element is subjected to that pressure based on the positive correlation mapping relationship. In another embodiment, there is a negative correlation between the pressure exerted by an array element and the strength of the detection signal, that is, the greater the pressure exerted by the array element, the weaker the detection signal. The electronic device can determine the strength of the detection signal detected when the array element is subjected to that pressure based on the negative correlation mapping relationship.

[0108] See Figure 3 This is a flowchart illustrating the second embodied intelligence simulation method provided in this application embodiment. The method includes steps S301-S308, compared to... Figure 1In the embodiment shown, steps S304-S308 are the same as steps S101-S105 above, and will not be repeated here. Steps S301-S303 are new steps.

[0109] S301 displays preset data packets for each object.

[0110] The object data packet must include at least the object's shape and the physical rules that the object must follow when it moves.

[0111] S302, in response to the simulation environment setup command input for each displayed object data packet, determines the target object data packet indicated by the simulation environment setup command.

[0112] S303 generates an object that has the shape in the target object data packet and conforms to the physical rules in the target object data packet, thus obtaining a three-dimensional virtual scene.

[0113] S304 showcases a three-dimensional virtual scene.

[0114] Among them, the three-dimensional virtual scene includes a carrier of embodied intelligence.

[0115] S305, in response to a layout command input for a 3D virtual scene, sets virtual geometry at multiple locations indicated by the layout command.

[0116] Among them, the layout instructions are used to indicate the position of each element in the multi-element sensor of the carrier, and there is no collision volume in each virtual assembly.

[0117] S306 controls the simulated movement of the carrier in a three-dimensional virtual scene and controls each virtual geometry to follow the movement of the carrier.

[0118] S307, during the simulated motion of the carrier, the contact depth between each virtual geometry and the objects in the three-dimensional virtual scene is determined, and is used as the corresponding contact depth for each virtual geometry.

[0119] S308: For each array element, the pressure of the array element is calculated based on the contact depth corresponding to the virtual geometry located at the array element, and the detection signal detected when the array element is under pressure is determined.

[0120] In existing technologies, building a 3D virtual environment requires users to create multiple objects step-by-step. First, users need to construct objects with preset shapes by selecting lines and geometric shapes, then set physical rules that these objects must follow to obtain objects with preset shapes that adhere to these physical rules, thus creating the 3D virtual environment. On the one hand, this method of generating a 3D virtual environment requires users to have professional simulation knowledge to complete its construction. On the other hand, this method is inefficient in building a 3D virtual environment. To reduce the difficulty of building a 3D virtual environment and improve its efficiency, the embodiments of this invention can execute... Figure 3 The example shown.

[0121] The following is an exemplary description of steps S301-S308.

[0122] Steps S304-S308 are similar to steps S101-S105 above, and will not be described again here.

[0123] In step S301 above, various object data packages are pre-set. Each object data package includes at least the object's shape and the physical rules that the object must follow when moving. Based on the object data package, an object with the shape specified in the object data package and conforming to the physical rules specified in the object data package can be generated. In this way, when building a 3D virtual environment, the user can directly select the target object data package from multiple object data packages to generate the corresponding object. The user does not need simulation knowledge to generate objects in the 3D virtual scene, reducing the difficulty of building a 3D virtual environment. Furthermore, the user does not need to generate objects step by step, improving the efficiency of building a 3D virtual environment.

[0124] In one embodiment of this application, objects in a three-dimensional virtual scene can be mainly categorized into three-dimensional rigid bodies, three-dimensional soft bodies, three-dimensional fluids, three-dimensional gases, and carriers. These objects can encompass the vast majority of objects in the real world.

[0125] The shape of the three-dimensional rigid body is a convex hull. The physical rules in the object data package of the three-dimensional rigid body are used to represent: the stiffness of the three-dimensional rigid body, and the friction force when the three-dimensional rigid body moves in a simulated three-dimensional virtual scene.

[0126] The shape of a 3D software is a set of tetrahedrons. The physical rules in the object data package of the 3D software are used to represent: the Poisson coefficient of the 3D software, and the friction force when the 3D software simulates motion in a 3D virtual scene.

[0127] The physical rules in the object data package of a three-dimensional fluid are used to represent: the Reynolds number of the three-dimensional fluid.

[0128] The physical rules in the object data package for three-dimensional gases are used to represent: the Reynolds number of the three-dimensional gas.

[0129] The carrier is cylindrical in shape. The physical rules in the carrier's object data package are used to represent: the carrier's motor torque and motor gain, as well as the friction force when the carrier simulates motion in a 3D virtual scene.

[0130] Furthermore, in this embodiment, the object data packet, i.e., the shape of the object and the physical rules that the object must follow when moving, can be stored in tabular form. See Table 1, which shows one storage format of the object data packet given in this embodiment.

[0131] Table 1

[0132]

[0133] In Table 1 above, the collider represents a virtual geometry attached to an object, used to define the object's collision boundaries and interaction range. Geometric properties represent the object's geometric representation. The mesh is used to precisely describe the object's shape, while the AABB bounding box is a simplified geometry that encloses the object in a cuboid shape. In short, the collider and geometric properties can be simply understood as the shapes of the objects in the aforementioned object data package. Friction and simulation coefficients in Table 1 can be simply understood as the physical rules that the objects in the object data package must follow when moving. The contents of the above tables can be added, deleted, searched, and modified according to actual needs.

[0134] In this embodiment of the application, the electronic device can display preset data packets of various objects to the user based on the configuration interface of the simulation software. For example... Figure 4 The diagram shown is a schematic of the configuration interface of a simulation software provided in an embodiment of this application. The object data package display bar on the left side of the configuration interface displays preset object data packages (such as object data package 1, object data package 2, object data package 3, object data package 4, etc.). The display area of ​​the configuration interface can display objects generated based on the object data packages.

[0135] In step S302 above, the electronic device can display the configuration page of the simulation software to the user, so that the user can input simulation environment building instructions for each displayed object data packet based on the configuration page. The simulation environment building instructions can include the identifier of the target object data packet. Then, the electronic device can respond to the simulation environment building instructions and determine the target object data packet indicated by the simulation environment building instructions. The number of determined target object data packets can be one or more.

[0136] In one embodiment of this application, the user selects an object data packet from the various object data packets displayed on the configuration page and drags it into the display area of ​​the simulation software, thus completing the input of the simulation environment setup command. Subsequently, the electronic device can respond to the simulation environment setup command and identify the object data packet dragged into the display area by the user as the target object data packet.

[0137] In step S303 above, after determining the target object data packet, the electronic device generates an object that has the shape in the target object data packet and conforms to the physical rules in the target object data packet, thus obtaining a three-dimensional virtual scene.

[0138] As can be seen from the above embodiments, in this application embodiment, an object data package is pre-stored. The object data package can indicate the shape of the object and the physical rules that the object must follow when moving. Therefore, the user only needs to determine the target object data package from the pre-stored object data package to generate an object with the shape and physical rules specified in the target object data package, thus obtaining a three-dimensional virtual scene. In this way, the user does not need professional simulation knowledge to complete the construction of the virtual scene, and there is no need for the user to generate objects step by step, improving the efficiency of virtual scene construction.

[0139] To make the 3D virtual scene more closely resemble the real environment, it is necessary to configure the 3D virtual environment to make it more like the real environment. This allows us to obtain RGBD (RGB-Depth, color-depth) data from the 3D virtual scene, which can then be used to train the embodied intelligence system.

[0140] Based on this, in one embodiment of this application, in addition to pre-stored object data packets, scene configuration data packets may also be pre-stored.

[0141] The above-mentioned scenario configuration data packets include: light data packets, and / or camera data packets, and / or media data packets.

[0142] The aforementioned light data package includes: the light emission spectrum, and / or color temperature, and / or spectral range, and / or masking.

[0143] The aforementioned camera data package includes: the camera's color matching function curve, and / or color, and / or depth.

[0144] The aforementioned media data packet includes: the density field and / or radiation field of the medium.

[0145] Furthermore, in this embodiment, the configuration data packet can be stored in the form of a table. See Table 2, which shows one storage format of the configuration data packet given in this embodiment.

[0146] Table 2

[0147]

[0148] In one embodiment of this application, the electronic device displays preset configuration data packages for various scenes based on the configuration page of the simulation software. Responding to the simulation environment configuration command input by the user for each displayed configuration data package, the device determines the target configuration data package indicated by the simulation environment configuration command, and then generates the lights and / or cameras and / or media indicated by the target configuration data package, thereby realizing the configuration of the three-dimensional virtual scene. Furthermore, a ray tracing rendering engine can be used to render the three-dimensional virtual environment to obtain RGBD data. For example... Figure 5 The image shown is a schematic diagram of a rendered three-dimensional virtual scene as provided in an embodiment of this application.

[0149] As can be seen from the above embodiments, in the embodiments of this application, a scene configuration data package is pre-stored. Then, the user only needs to determine the target configuration data package from the pre-stored scene configuration data package to generate the lights and / or cameras and / or media indicated by the target configuration data package, so that the three-dimensional virtual environment is closer to the real environment. In this way, RGBD data in the three-dimensional virtual scene can be obtained. The RGBD data can also be used to train the embodied intelligent system and enrich the information on which the embodied intelligent agent makes decisions.

[0150] See Figure 6 This is a flowchart illustrating the third embodied intelligence simulation method provided in this application embodiment. The method includes steps S601-S606, wherein steps S601-S605 are the same as steps S101-S105 above, and will not be described again here. Step S606 is a newly added step.

[0151] S601 showcases a three-dimensional virtual scene.

[0152] Among them, the three-dimensional virtual scene includes a carrier of embodied intelligence.

[0153] S602, in response to a layout instruction input for a 3D virtual scene, sets virtual geometry at multiple locations indicated by the layout instruction.

[0154] Among them, the layout instructions are used to indicate the position of each element in the multi-element sensor of the carrier, and there is no collision volume in each virtual assembly.

[0155] S603 controls the simulated movement of the carrier in a three-dimensional virtual scene and controls each virtual geometry to follow the movement of the carrier.

[0156] S604, during the simulated motion of the carrier, the contact depth between each virtual geometry and the objects in the three-dimensional virtual scene is determined, and is used as the corresponding contact depth for each virtual geometry.

[0157] S605: For each array element, the pressure of the array element is calculated based on the contact depth corresponding to the virtual geometry located at the array element, and the detection signal detected when the array element is under pressure is determined.

[0158] S606 generates a bitmap including multiple image regions based on the signal strength of the detection signals of each array element.

[0159] In this bitmap, each region corresponds to an array element, and different regions correspond to different array elements. The relative position between any two regions is the same as the relative position between the corresponding array elements. The pixel value of each region in the bitmap is related to the signal strength of the detection signal of the array element corresponding to each region.

[0160] For a multi-element sensor, the number of simulated detection signals may be large. To improve the training efficiency of training the embodied intelligent system based on the detection signals, the embodiments of this application can perform... Figure 6 The embodiments described above.

[0161] The following provides an exemplary description of steps S601-S606.

[0162] Steps S601-S605 are similar to steps S101-S105 above, so they will not be described in detail again.

[0163] In step S606 above, in order to facilitate understanding of the concept of bitmap, this application embodiment provides a schematic diagram of bitmap.

[0164] See Figure 7 This is a schematic diagram of a bitmap provided in an embodiment of this application. The bitmap is generated based on the signal strength of the detection signals of each element on a 6*10 capacitive array sensor, and includes 6*10 regions. It can be seen that each region in the bitmap corresponds to one element on the multi-element sensor, and different regions correspond to different elements on the multi-element sensor. The relative position between any two regions is the same as the relative position between the corresponding elements. That is, when an element is located in the first row and first column of the multi-element sensor, the region corresponding to that element is also located in the first row and first column of the bitmap; when an element is located in the first row and second column of the multi-element sensor, the region corresponding to that element is also located in the first row and second column of the bitmap, and so on.

[0165] The pixel value of each region in the bitmap is positively correlated with the signal strength of the detection signal of the corresponding array element. That is, the greater the signal strength of the detection signal of the array element, the greater the pixel value of the corresponding region in the bitmap, which can be represented as a darker color in that region. Figure 7(The color depth is represented by lines filling the bitmap area). Alternatively, the pixel value of each area in the bitmap can be negatively correlated with the signal strength of the detection signal of the corresponding array element. That is, the stronger the signal strength of the detection signal of the array element, the smaller the pixel value of the corresponding area in the bitmap, which can be represented as a lighter color in that area.

[0166] In this embodiment, the electronic device can generate a bitmap including multiple image regions based on the signal strength of the detection signals of each element on the multi-element sensor. In other words, the detection signals detected by multiple elements are integrated into a single bitmap. After obtaining the bitmap representing the signal strength of the detection signals of each element on the multi-element sensor, the embodied intelligent system can be trained directly using the bitmap. Compared with training using the detection signals detected by multiple elements individually, this can improve the training efficiency of training the embodied intelligent system based on the detection signals.

[0167] As can be seen from the above embodiments, in the embodiments of this application, bitmaps are used to represent the signal strength of the detection signals of each element on the multi-element sensor, thereby improving the training efficiency of training the embodied intelligent system based on the detection signals.

[0168] See Figure 8 This is a flowchart illustrating the fourth embodied intelligence simulation method provided in this application embodiment. The method includes steps S801-S807, wherein steps S801-S805 are the same as steps S101-S105, and steps S806-S807 are newly added steps.

[0169] S801 displays a three-dimensional virtual scene.

[0170] Among them, the three-dimensional virtual scene includes a carrier of embodied intelligence;

[0171] S802, in response to a layout command input for a 3D virtual scene, sets virtual geometry at multiple locations indicated by the layout command.

[0172] Among them, the layout instructions are used to indicate the position of each element in the multi-element sensor of the carrier, and there is no collision volume in each virtual assembly.

[0173] S803 controls the simulated movement of the carrier in a three-dimensional virtual scene and controls each virtual geometry to follow the movement of the carrier.

[0174] S804, during the simulated motion of the carrier, determines the contact depth between each virtual geometry and the objects in the three-dimensional virtual scene, which serves as the corresponding contact depth for each virtual geometry.

[0175] S805 calculates the pressure on each array element based on the contact depth corresponding to the virtual geometry located at the array element, and determines the detection signal detected when the array element is under pressure.

[0176] S806 determines the target noise algorithm corresponding to the type of multi-element sensor based on the preset correspondence between sensor type and noise algorithm.

[0177] S807 adds noise to the detection signals of each array element according to the target noise algorithm.

[0178] Since the detection signal of a real sensor contains noise when it comes into contact with an object, in order to make the detection signals of each array element obtained by simulation closer to the real detection signal, the embodiments of this application can perform... Figure 8 The example shown.

[0179] Steps S801-S807 are described below by example.

[0180] Steps S801-S805 are similar to steps S101-S105 above, so they will not be described in detail again.

[0181] In step S806 above, a correspondence between sensor type and noise algorithm is pre-stored. The electronic device can determine the target noise algorithm corresponding to the type of multi-element sensor based on this correspondence.

[0182] In one embodiment of this application, the electronic device can determine the dimension of the detected signal based on the type of the multi-element sensor. For example, a one-dimensional signal, a two-dimensional signal, and so on. For a one-dimensional signal, the corresponding noise algorithm can be to perturb the curve corresponding to the one-dimensional signal above and below. For a two-dimensional signal, the corresponding noise algorithm can be to perturb the surface corresponding to the two-dimensional signal above, below, left, and right. The perturbation can be a direct numerical perturbation or a perturbation based on gradient / reciprocal. The noise distribution sampling can also employ different methods, such as random scattering, uniform random sampling, Poisson disk sampling, etc.

[0183] In step S807 above, after determining the target noise algorithm corresponding to the type of multi-element sensor, the electronic device can add noise to the detection signal of each element. The detection signal after adding noise is closer to the real detection signal. The embodied intelligent system can be trained using the detection signal after adding noise, thereby improving the accuracy of the embodied intelligent agent when making decisions.

[0184] As can be seen from the above embodiments, in the embodiments of this application, noise is added to the detection signal of each array element according to the target noise algorithm corresponding to the type of multi-element sensor, so that the simulated detection signal is closer to the real detection signal.

[0185] The following section, in conjunction with the system architecture diagram of the embodied intelligence system, explains the improvements proposed in the embodiments of this application for the embodied intelligence system.

[0186] See Figure 9 This diagram illustrates the system architecture of an embodied intelligent system according to an embodiment of this application. The solid lines in the diagram represent prior art, which will not be elaborated upon further. The dashed lines represent the improvements proposed in this application to existing embodied intelligent systems.

[0187] The simulated asset database section in the diagram corresponds to the stage of building a 3D virtual scene.

[0188] At this stage, this application improves the sensor type. In the prior art, the contact point generated during sensor simulation is usually only a single point. However, this application, through the concept of differentiation, uses multiple virtual geometric shapes to simulate a multi-element sensor, thereby increasing the number of contact points and enabling a more realistic simulation of the working principle of the contact sensor.

[0189] At this stage, this application also provides a self-developed standard, namely a pre-set database for accurate physical simulation. This database can store object data packages, and based on these object data packages, objects with the shapes and physical rules specified in the data packages can be generated. This enables standardized and automated creation of simulation resources, thereby improving the efficiency of building 3D virtual environments. See details in [link to relevant documentation]. Figure 3 Description of the embodiments.

[0190] The diagram shows the core component section, which corresponds to the stage of simulating the sensor.

[0191] In this stage, this application improves the sensor simulation process. In existing technologies, limited by the performance of simulation software, only mechanical simulations are typically possible. This means simulating the mechanical signals when the sensor contacts objects in a virtual environment, using these signals as synthetic data for training embodied agents. However, compared to the original detection signals detected by the sensor, the mechanical signals contain less information. In this application, for each element of a multi-element sensor, the pressure on the element is calculated based on the contact depth corresponding to the virtual geometry located at the element. The detection signal detected when the element is under pressure is then determined. This detection signal can characterize the original detection signal detected by the sensor and has richer information. See details... Figure 1 Description of the Implementation Examples

[0192] In this stage, noise simulation was also performed. Noise was added to the detection signals of each array element to make the simulated detection signals closer to the real detection signals. See details in [link to relevant documentation]. Figure 8 Description of the embodiments.

[0193] Based on the same inventive concept, this application also provides a simulation device with embodied intelligence.

[0194] See Figure 10 This is a schematic diagram of the structure of a simulation device for embodied intelligence provided in an embodiment of this application. The device includes:

[0195] A three-dimensional virtual scene display module 1001 is used to display a three-dimensional virtual scene, which includes a carrier of embodied intelligence.

[0196] The virtual geometry setting module 1002 is used to set virtual geometries at multiple locations indicated by the layout command in response to the layout command input to the three-dimensional virtual scene. The layout command is used to indicate the position of each element in the multi-element sensor of the carrier, and each virtual geometry has no collision volume.

[0197] The carrier control module 1003 is used to control the carrier to simulate movement in the three-dimensional virtual scene, and to control each virtual geometry to follow the movement of the carrier;

[0198] The contact depth determination module 1004 is used to determine the contact depth between each virtual geometry and an object in the three-dimensional virtual scene during the simulation motion of the carrier, and to use the contact depth of each virtual geometry as the contact depth of each virtual geometry.

[0199] The detection signal determination module 1005 is used to calculate the pressure of each array element based on the contact depth corresponding to the virtual geometry located at the array element, and to determine the detection signal detected when the array element is subjected to the pressure.

[0200] In one embodiment of this application, the apparatus may further include:

[0201] The object data packet display module is used to display preset object data packets, which include at least: the shape of the object and the physical rules that the object must follow when it moves;

[0202] The target environment data packet determination module is used to determine the target object data packet indicated by the simulation environment setup instruction in response to the simulation environment setup instruction input for each displayed object data packet;

[0203] The 3D virtual scene generation module is used to generate objects that have the shape in the target object data packet and comply with the physical rules in the target object data packet, thereby obtaining a 3D virtual scene.

[0204] In one embodiment of this application, the shape of the three-dimensional rigid body is a convex hull, and the physical rules in the object data package of the three-dimensional rigid body are used to represent: the stiffness of the three-dimensional rigid body, and the frictional force of the three-dimensional rigid body during simulated motion in a three-dimensional virtual scene; and / or

[0205] The 3D software body is a set of tetrahedrons. The physical rules in the object data package of the 3D software body are used to represent: the Poisson coefficient of the 3D software body, and the frictional force when the 3D software body simulates motion in a 3D virtual scene; and / or

[0206] The physical rules in the object data package for three-dimensional fluids are used to represent: the Reynolds number of the three-dimensional fluid; and / or

[0207] The physical rules in the object data package for three-dimensional gases are used to represent: the Reynolds number of the three-dimensional gas; and / or

[0208] The carrier is cylindrical in shape, and the physical rules in the object data package of the carrier are used to represent: the motor torque and motor gain of the carrier, and the friction force when the carrier moves in a simulated three-dimensional virtual scene.

[0209] In one embodiment of this application, the apparatus further includes:

[0210] The scene configuration data package display module is used to display preset scene configuration data packages; wherein, the scene configuration data packages include: lighting data packages, and / or camera data packages, and / or media data packages; the lighting data packages include: the emission spectrum of the light, and / or color temperature, and / or spectral range, and / or mask; the camera data packages include: the camera's color matching function curve, and / or color, and / or depth; the media data packages include: the density field of the medium, and / or radiation field;

[0211] The target configuration data packet determination module is used to determine the target configuration data packet indicated by the simulation environment configuration command in response to the simulation environment configuration command input for each displayed configuration data packet;

[0212] Generate the lights and / or cameras and / or media indicated by the target configuration data packet.

[0213] In one embodiment of this application, the apparatus may further include:

[0214] The bitmap generation module is used to generate a bitmap including multiple image regions based on the signal strength of the detection signal of each array element. Each region in the bitmap corresponds to an array element, and different regions correspond to different array elements. The relative position between any two regions is the same as the relative position between the corresponding array elements. The pixel value of each region in the bitmap is related to the signal strength of the detection signal of the array element corresponding to each region.

[0215] In one embodiment of this application, the apparatus may further include:

[0216] The target noise algorithm determination module is used to determine the target noise algorithm corresponding to the type of the multi-element sensor based on a preset correspondence between sensor types and noise algorithms.

[0217] The noise addition module is used to add noise to the detection signal of each array element according to the target noise algorithm.

[0218] This application also provides an electronic device, such as... Figure 11 As shown, it includes:

[0219] Memory 1101 is used to store computer programs;

[0220] The processor 1102 is used to implement the simulation method steps of any of the above-mentioned embodied intelligence when executing the program stored in the memory 1101.

[0221] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1102, the communication interface, and the memory 1101 communicating with each other via the communication bus.

[0222] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0223] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0224] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0225] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0226] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described embodied intelligence simulation methods.

[0227] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the embodied intelligence simulation methods described in the above embodiments.

[0228] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0229] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0230] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0231] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A simulation method for embodied intelligence, characterized in that, The method includes: Displaying a three-dimensional virtual scene, wherein the three-dimensional virtual scene includes a carrier of embodied intelligence; In response to a layout command input to the three-dimensional virtual scene, virtual geometries are respectively set at multiple locations indicated by the layout command, wherein the layout command is used to indicate the position of each element in the multi-element sensor of the carrier, and each virtual geometry has no collision volume; The carrier is controlled to simulate movement in the three-dimensional virtual scene, and each of the virtual geometric objects is controlled to follow the movement of the carrier; During the simulated motion of the carrier, the contact depth between each virtual geometry and the object in the three-dimensional virtual scene is determined, and is used as the contact depth corresponding to each virtual geometry. For each array element, the pressure of the array element is calculated based on the contact depth corresponding to the virtual geometry located at the array element, and the detection signal detected when the array element is subjected to the pressure is determined.

2. The method according to claim 1, characterized in that, The method further includes: Display preset object data packets, which include at least: the shape of the object and the physical rules that the object must follow when it moves; In response to the simulation environment setup command input for each displayed object data packet, determine the target object data packet indicated by the simulation environment setup command; An object with the shape in the target object data packet and conforming to the physical rules in the target object data packet is generated to obtain a three-dimensional virtual scene.

3. The method according to claim 2, characterized in that, The shape of the three-dimensional rigid body is a convex hull. The physical rules in the object data package of the three-dimensional rigid body are used to represent: the stiffness of the three-dimensional rigid body, and the frictional force of the three-dimensional rigid body during simulated motion in a three-dimensional virtual scene; and / or The 3D software body is a set of tetrahedrons. The physical rules in the object data package of the 3D software body are used to represent: the Poisson coefficient of the 3D software body, and the frictional force when the 3D software body simulates motion in a 3D virtual scene; and / or The physical rules in the object data package for three-dimensional fluids are used to represent: the Reynolds number of the three-dimensional fluid; and / or The physical rules in the object data package for three-dimensional gases are used to represent: the Reynolds number of the three-dimensional gas; and / or The carrier is cylindrical in shape, and the physical rules in the object data package of the carrier are used to represent: the motor torque and motor gain of the carrier, and the friction force when the carrier moves in a simulated three-dimensional virtual scene.

4. The method according to claim 2, characterized in that, The method further includes: The system displays preset scene configuration data packages; wherein, the scene configuration data packages include: lighting data packages, and / or camera data packages, and / or media data packages; the lighting data packages include: the emission spectrum of the light, and / or color temperature, and / or spectral range, and / or mask; the camera data packages include: the camera's color matching function curve, and / or color, and / or depth; the media data packages include: the density field of the medium, and / or radiation field; In response to the simulation environment configuration command input for each of the displayed configuration data packets, determine the target configuration data packet indicated by the simulation environment configuration command; Generate the lights and / or cameras and / or media indicated by the target configuration data packet.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Based on the signal strength of the detection signal of each array element, a bitmap comprising multiple image regions is generated. Each region in the bitmap corresponds to an array element, and different regions correspond to different array elements. The relative position between any two regions is the same as the relative position between the corresponding array elements. The pixel value of each region in the bitmap is related to the signal strength of the detection signal of the array element corresponding to each region.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: Based on the preset correspondence between sensor types and noise algorithms, the target noise algorithm corresponding to the type of the multi-element sensor is determined. According to the target noise algorithm, noise is added to the detection signal of each array element.

7. A simulation device with embodied intelligence, characterized in that, The device includes: A 3D virtual scene display module is used to display 3D virtual scenes, which include carriers of embodied intelligence. The virtual geometry setting module is used to set virtual geometries at multiple locations indicated by the layout instructions in response to the layout instructions input to the three-dimensional virtual scene. The layout instructions are used to indicate the position of each element in the multi-element sensor of the carrier, and each virtual geometry has no collision volume. The carrier control module is used to control the carrier to simulate movement in the three-dimensional virtual scene, and to control each virtual geometry to follow the movement of the carrier; The contact depth determination module is used to determine the contact depth between each virtual geometry and an object in the three-dimensional virtual scene during the simulated motion of the carrier, and to use the contact depth of each virtual geometry as its respective contact depth. The detection signal determination module is used to calculate the pressure of each array element based on the contact depth corresponding to the virtual geometry located at the array element, and to determine the detection signal detected when the array element is subjected to the pressure.

8. The apparatus according to claim 7, characterized in that, The device further includes: The object data packet display module is used to display preset object data packets, which include at least: the shape of the object and the physical rules that the object must follow when it moves; The target environment data packet determination module is used to determine the target object data packet indicated by the simulation environment setup instruction in response to the simulation environment setup instruction input for each displayed object data packet; A 3D virtual scene generation module is used to generate objects that have the shape in the target object data packet and conform to the physical rules in the target object data packet, thereby obtaining a 3D virtual scene; and / or The shape of the three-dimensional rigid body is a convex hull. The physical rules in the object data package of the three-dimensional rigid body are used to represent: the stiffness of the three-dimensional rigid body, and the frictional force of the three-dimensional rigid body during simulated motion in a three-dimensional virtual scene; and / or The 3D software body is a set of tetrahedrons. The physical rules in the object data package of the 3D software body are used to represent: the Poisson coefficient of the 3D software body, and the frictional force when the 3D software body simulates motion in a 3D virtual scene; and / or The physical rules in the object data package for three-dimensional fluids are used to represent: the Reynolds number of the three-dimensional fluid; and / or The physical rules in the object data package for three-dimensional gases are used to represent: the Reynolds number of the three-dimensional gas; and / or The carrier is cylindrical in shape, and the physical rules in the object data package of the carrier are used to represent: the motor torque and motor gain of the carrier, and the frictional force of the carrier during simulated motion in a three-dimensional virtual scene; and / or The device further includes: The scene configuration data package display module is used to display preset scene configuration data packages; wherein, the scene configuration data packages include: lighting data packages, and / or camera data packages, and / or media data packages; the lighting data packages include: the emission spectrum of the light, and / or color temperature, and / or spectral range, and / or mask; the camera data packages include: the camera's color matching function curve, and / or color, and / or depth; the media data packages include: the density field of the medium, and / or radiation field; The target configuration data packet determination module is used to determine the target configuration data packet indicated by the simulation environment configuration command in response to the simulation environment configuration command input for each displayed configuration data packet; The lights and / or cameras and / or media indicated by the generated target configuration data packet; and / or The device further includes: The bitmap generation module is used to generate a bitmap comprising multiple image regions based on the signal strength of the detection signal of each array element. Each region in the bitmap corresponds to one array element, and different regions correspond to different array elements. The relative position between any two regions is the same as the relative position between their corresponding array elements. The pixel value of each region in the bitmap is related to the signal strength of the detection signal of the array element corresponding to that region; and / or The device further includes: The target noise algorithm determination module is used to determine the target noise algorithm corresponding to the type of the multi-element sensor based on a preset correspondence between sensor types and noise algorithms. The noise addition module is used to add noise to the detection signal of each array element according to the target noise algorithm.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images