Robot model simulation method and device, electronic equipment and storage medium

Abstract

The invention relates to a robot model simulation method and device, electronic equipment and a storage medium, and belongs to the technical field of robot model emulation.The method comprises the steps that corresponding sensing data are obtained in real time based on different sensor equipment on a robot, and therefore the data can be updated in real time according to the requirements of the robot; performing compression processing on the sensing data of different sensors to obtain compressed data; and performing model construction on the compressed data in the three-dimensional scene based on a Unity engine to obtain a target three-dimensional model, and performing real-time simulation on the robot according to the target three-dimensional model, so that the target three-dimensional model of the robot can be constructed according to the obtained sensing data, and real-time simulation on the robot can be performed according to the target three-dimensional model. And the simulation precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot model simulation, and in particular to a robot model simulation method, device, electronic equipment and storage medium. Background Art

[0002] With the development of computer technology, virtual simulation technology has gradually become a system of its own, becoming the third basic method for humans to understand the objective laws of nature after mathematical reasoning and scientific experiments, and is developing into a universal and strategic technology for humans to understand, transform and create the objective world. Mobile robots are given more and more functions, and have made great leaps in functions and technologies, and can complete many complex tasks. Mobile robots are also increasingly used in industrial production, family life, communities, hospitals and other scenarios to complete tasks. Then, how to make robots have intelligent behavior through self-research, self-learning, imitation and replication has become a major problem in research institutes, academia, enterprises and other fields.

[0003] In terms of complex task operations for intelligent systems, it is necessary to integrate the information from the robot system and the local geographic information to achieve the integration of simulation, supervision, and operation of the robot system. In the existing technology, it is impossible to modify the data at will according to one's own needs, which leads to low simulation accuracy.

[0004] Therefore, there is an urgent need to propose a robot model simulation method, device, electronic device and storage medium to solve the technical problem in the prior art that the current data cannot be modified arbitrarily according to one's own needs, resulting in low simulation accuracy. Summary of the invention

[0005] In view of this, it is necessary to provide a robot model simulation method, device, electronic device and storage medium to solve the technical problem in the prior art that data cannot be modified arbitrarily according to one's own needs, resulting in low simulation accuracy.

[0006] In order to solve the above problems, the present invention provides a robot model simulation method, comprising: Acquire corresponding sensor data based on different sensor devices on the robot; compressing the sensing data of the different sensors to obtain compressed data; The compressed data is modeled in a three-dimensional scene based on the Unity engine to obtain a target three-dimensional model, and the robot is simulated in real time according to the target three-dimensional model.

[0007] In a possible implementation manner, compressing the sensing data of the different sensors to obtain compressed data includes: Integrating the sensor data based on a fusion algorithm to obtain integrated data; Preprocessing the integrated data to obtain preprocessed data; The preprocessed data is compressed to obtain compressed data.

[0008] In a possible implementation manner, preprocessing the integrated data to obtain preprocessed data includes: filtering the integrated data to obtain filtered data; De-noising the filtered data to obtain de-noised data; The denoised data is registered to obtain preprocessed data.

[0009] In a possible implementation, constructing a model of the compressed data in a three-dimensional scene based on a Unity engine to obtain a target three-dimensional model includes: The dynamic modeling system based on the Unity engine constructs a model for the compressed data to obtain an initial three-dimensional model; The initial three-dimensional model is rendered with materials and simulated with lighting by using the real-time rendering technology of the Unity engine to obtain a target three-dimensional model.

[0010] In a possible implementation, the real-time simulation of the robot according to the target three-dimensional model includes: Managing the data of the target three-dimensional model based on a preset data structure to obtain a model data structure; The target three-dimensional model is updated and simulated in real time according to the model data structure.

[0011] In a possible implementation, the method further includes: Set up conflict resolution strategies; When a conflict occurs in data transmission between the Unity engine and the robot, the data is processed according to the conflict resolution strategy to obtain conflict-free data.

[0012] In a possible implementation, the method further includes: The connection interfaces and connection protocols between the different sensor devices and the robot are uniformly processed.

[0013] On the other hand, the present invention also provides a robot model simulation device, comprising: A data acquisition module is used to acquire corresponding sensor data based on different sensor devices on the robot; A data compression module, used for compressing the sensing data of the different sensors to obtain compressed data; The real-time simulation module is used to construct a model of the compressed data in a three-dimensional scene based on the Unity engine to obtain a target three-dimensional model, and to perform real-time simulation of the robot according to the target three-dimensional model.

[0014] On the other hand, an embodiment of the present invention discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the various steps of the above-mentioned robot model simulation method embodiment when executed by the processor.

[0015] On the other hand, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the above-mentioned robot model simulation method embodiment are implemented.

[0016] The beneficial effects of the present invention are: based on different sensor devices on the robot, corresponding sensor data is acquired in real time, so that the data can be updated in real time according to the robot's own needs; the sensor data of different sensors are compressed to obtain compressed data; based on the Unity engine, a model is constructed for the compressed data in a three-dimensional scene to obtain a target three-dimensional model, and the robot is simulated in real time according to the target three-dimensional model, so that the target three-dimensional model of the robot can be constructed according to the acquired sensor data, so that the robot can be simulated in real time according to the target three-dimensional model, thereby improving the simulation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic flow chart of an embodiment of the robot model simulation method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic flow chart of an embodiment of step S102; Figure 3 For the present invention Figure 2 A schematic flow chart of an embodiment of step S202; Figure 4 A schematic structural diagram of an embodiment of a robot model simulation device provided by the present invention; Figure 5 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0019] like Figure 1As shown, a specific embodiment of the present invention discloses a robot model simulation method, comprising: S101, acquiring corresponding sensor data based on different sensor devices on the robot; S102, compressing the sensing data of different sensors to obtain compressed data; S103, constructing a model of the compressed data in a three-dimensional scene based on the Unity engine to obtain a target three-dimensional model, and performing real-time simulation of the robot according to the target three-dimensional model.

[0020] It should be understood that: the robot can be equipped with a plurality of different sensor devices, such as high-precision laser radar, depth camera and high-definition camera, etc. The specific installation method and installation position can be set according to the actual situation, and the embodiment of the present invention is not limited here. Among them, each sensor can obtain sensor data in a different way, for example, the high-precision laser radar obtains point cloud data, the depth camera obtains image data, and the high-definition camera obtains video data. The specific acquisition method can be set according to different sensors, and the embodiment of the present invention is not limited here.

[0021] In a specific embodiment of the present invention, the sensing data corresponding to each sensor can be obtained through different sensor devices on the robot, integrated on the robot platform, and all-round scanning and data collection of the surrounding environment can be realized. The sensing data of different sensors can also be compressed to obtain compressed data of all data. The specific compression process can be set according to the actual situation, and the embodiment of the present invention is not limited here. After the data is compressed, the compressed data can be processed based on the Unity engine, so that the model can be constructed in the three-dimensional scene through the compressed data to obtain the target model corresponding to the robot, and then the Unity engine can be used to transmit data with the robot to realize robot simulation and remote monitoring.

[0022] Compared with the prior art, the present embodiment provides real-time acquisition of corresponding sensor data based on different sensor devices on the robot, so that the data can be updated in real time according to the robot's own needs; the sensor data of different sensors are compressed to obtain compressed data; based on the Unity engine, a model is constructed for the compressed data in a three-dimensional scene to obtain a target three-dimensional model, and the robot is simulated in real time according to the target three-dimensional model, so that the target three-dimensional model of the robot can be constructed according to the acquired sensor data, so that the robot can be simulated in real time according to the target three-dimensional model, thereby improving the accuracy of the simulation.

[0023] In some embodiments of the present invention, Figure 2 As shown, step S102 includes: S201, integrating the sensor data based on a fusion algorithm to obtain integrated data; S202, preprocessing the integrated data to obtain preprocessed data; S203: compress the preprocessed data to obtain compressed data.

[0024] In a specific embodiment of the present invention, the sensor data corresponding to multiple sensors can be integrated based on the sensor data fusion algorithm, so as to obtain integrated data. The sensor data fusion algorithm may include random methods and artificial intelligence methods. Random methods may include weighted average method, Kalman filter method, multi-Bayesian estimation method, Dempster-Shafer (DS) evidence reasoning and production rules. Artificial intelligence methods may include fuzzy logic theory, artificial neural network method, rough set theory and expert system. These methods have their own characteristics and are suitable for different application scenarios. For example, in autonomous driving, commonly used fusion algorithms include weighted average method, Kalman filter method and neural network, which are intuitive, fault-tolerant and universal. In addition, the sensor information fusion method also includes four basic methods: combination, synthesis, fusion and correlation. The sensor data fusion algorithm in the embodiment of the present invention can be set according to actual conditions, and the embodiment of the present invention is not limited here. After obtaining the integrated data, the integrated data can also be preprocessed to obtain preprocessed data. The specific preprocessing process can be set according to actual conditions, and the embodiment of the present invention is not limited here. An efficient data compression algorithm can also be used to compress the preprocessed data to obtain compressed data, thereby reducing the amount of data transmitted and improving transmission efficiency. Among them, the data compression algorithm mainly includes a lossless compression algorithm and a lossy compression algorithm. Data compression refers to a technical method that reduces the amount of data to reduce storage space and improve its transmission, storage and processing efficiency without losing useful information. The lossless compression algorithm can reduce the size of the file without losing any information. The lossy compression algorithm reduces the file size by discarding some unimportant data, so the original data cannot be completely restored. The specific data compression algorithm in the embodiment of the present invention can be set according to the actual situation, and the embodiment of the present invention is not limited here.

[0025] In some embodiments of the present invention, Figure 3 As shown, step S202 includes: S301, filtering the integrated data to obtain filtered data; S302, denoising the filtered data to obtain denoised data; S303: register the denoised data to obtain preprocessed data.

[0026] In a specific embodiment of the present invention, the process of preprocessing the integrated data may include preprocessing operations such as filtering, denoising, and alignment to ensure the accuracy of the data. Specifically, the integrated data may be first filtered to obtain filtered data, and then the filtered data may be denoised to obtain denoised data, and then the denoised data may be aligned to obtain preprocessed data. Other processing processes may be added during the preprocessing process, such as normalization and other operations, which may be set according to actual conditions and are not limited in the embodiments of the present invention.

[0027] In some embodiments of the present invention, step S103 includes: The dynamic modeling system based on the Unity engine builds a model for the compressed data to obtain an initial three-dimensional model; The real-time rendering technology based on the Unity engine is used to perform material rendering and lighting simulation on the initial 3D model to obtain the target 3D model.

[0028] In a specific embodiment of the present invention, a dynamic modeling system is designed in the Unity engine to receive scanning data transmitted in real time, that is, compressed data, and quickly build or update a three-dimensional model based on the compressed data, so as to obtain an initial three-dimensional model. Then, the real-time rendering technology of the Unity engine, such as Shader and Material technology, can be used to perform material rendering and lighting simulation on the model to obtain a target three-dimensional model. The three-dimensional model processed by Shader and Material technology is more realistic.

[0029] In some embodiments of the present invention, step S103 includes: Managing the data of the target three-dimensional model based on a preset data structure to obtain a model data structure; The target three-dimensional model is updated and simulated in real time according to the model data structure.

[0030] In a specific embodiment of the present invention, preset data structures such as octrees and KD trees can also be used to manage the three-dimensional spatial data of the target three-dimensional model, thereby obtaining a model data structure that is easy to manage, so that the target three-dimensional model can be updated and simulated in real time through the model data structure, thereby optimizing the model update and rendering performance.

[0031] In some embodiments of the present invention, the method further comprises: Set up conflict resolution strategies; When a conflict occurs in the data transmission between the Unity engine and the robot, the data is processed according to the conflict resolution strategy to obtain conflict-free data.

[0032] In a specific embodiment of the present invention, a conflict resolution strategy can be preset, and the conflict resolution strategies for data transmission mainly include the following: Version control: Each data item is accompanied by a version number, and each modification will update the version number. When resolving conflicts, the data with a larger version number is selected as the latest version. Timestamp: Each data item has a timestamp, which indicates its update time. When resolving conflicts, the data with a later timestamp is selected as the latest version. Atomic operation: Atomic operation is used to ensure the atomicity of data modification, that is, either all executions are successful or all fail, avoiding concurrent conflicts. Distributed lock: By introducing distributed locks, it is ensured that only one node can operate the modification of data to avoid conflicts caused by simultaneous modifications of other nodes. Coordinator: A coordinator is introduced to schedule and coordinate data modifications between different nodes. All requests are first sent to the coordinator, which makes a unified decision and then assigns them to the corresponding nodes for processing. Vector clock: The number of times a node modifies the data is recorded by the vector clock, which is used to resolve data conflicts between different nodes. These strategies can select appropriate strategies to solve data conflict problems according to specific application scenarios and data transmission requirements. Conflicts and inconsistencies may occur during the data transmission and synchronization between the Unity engine and the robot. However, when conflicts occur, the data can be processed to the extreme through conflict resolution strategies, so that the data can be transmitted without conflicts.

[0033] Furthermore, efficient data transmission protocols can be designed, such as TCP, UDP, HTTP, FTP, SMTP, Telnet, DNS, etc., to ensure that scanning data, control instructions and modeling data can be transmitted between the robot and the Unity engine with low latency and high reliability. A two-way communication mechanism can also be implemented to ensure real-time synchronization between the model status in the Unity engine and the real robot.

[0034] In some embodiments of the present invention, the method further comprises: The connection interfaces and connection protocols between different sensor devices and robots are uniformly handled.

[0035] In a specific embodiment of the present invention, standardized interfaces and protocols (such as ROS, OPC UA, etc.) are used to uniformly process the connection interfaces and connection protocols between different sensor devices and robots, ensuring that the present invention is compatible with different types of robots and scanning devices.

[0036] Furthermore, the modular design concept is adopted to divide the system into reusable components and modules, which is convenient for users to customize and expand according to actual needs. Rich documentation and sample codes can also be provided to help users quickly integrate and deploy.

[0037] The embodiment of the present invention can realize fast and accurate scanning of the robot's surrounding environment by integrating high-performance scanning equipment and real-time data processing algorithms, and transmit the data to the Unity engine in real time for processing. This process greatly shortens the delay of data processing and model updating, ensures the real-time and accuracy of the simulation environment, and provides a more reliable basis for robot decision-making. High-precision scanning equipment and advanced modeling technology can be used to build a highly detailed and realistic three-dimensional model. By optimizing material rendering and lighting simulation, the model can more accurately reflect the lighting, shadow and material effects in the real environment, thereby improving the user experience and the reliability of the simulation results. With the help of efficient data compression algorithms and optimized data transmission protocols, the embodiment of the present invention significantly reduces the amount of data transmission and improves the efficiency of data transmission. This not only reduces the occupation of network bandwidth, but also reduces the delay and packet loss rate during data transmission, ensuring the integrity and real-time of the data. Standardized interface and protocol design are adopted, which can be compatible with different types of robots and scanning equipment. At the same time, the modular system architecture allows users to customize and expand according to actual needs, thereby improving the flexibility and scalability of the system. In addition, rich documentation and sample codes provide users with a convenient integration and deployment experience. By integrating the mature Unity engine and efficient algorithm library, the embodiments of the present invention reduce the difficulty and cost of development. At the same time, the modular design makes system maintenance more convenient and reduces the cost and risk of long-term operation. The embodiments of the present invention provide strong technical support for robot simulation and remote monitoring, which helps to promote the innovation and development of robot technology. Through a real-time and accurate simulation environment, researchers and developers can have a deeper understanding of the behavioral characteristics of robots, optimize algorithm design, and improve the intelligence level of robots. At the same time, this technology can also be widely used in industrial automation, intelligent manufacturing, medical rehabilitation and other fields, providing strong support for industrial upgrading and transformation.

[0038] In order to better implement the robot model simulation method in the embodiment of the present invention, based on the robot model simulation method, the embodiment of the present invention also provides a robot model simulation device, such as Figure 4 As shown, the robot model simulation device 400 includes: A data acquisition module 401 is used to acquire corresponding sensor data based on different sensor devices on the robot; The data compression module 402 is used to compress the sensing data of different sensors to obtain compressed data; The real-time simulation module 403 is used to construct a model of the compressed data in a three-dimensional scene based on the Unity engine, obtain a target three-dimensional model, and perform real-time simulation of the robot according to the target three-dimensional model.

[0039] The robot model simulation device 400 provided in the above embodiment can implement the technical solution described in the above robot model simulation method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above robot model simulation method embodiment, which will not be repeated here.

[0040] like Figure 5 As shown, the present invention also provides an electronic device 500. The electronic device 500 includes a processor 501, a memory 502 and a display 503. Figure 5 Only some components of the electronic device 500 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0041] In some embodiments, the memory 502 may be an internal storage unit of the electronic device 500, such as a hard disk or memory of the electronic device 500. In other embodiments, the memory 502 may also be an external storage device of the electronic device 500, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 500.

[0042] Furthermore, the memory 502 may include both an internal storage unit of the electronic device 500 and an external storage device. The memory 502 is used to store application software installed in the electronic device 500 and various data.

[0043] In some embodiments, the processor 501 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 502, such as the robot model simulation method of the present invention.

[0044] In some embodiments, the display 503 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 503 is used to display information of the electronic device 500 and to display a visual user interface. The components 501-503 of the electronic device 500 communicate with each other through a system bus.

[0045] In some embodiments of the present invention, when the processor 501 executes the robot model simulation program in the memory 502, the following steps may be implemented: Acquire corresponding sensor data based on different sensor devices on the robot; Compress the sensing data of different sensors to obtain compressed data; Based on the Unity engine, the compressed data is modeled in a three-dimensional scene to obtain a target three-dimensional model, and the robot is simulated in real time according to the target three-dimensional model.

[0046] It should be understood that: when the processor 501 executes the robot model simulation program in the memory 502, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0047] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 500 mentioned, and the electronic device 500 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 500 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0048] Correspondingly, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, it can implement the robot model simulation method steps or functions provided in the above-mentioned method embodiments.

[0049] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0050] The robot model simulation method, device, electronic device and storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A robot model simulation method, characterized in that: include: Acquire corresponding sensor data based on different sensor devices on the robot; compressing the sensing data of the different sensors to obtain compressed data; The compressed data is modeled in a three-dimensional scene based on the Unity engine to obtain a target three-dimensional model, and the robot is simulated in real time according to the target three-dimensional model.

2. The robot model simulation method according to claim 1, characterized in that: The compressing the sensing data of the different sensors to obtain compressed data includes: Integrating the sensor data based on a fusion algorithm to obtain integrated data; Preprocessing the integrated data to obtain preprocessed data; The preprocessed data is compressed to obtain compressed data.

3. The robot model simulation method according to claim 2, characterized in that: The preprocessing of the integrated data to obtain preprocessed data includes: filtering the integrated data to obtain filtered data; De-noising the filtered data to obtain de-noised data; The denoised data is registered to obtain preprocessed data.

4. The robot model simulation method according to claim 1, characterized in that: The step of constructing a model of the compressed data in a three-dimensional scene based on the Unity engine to obtain a target three-dimensional model includes: The dynamic modeling system based on the Unity engine constructs a model for the compressed data to obtain an initial three-dimensional model; The initial three-dimensional model is rendered with materials and simulated with lighting by using the real-time rendering technology of the Unity engine to obtain a target three-dimensional model.

5. The robot model simulation method according to claim 1, characterized in that: The real-time simulation of the robot according to the target three-dimensional model includes: Managing the data of the target three-dimensional model based on a preset data structure to obtain a model data structure; The target three-dimensional model is updated and simulated in real time according to the model data structure.

6. The robot model simulation method according to claim 1, characterized in that: The method further comprises: Set up conflict resolution strategies; When a conflict occurs in data transmission between the Unity engine and the robot, the data is processed according to the conflict resolution strategy to obtain conflict-free data.

7. The robot model simulation method according to claim 1, characterized in that: The method further comprises: The connection interfaces and connection protocols between the different sensor devices and the robot are uniformly processed.

8. A robot model simulation device, characterized in that: include: A data acquisition module is used to acquire corresponding sensor data based on different sensor devices on the robot; A data compression module, used for compressing the sensing data of the different sensors to obtain compressed data; The real-time simulation module is used to construct a model of the compressed data in a three-dimensional scene based on the Unity engine to obtain a target three-dimensional model, and to perform real-time simulation of the robot according to the target three-dimensional model.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the robot model simulation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot model simulation method according to any one of claims 1 to 7 are implemented.

Images