Intelligent robot testing method and system based on virtual environment, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of robot testing, and provides an intelligent robot testing method and system based on a virtual environment, electronic equipment and a storage medium. The method comprises the steps of building data acquisition, virtual test environment reconstruction, digital twin robot construction, multi-dimensional test task setting, quantitative evaluation formula system construction, virtual test and quantitative evaluation. According to the method, the three-dimensional reconstruction model is improved, the multi-dimensional point cloud data is utilized for collaborative reconstruction, and the actual remote sensing image is utilized for dynamic adjustment of the reconstruction structure, so that a large amount of manual reconstruction data is not used, the situation that the model reconstruction effect is reduced due to unsupervised reconstruction is avoided, and the test scene reduction degree is greatly improved; the deviation between test and actual operation is reduced; by coupling and binding the digital twin robot and the target robot, the problem that the reliability of the test result is low due to excessive idealized setting is avoided, and the test result better fits the actual working condition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot testing, in particular to an intelligent robot testing method and system based on a virtual environment, an electronic device and a storage medium. BACKGROUND

[0002] The existing intelligent robot evaluation system takes standard, tool and scene adaptation as the core, focuses on technology landing and compliance to build a multi-dimensional framework: on the functional performance level, a hierarchical index system and dynamic weight adaptation are used to meet different needs in terms of motion accuracy, perception range and decision efficiency; in terms of safety and ethics, physical safety is controlled according to standards such as ISO10218 and GB9706, power and biocompatibility, and the ethical level emphasizes data privacy, algorithm transparency and responsibility division; in scene evaluation, industry focuses on digital twin verification and operation accuracy, service robots evaluate voice interaction and emotional companionship effect, and extreme environment testing adaptability and anti-interference; tools rely on high-fidelity simulation, hardware-in-the-loop testing and multi-modal fusion framework, combined with intelligent agent-driven evaluation to reduce costs; the whole system complies with international standards such as ISO / ANSI and domestic specifications to achieve precise matching of technical indicators and actual needs.

[0003] The existing intelligent robot testing method is mostly based on fixed standard procedures, but the testing process is risky when combined with complex or dangerous scenes, and there are the following problems: 1) traditional virtual testing relies on manual modeling or simple data collection, lacks accurate restoration of real scene structure details and material properties, resulting in large deviations between testing environment and actual working conditions; 2) traditional simulation testing often uses pure virtual robots, directly transplanting algorithms to virtual robots, but this process ignores factors such as delay, error and interference during actual robot operation, and the overly ideal response results in low reliability of the test; 3) existing entity testing methods are limited by environmental requirements and cannot cover multiple testing scene requirements; 4) traditional reconstruction models require a large amount of labeled data for pre-training, increasing time costs, and the model cannot be dynamically adjusted. SUMMARY

[0004] In order to overcome the deficiencies of the prior art, the purpose of the present application is to provide an intelligent robot testing method, system, electronic device and storage medium based on a virtual environment, which solves the problems of large environmental deviation, overly ideal simulation, inability to cover multiple testing scene requirements, and the need for a large amount of pre-constructed data for training of a reconstruction model.

[0005] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0006] An intelligent robot testing method based on a virtual environment, comprising:

[0007] Collecting building data;

[0008] The building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment;

[0009] Construct a digital twin robot based on the structure of the target robot;

[0010] Set up multi-dimensional test tasks based on the test project;

[0011] Construct a quantitative evaluation formula system;

[0012] The digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data;

[0013] The virtual test data is calculated using the aforementioned quantitative evaluation formula system to obtain the test results of the intelligent robot.

[0014] Preferably, the collection of building data includes:

[0015] By setting the acquisition angle, acquisition height, and distance from the center of the power station, several sets of acquisition parameter combinations are obtained; the acquisition height is greater than the building height of the target substation; the distance from the center of the power station is greater than the building radius of the target substation.

[0016] Data is collected from the target substation using the camera and radar in four orthogonal directions according to the collection parameter combination, resulting in one-way data. The four one-way data corresponding to the same collection parameter combination are set as a set of reconstructed data input. The one-way data includes: remote sensing images and remote sensing point clouds.

[0017] By integrating all the aforementioned reconstructed data inputs, the building data is obtained.

[0018] Preferably, a digital twin robot is constructed based on the structure of the target robot, including:

[0019] A three-dimensional model is created based on the geometric structure of the target robot to obtain the robot model;

[0020] Physical property mapping is performed on each component of the robot model based on the device parameters and device characteristics of the target robot;

[0021] A digital twin robot is obtained by setting virtual sensing devices on the robot model; the virtual sensing devices include: a visual sensor, a laser sensor, a pressure sensor, an angle sensor, and a sound sensor.

[0022] The data acquisition output interface of the target robot is replaced with the output interface of the virtual sensing device, and the command output interface of the target robot is embedded into the digital twin robot.

[0023] Preferably, the test items include: inspection path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, severe weather and nighttime travel stability test, equipment anomaly emergency response speed test, command coordination execution accuracy test, high-voltage area safety boundary compliance test, and full-condition endurance test; the severe weather includes: rainstorm weather, sleet weather, heavy snow weather, and typhoon weather.

[0024] Preferably, the expression of the quantitative evaluation formula system is: ;in, This is the overall test score; For the first Individual test weights; For the first Individual test scores; The evaluation formula is set according to the test items.

[0025] Preferably, the building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment, including:

[0026] An improved 3D reconstruction model is constructed; the improved 3D reconstruction model includes: an input layer, a matrix transformation layer, four parallel encoding layers, a feature fusion layer, a decoding layer, and an output layer connected in sequence;

[0027] Define the reconstruction loss function and the sliding window; the expression for the reconstruction loss function is:

[0028] ;in, ; The reconstruction loss value for a single sliding window; These are the loss weighting coefficients; For contour similarity function; Extracting contours within a single window of a virtual structure image; Extract the contour within a single window of the remote sensing image; These are similarity weight coefficients; , These are the shape similarity function and the positional overlap function, respectively.

[0029] The matrix transformation layer is used to perform orientation alignment transformation on the remote sensing point clouds in the four unidirectional data in the reconstructed data input to obtain unidirectional data input;

[0030] The four in-direction data inputs are input in parallel into four parallel coding layers for feature extraction, resulting in single-path features;

[0031] The four single-path features are fused using the feature fusion layer to obtain fused features;

[0032] The fused features are decoded using the decoding layer to obtain deconstructed features;

[0033] The deconstructed features are input into the output layer for mapping processing to obtain a three-dimensional reconstructed structure.

[0034] The virtual image of the three-dimensional reconstructed structure is acquired based on the combination of acquisition parameters to obtain the virtual structure image;

[0035] The similarity loss between the virtual structure image and the remote sensing image is calculated using the reconstruction loss function and the sliding window to obtain several window loss values;

[0036] The window loss values ​​corresponding to the four remote sensing images are integrated, and a weight map is generated based on the integration result to obtain an auxiliary weight map;

[0037] Based on the auxiliary weight map and the window loss value, the virtual structure image is iterated using the improved 3D reconstruction model to obtain the reconstructed structure of the substation;

[0038] The remote sensing image is used to identify objects, and the identification results are obtained.

[0039] The identification results are matched with a preset material property library to obtain matching results;

[0040] Based on the matching results, the material properties of the reconfigurable structure of the substation are set to obtain the virtual test environment.

[0041] Preferably, the digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data, including:

[0042] The constructed virtual test environment is invoked, and the digital twin robot is loaded into the virtual test environment through the test interface;

[0043] The multi-dimensional test tasks are sent to the target robot through a preset interface;

[0044] The algorithm embedded in the target robot is used to perform subtask splitting, sorting and instruction generation on the multi-dimensional test task to obtain the test task execution strategy, and the test task execution strategy is synchronously updated to the digital twin robot through a preset interface.

[0045] The virtual sensing device is used to collect environmental information, and the environmental information is synchronized to the target robot.

[0046] After the target robot responds to the environmental information, the digital twin robot is controlled to move using the control commands issued by the target robot, and the environmental information and the control commands are updated at a fixed frequency during the task execution.

[0047] After the multi-dimensional test task is completed, the indicator data of the digital twin robot during the task execution is integrated to obtain the virtual test data.

[0048] Preferably, a virtual environment-based intelligent robot testing system includes:

[0049] The data acquisition module is used to collect building data;

[0050] The structural reconstruction module is used to reconstruct the building data using an improved 3D reconstruction model to obtain a virtual testing environment.

[0051] A digital twin module is used to construct a digital twin robot based on the structure of the target robot.

[0052] The task setting module is used to set multi-dimensional test tasks according to the test project.

[0053] The quantitative evaluation module is used to construct a quantitative evaluation formula system;

[0054] The virtual testing module is used to load the digital twin robot into the virtual testing environment, test the digital twin robot according to multi-dimensional testing tasks, and obtain virtual test data.

[0055] The result output module is used to calculate the virtual test data using the quantitative evaluation formula system to obtain the test results of the intelligent robot.

[0056] Preferably, an electronic device includes: at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the aforementioned intelligent robot testing method based on a virtual environment.

[0057] Preferably, a non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the aforementioned intelligent robot testing method based on a virtual environment.

[0058] The present invention discloses the following technical effects:

[0059] This invention provides a method, system, electronic device, and storage medium for testing intelligent robots based on a virtual environment. By improving the 3D reconstruction model, utilizing multi-dimensional point cloud data for collaborative reconstruction, and dynamically adjusting the reconstructed structure using actual remote sensing images, it solves the problems caused by traditional virtual testing relying on manual modeling or simple data acquisition, and achieves reliable reconstruction of the 3D structure of distribution substations. By coupling and binding the digital twin robot and the target robot, it solves the problem of low reliability of test results caused by overly idealized settings, and achieves a testing process that is more in line with actual working conditions. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 A schematic diagram of a virtual environment-based intelligent robot testing process provided in an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the building data acquisition process provided in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the digital twin robot construction process provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of the virtual test environment generation process provided in an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of a virtual testing process provided in an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The purpose of this invention is to provide a method, system, electronic device and storage medium for testing intelligent robots based on a virtual environment, which solves the problems of large environmental deviation, overly idealized simulation, inability to cover multiple test scenario requirements and the need for a large amount of pre-built data for reconstructing model training in traditional methods.

[0068] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] Figure 1 This is a schematic diagram of a virtual environment-based intelligent robot testing process provided in an embodiment of the present invention, such as... Figure 1 As shown, this invention provides a method for testing intelligent robots based on a virtual environment, including:

[0070] Step 100: Collect building data;

[0071] Step 200: Reconstruct the building data using the improved 3D reconstruction model to obtain a virtual testing environment;

[0072] Step 300: Construct a digital twin robot based on the structure of the target robot;

[0073] Step 400: Set up multi-dimensional test tasks based on the test items;

[0074] Step 500: Construct a quantitative evaluation formula system;

[0075] Step 600: Load the digital twin robot into the virtual test environment, test the digital twin robot according to the multi-dimensional test tasks, and obtain virtual test data;

[0076] Step 700: Calculate the virtual test data using the quantitative evaluation formula system to obtain the intelligent robot test results.

[0077] refer to Figure 2 Collect building data, including:

[0078] Step 101: Set the acquisition angle, acquisition height, and distance from the center of the power station to obtain several sets of acquisition parameter combinations; the acquisition height is greater than the building height of the target substation; the distance from the center of the power station is greater than the building radius of the target substation.

[0079] Step 102: Using the camera and radar of the UAV, data is collected from the target substation in four orthogonal directions according to the collection parameter combination to obtain one-way data. The four one-way data corresponding to the same collection parameter combination are set as a set of reconstructed data input. The one-way data includes: remote sensing images and remote sensing point clouds.

[0080] Step 103: Integrate all the reconstructed data inputs to obtain the building data.

[0081] refer to Figure 3 Constructing a digital twin robot based on the structure of the target robot includes:

[0082] Step 301: Perform 3D modeling based on the geometric structure of the target robot to obtain the robot model;

[0083] Step 302: Map the physical properties of each component of the robot model according to the device parameters and device characteristics of the target robot;

[0084] Step 303: Set virtual sensing devices on the robot model to obtain the digital twin robot; the virtual sensing devices include: a visual sensor, a laser sensor, a pressure sensor, an angle sensor, and a sound sensor;

[0085] Step 304: Replace the data acquisition output interface of the target robot with the output interface of the virtual sensing device, and embed the command output interface of the target robot into the digital twin robot.

[0086] Optionally, the test items include: inspection path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, severe weather and nighttime travel stability test, equipment anomaly emergency response speed test, command coordination execution accuracy test, high-voltage area safety boundary compliance test, and full-condition endurance test; the severe weather includes: rainstorm weather, sleet weather, heavy snow weather, and typhoon weather.

[0087] Preferably, the expression of the quantitative evaluation formula system is: ;in, This is the overall test score; For the first Individual test weights; For the first Individual test scores; The evaluation formula is set according to the test items.

[0088] refer to Figure 4The building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment, including:

[0089] Step 201: Construct an improved 3D reconstruction model; the improved 3D reconstruction model includes: an input layer, a matrix transformation layer, four parallel encoding layers, a feature fusion layer, a decoding layer, and an output layer connected in sequence;

[0090] Step 202: Set the reconstruction loss function and the sliding window; the expression for the reconstruction loss function is:

[0091] ;in, ; The reconstruction loss value for a single sliding window; These are the loss weighting coefficients; For contour similarity function; Extracting contours within a single window of a virtual structure image; Extract the contour within a single window of the remote sensing image; These are similarity weight coefficients; , These are the shape similarity function and the positional overlap function, respectively.

[0092] Step 203: Use the matrix transformation layer to perform orientation alignment transformation on the remote sensing point clouds in the four unidirectional data in the reconstructed data input to obtain unidirectional data input;

[0093] Step 204: Input the four in-direction data into the four parallel coding layers in parallel to extract features and obtain single-path features;

[0094] Step 205: Use the feature fusion layer to fuse the four single-path features to obtain fused features;

[0095] Step 206: Decode the fused features using the decoding layer to obtain the deconstructed features;

[0096] Step 207: Input the deconstruction features into the output layer for mapping processing to obtain the three-dimensional reconstructed structure;

[0097] Step 208: Perform virtual image acquisition on the three-dimensional reconstructed structure according to the acquisition parameter combination to obtain the virtual structure image;

[0098] Step 209: Calculate the similarity loss between the virtual structure image and the remote sensing image using the reconstruction loss function and the sliding window to obtain several window loss values;

[0099] Step 210: Integrate the window loss values ​​corresponding to the four remote sensing images and generate a weight map based on the integration result to obtain an auxiliary weight map;

[0100] Step 211: Based on the auxiliary weight map and the window loss value, iterate the virtual structure image using the improved 3D reconstruction model to obtain the reconstructed structure of the substation;

[0101] Step 212: Use an image recognition model to perform object recognition on the remote sensing image to obtain the recognition result;

[0102] Step 213: Match the identification result with the preset material property library to obtain the matching result;

[0103] Step 214: Set the material properties of the substation reconfiguration structure according to the matching results to obtain the virtual test environment.

[0104] refer to Figure 5 The digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data, including:

[0105] Step 601: Invoke the constructed virtual test environment and load the digital twin robot into the virtual test environment through the test interface;

[0106] Step 602: Send the set multi-dimensional test task to the target robot through a preset interface;

[0107] Step 603: Use the algorithm embedded in the target robot to perform sub-task splitting, sorting and instruction generation on the multi-dimensional test task to obtain the test task execution strategy, and update the test task execution strategy to the digital twin robot through a preset interface.

[0108] Step 604: Collect environmental information using the virtual sensing device and synchronize the environmental information to the target robot;

[0109] Step 605: After the target robot responds to the environmental information, the digital twin robot is controlled to move using the control commands issued by the target robot, and the environmental information and the control commands are updated at a fixed frequency during the task execution.

[0110] Step 606: After the multi-dimensional test task is completed, integrate the indicator data of the digital twin robot during the task execution to obtain the virtual test data.

[0111] Specifically, this embodiment acquires building data including remote sensing images and remote sensing point clouds through a standardized UAV acquisition scheme, including: 1) setting three core acquisition parameters: acquisition angle, acquisition height, and distance from the center of the substation, combining them to form several sets of acquisition parameters. The acquisition height must be greater than the building height of the target substation, and the distance from the center of the substation must be greater than the building radius of the target substation to ensure that the acquisition range covers the entire substation; 2) conducting data acquisition in four orthogonal directions of the target substation, such as east, south, west, and north (the actual direction is set according to the direction of the main structure of the substation). According to each preset set of acquisition parameters, the camera and radar on the UAV are used to acquire data synchronously to obtain remote sensing images and remote sensing point clouds. The four sets of unidirectional data corresponding to the four orthogonal directions under the same acquisition parameter combination are grouped into one set of reconstructed data input; 3) collecting the reconstructed data input corresponding to all acquisition parameter combinations and integrating them to form building data.

[0112] Furthermore, this embodiment improves upon PointNet by designing an improved 3D reconstruction model. This improved model accurately adapts to the generation requirements of virtual testing environments. By integrating PointNet's point cloud processing advantages with the original multi-directional data collaboration logic, it further enhances the detail reproduction of the substation reconstruction structure and the accuracy of material property matching. The PointNet encoding layer structure is changed to four parallel encoding layers, and the original loss function is also replaced. The specific structure of the improved 3D reconstruction model is as follows: input layer, preprocessing layer, matrix transformation layer, four parallel encoding layers, feature fusion layer, decoding layer, and output layer. The specific functions of each network layer are as follows:

[0113] 1) Preprocessing layer: PointNet standard preprocessing is performed on the remote sensing point cloud in the preprocessing layer, including point cloud sampling and normalization.

[0114] 2) Matrix transformation layer: The remote sensing point cloud of four unidirectional data is transformed by a 3×3 transformation matrix to perform orientation alignment transformation and output point cloud data in the same direction.

[0115] 3) Four parallel coding layers: The MLP network of PointNet is used to extract local geometric features of point cloud through multilayer perceptron, and then global max pooling is used to obtain single-path point cloud features.

[0116] 4) Feature fusion layer: The splicing operation is used to integrate the single-path point cloud features output by the four coding layers into global fusion features.

[0117] 5) Decoding layer: Composed of 3 layers of deconvolutional network, it upsamples the global fusion features, restores the 3D structural details, and outputs deconstructed features.

[0118] 6) Output layer: The features are deconstructed through a fully connected layer to output a 3D reconstructed structure.

[0119] This embodiment adjusts the loss function and iteration mechanism during model iteration: a reconstruction loss function and a sliding window are set, and the expression for the reconstruction loss function is as follows:

[0120]

[0121]

[0122]

[0123]

[0124] In the formula The reconstruction loss value for a single sliding window; These are the loss weighting coefficients; For contour similarity function; Extracting contours within a single window of a virtual structure image; Extracting contours within a single window of a remotely sensed image; These are similarity weight coefficients; , These are the shape similarity function and the positional overlap function, respectively. , They are respectively , The Hu moment vector; Represents the pixel area of ​​the contour mask.

[0125] The initial 3D reconstructed structure is captured by a virtual camera according to the above-mentioned combination of acquisition parameters to obtain a virtual structure image with the same perspective as the remote sensing image. The reconstruction loss value of each window is calculated by traversing the virtual structure image and the remote sensing image using a sliding window. The window loss values ​​corresponding to the four remote sensing images are integrated, and high weights are assigned to areas with high loss values ​​to generate an auxiliary weight map. The auxiliary weight map is input into the improved 3D reconstruction model, and the model parameters are adjusted through backpropagation. The final substation reconstruction structure is obtained through model iteration.

[0126] Preferably, after generating the substation reconfiguration structure, a ResNet-18-based image recognition model is used to identify equipment objects in the remote sensing images, outputting the recognition results, including: equipment type: transformer, switchgear, cable trench; and structure type: wall, column, roof. The recognition results are then matched with a material property database using keywords. This database pre-stores physical parameters of substation building materials (steel, concrete, insulating rubber, etc.), including but not limited to hardness, reflectivity, and heat resistance. Based on the matching results, the material properties are mapped to the corresponding components of the substation reconfiguration structure, forming a virtual testing environment containing physical properties.

[0127] Specifically, based on the target robot's design drawings, a 1:1 geometric model is created to recreate the robot's joint structure, degrees of freedom, and external dimensions. Core component parameters of the target robot, such as motor power, reducer transmission ratio, sensor accuracy, and component characteristics like battery capacity and actuator response delay, are extracted and assigned mechanical and electrical properties using ANSYS. Sensors, including vision sensors, laser sensors, pressure sensors, angle sensors, sound sensors, inertial measurement units, GPS positioning modules, and electromagnetic interference sensors, are deployed on the virtual robot, with parameters consistent with the actual components on the target robot. The data acquisition interfaces of the target robot are mapped one-to-one with the output interfaces of the virtual sensing devices. The digital twin robot collects environmental data from the virtual test environment through the virtual sensing devices, converts the data according to the target robot's data transmission protocol, and pushes it to the target robot's interface in real time, replacing the input of actual environmental data. Control commands output by the target robot's internal algorithm are synchronized to the digital twin robot in real time through the interface embedding module, driving the virtual robot to perform corresponding actions in the virtual environment, with the synchronization frequency consistent with the target robot's control cycle.

[0128] Preferably, this embodiment selects 10 test items: inspection path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, stability test in severe weather and at night, equipment anomaly emergency response speed test, command coordination execution accuracy test, high-voltage area safety boundary compliance test, and full-condition endurance test; among which, severe weather includes: rainstorm weather, sleet weather, heavy snow weather, and typhoon weather. A quantitative evaluation formula system is constructed based on the above test items, the expression of which is as follows:

[0129]

[0130] In the formula, This is the overall test score; For the first Individual test weights; For the first Individual test scores; The evaluation formula is set according to the test items.

[0131] Specifically, the detailed testing process is as follows:

[0132] 1) Call the pre-built virtual test environment through the preset interface, load the three-dimensional reconstruction structure, material properties and preset scene of the target substation; package and deploy the digital twin robot model to the virtual test environment, complete the interface connection and communication verification; and send the multi-dimensional test tasks to the target robot.

[0133] 2) The target robot uses a built-in algorithm to break down multi-dimensional test tasks, generate sub-task sequences, and synchronize them to the digital twin robot; virtual sensing devices collect environmental information at a fixed frequency, and the target robot issues control commands at a fixed frequency. During the update process, timestamp alignment technology is used to ensure the consistency of data and command timing; the collected dimensions include motion indicators, performance indicators, safety indicators, etc.

[0134] 3) Integrate data according to test items, generate indicator datasets for each item, and calculate and evaluate the indicator datasets according to the quantitative evaluation formula system mentioned above, and output test reports.

[0135] As an optional implementation, this embodiment also provides an intelligent robot testing system based on a virtual environment, including:

[0136] The data acquisition module is used to collect building data;

[0137] The structural reconstruction module is used to reconstruct the building data using an improved 3D reconstruction model to obtain a virtual testing environment.

[0138] A digital twin module is used to construct a digital twin robot based on the structure of the target robot.

[0139] The task setting module is used to set multi-dimensional test tasks according to the test project.

[0140] The quantitative evaluation module is used to construct a quantitative evaluation formula system;

[0141] The virtual testing module is used to load the digital twin robot into the virtual testing environment, test the digital twin robot according to multi-dimensional testing tasks, and obtain virtual test data.

[0142] The result output module is used to calculate the virtual test data using the quantitative evaluation formula system to obtain the test results of the intelligent robot.

[0143] As an optional implementation, this embodiment also provides an electronic device, including: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to execute the aforementioned intelligent robot testing method based on a virtual environment.

[0144] As an optional implementation, this embodiment also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the aforementioned intelligent robot testing method based on a virtual environment.

[0145] The beneficial effects of this invention are as follows:

[0146] This invention improves the 3D reconstruction model by using multi-dimensional point cloud data for collaborative reconstruction and dynamically adjusting the reconstructed structure using actual remote sensing images. This avoids the need for a large amount of manual reconstruction data and prevents the decline in model reconstruction effect caused by unsupervised reconstruction. The accuracy of the test scene restoration is greatly improved, and the deviation between the test and the actual operation is reduced. By coupling and binding the digital twin robot and the target robot, the problem of low reliability of test results caused by overly idealized settings is avoided, and the test results are more in line with the actual working conditions.

[0147] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0148] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A testing method for intelligent robots based on a virtual environment, characterized in that, include: Collect building data; The building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment; Construct a digital twin robot based on the structure of the target robot; Set up multi-dimensional test tasks based on the test project; Construct a quantitative evaluation formula system; The digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data; The virtual test data is calculated using the aforementioned quantitative evaluation formula system to obtain the test results of the intelligent robot; Constructing a digital twin robot based on the structure of the target robot includes: A three-dimensional model is created based on the geometric structure of the target robot to obtain the robot model; Physical property mapping is performed on each component of the robot model based on the device parameters and device characteristics of the target robot; A digital twin robot is obtained by setting virtual sensing devices on the robot model; the virtual sensing devices include: a visual sensor, a laser sensor, a pressure sensor, an angle sensor, and a sound sensor. Replace the data acquisition output interface of the target robot with the output interface of the virtual sensing device, and embed the command output interface of the target robot into the digital twin robot; The digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data, including: The constructed virtual test environment is invoked, and the digital twin robot is loaded into the virtual test environment through the test interface; The multi-dimensional test tasks are sent to the target robot through a preset interface; The algorithm embedded in the target robot is used to perform subtask splitting, sorting and instruction generation on the multi-dimensional test task to obtain the test task execution strategy, and the test task execution strategy is synchronously updated to the digital twin robot through a preset interface. The virtual sensing device is used to collect environmental information, and the environmental information is synchronized to the target robot. After the target robot responds to the environmental information, the digital twin robot is controlled to move using the control commands issued by the target robot, and the environmental information and the control commands are updated at a fixed frequency during the task execution. After the multi-dimensional test task is completed, the indicator data of the digital twin robot during the task execution is integrated to obtain the virtual test data.

2. The intelligent robot testing method based on a virtual environment according to claim 1, characterized in that, Collect building data, including: By setting the acquisition angle, acquisition height, and distance from the center of the power station, several sets of acquisition parameter combinations are obtained; the acquisition height is greater than the building height of the target substation; the distance from the center of the power station is greater than the building radius of the target substation. Data is collected from the target substation using the camera and radar in four orthogonal directions according to the collection parameter combination, resulting in one-way data. The four one-way data corresponding to the same collection parameter combination are set as a set of reconstructed data input. The one-way data includes: remote sensing images and remote sensing point clouds. By integrating all the aforementioned reconstructed data inputs, the building data is obtained.

3. The intelligent robot testing method based on a virtual environment according to claim 1, characterized in that, The test items include: inspection path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, severe weather and nighttime travel stability test, equipment anomaly emergency response speed test, command coordination execution accuracy test, high-voltage area safety boundary compliance test, and full-condition endurance test; the severe weather includes: rainstorm weather, sleet weather, heavy snow weather, and typhoon weather.

4. The intelligent robot testing method based on a virtual environment according to claim 1, characterized in that, The expression for the quantitative evaluation formula system is as follows: ;in, This is the overall test score; For the first Individual test weights; For the first Individual test scores; The evaluation formula is set according to the test items.

5. The intelligent robot testing method based on a virtual environment according to claim 2, characterized in that, The building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment, including: An improved 3D reconstruction model is constructed; the improved 3D reconstruction model includes: an input layer, a matrix transformation layer, four parallel encoding layers, a feature fusion layer, a decoding layer, and an output layer connected in sequence; Define the reconstruction loss function and the sliding window; the expression for the reconstruction loss function is: ;in, ; The reconstruction loss value for a single sliding window; These are the loss weighting coefficients; For contour similarity function; Extracting contours within a single window of a virtual structure image; Extract the contour within a single window of the remote sensing image; These are similarity weight coefficients; , These are the shape similarity function and the positional overlap function, respectively. The matrix transformation layer is used to perform orientation alignment transformation on the remote sensing point clouds in the four unidirectional data in the reconstructed data input to obtain unidirectional data input; The four in-direction data inputs are input in parallel into four parallel coding layers for feature extraction, resulting in single-path features; The four single-path features are fused using the feature fusion layer to obtain fused features; The fused features are decoded using the decoding layer to obtain deconstructed features; The deconstructed features are input into the output layer for mapping processing to obtain a three-dimensional reconstructed structure. The virtual image of the three-dimensional reconstructed structure is acquired based on the combination of acquisition parameters to obtain the virtual structure image; The similarity loss between the virtual structure image and the remote sensing image is calculated using the reconstruction loss function and the sliding window to obtain several window loss values; The window loss values ​​corresponding to the four remote sensing images are integrated, and a weight map is generated based on the integration result to obtain an auxiliary weight map; Based on the auxiliary weight map and the window loss value, the virtual structure image is iterated using the improved 3D reconstruction model to obtain the reconstructed structure of the substation; The remote sensing image is used to identify objects, and the identification results are obtained. The identification results are matched with a preset material property library to obtain matching results; Based on the matching results, the material properties of the reconfigurable structure of the substation are set to obtain the virtual test environment.

6. A virtual environment-based intelligent robot testing system, characterized in that, The system for implementing the intelligent robot testing method based on a virtual environment as described in claim 1 includes: The data acquisition module is used to collect building data; The structural reconstruction module is used to reconstruct the building data using an improved 3D reconstruction model to obtain a virtual testing environment. A digital twin module is used to construct a digital twin robot based on the structure of the target robot. The task setting module is used to set multi-dimensional test tasks according to the test project. The quantitative evaluation module is used to construct a quantitative evaluation formula system; The virtual testing module is used to load the digital twin robot into the virtual testing environment, test the digital twin robot according to multi-dimensional testing tasks, and obtain virtual test data. The result output module is used to calculate the virtual test data using the quantitative evaluation formula system to obtain the test results of the intelligent robot.

7. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform a virtual environment-based intelligent robot testing method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the intelligent robot testing method based on a virtual environment according to any one of claims 1 to 5.

Images