A virtual environment-based intelligent robot testing method and system, an electronic device, and a storage medium

Abstract

The application belongs to the technical field of robot testing, and provides an intelligent robot testing method and system based on a virtual environment, an electronic device and a storage medium, the method comprising: building data acquisition, virtual testing environment reconstruction, digital twin robot construction, multi-dimensional testing task setting, quantitative evaluation formula system construction, virtual testing and quantitative evaluation; the application uses multi-dimensional point cloud data for collaborative reconstruction by improving a three-dimensional reconstruction model, and dynamically adjusts the reconstruction structure by using actual remote sensing images, so that the use of a large amount of artificial reconstruction data is avoided, the model reconstruction effect is improved, the test scene restoration degree is greatly improved, and the deviation between testing and actual operation is reduced; by coupling and binding the digital twin robot and the target robot, the problem that the test result reliability is low due to excessive idealization setting is avoided, and the test result is more in line with the actual working condition.

Description

Technical Field

[0001] This invention relates to the field of robot testing technology, and in particular to a method, system, electronic device, and storage medium for testing intelligent robots based on a virtual environment. Background Technology

[0002] The existing intelligent robot evaluation system is centered on standards, tools, and scenario adaptation, focusing on technology implementation and compliance to build a multi-dimensional framework: In terms of functional performance, it revolves around motion accuracy, perception range, and decision-making efficiency, adopting a hierarchical indicator system and dynamic weights to adapt to different needs; in terms of safety and ethics, physical safety follows standards such as ISO10218 and GB9706 to control force, power, and biocompatibility, while the ethical aspect emphasizes data privacy, algorithm transparency, and responsibility division; in scenario-based evaluation, industrial applications focus on digital twin verification and operational accuracy, service robots are assessed for voice interaction and emotional companionship effects, and extreme environment adaptability and anti-interference are tested; in terms of tools, it relies on high-fidelity simulation, hardware-in-the-loop testing, and a multimodal fusion framework, combined with agent-driven evaluation to reduce costs; overall, it follows international standards such as ISO / ANSI and domestic regulations to achieve precise matching between technical indicators and actual needs.

[0003] Existing intelligent robot testing methods are mostly based on fixed standard procedures, but these pose high risks for testing processes that require complex or dangerous scenarios. Furthermore, they suffer from the following problems: 1) Traditional virtual testing relies on manual modeling or simple data collection, lacking accurate reproduction of the structural details and material properties of real-world scenarios, leading to significant deviations between the testing environment and actual working conditions; 2) Traditional simulation testing often uses purely virtual robots, directly porting algorithms to them. However, this approach ignores factors such as latency, errors, and interference during actual robot operation, resulting in overly ideal responses and low test reliability; 3) Existing physical testing methods are limited by environmental requirements and cannot cover the needs of multiple testing scenarios; 4) Traditional reconstructed models require extensive pre-training with labeled data, increasing time costs, and the models cannot be dynamically adjusted. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a method, system, electronic device, and storage medium for testing intelligent robots based on a virtual environment, which solves the problems of large environmental deviations, overly idealized simulations, inability to cover multiple test scenario requirements, and the need for a large amount of pre-built data for reconstructing model training in traditional methods.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for testing intelligent robots based on a virtual environment, comprising:

[0007] Collect building data;

[0008] The building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment;

[0009] Construct a digital twin robot based on the structure of the target robot;

[0010] Set up multi-dimensional test tasks based on the test items;

[0011] Construct a quantitative evaluation formula system;

[0012] The digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data;

[0013] The virtual test data is calculated using the aforementioned quantitative evaluation formula system to obtain the test results of the intelligent robot.

[0014] Preferably, the collection of building data includes:

[0015] By setting the acquisition angle, acquisition height, and distance from the center of the power station, several sets of acquisition parameter combinations are obtained; the acquisition height is greater than the building height of the target substation; the distance from the center of the power station is greater than the building radius of the target substation.

[0016] Data is collected from the target substation using the camera and radar in four orthogonal directions according to the collection parameter combination, resulting in one-way data. The four one-way data corresponding to the same collection parameter combination are set as a set of reconstructed data input. The one-way data includes: remote sensing images and remote sensing point clouds.

[0017] By integrating all the aforementioned reconstructed data inputs, the building data is obtained.

[0018] Preferably, a digital twin robot is constructed based on the structure of the target robot, including:

[0019] A three-dimensional model is created based on the geometric structure of the target robot to obtain the robot model;

[0020] Physical property mapping is performed on each component of the robot model based on the device parameters and device characteristics of the target robot;

[0021] A digital twin robot is obtained by setting virtual sensing devices on the robot model; the virtual sensing devices include: a visual sensor, a laser sensor, a pressure sensor, an angle sensor, and a sound sensor.

[0022] The data acquisition output interface of the target robot is replaced with the output interface of the virtual sensing device, and the command output interface of the target robot is embedded into the digital twin robot.

[0023] Preferably, the test items include: inspection path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, severe weather and nighttime travel stability test, equipment anomaly emergency response speed test, command coordination execution accuracy test, high-voltage area safety boundary compliance test, and full-condition endurance test; the severe weather includes: rainstorm weather, sleet weather, heavy snow weather, and typhoon weather.

[0024] Preferably, the expression of the quantitative evaluation formula system is: ;in, This is the overall test score; For the first Individual test weights; For the first Individual test scores; The evaluation formula is set according to the test items.

[0025] Preferably, the building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment, including:

[0026] An improved 3D reconstruction model is constructed; the improved 3D reconstruction model includes: an input layer, a matrix transformation layer, four parallel encoding layers, a feature fusion layer, a decoding layer, and an output layer connected in sequence;

[0027] Define the reconstruction loss function and the sliding window; the expression for the reconstruction loss function is:

[0028] ;in, ; The reconstruction loss value for a single sliding window; These are the loss weighting coefficients; For contour similarity function; Extracting contours within a single window of a virtual structure image; Extract the contour within a single window of the remote sensing image; These are similarity weight coefficients; , These are the shape similarity function and the positional overlap function, respectively.

[0029] The matrix transformation layer is used to perform orientation alignment transformation on the remote sensing point clouds in the four unidirectional data in the reconstructed data input to obtain unidirectional data input;

[0030] The four in-direction data inputs are input in parallel into four parallel coding layers for feature extraction, resulting in single-path features;

[0031] The four single-path features are fused using the feature fusion layer to obtain fused features;

[0032] The fused features are decoded using the decoding layer to obtain deconstructed features;

[0033] The deconstructed features are input into the output layer for mapping processing to obtain a three-dimensional reconstructed structure.

[0034] The virtual image of the three-dimensional reconstructed structure is acquired based on the combination of acquisition parameters to obtain the virtual structure image;

[0035] The similarity loss between the virtual structure image and the remote sensing image is calculated using the reconstruction loss function and the sliding window to obtain several window loss values;

[0036] The window loss values ​​corresponding to the four remote sensing images are integrated, and a weight map is generated based on the integration result to obtain an auxiliary weight map;

[0037] Based on the auxiliary weight map and the window loss value, the virtual structure image is iterated using the improved 3D reconstruction model to obtain the reconstructed structure of the substation;

[0038] The remote sensing image is used to identify objects, and the identification results are obtained.

[0039] The identification results are matched with a preset material property library to obtain matching results;

[0040] Based on the matching results, the material properties of the reconfigurable structure of the substation are set to obtain the virtual test environment.

[0041] Preferably, the digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data, including:

[0042] The constructed virtual test environment is invoked, and the digital twin robot is loaded into the virtual test environment through the test interface;

[0043] The multi-dimensional test tasks are sent to the target robot through a preset interface;

[0044] The algorithm embedded in the target robot is used to perform subtask splitting, sorting and instruction generation on the multi-dimensional test task to obtain the test task execution strategy, and the test task execution strategy is synchronously updated to the digital twin robot through a preset interface.

[0045] The virtual sensing device is used to collect environmental information, and the environmental information is synchronized to the target robot.

[0046] After the target robot responds to the environmental information, the digital twin robot is controlled to move using the control commands issued by the target robot, and the environmental information and the control commands are updated at a fixed frequency during the task execution.

[0047] After the multi-dimensional test task is completed, the indicator data of the digital twin robot during the task execution is integrated to obtain the virtual test data.

[0048] Preferably, a virtual environment-based intelligent robot testing system includes:

[0049] The data acquisition module is used to collect building data;

[0050] The structural reconstruction module is used to reconstruct the building data using an improved 3D reconstruction model to obtain a virtual testing environment.

[0051] A digital twin module is used to construct a digital twin robot based on the structure of the target robot.

[0052] The task setting module is used to set multi-dimensional test tasks according to the test project.

[0053] The quantitative evaluation module is used to construct a quantitative evaluation formula system;

[0054] The virtual testing module is used to load the digital twin robot into the virtual testing environment, test the digital twin robot according to multi-dimensional testing tasks, and obtain virtual test data.

[0055] The result output module is used to calculate the virtual test data using the quantitative evaluation formula system to obtain the test results of the intelligent robot.

[0056] Preferably, an electronic device includes: at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the aforementioned intelligent robot testing method based on a virtual environment.

[0057] Preferably, a non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the aforementioned intelligent robot testing method based on a virtual environment.

[0058] The present invention discloses the following technical effects:

[0059] This invention provides a method, system, electronic device, and storage medium for testing intelligent robots based on a virtual environment. By improving the 3D reconstruction model, utilizing multi-dimensional point cloud data for collaborative reconstruction, and dynamically adjusting the reconstructed structure using actual remote sensing images, it solves the problems caused by traditional virtual testing relying on manual modeling or simple data acquisition, and achieves reliable reconstruction of the 3D structure of distribution substations. By coupling and binding the digital twin robot and the target robot, it solves the problem of low reliability of test results caused by overly idealized settings, and achieves a testing process that is more in line with actual working conditions. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 A schematic diagram of a virtual environment-based intelligent robot testing process provided in an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the building data acquisition process provided in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the digital twin robot construction process provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of the virtual test environment generation process provided in an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of a virtual testing process provided in an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The purpose of this invention is to provide a method, system, electronic device and storage medium for testing intelligent robots based on a virtual environment, which solves the problems of large environmental deviation, overly idealized simulation, inability to cover multiple test scenario requirements and the need for a large amount of pre-built data for reconstructing model training in traditional methods.

[0068] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] Figure 1 This is a schematic diagram of a virtual environment-based intelligent robot testing process provided in an embodiment of the present invention, such as... Figure 1 As shown, this invention provides a method for testing intelligent robots based on a virtual environment, including:

[0070] Step 100: Collect building data;

[0071] Step 200: Reconstruct the building data using the improved 3D reconstruction model to obtain a virtual testing environment;

[0072] Step 300: Construct a digital twin robot based on the structure of the target robot;

[0073] Step 400: Set up multi-dimensional test tasks based on the test items;

[0074] Step 500: Construct a quantitative evaluation formula system;

[0075] Step 600: Load the digital twin robot into the virtual test environment, test the digital twin robot according to the multi-dimensional test tasks, and obtain virtual test data;

[0076] Step 700: Calculate the virtual test data using the quantitative evaluation formula system to obtain the intelligent robot test results.

[0077] refer to Figure 2 Collect building data, including:

[0078] Step 101: Set the acquisition angle, acquisition height, and distance from the center of the power station to obtain several sets of acquisition parameter combinations; the acquisition height is greater than the building height of the target substation; the distance from the center of the power station is greater than the building radius of the target substation.

[0079] Step 102: Using the camera and radar of the UAV, data is collected from the target substation in four orthogonal directions according to the collection parameter combination to obtain one-way data. The four one-way data corresponding to the same collection parameter combination are set as a set of reconstructed data input. The one-way data includes: remote sensing images and remote sensing point clouds.

[0080] Step 103: Integrate all the reconstructed data inputs to obtain the building data.

[0081] refer to Figure 3 Constructing a digital twin robot based on the structure of the target robot includes:

[0082] Step 301: Perform 3D modeling based on the geometric structure of the target robot to obtain the robot model;

[0083] Step 302: Map the physical properties of each component of the robot model according to the device parameters and device characteristics of the target robot;

[0084] Step 303: Set virtual sensing devices on the robot model to obtain the digital twin robot; the virtual sensing devices include: a visual sensor, a laser sensor, a pressure sensor, an angle sensor, and a sound sensor;

[0085] Step 304: Replace the data acquisition output interface of the target robot with the output interface of the virtual sensing device, and embed the command output interface of the target robot into the digital twin robot.

[0086] Optionally, the test items include: inspection path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, severe weather and nighttime travel stability test, equipment anomaly emergency response speed test, command coordination execution accuracy test, high-voltage area safety boundary compliance test, and full-condition endurance test; the severe weather includes: rainstorm weather, sleet weather, heavy snow weather, and typhoon weather.

[0087] Preferably, the expression of the quantitative evaluation formula system is: ;in, This is the overall test score; For the first Individual test weights; For the first Individual test scores; The evaluation formula is set according to the test items.

[0088] refer to Figure 4The building data is reconstructed using an improved 3D reconstruction model to obtain a virtual testing environment, including:

[0089] Step 201: Construct an improved 3D reconstruction model; the improved 3D reconstruction model includes: an input layer, a matrix transformation layer, four parallel encoding layers, a feature fusion layer, a decoding layer, and an output layer connected in sequence;

[0090] Step 202: Set the reconstruction loss function and the sliding window; the expression for the reconstruction loss function is:

[0091] ;in, ; The reconstruction loss value for a single sliding window; These are the loss weighting coefficients; For contour similarity function; Extracting contours within a single window of a virtual structure image; Extract the contour within a single window of the remote sensing image; These are similarity weight coefficients; , These are the shape similarity function and the positional overlap function, respectively.

[0092] Step 203: Use the matrix transformation layer to perform orientation alignment transformation on the remote sensing point clouds in the four unidirectional data in the reconstructed data input to obtain unidirectional data input;

[0093] Step 204: Input the four in-direction data into the four parallel coding layers in parallel to extract features and obtain single-path features;

[0094] Step 205: Use the feature fusion layer to fuse the four single-path features to obtain fused features;

[0095] Step 206: Decode the fused features using the decoding layer to obtain the deconstructed features;

[0096] Step 207: Input the deconstruction features into the output layer for mapping processing to obtain the three-dimensional reconstructed structure;

[0097] Step 208: Perform virtual image acquisition on the three-dimensional reconstructed structure according to the acquisition parameter combination to obtain the virtual structure image;

[0098] Step 209: Calculate the similarity loss between the virtual structure image and the remote sensing image using the reconstruction loss function and the sliding window to obtain several window loss values;

[0099] Step 210: Integrate the window loss values ​​corresponding to the four remote sensing images and generate a weight map based on the integration result to obtain an auxiliary weight map;

[0100] Step 211: Based on the auxiliary weight map and the window loss value, iterate the virtual structure image using the improved 3D reconstruction model to obtain the reconstructed structure of the substation;

[0101] Step 212: Use an image recognition model to perform object recognition on the remote sensing image to obtain the recognition result;

[0102] Step 213: Match the identification result with the preset material property library to obtain the matching result;

[0103] Step 214: Set the material properties of the substation reconfiguration structure according to the matching results to obtain the virtual test environment.

[0104] refer to Figure 5 The digital twin robot is loaded into the virtual testing environment, and the digital twin robot is tested according to multi-dimensional testing tasks to obtain virtual test data, including:

[0105] Step 601: Invoke the constructed virtual test environment and load the digital twin robot into the virtual test environment through the test interface;

[0106] Step 602: Send the set multi-dimensional test task to the target robot through a preset interface;

[0107] Step 603: Use the algorithm embedded in the target robot to perform sub-task splitting, sorting and instruction generation on the multi-dimensional test task to obtain the test task execution strategy, and update the test task execution strategy to the digital twin robot through a preset interface.

[0108] Step 604: Collect environmental information using the virtual sensing device and synchronize the environmental information to the target robot;

[0109] Step 605: After the target robot responds to the environmental information, the digital twin robot is controlled to move using the control commands issued by the target robot, and the environmental information and the control commands are updated at a fixed frequency during the task execution.

[0110] Step 606: After the multi-dimensional test task is completed, integrate the indicator data of the digital twin robot during the task execution to obtain the virtual test data.

[0111] Specifically, this embodiment acquires building data including remote sensing images and remote sensing point clouds through a standardized UAV acquisition scheme, including: 1) setting three core acquisition parameters: acquisition angle, acquisition height, and distance from the center of the substation, combining them to form several sets of acquisition parameters. The acquisition height must be greater than the building height of the target substation, and the distance from the center of the substation must be greater than the building radius of the target substation to ensure that the acquisition range covers the entire substation; 2) conducting data acquisition in four orthogonal directions of the target substation, such as east, south, west, and north (the actual direction is set according to the direction of the main structure of the substation). According to each preset set of acquisition parameters, the camera and radar on the UAV are used to acquire data synchronously to obtain remote sensing images and remote sensing point clouds. The four sets of unidirectional data corresponding to the four orthogonal directions under the same acquisition parameter combination are grouped into one set of reconstructed data input; 3) collecting the reconstructed data input corresponding to all acquisition parameter combinations and integrating them to form building data.

[0112] Furthermore, this embodiment improves upon PointNet by designing an improved 3D reconstruction model. This improved model accurately adapts to the generation requirements of virtual testing environments. By integrating PointNet's point cloud processing advantages with the original multi-directional data collaboration logic, it further enhances the detail reproduction of the substation reconstruction structure and the accuracy of material property matching. The PointNet encoding layer structure is changed to four parallel encoding layers, and the original loss function is also replaced. The specific structure of the improved 3D reconstruction model is as follows: input layer, preprocessing layer, matrix transformation layer, four parallel encoding layers, feature fusion layer, decoding layer, and output layer. The specific functions of each network layer are as follows:

[0113] 1) Preprocessing layer: PointNet standard preprocessing is performed on the remote sensing point cloud in the preprocessing layer, including point cloud sampling and normalization.

[0114] 2) Matrix transformation layer: The remote sensing point cloud of four unidirectional data is transformed by a 3×3 transformation matrix to perform orientation alignment transformation and output point cloud data in the same direction.

[0115] 3) Four parallel coding layers: The MLP network of PointNet is used to extract local geometric features of point cloud through multilayer perceptron, and then global max pooling is used to obtain single-path point cloud features.

[0116] 4) Feature fusion layer: The splicing operation is used to integrate the single-path point cloud features output by the four coding layers into global fusion features.

[0117] 5) Decoding layer: Composed of 3 layers of deconvolutional network, it upsamples the global fusion features, restores the 3D structural details, and outputs deconstructed features.

[0118] 6) Output layer: The features are deconstructed through a fully connected layer to output a 3D reconstructed structure.

[0119] This embodiment adjusts the loss function and iteration mechanism during model iteration: a reconstruction loss function and a sliding window are set, and the expression for the reconstruction loss function is as follows:

[0120]

[0121]

[0122]

[0123]

[0124] In the formula The reconstruction loss value for a single sliding window; These are the loss weighting coefficients; For contour similarity function; Extracting contours within a single window of a virtual structure image; Extracting contours within a single window of a remotely sensed image; These are similarity weight coefficients; , These are the shape similarity function and the positional overlap function, respectively. , They are respectively , The Hu moment vector; Represents the pixel area of ​​the contour mask.

[0125] The initial 3D reconstructed structure is captured by a virtual camera according to the above-mentioned combination of acquisition parameters to obtain a virtual structure image with the same perspective as the remote sensing image. The reconstruction loss value of each window is calculated by traversing the virtual structure image and the remote sensing image using a sliding window. The window loss values ​​corresponding to the four remote sensing images are integrated, and high weights are assigned to areas with high loss values ​​to generate an auxiliary weight map. The auxiliary weight map is input into the improved 3D reconstruction model, and the model parameters are adjusted through backpropagation. The final substation reconstruction structure is obtained through model iteration.

[0126] Preferably, after generating the substation reconfiguration structure, a ResNet-18-based image recognition model is used to identify equipment objects in the remote sensing images, outputting the recognition results, including: equipment type: transformer, switchgear, cable trench; and structure type: wall, column, roof. The recognition results are then matched with a material property database using keywords. This database pre-stores physical parameters of substation building materials (steel, concrete, insulating rubber, etc.), including but not limited to hardness, reflectivity, and heat resistance. Based on the matching results, the material properties are mapped to the corresponding components of the substation reconfiguration structure, forming a virtual testing environment containing physical properties.

[0127] Specifically, based on the target robot's design drawings, a 1:1 geometric model is created to recreate the robot's joint structure, degrees of freedom, and external dimensions. Core component parameters of the target robot, such as motor power, reducer transmission ratio, sensor accuracy, and component characteristics like battery capacity and actuator response delay, are extracted and assigned mechanical and electrical properties using ANSYS. Sensors, including vision sensors, laser sensors, pressure sensors, angle sensors, sound sensors, inertial measurement units, GPS positioning modules, and electromagnetic interference sensors, are deployed on the virtual robot, with parameters consistent with the actual components on the target robot. The data acquisition interfaces of the target robot are mapped one-to-one with the output interfaces of the virtual sensing devices. The digital twin robot collects environmental data from the virtual test environment through the virtual sensing devices, converts the data according to the target robot's data transmission protocol, and pushes it to the target robot's interface in real time, replacing the input of actual environmental data. Control commands output by the target robot's internal algorithm are synchronized to the digital twin robot in real time through the interface embedding module, driving the virtual robot to perform corresponding actions in the virtual environment, with the synchronization frequency consistent with the target robot's control cycle.

[0128] Preferably, this embodiment selects 10 test items: inspection path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, stability test in severe weather and at night, equipment anomaly emergency response speed test, command coordination execution accuracy test, high-voltage area safety boundary compliance test, and full-condition endurance test; among which, severe weather includes: rainstorm weather, sleet weather, heavy snow weather, and typhoon weather. A quantitative evaluation formula system is constructed based on the above test items, the expression of which is as follows:

[0129]

[0130] In the formula, This is the overall test score; For the first Individual test weights; For the first Individual test scores; The evaluation formula is set according to the test items.

[0131] Specifically, the detailed testing process is as follows:

[0132] 1) Call the pre-built virtual test environment through the preset interface, load the three-dimensional reconstruction structure, material properties and preset scene of the target substation; package and deploy the digital twin robot model to the virtual test environment, complete the interface connection and communication verification; and send the multi-dimensional test tasks to the target robot.

[0133] 2) The target robot uses a built-in algorithm to break down multi-dimensional test tasks, generate sub-task sequences, and synchronize them to the digital twin robot; virtual sensing devices collect environmental information at a fixed frequency, and the target robot issues control commands at a fixed frequency. During the update process, timestamp alignment technology is used to ensure the consistency of data and command timing; the collected dimensions include motion indicators, performance indicators, safety indicators, etc.

[0134] 3) Integrate data according to test items, generate indicator datasets for each item, and calculate and evaluate the indicator datasets according to the quantitative evaluation formula system mentioned above, and output test reports.

[0135] As an optional implementation, this embodiment also provides an intelligent robot testing system based on a virtual environment, including:

[0136] The data acquisition module is used to collect building data;

[0137] The structural reconstruction module is used to reconstruct the building data using an improved 3D reconstruction model to obtain a virtual testing environment.

[0138] A digital twin module is used to construct a digital twin robot based on the structure of the target robot.

[0139] The task setting module is used to set multi-dimensional test tasks according to the test project.

[0140] The quantitative evaluation module is used to construct a quantitative evaluation formula system;

[0141] The virtual testing module is used to load the digital twin robot into the virtual testing environment, test the digital twin robot according to multi-dimensional testing tasks, and obtain virtual test data.

[0142] The result output module is used to calculate the virtual test data using the quantitative evaluation formula system to obtain the test results of the intelligent robot.

[0143] As an optional implementation, this embodiment also provides an electronic device, including: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to execute the aforementioned intelligent robot testing method based on a virtual environment.

[0144] As an optional implementation, this embodiment also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the aforementioned intelligent robot testing method based on a virtual environment.

[0145] The beneficial effects of this invention are as follows:

[0146] This invention improves the 3D reconstruction model by using multi-dimensional point cloud data for collaborative reconstruction and dynamically adjusting the reconstructed structure using actual remote sensing images. This avoids the need for a large amount of manual reconstruction data and prevents the decline in model reconstruction effect caused by unsupervised reconstruction. The accuracy of the test scene restoration is greatly improved, and the deviation between the test and the actual operation is reduced. By coupling and binding the digital twin robot and the target robot, the problem of low reliability of test results caused by overly idealized settings is avoided, and the test results are more in line with the actual working conditions.

[0147] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0148] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A virtual environment-based intelligent robot testing method, characterized by, The method comprises the following steps: collecting building data; reconstructing the building data by using an improved three-dimensional reconstruction model to obtain a virtual test environment; constructing a digital twin robot according to the structure of the target robot; setting up a multi-dimensional test task according to a test item; constructing a quantitative evaluation formula system; loading the digital twin robot into the virtual test environment, testing the digital twin robot according to the multi-dimensional test task, and obtaining virtual test data; calculating the virtual test data by using the quantitative evaluation formula system to obtain the test result of the intelligent robot; collecting building data, comprising: setting up a collection angle, a collection height, and a distance from the center of the power station to obtain a plurality of sets of collection parameter combinations; the collection height is greater than the building height of the target variable distribution power station; the distance from the center of the power station is greater than the building radius of the target variable distribution power station; collecting data of the target variable distribution power station in four orthogonal directions according to the collection parameter combinations by using the camera and radar of the unmanned aerial vehicle to obtain one-way data, and setting up four one-way data corresponding to the same collection parameter combination as a set of reconstruction data input; the one-way data includes remote sensing images and remote sensing point clouds; integrating all the reconstruction data inputs to obtain the building data; reconstructing the building data by using an improved three-dimensional reconstruction model to obtain a virtual test environment, comprising: constructing an improved three-dimensional reconstruction model; the improved three-dimensional reconstruction model comprises an input layer, a matrix conversion layer, four parallel encoding layers, a feature fusion layer, a decoding layer, and an output layer connected in sequence; setting up a reconstruction loss function and a sliding window; the expression of the reconstruction loss function is: ; wherein, ; is a reconstruction loss value for a single sliding window; is a loss weight coefficient; is a contour similarity function; is an extracted contour within a single window of the virtual structure image; is an extracted contour within a single window of the remote sensing image; is a similarity weight coefficient; 、 are shape similarity function, position overlap function, respectively; aligning and converting the remote sensing point clouds in the four one-way data in the reconstruction data input by using the matrix conversion layer to obtain same-direction data input; parallelly inputting the four same-direction data inputs into the four parallel encoding layers to extract features and obtain single-path features; fusing the four single-path features by using the feature fusion layer to obtain fused features; decoding the fused features by using the decoding layer to obtain deconstructed features; inputting the deconstructed features into the output layer for mapping processing to obtain a three-dimensional reconstruction structure; collecting virtual images according to the collection parameter combinations to obtain the virtual structure images; calculating the similarity loss of the virtual structure images and the remote sensing images by using the reconstruction loss function and the sliding window to obtain a plurality of window loss values; integrating the window loss values corresponding to the four remote sensing images and generating a weight map according to the integration result to obtain an auxiliary weight map; iterating the virtual structure images by using the improved three-dimensional reconstruction model according to the auxiliary weight map and the window loss values to obtain a variable distribution power station reconstruction structure; identifying objects in the remote sensing images by using an image recognition model to obtain an identification result; matching the identification result with a preset material attribute library to obtain a matching result; setting up material attributes of the variable distribution power station reconstruction structure according to the matching result to obtain the virtual test environment.

2. The virtual environment based intelligent robot testing method of claim 1, wherein, The test items include: patrol path optimization and coverage integrity test, equipment defect identification accuracy and response speed test, target data acquisition accuracy test, fixed equipment obstacle avoidance effectiveness test, dynamic obstacle avoidance flexibility test, severe weather and night travel stability test, equipment emergency response speed test, instruction coordination execution accuracy test, high pressure area safety boundary compliance test, and full working condition endurance test; the severe weather includes: heavy rain weather, sleet weather, snow weather, and typhoon weather.

3. The virtual environment based intelligent robot testing method of claim 1, wherein, The expression of the quantitative evaluation formula system is: ; wherein, is a comprehensive test score; is a weight of an individual test; is a score of an individual test; is an evaluation formula set according to the test item.

4. The virtual environment based intelligent robot testing method of claim 1, wherein, A digital twin robot is constructed according to a structure of a target robot, including: a three-dimensional model is established according to a geometric structure of the target robot, to obtain a robot model; physical property mapping is performed on each component of the robot model according to device parameters and device characteristics of the target robot; a virtual perception device is arranged on the robot model, to obtain the digital twin robot; the virtual perception device includes a visual sensor, a laser sensor, a pressure sensor, an angle sensor, and a sound sensor; an output interface of the target robot is replaced with an output interface of the virtual perception device, and an instruction output interface of the target robot is embedded into the digital twin robot.

5. The virtual environment based intelligent robot testing method of claim 4, wherein, The digital twin robot is loaded into the virtual test environment, and the digital twin robot is tested according to multi-dimensional test tasks, to obtain virtual test data, including: the virtual test environment is called, and the digital twin robot is loaded into the virtual test environment through a test interface; the multi-dimensional test tasks are issued to the target robot through a preset interface; an algorithm embedded in the target robot is used to split, sort, and generate instructions for the multi-dimensional test tasks, to obtain a test task execution strategy, and the test task execution strategy is synchronously updated to the digital twin robot through a preset interface; environmental information is collected by the virtual perception device, and the environmental information is synchronously updated to the target robot; after the target robot responds to the environmental information, the digital twin robot is controlled by a control instruction issued by the target robot, and the environmental information and the control instruction are updated at a fixed frequency during task execution; when the multi-dimensional test tasks are completed, index data of the digital twin robot during task execution is integrated, to obtain the virtual test data.

6. A virtual environment based intelligent robot testing system, characterized in that, A system for implementing the intelligent robot testing method based on a virtual environment is provided, including: a data acquisition module for acquiring building data; a structure reconstruction module for reconstructing the building data by using an improved three-dimensional reconstruction model to obtain a virtual test environment; a digital twin module for constructing a digital twin robot according to a structure of a target robot; a task setting module for setting multi-dimensional test tasks according to test items; a quantitative evaluation module for constructing a quantitative evaluation formula system; A virtual test module is configured to load the digital twin robot into the virtual test environment, test the digital twin robot according to a multi-dimensional test task, and obtain virtual test data. A result output module is configured to calculate the virtual test data by using the quantitative evaluation formula system, and obtain an intelligent robot test result.

7. An electronic device, comprising: Comprise: At least one processor and a memory connected to the processor in communication; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to perform the method of any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1 to 5.

Images