Intelligent test service system and method, electronic device, and storage medium

Abstract

The application provides an intelligent test service system and method, an electronic device and a storage medium, and relates to the technical field of artificial intelligence testing. The system comprises: a test control component, which receives a test task type of test task configuration information for distribution; a static test component, which receives a static test task and obtains an original to-be-tested data set address; a dynamic test component, which receives a dynamic test task and obtains simulation task configuration information; an intelligent test algorithm component, which performs enhancement processing on the original to-be-tested data set to obtain a new to-be-tested data set, and performs variable element sampling on the simulation task configuration information to obtain new simulation task configuration information; and a to-be-tested algorithm service component, which performs static testing according to the new to-be-tested data set, and performs dynamic testing according to the new simulation task configuration information, to obtain a test result. Through the fusion of modular design and intelligent test algorithms, the application solves the problems of low test efficiency, weak pertinence and poor expansibility of intelligent algorithms of unmanned systems.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence testing technology, and more specifically, to an intelligent testing service system and method, electronic device and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, its application in the field of autonomous driving systems is becoming increasingly widespread, covering multiple core sub-fields such as perception, decision-making, and planning. The rapid iteration of various algorithms has placed stringent requirements on comprehensive testing before algorithm deployment—not only is it necessary to verify the robustness of the algorithm in complex scenarios, but it is also necessary to consider testing efficiency and avoid redundant testing consuming too many resources.

[0003] Currently, algorithm testing for unmanned systems mainly falls into two core scenarios: one is testing perception algorithms such as YOLO, CADDN, and BEV. This type of testing relies on fixed datasets collected by devices such as LiDAR, cameras, and radar. By covering diverse scenarios and targets with these datasets, the robustness of the algorithms can be ensured. The other type is testing decision-making and planning algorithms. These algorithms need to drive the interaction between the vehicle and the environment to complete tasks. There are no fixed standard answers, so a high-fidelity simulation environment and vehicle model are required to reproduce real-world scenarios for effective testing.

[0004] However, existing testing frameworks have significant flaws in design and application, making it difficult to meet the comprehensive needs of the two types of testing mentioned above. Specifically:

[0005] First, the modules are highly coupled and have poor scalability. Existing testing frameworks often integrate the testing of perception, decision-making, and planning algorithms into the overall system without modular decomposition. When it is necessary to replace a testing submodule in a certain area (such as changing the test dataset type for the perception algorithm or updating the simulation environment parameters for the decision-making algorithm), a large-scale adjustment to the entire framework is required. This is complex and prone to triggering a chain of failures, severely limiting the adaptability and iteration efficiency of the testing framework.

[0006] Second, there is a lack of an independent perception algorithm testing module. The existing framework integrates perception algorithm testing into the overall testing process for unmanned systems, resulting in single-scenario testing with fixed target types. This makes it impossible to comprehensively verify core indicators of the perception algorithm, such as target detection and recognition accuracy. To cover a sufficient number of test scenarios, multiple overall tests need to be executed, which not only increases testing costs but also reduces the specificity of the tests.

[0007] Third, the lack of intelligent testing logic leads to low testing efficiency. Existing testing frameworks often employ a "simple stacking of test cases" approach to achieve comprehensive testing, without incorporating intelligent testing algorithms to optimize the testing process. For example, in perceptual algorithm testing, they rely solely on the original, fixed dataset, failing to utilize data augmentation techniques to expand the diversity of test samples. In decision-making and planning algorithm testing, they simply configure simulation scenarios with fixed parameters, without using intelligent sampling algorithms to uncover key test scenarios. This results in a large number of repetitive and invalid test cases, consuming significant computational resources and time, and potentially overlooking core test scenarios, thus impacting testing effectiveness.

[0008] Fourth, it is difficult to accommodate the differentiated requirements of the two types of testing tasks. Perception algorithm testing requires efficient processing of large-scale static datasets, while decision and planning algorithm testing requires a real-time interactive dynamic simulation environment. The existing framework is not specifically designed for the characteristics of these two types of tasks, resulting in poor integration of the testing process. For example, the access to static datasets and the initialization of the dynamic simulation environment interfere with each other, and the data storage and result feedback formats are inconsistent, further reducing testing efficiency.

[0009] Therefore, there is an urgent need for a modular, intelligent, and multi-type test service framework that can solve the problems of high coupling, weak targeting, and low efficiency of existing frameworks, and provide support for the rapid iteration and reliable implementation of various intelligent algorithms for unmanned systems. Summary of the Invention

[0010] To address at least one of the aforementioned problems, this application proposes an intelligent testing service system and method, electronic device, and storage medium. By integrating modular design with intelligent testing algorithms, it solves the problems of low efficiency, weak targeting, and poor scalability in intelligent algorithm testing of unmanned systems.

[0011] According to a first aspect of this application, at least one embodiment of this application provides an intelligent testing service system, comprising: a test control component, configured to receive test task configuration information input by a user, and allocate tasks according to the test task type in the test task configuration information, wherein the test task configuration information includes the test task type, and the test task type includes static test tasks and dynamic test tasks; a static testing component, communicatively connected to the test control component, configured to receive configuration information of static test tasks, and obtain the address of the original dataset to be tested according to the configuration information of the static test tasks; a dynamic testing component, communicatively connected to the test control component, configured to receive configuration information of dynamic test tasks, and obtain simulation task configuration information according to the configuration information of the dynamic test tasks; and an intelligent testing algorithm component, communicatively connected to the static testing component and the dynamic testing component respectively, configured to receive the address of the original dataset to be tested, and allocate tasks according to the configuration information of the static test tasks. The system retrieves the original test dataset from its address, performs enhancement processing on the original test dataset to obtain a new test dataset, and returns the address of the new test dataset to the static testing component based on the new test dataset. It also receives simulation task configuration information, performs variable feature sampling on the simulation task configuration information to obtain new sampled simulation task configuration information, and sends it to the dynamic testing component. The tested algorithm service component, which is communicatively connected to both the static and dynamic testing components, receives the address of the new test dataset from the static testing component, retrieves the new test dataset based on the address, performs static testing based on the new test dataset, obtains test results, and sends them to the static testing component. Furthermore, it receives the new simulation task configuration information from the dynamic testing component to perform dynamic testing, obtains test results, and sends them to the dynamic testing component.

[0012] For example, in some embodiments of this application, it further includes: a simulation component, communicatively connected to the dynamic testing component and the algorithm under test service component, respectively, for receiving the new simulation task configuration information sent by the dynamic testing component and performing simulation, completing the construction and initialization of the simulation environment, and feeding back an initialization completion signal to the dynamic testing component; when the dynamic testing component receives the initialization completion signal, the dynamic testing component is used to start the algorithm under test service component; during the dynamic testing process, the simulation component is also used to interact with the algorithm under test service component, provide the simulation environment to the algorithm under test service component, and receive carrier control instructions sent by the algorithm under test service component to control the carrier in the simulation environment, and to feed back the carrier's operating status data to the algorithm under test service component in real time; wherein, the algorithm under test service component is used to adjust the carrier control instructions according to the carrier's operating status data, and the algorithm under test service component is also used to send the test results to the dynamic testing component.

[0013] For example, in some embodiments of this application, the algorithm under test service component includes: a static algorithm sub-component, used to receive the address of the new dataset to be tested sent by the static testing component, and obtain the new dataset to be tested according to the address of the new dataset to be tested for static testing, obtain test results and send them to the static testing component; and a dynamic algorithm sub-component, used to receive the start signal and the new simulation task configuration information sent by the dynamic testing component, to perform dynamic testing according to the simulation environment provided by the simulation component, generate the carrier control instructions to interact with the simulation component, and adjust the carrier control instructions according to the running status data of the carrier to obtain the test results and send them to the dynamic testing component.

[0014] For example, in some embodiments of this application, it further includes: a database, which is communicatively connected to the static test component, the dynamic test component, the intelligent test algorithm component, the simulation component and the algorithm under test service component, respectively, for storing the original dataset to be tested, the enhanced new dataset to be tested, the simulation task configuration information, the sampled new simulation task configuration information, the test process data and the test results, so as to form a test record.

[0015] For example, in some embodiments of this application, it further includes: a front-end interaction component, used to obtain test task configuration information input by the user, and also used to visualize and display test results.

[0016] For example, in some embodiments of this application, the test control component is further configured to receive a stop task control command input by the user, and send a termination command to the static test component or the dynamic test component according to the test task type in the test task configuration information, so as to stop the test service.

[0017] According to a second aspect of this application, at least one embodiment of this application provides an intelligent testing service method executed by an intelligent testing service system as described in any one of the first aspects. The intelligent testing service method includes: receiving test task configuration information input by a user; allocating test task according to the test task type in the test task configuration information, wherein the test task configuration information includes the test task type, and the test task type includes static test tasks and dynamic test tasks; obtaining the address of an original dataset to be tested according to the configuration information of the static test task; obtaining the original dataset to be tested according to the address of the original dataset to be tested, and performing enhancement processing on the original dataset to be tested to obtain a new dataset to be tested; performing static testing on the new dataset to be tested to obtain a first test result; obtaining simulation task configuration information according to the configuration information of the dynamic test task; performing variable element sampling on the simulation task configuration information to obtain new simulation task configuration information after sampling; performing simulation according to the new simulation task configuration information to obtain a simulation environment; and performing dynamic testing according to the new simulation task configuration information and the simulation environment to obtain a second test result.

[0018] For example, in some embodiments of this application, the method further includes: receiving a stop task control command input by a user, and sending a termination command to the static test component or the dynamic test component according to the test task type in the test task configuration information; when the static test component receives the termination command, sending a termination request to the intelligent test algorithm component or the algorithm under test service component to stop the test service; when the dynamic test component receives the termination command, sending a termination request to the intelligent test algorithm component, or the simulation component and the algorithm under test service component to stop the test service.

[0019] According to a third aspect of this application, at least one embodiment of this application provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to perform the method as described in any one of the second aspects.

[0020] According to a fourth aspect of this application, at least one embodiment of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method as described in any one of the second aspects.

[0021] Through the above exemplary embodiments, the intelligent testing service system and method, electronic device and storage medium provided in this application have at least one of the following beneficial effects:

[0022] Modular design reduces coupling and enhances scalability: The framework breaks down the testing process into independent components such as test control, static testing, dynamic testing, and intelligent testing algorithms. Each component communicates through standardized interfaces. Replacing or upgrading a single component does not require modifying the overall framework, adapting to the iterative needs of different types of algorithms under test and test scenarios.

[0023] Independent static testing components enhance the comprehensiveness of perception algorithm testing: Static testing processes are specifically designed for perception algorithms, and intelligent data augmentation technology expands the diversity of test samples. Targeted verification of perception algorithms can be achieved without relying on overall system testing, avoiding the limitations of single-scenario testing and reducing the cost of repetitive testing.

[0024] Intelligent testing algorithms optimize testing efficiency and reduce resource consumption: Through static data augmentation algorithms and dynamic scene sampling algorithms, key test samples and simulation configurations are quickly generated, avoiding the simple stacking of test cases. While ensuring test coverage, invalid tests are significantly reduced, testing efficiency is improved and computing resource consumption is reduced.

[0025] It is compatible with two types of test tasks and has wide adaptability: The framework supports both static dataset testing of perception algorithms and dynamic simulation testing of decision and planning algorithms. Through a unified central control scheduling and data storage mechanism, it achieves seamless connection between the two types of test tasks and meets the algorithm testing needs of the entire process of unmanned systems.

[0026] Visualized interaction and full-process traceability, with high ease of use: The front-end interactive components provide intuitive configuration and display functions, test process data and results are stored in a structured manner, and support real-time query of test progress, visual analysis of test results and traceability of historical test records, reducing the user's operating threshold.

[0027] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0028] The above and other objects, features, and advantages of this application will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this application and are not intended to limit the scope of this application.

[0029] Figure 1 A schematic diagram of the structure of an intelligent testing service system of an exemplary embodiment is shown;

[0030] Figure 2A flowchart illustrating an exemplary embodiment of an intelligent testing service method is provided.

[0031] Figure 3 This diagram illustrates the structure of an electronic device provided in this application. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0033] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of these specific details, or other methods, components, materials, devices, etc. In these cases, well-known structures, methods, devices, implementations, materials, or operations will not be shown or described in detail.

[0034] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0035] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0036] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0037] Figure 1 A schematic diagram of the structure of an intelligent testing service system of an exemplary embodiment is shown.

[0038] like Figure 1As shown, the intelligent testing service system includes: a test control component 101, a static testing component 102, a dynamic testing component 103, an intelligent testing algorithm component 104, a simulation component 105, and a test algorithm service component 106.

[0039] The test control component 101 receives test task configuration information input by the user and allocates it to either the static test component 102 or the dynamic test component 103 according to the test task type in the configuration information. The test task configuration information includes the test task type, which includes static test tasks and dynamic test tasks.

[0040] The static test component 102 is communicatively connected to the test control component 101, the intelligent test algorithm component 104, and the algorithm under test service component 106, and is specifically used for independent testing of perception-type algorithms. Its core function is to receive the configuration information of the static test task, obtain the address of the original dataset under test according to the configuration information of the static test task sent by the test control component 101, and transmit the address of the original dataset under test to the intelligent test algorithm component 104.

[0041] The dynamic test component 103 is communicatively connected to the test control component 101, the intelligent test algorithm component 104, the simulation component 105, and the algorithm under test service component 106. It is suitable for dynamic interactive testing of decision-making and planning algorithms, and is also compatible with dynamic scene testing of perception algorithms. Its core function is to receive the configuration information of the dynamic test task sent by the test control component 101, obtain the simulation task configuration information according to the configuration information of the dynamic test task, and send the simulation task configuration information to the intelligent test algorithm component 104.

[0042] The intelligent testing algorithm component 104 is communicatively connected to the static testing component 102 and the dynamic testing component 103, and incorporates multiple intelligent testing algorithms. The intelligent testing algorithm component 104 receives the address of the original test dataset sent by the static testing component 102, obtains the original test dataset based on the address, performs enhancement processing on the original test dataset to obtain a new test dataset, and returns the address of the new test dataset to the static testing component 102. The intelligent testing algorithm component 104 also receives simulation task configuration information sent by the dynamic testing component 103, performs variable feature sampling on the simulation task configuration information to obtain sampled new simulation task configuration information, and sends it to the dynamic testing component 103.

[0043] The simulation component 105 is communicatively connected to the dynamic testing component 103 and the algorithm under test service component 106, respectively. It receives new simulation task configuration information sent by the dynamic testing component 103 and performs simulation. Based on this configuration information, it constructs a high-fidelity simulation environment, completes the simulation environment construction and initialization, and sends an initialization completion signal back to the dynamic testing component 103. Upon receiving the initialization completion signal, the dynamic testing component 103 starts the algorithm under test service component 106. During dynamic testing, the simulation component 105 also interacts with the algorithm under test service component 106, providing interactive scenario support for the dynamic algorithm under test, providing a simulation environment to the algorithm under test service component 106, receiving carrier control commands sent by the algorithm under test service component 106 to control the carrier in the simulation environment, and providing real-time feedback of carrier operating status data to the algorithm under test service component 106.

[0044] The algorithm under test service component 106 is communicatively connected to the static testing component 102, the dynamic testing component 103, and the simulation component 105, respectively. The algorithm under test service component 106 receives the address of a new dataset to be tested and the name of the algorithm under test sent by the static testing component 102, enabling it to start the corresponding algorithm. It then retrieves the new dataset to be tested based on the address, performs static testing on the new dataset, obtains the test results, and sends them to the static testing component 102. The algorithm under test service component 106 also receives new simulation task configuration information, start instructions, and the name of the algorithm under test from the dynamic testing component 103, enabling it to start the corresponding algorithm. After starting the algorithm, the algorithm under test service component 106 interacts with the simulation component 105, receives the simulation environment sent by the simulation component 105, performs dynamic testing according to the algorithm, generates carrier control instructions, outputs them to the simulation component 105, and controls the carrier in the simulation environment. The algorithm under test service component 106 is also used to receive the motion state data of the carrier returned by the simulation component 105, and adjust the carrier control commands according to the carrier's operating state data until the dynamic test ends, thereby obtaining the test results. The algorithm under test service component 106 is also used to send the test results to the dynamic test component 103.

[0045] like Figure 1 As shown, the algorithm service component 106 under test includes: a static algorithm subcomponent 1061 and a dynamic algorithm subcomponent 1062.

[0046] The static algorithm sub-component 1061 is used to receive the address of the new dataset to be tested and the name of the algorithm to be tested sent by the static test component 102, so that the static algorithm sub-component 1061 starts the corresponding algorithm, obtains the new dataset to be tested according to the address of the new dataset to be tested, performs static testing, obtains the test results and sends them to the static test component 102.

[0047] The dynamic algorithm sub-component 1062 is used to receive new simulation task configuration information, start instructions and the name of the algorithm under test sent by the dynamic test component, so that the algorithm under test service component can start the corresponding algorithm to perform dynamic testing according to the simulation environment provided by the simulation component 105, generate carrier control instructions to interact with the simulation component 105, and also receive the motion state data of the carrier returned by the simulation component 105, and adjust the carrier control instructions according to the carrier's running state data until the dynamic test ends, so as to obtain the test results and send them to the dynamic test component 103.

[0048] According to an example embodiment, the intelligent testing service system further includes a database 107.

[0049] Database 107 is communicatively connected to static test component 102, dynamic test component 103, intelligent test algorithm component 104, simulation component 105 and test algorithm service component 106 respectively. It is used to store the original test dataset, the enhanced new test dataset, simulation task configuration information, the sampled new simulation task configuration information, test process data and test results to form a structured test record, which supports subsequent query and traceability.

[0050] The static testing component 102 receives the configuration information of the static testing task, retrieves the address of the original dataset to be tested from the database 107 according to the configuration information, transmits the original dataset address to the intelligent testing algorithm component 104, receives the processed new dataset address and forwards it to the algorithm under test service component 106 to start the static algorithm under test in the algorithm under test service component 106. The static testing component 102 monitors the processing progress in real time, receives the processing results, and finally stores the test-related data in the database 107 to form test records.

[0051] The dynamic testing component 103 receives the configuration information of the dynamic testing task, retrieves the simulation task configuration information from the database 107 based on the configuration information, transmits the simulation task configuration information to the intelligent testing algorithm component 104, receives the sampled new simulation task configuration information and stores it in the database 107, and simultaneously forwards the new simulation task configuration information to the simulation component 105. After the simulation component 105 completes environment initialization, it sends a start request to the algorithm under test service component 106, triggering the algorithm under test service component 106 to execute the test. The dynamic testing component 103 monitors the processing progress in real time, receives the processing results, and finally stores the test-related data in the database 107 to form test records.

[0052] According to an example embodiment, the intelligent testing service system further includes a front-end interaction component 108.

[0053] The front-end interaction component 108 is communicatively connected to the test control component 101 to obtain test task configuration information and stop task control commands input by the user. The front-end interaction component 108 transmits the test task configuration information and stop task control commands to the test control component 101. The front-end interaction component 108 is also used to visualize test results, including test progress, test stages, comparison of the original and new datasets, real-time simulation footage, and algorithm test performance data. This application uses this as an example only, but the visualized data is not limited to this.

[0054] The test control component 101 is also used to receive the stop task control command forwarded by the front-end interaction component 108, and send a termination command to the static test component 102 or the dynamic test component 103 according to the test task type in the test task configuration information to stop the test service.

[0055] The static test component 102 sends a termination request to the intelligent test algorithm component 104 or the algorithm under test service component 106 according to the current stage, stops the test, and stores the generated test data.

[0056] The dynamic test component 103 sends a termination request to the intelligent test algorithm component 104, or the simulation component 105 and the algorithm under test service component 106 according to the current stage, stops the test and stores the generated test data.

[0057] Figure 2 A flowchart illustrating an exemplary embodiment of an intelligent testing service method is shown.

[0058] like Figure 2 As shown, the intelligent testing service method includes steps S201-S206:

[0059] In step S201, the test task configuration information input by the user is received, and the test task is assigned according to the test task type in the test task configuration information.

[0060] According to the example embodiment, the front-end interaction component obtains the test task configuration information input by the user, including the name of the algorithm under test, the name of the dataset to be tested, and the name of the target intelligent test algorithm. The system automatically generates a test ID and transmits the test task configuration information to the test control component. The test control component parses the test task configuration information, identifies it as a static test task, and distributes the test task configuration information to the static test component. The test control component parses the configuration information, identifies it as a dynamic test task, and distributes the test task configuration information to the dynamic test component.

[0061] Static testing process:

[0062] In step S2021, the address of the original dataset to be tested is obtained according to the configuration information of the static test task.

[0063] According to the example embodiment, the static testing component obtains the address of the original dataset to be tested from the database based on the configuration information of the static testing task, and transmits the address, the name of the target intelligent testing algorithm, and the test ID to the intelligent testing algorithm component.

[0064] In step S2031, the original dataset to be tested is obtained according to the address of the original dataset to be tested, and the original dataset to be tested is enhanced to obtain a new dataset to be tested.

[0065] According to the example embodiment, the intelligent testing algorithm component obtains the original dataset to be tested based on the address of the original dataset to be tested, starts the corresponding static testing algorithm, performs enhancement processing on the original dataset to be tested, generates a new dataset to be tested and stores it in the database, and returns the address of the new dataset to be tested to the static testing component.

[0066] In step S2041, static testing is performed based on the new dataset to be tested to obtain test results.

[0067] According to the example embodiment, the static testing component forwards the address of the new dataset to be tested to the static algorithm sub-component of the algorithm under test service component, and transmits the name of the algorithm under test, so that the static algorithm sub-component starts the corresponding algorithm under test. The static algorithm sub-component obtains the new dataset to be tested based on the new dataset to be tested, executes the test based on the new dataset to be tested, and outputs the test results to the static testing component. The static testing component associates and stores the test ID, the original dataset to be tested address, the new dataset to be tested address, and the test results in the database to form a test record, and synchronizes it to the front-end interactive component for visualization.

[0068] Dynamic testing process:

[0069] In step S2022, the simulation task configuration information is obtained based on the configuration information of the dynamic test task.

[0070] According to the example embodiment, the dynamic testing component obtains the simulation task configuration information from the database based on the test task configuration information, and transmits the simulation task configuration information, the target intelligent test algorithm name, the name of the algorithm under test, and the test ID to the intelligent test algorithm component.

[0071] In step S2032, variable element sampling is performed on the simulation task configuration information to obtain the new simulation task configuration information after sampling.

[0072] According to the example embodiment, the intelligent test algorithm component starts the corresponding dynamic test algorithm, samples the variable elements in the simulation task configuration information, generates new simulation task configuration information and stores it in the database, and returns the new simulation task configuration information to the dynamic test component.

[0073] In step S2042, simulation is performed according to the new simulation task configuration information to obtain the simulation environment.

[0074] According to the example embodiment, the dynamic testing component forwards the new simulation task configuration information to the simulation component, triggering the simulation component to initialize the simulation environment. After completing the environment construction, the simulation component sends an initialization completion signal back to the dynamic testing component.

[0075] In step S2052, dynamic testing is performed based on the new simulation task configuration information and simulation environment to obtain test results.

[0076] According to the example embodiment, upon receiving the initialization completion signal, the dynamic testing component forwards the new simulation task configuration information, the start signal, and the corresponding name of the algorithm under test to the dynamic algorithm sub-component of the algorithm under test service component, thus starting the dynamic algorithm under test. The dynamic algorithm sub-component executes the test based on the simulation environment, configuration information, and the corresponding name of the algorithm under test, interacting with the simulation component and outputting carrier control commands to it. The simulation component controls the carrier in the simulation environment according to the carrier control commands and re-outputs the carrier's operating status data to the dynamic algorithm sub-component. The dynamic algorithm sub-component adjusts the carrier control commands based on the carrier's operating status data. Until the test ends, the dynamic algorithm sub-component outputs the test results to the dynamic testing component. The dynamic testing component associates and stores the test ID, simulation task configuration information, new simulation task configuration information, simulation process data (including received control commands, key events, etc.), simulation results, and test results in the database to form a test record, and synchronizes it to the front-end interactive component for visualization.

[0077] The intelligent testing service method in this application also includes a test termination process.

[0078] In step S301, a stop task control command input by the user is received, and a termination command is sent to the static test component or the dynamic test component according to the test task type in the test task configuration information.

[0079] According to the example embodiment, the front-end interaction component obtains the user's input stop task control command and transmits the command and the corresponding test ID to the test control component. The test control component queries the current test task's running status (static test / dynamic test, current stage) based on the test ID. If it is a static test, the test control component sends a termination command to the static test component. If it is a dynamic test, the test control component sends a termination command to the dynamic test component.

[0080] In step S302, when the static test component receives a termination instruction, it sends a termination request to the intelligent test algorithm component or the algorithm under test service component to stop the test service.

[0081] According to the example embodiment, the static testing component sends a termination request to the intelligent testing algorithm component or the algorithm under test service component based on the current stage, stops the test, and stores the generated test data.

[0082] In step S303, when the dynamic test component receives a termination instruction, it sends a termination request to the intelligent test algorithm component, or the simulation component and the algorithm under test service component, to stop the test service.

[0083] According to the example embodiment, the dynamic testing component sends a termination request to the intelligent testing algorithm component, or the simulation component and the algorithm under test service component, based on the current stage, to stop the test and store the generated test data. When in the intelligent testing service stage, the dynamic testing component requests the termination interface of the intelligent testing component; when in the algorithm under test stage, the dynamic testing component first requests the termination interface of the simulation component, and then requests the termination interface of the algorithm under test service component.

[0084] In step S304, the test control component synchronizes the termination status to the front-end interaction component, thus completing the test termination.

[0085] The following detailed description of the intelligent testing service system and method of this application, combined with specific scenarios, enables those skilled in the art to accurately implement the technical solution based on the description.

[0086] I. Static Test Implementation Example (Taking YOLOV8 Perception Algorithm Test as an Example)

[0087] 1. Test configuration phase: Users select the original dataset to be tested as the COCO dataset through the front-end interactive component, the name of the algorithm to be tested as the YOLOV8 object detection algorithm, the object intelligence testing algorithm as the Gaussian noise addition algorithm, the system automatically matches the test task type as static test, and generates a unique test ID (such as TEST_STATIC_001). After the user confirms, the test task configuration information is submitted.

[0088] 2. Task Distribution Phase: The front-end interactive component transmits the test task configuration information to the test control component. The test control component parses the task type as static test and forwards the corresponding configuration information (test ID, name of the algorithm under test, name of the dataset, name of the target intelligent test algorithm) to the static test component.

[0089] 3. Original dataset address acquisition stage: The static testing component queries the database based on the COCO dataset name and obtains the storage address of the original dataset to be tested (e.g., / data / original / COCO / ).

[0090] 4. Data Augmentation Scheduling Phase: The static test component transmits the original dataset address to be tested, the name of the Gaussian noise addition algorithm, and the test ID TEST_STATIC_001 to the intelligent test algorithm component. The intelligent test algorithm component starts the corresponding algorithm execution environment and returns the intelligent test ID (such as ALGORITHM_ID_001) for progress query.

[0091] 5. Data Augmentation Processing Phase: The intelligent testing algorithm component initiates the Gaussian noise addition algorithm, downloads image data from the original test dataset address, generates and adds Gaussian noise of random intensity to each image sequentially, generating an enhanced new test dataset. During processing, the static testing component queries the progress in real time using the intelligent test ID ALGORITHM_ID_001 (progress is expressed as a percentage of the dataset processed) and feeds the progress back to the test control component. The front-end interactive component queries and displays the status by test ID, such as "Data Augmentation in Progress: 60%".

[0092] 6. New Dataset Storage and Feedback Phase: After the intelligent testing algorithm component completes data augmentation, it stores the new dataset to be tested in the specified path of the database (e.g., / data / enhanced / COCO_Gaussian_001 / ) and returns the address of the new dataset to be tested to the static testing component. At the same time, the front-end interactive component updates its display status to "Data Augmentation Complete".

[0093] 7. Startup Phase of the Algorithm Under Test: After receiving the address of the new dataset to be tested, the static testing component sends a startup request to the static algorithm sub-component of the algorithm under test service component, carrying the address of the new dataset to be tested, the name of the algorithm under test (yoloV8), and the test ID. The static algorithm sub-component starts the corresponding yoloV8 algorithm execution environment and returns the ID of the algorithm under test (e.g., DETECTOR_ID_001) for progress query.

[0094] 8. Static Test Execution Phase: The static test component queries the running progress of the YOLOV8 algorithm using the tested algorithm ID DETECTOR_ID_001, and simultaneously updates the test task phase to "Tested Algorithm Running," which is then displayed by the front-end interactive component. The YOLOV8 algorithm reads the new test dataset and performs object detection, transmitting the processing results (including the coordinates of the object detection bounding boxes, category, and confidence score for each image) to the static test component in real time.

[0095] 9. Test record storage stage: The static test component associates and stores the test ID TEST_STATIC_001, the address of the original dataset to be tested, the address of the new dataset to be tested, and the processing results (detection boxes, categories) of the YOLOV8 algorithm in the database to form a complete test record.

[0096] 10. Result Visualization Stage: Users query the results for test ID TEST_STATIC_001 through the front-end interactive component. The front-end interactive component sends a request to the test control component, which retrieves the corresponding test record from the database and returns the address of the original test dataset, the address of the new test dataset, and the detection results. The front-end interactive component displays the original image and the enhanced image in sequence, and overlays the detection box, category label, and confidence score on the enhanced image for users to intuitively compare algorithm performance.

[0097] II. Dynamic Test Implementation Example (Taking A* Path Planning Algorithm Test as an Example)

[0098] 1. Test Configuration Phase: Users select path planning as the test task, A* algorithm as the algorithm under test, and Monte Carlo sampling algorithm as the target intelligent test algorithm through the front-end interactive component. The system automatically matches the test task type as dynamic test and generates a test ID (such as TEST_DYNAMIC_001). After the user confirms, the configuration information is submitted.

[0099] 2. Task distribution phase: The front-end interaction component transmits the configuration information to the test control component, which parses it into a dynamic test and forwards it to the dynamic test component.

[0100] 3. Simulation Configuration Acquisition Phase: The dynamic testing component retrieves simulation task configuration information from the database based on the path planning task name, including the simulation map (city road map), sampleable environmental elements and range (number of obstacles: 1-10, location: random area of ​​the road; weather: sunny / rainy / foggy), and vehicle parameters (maximum speed 30km / h, turning radius 5m).

[0101] 4. Simulation configuration sampling scheduling phase: The dynamic test component transmits the simulation task configuration information, Monte Carlo sampling algorithm name and test ID TEST_DYNAMIC_001 to the intelligent test algorithm component. The intelligent test algorithm component starts the corresponding algorithm execution environment and returns the intelligent test ID (such as ALGORITHM_ID_002).

[0102] 5. Simulation Configuration Sampling Phase: The intelligent testing algorithm component initiates the Monte Carlo sampling algorithm, parses the sampleable elements and ranges in the simulation task configuration information, randomly samples within the range, and generates 10 sets of differentiated new simulation task configuration files (e.g., Configuration 1: 3 obstacles, sunny day; Configuration 2: 8 obstacles, rainy day, etc.). The dynamic testing component queries the progress (represented by the percentage of simulation rounds with completed sampling out of the total rounds) through the intelligent testing ID ALGORITHM_ID_002, and displays it on the front-end interactive component, for example: "Simulation configuration sampling in progress: 80%".

[0103] 6. New Simulation Configuration Storage and Forwarding Phase: After sampling, the intelligent test algorithm component stores 10 sets of new simulation task configuration files in the database (path: / data / simulation / config / TEST_DYNAMIC_001 / ) and returns the configuration files to the dynamic test component. The dynamic test component stores the configuration files with the test ID and forwards them to the simulation component, sending an environment initialization request.

[0104] 7. Simulation Environment Construction Phase: After receiving the new simulation task configuration file, the simulation component asynchronously constructs the corresponding 10 sets of simulation environments. Upon completion, it returns "Initialization Complete" feedback to the dynamic testing component. The front-end interactive component updates its display status to "Simulation Environment Ready".

[0105] 8. Startup Phase of the Algorithm Under Test: After receiving the initialization completion feedback, the dynamic test component sends a startup request to the dynamic algorithm sub-component of the algorithm under test service component, carrying the new simulation task configuration information, the name of algorithm A, the carrier control interface information, and the test ID. The dynamic algorithm sub-component starts the execution environment of algorithm A and returns the ID of the algorithm under test (e.g., PLANNER_ID_001).

[0106] 9. Dynamic Test Execution Phase: The dynamic test component queries the running progress of Algorithm A through the tested algorithm ID PLANNER_ID_001, updates the test phase to "Algorithm under test is running," and feeds back to the front-end interactive component. The dynamic algorithm sub-component executes Algorithm A, obtains the global semantic map of the simulation environment through the carrier data acquisition interface, plans the globally optimal path based on obstacle positions, weather conditions, and other information in the configuration file of each new simulation task, and sends motion commands (such as turning, accelerating, and decelerating) to the simulation component through the control interface. The simulation component executes the commands and provides real-time feedback on the carrier status (such as position, speed, and whether a collision has occurred) until the dynamic test ends and the test results are obtained. The front-end interactive component initiates a push request to the simulation component to display the simulation process in real time (such as the carrier avoiding obstacles on urban roads).

[0107] 10. Test Result Storage Phase: When a new simulation configuration's test meets the termination condition (vehicle reaches its destination), the dynamic algorithm subcomponent sends the test results to the dynamic testing component. The dynamic testing component records the test results for that configuration (e.g., planned path length) and simulation process data (travel time, collision status). If the timeout occurs and the destination is not reached, or a collision occurs, the results are also recorded, and the dynamic algorithm subcomponent sends the test results to the dynamic testing component. After all 10 configurations have been tested, the dynamic testing component associates and stores the test ID, simulation task configuration information, new simulation task configuration file, and each set of test results in the database, forming a test record.

[0108] III. Test Termination Implementation Example (Taking Dynamic Test Termination as an Example)

[0109] 1. Termination Request Initiation Phase: When the dynamic test component executes the A* algorithm test, the user queries the current test phase through the front-end interactive component and finds that "the algorithm under test is running: 50%". Because the simulation configuration parameters are found to be unreasonable, the user clicks the "Terminate Test" button. The front-end interactive component sends a stop task control command to the test control component with the test ID TEST_DYNAMIC_001.

[0110] 2. Termination command distribution phase: The test control component queries the current running status corresponding to the test ID and finds it to be "Dynamic Test - Algorithm under test is running". It then sends a termination command to the dynamic test component, carrying the test ID.

[0111] 3. Termination Phase: After receiving the termination command, the dynamic testing component determines the current phase of the dynamic test. When in the intelligent test service phase, the dynamic testing component requests the termination interface of the intelligent test component; when in the algorithm under test phase, the dynamic testing component first requests the termination interface of the simulation component, and then requests the termination interface of the algorithm under test service component. The A* algorithm execution environment stops running and returns "Terminated".

[0112] 4. Termination Result Storage and Display Phase: The dynamic testing component stores the test termination status, the test data of the 5 completed new simulation configurations, and the test IDs in the database. The front-end interactive component displays "Test terminated, 5 / 10 configuration tests completed".

[0113] Through the above-described exemplary embodiments, the intelligent testing service system and method provided by this application have the following beneficial effects: First, this application reduces system coupling through modular design, possessing strong scalability. The testing process is divided into independent components such as test control, static testing, dynamic testing, and intelligent testing algorithms. Each component interacts with data through standardized interfaces. Replacing or upgrading a single testing component does not require modifying the overall system, flexibly adapting to the iterative needs of different types of tested algorithms and test scenarios. Second, this application enhances the comprehensiveness of perception algorithm testing through independent static testing components. A dedicated testing process is designed specifically for perception algorithms, and intelligent data augmentation technology expands the diversity of test samples. Targeted verification can be achieved without relying on overall system testing, avoiding the limitations of single-scenario testing and significantly reducing repetitive testing costs. Furthermore, this application optimizes testing efficiency and reduces resource consumption by leveraging intelligent testing algorithm components. It rapidly generates key test samples and simulation configurations through static data augmentation algorithms (such as Gaussian noise addition and deep learning perturbation models) and dynamic scene sampling algorithms (such as Monte Carlo sampling), avoiding simple stacking of test cases and significantly reducing invalid tests while ensuring test coverage. Finally, this application is compatible with both types of testing tasks and has broad adaptability. Through a unified central control scheduling and structured data storage mechanism, it achieves seamless integration of static dataset testing of perception algorithms and dynamic simulation testing of decision and planning algorithms, meeting the full-process algorithm testing needs of unmanned systems. Simultaneously, it is equipped with front-end visual interaction and full-process test record traceability functions, lowering the user's operational threshold and further expanding the application scope of intelligent testing technology in fields such as autonomous driving.

[0114] Figure 3 This diagram illustrates the structure of an electronic device provided in this application.

[0115] See Figure 3 , Figure 3 An electronic device is provided, including a processor and a memory. The memory stores computer instructions, which, when executed by the processor, cause the processor to perform the computer instructions to achieve the following: Figure 2 The method and its detailed scheme are shown.

[0116] It should be understood that the above-described device embodiments are merely illustrative, and the device disclosed in this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0117] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0118] When an integrated unit / module is implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor or chip can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, on-chip cache, off-chip memory, and storage can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0119] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments disclosed herein. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0120] This application also provides a non-transitory computer storage medium storing a computer program, which, when executed by multiple processors, causes the processors to perform actions such as... Figure 2 The method and its detailed scheme are shown.

[0121] It should be clearly understood that this application describes how specific examples are formed and used, but this application is not limited to any details of these examples. Rather, based on the teachings of the disclosure of this application, these principles can be applied to many other embodiments.

[0122] Furthermore, it should be noted that the above figures are merely illustrative representations of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0123] Exemplary embodiments of this application have been specifically shown and described above. It should be understood that this application is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, this application is intended to cover various modifications and equivalent arrangements that fall within the objectives and scope of the appended claims.

Claims

1. An intelligent test service system, characterized by, Comprise: A test control component for receiving user input test task configuration information and distributing according to the test task type of the test task configuration information, the test task configuration information comprising the test task type, the test task type comprising static test tasks and dynamic test tasks; A static test component in communication connection with the test control component, for receiving configuration information of static test tasks and obtaining original test data set addresses according to the configuration information of the static test tasks; A dynamic test component in communication connection with the test control component, for receiving configuration information of dynamic test tasks and obtaining simulation task configuration information according to the configuration information of the dynamic test tasks; An intelligent test algorithm component in communication connection with the static test component and the dynamic test component respectively, for receiving the original test data set addresses, obtaining original test data sets according to the original test data set addresses, enhancing the original test data sets to obtain new test data sets, and returning new test data set addresses to the static test component according to the new test data sets, and for receiving the simulation task configuration information, sampling variable elements of the simulation task configuration information to obtain sampled new simulation task configuration information and sending to the dynamic test component; A measured algorithm service component in communication connection with the static test component and the dynamic test component respectively, for receiving the new test data set addresses sent by the static test component, obtaining new test data sets according to the new test data set addresses, and performing static test according to the new test data sets to obtain test results and send to the static test component, and for receiving the new simulation task configuration information sent by the dynamic test component to perform dynamic test, obtaining test results and sending to the dynamic test component. Also comprise:

2. The intelligent test service system of claim 1, wherein, A simulation component in communication connection with the dynamic test component and the measured algorithm service component respectively, for receiving the new simulation task configuration information sent by the dynamic test component and performing simulation, completing simulation environment construction and initialization, and feeding back initialization completion signals to the dynamic test component; In the case that the dynamic test component receives the initialization completion signal, the dynamic test component is used to start the measured algorithm service component; During dynamic testing, the simulation component is also used to interact with the measured algorithm service component, provide the simulation environment to the measured algorithm service component, receive carrier control instructions sent by the measured algorithm service component to control carriers in the simulation environment, and feed back running state data of the carriers to the measured algorithm service component in real time; wherein the measured algorithm service component is used to adjust the carrier control instructions according to the running state data of the carriers, and the measured algorithm service component is also used to send the test results to the dynamic test component. The measured algorithm service component comprises:

3. The intelligent test service system of claim 2, wherein, ​ a static algorithm subcomponent configured to receive the new to-be-tested data set address sent by the static test component, and to obtain a new to-be-tested data set according to the new to-be-tested data set address for static testing, and to obtain a test result and send the test result to the static test component; a dynamic algorithm subcomponent configured to receive a start signal and the new simulation task configuration information sent by the dynamic test component, to perform dynamic testing according to a simulation environment provided by the simulation component, to generate the carrier control instruction and interact with the simulation component, and to adjust the carrier control instruction according to the running state data of the carrier, so as to obtain the test result and send the test result to the dynamic test component.

4. The intelligent test service system of claim 2, wherein, Further comprising: a database in communication connection with the static test component, the dynamic test component, the intelligent test algorithm component, the simulation component, and the measured algorithm service component respectively, configured to store original to-be-tested data sets, enhanced new to-be-tested data sets, simulation task configuration information, sampled new simulation task configuration information, test process data, and test results, so as to form a test record.

5. The intelligent test service system of claim 1, wherein, Further comprising: a front-end interaction component configured to obtain test task configuration information input by a user, and to visually display a test result.

6. The intelligent test service system of claim 1, wherein, The test control component is further configured to receive a stop task control instruction input by a user, and to send a termination instruction to the static test component or the dynamic test component according to a test task type of the test task configuration information, so as to stop the test service.

7. An intelligent test service method characterized by, The intelligent test service method is executed by the intelligent test service system according to any one of claims 1-6, and the intelligent test service method comprises: receiving test task configuration information input by a user, and distributing according to a test task type of the test task configuration information, wherein the test task configuration information comprises the test task type, and the test task type comprises a static test task and a dynamic test task; obtaining an original to-be-tested data set address according to configuration information of the static test task; obtaining an original to-be-tested data set according to the original to-be-tested data set address, and performing enhancement processing on the original to-be-tested data set, so as to obtain a new to-be-tested data set; performing static testing according to the new to-be-tested data set, and obtaining a first test result; obtaining simulation task configuration information according to configuration information of the dynamic test task; sampling a variable element of the simulation task configuration information, and obtaining sampled new simulation task configuration information; performing simulation according to the new simulation task configuration information, and obtaining a simulation environment; performing dynamic testing according to the new simulation task configuration information and the simulation environment, and obtaining a second test result.

8. The intelligent test service method of claim 7, wherein, Further comprising: receiving a stop task control instruction input by a user, and sending a termination instruction to the static test component or the dynamic test component according to a test task type of the test task configuration information; in a case where the static test component receives the termination instruction, sending a termination request to the intelligent test algorithm component or the measured algorithm service component, so as to stop the test service; In the case that the dynamic testing component receives the termination instruction, a termination request is sent to the intelligent testing algorithm component, or the simulation component and the measured algorithm service component, to stop the testing service.

9. An electronic device, comprising: comprising: one or more processors; memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to claim 7 or 8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, the program is executed by the processor to implement the method according to claim 7 or 8. the program is executed by the processor to implement the method according to claim 7 or 8.

Images