Vehicle cloud and road cloud integrated test evaluation system and method

Abstract

The invention discloses a vehicle cloud and road cloud integrated test evaluation system and method, and the system comprises a test management module which is used for creating, scheduling and monitoring a vehicle cloud and road cloud integrated test task; the test scene construction module is used for constructing diversified test scenes; the data acquisition module is used for acquiring vehicle cloud data and road cloud data; the data analysis module is used for analyzing the collected vehicle cloud data and road cloud data; and the evaluation decision module is used for comprehensively evaluating the system performance according to the multi-dimensional evaluation index system based on the data analysis result. According to the invention, comprehensive and integrated testing can be carried out on vehicle cloud and road cloud interaction, the association between vehicle cloud and road cloud interaction data is deeply mined, and a perfect multi-dimensional evaluation system is established, so that the overall performance of the vehicle cloud and road cloud integration system is accurately evaluated.

Description

Technical Field

[0001] This application relates to the field of intelligent connected vehicles, and in particular to a vehicle-cloud and road-cloud integrated testing and evaluation system and method. Background Technology

[0002] With the rapid development of vehicle-road-cloud integration technology, the complexity of vehicle-cloud and road-cloud integrated systems is increasing day by day, and their performance and reliability are directly related to the overall safety and efficiency of intelligent connected vehicles.

[0003] Traditional testing methods often focus on the independent verification of vehicles or roadside units, lacking a systematic evaluation method for the deep interaction and collaborative performance among vehicles, roads, and the cloud. This fragmented testing model makes it difficult to reproduce real-world complex operating scenarios and cannot effectively evaluate the system's functional compliance, data transmission timeliness, and overall stability during dynamic interactions. This results in incomplete test coverage and a single evaluation dimension, which has become a key bottleneck restricting system iteration optimization and large-scale application deployment.

[0004] Therefore, building an integrated testing and evaluation system that can simulate real-world environments and achieve automated testing and multi-dimensional comprehensive evaluation has become an urgent technical need for the industry. Summary of the Invention

[0005] The purpose of this invention is to provide an integrated testing and evaluation system and method for vehicle-cloud and road-cloud systems, so as to solve the technical problems of lack of integration in testing, insufficient data correlation analysis and imperfect evaluation system in the prior art.

[0006] This invention provides the following solution:

[0007] A vehicle-cloud and road-cloud integrated testing and evaluation system includes:

[0008] The test management module is used to create, schedule, and monitor test tasks for the integration of vehicle cloud and road cloud.

[0009] The test scenario building module is used to construct diverse test scenarios;

[0010] The data acquisition module is used to collect vehicle cloud data and road cloud data;

[0011] The data analysis module is used to analyze the collected vehicle cloud data and road cloud data;

[0012] The evaluation and decision-making module is used to comprehensively evaluate the system performance based on data analysis results and a multi-dimensional evaluation index system.

[0013] Furthermore, the test management module generates executable test tasks and breaks them down into multiple sub-tasks based on the test requirements submitted by the user. It selects matching scenarios from a preset scenario library, configures functional and non-functional test parameters, monitors resource consumption and data quality indicators in real time during test execution, and finally summarizes the test results and generates a report.

[0014] Furthermore, the test scenario construction module includes:

[0015] The scene mapping unit is used to determine key scene elements according to the actual application scenario and to assign weights to the elements based on the analytic hierarchy process.

[0016] The scene element modeling unit is used to perform structured modeling of vehicle, roadside, platform and environmental elements using a unified modeling language, and to analyze the relationships between elements based on graph theory.

[0017] The scene generation unit is used to convert the modeled scene into an executable simulation script and adaptively adjust the scene parameters based on reinforcement learning algorithms.

[0018] Furthermore, the scene mapping unit confirms the core elements of the scene based on the actual application scenario. The core elements of the scene include multiple factors such as intersection size, traffic flow, intersection queue length, weather interference, and road facility density.

[0019] Furthermore, in the element association graph constructed by the scene element modeling unit, nodes represent scene elements, edges represent interaction relationships between elements, and key interaction relationships are filtered through association coefficients;

[0020] The scene generation unit dynamically adjusts scene parameters using the Q-Learning algorithm, with the matching degree between the scene and the real environment as the reward function.

[0021] Furthermore, the data analysis module includes:

[0022] The data processing unit is used to extract relevant fields from the raw data stream and classify them according to the test item type;

[0023] The data analysis unit is used to perform functional compliance verification and non-functional performance calculations on the categorized test data.

[0024] Furthermore, the data processing unit extracts the corresponding field information from the original communication data stream for each test subtask based on the data structures defined in T / CSAE 295.2-2023 and T / CSAE 295.3-2023.

[0025] Furthermore, the data analysis unit performs rule matching and integrity verification for functional tests; performs quantitative calculations for non-functional tests; and calls protocol testing tools to automatically verify the extracted encoded fields for protocol consistency testing, determining whether all messages conform to the T / CSAE 295.2-2023 protocol specification.

[0026] Furthermore, the evaluation and decision-making module includes:

[0027] The indicator import unit is used to configure corresponding evaluation indicator parameters for each test subtask before or during the execution of the test task, and to establish an evaluation benchmark.

[0028] The evaluation and decision-making unit is used to calculate the pass rate of test tasks and determine the overall performance of the system based on the execution results of test sub-tasks and preset evaluation indicators.

[0029] According to another aspect of this application, a method for testing and evaluating the integration of vehicle-cloud and road-cloud is provided, comprising:

[0030] Step S101: Create, schedule, and monitor the vehicle cloud and road cloud integration test task;

[0031] Step S102: Construct diverse test scenarios;

[0032] Step S103: Collect vehicle cloud data and road cloud data;

[0033] Step S104: Analyze the collected vehicle cloud data and road cloud data;

[0034] Step S105: Based on the data analysis results, conduct a comprehensive evaluation of the system performance according to the multi-dimensional evaluation index system.

[0035] The above solution achieves the following beneficial technical effects:

[0036] By organically coordinating modules such as test management, scenario construction, data collection, analysis, and evaluation decision-making, an automated, standardized, and systematic vehicle-cloud and road-cloud integrated testing and evaluation platform has been built. It effectively overcomes the limitations of traditional testing methods, achieving a closed-loop process from test task initialization, diverse scenario generation, real-time data collection to in-depth analysis and intelligent decision-making. This system significantly improves the breadth, depth, and efficiency of testing, enabling comprehensive evaluation of the system's functional compliance and non-functional performance under various complex operating conditions. Its ultimate value lies in providing comprehensive and reliable data support and clear optimization directions for system development, greatly accelerating the verification process of the vehicle-road-cloud integrated system, and playing a crucial role in ensuring the reliability, security, and efficiency of the final system application. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of a vehicle-cloud and road-cloud integrated testing and evaluation system provided by one or more embodiments of the present invention.

[0038] Figure 2 This is a schematic diagram of the test scenario construction module of the present invention.

[0039] Figure 3 This is a schematic diagram of the data analysis module of the present invention.

[0040] Figure 4 This is a schematic diagram of the evaluation and decision-making module of the present invention.

[0041] Figure 5 This is a flowchart of the vehicle-cloud and road-cloud integration testing and evaluation method of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Specifically, the system described in this embodiment utilizes a vehicle-road-cloud integrated testing environment. The test management module initializes test tasks, the scenario construction module generates test scenarios, the data acquisition module collects interactive data in real time, the data analysis module processes and analyzes the data, and the evaluation and decision-making module outputs evaluation results. The system supports multiple test types, enabling comprehensive evaluation of the performance of the vehicle-cloud and road-cloud integrated system, and providing data support for system optimization.

[0044] Please see Figure 1 As shown, this is a schematic diagram of the integrated testing and evaluation system for vehicle-cloud and road-cloud in this embodiment. The system includes:

[0045] The test management module is used to create, schedule, and monitor vehicle cloud and road cloud integration test tasks.

[0046] Specifically, the test management module generates executable test tasks and breaks them down into multiple sub-tasks based on the test requirements submitted by the user. It selects matching scenarios from a preset scenario library, configures functional and non-functional test parameters, monitors resource consumption and data quality indicators in real time during test execution, and finally summarizes the test results and generates a report.

[0047] For example, the test management module receives test requirements submitted by users, such as "testing the response performance of vehicle-to-cloud remote control commands in urban road environments." Based on this, the system creates vehicle-to-cloud data interaction test tasks and / or road-to-cloud data interaction test tasks, and breaks down the tasks into multiple test sub-tasks that can be executed in parallel. The total number of sub-tasks is set to N, where N≥1. In this embodiment, the vehicle-to-cloud data interaction test task includes 11 sub-tasks, specifically including: vehicle-to-cloud network connection and rule testing, transmission timing and data packet structure testing, vehicle static parameter information return testing, remote control command issuance and response testing, navigation path adjustment suggestion issuance testing, fault alarm upload testing, data transmission latency testing, packet loss rate testing, protocol consistency testing, system stability stress testing, and multi-scenario switching adaptability testing. This embodiment does not specifically limit the settings of the sub-tasks; those skilled in the art can freely set them according to their needs.

[0048] According to the testing requirements, select a matching test scenario from the preset scenario library, such as "urban commuting", "highway cruising" and "low visibility in rainy weather", and configure test parameters, including functional test parameters and non-functional test parameters. The functional test parameters can be the frequency of vehicle-cloud decision suggestions and the type of real-time control suggestions in the cloud. The non-functional test parameters can be the data transmission latency requirements and packet loss rate requirements.

[0049] During the test execution, the following metrics are monitored in real time: hardware resource consumption, network bandwidth consumption, data collection completion rate, and protocol test pass rate;

[0050] After the test is completed, the execution results of each subtask are automatically summarized, a structured test report is generated, and users can edit and confirm parameters such as test metrics and evaluation methods.

[0051] Specifically, the test management module significantly improves the overall efficiency and manageability of the testing process by automating the creation, scheduling, and monitoring of test tasks. It can flexibly generate test tasks according to user needs and break them down into parallel sub-tasks, optimizing resource allocation and utilization. Real-time monitoring ensures that anomalies and resource consumption issues during test execution are detected and corrected promptly, while automatically generated structured test reports simplify the results summary and review process, providing a reliable basis for system optimization.

[0052] Please continue reading. Figure 1 As shown, the vehicle-cloud and road-cloud integrated testing and evaluation system also includes:

[0053] A test scenario building module, which is connected to the test management module, is used to build diverse test scenarios.

[0054] Please see Figure 2As shown, the test scenario construction module includes:

[0055] The scene mapping unit is used to determine key scene elements based on the actual application scenario and to assign weights to the elements based on the analytic hierarchy process.

[0056] Specifically, this unit first selects the target scenario type (such as "urban intersection" or "highway mainline") from a predefined scenario library based on the test requirements (such as testing the vehicle-road cooperative obstacle avoidance function at urban intersections or the remote control response performance of highways).

[0057] Next, the unit uses the analytic hierarchy process (AHP) to determine the weights of core elements in each scenario. Specifically, based on expert experience, a judgment matrix is ​​constructed for each selected scenario type, comprising core elements (including but not limited to traffic flow, intersection queue length, traffic light status, weather interference, and road infrastructure density). For example, for the "urban intersection" scenario, the judgment matrix can be scaled as follows: traffic flow is slightly more important than intersection queue length (scale 3), and traffic flow is significantly more important than weather interference (scale 5). Subsequently, a consistency check is performed on the judgment matrix. When the consistency ratio CR < 0.1, the consistency of the judgment matrix is ​​considered acceptable, and the weight vector for each element is calculated. For example, the final weight allocation is: traffic flow weight 0.30, intersection queue length weight 0.25, traffic light status weight 0.20, weather interference weight 0.15, and road infrastructure density weight 0.10. This process ensures that the scenario design prioritizes and accurately reflects key influencing factors.

[0058] Please continue reading. Figure 2 As shown, the test scenario construction module also includes:

[0059] The scene element modeling unit, which is connected to the scene mapping unit, is used to perform structured modeling of vehicle, roadside, platform and environmental elements using a unified modeling language, and to analyze the relationship between elements based on graph theory.

[0060] Specifically, this unit first constructs a UML scene model, which contains the following four types of elements:

[0061] Vehicle elements include attributes such as vehicle type, speed, acceleration, position coordinates, driving trajectory, and onboard equipment status;

[0062] Roadside elements include attributes such as roadside unit type, radar / camera parameters, traffic light timing, and roadside computing unit status;

[0063] Platform elements: mainly refers to the cloud control platform, including its command issuance rules, data processing logic, and other attributes;

[0064] Environmental elements include weather conditions (such as sunny, rainy, foggy, snowy), rainfall, visibility, and light intensity.

[0065] To optimize the model structure and accurately capture key interactions between elements, this unit integrates a graph theory-based algorithm for analyzing the correlation between scene elements. The specific implementation steps are as follows:

[0066] Each element instance in the UML model is regarded as a node in the graph, and the interaction relationship between elements (such as "traffic light status change → vehicle deceleration" and "roadside radar detects obstacles → cloud platform issues warning command") is regarded as the edge connecting the nodes, thus constructing a directed or undirected scene element relationship graph.

[0067] For each pair of nodes (i.e., element instances) in the graph, the Pearson correlation coefficient of their interaction relationship is calculated as the correlation coefficient based on historical test data or simulation data. The specific calculation formula is as follows:

[0068] ;

[0069] Where Xi and Yi represent the interaction data sequences of two elements over multiple test periods, and n is the number of data points. and are the sequence mean, and r is the correlation coefficient;

[0070] Based on actual testing requirements and data distribution characteristics, the preset correlation coefficient threshold is 0.7. Interactions with a correlation coefficient greater than or equal to 0.7 are identified as strong correlations (e.g., "roadside equipment status - cloud platform feedback") and are highlighted and retained in the model; interactions with a correlation coefficient less than 0.7 are identified as redundant or weak correlations (e.g., "light intensity - vehicle static parameters") and are simplified or eliminated.

[0071] Based on the selected key interaction relationships, the UML model is optimized, redundant structures are removed, and the optimized scene model is finally stored in XML format for easy parsing, transmission, and simulation script generation.

[0072] Please continue reading. Figure 2 As shown, the test scenario construction module also includes:

[0073] A scene generation unit, connected to the scene element modeling unit, is used to convert the modeled scene into an executable simulation script and adaptively adjust the scene parameters based on a reinforcement learning algorithm.

[0074] Specifically, this unit calls the scene generation engine (such as the interface of simulation tools such as Prescan and Simulink) to convert the XML format scene model output by the aforementioned UML modeling unit into an executable simulation script that conforms to standards such as OpenSCENARIO. The test scene is not limited to a pure simulation environment, but also supports integration with real road test fields, integrating the actual test vehicle and roadside equipment in the scene.

[0075] To achieve high-fidelity matching between the test scenario and the real environment, this unit designs a scene parameter adaptive adjustment algorithm based on Q-Learning. This algorithm defines the "matching degree between the scene and the real environment" as a reward function. The matching degree is calculated comprehensively through multiple quantifiable indicators, with a maximum score of 100 points. Specific indicators include: the degree of consistency between simulated traffic flow and real traffic flow trends, the similarity between simulated vehicle trajectories and expected trajectories, and the deviation between device interaction response latency and measured latency. The agent in the algorithm continuously interacts with the simulation environment or the real test environment, and dynamically adjusts various parameters of the test scenario based on the feedback from the reward function. For example:

[0076] When the actual traffic flow during the morning rush hour is detected to increase from 500 vehicles / hour to 1200 vehicles / hour, the algorithm will automatically shorten the vehicle generation interval in the simulation scenario from 5 seconds to 2 seconds.

[0077] When simulating rainy conditions and visibility dropping from 100 meters to 50 meters, the algorithm automatically lowers the effective perception distance parameter of the roadside camera in the scene.

[0078] Specifically, the test scenario building module supports the construction of diverse and highly realistic test scenarios, significantly enhancing the system's verification coverage capabilities in complex environments. Through standardized modeling and automated script generation, this module enables rapid construction of test scenarios and multi-platform compatibility, effectively improving the flexibility of testing and the accuracy of scenario reproduction, laying a solid foundation for comprehensively evaluating the system's adaptability under different operating conditions.

[0079] Please continue reading. Figure 1 As shown, the vehicle-cloud and road-cloud integrated testing and evaluation system also includes:

[0080] The data acquisition module, connected to the test scenario construction module, is used to collect vehicle-cloud data and road-cloud data. The vehicle-cloud data includes vehicle status data, command data, and feedback data. The vehicle status data includes, but is not limited to, vehicle speed, acceleration, GPS coordinates, and direction angle. The command data consists of remote control commands issued from the cloud, such as acceleration, deceleration, and lane change. The feedback data includes vehicle status reporting and command execution confirmation. The road-cloud data includes roadside perception data and road-cloud interaction data. The roadside perception data includes, but is not limited to, traffic flow, road conditions, and traffic light phases. The road-cloud interaction data includes, but is not limited to, road condition information sharing, data reporting, and control command feedback.

[0081] For example, in this embodiment, vehicle-to-cloud data can be collected through on-board testing equipment and road-to-cloud data can be collected through roadside equipment; this embodiment does not specifically limit the data collection method, and those skilled in the art can freely set it according to their needs.

[0082] Please continue reading. Figure 1 As shown, the vehicle-cloud and road-cloud integrated testing and evaluation system also includes:

[0083] A data analysis module, which is connected to the data acquisition module, is used to analyze the acquired vehicle cloud data and road cloud data.

[0084] For details, please refer to Figure 3 As shown, the data analysis module includes:

[0085] The data processing unit is used to extract relevant fields from the raw data stream and classify them according to the test item type.

[0086] Specifically, the data processing unit, based on the data structures defined in T / CSAE 295.2-2023 and T / CSAE 295.3-2023, extracts the corresponding field information from the original communication data stream for each test subtask, as follows:

[0087] For vehicle-to-cloud network connectivity and rule testing, extract the vehicle-to-cloud heartbeat message field from each frame of communication data, including message ID, sending period, and CRC checksum.

[0088] For transmission timing and data packet structure testing, extract the data packet header information, including protocol version number, message type, data length, timestamp, and sequence number, and verify whether its format conforms to the requirements of Section 7.5 of the standard;

[0089] For the test of returning vehicle static parameter information, extract the static attribute fields reported by the vehicle, including the vehicle identification number (VIN), license plate number, vehicle model, and manufacturer information;

[0090] For vehicle dynamic status reporting tests, extract the dynamic status fields periodically reported by the vehicle, including vehicle speed, acceleration, GPS latitude and longitude coordinates, direction angle, and timestamp;

[0091] For remote control command issuance and response testing, extract the control command fields (such as acceleration, deceleration, and speed limit commands) issued from the cloud and the response confirmation fields (such as "received" and "executed") returned by the vehicle.

[0092] For testing the distribution of navigation route adjustment suggestions, extract the route planning fields from the route suggestion message, including the starting point coordinates, the destination coordinates, the list of waypoints, the estimated time of arrival (ETA), and the recommended lane;

[0093] For fault alarm upload testing, extract the fault alarm fields actively reported by the vehicle, including fault code, alarm level, occurrence time, and associated systems (such as battery and brake).

[0094] For data transmission latency testing and packet loss rate testing, extract the timestamp field T1 (time of transmission by vehicle or roadside equipment) and the timestamp T2 (time of reception by cloud control platform) from the data packet, as well as the sequence number field;

[0095] For protocol conformance testing, extract the protocol encoding fields of all communication messages, including message headers, payload structure, and encoding format (such as JSON Schema or ASN.1).

[0096] For multi-scenario switching adaptability testing, extract the scenario switching record fields from the system log, including the scenario before switching, the scenario after switching, the switching time, and the system status code.

[0097] Please continue reading. Figure 3 As shown, the data analysis module further includes:

[0098] A data analysis unit, connected to the data processing unit, is used to perform functional compliance verification and non-functional performance calculation on the classified test data.

[0099] Specifically, the data analysis unit performs rule matching and integrity verification for functional tests; performs quantitative calculations for non-functional tests; and calls protocol testing tools to automatically verify the extracted encoded fields for protocol consistency testing, determining whether all messages conform to the T / CSAE 295.2-2023 protocol specification.

[0100] For example, for functional tests such as vehicle-to-cloud network connectivity, remote control response, and fault alarm uploading, rule matching and integrity verification are performed:

[0101] Check if the vehicle-to-cloud heartbeat message is sent every ±10% of 1 second, and if there are 5 consecutive messages without loss; verify if the vehicle returns a "received" confirmation within 500ms after receiving a remote control command; determine if fault alarm information is uploaded to the cloud within 200ms after the event occurs; verify if the navigation route suggestion includes complete fields such as origin, destination, and ETA.

[0102] For non-functional tests, such as latency, packet loss, and stability, quantitative calculations are performed:

[0103] Calculate data transmission delay: ΔT = T2 - T1, and calculate the average delay and maximum delay;

[0104] Calculate the packet loss rate: Determine the number of lost data packets based on the sequence number continuity. Packet loss rate = (Number of lost packets / Total number of packets sent) × 100%;

[0105] Analyze system stability: Monitor CPU and memory usage during stress testing to determine if crashes or timeouts occur;

[0106] Assess adaptability to multi-scenario switching: During the continuous switching of 5 preset scenarios, such as urban commuting, highway cruising, and low visibility in rainy weather, check whether the system function recovery time is ≤2s.

[0107] Specifically, the data analysis module systematically identifies system functional compliance and performance through structured processing and multi-dimensional analysis of the collected data. It can perform rule verification and quantitative calculations based on industry standards, effectively revealing potential problems in system timing, latency, and consistency.

[0108] Please continue reading. Figure 1 As shown, the vehicle-cloud and road-cloud integrated testing and evaluation system also includes:

[0109] The evaluation and decision-making module is connected to the data analysis module. The evaluation and decision-making module is used to comprehensively evaluate the system performance based on the data analysis results and according to a multi-dimensional evaluation index system.

[0110] Please see Figure 4 As shown, the evaluation and decision-making module includes:

[0111] The indicator import unit is used to configure corresponding evaluation indicator parameters for each test subtask before or during the execution of the test task, and to establish an evaluation benchmark.

[0112] Specifically, the indicator import unit configures clear evaluation indicators for each test sub-task based on the test task type. In this embodiment, the vehicle-cloud data interaction test task includes 11 sub-tasks, and each sub-task and its corresponding evaluation indicator are as follows:

[0113] Car-cloud network connection and rule test: Successfully establish a TCP long connection, with a heartbeat message period of 1s±10%, and pass 5 consecutive heartbeats without loss;

[0114] Transmission timing and data packet structure test: Data packet fields conform to the format requirements of Section 7.5 of T / CSAE 295.2-2023, and the CRC check pass rate is 100%;

[0115] Vehicle static parameter information return test: The completeness rate of reporting static information such as vehicle VIN code, license plate number, and vehicle model is ≥98%, and the format is compliant;

[0116] Vehicle dynamic status reporting test: The reporting frequency of dynamic data such as vehicle speed, acceleration, position, and steering angle is ≥10Hz, and the timestamp is continuous without jumps;

[0117] Remote control command issuance and response test: After the cloud issues commands such as acceleration, deceleration, and speed limit, the vehicle returns a "received" confirmation within 500ms;

[0118] Navigation route adjustment suggestions have been sent out for testing: The suggested routes include the origin, destination, waypoints, and estimated time of arrival (ETA), and the format conforms to the standard.

[0119] Fault alarm upload test: After a vehicle malfunctions (such as battery overheating or abnormal braking), the alarm information is uploaded to the cloud within 200ms;

[0120] Data transmission latency test: The time difference ΔT from the vehicle reporting data to the cloud control platform receiving it is ≤20ms;

[0121] Data packet loss rate test: During a 10-minute test, the data packet loss rate of the vehicle-to-cloud interaction was ≤ 1%;

[0122] Protocol conformance testing: All message encoding / decoding conforms to the T / CSAE 295.2-2023 protocol specification, and the protocol testing tool verification pass rate is 100%.

[0123] Multi-scenario switching adaptability test: Under the continuous switching of 5 scenarios including urban commuting, highway cruising, and low visibility in rainy weather, the system did not crash or time out, and the function recovery time was ≤2s;

[0124] This embodiment does not impose specific limitations on the setting of the vehicle-cloud data interaction test task and its evaluation criteria; those skilled in the art can set them freely according to their needs.

[0125] Please continue reading. Figure 4 As shown, the evaluation and decision-making module further includes:

[0126] The evaluation and decision-making unit, which is connected to the indicator import unit, is used to calculate the pass rate of the test tasks and determine the overall performance of the system based on the execution results of the test sub-tasks and the preset evaluation indicators.

[0127] Specifically, the evaluation decision unit obtains the total number N of test sub-tasks, counts the number M of successful sub-tasks, and calculates the test task pass rate:

[0128] Pass rate = M / N × 100%;

[0129] If the pass rate is ≥90%, the overall performance of the vehicle-cloud and road-cloud integrated system is deemed qualified; otherwise, it is deemed unqualified.

[0130] An adjustment coefficient is constructed based on the data acquisition success rate c0 and the test scenario coverage f0, and the pass rate is updated based on the adjustment coefficient:

[0131] The data collection success rate c0 is compared with the preset success rate c1 to construct a data collection factor. If c0≤c1, the data collection factor is set to ln[2×(c1-c0) / c1+1] / ln3. If c0>c1, the data collection factor is set to 0.

[0132] The test scenario coverage f0 is compared with the preset coverage f1 to construct the scenario factor. If f0≤f1, the scenario factor is set to (f1-f0) / f1. If f0>f1, the scenario factor is set to 0.

[0133] An adjustment coefficient H is constructed based on the data acquisition factor and the scenario factor, where H = x1 × data acquisition factor + x2 × scenario factor, x1 is the data acquisition weight, x2 is the scenario coverage weight, and x1 + x2 = 1.

[0134] The pass rate is updated based on the adjustment coefficient, and the updated pass rate is set as TG, where TG = pass rate × (1 - 0.1 × H).

[0135] Specifically, the data acquisition success rate refers to the ratio of the number of valid data points actually successfully acquired to the total number of data points expected to be acquired during the testing process. It is used to quantitatively evaluate the completeness and reliability of the data acquisition process. The scenario coverage rate refers to the ratio of the number of test scenario types actually covered to the total number of related scenario types in the preset scenario library during the execution of the test task. It is used to evaluate the comprehensiveness and scenario diversity of the test. The data acquisition success rate can be obtained by statistically analyzing the ratio of the number of valid data points actually acquired to the total number of data points expected to be acquired. The scenario coverage rate can be obtained by calculating the ratio of the number of test scenario types actually executed to the total number of related scenario types in the preset scenario library.

[0136] Specifically, in this embodiment, the preset success rate is 95%, the preset coverage rate is 90%, the data collection weight is 0.6, and the scene coverage weight is 0.4.

[0137] Specifically, the evaluation and decision-making module comprehensively evaluates test results based on a multi-dimensional indicator system, achieving objective quantification and overall judgment of system performance. Through configurable evaluation benchmarks and pass rate statistics, this module provides clear conclusions on whether the system meets design requirements, helping the team quickly identify weaknesses and develop targeted optimization strategies.

[0138] Please see Figure 5 As shown, the integrated testing and evaluation method for vehicle-cloud and road-cloud includes:

[0139] Step S101: Create, schedule, and monitor the vehicle cloud and road cloud integration test task;

[0140] Step S102: Construct diverse test scenarios;

[0141] Step S103: Collect vehicle cloud data and road cloud data;

[0142] Step S104: Analyze the collected vehicle cloud data and road cloud data;

[0143] Step S105: Based on the data analysis results, conduct a comprehensive evaluation of the system performance according to the multi-dimensional evaluation index system.

[0144] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is impossible to exhaustively list all the implementation methods here. All obvious variations or modifications derived from the technical solutions of the present invention are still within the protection scope of the present invention.

Claims

1. A vehicle-cloud and road-cloud integrated testing and evaluation system, characterized in that, include: The test management module is used to create, schedule, and monitor test tasks for the integration of vehicle cloud and road cloud. The test scenario building module is used to construct diverse test scenarios; The data acquisition module is used to collect vehicle cloud data and road cloud data; The data analysis module is used to analyze the collected vehicle cloud data and road cloud data; The evaluation and decision-making module is used to comprehensively evaluate the system performance based on data analysis results and a multi-dimensional evaluation index system.

2. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 1, characterized in that, The test management module generates executable test tasks based on the test requirements submitted by the user, breaks them down into multiple sub-tasks, selects matching scenarios from the preset scenario library, configures functional and non-functional test parameters, monitors resource consumption and data quality indicators in real time during test execution, and finally summarizes test results and generates reports.

3. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 2, characterized in that, The test scenario construction module includes: The scene mapping unit is used to determine key scene elements according to the actual application scenario and to assign weights to the elements based on the analytic hierarchy process. The scene element modeling unit is used to perform structured modeling of vehicle, roadside, platform and environmental elements using a unified modeling language, and to analyze the relationships between elements based on graph theory. The scene generation unit is used to convert the modeled scene into an executable simulation script and adaptively adjust the scene parameters based on reinforcement learning algorithms.

4. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 3, characterized in that, The scene mapping unit determines the core elements of the scene based on the actual application scenario. The core elements of the scene include multiple factors such as intersection size, traffic flow, intersection queue length, weather interference, and road facility density.

5. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 3, characterized in that, In the element association graph constructed by the scene element modeling unit, nodes represent scene elements, edges represent the interaction relationships between elements, and key interaction relationships are filtered by association coefficients. The scene generation unit dynamically adjusts scene parameters using the Q-Learning algorithm, with the matching degree between the scene and the real environment as the reward function.

6. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 1, characterized in that, The data analysis module includes: The data processing unit is used to extract relevant fields from the raw data stream and classify them according to the test item type; The data analysis unit is used to perform functional compliance verification and non-functional performance calculations on the categorized test data.

7. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 6, characterized in that, The data processing unit extracts the corresponding field information from the original communication data stream for each test subtask based on the data structure defined in T / CSAE 295.2-2023 and T / CSAE 295.3-2023.

8. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 7, characterized in that, The data analysis unit performs rule matching and integrity verification for functional tests; quantitative calculations for non-functional tests; and automated verification of extracted encoded fields by calling protocol testing tools for protocol consistency tests to determine whether all messages conform to the T / CSAE 295.2-2023 protocol specification.

9. The vehicle-cloud and road-cloud integrated testing and evaluation system according to claim 8, characterized in that, The evaluation and decision-making module includes: The indicator import unit is used to configure corresponding evaluation indicator parameters for each test subtask before or during the execution of the test task, and to establish an evaluation benchmark. The evaluation and decision-making unit is used to calculate the pass rate of test tasks and determine the overall performance of the system based on the execution results of test sub-tasks and preset evaluation indicators.

10. A method for testing and evaluating the integration of vehicle-cloud and road-cloud systems, applied to the vehicle-cloud and road-cloud integration testing and evaluation system as described in any one of claims 1-9, characterized in that, include: Step S101: Create, schedule, and monitor the vehicle cloud and road cloud integration test task; Step S102: Construct diverse test scenarios; Step S103: Collect vehicle cloud data and road cloud data; Step S104: Analyze the collected vehicle cloud data and road cloud data; Step S105: Based on the data analysis results, conduct a comprehensive evaluation of the system performance according to the multi-dimensional evaluation index system.

Images