Vehicle network interaction vertical application large model demonstration performance test evaluation method

Abstract

According to the vehicle network interaction vertical application large model demonstration performance test and evaluation method, a large model special test index system suitable for vehicle network interaction is systematically constructed, and the defect that in the prior art, an effective test and evaluation mechanism is lacked in the field is overcome. The evaluation model breaks through the limitation of single evaluation dimension and poor adaptability in related technologies, fully introduces multi-source heterogeneous data including vehicle operation data, charging infrastructure data, road traffic information, power grid operation parameters, meteorological data and the like, constructs a multi-element dynamic test scene in combination with conventional and extreme working conditions, and improves the evaluation accuracy. And the comprehensiveness and the actual adaptability of evaluation are improved. Besides, a robustness test mechanism is also introduced, so that the strategy rationality and application universality of the large model under a complex operation condition can be effectively tested, and the application reliability of the large model in an actual scene is improved.

Description

Technical Field

[0001] This invention relates to the technical field of the cross-integration of smart grids and artificial intelligence, and in particular to a method for performance testing and evaluation of a large-scale demonstration model of vehicle-grid interaction vertical applications. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, large-scale models have demonstrated outstanding capabilities in fields such as natural language processing, image recognition, and intelligent decision-making, gradually promoting the mature evolution of cross-industry integration and vertical applications. Meanwhile, in the energy sector, the world is undergoing profound changes in its energy structure, further accelerating the deep integration of electric vehicles and smart grids, and helping the energy internet move towards a more intelligent and collaborative direction.

[0003] As a crucial pathway for building new power systems and achieving energy transition, vehicle-to-grid (V2G) interaction has become a focal point of research and practice. Its core lies in establishing dynamic interconnections between electric vehicles, charging infrastructure, and the power grid system, enabling multi-faceted collaboration and intelligent control across the "vehicle-charging-grid-load" system. To this end, V2G technology not only needs to support real-time data acquisition, transmission, and processing from a large number of heterogeneous terminals, but also places higher demands on distribution network scheduling optimization, precise load sensing in distribution areas, and participation in the electricity market. Constructing a large-scale vertical application system for V2G and conducting systematic testing and evaluation of its demonstration performance have become key steps in promoting the intelligent development and large-scale application of this field.

[0004] However, the integration of large-scale models with vehicle-to-everything (V2X) interaction is still in its early stages, and evaluating the effectiveness of its outputs faces numerous challenges. First, there is a lack of unified standards for evaluation metrics; second, data collection and scenario coverage are insufficient; furthermore, verification of real-time performance and robustness is inadequate; and finally, collaborative testing between different system components and security and privacy protection mechanisms need further improvement. To address these issues, it is necessary to further refine the indicator system, enrich test scenarios, enhance the system's real-time response and robustness, and strengthen cross-system collaboration and security protection, thereby laying a solid foundation for the large-scale application of this technology in the future. Summary of the Invention The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a method for performance testing and evaluation of a large-scale demonstration model of vehicle-to-everything (V2X) interactive vertical applications.

[0006] The second objective of this invention is to propose a large-scale demonstration performance testing and evaluation device for vehicle-to-everything (V2X) interactive vertical applications.

[0007] The third objective of this invention is to provide an electronic device.

[0008] The fourth objective of this invention is to provide a computer-readable storage medium.

[0009] The fifth objective of this invention is to provide a computer program product.

[0010] To achieve the above objectives, a first aspect of the present invention proposes a method for performance testing and evaluation of a large-scale demonstration model of vehicle-to-everything (V2X) interactive vertical applications, comprising: A basic performance evaluation index system for large models is constructed to evaluate the input processing efficiency of large model input data. At the same time, adversarial attack testing is used to evaluate the data quality sensitivity of large models, and data bias index is used to measure the error rate of its prediction data in vehicle-pile-road-network-weather forecasting. From the perspectives of logic and compliance, the performance of the large model in generating strategies under given input data and constraints is comprehensively evaluated. Based on the evaluation test results, a systematic test evaluation report is generated, and based on the report content, strategy defects are identified and optimization suggestions are output for the large model to support the iterative upgrade of the model.

[0011] Optionally, the input processing efficiency includes data throughput, transmission latency, and data processing resource consumption, wherein: Data throughput refers to the total amount of data that a system can process per unit of time, and is used to reflect the data processing capability and efficiency of a model; Transmission latency refers to the end-to-end response time from data input to policy generation completion, and is used to measure the system's responsiveness to real-time requirements; System resource consumption includes CPU / GPU utilization and network bandwidth usage, which are used to assess the hardware dependence and deployment cost of the system during operation.

[0012] Optionally, adversarial attack testing can be used to evaluate the data quality sensitivity of large models, including: The data quality sensitivity is evaluated by superimposing Gaussian noise into the input data or introducing adversarial examples based on the fast gradient sign method, and the output deviation formula is used to quantify the evaluation.

[0013] Optionally, the error rate of the predicted data is quantitatively evaluated using the error rate.

[0014] Optionally, from a logical dimension, the steps for evaluating the performance of a large model in generating strategies under given input data and constraints include: Based on the generative adversarial network, three low-probability events were generated: saturated operation of highway service areas during holidays, maximum power grid load exceeding historical values, and user charging behavior during typhoons. The large model was trained to analyze whether the strategy generation logic of the large model is feasible. Among them: the saturated operation scenario of highway service areas during holidays is used to simulate the extreme saturation state of service areas during holidays, and to verify whether the power configuration of charging facilities is effectively improved based on the strategy output by the large model under high load conditions; the scenario of the maximum load of the power grid exceeding the historical value is used to simulate the operation state of the model city under the condition of a surge in electricity load, and to verify the generation accuracy of the guidance strategy and its effectiveness in alleviating the load pressure of the power grid and improving the peak shaving rate; the user charging behavior scenario under typhoon conditions is used to verify whether the behavior distribution generated by the large model tends to be non-charging or home charging, and to evaluate the degree of matching between its guidance strategy and the actual user choices.

[0015] Optionally, from a compliance perspective, the steps for evaluating the performance of large models in generating strategies under given input data and constraints include: The compliance of the large model generation strategy is systematically examined from three dimensions: time-of-use electricity pricing strategy, charging and discharging guidance strategy, and user data privacy. Specifically: the time-of-use pricing strategy is used to detect whether the pricing strategy generated by the large model violates existing time-of-use pricing rules; the charging and discharging guidance strategy is used to conduct compliance checks on the charging and discharging schemes generated by the large model, and to determine whether they may lead to overload of the distribution network and affect the stability of the power grid system; and the user data privacy strategy is used to assess whether the large model has taken privacy protection measures such as desensitization and encryption when processing sensitive information such as user location and charging records, to ensure that data use is compliant.

[0016] To achieve the above objectives, a second aspect of the present invention provides a performance testing and evaluation device for a large-scale demonstration model of vehicle-to-everything (V2X) interactive vertical applications, comprising: The first evaluation module is used to construct a basic performance evaluation index system for large models, evaluate the input processing efficiency of large models, evaluate the data quality sensitivity of large models using adversarial attack testing, and use data bias index to measure the error rate of its prediction data in vehicle-pile-road-network-weather forecasting. The second evaluation module is used to comprehensively evaluate the performance of the large model in generating strategies under given input data and constraints from two dimensions: logic and compliance. The output module is used to generate a systematic test evaluation report based on the evaluation test results, and to identify strategy defects and provide optimization suggestions for the large model based on the report content, so as to support the iterative upgrade of the model.

[0017] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of the first aspects.

[0018] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of the first aspects.

[0019] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product that, when executed by a processor, implements the method described in any one of the first aspects.

[0020] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects: Addressing the multi-source heterogeneous data (vehicles, charging piles, roads, networks, and weather) and their highly coupled and dynamic characteristics in vehicle-to-everything (V2X) interaction scenarios, this paper pioneers a comprehensive indicator system encompassing both basic performance and strategy generation quality, filling the gap in this field's lack of dedicated evaluation metrics. Basic performance covers input processing efficiency (data throughput, transmission latency, resource consumption), data quality sensitivity (output deviation under adversarial attacks), and prediction data error rate; strategy generation quality covers logicality and compliance, overcoming the limitations of existing technologies with their single evaluation dimension and poor adaptability.

[0021] Generative Adversarial Networks (GANs) are used to generate low-probability extreme scenarios such as highway service area saturation during holidays, power grid load exceeding historical values, and typhoon weather. The rationality of the large model's strategy under extreme conditions (such as power boost rate > 0, load relief rate < 0, and KL divergence of user behavior distribution ≤ 0.1) is tested to enhance robustness verification.

[0022] From three dimensions—the violation rate of time-of-use pricing strategy, the overload rate of charging and discharging guidance strategy, and user data privacy risks (the difference in entropy value before and after desensitization)—we ensure that the strategy complies with the rules and security requirements of the actual scenario. Existing technologies have not designed robustness tests and multi-dimensional compliance verifications for such extreme scenarios in vehicle-to-grid interaction scenarios.

[0023] A closed-loop testing process is proposed, which includes "dataset reading (spatial-temporal alignment of multi-source data) - scenario modeling (covering regular and extreme events) - model deployment and evaluation (multi-dimensional performance verification) - result analysis and optimization". This process enables the full-chain evaluation of the vehicle-network interaction model from data integration to strategy optimization. Existing technologies lack such a systematic process that combines multi-source heterogeneous data and dynamic scenarios.

[0024] By integrating real-time grid load, vehicle trajectory, charging facility distribution, charging and swapping behavior, and meteorological grid data, a comprehensive test dataset with spatiotemporal alignment characteristics is constructed. This provides a data foundation for model testing that fits the actual scenario of vehicle-grid interaction. Existing technologies have not achieved spatiotemporal collaborative integration of such multi-source heterogeneous data.

[0025] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating a performance testing and evaluation method for a large-scale demonstration model of vehicle-to-everything (V2X) interactive vertical applications provided in an embodiment of the present invention. Figure 2 This is a flowchart illustrating another method for performance testing and evaluation of a large-scale model for vehicle-to-everything (V2X) interactive vertical applications, provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a large-scale demonstration performance testing and evaluation device for vehicle-to-everything (V2X) interactive vertical application provided in an embodiment of the present invention. Detailed Implementation

[0027] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0028] Vehicle-to-grid (V2G) interaction systems typically involve multi-source heterogeneous data from vehicles, charging infrastructure, transportation networks, power systems, and meteorological information. Their high coupling and dynamic nature make existing evaluation methods insufficient to meet their requirements for accuracy, timeliness, and adaptability, thus limiting the effective implementation of performance testing and comprehensive evaluation of V2G systems. To address the shortcomings of existing performance testing and evaluation methods for large-scale V2G models, this invention proposes a demonstration performance testing and evaluation method for large-scale V2G vertical applications. This invention establishes a complete evaluation index system based on the basic performance assessment of large-scale models and the quality assessment of policy generation, and clarifies the large-scale model testing and evaluation process. The method not only proposes dedicated testing indicators for V2G, filling a gap in this field, but also fully considers the impact of multi-source heterogeneous data (vehicles, charging infrastructure, roads, networks, and meteorological information), going beyond traditional single evaluation dimensions. Furthermore, it constructs an evaluation system for extreme scenarios of V2G for robustness testing, thereby achieving an objective evaluation of the comprehensive performance of the large-scale model.

[0029] Figure 1 and Figure 2 This is a flowchart illustrating different formats of a performance testing and evaluation method for a large-scale demonstration model of vehicle-to-everything (V2X) interactive vertical applications provided in an embodiment of the present invention. (Refer to...) Figure 1 and Figure 2 As shown, the method includes the following steps: Step S1: Construct a basic performance evaluation index system for the large model, evaluate the input processing efficiency of the large model's input data, use adversarial attack testing to evaluate the data quality sensitivity of the large model, and use the data bias index to measure its error rate in vehicle-pile-road-network-weather forecasting.

[0030] In one embodiment of the present invention, a basic performance evaluation index system for a large model is first constructed.

[0031] This fundamental performance evaluation refers to the comprehensive measurement of the core capabilities and limitations of large-scale systems by establishing a multi-dimensional basic application indicator system. This evaluation system mainly covers key indicators such as input processing efficiency, data quality sensitivity, and prediction data error rate. By quantifying these indicators, the model's performance in handling high-speed inputs, coping with data fluctuations, and generating accurate outputs in complex scenarios can be systematically analyzed, thereby identifying potential bottlenecks and providing theoretical guidance for model optimization and improvement.

[0032] Specifically, input processing efficiency is mainly used to evaluate the capabilities of large models in data reading and processing. Its indicator system includes data throughput, transmission latency, and data processing resource consumption. By quantifying these key indicators, the response speed and resource utilization efficiency of large models in high-speed data flow environments can be comprehensively reflected, providing strong support for systematic large model testing.

[0033] Data throughput: refers to the total amount of data that a system can process per unit of time. It is usually measured by the number of samples or bytes processed per second and is used to reflect the data processing capability and efficiency of the model. The formula is shown below.

[0034]

[0035] In the formula: : This represents the data read rate per second, in MB / s; Total number of samples read. (MB) : Processing time, in seconds.

[0036] Transmission latency rate: refers to the end-to-end response time from data input to policy generation completion. It is used to measure the system's responsiveness to real-time requirements. The formula is shown below.

[0037]

[0038] In the formula: : Data transmission delay, in seconds; : The output time, in seconds, after preprocessing the data collected from the large model; : The time taken, in seconds, for inputting data into the large model strategy analysis.

[0039] System resource consumption: This includes metrics such as CPU / GPU utilization and network bandwidth usage, used to assess the system's hardware dependence and deployment costs during operation. It can be reflected through data resource utilization and data transfer rate. The formula is shown below.

[0040]

[0041] In the formula: Data resource utilization rate; Actual computer resource usage; Total computer resources.

[0042]

[0043] In the formula: Data transfer rate, measured in MB / s; Total amount of data transmitted, in MB; Processing time, in seconds.

[0044] In one embodiment of the present invention, data quality sensitivity is primarily evaluated through adversarial attack testing. Specific methods include adding Gaussian noise (signal-to-noise ratio (SNR) ≤ 20 dB) to the input data or introducing adversarial examples (such as attacks based on the Fast Gradient Sign Method (FGSM)) to simulate data perturbations that may occur in real-world environments. Based on this, the stability and robustness to changes in input data quality are measured by calculating the deviation of the model output. The relevant evaluation formula is shown below:

[0045] In the formula: Data output deviation; The number of data points in the data sample; Changes in data after an attack; : Original data value.

[0046] In one embodiment of the present invention, the prediction data error rate is mainly quantitatively evaluated through the error rate. During the process of generating prediction data using a large model, problems such as logical contradictions, factual errors, or non-standard formats may occur. This indicator comprehensively measures the probability of errors generated by the model during data prediction by analyzing the degree of deviation between the prediction results and the actual data, thereby reflecting the accuracy and reliability of its output results. The formula for calculating the sample data error rate is as follows:

[0047] In the formula: Sample data error rate; The number of data points in the data sample; : The predicted value of the data; : The actual value of the data.

[0048] In one embodiment of the present invention, the test content and detection rules for data error rate are shown in Table 1.

[0049] Table 1

[0050] In another embodiment of the present invention, the test content and detection rules for data error rate are shown in Table 2.

[0051] Table 2

[0052] Step S2 involves comprehensively evaluating the performance of the large model's generation strategy under given input data and constraints from both logical and compliance perspectives.

[0053] It is understandable that after receiving data input, the large model will analyze it according to user needs and output corresponding strategies.

[0054] In one embodiment of the present invention, the generation quality is comprehensively evaluated from the two dimensions of logic and compliance of the strategy generation results, thereby fully reflecting the decision-making ability and reliability of the large model in actual application scenarios.

[0055] Specifically, the logical quality of large-scale model strategy generation refers to whether the large-scale model can be analyzed to obtain reasonable and feasible strategies under extreme conditions. This invention trains a large-scale model based on three low-probability events generated by a Generative Adversarial Network (GAN): saturated operation of highway service areas during holidays, maximum grid load exceeding historical values, and user charging behavior during typhoons. The analysis then examines the feasibility of the large-scale model's strategy generation logic.

[0056] The holiday highway service area saturation operation scenario is used to simulate the extreme saturation state of service areas during holidays, verifying whether the power configuration of charging facilities is effectively improved under high load conditions based on the large model output strategy. The formula is:

[0057] In the formula: : Charging power improvement rate of high-speed service area stations >0 considers it reasonable; : After adjusting the output strategy based on the large model, the operating power of the charging station, in kW; : Charging station power during actual saturated operation, in kW; The scenario of the power grid's maximum load exceeding historical values ​​is used to simulate the operational status of a model city under a surge in electricity load, verifying the accuracy of the guidance strategy generation and its effectiveness in alleviating power grid load pressure and improving peak shaving rate. The formula is:

[0058] In the formula: Load relief rate under extreme power grid conditions <0 is considered reasonable; Based on the large model output strategy, the urban power grid load after power load adjustment, in kW; : Load of urban power grid under extreme conditions, kW; A typhoon weather scenario simulation of user charging behavior was constructed to verify whether the behavior distribution generated by the large model tends towards not charging or charging at home, and to evaluate the degree of match between the guidance strategy and actual user choices. The formula is:

[0059] In the formula: If the probability of users charging at home is DKL≤0.1, the model distribution is considered reasonably biased (e.g., the probability of charging at home increases significantly); if DKL>0.1, there is an unreasonable bias.

[0060] A set of user behavior categories, including "not charging", "charging at home", "charging at work", "charging in shopping mall", etc. : The behavioral category predicted by the model The probability of; Behavioral categories observed during typhoon weather The probability of.

[0061] Specifically, the compliance of large-scale model strategy generation means that it must not deviate from reality during data processing and strategy generation, should conform to real-world scenarios, and meet the requirements of strategy executability. This invention systematically examines the compliance of large-scale model generation strategies from three dimensions: time-of-use pricing strategies, charging and discharging guidance strategies, and user data privacy.

[0062] Time-of-use pricing strategy: Used to detect whether the pricing strategy generated by a large model violates existing time-of-use pricing rules. The formula is:

[0063] In the formula: : violation rate; The number of strategies that violate time-of-use pricing rules; Total number of strategies.

[0064] Charge / discharge guidance strategy: Used to perform compliance checks on charge / discharge schemes generated by large models, determining whether they may lead to overload of the distribution network and affect the stability of the power grid system. The formula is:

[0065] In the formula: Overload rate; : Node number of the distribution area; : Rated power of the node; : The set of nodes in the transformer area.

[0066] User data privacy: This is used to assess whether large models have taken privacy protection measures such as anonymization and encryption when processing sensitive information such as user location and charging records, ensuring compliant data use. The formula is:

[0067] In the formula: Privacy risks; : Original data entropy; Entropy of data after anonymization.

[0068] Step S3: Generate a systematic test evaluation report based on the evaluation test results, and output strategy defect identification and optimization suggestions for the large model according to the report content to support the iterative upgrade of the model.

[0069] Specifically, this step is a crucial closed-loop feedback link in the testing and evaluation method of this invention. Its technical implementation principle is based on the quantitative analysis of multi-dimensional performance indicators and the structured evaluation of strategy generation quality, aiming to provide traceable, verifiable, and optimizable decision-making basis for large-scale models in vehicle-to-everything (V2X) interaction scenarios. This step realizes a closed-loop feedback mechanism from test data to model optimization, effectively improving the strategy generation quality and system stability of large-scale models in V2X interaction scenarios. It provides scientific support for continuous model iteration and performance improvement, demonstrating significant engineering practical value and innovation.

[0070] The vehicle-to-grid (V2G) interaction scenario comprehensive performance testing and evaluation method of this invention generates a systematic test evaluation report based on the test results, and outputs strategy defect identification and optimization suggestions accordingly. This further realizes the continuous iteration and performance optimization of the V2G interaction model, and significantly improves the model's adaptability and decision quality in complex traffic scenarios.

[0071] To achieve the above embodiments, the present invention also proposes a large-scale demonstration performance testing and evaluation device for vehicle-to-everything (V2X) interactive vertical applications. Figure 3 This is a schematic diagram of the structure of a large-scale demonstration performance testing and evaluation device for vehicle-to-everything (V2X) interactive vertical applications, provided as an embodiment of the present invention. Figure 3 As shown, the device includes: The first evaluation module 100 is used to construct a basic performance evaluation index system for large models, evaluate the input processing efficiency of large model input data, evaluate the data quality sensitivity of large models using adversarial attack testing, and use data deviation index to measure the error rate of its prediction data in vehicle-pile-road-network-weather prediction. The second evaluation module 200 is used to comprehensively evaluate the performance of the large model in generating strategies under given input data and constraints from two dimensions: logic and compliance. The output module 300 is used to generate a systematic test evaluation report based on the evaluation test results, and to identify strategy defects and output optimization suggestions for the large model based on the report content, so as to support the iterative upgrade of the model.

[0072] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0073] To implement the above embodiments, the present invention also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0074] To implement the above embodiments, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0075] To implement the above embodiments, the present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0076] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0077] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0078] This invention is intended to provide implementation schemes for users to selectively prevent the use or access to personal information data. That is, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0079] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0080] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0081] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0082] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0083] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0084] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0085] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0086] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0087] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0088] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for performance testing and evaluation of a large-scale demonstration model of vehicle-to-everything (V2X) interactive vertical applications, characterized in that: Includes the following steps: A basic performance evaluation index system for large models is constructed to evaluate the input processing efficiency of large model input data. At the same time, adversarial attack testing is used to evaluate the data quality sensitivity of large models, and data bias index is used to measure the error rate of its prediction data in vehicle-pile-road-network-weather forecasting. From the perspectives of logic and compliance, the performance of the large model in generating strategies under given input data and constraints is comprehensively evaluated. Based on the evaluation test results, a systematic test evaluation report is generated, and based on the report content, strategy defects are identified and optimization suggestions are output for the large model to support the iterative upgrade of the model.

2. The method according to claim 1, characterized in that, The input processing efficiency includes data throughput, transmission latency, and data processing resource consumption, among which: Data throughput refers to the total amount of data that a system can process per unit of time, and is used to reflect the data processing capability and efficiency of a model; Transmission latency refers to the end-to-end response time from data input to policy generation completion, and is used to measure the system's responsiveness to real-time requirements; System resource consumption includes CPU / GPU utilization and network bandwidth usage, which are used to assess the hardware dependence and deployment cost of the system during operation.

3. The method according to claim 2, characterized in that, Using adversarial attack testing to evaluate the data quality sensitivity of large models, including: The data quality sensitivity is evaluated by superimposing Gaussian noise into the input data or introducing adversarial examples based on the fast gradient sign method, and the output deviation formula is used to quantify the evaluation.

4. The method according to claim 3, characterized in that, The error rate of the predicted data is quantitatively evaluated using the error rate.

5. The method according to claim 4, characterized in that, From a logical perspective, the steps for evaluating the performance of a large model in generating strategies under given input data and constraints include: Based on the generative adversarial network, three low-probability events were generated: saturated operation of highway service areas during holidays, maximum power grid load exceeding historical values, and user charging behavior during typhoons. The large model was trained to analyze whether the strategy generation logic of the large model is feasible. Among them: the saturated operation scenario of highway service areas during holidays is used to simulate the extreme saturation state of service areas during holidays, and to verify whether the power configuration of charging facilities is effectively improved based on the strategy output by the large model under high load conditions; the scenario of the maximum load of the power grid exceeding the historical value is used to simulate the operation state of the model city under the condition of a surge in electricity load, and to verify the generation accuracy of the guidance strategy and its effectiveness in alleviating the load pressure of the power grid and improving the peak shaving rate; the user charging behavior scenario under typhoon conditions is used to verify whether the behavior distribution generated by the large model tends to be non-charging or home charging, and to evaluate the degree of matching between its guidance strategy and the actual user choices.

6. The method according to claim 5, characterized in that, From a compliance perspective, the steps for evaluating the performance of a large model in generating strategies under given input data and constraints include: The compliance of the large model generation strategy is systematically examined from three dimensions: time-of-use electricity pricing strategy, charging and discharging guidance strategy, and user data privacy. Specifically: the time-of-use pricing strategy is used to detect whether the pricing strategy generated by the large model violates existing time-of-use pricing rules; the charging and discharging guidance strategy is used to conduct compliance checks on the charging and discharging schemes generated by the large model, and to determine whether they may lead to overload of the distribution network and affect the stability of the power grid system; and the user data privacy strategy is used to assess whether the large model has taken privacy protection measures such as desensitization and encryption when processing sensitive information such as user location and charging records, to ensure that data use is compliant.

7. A performance testing and evaluation device for a large-scale demonstration model of vehicle-to-everything (V2X) interactive vertical applications, characterized in that, include: The first evaluation module is used to construct a basic performance evaluation index system for large models, evaluate the input processing efficiency of large models, evaluate the data quality sensitivity of large models using adversarial attack testing, and use data bias index to measure the error rate of its prediction data in vehicle-pile-road-network-weather forecasting. The second evaluation module is used to comprehensively evaluate the performance of the large model in generating strategies under given input data and constraints from two dimensions: logic and compliance. The output module is used to generate a systematic test evaluation report based on the evaluation test results, and to identify strategy defects and provide optimization suggestions for the large model based on the report content, so as to support the iterative upgrade of the model.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images