A performance verification method, device and equipment of a large language model and a medium

Abstract

The application discloses a performance verification method, device and equipment of a large language model and a medium. The method comprises the following steps: in response to the triggering of a version change event, a test task set of a target verification model is obtained; each test task in the test task set is run using the target verification model through a pre-set continuous pipeline, and a performance test baseline matched with each test task is obtained; a historical performance baseline matched with each test task in the test task set is obtained, and a performance verification result of the target verification model is obtained based on each performance test baseline and each historical performance baseline. Through the technical scheme of the application, the automatic performance verification of the large language model can be realized, and the efficiency of the model performance verification work and the reliability of the verification result are improved.

Description

Technical Field

[0001] This invention relates to the field of software testing, and more particularly to a method, apparatus, device, and medium for performance verification of large language models. Background Technology

[0002] With the rapid development of large language model technology, inference services based on large language models have been widely applied in various production scenarios such as intelligent customer service, content generation, and code assistance. To cope with rapidly changing business needs, large language model inference services typically employ continuous integration and continuous delivery models for iterative deployment, involving multiple factors such as frequent model version iterations, continuous upgrades of the inference engine, and dynamic adjustments to the underlying hardware environment. However, in existing technologies, the performance verification of large language model inference services still faces significant challenges. Regarding model version iteration, when a model is upgraded from LLaMA2 to LLaMA3 or its parameters are fine-tuned, its inference computation characteristics, memory usage patterns, and concurrent processing capabilities may undergo unpredictable changes. Regarding inference engine upgrades, although version updates of the inference framework aim to optimize scheduling strategies or kernel implementations, the compatibility of newly introduced optimization mechanisms with specific model structures or hardware configurations is uncertain, potentially leading to performance fluctuations or even regressions. Regarding hardware environment changes, replacing the GPU model from A100 to H100, adjusting instance specifications, or changing cluster topology will directly affect key performance indicators such as service throughput, latency distribution, and maximum concurrent connections.

[0003] Current mainstream CI / CD pipelines primarily focus on functional correctness verification, unit testing, code style checks, and model accuracy evaluation, but have not yet established a standardized, automated verification mechanism for the runtime performance of inference services. Performance testing largely relies on developers or operations personnel manually triggering offline scripts. The configuration of test cases, the setting of load benchmarks, and the setup of the execution environment all require manual intervention, resulting in low testing efficiency, difficulty in aligning load benchmarks, and poor reproducibility of test results. Furthermore, manual testing cannot be deeply integrated with CI / CD pipelines and lacks the ability to automatically block the release of performance-degrading changes as a quality gatekeeper. This makes it difficult to detect performance regression issues early in delivery, often delaying their resolution until they are exposed in the production environment, significantly increasing operational costs and business risks.

[0004] In summary, existing performance verification methods for large language models suffer from problems such as the inability to automate model performance verification, poor efficiency in model performance verification, and low reliability of verification results. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for performance verification of large language models, which can solve the problems of existing performance verification methods for large language models, such as the inability to achieve automated verification of model performance and the poor efficiency and reliability of verification results.

[0006] In a first aspect, embodiments of the present invention provide a performance verification method for a large language model, the method comprising: In response to a version change event, retrieve the set of test tasks for the target validation model; The target verification model is used to run each test task in the test task set through a pre-defined continuous pipeline to obtain a performance test baseline that matches each test task. Obtain the historical performance baselines that match each test task in the test task set, and obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

[0007] Secondly, embodiments of the present invention provide a performance verification device for a large language model, the device comprising: The event triggering module is used to respond to version change events and obtain the set of test tasks for the target verification model. The performance baseline acquisition module is used to run each test task in the test task set using the target verification model through a pre-set continuous pipeline to obtain the performance test baseline that matches each test task. The verification result generation module is used to obtain the historical performance baselines that match each test task in the test task set, and to obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

[0008] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a performance verification method for a large language model as described in any embodiment of the present invention.

[0009] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute and implement a performance verification method for a large language model as described in any embodiment of the present invention.

[0010] The technical solution of this invention, in response to a version change event, obtains a set of test tasks for a target verification model. Then, it runs each test task in the set using the target verification model through a pre-defined continuous pipeline, obtaining a performance test baseline matching each test task. Finally, it obtains historical performance baselines matching each test task in the set, and based on these historical performance baselines, it obtains the performance verification result of the target verification model. This solves the problems of existing large language model performance verification methods, such as the inability to automate model performance verification, poor efficiency in model performance verification, and low reliability of verification results. It achieves automated performance verification of large language models, improving the efficiency of model performance verification and the reliability of verification results.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a performance verification method for a large language model according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a performance verification method for a large language model according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a performance verification device for a large language model according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements a performance verification method for a large language model according to an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, any variations of the terms "comprising" and "having" are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1 This is a flowchart of a performance verification method for a large language model provided in Embodiment 1 of the present invention. This embodiment can be applied to the automated verification of the performance of a large language model that has version change events. The method can be executed by a performance verification device for the large language model, which can be implemented in hardware and / or software. The performance verification device for the large language model can be configured in a terminal or server with performance verification function for the large language model.

[0017] like Figure 1 As shown, the method includes: S110. In response to the triggering of a version change event, obtain the set of test tasks for the target verification model.

[0018] The test task includes: version information of the target verification model, inference engine information, target hardware environment information, declaration configuration file, and scenario information. The declaration configuration file is used to define the target verification model, inference engine, test case set, and performance acceptance criteria.

[0019] The target validation model is a large language model that can run on different GPU models.

[0020] Furthermore, the version information of the target validation model is used to uniquely identify the weights and architecture configuration of the model under test, the inference engine information is used to specify the underlying acceleration framework and version on which the model inference depends, and the target hardware environment information is used to describe the specifications and topology of computing resources required for model deployment.

[0021] In this embodiment, optionally, the declaration configuration file is organized using a structured data format to define the target verification model, inference engine, test case set, and performance acceptance criteria. For example, the declaration configuration file may specify the input and output length distribution combination of test cases, the number of concurrent users, the request arrival mode, and specific indicators such as throughput threshold and latency percentile upper limit in the performance acceptance criteria.

[0022] The scenario information is used to characterize the actual load characteristics faced by the model inference service, including: context length, concurrency, request arrival patterns, and input / output length distribution combinations. Furthermore, the test task set is composed of multiple test tasks aggregated according to preset combination rules. For example, a declarative test matrix in existing technology can be used to expand different model versions, engine versions, hardware configurations, and scenario information using a Cartesian product to generate a complete test task set covering multi-dimensional change factors, ensuring the comprehensiveness and systematic nature of performance verification.

[0023] The version change events include at least one of the following: model version iteration events, inference engine version update events, and underlying hardware environment change events.

[0024] Specifically, the version change event refers to various change operations that cause changes to the operating environment or service capabilities of the large language model inference service. These can be captured through an event listening mechanism. For example, when the code repository detects an update to the model weight file, a change to the inference engine image tag, or a modification to the Infrastructure as Code (IaC) configuration, the corresponding version change event is automatically triggered and an event notification is sent to the continuous pipeline.

[0025] S120. Run each test task in the test task set using the target verification model through a pre-set continuous pipeline to obtain a performance test baseline that matches each test task.

[0026] The process involves running each test task in the test task set using a pre-defined continuous pipeline and a target verification model to obtain a performance test baseline matching each test task. This includes: in the continuous pipeline, generating test task instances corresponding to each test task in the test task set through a declaration configuration file; in response to the generation of each test task instance, running each test task instance through a pre-configured automated testing tool to obtain raw performance index data corresponding to each test task instance; and standardizing each raw performance index data to generate a performance test baseline matching each test task. The performance test baseline includes: throughput baseline, latency distribution baseline, maximum concurrent load baseline, and resource utilization baseline.

[0027] The continuous pipeline refers to an execution framework that automates the building, testing, and deployment processes in the software delivery process. Those skilled in the art should understand that the continuous pipeline can be built using tools such as Jenkins, GitLab CI, and GitHub Actions. The continuous pipeline in this embodiment is specifically customized for the performance verification scenario of large language model inference services, integrating dedicated stages such as service deployment, readiness detection, load testing execution, metric collection, and baseline generation.

[0028] Furthermore, the test task instance is an executable representation of the test task, including specific running parameters, environment configuration, and execution script; the automated testing tool is a dedicated tool for simulating load, sending requests, and collecting performance data; furthermore, the automated testing tool generates a request sequence that conforms to a specified arrival pattern based on the scenario information defined in the test task instance, continuously sends inference requests to the deployed target verification model inference server endpoint, and records raw performance metrics such as response time, first token latency, complete response latency, and request success rate for each request in real time, while collecting resource consumption data such as GPU utilization, video memory usage, CPU utilization, and network bandwidth through the node exporter or container monitoring interface.

[0029] In this embodiment, the standardization process refers to the process of cleaning, aggregating, statistically calculating, and formatting the collected raw performance index data in order to eliminate abnormal fluctuations and extract representative performance characteristics.

[0030] Furthermore, the performance test baselines generated after standardization include: throughput baseline, latency distribution baseline, maximum concurrent user capacity baseline, and resource utilization baseline. Specifically, the throughput baseline represents the number of inference requests successfully processed per unit time, such as requests per second or tokens generated per second. The latency distribution baseline includes multi-dimensional quantile data covering the latency of the first token, adjacent tokens, and the complete response latency. The maximum concurrent user capacity baseline represents the maximum number of concurrent users the system can support under the condition of meeting a preset latency threshold. The resource utilization baseline represents the consumption level of key resources such as GPU computing unit utilization, video memory occupancy, and memory bandwidth utilization.

[0031] Those skilled in the art should understand that the method of generating test task instances through a continuous pipeline and running the test task instances through a continuous pipeline to obtain the corresponding baseline of the running results, given that the declaration configuration file and test tasks are known, is a mature existing technology. This embodiment only introduces its steps and does not elaborate on its principles and specific calculation methods.

[0032] S130. Obtain the historical performance baselines that match each test task in the test task set, and obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

[0033] The historical performance baseline refers to the performance data records that have been standardized and stored during the historical version verification process for the same or similar test tasks.

[0034] Optionally, after running each test task in the test task set using the target verification model through a pre-defined continuous pipeline, the method further includes: setting at least one performance quality gate node in the continuous pipeline, and performing an admission judgment on each performance verification result through each performance quality gate node; if the admission judgment of the performance quality gate node shows that there is performance regression and exceeds the preset tolerance range, then the performance quality gate node is determined to fail, the subsequent release process of the continuous pipeline is automatically blocked, and a warning message is generated and sent to the user; if the admission judgment of the performance quality gate node shows that the preset admission rules are met, then the quality gate is determined to pass.

[0035] The performance quality access control node refers to an automated judgment checkpoint embedded in a specific stage of a continuous pipeline, used to determine whether the pipeline is allowed to continue based on performance verification results.

[0036] In this embodiment, specifically, if the admission judgment of the performance quality access control node shows performance regression exceeding a preset tolerance range, the performance quality access control node is deemed to fail, automatically blocking subsequent release processes in the continuous pipeline and generating a warning message sent to the user. The preset tolerance range is defined by a threshold parameter in a pre-set access rule. If the admission judgment of the performance quality access control node shows that the preset access rule is met, the quality access control is deemed to pass. The preset access rule refers to a set of multiple judgment conditions that performance indicators must simultaneously meet. For example, in this embodiment, it could specifically include throughput not being lower than a specified percentage of the historical baseline, latency at each percentile not exceeding a specified multiple of the historical baseline, request success rate not being lower than a preset lower limit, and resource utilization being within a reasonable range.

[0037] Those skilled in the art should understand that the method for setting up and applying access control nodes in a continuous pipeline is a mature existing technology, and this embodiment will not elaborate on its principle and specific application method.

[0038] The technical solution of this invention obtains a set of test tasks for a target verification model in response to a version change event. Then, it runs each test task in the set of test tasks using the target verification model through a pre-set continuous pipeline to obtain a performance test baseline that matches each test task. Finally, it obtains historical performance baselines that match each test task in the set of test tasks and obtains the performance verification result of the target verification model based on each performance test baseline and each historical performance baseline. This achieves automated performance verification of large language models and improves the efficiency of model performance verification and the reliability of verification results.

[0039] Example 2 Figure 2 This is a flowchart of a performance verification method for a large language model provided in Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiment. Specifically, this embodiment refines the method of obtaining historical performance baselines that match each test task in the test task set, and obtaining the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

[0040] like Figure 2 As shown, the method includes: S210. In response to the triggering of a version change event, obtain the set of test tasks for the target verification model.

[0041] S220. Run each test task in the test task set using the target verification model through a pre-set continuous pipeline to obtain a performance test baseline that matches each test task.

[0042] S230. Obtain historical performance baselines from a pre-built performance baseline library that match the test task in terms of model version information, inference engine information, target hardware environment information, and scene information.

[0043] The performance baseline library refers to a structured data warehouse used for versioned storage of performance test baselines. It is indexed according to multiple dimensions such as model version information, inference engine information, target hardware environment information, and scenario information to support efficient retrieval and accurate matching. Matching means that the attribute values ​​of the above-mentioned dimensions are completely consistent except for the test execution timestamp. That is, the historical performance baseline and the current test task have the same model version information, inference engine information, target hardware environment information, and scenario information, with only the test execution time being different.

[0044] S240. Calculate the difference between the performance test baseline and the corresponding historical performance baseline.

[0045] The difference refers to a numerical indicator that quantifies the degree of change of the current performance test baseline relative to the historical performance baseline. Furthermore, the difference includes the throughput change rate, latency change rate, and concurrent load capacity change rate; the positive or negative sign of the change rate is used to indicate the direction of performance optimization or degradation.

[0046] It should be noted that the method for calculating the difference is a mature mathematical calculation method, and the specific calculation process will not be described in detail in this embodiment.

[0047] S250. Based on the preset performance regression detection rules, determine whether the difference exceeds the tolerance threshold range, and obtain the regression detection results of each test task.

[0048] The performance regression detection rule refers to a set of conditional expressions used to determine whether unacceptable performance degradation has occurred. The tolerance threshold range refers to the upper and lower boundary intervals within which performance indicators are allowed to fluctuate. If the difference falls within the tolerance threshold range, it is determined that the corresponding test task has not experienced performance regression; otherwise, it is determined that performance regression has occurred. Furthermore, the tolerance threshold range can be configured differently for different types of performance indicators. For example, the throughput indicator uses a preset lower threshold, and the latency indicator uses a preset lower threshold.

[0049] S260. Based on the regression detection results of each test task in the test task set, generate the performance verification results of the target verification model.

[0050] Specifically, the performance verification result of the target verification model is generated based on the regression detection results of each test task in the test task set, including: performing a weighted aggregation operation on the regression detection results of each test task to obtain a comprehensive performance score of the target verification model; and comparing the comprehensive performance score with a preset verification threshold to generate the performance verification result of the target verification model.

[0051] Specifically, the weighted aggregation operation refers to the mathematical processing of weighted summation or weighted average of each regression detection result based on the importance weights of each test task set in advance. The preset verification threshold is the critical score used to determine whether the overall performance of the target verification model passes the verification. If the comprehensive performance score reaches or exceeds the verification threshold, a performance verification result that has passed the verification is generated; otherwise, a performance verification result that has failed the verification is generated.

[0052] The technical solution of this invention, in response to a version change event, obtains a set of test tasks for a target verification model. Then, it runs each test task in the set using the target verification model through a pre-defined continuous pipeline, obtaining a performance test baseline matching each test task. Next, it retrieves historical performance baselines from a pre-built performance baseline library that match the test tasks in terms of model version information, inference engine information, target hardware environment information, and scene information. It then calculates the difference between the performance test baseline and the corresponding historical performance baseline, and based on preset performance regression detection rules, determines whether the difference exceeds a tolerance threshold, obtaining the regression detection results for each test task. Finally, based on the regression detection results of each test task in the set of test tasks, it generates the performance verification results for the target verification model. This achieves automated performance verification of large language models, improving the efficiency of model performance verification and the reliability of the verification results.

[0053] Example 3 Figure 3 This is a schematic diagram of the structure of a performance verification device for a large language model provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: Event triggering module 310 is used to obtain the test task set of the target verification model in response to the triggering of version change events; The performance baseline acquisition module 320 is used to run each test task in the test task set using the target verification model through a pre-set continuous pipeline to obtain a performance test baseline that matches each test task. The verification result generation module 330 is used to obtain the historical performance baselines that match each test task in the test task set, and to obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

[0054] The technical solution of this invention obtains a set of test tasks for a target verification model in response to a version change event. Then, it runs each test task in the set of test tasks using the target verification model through a pre-set continuous pipeline to obtain a performance test baseline that matches each test task. Finally, it obtains historical performance baselines that match each test task in the set of test tasks and obtains the performance verification result of the target verification model based on each performance test baseline and each historical performance baseline. This achieves automated performance verification of large language models and improves the efficiency of model performance verification and the reliability of verification results.

[0055] Based on the above embodiments, the performance baseline acquisition module 320 includes: The instance generation unit is used to generate test task instances corresponding to each test task in the continuous pipeline based on each test task in the test task collection through a declaration configuration file. The instance execution unit is used to respond to the generation of each test task instance, run each test task instance through pre-configured automated testing tools, and obtain the raw performance index data corresponding to each test task instance. The standardization processing unit is used to standardize the raw performance index data and generate performance test baselines that match each test task. The performance test baselines include: throughput baseline, latency distribution baseline, maximum concurrent load baseline, and resource utilization baseline.

[0056] Based on the above embodiments, the verification result generation module 330 includes: The historical baseline acquisition unit is used to acquire historical performance baselines that match the test task in terms of model version information, inference engine information, target hardware environment information, and scene information from a pre-built performance baseline library. A difference calculation unit is used to calculate the difference between the performance test baseline and the corresponding historical performance baseline. The regression detection unit is used to determine whether the difference exceeds the tolerance threshold range based on the preset performance regression detection rules, and to obtain the regression detection results of each test task. The result generation unit is used to generate the performance verification results of the target verification model based on the regression detection results of each test task in the test task set.

[0057] Based on the above embodiments, the performance baseline acquisition module 320 is further configured to: after running each test task in the test task set using the target verification model through a pre-set continuous pipeline, set at least one performance quality gate node in the continuous pipeline, and perform admission judgment on each performance verification result through each performance quality gate node; if the admission judgment of the performance quality gate node shows that there is performance regression and exceeds the preset tolerance range, then the performance quality gate node is determined to fail, the subsequent release process of the continuous pipeline is automatically blocked, and a warning message is generated and sent to the user; if the admission judgment of the performance quality gate node shows that the preset admission rules are met, then the quality gate is determined to pass.

[0058] Based on the above embodiments, the result generation unit includes: The weighted aggregation unit is used to perform weighted aggregation operations on the regression detection results of each test task to obtain the comprehensive performance score of the target verification model; The threshold comparison unit is used to compare the comprehensive performance score with a preset verification threshold to generate the performance verification result of the target verification model.

[0059] The performance verification device for a large language model provided in this embodiment of the invention can execute the performance verification method for a large language model provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0060] Example 4 Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0061] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0062] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0063] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a performance verification method for a large language model.

[0064] Accordingly, the method includes: In response to a version change event, retrieve the set of test tasks for the target validation model; The target verification model is used to run each test task in the test task set through a pre-defined continuous pipeline to obtain a performance test baseline that matches each test task. Obtain the historical performance baselines that match each test task in the test task set, and obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

[0065] In some embodiments, a performance verification method for a large language model can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the performance verification method for a large language model described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a performance verification method for a large language model by any other suitable means (e.g., by means of firmware).

[0066] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0067] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0068] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0069] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0070] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0071] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0072] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

Claims

1. A performance verification method for a large language model, characterized in that, include: In response to a version change event, retrieve the set of test tasks for the target validation model; The target verification model is used to run each test task in the test task set through a pre-defined continuous pipeline to obtain a performance test baseline that matches each test task. Obtain the historical performance baselines that match each test task in the test task set, and obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

2. The method according to claim 1, characterized in that, The test task includes: version information of the target verification model, inference engine information, target hardware environment information, declaration configuration file, and scenario information. The declaration configuration file is used to define the target verification model, inference engine, test case set, and performance acceptance criteria.

3. The method according to claim 1, characterized in that, The test tasks in the test task set are run using a pre-defined continuous pipeline and a target verification model to obtain a performance test baseline that matches each test task, including: In the continuous pipeline, based on each test task in the test task collection, test task instances corresponding to each test task are generated through a declaration configuration file; In response to the generation of each test task instance, the pre-configured automated testing tools are used to run each test task instance to obtain the raw performance index data corresponding to each test task instance. The original performance index data are standardized to generate performance test baselines that match each test task. These performance test baselines include: throughput baseline, latency distribution baseline, maximum concurrent load baseline, and resource utilization baseline.

4. The method according to claim 1, characterized in that, Obtain the historical performance baselines that match each test task in the test task set, and obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline, including: From a pre-built performance baseline library, obtain historical performance baselines that match the test task in terms of model version information, inference engine information, target hardware environment information, and scene information. Calculate the difference between the performance test baseline and the corresponding historical performance baseline; Based on the preset performance regression detection rules, it is determined whether the difference exceeds the tolerance threshold range, and the regression detection results of each test task are obtained. Based on the regression detection results of each test task in the test task set, the performance verification results of the target verification model are generated.

5. The method according to claim 1, characterized in that, After running each test task in the test task set using the target verification model through a pre-defined continuous pipeline, the process also includes: At least one performance quality gate node is set in the continuous pipeline, and each performance quality gate node performs an admission judgment on each performance verification result. If the admission judgment of the performance quality access control node shows that there is performance regression and exceeds the preset tolerance range, the performance quality access control node is determined to be unqualified, the subsequent release process of the continuous pipeline is automatically blocked, and a warning message is generated and sent to the user. If the access control of the performance quality access node shows that it meets the preset access rules, then the quality access control is deemed to be passed.

6. The method according to claim 4, characterized in that, Based on the regression detection results of each test task in the test task set, the performance verification results of the target verification model are generated, including: The regression detection results of each test task are weighted and aggregated to obtain the comprehensive performance score of the target verification model; The overall performance score is compared with a preset verification threshold to generate the performance verification result of the target verification model.

7. The method according to claim 1, characterized in that, The version change events include at least one of the following: model version iteration events, inference engine version update events, and underlying hardware environment change events.

8. A performance verification device for a large language model, characterized in that, include: The event triggering module is used to respond to version change events and obtain the set of test tasks for the target verification model. The performance baseline acquisition module is used to run each test task in the test task set using the target verification model through a pre-set continuous pipeline to obtain the performance test baseline that matches each test task. The verification result generation module is used to obtain the historical performance baselines that match each test task in the test task set, and to obtain the performance verification results of the target verification model based on each performance test baseline and each historical performance baseline.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a performance verification method for a large language model according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute a performance verification method for a large language model according to any one of claims 1-7.

Images