Version management method and apparatus for large language model, and device and medium

Abstract

A version management method and apparatus for a large language model, and a device and a medium. The method comprises: acquiring version information and metadata which correspond to a large language model training task within a target tenant, wherein the version information comprises timestamp information and region information, and the metadata comprises performance indicator data; on the basis of the timestamp information, the region information and the performance indicator data, determining a version rollback strategy for the large language model; and on the basis of the version rollback strategy, executing a version rollback operation on the large language model, so as to complete the version rollback of the large language model.

Description

Version management methods, devices, equipment, and media for large language models

[0001] This application claims priority to Chinese Patent Application No. 202510193916.1, filed with the Chinese Patent Office on February 21, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence technology, such as a method, apparatus, device, and medium for version management of a large language model. Background Technology

[0003] With the widespread application of large language models across various industries, the demand for their application in the automotive finance sector is growing and becoming increasingly sophisticated.

[0004] Large model fine-tuning, as an efficient and economical adaptation method, has become a widespread application model in the field by utilizing a base model and combining it with domain-specific data for targeted training. However, in the special scenario of auto finance, large model fine-tuning still faces many challenges, especially the differences in version requirements caused by changes in time points and functionalities. This not only increases the complexity of version management but also makes version rollback operations particularly difficult. Summary of the Invention

[0005] This application provides a version management method, apparatus, device, and medium for large language models, which can significantly simplify the complexity of version management and improve the ability to quickly and accurately roll back for specific errors.

[0006] According to one aspect of this application, a version management method for a large language model is provided, comprising:

[0007] Obtain version information and metadata corresponding to the large language model training task within the target tenant; wherein, the version information includes timestamp information and region information; the metadata includes performance index data;

[0008] The version rollback strategy for the large language model is determined based on the timestamp information, the geographic information, and the performance index data.

[0009] The version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0010] According to another aspect of this application, a version management device for a large language model is provided, comprising:

[0011] The data acquisition module is configured to acquire version information and metadata corresponding to the large language model training task within the target tenant; wherein, the version information includes timestamp information and geographic information; and the metadata includes performance index data.

[0012] The strategy determination module is configured to determine the version rollback strategy of the large language model based on the timestamp information, the regional information, and the performance index data.

[0013] The rollback operation execution module is configured to perform a version rollback operation on the large language model according to the version rollback strategy, so as to complete the version rollback of the large language model.

[0014] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the version management method for a large language model as described in any embodiment of this application.

[0018] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions configured to cause a processor to execute and implement the version management method of the large language model described in any embodiment of this application.

[0019] The technical solution of this application embodiment obtains version information and metadata corresponding to the training task of a large language model within the target tenant. The version information includes timestamp information and region information; the metadata includes performance metric data. A version rollback strategy for the large language model is determined based on the timestamp information, the region information, and the performance metric data. A version rollback operation is then performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model. This technical solution can determine the corresponding version rollback strategy based on the performance data and timestamp information of the large language model, thereby significantly simplifying the complexity of version management and improving the ability to quickly and accurately roll back for specific errors. Attached Figure Description

[0020] Figure 1 is a flowchart of a version management method for a large language model according to Embodiment 1 of this application;

[0021] Figure 2 is a flowchart of a version management method for a large language model according to Embodiment 2 of this application;

[0022] Figure 3 is a schematic diagram of a version management device for a large language model according to Embodiment 3 of this application;

[0023] Figure 4 is a schematic diagram of the structure of an electronic device according to Embodiment 4 of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," and "target," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] Figure 1 is a flowchart of a version management method for a large language model according to Embodiment 1 of this application. This embodiment is applicable to the situation of version management of a large language model. The method can be executed by a version management device for a large language model, which can be implemented in hardware and / or software. The version management device for a large language model can be configured in an electronic device with data processing capabilities. As shown in Figure 1, the method includes:

[0028] S110. Obtain the version information and metadata corresponding to the large language model training task within the target tenant.

[0029] The version information includes timestamp information and geographic information; metadata includes performance metrics. Timestamp information can include the start time, end time, and deployment time of the training for each training task. Version information typically refers to the model version used in the large language model training task. Metadata can refer to the data of the large language model training task, specifically including training data, algorithm parameters, performance metrics, etc. Geographic information can refer to the geographical region or market information used by the large language model. It is understood that business requirements differ across regions in this embodiment, therefore the large language models corresponding to different regions will also differ. A large language model refers to a deep learning model trained using a large amount of text data, enabling the model to generate natural language text or understand the meaning of language text. In this embodiment, the large language model refers to an advanced model that provides customized natural language processing services to tenants in a multi-tenant service environment.

[0030] The target tenant can be any tenant on the training platform. This embodiment further refines the resource allocation and permission management for each tenant combination to achieve efficient resource utilization and resource isolation between tenants. This embodiment designs an enhanced resource isolation method for each tenant on the training platform.

[0031] Regarding resource isolation, this embodiment leverages the powerful container orchestration and resource management capabilities of Kubernetes (k8s) with targeted optimizations and innovations. Specifically, this embodiment addresses the establishment and database-level isolation of the training platform's tenant database. First, after a new tenant completes information entry through the training platform's registration system, the platform automatically triggers the database creation process, generating an independent database instance for each new tenant. These database instances achieve strict database-level isolation at both the physical and logical levels, ensuring the security of tenant data. Next, a Role-Based Access Control (RBAC) mechanism is introduced to further refine permission management between tenants. Based on the tenant's business needs and responsibilities, various roles such as administrator, developer, and tester are defined, and clear permission boundaries are assigned to each role. These permission boundaries are implemented through k8s' RBAC mechanism, i.e., fine-grained permission management is achieved by defining Roles and RoleBindings. In terms of implementation details, precise permission control is achieved by ensuring that role definitions are detailed down to specific API resources, operation types, and scopes. Simultaneously, we regularly audit permission usage to promptly identify and correct any abnormal or abusive behavior, ensuring that operations between tenants do not interfere with each other. In terms of resource allocation and management, we utilized Kubernetes' Custom Resource Definition (CRD) feature to define a custom resource type, "TenantResource." This innovative approach allows us to configure dedicated resource quotas for each "TenantResource," including critical resources such as CPU, memory, and storage, based on the specific needs of each tenant. In terms of implementation details, the definition of "TenantResource" includes key information such as tenant identifier, resource type, and resource quota to ensure the accuracy and rationality of resource allocation. Simultaneously, we dynamically adjust resource quota configurations based on tenant business scale, growth trends, and peak demand to meet the ever-changing business needs of tenants. Furthermore, we developed appropriate scheduling strategies, comprehensively considering resource availability, tenant priority, and affinity / anti-affinity rules to achieve efficient resource utilization and isolation between tenants. Finally, network security was also given high priority. By selecting Kubernetes network plugins that support virtual network isolation (such as Calico or Cilium), we created an independent virtual network for each tenant. These virtual networks are logically isolated from each other, effectively preventing network attacks and data leakage risks between tenants. In terms of implementation details, the creation process of the virtual network is meticulously planned based on the tenant's business needs and network topology, ensuring that network policies are configured down to specific network ports, protocols, and IP address ranges. Simultaneously, network traffic and abnormal behavior are regularly monitored to promptly detect and address any potential network security threats, ensuring a secure and stable network environment for tenants.

[0032] In this implementation, each tenant corresponds to different physical resources. The large language model is deployed on the physical resources corresponding to each tenant. This embodiment utilizes Kubernetes Annotations to record detailed version information and metadata for each training task, including training data, algorithm parameters, performance metrics, etc. Specifically, when recording version information and metadata using Kubernetes Annotations, a timestamp is added for each version, including the training start time, end time, and deployment time. This helps track the model's performance over different time periods and provides data support for time-based rollback strategies. A geographic label is added to each version to indicate the geographical region or market to which the model is applicable. This helps to quickly locate the region when model rollback is required. Therefore, in this embodiment, when managing versions for different tenants, detailed version information, training data, algorithm parameters, performance metrics, etc., corresponding to each training task of the target tenant's large language model can be obtained.

[0033] S120. Determine the version rollback strategy for the large language model based on timestamp information, regional information, and performance index data.

[0034] Among them, the version rollback strategy can be considered as the most suitable model version determined when a large language model needs to be rolled back.

[0035] In this embodiment, the model performance data of each version of the large language model can be analyzed based on timestamp information, regional information and performance index data. The model version with the best performance or the fewest problems can be selected as the rollback target. If there are multiple candidate versions, the best rollback version can be determined through further testing or performance evaluation to obtain the version rollback strategy of the large language model.

[0036] In this embodiment, when a rollback operation is required for a large language model, the system not only locates the most suitable rollback version based on timestamp information, regional information, and performance metrics, but also automatically analyzes the cause of the error and generates a rollback suggestion report for user reference. Furthermore, this embodiment introduces the concept of a version chain, allowing users to select a specific version chain for rollback, thereby avoiding the impact of erroneous data or algorithms on subsequent versions. In special business scenarios, such as when updating a credit assessment model, a time window-based rollback strategy is also designed to ensure a smooth model rollback without affecting business continuity.

[0037] S130. Perform a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0038] In this embodiment, after determining the optimal model version in the version rollback strategy, the current version of the large language model service can be stopped first to avoid data conflicts or inconsistencies. Then, based on the version rollback operation performed on the large language model, the large language model is restored to the specified optimal model version, and the integrity and correctness of the rolled-back model files and data are verified, thereby completing the smooth rollback of the large language model and ensuring the model performance of the large language model without affecting business continuity.

[0039] The technical solution of this application embodiment obtains version information and metadata corresponding to the training task of a large language model within the target tenant. The version information includes timestamp information and region information; the metadata includes performance metric data. A version rollback strategy for the large language model is determined based on the timestamp information, region information, and performance metric data. A version rollback operation is then performed on the large language model according to the rollback strategy to complete the version rollback. This technical solution can determine the corresponding version rollback strategy based on the performance data and timestamp information of the large language model, thereby significantly simplifying the complexity of version management and improving the ability to quickly and accurately roll back for specific errors.

[0040] Example 2

[0041] Figure 2 is a flowchart of a version management method for a large language model according to Embodiment 2 of this application. This embodiment is an optimization based on the above embodiment. Specifically, the optimization involves determining the version rollback strategy of the large language model based on timestamp information, regional information, and performance indicator data, including: determining the target performance data for each version of the large language model based on timestamp information and performance indicator data; and determining the version rollback strategy of the large language model based on the target performance data and / or regional information for each version. As shown in Figure 2, the method includes:

[0042] S210. Obtain the version information and metadata corresponding to the large language model training task within the target tenant.

[0043] The version information includes timestamp information and region information; the metadata includes performance metrics data.

[0044] S220. Determine the target performance data for each version in the large language model based on timestamp information and performance index data.

[0045] The target performance data can be considered as the specified performance data of the large language model in each version. For example, the target performance data can be performance data such as business continuity and credit assessment accuracy. In this embodiment, the corresponding target performance data can be set according to the user's actual needs.

[0046] In this embodiment, the credit assessment results of the large language model under different versions can be evaluated using performance index data, and the changes in different credit assessment results within the time window can be determined by combining timestamp information, thereby determining the impact of rolling back to different versions on the model performance data.

[0047] In this embodiment, optionally, the target performance data for each version of the large language model is determined based on timestamp information and performance index data, including: evaluating the credit assessment impact of the large language model in each version based on performance index data; and determining the target performance data of the large language model based on the credit assessment impact and timestamp information.

[0048] The impact of credit assessment can refer to the expected changes in the credit assessment results corresponding to different versions. In this embodiment, the impact of credit assessment results can be determined by performance index data and machine learning algorithm prediction models to identify the impact of credit assessment results for different versions. This embodiment uses performance index data and machine learning algorithms to obtain the impact of credit assessment results for each version, then combines timestamp information to analyze the changes in the impact of credit assessment results of the large language model over various time periods. Finally, based on time series analysis and version comparison results, the target performance data for each version of the large language model is determined.

[0049] In this embodiment, by combining timestamp information and performance index information, the target performance data of each version of the large language model can be analyzed, so as to determine the best rollback version of the large language model through analysis and comparison.

[0050] In this embodiment, optionally, the performance indicator data includes accuracy indicator data, recall indicator data, and harmonic indicator data; evaluating the impact of the large language model on credit assessment in each version based on the performance indicator data includes: inputting the accuracy indicator data, recall indicator data, and harmonic indicator data corresponding to each version into the credit assessment prediction model to obtain the credit assessment impact results for each version.

[0051] Among these metrics, accuracy can be considered the precision data of a large language model, used to evaluate the correctness of the model's processing results. Recall can be considered the recall rate data of a large language model, also known as the total recall rate, which is the proportion of instances correctly identified as positive (true positives) out of all actual positive instances. Harmonic metrics can refer to the F1 score, which is expressed as the harmonic mean of precision and recall. Harmonic metrics take into account both metrics to reduce the misleading nature of relying on a single metric.

[0052] In this embodiment, the metadata includes key performance indicators for credit assessment, such as accuracy, recall, and F1 score, as well as the trends of these indicators over time, in order to evaluate the performance of the large language model in the credit assessment task and provide a basis for rollback decisions.

[0053] In this embodiment, the accuracy, recall, and F1 score data corresponding to each version can be input into the trained credit assessment prediction model to obtain the credit assessment impact results corresponding to each version.

[0054] Furthermore, in this embodiment, a rollback strategy based on a time window is designed for the credit assessment prediction model. For example, before peak business periods or within a specific time period, the system automatically evaluates the performance of the current large language model and decides whether to roll back based on a preset threshold. When performing a rollback operation, the most recent version that has shown stable performance and good credit assessment performance within the time window can be selected for deployment.

[0055] This embodiment, through such a setup, can utilize historical data and machine learning algorithms to predict the expected changes in credit assessment results after rolling back to different versions, helping to make more informed rollback decisions and improving the reliability of the rollback operation.

[0056] S230. Determine the version rollback strategy for the large language model based on the target performance data and / or regional information of each version.

[0057] In this embodiment, the version chain not only records the linear relationship between versions but also marks the regional applicability and target performance data of each version. The version rollback strategy for the large language model can be determined based on the target performance data, regional information, or both. In this embodiment, the version rollback strategy can include the model version to be rolled back and the specific rollback method.

[0058] Specifically, in this embodiment, during the rollback operation, the system can select the version chain with the least impact on a specific region or credit assessment performance for rollback. When determining the version rollback strategy based on regional information, if the error or performance degradation only affects a specific region, the system recommends rolling back only the model for that region to maintain the stability and accuracy of models in other regions.

[0059] In this embodiment, optionally, determining the version rollback strategy of the large language model based on the target performance data and / or regional information of each version includes: generating a rollback recommendation report of the large language model based on the target performance data and / or regional information of each version; and determining the version rollback strategy of the large language model based on the rollback recommendation report.

[0060] The rollback recommendation report can include multiple rollback versions, rollback methods, and risk assessments. In this embodiment, one or more historical versions can be selected as rollback targets based on target performance data and regional information. Then, the specific rollback method is determined according to actual needs, such as directly rolling back to a specified version or fine-tuning based on a specific version. Direct rollback may involve operations of the version control system, while fine-tuning requires adjustments and optimizations based on the selected version. The potential risks of executing each rollback version are then assessed, such as performance degradation and data loss. These risks should be identified and mitigated through thorough testing. This embodiment generates a large language model rollback recommendation report by combining the rollback target, rollback method, and risk assessment. Then, based on the rollback recommendation report and the user's actual needs, the most suitable rollback version and rollback method are determined as the version rollback strategy.

[0061] Furthermore, when generating the rollback suggestion report in this embodiment, the system not only analyzes the cause of the error, but also combines the changes in credit assessment indicators within the time window to assess the impact of rolling back to different versions on business continuity and the accuracy of credit assessment.

[0062] In this embodiment, by setting up the most suitable version rollback strategy for the large language model based on the target performance data and regional data of each version, the stability and performance of the large language model in different tasks and regions can be ensured.

[0063] S240. Perform a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0064] In this embodiment, optionally, a version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model, including: performing a version rollback operation within the first rollback range of the large language model based on the version rollback strategy to obtain model performance data after the version rollback; and continuing to perform a version rollback operation on the large language model based on the model performance data after the version rollback to complete the version rollback of the large language model.

[0065] The first rollback range can be a small range within the environment where the large language model is deployed. In this embodiment, the first rollback range can be set according to actual needs.

[0066] In this embodiment, for large-scale deployed language models, a phased rollback approach can be used to roll back the large language model. Rollback testing can be performed initially on a small scale to ensure the model's performance is stable and meets expectations after the rollback. Then, the rollback scope can be gradually expanded to complete the version rollback of the large language model. For example, a large-scale deployed language model can be considered as a model that is already widely used, such as dozens of servers on an online platform all using the large language model.

[0067] In this embodiment, a version rollback operation can be performed within a pre-defined first rollback range of the large language model according to a determined version rollback strategy. Performance data after version rollback within the first rollback range is obtained. Based on the performance data of the large language model after rollback within the first rollback range, it is determined whether the expected effect is met. Then, the rollback range is gradually expanded to continue performing version rollback operations on the large language model, thereby gradually realizing the version rollback of the large language model.

[0068] In this embodiment, this configuration allows for a gradual rollback operation on large language models deployed on a large scale, further ensuring the performance and rollback effect of the large language models.

[0069] In this embodiment, optionally, the version rollback operation is continued to be performed on the large language model based on the model performance data after the version rollback, so as to complete the version rollback of the large language model. This includes: if the model performance data after the version rollback meets the set conditions, the version rollback operation is gradually continued to be performed in the second rollback range until the version rollback operation is completed in the entire range of the large language model.

[0070] The second rollback range is larger than the first rollback range. The setting conditions can be based on model performance data or can be pre-defined. In this embodiment, the setting conditions can be that the model performance data meets the model's expected data; the specific expected data can be configured according to the actual large language model.

[0071] In this embodiment, for models deployed on a large scale, a phased rollback strategy is adopted. First, rollback testing is performed within the first rollback scope to ensure that the model performance data after the rollback meets the set commit parameters, i.e., the model performance after the rollback is stable and meets expectations. Then, the rollback scope is gradually expanded, and the rollback operation continues to be performed within the second rollback scope of the large language model until the rollback operation is completed within the entire scope of the large language. If the model performance data after the rollback does not meet the set conditions, the rollback operation for the large language model can be stopped, and the rollback strategy can be adjusted according to the model performance.

[0072] In this embodiment, during the rollback process of the large language model, the model's performance and credit evaluation metrics are monitored in real time to promptly detect and handle any anomalies. Simultaneously, user feedback is collected to continuously evaluate and optimize the rollback effect.

[0073] This embodiment uses this configuration to allow for a gradual rollback of large-scale deployed language models, ensuring the flexibility and reliability of the rollback and improving the stability of the large language model's performance.

[0074] This embodiment further improves and optimizes the version rollback process by combining the time-sensitive, geographical, and credit assessment attributes of auto finance. This not only enhances the flexibility and accuracy of the rollback but also ensures business continuity and the accuracy of credit assessment.

[0075] The technical solution of this application embodiment obtains version information and metadata corresponding to the training task of a large language model within the target tenant. The version information includes timestamp information and region information; the metadata includes performance metric data. Based on the timestamp information and performance metric data, the target performance data for each version in the large language model is determined. Based on the target performance data and / or region information for each version, a version rollback strategy for the large language model is determined. A version rollback operation is performed on the large language model according to the rollback strategy to complete the version rollback. This technical solution can determine the corresponding version rollback strategy based on the performance data and timestamp information of the large language model, thereby significantly simplifying the complexity of version management and improving the ability to quickly and accurately roll back for specific errors.

[0076] Example 3

[0077] Figure 3 is a structural schematic diagram of a version management device for a large language model according to Embodiment 3 of this application. As shown in Figure 3, the device includes:

[0078] The data acquisition module 310 is configured to acquire version information and metadata corresponding to the large language model training task within the target tenant; the version information includes timestamp information and geographic information; the metadata includes performance metric data.

[0079] The strategy determination module 320 is configured to determine the version rollback strategy of the large language model based on timestamp information, regional information, and performance index data.

[0080] The rollback operation execution module 330 is configured to perform a version rollback operation on the large language model according to the version rollback strategy in order to complete the version rollback of the large language model.

[0081] Optionally, the strategy determination module 320 includes:

[0082] The target performance data determination unit is set to determine the target performance data of each version in the large language model based on timestamp information and performance index data.

[0083] The version rollback strategy determination unit is set to determine the version rollback strategy of the large language model based on the target performance data and / or regional information of each version.

[0084] Optionally, the target performance data determination unit includes:

[0085] The sub-unit for evaluating impact results is set up to evaluate the impact of the large language model on credit assessment results in different versions based on performance index data.

[0086] The performance data determination subunit is set up to determine the target performance data of the large language model based on the credit assessment impact results and timestamp information.

[0087] Optional, performance metrics include accuracy, recall, and harmonic metrics;

[0088] The sub-unit for evaluating the impact on the outcome is specifically set as follows:

[0089] The accuracy, recall, and harmonization metrics corresponding to each version are input into the credit assessment prediction model to obtain the credit assessment impact results for each version.

[0090] Optional, the version rollback strategy determination unit is specifically set as follows:

[0091] Generate a rollback recommendation report for the large language model based on the target performance data and / or regional information of each version;

[0092] Determine the version rollback strategy for the large language model based on the rollback recommendation report.

[0093] Optionally, the rollback operation execution module 330 includes:

[0094] The first rollback processing unit is set to perform version rollback operations within the first rollback range of the large language model based on the version rollback strategy, so as to obtain the model performance data after the version rollback.

[0095] The rollback operation continues execution unit is set to continue performing the version rollback operation on the large language model based on the model performance data after the version rollback, so as to complete the version rollback of the large language model.

[0096] Optionally, the rollback operation can continue execution of the unit, specifically configured as follows:

[0097] If the model performance data after the version rollback meets the set conditions, the version rollback operation will continue to be executed gradually in the second rollback range until the version rollback operation is completed in the entire range of the large language model; wherein, the second rollback range is larger than the first rollback range.

[0098] The version management device for large language models provided in this application can execute the version management method for large language models provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the execution method.

[0099] Example 4

[0100] Figure 4 is a schematic diagram of an electronic device according to Embodiment 4 of this application. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0101] As shown in Figure 4, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0103] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as version management methods for large language models.

[0104] In some embodiments, the version management method for a large language model can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the version management method for a large language model described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the version management method for a large language model by any other suitable means (e.g., by means of firmware).

[0105] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0109] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0110] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0111] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

Claims

1. A version management method for a large language model, comprising: Obtain version information and metadata corresponding to the large language model training task within the target tenant; wherein, the version information includes timestamp information and region information; the metadata includes performance index data; The version rollback strategy for the large language model is determined based on the timestamp information, the geographic information, and the performance index data. The version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

2. The method according to claim 1, wherein, The version rollback strategy for the large language model is determined based on the timestamp information, the geographic information, and the performance metric data, including: The target performance data for each version in the large language model is determined based on the timestamp information and the performance metric data. The version rollback strategy for the large language model is determined based on the target performance data of each version and / or the regional information.

3. The method according to claim 2, wherein, Based on the timestamp information and the performance metric data, the target performance data for each version of the large language model is determined, including: The impact of the large language model on credit assessment results in each version is evaluated based on the performance index data. The target performance data of the large language model is determined based on the credit assessment impact results and the timestamp information.

4. The method according to claim 3, wherein, The performance metrics data include accuracy metrics data, recall metrics data, and harmonic metrics data; The impact of the large language model on credit assessment results in each version is evaluated based on the performance metric data, including: The accurate indicator data, recall indicator data, and harmonization indicator data corresponding to each version are input into the credit assessment prediction model to obtain the credit assessment impact results of each version.

5. The method according to claim 2, wherein, Based on the target performance data of each version and / or the regional information, the version rollback strategy of the large language model is determined, including: A rollback recommendation report for the large language model is generated based on the target performance data of each version and / or the regional information. The version rollback strategy for the large language model is determined based on the rollback recommendation report.

6. The method according to claim 1, wherein, Perform a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model, including: Based on the version rollback strategy, a version rollback operation is performed within the first rollback range of the large language model to obtain the model performance data after the version rollback. Based on the model performance data after the version rollback, the large language model continues to be rolled back to complete the version rollback of the large language model.

7. The method according to claim 6, wherein, Based on the model performance data after the version rollback, the large language model continues to undergo version rollback operations to complete the version rollback of the large language model, including: If the model performance data after the version rollback meets the set conditions, the version rollback operation will continue to be executed gradually in the second rollback range until the version rollback operation is completed in the entire range of the large language model; wherein, the second rollback range is larger than the first rollback range.

8. A version management device for a large language model, comprising: The data acquisition module is configured to acquire version information and metadata corresponding to the large language model training task within the target tenant; wherein, the version information includes timestamp information and geographic information; and the metadata includes performance index data. The strategy determination module is configured to determine the version rollback strategy of the large language model based on the timestamp information, the regional information, and the performance index data. The rollback operation execution module is configured to perform a version rollback operation on the large language model according to the version rollback strategy, so as to complete the version rollback of the large language model.

9. An electronic device, the electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the version management method for the large language model according to any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions configured to cause a processor to execute and implement the version management method for a large language model according to any one of claims 1-7.

Images