Version management method and device of large language model, equipment and medium

Abstract

The invention discloses a version management method and device for a large language model, equipment and a medium. The method comprises the steps of obtaining version information and metadata corresponding to a large language model training task in a target tenant; wherein the version information comprises timestamp information and regional information; the metadata comprises performance index data; determining a version rollback strategy of the large language model according to the timestamp information, the regional information and the performance index data; and executing a version rollback operation on the large language model according to the version rollback strategy so as to complete version rollback of the large language model. According to the technical scheme, the corresponding version rollback strategy can be determined according to the performance data of the large language model and the timestamp information, so that the complexity of version management can be greatly simplified, and the capability of carrying out rapid and accurate rollback for specific errors is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a version management method, device, equipment and medium for a large language model. Background Art

[0002] With the widespread penetration of large language models in various industries, the demand for their application in the field of automotive finance is growing and becoming more sophisticated.

[0003] As an efficient and economical adaptation method, large model fine-tuning has become a popular application mode in this field by using basic models and combining them with data from specific fields for targeted training. However, in the special scenario of auto finance, large model fine-tuning still faces many challenges, especially the differences in version requirements caused by changes in time nodes and functional characteristics, which not only increases the complexity of version management, but also makes version rollback operations particularly difficult. Summary of the invention

[0004] The present invention provides a version management method, apparatus, device and medium for a large language model, which can greatly simplify the complexity of version management and improve the ability to quickly and accurately roll back specific errors.

[0005] According to one aspect of the present invention, a version management method for a large language model is provided, comprising:

[0006] Obtaining version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and region information; the metadata includes performance indicator data;

[0007] Determine a version rollback strategy of the large language model according to the timestamp information, the region information, and the performance indicator data;

[0008] A version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0009] According to another aspect of the present invention, there is provided a version management device for a large language model, comprising:

[0010] A data acquisition module, used to acquire version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and region information; and the metadata includes performance indicator data;

[0011] A strategy determination module, configured to determine a version rollback strategy of a large language model according to the timestamp information, the region information, and the performance indicator data;

[0012] The rollback operation execution module is used to execute a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the version management method for a large language model described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the version management method of a large language model described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention is to obtain the version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and regional information; the metadata includes performance indicator data; determine the version rollback strategy of the large language model according to the timestamp information, the regional information and the performance indicator data; perform a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model. This technical solution can determine the corresponding version rollback strategy based on the performance data and timestamp information of the large language model, thereby greatly simplifying the complexity of version management and improving the ability to quickly and accurately roll back specific errors.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1is a flowchart of a version management method for a large language model provided according to Embodiment 1 of the present invention;

[0022] Figure 2 is a flow chart of a version management method for a large language model provided according to Embodiment 2 of the present invention;

[0023] Figure 3 2 is a schematic diagram of the structure of a version management device for a large language model provided according to Embodiment 3 of the present invention;

[0024] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second" and "target" in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1 This is a flowchart of a version management method for a large language model provided according to the first embodiment of the present invention. This embodiment is applicable to the case of performing version management on a large language model. The method can be executed by a version management device for a large language model. The version management device for a large language model can be implemented in the form of hardware and / or software. The version management device for a large language model can be configured in an electronic device with data processing capabilities. Figure 1 As shown, the method includes:

[0029] S110. Obtain version information and metadata corresponding to the large language model training task in the target tenant.

[0030] Among them, the version information includes timestamp information and regional information; the metadata includes performance indicator data. The timestamp information may include the training start time, end time and deployment time of the version corresponding to each training task. The version information generally refers to the model version information used by the large language model training task. Metadata may refer to the data of the large language model training task, which may specifically include training data, algorithm parameters, performance indicators and other data. Regional information may refer to the geographical area or market information used by the large language model. It can be understood that in this embodiment, there are differences in business requirements for different regions, so the large language models corresponding to different regions are also different. A large language model refers to a deep learning model trained using a large amount of text data, so that the model can generate natural language text or understand the meaning of language text. The large language model in this embodiment refers to an advanced model that provides customized natural language processing services to tenants in a multi-tenant service environment.

[0031] The target tenant can be any tenant on the training platform. In this embodiment, the resource allocation management and permission management of each tenant are further refined to achieve efficient resource utilization and resource isolation between tenants. In this embodiment, an enhanced resource isolation method is designed for each tenant of the training platform.

[0032] Regarding the resource isolation part, in this embodiment, based on the powerful container orchestration and resource management capabilities of Kubernetes (k8s), targeted optimizations and innovations have been carried out. Specifically, in this embodiment, the establishment and library-level isolation of the tenant databases of the training platform are addressed. First, after a new tenant completes information entry through the registration system of the training platform, the platform will automatically trigger the database creation process and generate an independent database instance for each new tenant. These database instances achieve strict library-level isolation both physically and logically, ensuring the security of tenant data. Then, a Role-Based Access Control (RBAC) mechanism is introduced to further refine the permission management among tenants. According to the business requirements and responsibilities of the tenants, multiple roles such as administrators, developers, and testers are defined, and clear permission boundaries are assigned to each role. These permission boundaries are implemented through the RBAC mechanism of k8s, that is, fine-grained permission management is achieved by defining Role and RoleBinding. In terms of implementation details, by ensuring that the role definitions are detailed to specific API resources, operation types, and scopes, precise control of permissions is realized. At the same time, we also regularly audit the usage of permissions, promptly discover and correct any abnormal or abusive permission behaviors, and ensure that the operations among tenants do not interfere with each other. In terms of resource allocation and management, using the CustomResource Definition (CRD) function of k8s, a custom resource type "TenantResource" is defined. This innovative measure enables us to configure dedicated resource quotas for each "TenantResource" according to the specific needs of the tenants, including key resources such as CPU, memory, and storage. In terms of implementation details, the definition of "TenantResource" includes key information such as tenant identification, resource type, and resource quota to ensure the accuracy and reasonableness of resource allocation. At the same time, the configuration of resource quotas is also dynamically adjusted according to the business scale, growth trend, and peak demand of the tenants to meet the changing business needs of the tenants. In addition, appropriate scheduling strategies are also formulated, taking into account the availability of resources, the priorities of tenants, and affinity / anti-affinity rules to achieve efficient utilization of resources and isolation among tenants. Finally, at the network security level, high attention is also given. By selecting a k8s network plugin that supports virtual network isolation (such as Calico or Cilium), an independent virtual network is created for each tenant. These virtual networks are logically isolated from each other, effectively preventing the risks of network attacks and data leakage among tenants. In terms of implementation details, the creation process of the virtual network is carefully planned according to the business requirements and network topology of the tenants, and the configuration of network policies is ensured to be detailed to specific network ports, protocols, and IP address ranges. At the same time, network traffic and abnormal behaviors are also regularly monitored, and any potential network security threats are promptly discovered and handled to ensure the security and stability of the tenant's network environment.

[0033] In this implementation, the physical resources corresponding to each tenant are different, and the large language model will be deployed on the physical resources corresponding to each tenant. In this embodiment, the Annotations function of k8s can be used to record detailed version information and metadata for each training task, including training data, algorithm parameters, performance indicators, etc. Specifically, when using k8s Annotations to record version information and metadata, add the timestamp of each version, including the training start time, end time, and deployment time, which helps to track the performance of the model in different time periods and provide data support for time-based rollback strategies. Add a regional label to each version to indicate the geographic area or market to which the model is applicable, which helps to quickly locate when a model rollback is required for a specific region. Therefore, in this embodiment, when performing version management processing for different tenants, the detailed version information training data, algorithm parameters, performance indicators and other data corresponding to each training task of the large language model of the target tenant can be obtained.

[0034] S120: Determine a version rollback strategy for the large language model according to the timestamp information, the region information, and the performance indicator data.

[0035] Among them, the version rollback strategy can be considered as the most appropriate model version determined when a large language model needs to be rolled back.

[0036] In this embodiment, the model performance data of each version of the large language model can be analyzed according to the timestamp information, regional information and performance indicator data, and the model version with the best model performance or the least problems can be selected as the rollback target; if there are multiple candidate versions, the best rollback version can also be determined through further testing or performance evaluation to obtain a version rollback strategy for the large language model.

[0037] In this embodiment, when a large language model needs to be rolled back, the system will not only locate the most appropriate rollback version based on timestamp information, regional information, and performance indicator data, but will also automatically analyze the cause of the error and generate a rollback recommendation report for user reference. In addition, the concept of version chain is introduced in this embodiment, allowing users to select a specific version chain for rollback when rolling back, thereby avoiding the impact of erroneous data or algorithms on subsequent versions. In special business scenarios, such as when updating the credit assessment model, a rollback strategy based on a time window is also designed to ensure that the model is rolled back smoothly without affecting business continuity.

[0038] S130: Perform a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0039] In this embodiment, after determining the best model version in the version rollback strategy, the large language model service of the current version of the large language model can be stopped first to avoid data conflicts or inconsistencies. Then, based on performing a version rollback operation on the large language model, the large language model is restored to the specified best model version, and it is verified whether the model files and data after the rollback are complete and correct, thereby completing a smooth rollback of the large language model to ensure the model performance of the large language model without affecting business continuity.

[0040] The technical solution of the embodiment of the present invention obtains the version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and regional information; the metadata includes performance indicator data; the version rollback strategy of the large language model is determined according to the timestamp information, regional information and performance indicator data; and the version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model. This technical solution can determine the corresponding version rollback strategy according to the performance data and timestamp information of the large language model, thereby greatly simplifying the complexity of version management and improving the ability to quickly and accurately roll back specific errors.

[0041] Embodiment 2

[0042] Figure 2 This is a flowchart of a version management method for a large language model provided according to the second embodiment of the present invention. This embodiment is optimized based on the above embodiment. The specific optimization is: determining the version rollback strategy of the large language model according to the timestamp information, regional information and performance indicator data, including: determining the target performance data of each version in the large language model according to the timestamp information and performance indicator data; determining the version rollback strategy of the large language model based on the target performance data and / or regional information of each version. Figure 2 As shown, the method includes:

[0043] S210. Obtain version information and metadata corresponding to the large language model training task in the target tenant.

[0044] The version information includes timestamp information and region information; the metadata includes performance indicator data.

[0045] S220: Determine target performance data for each version in the large language model according to the timestamp information and the performance indicator data.

[0046] Among them, the target performance data can be considered as the specified performance data corresponding to the large language model in each version. For example, the target performance data can be performance data such as business continuity and credit assessment accuracy. In this embodiment, the corresponding target performance data can be set according to the actual needs of the user.

[0047] In this embodiment, the credit evaluation results of the large language model in different versions can be evaluated through performance indicator data, and the changes in different credit evaluation results within the time window can be determined in combination with timestamp information, so as to determine the impact of rolling back to different versions on the model performance data.

[0048] In this embodiment, optionally, target performance data of each version in the large language model is determined based on timestamp information and performance indicator data, including: evaluating credit assessment impact results of the large language model in each version based on the performance indicator data; and determining the target performance data of the large language model based on the credit assessment impact results and timestamp information.

[0049] Among them, the credit assessment impact result may refer to the expected change result of the credit assessment results corresponding to different versions. In this embodiment, the credit assessment impact result can be determined by the performance indicator data and the machine learning algorithm prediction model to determine the credit assessment impact results of different versions. In this embodiment, the performance indicator data corresponding to each version and the machine learning algorithm can be used to obtain the credit assessment impact results of each version, and then the changes in the credit assessment impact results of the large language model in each time period are analyzed in combination with the timestamp information, and then the target performance data of each version of the large language model is determined according to the results of time series analysis and version comparison.

[0050] In this embodiment, through such a setting, the target performance data of each version of the large language model can be analyzed in combination with the timestamp information and the performance indicator information, so as to determine the best rollback version of the large language model through analysis and comparison.

[0051] In this embodiment, optionally, the performance indicator data includes accuracy indicator data, recall indicator data and reconciliation indicator data; accordingly, the credit assessment impact results of the large language model in each version are evaluated according to the performance indicator data, including: inputting the accuracy indicator data, recall indicator data and reconciliation indicator data corresponding to each version into the credit assessment prediction model to obtain the credit assessment impact results of each version.

[0052] Among them, the accuracy index data can be considered as the accuracy data of the large language model, which can be used as an indicator to evaluate the accuracy of the model processing results. The recall index data can be considered as the recall data of the large language model, which can also be called the recall rate, which is the proportion of instances correctly identified as positive classes (true classes) by the model to all actual positive class instances. The reconciliation index data can refer to the F1 score, which is expressed as the reconciled mean of the accuracy and recall rates. The reconciliation index data takes both indicators into account to reduce the misleading nature of a single indicator.

[0053] In this embodiment, the metadata includes key performance indicator data of credit assessment, such as accuracy, recall rate, F1 score and other indicator data, as well as the changing trends of these indicators over time, so as to evaluate the performance of the large language model in the credit assessment task and provide a basis for rollback decisions.

[0054] In this embodiment, the accuracy, recall rate, F1 score and other indicator data corresponding to each version can be input into the trained credit assessment prediction model to obtain the credit assessment impact results corresponding to each version.

[0055] Furthermore, in this embodiment, a rollback strategy based on a time window is designed for the credit assessment prediction model. For example, before the business peak or within a specific time period, the system automatically evaluates the performance of the current large language model and decides whether to roll back based on a preset threshold. When performing a rollback operation, the most recent version that performs stably within the time window and has good credit assessment performance can be selected for deployment.

[0056] Through such a setting in this embodiment, historical data and machine learning algorithms can be used to predict the expected changes in credit assessment results after rolling back to different versions, helping to make more informed rollback decisions and improving the reliability of rollback operations.

[0057] S230: Determine a version rollback strategy for the large language model based on target performance data and / or regional information of each version.

[0058] In this embodiment, in the version chain, not only the linear relationship of the versions is recorded, but also the regional applicability and target performance data of each version are marked. The version rollback strategy of the large language model can be determined based on the target performance data of each version, or regional information, or target performance data and regional information. In this embodiment, the version rollback strategy can include the model version to be rolled back and the specific rollback method.

[0059] Specifically, in this embodiment, when performing a rollback operation, the version chain that has the least impact on a specific region or credit assessment performance can be selected for rollback. In the process of determining the version rollback strategy based on regional information, if an error or performance degradation only affects a specific region, the system recommends rolling back only the model in that region to maintain the stability and accuracy of models in other regions.

[0060] In this embodiment, optionally, a version rollback strategy for the large language model is determined based on the target performance data and / or regional information of each version, including: generating a rollback recommendation report for the large language model based on the target performance data and / or regional information of each version; and determining the version rollback strategy for the large language model according to the rollback recommendation report.

[0061] Among them, the rollback recommendation report may include multiple rollback versions, rollback methods and risk assessment components. In this embodiment, one or more historical versions can be selected as rollback targets based on target performance data and regional information, and then the specific method of rollback is determined according to actual needs, such as directly rolling back to a specified version, fine-tuning based on a specific version, etc. Direct rollback may involve the operation of the version control system, while fine-tuning requires adjustment and optimization based on the selected version. Then evaluate the risks that may be brought about when each rollback version is executed, such as performance degradation, data loss, etc. These risks should be identified and mitigated through sufficient testing. In this embodiment, a rollback recommendation report of a large language model is generated by combining the three aspects of rollback target, rollback method and risk assessment, and then the most suitable rollback version and rollback method are determined as the version rollback strategy based on the rollback recommendation report and the actual needs of the user.

[0062] In addition, when generating a rollback recommendation report in this embodiment, the system not only analyzes the cause of the error, but also evaluates the impact of rolling back to different versions on business continuity and credit assessment accuracy in combination with changes in credit assessment indicators within the time window.

[0063] In this embodiment, through such a setting, the most suitable version rollback strategy for the large language model is determined according to the target performance data and regional data of each version, which can ensure the stability and performance of the large language model in different tasks and regions.

[0064] S240: Perform a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0065] In this embodiment, optionally, a version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model, including: performing a version rollback operation within a first rollback range of the large language model based on the version rollback strategy to obtain model performance data after the version rollback; and continuing to perform the version rollback operation on the large language model based on the model performance data after the version rollback to complete the version rollback of the large language model.

[0066] The first rollback range may be a small range in the environment where the large language model is deployed. In this embodiment, the first rollback range may be set according to actual needs.

[0067] In this embodiment, for a large language model deployed on a large scale, a gradual rollback method can be used to roll back the large language model. A rollback test can be performed in a small range first to ensure that the performance of the model after the rollback is stable and meets expectations, and then the rollback range can be gradually expanded to complete the version rollback of the large language model. Exemplarily, a large language model deployed on a large scale can be considered as a currently deployed large model that has been a large-scale application. For example, dozens of services deployed on online servers are all using large language models.

[0068] In this embodiment, a version rollback operation can be performed within a preset first rollback range of the large language model according to a determined version rollback strategy, and performance data after the version rollback within the first rollback range can be obtained. It is determined whether the expected effect is met based on the model performance data of the large language model within the first rollback range after the rollback, and then the rollback range is gradually expanded to continue to perform the version rollback operation on the large language model, thereby gradually realizing the version rollback of the large language model.

[0069] Through such a setting in this embodiment, a large language model deployed on a large scale can be rolled back in a gradual rollback manner, further ensuring the performance and rollback effect of the large language model.

[0070] In this embodiment, optionally, the version rollback operation is continued on the large language model based on the model performance data after the version rollback to complete the version rollback of the large language model, including: if the model performance data after the version rollback meets the set conditions, the version rollback operation is gradually continued to the second rollback range until the version rollback operation is completed within the entire range of the large language model.

[0071] The second rollback range is larger than the first rollback range. The setting condition may be set for the model performance data, and may be pre-set. The setting condition in this embodiment may be that the model performance data meets the expected data of the model, and the specific expected data may be configured according to the actual large language model.

[0072] In this embodiment, for large-scale deployed models, a gradual rollback strategy is adopted. First, a rollback test is performed within the first rollback range to ensure that the model performance data after the version rollback meets the set submission, that is, the model performance after the version rollback is stable and meets expectations, and then the rollback range is gradually expanded, so as to gradually continue to perform the version rollback operation on the large language model within the second rollback range until the version rollback operation is completed within the entire range of the large language; if the model performance data after the version rollback does not meet the set conditions, the version rollback operation on the large language model can be stopped, and the version rollback strategy can be adjusted according to the model performance.

[0073] In this embodiment, during the rollback process of the large language model, the performance and credit evaluation indicators of the model are monitored in real time to promptly detect and handle any abnormalities. At the same time, user feedback is collected to continuously evaluate and optimize the rollback effect.

[0074] Through such a setting in this embodiment, a gradual rollback method can be adopted for a large language model deployed on a large scale, which ensures the flexibility and reliability of the rollback of the large language model and improves the stability of the performance of the large language model.

[0075] In this embodiment, the version rollback process is further improved and optimized by combining the timeliness, regionality and credit assessment attributes of automobile finance, which not only improves the flexibility and accuracy of rollback, but also ensures business continuity and the accuracy of credit assessment.

[0076] The technical solution of the embodiment of the present invention obtains the version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and regional information; the metadata includes performance indicator data; the target performance data of each version in the large language model is determined according to the timestamp information and the performance indicator data; the version rollback strategy of the large language model is determined based on the target performance data and / or regional information of each version. A version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model. The technical solution can determine the corresponding version rollback strategy based on the performance data and timestamp information of the large language model, thereby greatly simplifying the complexity of version management and improving the ability to quickly and accurately roll back specific errors.

[0077] Embodiment 3

[0078] Figure 3 1 is a schematic diagram of a structure of a version management device for a large language model according to Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0079] The data acquisition module 310 is used to obtain the version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and region information; the metadata includes performance indicator data;

[0080] A strategy determination module 320, configured to determine a version rollback strategy of a large language model according to timestamp information, region information, and performance indicator data;

[0081] The rollback operation execution module 330 is used to execute a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

[0082] Optionally, the policy determination module 320 includes:

[0083] A target performance data determination unit, used to determine target performance data of each version in the large language model according to the timestamp information and the performance indicator data;

[0084] The version rollback strategy determination unit is used to determine the version rollback strategy of the large language model based on the target performance data and / or regional information of each version.

[0085] Optionally, the target performance data determination unit includes:

[0086] The evaluation impact result subunit is used to evaluate the credit evaluation impact results of the large language model in each version based on the performance indicator data;

[0087] The performance data determination subunit is used to determine the target performance data of the large language model according to the credit assessment impact result and the timestamp information.

[0088] Optionally, the performance indicator data includes an accuracy indicator, a recall indicator, and a reconciliation indicator;

[0089] Accordingly, the evaluation impact results subunit is specifically used to:

[0090] The accuracy index, recall index and reconciliation index corresponding to each version are input into the credit assessment prediction model to obtain the credit assessment impact results of each version.

[0091] Optionally, the version rollback strategy determination unit is specifically used to:

[0092] Generate a rollback recommendation report for a large language model based on target performance data and / or regional information of each version;

[0093] Determine the version rollback strategy for the large language model based on the rollback recommendation report.

[0094] Optionally, the rollback operation execution module 330 includes:

[0095] A first-range rollback processing unit, configured to perform a version rollback operation within a first rollback range of the large language model based on a version rollback strategy, so as to obtain model performance data after the version rollback;

[0096] The rollback operation continuing execution unit is used to continue to execute the version rollback operation on the large language model based on the model performance data after the version rollback, so as to complete the version rollback of the large language model.

[0097] Optionally, the rollback operation continues to execute the unit, specifically for:

[0098] If the model performance data after the version rollback meets the set conditions, the version rollback operation is gradually continued to the second rollback range until the version rollback operation is completed within the entire range of the large language model; wherein the second rollback range is larger than the first rollback range.

[0099] The version management device for a large language model provided in the embodiment of the present invention can execute the version management method for a large language model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0100] Embodiment 4

[0101] Figure 4 1 is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0102] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0103] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0104] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a version management method for a large language model.

[0105] In some embodiments, the version management method of the large language model may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the version management method of the large language model described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the version management method of the large language model in any other appropriate manner (e.g., by means of firmware).

[0106] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0107] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0108] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0109] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0110] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0111] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0113] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A version management method for a large language model, characterized in that: include: Obtaining version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and region information; the metadata includes performance indicator data; Determine a version rollback strategy of the large language model according to the timestamp information, the region information, and the performance indicator data; A version rollback operation is performed on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

2. The method according to claim 1, characterized in that Determining a version rollback strategy of a large language model according to the timestamp information, the region information, and the performance indicator data includes: Determine target performance data of each version in the large language model according to the timestamp information and the performance indicator data; A version rollback strategy for the large language model is determined based on the target performance data of each version and / or the regional information.

3. The method according to claim 2, characterized in that Determining target performance data of each version in the large language model according to the timestamp information and the performance indicator data includes: Evaluate the credit assessment impact results of the large language model in each version according to the performance indicator data; Target performance data of the large language model is determined according to the credit assessment impact result and the timestamp information.

4. The method according to claim 3, characterized in that: The performance index data includes accuracy index data, recall index data and reconciliation index data; Accordingly, the credit assessment impact results of the large language model in each version are evaluated according to the performance indicator data, including: The accuracy index data, recall index data and reconciliation index data corresponding to each version are input into the credit assessment prediction model to obtain the credit assessment impact results of each version.

5. The method according to claim 2, characterized in that: Determining a version rollback strategy of the large language model based on the target performance data of each version and / or the regional information includes: Generate a rollback recommendation report for the large language model based on the target performance data of each version and / or the regional information; A version rollback strategy for the large language model is determined according to the rollback suggestion report.

6. The method according to claim 1, characterized in that Performing a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model includes: Performing a version rollback operation within a first rollback range of the large language model based on the version rollback strategy to obtain model performance data after the version rollback; The version rollback operation is continued to be performed on the large language model based on the model performance data after the version rollback, so as to complete the version rollback of the large language model.

7. The method according to claim 6, characterized in that Continuing to perform a version rollback operation on the large language model based on the model performance data after the version rollback to complete the version rollback of the large language model includes: If the model performance data after the version rollback meets the set conditions, the version rollback operation is gradually continued to the second rollback range until the version rollback operation is completed within the entire range of the large language model; wherein the second rollback range is larger than the first rollback range.

8. A version management device for a large language model, characterized in that: include: A data acquisition module, used to acquire version information and metadata corresponding to the large language model training task in the target tenant; wherein the version information includes timestamp information and region information; and the metadata includes performance indicator data; A strategy determination module, configured to determine a version rollback strategy of a large language model according to the timestamp information, the region information, and the performance indicator data; The rollback operation execution module is used to execute a version rollback operation on the large language model according to the version rollback strategy to complete the version rollback of the large language model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the version management method for a large language model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the version management method of a large language model according to any one of claims 1 to 7 when executed.

Images