Large language model deployment method and device, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a large language model deployment method and device, equipment and a storage medium, and the method comprises the steps: starting a parallel computing platform and a deep learning framework in a virtual environment; when the test result of the parallel computing platform and the test result of the deep learning framework are both test passing results, merging a plurality of sub-files of the large language model to obtain a model file of the large language model, and obtaining a recommended memory; and when the physical memory is smaller than the recommended memory, packaging the enlarging instruction of the virtual memory, the cache cleaning instruction of the physical memory and the optimization instruction of the model loading parameter to obtain an automatic script, executing the automatic script, and after the automatic script is executed, executing the model loading parameter. Obtaining a loading state identifier of the model file from a log returned by the deep learning framework; and when the loading state identifier is a loading success identifier, outputting a deployment success message of the large language model. The deployment efficiency of the large language model can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for deploying large language models. Background Technology

[0002] Large language models are complex neural network models built upon deep learning training on massive amounts of text data. Through in-depth mining and learning of the semantic and grammatical relationships between words, sentences, and paragraphs in text, large language models acquire rich linguistic knowledge and powerful language processing capabilities. They can simulate human language thought processes, achieving efficient processing and accurate responses to natural language. By deploying large language models, enterprises can systematically improve operational efficiency and strategic competitiveness.

[0003] However, the deployment process of existing large language models is cumbersome, which hinders the improvement of deployment efficiency. This is because current technologies primarily employ manual deployment, which consumes significant human and time resources, increases deployment time, and is susceptible to human intervention, thus hindering the improvement of deployment efficiency. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for deploying large language models, in order to solve the technical problem that the deployment process of existing large language models is cumbersome and not conducive to improving the deployment efficiency of large language models.

[0005] In a first aspect, embodiments of this application provide a method for deploying a large language model, applied to an electronic device, the method comprising: Obtain the installation files for the parallel computing platform and the deep learning framework. Perform the installation operation on the installation files for the parallel computing platform and the deep learning framework in the virtual environment. After the installation is complete, start the parallel computing platform and the deep learning framework in the virtual environment. The parallel computing platform was tested using the first test file, and the test results of the parallel computing platform were obtained. The deep learning framework was tested using the second test file, and the test results of the deep learning framework were obtained. When both the test results of the parallel computing platform and the test results of the deep learning framework are passed, the multiple sub-files of the large language model are merged to obtain the model file of the large language model, and the recommended memory is obtained. The recommended memory is the memory required by the model file when loading. Obtain physical memory from the memory management system. When physical memory is less than the recommended memory, encapsulate the instructions to increase virtual memory, clear physical memory cache, and optimize model loading parameters to obtain an automated script. Execute the automated script. After the automated script is executed, call the deep learning framework to load the model file and obtain the loading status identifier of the model file from the log returned by the deep learning framework. When the loading status is marked as successful, the inference task and the target result corresponding to the subway project are obtained. The inference task is decomposed into multiple parallel subtasks through the large language model. The calculation results of each parallel subtask are passed to the large language model through the parallel computing platform. The large language model processes the calculation results of each parallel subtask to obtain the inference result. When the inference result is the target result, the deployment success message of the large language model is output.

[0006] In one possible implementation of the first aspect, obtaining the installation files of the parallel computing platform and the deep learning framework, performing an installation operation on the installation files of the parallel computing platform and the deep learning framework in a virtual environment, and starting the parallel computing platform and the deep learning framework in the virtual environment after installation includes: Access the data platform and obtain the installation files for the parallel computing platform and the deep learning framework from it; Create a virtual environment and install the parallel computing platform and deep learning framework within it. Once installation is complete, start the parallel computing platform and deep learning framework in the virtual environment.

[0007] In one possible implementation of the first aspect, the step of testing the parallel computing platform using a first test file to obtain test results for the parallel computing platform, and testing the deep learning framework using a second test file to obtain test results for the deep learning framework, includes: Obtain startup test cases, dependency test cases, and parallel task test cases from the first test file. Test the parallel computing platform using these test cases to obtain the test results. Obtain import test cases and computation test cases from the second test file, and test the deep learning framework using the import test cases and computation test cases to obtain the test results of the parallel computing platform.

[0008] In one possible implementation of the first aspect, when both the test results of the parallel computing platform and the test results of the deep learning framework are pass results, multiple sub-files of the large language model are merged to obtain the model file of the large language model, and the recommended memory is obtained. The recommended memory is the memory required by the model file when loading, including: When both the test results of the parallel computing platform and the test results of the deep learning framework are pass results; Download multiple sub-files of the large language model from the specified repository, merge the multiple sub-files of the large language model to obtain the model file of the large language model, and obtain the recommended memory, which is the memory required by the model file when loading.

[0009] In one possible implementation of the first aspect, physical memory is obtained from the memory management system. When the physical memory is less than the recommended memory, the instructions for increasing virtual memory, clearing physical memory cache, and optimizing model loading parameters are encapsulated to obtain an automated script. The automated script is executed, and after its execution, the deep learning framework is invoked to load the model file. The loading status identifier of the model file is obtained from the logs returned by the deep learning framework, including: Obtain physical memory from the memory management system. When the physical memory is less than the recommended memory for the model file, obtain the virtual memory and model loading parameters corresponding to the physical memory. The instructions for increasing virtual memory, clearing physical memory cache, and optimizing model loading parameters are encapsulated to obtain an automated script. The instructions for optimizing model loading parameters include instructions for optimizing the number of CPU threads and instructions for optimizing quantization. Execute the automated script. After the automated script has finished executing, call the deep learning framework to load the model file and obtain the loading status identifier of the model file from the log returned by the deep learning framework.

[0010] In one possible implementation of the first aspect, when the loading status identifier is a loading success identifier, the inference task of the subway project and the target result corresponding to the inference task are obtained. The inference task is decomposed into multiple parallel subtasks through a large language model. The computation result of each parallel subtask is passed to the large language model through a parallel computing platform. The large language model processes the computation result of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message of the large language model is output, including: When the loading status is marked as successful, configure the access interface of the large language model to obtain the inference task of the subway project and the target result corresponding to the inference task. The system submits inference tasks to the large language model via an access interface. The large language model then breaks down the inference tasks into multiple parallel subtasks. The large language model transmits these multiple parallel subtasks to a parallel computing platform. The parallel computing platform then transmits the computation results of each parallel subtask to the large language model. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message for the large language model is output.

[0011] In one possible implementation of the first aspect, when the loading status identifier is a loading success identifier, the inference task of the subway project and the target result corresponding to the inference task are obtained. The inference task is decomposed into multiple parallel subtasks through a large language model. The computation result of each parallel subtask is passed to the large language model through a parallel computing platform. The large language model processes the computation result of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message of the large language model is output. The large language model deployment method includes: Create a display window to show a message indicating successful deployment of the large language model.

[0012] Secondly, embodiments of this application provide a large language model deployment apparatus, applied to an electronic device, comprising: The acquisition module is used to acquire the installation files of the parallel computing platform and the deep learning framework, perform the installation operation on the installation files of the parallel computing platform and the deep learning framework in the virtual environment, and start the parallel computing platform and the deep learning framework in the virtual environment after the installation is completed. The testing module is used to test the parallel computing platform using the first test file and obtain the test results of the parallel computing platform, and to test the deep learning framework using the second test file and obtain the test results of the deep learning framework. The merging module is used to merge multiple sub-files of the large language model to obtain the model file of the large language model when both the test results of the parallel computing platform and the test results of the deep learning framework are passed. The recommended memory is the memory required by the model file when loading. The execution module is used to obtain physical memory from the memory management system. When the physical memory is less than the recommended memory, it encapsulates the instructions to increase virtual memory, clear physical memory cache, and optimize model loading parameters to obtain an automated script. The automated script is executed, and after the automated script is completed, the deep learning framework is called to load the model file. The loading status identifier of the model file is obtained from the log returned by the deep learning framework. The deployment module is used to obtain the inference task and the target result corresponding to the inference task of the subway project when the loading status is marked as successful. The inference task is decomposed into multiple parallel subtasks through the large language model. The calculation result of each parallel subtask is passed to the large language model through the parallel computing platform. The large language model processes the calculation result of each parallel subtask to obtain the inference result. When the inference result is the target result, the deployment success message of the large language model is output.

[0013] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large language model deployment method described in the first aspect above.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the large language model deployment method described in the first aspect above.

[0015] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the large language model deployment method described in the first aspect above.

[0016] The beneficial effects of this application embodiment are twofold. Firstly, when the loading status is marked as successful, the inference task and the corresponding target result of the subway project are obtained. The inference task is decomposed into multiple parallel subtasks using a large language model. The computation results of each parallel subtask are transmitted to the large language model through a parallel computing platform. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message for the large language model is output. Since no manual deployment is required, the deployment time of the large language model is reduced, which is beneficial to improving the deployment efficiency of the large language model. Secondly, it is not affected by manual intervention, which is beneficial to improving the reliability of the large language model. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 An application scenario diagram of the large language model deployment method provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the large language model deployment method provided in an embodiment of this application; Figure 3 A flowchart of S204 provided in the embodiments of this application; Figure 4 A schematic block diagram of a large language model deployment apparatus provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0020] The large language model deployment method provided in this application can be applied to electronic devices such as servers, mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0021] Please see Figure 1 , Figure 1 The application scenario diagram of the large language model deployment method provided in the embodiments of this application is described in detail below: Electronic devices access the data platform to obtain the installation files of the parallel computing platform and the deep learning framework from the data platform; For ease of explanation, details are provided below: Electronic devices access the data platform via wired or wireless networks to obtain the installation files for the parallel computing platform and the deep learning framework.

[0022] In this embodiment, the data platform integrates a rich variety of deep learning framework resources. Users can quickly obtain the installation files of the parallel computing platform and the deep learning framework by simply searching within the data platform, which greatly saves time.

[0023] Please see Figure 2 , Figure 2 This is a flowchart illustrating the large language model deployment method provided in this application embodiment, which can be applied to electronic devices.

[0024] like Figure 2 As shown in the embodiments of this application, the method for deploying a large language model includes the following steps, which are detailed below: S201: Obtain the installation files of the parallel computing platform and the deep learning framework. Perform the installation operation on the installation files of the parallel computing platform and the deep learning framework in the virtual environment. After the installation is completed, start the parallel computing platform and the deep learning framework in the virtual environment. The steps include obtaining the installation files of the parallel computing platform and the deep learning framework, performing the installation operation on the installation files of the parallel computing platform and the deep learning framework in a virtual environment, and starting the parallel computing platform and the deep learning framework in the virtual environment after installation. Access the data platform and obtain the installation files for the parallel computing platform and the deep learning framework from it; Create a virtual environment and install the parallel computing platform and deep learning framework within it. Once installation is complete, start the parallel computing platform and deep learning framework in the virtual environment.

[0025] For example, a virtual environment is created, and installation operations are performed on the installation files of the parallel computing platform and the deep learning framework within the virtual environment, including: Create a virtual environment using the environment manager, and then perform the installation operations on the installation files of the parallel computing platform and the deep learning framework within the virtual environment.

[0026] S202, the parallel computing platform is tested using the first test file to obtain the test results of the parallel computing platform, and the deep learning framework is tested using the second test file to obtain the test results of the deep learning framework. The test involves using a first test file to test the parallel computing platform and obtaining test results for the parallel computing platform. A second test file is used to test the deep learning framework and obtain test results for the deep learning framework, including: Obtain startup test cases, dependency test cases, and parallel task test cases from the first test file. Test the parallel computing platform using these test cases to obtain the test results. Obtain import test cases and computation test cases from the second test file, and test the deep learning framework using the import test cases and computation test cases to obtain the test results of the parallel computing platform.

[0027] For example, the test results of the parallel computing platform include the test results of the startup test cases, the test results of the dependency test cases, and the test results of the parallel task test cases.

[0028] For example, the test results of a deep learning framework include the test results of test cases and the test results of computation test cases.

[0029] Start the test case: Check if the platform service process is running normally on all nodes; In software development and system operation, a program or function often depends on other components, libraries, software packages, etc. to function properly. These dependent parts are called dependencies.

[0030] Dependency test cases: Test plans to check whether the status of dependencies meets the operational requirements of the parallel computing platform.

[0031] Parallel task test cases: Parallel task test cases are test plans designed to assess the parallel processing capabilities of a parallel computing platform. For ease of explanation, an example of a parallel task test case is provided below: For example, consider a scenario where four computing nodes simultaneously participate in a task to sum integers up to 10. The task is split so that each node handles a portion of the calculation. For instance, node 1 calculates the sum of numbers 1 to 3, node 2 calculates the sum of numbers 4 to 5, node 3 calculates the sum of numbers 6 to 8, and node 4 calculates the sum of numbers 9 to 10. If, during testing, each of the four nodes can complete its assigned task, smoothly transmit sub-results, and the master node sums the result to 45, then the node collaboration mechanism of the parallel computing system is effective, and the data transmission and result summarization functions are normal. Conversely, if node calculation errors, communication interruptions, or summarization results deviating from 45 occur, it indicates a problem in the parallel computing platform's task allocation, node collaboration, or data synchronization, requiring further investigation and optimization.

[0032] Importing test cases: In the testing process of software development or deep learning frameworks, specific code instructions are executed to verify whether the core modules of the deep learning framework can be successfully loaded by the current environment without import errors or missing dependency prompts.

[0033] Computational test cases: Run basic mathematical operations to verify whether the deep learning framework can execute the computational logic normally, and whether the numerical values ​​and data types of the output results meet expectations.

[0034] S203. When both the test results of the parallel computing platform and the test results of the deep learning framework are passed, the multiple sub-files of the large language model are merged to obtain the model file of the large language model, and the recommended memory is obtained. The recommended memory is the memory required by the model file when loading. When both the test results for the parallel computing platform and the test results for the deep learning framework are "passed", it means that the test results for the startup test case, the dependency test case, the parallel task test case, the test case, and the computation test case are all "passed".

[0035] Specifically, when both the test results of the parallel computing platform and the test results of the deep learning framework are passed, the multiple sub-files of the large language model are merged to obtain the model file of the large language model. The recommended memory is then obtained; the recommended memory is the memory required by the model file during loading, including: When both the test results of the parallel computing platform and the test results of the deep learning framework are pass results; Download multiple sub-files of the large language model from the specified repository, merge the multiple sub-files of the large language model to obtain the model file of the large language model, and obtain the recommended memory, which is the memory required by the model file when loading.

[0036] Optionally, the sub-files can be in GGUF format. GGUF (General GPU Unified File Format) is a general-purpose file storage format designed specifically for machine learning models, aiming to solve efficiency and compatibility challenges in model deployment.

[0037] Knowing in advance the memory required for loading the model file allows for accurate assessment of whether the device meets the operational requirements, ensuring that the model file can function stably and efficiently in real-world scenarios.

[0038] S204: Obtain physical memory from the memory management system. When the physical memory is less than the recommended memory, encapsulate the instructions to increase virtual memory, clear physical memory cache, and optimize model loading parameters to obtain an automated script. Execute the automated script. After the automated script is executed, call the deep learning framework to load the model file and obtain the loading status identifier of the model file from the log returned by the deep learning framework. Increasing the virtual memory size allows some infrequently used data to be temporarily stored in virtual memory on the disk when physical memory is exhausted, thus freeing up more storage space for loading model files and ensuring that the loading process can continue.

[0039] The physical memory cache may contain a large amount of data irrelevant to the current model loading. This redundant data consumes physical memory storage space, reducing the available physical memory. Clearing the cache frees up more clean storage space for model loading, avoiding loading failures or frequent disk swapping caused by insufficient physical memory, thereby improving the stability and speed of model loading. Furthermore, optimizing model loading parameters can further accelerate the model file loading process, ensuring that the model file can quickly and accurately reach a usable state.

[0040] S205: When the loading status is marked as successful, obtain the reasoning task and the target result corresponding to the subway project. Decompose the reasoning task into multiple parallel subtasks through the large language model. Pass the calculation result of each parallel subtask to the large language model through the parallel computing platform. Process the calculation result of each parallel subtask through the large language model to obtain the reasoning result. When the reasoning result is the target result, output the deployment success message of the large language model.

[0041] Subway projects involve numerous complex reasoning tasks. From the planning stage, it is necessary to deduce the optimal route and station layout based on information such as the city's population distribution, functional zoning, and traffic flow forecasts. For example, planning routes between densely populated commercial and residential areas to meet the high demand for commuting and travel. During construction, appropriate construction methods must be selected based on factors such as geological conditions, engineering feasibility, and cost budget; for instance, tunnel boring machines may be more suitable for soft soil layers.

[0042] For ease of explanation, the following example is provided: For example, the reasoning task for the subway project is: The construction site of a subway tunnel is 150 meters long and 60 meters wide. It needs to accommodate a shield machine operator's room, a segment stacking area, a spoil transport channel, and a temporary power distribution room. Please deduce a standardized layout plan.

[0043] The reasoning result is: The tunnel boring machine operator's cab is located on the extended line of the tunnel axis, 30 meters from the tunnel entrance; Segment stacking area: adjacent to the east side of the control room, stacked in layers according to model, with a height of ≤3 meters; Construction waste transportation channel: arranged along the west side of the site, 6 meters wide, with a slope of ≤5%; Temporary power distribution room: located in the northeast corner of the site, at least 15 meters away from flammable areas; Safety measures: Reflective signs are installed on both sides of the passageway, and fire extinguishers are provided in the power distribution room.

[0044] When the inference result is the target result, output a message indicating successful deployment of the large language model.

[0045] Specifically, when the inference result is not the target result, a message indicating that the deployment of the large language model was unsuccessful is output.

[0046] S205 includes: When the loading status is marked as successful, configure the access interface of the large language model to obtain the inference task of the subway project and the target result corresponding to the inference task. The system submits inference tasks to the large language model via an access interface. The large language model then breaks down the inference tasks into multiple parallel subtasks. The large language model transmits these multiple parallel subtasks to a parallel computing platform. The parallel computing platform then transmits the computation results of each parallel subtask to the large language model. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message for the large language model is output.

[0047] The method for deploying the large language model, following S205, includes: Create a display window to show a message indicating successful deployment of the large language model.

[0048] The beneficial effects of this application embodiment are twofold. Firstly, when the loading status is marked as successful, the inference task and the corresponding target result of the subway project are obtained. The inference task is decomposed into multiple parallel subtasks using a large language model. The computation results of each parallel subtask are transmitted to the large language model through a parallel computing platform. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message for the large language model is output. Since no manual deployment is required, the deployment time of the large language model is reduced, which is beneficial to improving the deployment efficiency of the large language model. Secondly, it is not affected by manual intervention, which is beneficial to improving the reliability of the large language model.

[0049] Please see Figure 3 , Figure 3 The flowchart of S204 provided in the embodiments of this application is described in detail below: S301 retrieves physical memory from the memory management system. When the physical memory is less than the recommended memory for the model file, it retrieves the virtual memory and model loading parameters corresponding to the physical memory. S302 encapsulates instructions for increasing virtual memory, clearing physical memory cache, and optimizing model loading parameters to obtain an automated script. The optimization instructions for model loading parameters include CPU thread count optimization instructions and quantization level optimization instructions. Among them, the number of CPU threads refers to the number of threads that the CPU loads in parallel.

[0050] Among them, the CPU thread number optimization instructions are instructions that reduce the number of CPU threads loading in parallel.

[0051] Among them, quantization-level optimization instructions are hardware or software instructions used to accelerate low-precision calculations. The core goal is to significantly improve computing efficiency, reduce memory bandwidth requirements, and reduce power consumption by converting high-precision numbers into low-precision formats and taking advantage of the hardware's special support for low-precision operations.

[0052] S303 executes the automation script. After the automation script is completed, it calls the deep learning framework to load the model file and obtains the loading status identifier of the model file from the log returned by the deep learning framework.

[0053] In this embodiment, the instructions for increasing virtual memory, clearing physical memory cache, and optimizing model loading parameters are encapsulated to obtain an automated script. Executing the automated script can quickly complete the operations of increasing virtual memory, clearing physical memory cache, and optimizing model loading parameters, saving a significant amount of manpower and time.

[0054] For the large language model deployment method described in the above embodiments, please refer to [link / reference]. Figure 4 , Figure 4 A schematic block diagram of a large language model deployment apparatus provided in an embodiment of this application. Figure 4 The large language model deployment device 400 shown can be applied to, for example... Figure 1 The application scenario diagram shows electronic devices. The following section uses electronic devices as an example to illustrate this. Figure 4 The large language model deployment device 400 shown will be described in detail. The large language model deployment device 400 may include an acquisition module 401, a testing module 402, a merging module 403, an execution module 404, and a deployment module 405.

[0055] The module 401 is used to obtain the installation files of the parallel computing platform and the deep learning framework, perform the installation operation on the installation files of the parallel computing platform and the deep learning framework in the virtual environment, and start the parallel computing platform and the deep learning framework in the virtual environment after the installation is completed. Test module 402 is used to test the parallel computing platform using the first test file and obtain the test results of the parallel computing platform, and to test the deep learning framework using the second test file and obtain the test results of the deep learning framework. The merging module 403 is used to merge multiple sub-files of the large language model to obtain the model file of the large language model when both the test results of the parallel computing platform and the test results of the deep learning framework are passing results, and to obtain the recommended memory, which is the memory required by the model file when loading. Execution module 404 is used to obtain physical memory from the memory management system. When the physical memory is less than the recommended memory, it encapsulates the instructions to increase virtual memory, clear physical memory cache, and optimize model loading parameters to obtain an automated script. The automated script is executed, and after the automated script is executed, the deep learning framework is called to load the model file. The loading status identifier of the model file is obtained from the log returned by the deep learning framework. The deployment module 405 is used to obtain the reasoning task and the target result corresponding to the reasoning task of the subway project when the loading status is marked as successful. The reasoning task is decomposed into multiple parallel subtasks through the large language model. The calculation result of each parallel subtask is passed to the large language model through the parallel computing platform. The large language model processes the calculation result of each parallel subtask to obtain the reasoning result. When the reasoning result is the target result, the deployment success message of the large language model is output.

[0056] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0057] The beneficial effects of this application embodiment are twofold. Firstly, when the loading status is marked as successful, the inference task and the corresponding target result of the subway project are obtained. The inference task is decomposed into multiple parallel subtasks using a large language model. The computation results of each parallel subtask are transmitted to the large language model through a parallel computing platform. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message for the large language model is output. Since no manual deployment is required, the deployment time of the large language model is reduced, which is beneficial to improving the deployment efficiency of the large language model. Secondly, it is not affected by manual intervention, which is beneficial to improving the reliability of the large language model.

[0058] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0059] like Figure 5 As shown, Figure 5 The electronic device 2 includes: at least one processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the at least one processor 20, wherein the processor 20 executes the computer program 22 to implement the steps in any of the above method embodiments.

[0060] The electronic device 2 may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 2 and does not constitute a limitation on electronic device 2. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0061] The processor 20 is used to run a computer program 22 stored in the memory 21, and performs the following steps when executing the computer program 22: Obtain the installation files for the parallel computing platform and the deep learning framework. Perform the installation operation on the installation files for the parallel computing platform and the deep learning framework in the virtual environment. After the installation is complete, start the parallel computing platform and the deep learning framework in the virtual environment. The parallel computing platform was tested using the first test file, and the test results of the parallel computing platform were obtained. The deep learning framework was tested using the second test file, and the test results of the deep learning framework were obtained. When both the test results of the parallel computing platform and the test results of the deep learning framework are passed, the multiple sub-files of the large language model are merged to obtain the model file of the large language model, and the recommended memory is obtained. The recommended memory is the memory required by the model file when loading. Obtain physical memory from the memory management system. When physical memory is less than the recommended memory, encapsulate the instructions to increase virtual memory, clear physical memory cache, and optimize model loading parameters to obtain an automated script. Execute the automated script. After the automated script is executed, call the deep learning framework to load the model file and obtain the loading status identifier of the model file from the log returned by the deep learning framework. When the loading status is marked as successful, the inference task and the target result corresponding to the subway project are obtained. The inference task is decomposed into multiple parallel subtasks through the large language model. The calculation results of each parallel subtask are passed to the large language model through the parallel computing platform. The large language model processes the calculation results of each parallel subtask to obtain the inference result. When the inference result is the target result, the deployment success message of the large language model is output.

[0062] The processor 20 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0063] In some embodiments, the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or memory of the electronic device 2. In other embodiments, the memory 21 may be an external storage device of the electronic device 2, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the electronic device 2.

[0064] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0065] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0066] The computer-readable storage medium stores program code that can be called by a processor to execute the large language model deployment method described in the above method embodiments.

[0067] Since the computer program stored in the computer-readable storage medium can execute any of the large language model deployment methods provided in the embodiments of this application, the computer-readable storage medium can achieve the beneficial effects that any of the large language model deployment methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0068] This application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the aforementioned large language model deployment method.

[0069] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0070] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for deploying a large language model, characterized in that, The large language model deployment method, applied to electronic devices, includes: Obtain the installation files for the parallel computing platform and the deep learning framework. Perform the installation operation on the installation files for the parallel computing platform and the deep learning framework in the virtual environment. After the installation is complete, start the parallel computing platform and the deep learning framework in the virtual environment. The parallel computing platform was tested using the first test file, and the test results of the parallel computing platform were obtained. The deep learning framework was tested using the second test file, and the test results of the deep learning framework were obtained. When both the test results of the parallel computing platform and the test results of the deep learning framework are passed, the multiple sub-files of the large language model are merged to obtain the model file of the large language model, and the recommended memory is obtained. The recommended memory is the memory required by the model file when loading. Obtain physical memory from the memory management system. When physical memory is less than the recommended memory, encapsulate the instructions to increase virtual memory, clear physical memory cache, and optimize model loading parameters to obtain an automated script. Execute the automated script. After the automated script is executed, call the deep learning framework to load the model file and obtain the loading status identifier of the model file from the log returned by the deep learning framework. When the loading status is marked as successful, the inference task and the target result corresponding to the subway project are obtained. The inference task is decomposed into multiple parallel subtasks through the large language model. The calculation results of each parallel subtask are passed to the large language model through the parallel computing platform. The large language model processes the calculation results of each parallel subtask to obtain the inference result. When the inference result is the target result, the deployment success message of the large language model is output.

2. The method for deploying a large language model according to claim 1, characterized in that, The process of obtaining the installation files for the parallel computing platform and the deep learning framework, performing the installation operation on the installation files for the parallel computing platform and the deep learning framework in a virtual environment, and then starting the parallel computing platform and the deep learning framework in the virtual environment after installation includes: Access the data platform and obtain the installation files for the parallel computing platform and the deep learning framework from it; Create a virtual environment, and in the virtual environment, perform the installation operations on the installation files of the parallel computing platform and the deep learning framework. After the installation is complete, start the parallel computing platform and the deep learning framework in the virtual environment.

3. The method for deploying a large language model according to claim 1, characterized in that, The test results for the parallel computing platform are obtained by using a first test file and the deep learning framework are obtained by using a second test file. This includes: Obtain startup test cases, dependency test cases, and parallel task test cases from the first test file. Test the parallel computing platform using these test cases to obtain the test results. Obtain import test cases and computation test cases from the second test file, and test the deep learning framework using the import test cases and computation test cases to obtain the test results of the parallel computing platform.

4. The method for deploying a large language model according to claim 1, characterized in that, When both the test results of the parallel computing platform and the test results of the deep learning framework are passed, the multiple sub-files of the large language model are merged to obtain the model file of the large language model. The recommended memory is then obtained; the recommended memory is the memory required for the model file to be loaded, including: When both the test results of the parallel computing platform and the test results of the deep learning framework are pass results; Download multiple sub-files of the large language model from the specified repository, merge the multiple sub-files of the large language model to obtain the model file of the large language model, and obtain the recommended memory, which is the memory required by the model file when loading.

5. The method for deploying a large language model according to claim 1, characterized in that, The process involves retrieving physical memory from the memory management system. When physical memory is less than the recommended memory, the instructions for increasing virtual memory, clearing physical memory cache, and optimizing model loading parameters are encapsulated into an automated script. This automated script is then executed. After execution, the deep learning framework is invoked to load the model file. The loading status identifier of the model file is retrieved from the logs returned by the deep learning framework, including: Obtain physical memory from the memory management system. When the physical memory is less than the recommended memory for the model file, obtain the virtual memory and model loading parameters corresponding to the physical memory. The instructions for increasing virtual memory, clearing physical memory cache, and optimizing model loading parameters are encapsulated to obtain an automated script. The instructions for optimizing model loading parameters include instructions for optimizing the number of CPU threads and instructions for optimizing quantization. Execute the automated script. After the automated script has finished executing, call the deep learning framework to load the model file and obtain the loading status identifier of the model file from the log returned by the deep learning framework.

6. The method for deploying a large language model according to claim 1, characterized in that, When the loading status is marked as successful, the inference task and its corresponding target result for the subway project are obtained. The inference task is then decomposed into multiple parallel subtasks using a large language model. The computation results of each parallel subtask are transmitted to the large language model via a parallel computing platform. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result matches the target result, a deployment success message for the large language model is output, including: When the loading status is marked as successful, configure the access interface of the large language model to obtain the inference task of the subway project and the target result corresponding to the inference task. The system submits inference tasks to the large language model via an access interface. The large language model then breaks down the inference tasks into multiple parallel subtasks. The large language model transmits these multiple parallel subtasks to a parallel computing platform. The parallel computing platform then transmits the computation results of each parallel subtask to the large language model. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message for the large language model is output.

7. The method for deploying a large language model according to claim 1, characterized in that, When the loading status is marked as successful, the inference task and its corresponding target result for the subway project are obtained. The inference task is then decomposed into multiple parallel subtasks using a large language model. The computation results of each parallel subtask are transmitted to the large language model via a parallel computing platform. The large language model processes the computation results of each parallel subtask to obtain the inference result. When the inference result is the target result, a deployment success message for the large language model is output. The large language model deployment method includes: Create a display window to show a message indicating successful deployment of the large language model.

8. A large language model deployment device, characterized in that, Applied to electronic devices, including: The acquisition module is used to acquire the installation files of the parallel computing platform and the deep learning framework, perform the installation operation on the installation files of the parallel computing platform and the deep learning framework in the virtual environment, and start the parallel computing platform and the deep learning framework in the virtual environment after the installation is completed. The testing module is used to test the parallel computing platform using the first test file and obtain the test results of the parallel computing platform, and to test the deep learning framework using the second test file and obtain the test results of the deep learning framework. The merging module is used to merge multiple sub-files of the large language model to obtain the model file of the large language model when both the test results of the parallel computing platform and the test results of the deep learning framework are passed. The recommended memory is the memory required by the model file when loading. The execution module is used to obtain physical memory from the memory management system. When the physical memory is less than the recommended memory, it encapsulates the instructions to increase virtual memory, clear physical memory cache, and optimize model loading parameters to obtain an automated script. The automated script is executed, and after the automated script is completed, the deep learning framework is called to load the model file. The loading status identifier of the model file is obtained from the log returned by the deep learning framework. The deployment module is used to obtain the inference task and the target result corresponding to the inference task of the subway project when the loading status is marked as successful. The inference task is decomposed into multiple parallel subtasks through the large language model. The calculation result of each parallel subtask is passed to the large language model through the parallel computing platform. The large language model processes the calculation result of each parallel subtask to obtain the inference result. When the inference result is the target result, the deployment success message of the large language model is output.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the large language model deployment method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the large language model deployment method as described in any one of claims 1 to 7.

Images