Large model service starting method and device, equipment and medium

By loading large models in advance to shared memory and mapping the memory in the target process, the problem of slow starting speed of large model services has been solved, and rapid startup and high stability services are realized.

CN119376971APending Publication Date: 2025-01-28BAIDU (CHINA) CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411297493.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2025-01-28

AI Technical Summary

Technical Problem

In the prior art, large-model services cannot be started quickly when the number of user visits increases, resulting in a decline in service stability and user experience.

Method used

By loading large models in advance to sharing memory, and when creating the target process, the shared memory is mapped into the target process, and the large model in the shared memory is directly called to provide services.

Benefits of technology

The rapid startup of large model services has been achieved, the service stability and user experience are improved, and the model initialization time of the model at the time of each process is avoided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119376971A_ABST
    Figure CN119376971A_ABST
Patent Text Reader

Abstract

The invention provides a large model service starting method and device, equipment and a medium, relates to the technical field of artificial intelligence, in particular to the technical fields of large models, deep learning and the like, and can be used for application scenes such as generative search, document intelligent editing, intelligent assistants, virtual assistants, intelligent e-commerce and the like. The artificial large model service starting method comprises the steps of receiving a creation instruction; the creation instruction comprises memory information of a shared memory, and a large model is pre-loaded in the shared memory; creating a target process in response to the creation instruction, and mapping the shared memory into the target process based on the memory information; and calling the large model in the shared memory through the target process, and providing a large model service by using the large model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of big models, deep learning, etc., and can be used in application scenarios such as generative search, intelligent document editing, smart assistants, virtual assistants, smart e-commerce, etc., and in particular to a big model service startup method, device, equipment and medium. Background Art

[0002] When the number of user visits increases and a single large model service cannot handle user requests, a new large model service process needs to be created to handle more requests. Since large model services have many parameters, it usually takes a long time to start a large model service. Therefore, how to quickly start a large model service is a problem that needs to be solved. Summary of the invention

[0003] The present disclosure provides a large model service startup method, device, equipment and medium.

[0004] According to one aspect of the present disclosure, a large model service startup method is provided, comprising: receiving a creation instruction; the creation instruction includes memory information of a shared memory, and a large model is pre-loaded in the shared memory; creating a target process in response to the creation instruction, and mapping the shared memory to the target process based on the memory information; calling the large model in the shared memory through the target process, and using the large model to provide a large model service.

[0005] According to another aspect of the present disclosure, a large model service startup device is provided, including: a receiving module for receiving a creation instruction; the creation instruction includes memory information of a shared memory, and a large model is pre-loaded in the shared memory; a creation module for creating a target process in response to the creation instruction, and mapping the shared memory to the target process based on the memory information; a calling module for calling the large model in the shared memory through the target process, and using the large model to provide a large model service.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method as described in any of the above aspects.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the methods described in any one of the above aspects.

[0009] According to the embodiment of the present disclosure, a large model service can be quickly started.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram of an application scenario for implementing an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a second embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;

[0016] Figure 5 It is a schematic diagram of an electronic device used to implement the large model service startup method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0018] In order to quickly start a large model service, the present disclosure provides the following embodiments.

[0019] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a large model service startup method, such as Figure 1 As shown, the method includes:

[0020] 101. Receive a creation instruction; the creation instruction includes memory information of a shared memory, and a large model is pre-loaded in the shared memory.

[0021] 102. Create a target process in response to the creation instruction, and map the shared memory to the target process based on the memory information.

[0022] 103. Call the large model in the shared memory through the target process, and use the large model to provide large model services.

[0023] The large model refers to the Large Language Model (LLM), a natural language processing model based on deep learning. Its core features are its huge number of parameters and complex structure, which enables it to process and understand a large amount of natural language data. Through the process of pre-training and fine-tuning, the LLM model can perform well in a variety of natural language processing tasks, including but not limited to text generation, language understanding, machine translation, etc.

[0024] With the development of artificial intelligence technology, large models can provide services to users in a variety of fields.

[0025] The large model service is usually provided by creating a service process, which can be associated with the large model, thereby providing services to users through the large model.

[0026] In the related art, when creating a service process, a model object is usually created first, and then the large model parameters are obtained from the disk, and the model object is initialized using the parameters to obtain the initialized large model, and the initialized large model is used for subsequent services.

[0027] However, creating model objects and loading model parameters from disk takes a long time, which results in slow startup of large model services, affects service stability, and reduces user experience.

[0028] To this end, in the embodiment of the present disclosure, the large model is pre-stored in the shared memory. When the service process is created, the large model is obtained through the shared memory mapping technology, rather than creating and initializing the model object every time the process is created. This can increase the startup speed of the large model service, improve service stability, and enhance user experience.

[0029] The target process refers to the process to provide large model services. The number of target processes can be one or more.

[0030] Create an instruction to trigger the creation of a target process.

[0031] The creation instruction includes memory information of the shared memory, such as a key and a memory address, so that the shared memory can be mapped to the target process according to the memory information.

[0032] The large model is pre-stored in the shared memory. The large model is a large model to be provided. The specific model structure can be selected according to actual needs, for example, the llama2 large model. The llama2 large model is one of the llama (Large Language Model Meta AI) series models and is an open and efficient large model.

[0033] After obtaining the memory information from the creation instruction, based on the memory information, the shared memory is mapped to the target process using the shared memory mapping technology, so that the target process can obtain the large model from the shared memory and use the large model to provide large model services.

[0034] In this embodiment, since the large model is pre-stored in the shared memory, after the target process is created, the shared memory is mapped to the target process. The large model already stored in the shared memory can be directly used without recreating the model object and initializing it. This allows the large model service to be started quickly, improving service stability and enhancing user experience.

[0035] In order to better understand the present disclosure, the application scenarios involved in the present disclosure are described as follows:

[0036] Figure 2 It is a schematic diagram of an application scenario for implementing the embodiment of the present disclosure.

[0037] like Figure 2 As shown, the application scenario includes: a large model dynamic module 201, a large model control module 202 and a process creation module 203.

[0038] The large model dynamic module 201 is used to create a shared memory, preload the large model into the shared memory, and send the memory information (key and memory address) of the shared memory to the large model control module 202.

[0039] Specifically, the large model dynamic module can load the large model object from the database, initialize the large model parameters, obtain the large model, and store the large model in the shared memory.

[0040] The database can be a dynamic library, which can be loaded on demand, saving resources and improving efficiency.

[0041] Generally speaking, a shared memory is a memory space with continuous addresses.

[0042] Since the large model is large in size, it is difficult to apply for a large continuous address space. For this reason, in some embodiments, the large model can be divided into multiple sub-models, and accordingly, multiple shared memories are created, each of which is used to store a sub-model, and the multiple shared memories do not overlap in space. For example, the large model can be divided into an input layer, a hidden layer, and an output layer. The input layer part can be used as the first sub-model and stored in the first shared memory; the hidden layer part can be used as the second sub-model and stored in the second shared memory; and the output layer part can be used as the third sub-model and stored in the third shared memory. In this way, by storing the large model in multiple shared memories, the feasibility of large model storage can be improved and resource utilization can be improved.

[0043] Furthermore, the shared memory may be unstable, for example, it may be closed or damaged by the process by mistake, so the shared memory can be backed up. The number of backups can be set according to actual needs. For example, there are three copies, and each large model corresponds to three shared memories, then 9 shared memories are needed. When a group of shared memories has problems (such as being closed by mistake), the shared memories of other groups continue to provide services. At this time, the memory information of the 9 shared memories can be sent to the large model control module, and the large model control module selects the memory information of one group (three) of shared memories to send to the process creation module. After at least one shared memory in the group is closed by mistake, the process creation module can feedback the relevant error information to the large model control module, and then the large model control module sends the memory information of one of the other two groups of shared memories to the process creation module.

[0044] The large model control module 202 is used to send a creation instruction to the process creation module, and the creation instruction includes memory information of the shared memory.

[0045] Specifically, when necessary, the administrator can trigger the large model control module to send a creation instruction through the management interface, such as when the service response speed is found to be slow.

[0046] The creation instruction may also include the number of processes to be created, such as n, so that the process creation module can create n processes, where n is a positive integer, and thus one or more processes can be created.

[0047] The process creation module 203 is used to create a target process after receiving a creation instruction, and, according to the memory information contained in the creation instruction, map the shared memory to the target process, obtain the large model from the shared memory through the target process, and use the large model to provide large model services.

[0048] Specifically, the process creation module creates a target process according to a creation instruction, executes shared memory mapping, and initializes a communication network framework, so that the target process can use the large model in the shared memory to provide large model services.

[0049] Initializing the communication network framework usually refers to the necessary configuration and settings of network components when building network applications or services to ensure that they can work properly. This may include setting network parameters, starting network services, and initializing the underlying system components.

[0050] The above-mentioned large model dynamic module 201, process creation module 203 and the process created by them are usually located on the same physical device, and the large model control module 202 can be located on the same physical device or other physical devices.

[0051] In combination with the above application scenarios, the present disclosure also provides the following embodiments.

[0052] Figure 3 is a schematic diagram according to the second embodiment of the present disclosure. This embodiment provides a large model service startup method, the method comprising:

[0053] 301. The large model dynamic module creates a shared memory and preloads the large model into the shared memory.

[0054] Preloading refers to loading before providing large model services.

[0055] Specifically, large models can be loaded from dynamic libraries to save resources and improve efficiency.

[0056] In some embodiments, the large model is divided into multiple sub-models, and the shared memory is multiple; each shared memory is used to store a sub-model of the large model.

[0057] For example, if a large model is divided into three sub-models, three shared memories can be created, each storing one sub-model.

[0058] Since shared memory is a memory space with continuous addresses, large models are large in size. Therefore, dividing the large model into multiple sub-models and creating multiple shared memories to store the sub-models respectively can improve feasibility and resource utilization.

[0059] In some embodiments, the large model has multiple sets of copies, and the shared memory has multiple sets; each set of shared memory is used to store a set of copies of the large model.

[0060] Each group of replicas includes a large model, or the large model is divided into multiple sub-models, and these sub-models serve as a group of replicas.

[0061] Assuming that a group of replicas includes three sub-models and the shared memory is divided into three groups, nine shared memories are created, with three shared memories as a group of shared memories, each storing one sub-model.

[0062] In this way, shared memory backup can be achieved to improve reliability.

[0063] 302. The large model dynamic module sends the memory information of the shared memory to the large model control module.

[0064] After the large model dynamic model creates a shared memory and loads the large model into the shared memory, the memory information (key and address) of the shared memory is sent to the large model control module.

[0065] 303. The large model control module sends a creation instruction to the process creation module, which includes memory information of the shared memory.

[0066] Among them, the large model control module can generate a creation instruction according to the trigger operation of the administrator and carry the memory information in the creation instruction.

[0067] 304. A process creation module creates a target process in response to the creation instruction, and maps the shared memory to the target process based on the memory information.

[0068] Among them, a prototype process can be created in response to the creation instruction; based on the memory information, the shared memory is mapped to the prototype process to obtain a parent process; the parent process is copied to create a child process; and the parent process and the child process are used as the target process.

[0069] Specifically, the clone function can be used to create a target process, so that the parent and child processes can share memory resources, so that both the parent and child processes can schedule the large model to provide services.

[0070] In this embodiment, by copying the parent process to obtain the child process, resources can be saved and the target process can be created efficiently and conveniently.

[0071] 305. A process creation module calls the large model in the shared memory through the target process and uses the large model to provide large model services.

[0072] Specifically, the communication network framework can be initialized to obtain an initialized communication network framework; based on the initialized communication network framework, a user request is received through the target process, and the user request is sent to the large model in the shared memory to process the user request through the large model.

[0073] In this embodiment, by initializing the communication network framework, using the initialized communication network framework to call the big model, and providing the big model service through the big model, the success rate of starting the big model service can be improved.

[0074] Figure 4 4 is a schematic diagram of a third embodiment of the present disclosure, wherein the present embodiment provides a large model service startup device. The device 400 includes: a receiving module 401, a creating module 402 and a calling module 403.

[0075] The receiving module 401 is used to receive a creation instruction; the creation instruction contains memory information of the shared memory, and the large model is pre-loaded in the shared memory; the creation module 402 is used to create a target process in response to the creation instruction, and map the shared memory to the target process based on the memory information; the calling module 403 is used to call the large model in the shared memory through the target process, and use the large model to provide large model services.

[0076] In this embodiment, since the large model is pre-stored in the shared memory, after the target process is created, the shared memory is mapped to the target process. The large model already stored in the shared memory can be directly used without recreating the model object and initializing it. This allows the large model service to be started quickly, improving service stability and enhancing user experience.

[0077] In some embodiments, the large model is divided into multiple sub-models, and the shared memory is multiple; each shared memory is used to store a sub-model of the large model.

[0078] Since shared memory is a memory space with continuous addresses, large models are large in size. Therefore, dividing the large model into multiple sub-models and creating multiple shared memories to store the sub-models respectively can improve feasibility and resource utilization.

[0079] In some embodiments, the large model has multiple sets of copies, and the shared memory has multiple sets; each set of shared memory is used to store a set of copies of the large model.

[0080] In this way, shared memory backup can be achieved to improve reliability.

[0081] In some embodiments, the creation module 402 is further used to:

[0082] In response to the creation instruction, creating a prototype process;

[0083] Based on the memory information, mapping the shared memory into the prototype process to obtain a parent process;

[0084] Copying the parent process to create a child process;

[0085] The parent process and the child process are used as the target process.

[0086] In this embodiment, by copying the parent process to obtain the child process, resources can be saved and the target process can be created efficiently and conveniently.

[0087] In some embodiments, the calling module 403 is further used to:

[0088] Initializing the communication network framework to obtain an initialized communication network framework;

[0089] Based on the initialized communication network framework, a user request is received through the target process, and the user request is sent to the large model in the shared memory so as to process the user request through the large model.

[0090] In this embodiment, by initializing the communication network framework, using the initialized communication network framework to call the big model, and providing the big model service through the big model, the success rate of starting the big model service can be improved.

[0091] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0092] It can be understood that the “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0093] It is understandable that unless there is any special limitation on the sequence of steps in the process, it means that the timing relationship between these steps is not limited.

[0094] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0095] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0096] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0097] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0098] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0099] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the large model service startup method. For example, in some embodiments, the large model service startup method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the large model service startup method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the large model service startup method in any other appropriate manner (e.g., by means of firmware).

[0100] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable task processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0102] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0104] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0105] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0106] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0107] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A large model service startup method, comprising: Receive creation instructions; The creation instruction includes memory information of the shared memory, and the large model is pre-loaded in the shared memory; Creating a target process in response to the creation instruction, and mapping the shared memory to the target process based on the memory information; The large model in the shared memory is called through the target process, and the large model is used to provide a large model service.

2. The method according to claim 1, wherein: The large model is divided into a plurality of sub-models, and the shared memory is a plurality of sub-models; Each shared memory is used to store a sub-model of the large model.

3. The method according to claim 1, wherein: The large model has multiple sets of copies, and the shared memory has multiple sets; Each set of shared memory is used to store a set of copies of the large model.

4. The method according to claim 1, wherein: The creating a target process and mapping the shared memory to the target process based on the memory information includes: In response to the creation instruction, creating a prototype process; Based on the memory information, mapping the shared memory into the prototype process to obtain a parent process; Copying the parent process to create a child process; The parent process and the child process are used as the target process.

5. The method according to claim 1, wherein: The step of calling the large model in the shared memory through the target process and using the large model to provide a large model service includes: Initializing the communication network framework to obtain an initialized communication network framework; Based on the initialized communication network framework, a user request is received through the target process, and the user request is sent to the large model in the shared memory so as to process the user request through the large model.

6. A large model service startup device, comprising: A receiving module, used for receiving a creation instruction; The creation instruction includes memory information of the shared memory, and the large model is pre-loaded in the shared memory; A creation module, configured to create a target process in response to the creation instruction, and map the shared memory to the target process based on the memory information; The calling module is used to call the large model in the shared memory through the target process and use the large model to provide large model services.

7. The device according to claim 6, wherein: The large model is divided into a plurality of sub-models, and the shared memory is a plurality of sub-models; Each shared memory is used to store a sub-model of the large model.

8. The device according to claim 6, wherein: The large model has multiple sets of copies, and the shared memory has multiple sets; Each set of shared memory is used to store a set of copies of the large model.

9. The device according to claim 6, wherein: The creation module is further used to: In response to the creation instruction, creating a prototype process; Based on the memory information, mapping the shared memory into the prototype process to obtain a parent process; Copying the parent process to create a child process; The parent process and the child process are used as the target process.

10. The device according to claim 6, wherein: The calling module is further used for: Initializing the communication network framework to obtain an initialized communication network framework; Based on the initialized communication network framework, a user request is received through the target process, and the user request is sent to the large model in the shared memory so as to process the user request through the large model.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Citation Information

Cited By

  • Model loading method and device, storage medium and electronic equipment

    CN120276791A

  • Model processing method, terminal device, system, storage medium and program product

    CN121144057A