Cloud native large model starting method, device and system

Abstract

The invention provides a cloud native large model starting method, device and system, relates to the field of artificial intelligence such as deep learning, large models, cloud computing and vertical search, and can be used for application scenes such as generative search, document intelligent editing, intelligent assistants, virtual assistants and intelligent e-commerce. The method comprises the following steps: acquiring a cloud native large model from a central control service, and generating a target mirror image corresponding to the cloud native large model; the target mirror image is sent to a container management service, so that the container management service starts model instances corresponding to the cloud native large model for preset M geographic areas, M is a positive integer, the model instances comprise initial processes, the initial processes are used for creating service processes after a trigger instruction is obtained, and the service processes are used for storing the target mirror image; the service process is used for processing the access request of the user.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to cloud-native large model startup methods, devices and systems in the fields of deep learning, large models, cloud computing and vertical search. Background Art

[0002] Since large models have many parameters, the speed of starting large model services is usually slow. Therefore, when user access requests increase, the large model service may not be able to withstand the large amount of traffic, resulting in many request failures and affecting service availability. Summary of the invention

[0003] The present disclosure provides a cloud-native large model startup method, device and system.

[0004] A cloud-native large model startup method includes:

[0005] Obtain a cloud-native big model from a central control service, and generate a target image corresponding to the cloud-native big model;

[0006] The target image is sent to a container management service, and the container management service starts model instances corresponding to the cloud native large model for predetermined M geographical areas, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process user access requests.

[0007] A cloud-native large model startup method includes:

[0008] Obtain a target image from an image service, where the target image is a target image generated by the image service for a cloud native large model obtained from a central control service;

[0009] According to the target image, model instances corresponding to the cloud native big model are started for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0010] A cloud-native large model startup method includes:

[0011] Get the cloud-native large model uploaded by the user;

[0012] The cloud native big model is sent to the image service, which is used by the image service to generate a target image corresponding to the cloud native big model and send it to the container management service. The target image is used by the container management service to start the model instances corresponding to the cloud native big model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0013] A cloud-native large model startup device, comprising: an image generation module and an image sending module;

[0014] The image generation module is used to obtain a cloud native large model from a central control service and generate a target image corresponding to the cloud native large model;

[0015] The image sending module is used to send the target image to the container management service, so that the container management service can start the model instances corresponding to the cloud native large model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0016] A cloud native large model startup device, comprising: an image acquisition module and an instance startup module;

[0017] The image acquisition module is used to acquire a target image from an image service, where the target image is a target image generated by the image service for a cloud native large model acquired from a central control service;

[0018] The instance startup module is used to start the model instances corresponding to the cloud native large model for the predetermined M geographical areas according to the target image, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0019] A cloud-native large model startup device includes: an information acquisition module and an information processing module;

[0020] The information acquisition module is used to obtain the cloud-native large model uploaded by the user;

[0021] The information processing module is used to send the cloud native big model to the image service, and the image service is used to generate a target image corresponding to the cloud native big model and send it to the container management service. The target image is used by the container management service to start the model instances corresponding to the cloud native big model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0022] A cloud-native large model startup system includes: the three cloud-native large model startup devices as described above.

[0023] An electronic device, comprising:

[0024] at least one processor; and

[0025] a memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0027] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0028] A computer program product comprises a computer program / instruction, wherein the computer program / instruction implements the above method when executed by a processor.

[0029] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0031] Figure 1 This is a flowchart of the first embodiment of the cloud native large model startup method described in the present disclosure;

[0032] Figure 2 This is a flowchart of the second embodiment of the cloud native large model startup method described in the present disclosure;

[0033] Figure 3 This is a flowchart of the third embodiment of the cloud native large model startup method described in the present disclosure;

[0034] Figure 4This is a schematic diagram of the overall implementation process of the cloud native large model startup method described in the present disclosure;

[0035] Figure 5 This is a schematic diagram of the composition structure of the first embodiment 500 of the cloud native large model startup device described in the present disclosure;

[0036] Figure 6 This is a schematic diagram of the composition structure of the second embodiment 600 of the cloud native large model startup device described in the present disclosure;

[0037] Figure 7 This is a schematic diagram of the composition structure of the third embodiment 700 of the cloud native large model startup device described in the present disclosure;

[0038] Figure 8 This is a schematic diagram of the composition structure of an embodiment 800 of the cloud-native large model startup system described in the present disclosure;

[0039] Fig. 9 A schematic block diagram of an electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0040] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0041] In addition, it should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0042] Figure 1 This is a flow chart of the first embodiment of the cloud native large model startup method described in the present disclosure. Figure 1 As shown, the following specific implementation methods are included.

[0043] In step 101, a cloud native big model is obtained from a central control service, and a target image corresponding to the cloud native big model is generated.

[0044] In step 102, the target image is sent to the container management service, which is used by the container management service to start model instances (i.e., large model services) corresponding to the cloud native large model for the predetermined M geographical areas, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process user access requests.

[0045] By adopting the scheme described in the above method embodiment, a model instance can be started for M geographical areas according to the target image corresponding to the cloud-native big model, including an initial process. Moreover, the initial process can create a service process after obtaining the trigger instruction to process the user's access request, so that the big model service can be started quickly and on a large scale, thereby improving the service availability and user experience.

[0046] The solution described in the present disclosure mainly involves three execution entities: central control service, image service and container management service. Among them, the central control service refers to the cloud native large model central control service, the image service can be a cloud native large model image service based on container (docker), and the container management service can be a container management service based on container cluster management system (k8s, kubernetes). Figure 1 The execution subject of the illustrated embodiment may be a mirror service.

[0047] The image service can obtain the cloud-native big model from the central control service and generate a target image corresponding to the cloud-native big model.

[0048] In some embodiments of the present disclosure, a code file corresponding to a cloud-native large model may be generated first, wherein the code file is a Python language code file, and then the code file may be made into an image as an initial image, and then a target image may be generated based on the initial image.

[0049] The cloud-native big models in the solutions disclosed in this disclosure are all big models implemented using the open source deep learning framework (pytorch) for machine learning, and accordingly, their codes can be cloud-native. After obtaining the cloud-native big model, the corresponding code file, i.e., the python file, can be generated first. The python language is a widely used interpreted, high-level and general programming language that supports a variety of programming paradigms, including structured, procedural, reflective, object-oriented and functional programming, etc. It has a dynamic type system and garbage collection function, can automatically manage memory usage, and has a huge and extensive standard library. Accordingly, generating a python file can improve subsequent processing efficiency and the accuracy of processing results.

[0050] In some embodiments of the present disclosure, the python file may include: a first code line, a second code line, a predetermined functional module, and a remote procedure call (RPC) service framework code, wherein the first code line is used to illustrate the introduction of the cloud native big model into the python file, the second code line is used to initialize the cloud native big model, the functional module is used to create a service process after obtaining a trigger instruction, and stop subsequent execution when running to the functional module but not obtaining a trigger instruction.

[0051] The solution disclosed in the present invention does not limit the specific model of the cloud native big model. For example, it can be the open source artificial intelligence large language model 2 (LLAMA2, Large Language Model MetaAI 2), where AI refers to artificial intelligence (Artificial Intelligence).

[0052] Taking the LLAMA2 model as an example, after the image service obtains the model, it can generate a Python file as shown below:

[0053]

[0054]

[0055] Rpc service framework code.

[0056] Among them, "from llama import Llama, Dialog" is the first code line, which is used to illustrate the introduction of the LLAMA2 model into the python file. The content corresponding to the generator is the second code line, which is used to initialize the LLAMA2 model. In addition, ckpt_dir represents the model parameter directory, tokenizer_path represents the tokenizer path, max_seq_len represents the maximum sequence length, and max_batch_size represents the maximum block length.

[0057] The specific content of the Rpc service framework code can be determined according to the actual situation. Between the Rpc service framework code and the second code line, there are predetermined functional modules, such as the DistributedModelSplit module shown above, which calls the Python operating system. Fork (os.fork) command to quickly create a new process.

[0058] After generating the Python file, you can use Docker to make it into an image. For easy distinction, it can be called the initial image and can be represented by llama2_model_local.

[0059] Furthermore, in some embodiments of the present disclosure, an initial image can be started, and in response to determining that the initial image stops when running to a functional module, the initial image in a stopped state after running can be packaged into a new image as a target image, and can be represented by llama2_model_online.

[0060] The initial image can be started. After it is started, the initial image will stop when it runs to the DistributedModelSplit module. This is because this module needs to receive an external trigger instruction to execute the os.fork command, otherwise it will stop. After that, the initial image in the stopped state after running can be packaged into a new image, that is, the target image, through the container save (docker save) command, and then the target image can be sent to the container management service.

[0061] Correspondingly, the container management service can start the model instances corresponding to the cloud native large model for the predetermined M geographical areas according to the acquired target image, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining the trigger instruction, and the service process is used to process the user's access request.

[0062] After the initial process obtains the trigger instruction, it can create a new process, namely the service process, by executing the os.fork command. The initial process creates the service process very quickly because the initial process is equivalent to having executed the DistributedModelSplit module, that is, the slowest part has been executed, so it usually only takes a few seconds to create the required service process.

[0063] The initial process and service process corresponding to the same geographical area belong to the same model instance. The main difference between the two is that the initial process is usually only responsible for creating a new process, namely the service process, and is not responsible for processing user access requests, while the service process is usually only responsible for processing user access requests and is not responsible for creating new processes. In addition, the initial process and the service process are located on the same physical machine.

[0064] The above mainly describes the solution of the present disclosure from the perspective of the image service. The following further describes the solution of the present disclosure from the perspectives of the container management service and the central control service.

[0065] Figure 2 This is a flow chart of the second embodiment of the cloud native large model startup method described in this disclosure. The execution subject of the method described in this embodiment can be a container management service. Figure 2 As shown, the following specific implementation methods are included.

[0066] In step 201, a target image is obtained from an image service, where the target image is a target image generated by the image service for a cloud native large model obtained from a central control service.

[0067] In step 202, according to the target image, model instances corresponding to the cloud native large model are started for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process user access requests.

[0068] By adopting the scheme described in the above method embodiment, a model instance can be started for M geographical areas according to the target image corresponding to the cloud-native big model, including an initial process. Moreover, the initial process can create a service process after obtaining the trigger instruction to process the user's access request, so that the big model service can be started quickly and on a large scale, thereby improving the service availability and user experience.

[0069] In some embodiments of the present disclosure, the trigger instruction may include: a trigger instruction from a central control service, and the service process may include: a target number of service processes created according to the target number carried in the trigger instruction.

[0070] Assuming that the country is divided into 6 geographical regions, such as Northeast China and North China, the value of M can be 6 or less than 6, that is, the model instances corresponding to the cloud native big model can be started for all 6 geographical regions, or the model instances corresponding to the cloud native big model can be started only for some of the geographical regions. Among them, each model instance can be started based on the container orchestration function of k8s, that is, a part of the model instances can be started in advance with the target image.

[0071] After that, after the initial process in each model instance receives the trigger instruction from the central control service, the DistributedModelSplit module can quickly create a new process, namely the service process, by executing the os.fork command. Generally speaking, the initial process is usually only responsible for creating a new process and is not responsible for processing user access requests, while the service process is usually only responsible for processing user access requests and is not responsible for creating a new process.

[0072] In addition, the trigger instruction may also carry a target number. For example, taking the Northeast as an example, the user may pre-set the target number, i.e., the number of service processes to be created. Accordingly, the initial process corresponding to the geographical area may create the target number of service processes. Through this processing, the user may effectively control the created service processes.

[0073] Figure 3This is a flow chart of the third embodiment of the cloud native large model startup method described in this disclosure. The execution subject of the method described in this embodiment can be a central control service. Figure 3 As shown, the following specific implementation methods are included.

[0074] In step 301, a cloud-native large model uploaded by a user is obtained.

[0075] In step 302, the cloud native big model is sent to the image service, which is used by the image service to generate a target image corresponding to the cloud native big model and send it to the container management service. The target image is used by the container management service to start the model instances corresponding to the cloud native big model for the predetermined M geographical areas, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process user access requests.

[0076] By adopting the scheme described in the above method embodiment, a model instance can be started for M geographical areas according to the target image corresponding to the cloud-native big model, including an initial process. Moreover, the initial process can create a service process after obtaining the trigger instruction to process the user's access request, so that the big model service can be started quickly and on a large scale, thereby improving the service availability and user experience.

[0077] In some embodiments of the present disclosure, in response to obtaining the first response information returned after the image service sends the target image to the container management service, the container management service may be notified to start the model instance for M geographical regions respectively, and in response to obtaining the second response information returned after the container management service starts the model instance for M geographical regions respectively, a trigger instruction may be sent to the initial process. The specific form of the trigger instruction is not limited.

[0078] Through the above interaction, the central control service can timely understand the progress of the entire processing process, and can achieve effective management and control of the entire processing process.

[0079] In addition, in some embodiments of the present disclosure, the target numbers set by the user for M geographical areas can also be obtained, and the target numbers of the corresponding geographical areas can be carried in the trigger instruction sent to the initial process, so that the initial process can create the target number of service processes.

[0080] Combined with the above introduction, Figure 4 The figure is a schematic diagram of the overall implementation process of the cloud-native large model startup method described in the present invention. Figure 4 In , we assume that the value of M is 2, and that the number of targets corresponding to each geographical area is 2. Figure 4As shown, the central control service can obtain the cloud native big model uploaded by the user and the target quantity set for the two geographical regions respectively, and can send the cloud native big model to the image service. The image service can generate a target image corresponding to the obtained cloud native big model. For example, it can first generate a python file corresponding to the cloud native big model, and then make the python file into an image as the initial image, and then generate the target image based on the initial image. Furthermore, the image service can send the target image to the container management service, and can return the first response information to the central control service. Afterwards, the central control service can notify the container management service to start a model instance for each of the two geographical regions, that is, Figure 4 , correspondingly, the container management service can start a model instance for each of the two geographical regions according to the target image, the model instance includes an initial process, and can return a second response message to the central control service. Afterwards, the central control service can send a trigger instruction to each initial process, and can carry the target number 2 in the trigger instruction. Correspondingly, each initial process can create two service processes according to the trigger instruction, and the service process can be used to actually process the user's access request and return the processing result to the user.

[0081] If the user wants to close a service process for some reason, he can send a close command to the central control service. Accordingly, the central control service can send a close command to the control script in the model instance where the service process is located, and the control script can then delete the service process.

[0082] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the described order of actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure. In addition, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0083] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0084] Figure 5 FIG. 5 is a schematic diagram of the composition structure of the first embodiment 500 of the cloud native large model startup device described in the present disclosure. Figure 5 As shown, it includes: a mirror image generation module 501 and a mirror image sending module 502.

[0085] The image generation module 501 is used to obtain the cloud native big model from the central control service and generate a target image corresponding to the cloud native big model.

[0086] The image sending module 502 is used to send the target image to the container management service, so that the container management service can start the model instances corresponding to the cloud native large model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0087] In some embodiments of the present disclosure, the image generation module 501 may first generate a code file corresponding to the cloud-native large model, wherein the code file is a Python language code file, and then the code file may be made into an image as an initial image, and then a target image may be generated based on the initial image.

[0088] In some embodiments of the present disclosure, the code file may include: a first code line, a second code line, a predetermined functional module, and an RPC service framework code, wherein the first code line is used to illustrate the introduction of the cloud native big model into the code file, the second code line is used to initialize the cloud native big model, and the functional module is used to create a service process after obtaining a trigger instruction, and stop running when running to the functional module but not obtaining a trigger instruction.

[0089] In addition, in some embodiments of the present disclosure, the image generation module 501 can start the initial image. In response to determining that the initial image stops when running to the functional module, the initial image in a stopped state after running can be packaged into a new image as the target image, and then the image sending module 502 can send the target image to the container management service.

[0090] Figure 6 FIG. 6 is a schematic diagram of the structure of the second embodiment 600 of the cloud native large model startup device described in the present disclosure. Figure 6 As shown, it includes: an image acquisition module 601 and an instance startup module 602.

[0091] The image acquisition module 601 is used to acquire a target image from an image service, where the target image is a target image generated by the image service for a cloud native large model acquired from a central control service.

[0092] The instance startup module 602 is used to start the model instances corresponding to the cloud native large model for the predetermined M geographical areas according to the target image, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0093] In some embodiments of the present disclosure, the trigger instruction may include: a trigger instruction from a central control service, and the service process may include: a target number of service processes created according to the target number carried in the trigger instruction.

[0094] Figure 7 FIG. 7 is a schematic diagram of the composition structure of the third embodiment 700 of the cloud native large model startup device described in the present disclosure. Figure 7 As shown, it includes: an information acquisition module 701 and an information processing module 702.

[0095] The information acquisition module 701 is used to obtain the cloud-native large model uploaded by the user.

[0096] Information processing module 702 is used to send the cloud native big model to the image service, and the image service is used to generate a target image corresponding to the cloud native big model and send it to the container management service. The target image is used by the container management service to start the model instances corresponding to the cloud native big model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

[0097] In some embodiments of the present disclosure, the information processing module 702 may notify the container management service to start model instances for M geographical regions respectively in response to obtaining the first response information returned after the image service sends the target image to the container management service, and may send a trigger instruction to the initial process in response to obtaining the second response information returned after the container management service starts the model instances for M geographical regions respectively.

[0098] In addition, in some embodiments of the present disclosure, the information acquisition module 701 can also obtain the target numbers set by the user for M geographical areas respectively. Accordingly, the information processing module 702 can carry the target numbers of the corresponding geographical areas in the trigger instruction sent to the initial process, so that the initial process can create the target number of service processes.

[0099] Figure 8 FIG. 8 is a schematic diagram of the structure of the cloud native large model startup system embodiment 800 described in the present disclosure. Figure 8 As shown, it includes: a first cloud native large model starting device 801, a second cloud native large model starting device 802 and a third cloud native large model starting device 803.

[0100] Among them, the first cloud native large model starting device 801 can be Figure 5 The second cloud native large model starting device 802 may be Figure 6 The third cloud native large model starting device 803 can be Figure 7 The device shown.

[0101] The specific working process of the above-mentioned device and system embodiments can refer to the relevant description in the aforementioned method embodiments and will not be repeated here.

[0102] In summary, by adopting the solution described in the present disclosure, large model services can be started quickly and on a large scale, thereby improving the availability of services and user experience.

[0103] The solution disclosed in the present disclosure can be applied to the field of artificial intelligence, especially to the fields of deep learning, large models, cloud computing, and vertical search. Artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0104] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0105] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0106] Fig. 9 A schematic block diagram of an electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0107] like Fig. 9As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0108] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0109] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI, Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP, Digital Signal Processing), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the methods described in the present disclosure may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the method described in the present disclosure in any other appropriate manner (for example, by means of firmware).

[0110] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0112] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM, Electronically Programmable Read-Only Memory), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM, Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0115] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0116] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0117] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A cloud-native large model startup method, comprising: Obtain a cloud-native big model from a central control service, and generate a target image corresponding to the cloud-native big model; The target image is sent to a container management service, and the container management service starts model instances corresponding to the cloud native large model for predetermined M geographical areas, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process user access requests.

2. The method according to claim 1, wherein: Generating a target image corresponding to the cloud native large model includes: Generate a code file corresponding to the cloud native large model, wherein the code file is a Serpentine language code file; Making the code file into an image as an initial image; The target image is generated according to the initial image.

3. The method according to claim 2, wherein: The code file includes: a first code line, a second code line, a predetermined functional module and a remote procedure call service framework code; Among them, the first code line is used to illustrate the introduction of the cloud native big model into the code file, the second code line is used to initialize the cloud native big model, the functional module is used to create the service process after obtaining the trigger instruction, and stop subsequent operation when running to the functional module but not obtaining the trigger instruction.

4. The method according to claim 3, wherein: Generating the target image according to the initial image includes: The initial image is started, and in response to determining that the initial image stops when running to the functional module, the initial image in a stopped state after running is packaged into a new image as the target image.

5. A cloud-native large model startup method, comprising: Obtain a target image from an image service, where the target image is a target image generated by the image service for a cloud native large model obtained from a central control service; According to the target image, model instances corresponding to the cloud native big model are started for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

6. The method according to claim 5, wherein: The trigger instruction includes: a trigger instruction from the central control service; The service process includes: a target number of service processes created according to the target number carried in the trigger instruction.

7. A cloud-native large model startup method, comprising: Get the cloud-native large model uploaded by the user; The cloud native big model is sent to the image service, which is used by the image service to generate a target image corresponding to the cloud native big model and send it to the container management service. The target image is used by the container management service to start the model instances corresponding to the cloud native big model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

8. The method according to claim 7, further comprising: In response to obtaining first response information returned by the image service after sending the target image to the container management service, informing the container management service to start the model instances for the M geographical areas respectively; In response to obtaining second response information returned by the container management service after starting the model instances for the M geographical areas respectively, sending the trigger instruction to the initial process.

9. The method according to claim 7, further comprising: The target numbers set by the user for the M geographical areas are obtained, and the target numbers of the corresponding geographical areas are carried in the trigger instruction sent to the initial process, so that the initial process creates the target number of service processes.

10. A cloud-native large model startup device, comprising: Image generation module and image sending module; The image generation module is used to obtain a cloud native large model from a central control service and generate a target image corresponding to the cloud native large model; The image sending module is used to send the target image to the container management service, so that the container management service can start the model instances corresponding to the cloud native large model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

11. The device according to claim 10, wherein: The image generation module generates a code file corresponding to the cloud native large model, where the code file is a Serpentine language code file, and makes the code file into an image as an initial image, and generates the target image based on the initial image.

12. The device according to claim 11, wherein The code file includes: a first code line, a second code line, a predetermined functional module and a remote procedure call service framework code; Among them, the first code line is used to illustrate the introduction of the cloud native big model into the code file, the second code line is used to initialize the cloud native big model, the functional module is used to create the service process after obtaining the trigger instruction, and stop subsequent operation when running to the functional module but not obtaining the trigger instruction.

13. The device according to claim 12, wherein: The image generation module starts the initial image, and in response to determining that the initial image stops when running to the functional module, packages the initial image in a stopped state after running into a new image as the target image.

14. A cloud-native large model startup device, comprising: Image acquisition module and instance startup module; The image acquisition module is used to acquire a target image from an image service, where the target image is a target image generated by the image service for a cloud native large model acquired from a central control service; The instance startup module is used to start the model instances corresponding to the cloud native large model for the predetermined M geographical areas according to the target image, where M is a positive integer. The model instance includes an initial process, which is used to create a service process after obtaining a trigger instruction, and the service process is used to process the user's access request.

15. The device according to claim 14, wherein: The trigger instruction includes: a trigger instruction from the central control service; The service process includes: a target number of service processes created according to the target number carried in the trigger instruction.

16. A cloud-native large model startup device, comprising: Information acquisition module and information processing module; The information acquisition module is used to obtain the cloud-native large model uploaded by the user; The information processing module is used to send the cloud native big model to the image service, and the image service is used to generate a target image corresponding to the cloud native big model and send it to the container management service. The target image is used by the container management service to start the model instances corresponding to the cloud native big model for the predetermined M geographical areas respectively, where M is a positive integer. The model instance includes an initial process, and the initial process is used to create a service process after obtaining a trigger instruction. The service process is used to process the user's access request.

17. The device according to claim 16, wherein: The information processing module is further used to, in response to obtaining first response information returned after the image service sends the target image to the container management service, notify the container management service to start the model instances for the M geographical regions respectively, and in response to obtaining second response information returned after the container management service starts the model instances for the M geographical regions respectively, send the trigger instruction to the initial process.

18. The device according to claim 16, wherein: The information acquisition module is further used to obtain the target quantities set by the user for the M geographical areas respectively; The information processing module is further used to carry a target number of corresponding geographical areas in the trigger instruction sent to the initial process, so that the initial process can create the target number of service processes.

19. A cloud-native large model startup system, comprising: A cloud-native large model starting device as described in any one of claims 1 to 4, a cloud-native large model starting device as described in any one of claims 5 to 6, and a cloud-native large model starting device as described in any one of claims 7 to 9.

20. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 9.

22. A computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images