Memory Management Method, Apparatus, Device, and Storage Medium for Deep Learning Model

Abstract

The present invention provides a memory management method, device, equipment and storage medium for a deep learning model. The method includes: obtaining multiple storage spaces on a device to be deployed; using a preset storage function to establish communication interface connections between the storage spaces to achieve data transmission between different storage spaces; adding algorithms for tensor data associated with the deep learning model to each storage space and configuring personalized expansion options for the deep learning model; and deploying the deep learning model on the device to be deployed according to the personalized expansion options and communication interfaces. By setting the tensor information required by the model in a personalized manner, the memory occupancy is reduced. The workload of the deployment personnel is greatly reduced, making the deployment work more convenient and fast.

Description

Technical Field

[0001] This application relates to the technical field of model deployment, and specifically relates to a memory management method, device, equipment, and storage medium for a deep learning model. Background Art

[0002] Deep neural networks require a large amount of computing and memory, and among them, GPUs are currently the most popular choice for training devices. The training of deeper and wider neural networks poses a huge challenge to the memory of GPUs. Since the memory size of general computers is very limited, and multiple general-purpose dependency libraries are set up on GPUs or CPUs, the memory is severely compressed. Therefore, when deploying relatively large deep learning models to a computer, a common problem faced is insufficient memory, resulting in the inability to deploy the model. Therefore, a memory management method, device, equipment, and storage medium for a deep learning model are needed. Summary of the Invention

[0003] In view of the above-mentioned disadvantages of the prior art, the present invention provides a memory management method, device, equipment, and storage medium for a deep learning model to solve the technical problem that when the existing deep learning model is deployed to a device, due to too many general-purpose dependency libraries, the storage space cannot accommodate the deep learning model.

[0004] The memory management method for a deep learning model provided by the present invention is used to deploy a deep learning model on a device, and includes:

[0005] Obtain multiple storage spaces on the device to be deployed;

[0006] Use a preset storage function to establish a communication interface connection between each storage space to achieve data transmission between different storage spaces;

[0007] In each storage space, add an algorithm for tensor data associated with the deep learning model, and configure personalized expansion options for the deep learning model;

[0008] Deploy the deep learning model on the device to be deployed according to the personalized expansion options and the communication interface.

[0009] In an embodiment of the present invention, after obtaining multiple storage spaces on the device to be deployed, it further includes:

[0010] Judge whether each storage space can accommodate the deep learning model to be deployed;

[0011] If each storage space can accommodate the deep learning model to be deployed, establish a communication interface connection between each storage space;

[0012] If there is at least one storage space that cannot accommodate the deep learning model to be deployed, delete the external dependency libraries unrelated to the deep learning model in the storage space that cannot accommodate the deep learning model.

[0013] In an embodiment of the present invention, the types of the storage spaces include CPU storage spaces and GPU storage spaces.

[0014] In an embodiment of the present invention, the establishing communication interface connections between the storage spaces by using a preset storage function includes:

[0015] According to the types of the storage spaces, obtain the storage functions corresponding to the storage spaces;

[0016] Input the interface information of each storage space into the corresponding storage function, and use the storage function to establish communication interface connections between different storage spaces.

[0017] In an embodiment of the present invention, the adding algorithms of tensor data associated with the deep learning model in each storage space and configuring personalized expansion options for the deep learning model include:

[0018] Based on the characteristics of the deep learning model, obtain the tensor data required by the deep learning model;

[0019] Deploy the tensor data in the corresponding storage space in the form of operators or functions to form personalized expansion options.

[0020] In an embodiment of the present invention, after obtaining multiple storage spaces on the device to be deployed, it further includes: adding cuda streams in each storage space to achieve data synchronous transmission.

[0021] In an embodiment of the present invention, after deploying the deep learning model on the device to be deployed, it further includes: testing the device to be deployed to ensure the normal operation of the deep learning model.

[0022] In an embodiment of the present invention, there is also provided a memory management device for a deep learning model, and the device includes:

[0023] A storage space acquisition module configured to acquire multiple storage spaces on a device to be deployed;

[0024] A communication interface deployment module configured to use a preset storage function to establish communication interface connections between the storage spaces to achieve data transmission between different storage spaces;

[0025] The personalized extension option deployment module is configured to add an algorithm for tensor data associated with the deep learning model in each storage space and configure the personalized extension options of the deep learning model.

[0026] The deep learning model deployment module is configured to deploy the deep learning model on the device to be deployed according to the personalized extension options and the communication interface.

[0027] In an embodiment of the present invention, an electronic device is further provided, including:

[0028] One or more processors;

[0029] A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the memory management method of the deep learning model described in any one of the above.

[0030] In an embodiment of the present invention, a computer-readable storage medium is further provided, on which computer-readable instructions are stored, which, when executed by a processor of a computer, cause the computer to execute the memory management method of the deep learning model described in any one of the above.

[0031] The beneficial effects of the present invention: In the present invention, multiple storage spaces are allocated for the deep learning model on the device to be deployed, and according to a preset storage function, with the relevant interfaces of each storage space as parameters, data transmission between different storage spaces is established. Thus, normal data transmission between different storage spaces is achieved. Then, according to the characteristics of the deep learning model to be deployed, an algorithm for tensor data related to the characteristics of the deep learning model is added to each storage space, thereby configuring the personalized extension options of the deep learning model. Then, according to the personalized extension options and the communication interface, the deep learning model is deployed on the device to be deployed, thus completing the deployment work of the deep learning model. In the present invention, first, the deployment of the deep learning model is sorted out, and an overall understanding of the storage requirements of deep learning storage is obtained. Finally, the overall storage structure is coded according to relevant storage theories to implement the deployment work of the model. For different deep learning models, the required tensor information is set personalized, thereby reducing memory occupancy. The entire deep learning deployment step is simple and clear, greatly reducing the workload of the deployment personnel and making the deployment work more convenient and fast.

[0032] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts. In the accompanying drawings:

[0034] Figure 1 is a schematic diagram of the implementation environment of the memory management method for a deep learning model shown in an exemplary embodiment of the present application;

[0035] Figure 2 is a flowchart of the memory management method for a deep learning model shown in an exemplary embodiment of the present application;

[0036] Figure 3 is Figure 2 a flowchart of the microservice architecture communication in the shown embodiment in an exemplary embodiment;

[0037] Figure 4 is Figure 2 a flowchart of the communication interface connection establishment process in the shown embodiment in an exemplary embodiment;

[0038] Figure 5 is Figure 2 a flowchart of the personalized expansion option deployment in the shown embodiment in an exemplary embodiment;

[0039] Figure 6 is a block diagram of the memory management device for a deep learning model shown in an exemplary embodiment of the present application;

[0040] Figure 7 shows a schematic diagram of the structure of a computer system of an electronic device suitable for implementing the embodiments of the present application. Detailed Embodiments

[0041] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for explaining the present invention and not for limiting the protection scope of the present invention.

[0042] It should be noted that the illustrations provided in the following embodiments only schematically illustrate the basic concept of the present invention. Therefore, only the components related to the present invention are shown in the drawings, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0043] In the following description, numerous details are explored to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0044] Figure 1 It is a schematic diagram of the implementation environment of the memory management method of the deep learning model shown in an exemplary embodiment of the present application. Figure 1 The illustrated intelligent terminal 110 can be any memory management terminal device that supports the installation of a deep learning model, such as a smart phone, in-vehicle computer, tablet computer, laptop computer, or wearable device, but is not limited thereto. Figure 1 The illustrated filling server 120 is a navigation server. For example, it can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud information libraries, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and large information and artificial intelligence platforms. There is no limitation here either. The intelligent terminal 110 can communicate with the navigation server 220 through wireless networks such as 3G (third-generation mobile information technology), 4G (fourth-generation mobile information technology), 5G (fifth-generation mobile information technology), etc. There is no limitation here either.

[0045] Please refer to Figure 2 , Figure 2 is a flowchart of the memory management method of the deep learning model shown in an exemplary embodiment of the present application. This method can be applied to Figure 1 the illustrated implementation environment and is specifically executed by the intelligent terminal 110 in this implementation environment. It should be understood that this method can also be applicable to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which this method applies.

[0046] As Figure 2As shown, in an exemplary embodiment, the memory management method of the deep learning model at least includes steps S210 to S240, which are introduced in detail as follows:

[0047] Step S210, obtain multiple storage spaces on the device to be deployed.

[0048] The device to be deployed is the device used to deploy the deep learning model, including but not limited to servers, client computers, etc. Considering that the deep learning model has a certain size, it is first necessary to determine whether the memory of the device to be deployed can meet the size of the deep learning model. When the memory is sufficient to accommodate the deep learning model, it means that the deep learning model can run on the device to be deployed, and the deep learning model can be deployed on the device to be deployed. Further, considering that the deep learning model usually involves processing data such as images, audio, and text, parallel data operations are required. However, since the deep learning model also involves processing serial data, in order to improve the operation efficiency, in an embodiment of the present invention, the types of storage spaces on the device to be deployed include CPU (central processing unit) and GPU (graphics processing unit). Among them, the CPU is good at processing logically complex and serial computing tasks. The CPU can be used to process very complex control logics and does well in prediction branches, out-of-order execution, multi-stage pipelining, etc.; the GPU is good at large-scale data parallel computing tasks, such as textures and dot matrices. It should be noted that the number of CPUs and GPUs is not limited. That is, on the device to be deployed, there can be multiple CPUs and multiple GPUs, or only one or more CPUs, or only one or more GPUs. Relevant deployments can be made according to the specific application of the deep learning model, and no requirements are made here. By allocating appropriate memory for the deep learning model on the CPU and GPU, the data generated by the deep learning model can be exchanged on different storage spaces and run on this storage space after the exchange is completed.

[0049] Step S220, use a preset storage function to establish a communication interface connection between each storage space to achieve data transmission between different storage spaces.

[0050] Since CUDA (Compute Unified Device Architecture) is used as the computing engine for parallel operations on the GPU, that is, information interaction between the CPU and the GPU is achieved through CUDA. Specifically, in the CUDA architecture, CUDA will assign the main program of the deep learning model to the CPU for execution. When encountering data that needs to be processed in parallel, CUDA will compile this part into a program that can be executed by the GPU and transmit it to the GPU for processing. During this process, since it involves different storage spaces, that is, data is transmitted between the GPU and the GPU, the GPU and the CPU, the CPU and the GPU, and the CPU and the CPU, it is necessary to establish communication interface connections between the storage spaces so that data can interact in different storage spaces through the communication interface. Specifically, relevant storage functions in CUDA can be used, with the interfaces of the two storage spaces to be connected as the relevant parameters of the storage function, to establish a communication interface connection between the two storage spaces. Thus, the normal transmission of data between different storage spaces is achieved, and the normal operation of data copy and migration between different storage spaces is realized.

[0051] Step S230, configure the personalized expansion options of the deep learning model based on the algorithm for adding tensor data associated with the deep learning model in each storage space.

[0052] Considering different application scenarios, the corresponding deep learning models are also different. To achieve the accurate deployment of deep learning models, a series of personalized settings need to be performed on the deep learning models to be deployed. Specifically, different tensor data can be added to each storage space according to the specific deep learning model, such as dimension conversion, sorting, screening, difference conversion, etc., to construct personalized expansion options. Among them, the personalized expansion option is an open interface function constructed in the storage space based on the tensor data. When the deep learning model needs to be deployed to the storage space, those skilled in the art can select the corresponding interface function according to the characteristics of the deep learning model to achieve the normal circulation of data. Exemplarily, the deep learning model to be deployed is a model for image processing. The corresponding tensor data can be matrices of different sizes composed of image pixels. The algorithms for the relevant tensor data can be mask processing, sharpening, transposing, sorting, screening, parallel processing, etc. Correspondingly, the involved algorithm functions can be operations such as permute, sort, index, mask, etc. After adding these algorithm functions to the relevant parameters of the deep learning model, they are added to the storage space to form personalized expansion options. When deploying this deep learning model of the image processing type, one or more personalized expansion options related to the deep learning model can be selected to achieve the normal circulation of data. It should be noted that before each deployment of the deep learning model, one or more personalized expansion options related to the deep learning model can be custom-set. It can also be that one or more general personalized expansion options are pre-set, and then according to the characteristics of the current deep learning model, the personalized expansion options suitable for the deep learning model are selected from the set personalized expansion options, and adaptive deletion or addition modifications can be made according to actual needs. The specific method is not limited here.

[0053] Step S240, deploy the deep learning model on the device to be deployed according to the personalized expansion option and the communication interface.

[0054] According to the foregoing, a series of deployment work has been completed for the deep learning model in each storage space. At this time, the storage space is more suitable for deploying the deep learning model. Therefore, when deploying the deep learning model, the interface in the deep learning model program can be set to the corresponding communication interface, and the parameters required for the corresponding tensor algorithm can be set as the parameters of the deep learning. At the same time, modify the corresponding parameters in cuda to ensure the normal circulation of data. Thus, the deployment work of the deep learning model is completed.

[0055] As Figure 3 shown, Figure 3 is Figure 2The flow chart of the microservice architecture communication in the embodiment shown in an exemplary embodiment. In step S210, after obtaining multiple storage spaces on the device to be deployed, the process further includes:

[0056] Step S310: determining whether each storage space can accommodate the deep learning model to be deployed;

[0057] Step S320: If each storage space can accommodate the deep learning model to be deployed, a communication interface connection is established between the storage spaces;

[0058] Step S330: If there is at least one storage space that cannot accommodate the deep learning model to be deployed, delete the external dependent libraries that are not related to the deep learning model in the storage space that cannot accommodate the deep learning model.

[0059] Since the size of the storage space is not unique and unchanging, before deploying the deep learning model to the storage space, you should first determine whether each storage space is greater than or equal to the space occupied by the deep learning model. If the storage space is greater than or equal to the space occupied by the deep learning model, it means that each storage space is sufficient to accommodate the deep learning model and subsequent deployment can be carried out. If one or more storage spaces are smaller than the space occupied by the deep learning model, it means that these storage spaces cannot accommodate the deep learning model and the model cannot be deployed temporarily. Since the storage space comes with multiple external dependent libraries, in order to increase the memory of the storage space. Therefore, it is necessary to delete some external dependent libraries that are not related to the deep learning model in the storage space, retain the external dependent libraries related to the deep learning model, and perform secondary packaging to increase the capacity of the storage space to accommodate the deep learning model.

[0060] like Figure 4 As shown, Figure 4 yes Figure 2 The communication interface connection establishment process in the illustrated embodiment is a flow chart in an exemplary embodiment. In step S220, the communication interface connection between the storage spaces is established using a preset storage function, including:

[0061] Step S410, obtaining a storage function corresponding to each storage space according to the type of the storage space;

[0062] Step S420: Input the interface information of each storage space into the corresponding storage function, and use the storage function to establish a communication interface connection between different storage spaces.

[0063] Since the storage space is divided into CPU and GPU, the storage functions cudamalloc for GPU and malloc for CPU can be obtained according to the different types of storage space. Then, the interface information of the storage space is used as a parameter and input into the corresponding storage function, so as to establish the connection of the communication interface between different storage spaces. Then, relevant cuda storage functions are used to ensure that data copy and migration can be carried out normally from CPU to GPU, from GPU to CPU, from GPU to GPU, and from CPU to CPU.

[0064] As Figure 5 shown, Figure 5 is Figure 2 the flowchart of the deployment of personalized expansion options in the exemplary embodiment shown. In step S230, in each storage space, an algorithm for adding tensor data associated with the deep learning model is added, and the personalized expansion options of the deep learning model are configured, including:

[0065] Step S510, obtaining the tensor data required by the deep learning model based on the characteristics of the deep learning model;

[0066] Step S520, deploying the tensor data in the corresponding storage space in the form of an operator or a function to form a personalized expansion option.

[0067] Since different deep learning models require different tensor data, first, the tensor data required by the current deep learning model needs to be obtained. For example, for an image processing model, the required tensor data may be sorting, screening, and differential transformation of the pixel matrix. For a model that converts high-dimensional text to low-dimensional text, the required tensor data may be high-dimensional data, low-dimensional data, text sequences, etc. Then, each tensor data is written into the storage space in the form of an operator or a function to form a personalized expansion option suitable for the current deep learning model. For example, for the processing methods of tensor data, such as transpose, sorting, screening, and parallel processing, relevant operators or functions are selected correspondingly. These operators or functions can be operations such as permute, sort, index, and mask. Among them, the permute function exchanges the dimensions of the matrix, the sort function sorts the elements, the index function returns data of a specified type, and the mask function performs masking processing on the matrix, etc.

[0068] In an embodiment of the present invention, after obtaining multiple storage spaces on the device to be deployed, the following steps are further included: adding CUDA streams to each storage space to achieve synchronous data transmission. A CUDA stream represents a GPU operation queue, and the operations in this queue will be executed in the order of addition to the stream. A stream can be regarded as a task on the GPU, and different tasks can be executed in parallel. According to the synchronous requirements of different storage spaces, corresponding CUDA streams can be added to achieve synchronous processing and transmission of data, avoiding the problem of inaccurate model processing effects caused by asynchronous data transmission. Further, in an embodiment of the present invention, after adding CUDA streams to each storage space, the following steps are further included: adding zeroing and copying operation interfaces to each storage space. Considering that after a deep learning model runs once, log files and other data will be left in the storage space, occupying memory. Therefore, in order to reduce unnecessary memory occupation, a zeroing interface is added before deploying the deep learning model. When the deep learning model runs in this storage space for a period of time, starting the zeroing interface can zero the memory of the storage space, thereby increasing the capacity of the storage space. The copying interface is for data generated during the operation of the deep learning model to be transmitted between different storage spaces. The data needs to be copied and then transferred to other storage spaces.

[0069] In an embodiment of the present invention, after deploying the deep learning model on the device to be deployed, the following steps are further included: testing the device to be deployed to ensure the normal operation of the deep learning model. In order to verify whether the model can run normally on the device to be deployed, after deploying the deep learning model, relevant testing work needs to be carried out. And a warning message is sent when the test fails, so that the operator can handle relevant problems in a timely manner. Thereby ensuring the soundness of different functions, making the entire deep learning deployment process simple and clear, greatly reducing the workload of the deployment personnel, and making the deployment work more convenient and fast.

[0070] It should be noted that the present invention can be implemented in many different forms and is not limited to the embodiments described herein. For example, it can be based on different operating systems, including but not limited to operating systems such as Linux and QNX, and based on different hardware platforms, including but not limited to hardware platforms such as x86, Xavier, and Orin.

[0071] In the present invention, multiple storage spaces are allocated for the deep learning model on the device to be deployed, and according to a preset storage function, with the relevant interfaces of each storage space as parameters, data transmission between different storage spaces is established, thereby realizing the normal data transmission between different storage spaces. Then, according to the characteristics of the deep learning model to be deployed, algorithms for tensor data related to the characteristics of the deep learning model are added to each storage space, thereby configuring the personalized expansion options of the deep learning model. Then, according to the personalized expansion options and communication interfaces, the deep learning model is deployed on the device to be deployed, thus completing the deployment work of the deep learning model. In the present invention, first, the deployment of the deep learning model is sorted out to have an overall understanding of the storage requirements of deep learning, and finally, the overall storage structure is coded according to relevant storage theories to realize the deployment work of the model. For different deep learning models, the required tensor information is set personalized, thereby reducing the memory occupancy. The entire deep learning deployment step is simple and clear, greatly reducing the workload of the deployment personnel and making the deployment work more convenient and fast. Through the interaction of data between different storage spaces, the related methods lay a foundation for the deployment related to deep learning, and at the same time simplify the large amount of space required by traditional storage, making it more suitable for the deployment of deep learning on embedded models.

[0072] Figure 6 is a block diagram of a memory management device for a deep learning model shown in an exemplary embodiment of the present application. This device can be applied to Figure 2 the implementation environment shown, and is specifically configured in the intelligent terminal 210. This device can also be applicable to other exemplary implementation environments and is specifically configured in other devices. This embodiment does not limit the implementation environment applicable to this device.

[0073] As Figure 6 shown, the exemplary memory management device 600 for a deep learning model includes:

[0074] A storage space acquisition module 601, configured to acquire multiple storage spaces on the device to be deployed; a communication interface deployment module 602, configured to use a preset storage function to establish communication interface connections between each storage space to realize data transmission between different storage spaces; a personalized expansion option deployment module 603, configured to add algorithms for tensor data associated with the deep learning model to each storage space to configure the personalized expansion options of the deep learning model; a deep learning model deployment module 604, configured to deploy the deep learning model on the device to be deployed according to the personalized expansion options and communication interfaces.

[0075] In another exemplary embodiment, the storage space acquisition module 601 includes:

[0076] A storage space judgment unit, configured to judge whether each storage space is greater than or equal to the size of the deep learning model to be deployed;

[0077] A communication interface establishment unit, configured to establish a communication interface connection between each storage space if each storage space is greater than or equal to the size of the deep learning model to be deployed;

[0078] An external dependency library deletion unit, configured to delete the external dependency libraries irrelevant to the deep learning model in the storage space if there is a storage space smaller than the size of the deep learning model to be deployed.

[0079] In another exemplary embodiment, the communication interface deployment module 602 includes:

[0080] A unit storage function acquisition unit, configured to acquire the storage functions corresponding to each storage space according to the type of the storage space;

[0081] An interface connection unit, configured to input the interface information of each storage space into the corresponding storage function, and use the storage function to establish a communication interface connection between different storage spaces.

[0082] In another exemplary embodiment, the personalized extension option deployment module 603 includes:

[0083] A tensor acquisition unit, configured to acquire the tensor data required by the deep learning model based on the characteristics of the deep learning model;

[0084] A personalized extension option acquisition unit, configured to deploy the tensor data in the corresponding storage space in the form of an operator or a function to form a personalized extension option.

[0085] It should be noted that the memory management device of the deep learning model provided in the above embodiments and the memory management method of the deep learning model provided in the above embodiments belong to the same concept. The specific manners in which each module and unit perform operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the memory management device of the deep learning model provided in the above embodiments can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited here either.

[0086] An embodiment of the present application further provides an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the memory management method of the deep learning model provided in each of the above embodiments.

[0087] Figure 7 The figure shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. It should be noted that Figure 7 The computer system 700 of the shown electronic device is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0088] As Figure 7 shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage section 708 into the random access memory (RAM) 703, such as executing the method described in the above embodiments. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0089] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read from it can be installed into the storage section 708 as needed.

[0090] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 709, and / or installed from a removable medium 711. When the computer program is executed by a central processing unit (CPU) 701, various functions defined in the system of the present application are executed.

[0091] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can, for example, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable computer program is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program included on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, and the above-mentioned module, segment of a program, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0093] The units described in the embodiments of the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the units themselves in some cases.

[0094] Another aspect of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor of a computer, it causes the computer to execute the memory management method of the deep learning model as described above. The computer-readable storage medium can be included in the electronic device described in the above embodiments, or can exist alone without being assembled into the electronic device.

[0095] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the memory management method of the deep learning model provided in the above various embodiments.

[0096] The above embodiments are only used to exemplarily illustrate the principles and effects of the present invention, rather than to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A memory management method for a deep learning model, characterized in that For deploying a deep learning model on a device, the method includes: Obtain multiple storage spaces on the device to be deployed; Use a preset storage function to establish communication interface connections between the storage spaces to enable data transmission between different storage spaces; In each storage space, add an algorithm for tensor data associated with the deep learning model and configure personalized expansion options for the deep learning model; Deploy the deep learning model on the device to be deployed according to the personalized expansion options and the communication interface; The step of adding an algorithm for tensor data associated with the deep learning model and configuring personalized expansion options for the deep learning model in each storage space includes: Based on the characteristics of the deep learning model, obtain the tensor data required by the deep learning model; Deploy the tensor data in the corresponding storage space in the form of an operator or a function to form personalized expansion options.

2. The memory management method of the deep learning model according to claim 1, characterized in that After obtaining multiple storage spaces on the device to be deployed, it further includes: Determine whether each storage space can accommodate the deep learning model to be deployed; If each storage space can accommodate the deep learning model to be deployed, establish communication interface connections between the storage spaces; If there is at least one storage space that cannot accommodate the deep learning model to be deployed, delete the external dependency libraries unrelated to the deep learning model in the storage space that cannot accommodate the deep learning model.

3. The memory management method of the deep learning model according to claim 1, wherein The types of the storage spaces include CPU storage spaces and GPU storage spaces.

4. The memory management method of the deep learning model according to claim 3, wherein The step of using a preset storage function to establish communication interface connections between the storage spaces includes: According to the type of the storage space, obtain the storage function corresponding to each storage space; Input the interface information of each storage space into the corresponding storage function and use the storage function to establish communication interface connections between different storage spaces.

5. The memory management method of the deep learning model according to claim 1, characterized in that, After obtaining multiple storage spaces on the device to be deployed, it further includes: adding cuda streams to each storage space to achieve synchronous data transmission.

6. The memory management method of the deep learning model according to claim 1, characterized in that After deploying the deep learning model on the device to be deployed, it further includes: testing the device to be deployed to ensure the normal operation of the deep learning model.

7. A memory management device for a deep learning model, characterized in that, The device includes: A storage space acquisition module configured to obtain multiple storage spaces on the device to be deployed; A communication interface deployment module configured to use a preset storage function to establish communication interface connections between the storage spaces to enable data transmission between different storage spaces; A personalized expansion option deployment module configured to add an algorithm for tensor data associated with the deep learning model and configure personalized expansion options for the deep learning model in each storage space; A deep learning model deployment module configured to deploy the deep learning model on the device to be deployed according to the personalized expansion options and the communication interface; The step of adding an algorithm for tensor data associated with the deep learning model and configuring personalized expansion options for the deep learning model in each storage space includes: Based on the characteristics of the deep learning model, obtain the tensor data required by the deep learning model; Deploy the tensor data in the form of an operator or a function in a corresponding storage space to form a personalized expansion option.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the memory management method of the deep learning model according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, which, when executed by a processor of a computer, causes the computer to execute the memory management method of the deep learning model according to any one of claims 1 to 6.

Images