Weight file sending method and device, storage medium and electronic device

By combining shared memory and hard disk, the file reading interface is used to directly obtain and send weight files from shared memory, which solves the problem of large-scale model weight files occupying a large amount of memory, and realizes efficient memory management and file transfer.

CN119046245APending Publication Date: 2024-11-29BEIJING SENSETIME TECH DEV CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411147447.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The weight files of large models occupy a large amount of memory during training, fine-tuning and inference, and cause excessive memory consumption of hosts. Due to the serious waste of memory space due to multi-process replication, the existing technology has failed to effectively solve this problem.

Method used

Determine whether there is a weight file in the shared memory through the file reading interface, and directly obtain it from the shared memory when it exists and send it to the target acceleration chip, avoid repeated reading from the hard disk, and use the file index table to manage file reference count to ensure the validity of the file Release and delete.

Benefits of technology

It effectively avoids the weight files that each acceleration chip read into the host memory separately, reduces the host memory consumption, reduces hard disk file copying, and improves file reading efficiency and storage space utilization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046245A_ABST
    Figure CN119046245A_ABST
Patent Text Reader

Abstract

The invention discloses a weight file sending method and device, a storage medium and an electronic device.The method comprises the steps that the file name of a first weight file is transmitted into a file reading interface, and whether a file corresponding to the file name exists in a shared memory or not is determined through the file reading interface; and when it is determined that the file corresponding to the file name exists in the shared memory, obtaining the first weight file in the shared memory, and sending the first weight file to a target acceleration chip. By adopting the technical scheme, the problem that in the related technology, each acceleration chip reads the weight file of the same model into the host memory, and consequently the host memory consumption is large is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular, to a method and device for sending a weight file, a storage medium, and an electronic device. Background Art

[0002] As a new productivity in the digital economy era, computing power is a solid foundation for supporting the development of the digital economy. It plays an important role in promoting scientific and technological progress, promoting the digital transformation of industries, and supporting economic and social development. The emergence of large models has epoch-making significance for the development of the artificial intelligence industry, providing more possibilities for the future.

[0003] The weight files of large models contain the parameter values ​​learned during the training process, reflecting the model's understanding of language structure and semantics. These weight files are the result of pre-training of large models and the basis for continued training, fine-tuning, and inference.

[0004] Currently, each deep learning framework provides an interface for model weight files, which can read these files from the hard disk to the host memory or the device memory of the AI ​​acceleration chip, and deserialize them to obtain the parameter weights of each layer of the model.

[0005] The training, fine-tuning, and inference of large models have two common characteristics: first, the model weight files are large; second, large models use multi-process distributed training. Based on the above two points, if each process repeatedly reads the model weight files into the host memory, the host side memory requirements are large.

[0006] This large-scale file transfer not only requires a lot of memory, but also causes serious time overhead. At the same time, since the weight files copied by the 8 processes are the same, there is an obvious waste of memory space.

[0007] Regarding the related technology, each acceleration chip reads the weight file of the same model into the host memory separately, resulting in the problem of large host memory consumption. No effective solution has been proposed yet.

[0008] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention

[0009] Embodiments of the present application provide a method and device for sending a weight file, a storage medium, and an electronic device.

[0010] According to one aspect of an embodiment of the present application, a method for sending a weight file is provided, including: passing the file name of a first weight file into a file reading interface, and determining through the file reading interface whether there is a file corresponding to the file name in the shared memory; when it is determined that there is a file corresponding to the file name in the shared memory, obtaining the first weight file in the shared memory, and sending the first weight file to a target acceleration chip.

[0011] In an exemplary embodiment, determining whether there is a file corresponding to the file name in the shared memory through the file reading interface includes: reading a file index table based on the file reading interface; determining whether there is the file name in the file index table; if the file name exists in the file index table, determining that there is a file corresponding to the file name in the shared memory; if the file name does not exist in the file index table, determining that there is no file corresponding to the file name in the shared memory.

[0012] In an exemplary embodiment, after determining that a file corresponding to the file name exists in the shared memory, the method further includes: determining status information of the first weight file according to the file index table; when the status information indicates a read status, pausing obtaining the first weight file in the shared memory; when the status information indicates a read status, obtaining the first weight file in the shared memory and updating the value of the file reference count field corresponding to the first weight file in the file index table.

[0013] In an exemplary embodiment, after determining that there is no file corresponding to the file name in the shared memory, the method further includes: obtaining a first weight file corresponding to the file name in the hard disk based on the file reading interface, wherein the first weight file stores serialized parameter tensors of the target model; reading the serialized parameter tensors in the first weight file, and deserializing the serialized parameter tensors to obtain deserialized parameter tensors; storing the deserialized parameter tensors in the first weight file; and adding the file name corresponding to the first weight file in the file index table.

[0014] In an exemplary embodiment, after sending the first weight file to the target acceleration chip, the method further includes: when the target acceleration chip releases the first weight file, calling a file release interface to read the file index table through the file release interface, and updating the value of the file reference count field corresponding to the first weight file in the file index table; determining whether the value of the file reference count field meets a preset condition; when it is determined that the value of the file reference count field meets the preset condition, releasing the first weight file in the shared memory, and deleting the information corresponding to the first weight file in the file index table, wherein the information includes: the file name of the first weight file, the value of the file reference count field, and status information.

[0015] In an exemplary embodiment, when the target acceleration chip requests to obtain partial data of the first weight file, the method further includes: passing the file name and weight layer index information of the first weight file into a file reading interface, so as to determine through the file reading interface whether there is a file corresponding to the file name in the shared memory, wherein the first weight file includes: parameter tensors corresponding to multiple weight layers; when it is determined that there is a file corresponding to the file name in the shared memory, obtaining the first weight file in the shared memory, and obtaining the parameter tensor corresponding to the weight layer index information in the first weight file; and sending the parameter tensor corresponding to the weight layer index information to the target acceleration chip.

[0016] According to another aspect of an embodiment of the present application, a device for sending a weight file is also provided, including: a determination module, used to pass the file name of a first weight file into a file reading interface, and determine through the file reading interface whether there is a file corresponding to the file name in the shared memory; a sending module, used to obtain the first weight file in the shared memory when it is determined that there is a file corresponding to the file name in the shared memory, and send the first weight file to the target acceleration chip.

[0017] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for sending the weight file when running.

[0018] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for sending the weight file through the computer program.

[0019] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the method for sending the weight file is implemented.

[0020] Through this application, the file name of the first weight file is passed into the file reading interface, and the file reading interface is used to determine whether there is a file corresponding to the file name in the shared memory; when it is determined that there is a file corresponding to the file name in the shared memory, the first weight file is obtained in the shared memory, and the first weight file is sent to the target acceleration chip. That is, in the embodiment of the present application, when different processes (or acceleration chips) read the same weight file, it is no longer read from the hard disk again, but the file in the shared memory is directly returned through the file reading interface; the above technical solution is adopted to solve the problem that each acceleration chip in the related technology reads the weight file of the same model into the host memory respectively, resulting in a large consumption of the host memory, thereby avoiding the hard disk from copying the same file to the host. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The exemplary embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 It is a hardware structure block diagram of a computer device of a method for sending a weight file according to an embodiment of the present application;

[0023] Figure 2 is a flowchart of a method for sending a weight file according to an embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of a method for reading a weight file in the related art;

[0025] Figure 4 is a schematic diagram of a method for reading a weight file according to an embodiment of the present application;

[0026] Figure 5 It is a structural block diagram of a device for sending a weight file according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms and "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a computer device or a similar computing device. Taking running on a computer device as an example, Figure 1 1 is a hardware structure block diagram of a computer device for a method for sending a weight file according to an embodiment of the present application. Figure 1 As shown, the computer device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the computer device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer device. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the method for sending weight files in the embodiment of the present application, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, which is equivalent to implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] In this embodiment, a method for sending a weight file is provided, which is applied to the above-mentioned computer device. Figure 2 : is a flowchart of a method for sending a weight file according to an embodiment of the present application, and the process includes the following steps:

[0033] Step S202: passing the file name of the first weight file into the file reading interface, and determining through the file reading interface whether there is a file corresponding to the file name in the shared memory;

[0034] Step S204: when it is determined that a file corresponding to the file name exists in the shared memory, the first weight file is obtained from the shared memory, and the first weight file is sent to the target acceleration chip.

[0035] It should be noted that acceleration chips are chips used to accelerate specific computing tasks, usually used to process large-scale data, artificial intelligence, deep learning and other tasks. In the host, acceleration chips are usually used to improve computing efficiency and performance, and reduce processing time and energy consumption.

[0036] The acceleration chips in the host include but are not limited to: graphics processors, field programmable gate arrays, application-specific integrated circuits, and tensor processing units; among them, the acceleration chips can work in conjunction with the host's central processing unit to improve the performance and efficiency of the entire system.

[0037] Through the above steps, the file name of the first weight file is passed into the file reading interface, and the file reading interface is used to determine whether there is a file corresponding to the file name in the shared memory; when it is determined that there is a file corresponding to the file name in the shared memory, the first weight file is obtained in the shared memory, and the first weight file is sent to the target acceleration chip. That is, in the embodiment of the present application, when different processes (or acceleration chips) read the same weight file, it is no longer read from the hard disk again, but the file in the shared memory is directly returned through the file reading interface; the above technical solution is adopted to solve the problem that each acceleration chip in the related technology reads the weight file of the same model into the host memory respectively, resulting in a large consumption of the host memory, thereby avoiding the hard disk from copying the same file to the host.

[0038] Optionally, in order to better understand the above step S202, a method for determining whether there is a file corresponding to the file name in the shared memory is also given: read the file index table based on the file reading interface; determine whether the file name exists in the file index table; if the file name exists in the file index table, determine that there is a file corresponding to the file name in the shared memory; if the file name does not exist in the file index table, determine that there is no file corresponding to the file name in the shared memory.

[0039] In an embodiment of the present application, a file index table is also established in the shared memory, wherein the file index table is used to record the file name, file location, file status information and reference count (which can also be understood as the value of the reference count field of the file) in the shared memory; the names of all the files recorded in the file index table are retrieved to determine whether the file name passed into the file reading interface exists; in the case where the file name passed into the file reading interface exists in the file index table, it is determined that the file corresponding to the file name exists in the shared memory; in the case where the file name passed into the file reading interface does not exist in the file index table, it is determined that the file corresponding to the file name does not exist in the shared memory.

[0040] Optionally, after determining that a file corresponding to the file name exists in the shared memory, it is also necessary to determine whether the first weight file can be read based on the status information of the first weight file, specifically: determining the status information of the first weight file based on the file index table; when the status information indicates a read status, pausing the acquisition of the first weight file in the shared memory; when the status information indicates a read status, acquiring the first weight file in the shared memory and updating the value of the file reference count field corresponding to the first weight file in the file index table.

[0041] That is to say, in an embodiment of the present application, when the status of the first weight file is a read status, it indicates that other processes or acceleration chips are reading the first weight file or other processes are writing the weight file to the shared memory. In order to avoid read conflicts, the acquisition of the first weight file in the shared memory is paused at this time; when the status information indicates a read status, it indicates that only the current process or acceleration chip is reading the first weight file at this time. The first weight file can be directly acquired in the shared memory, and the value of the file reference count field corresponding to the first weight file in the file index table is updated, for example, the value of the file reference count field is increased by 1.

[0042] It should be noted that, when the state of the first weight file is the read state, the first weight file can also be obtained in the shared memory in the following manner:

[0043] 1) monitoring the status information of the first weight file in real time, and obtaining the first weight file in the shared memory when the status information of the first weight file is updated to a read state;

[0044] 2) Setting a timer, obtaining the status information of the first weight file when the timing duration is reached, and obtaining the first weight file in the shared memory when the status information of the first weight file is updated to a read state.

[0045] Optionally, after determining that no file corresponding to the file name exists in the shared memory, the first weight file can be obtained in the following manner: obtaining a second weight file from the hard disk based on the file reading interface, wherein the second weight file stores serialized parameter tensors of the target model; reading the serialized parameter tensors in the second weight file, and deserializing the serialized parameter tensors to obtain deserialized parameter tensors; and storing the deserialized parameter tensors in the first weight file.

[0046] In an embodiment of the present application, the hardware device is opened and the file system is accessed. For example, standard library functions in C language such as fopen() and fread() can be used to read files; the file path of the hardware device is specified through the API function, and the serialized parameter tensor of the target model is obtained; the serialized parameter tensor is deserialized to obtain the deserialized parameter tensor, and the deserialized parameter tensor is stored in the first weight file; when the first weight file is obtained, the first weight file is stored in the shared memory.

[0047] In the embodiment of the present invention, by storing files in a shared memory and a hard disk in combination, the file reading efficiency can be improved.

[0048] It should be noted that when calling the file reading interface, the path of the second weight file on the hard disk will also be passed into the file reading interface. Therefore, after determining that there is no file corresponding to the file name in the shared memory, the file reading interface obtains the second weight file on the hard disk based on the passed-in path.

[0049] Optionally, after storing the deserialized parameter tensor in the first weight file, it also includes: adding a file name corresponding to the first weight file in the file index table, and updating the status information corresponding to the first weight file to a read status.

[0050] When the first weight file is obtained, the information of the first weight file is added to the file index table, and the status information of the first weight file is updated to a read status, so that other processes can obtain the first weight file based on the file index table.

[0051] After creating the file index table, you can quickly locate the required file based on keywords or file names, saving retrieval time. In addition, the file index table can prevent users from storing multiple identical files in their computers, saving storage space.

[0052] Optionally, as shown below, a method for releasing a weight file is also provided in an embodiment of the present application: when the target acceleration chip releases the first weight file, a file release interface is called to read the file index table through the file release interface, and the value of the file reference count field corresponding to the first weight file in the file index table is updated; it is determined whether the value of the file reference count field meets a preset condition; when it is determined that the value of the file reference count field meets the preset condition, the first weight file is released in the shared memory, and the information corresponding to the first weight file in the file index table is deleted, wherein the information includes: the file name of the first weight file, the value of the file reference count field, and status information.

[0053] In an embodiment of the present application, a file release interface is defined. When the target acceleration chip releases the first weight file, the weight file is released by calling the file release interface.

[0054] After releasing the first weight file, the information corresponding to the first weight file in the file index table needs to be updated, for example: the value of the file reference count field corresponding to the first weight file is reduced by one; and when the value of the file reference count field corresponding to the first weight file meets the preset conditions, for example, when the value of the file reference count field corresponding to the first weight file is 0, the first weight file is released in the shared memory, and the information corresponding to the first weight file in the file index table is deleted.

[0055] In the case where the first weight file is not applied, timely deleting the first weight file in the shared memory can reduce the resource usage of the shared memory and improve system performance and response speed; and timely updating the information in the index table can ensure the accuracy and completeness of the file and avoid file loss or confusion.

[0056] Since the target acceleration chip may obtain part of the weight file, when the target acceleration chip obtains part of the weight file, it is obtained in the following manner: the file name and weight layer index information of the first weight file are passed into the file reading interface, so as to determine through the file reading interface whether there is a file corresponding to the file name in the shared memory, wherein the first weight file includes: parameter tensors corresponding to multiple weight layers; when it is determined that there is a file corresponding to the file name in the shared memory, the first weight file is obtained in the shared memory, and the parameter tensor corresponding to the weight layer index information is obtained in the first weight file; the parameter tensor corresponding to the weight layer index information is sent to the target acceleration chip.

[0057] That is, when obtaining a partial weight file, it is also necessary to provide weight layer index information, and pass the file name and weight layer index information of the first weight file into the file reading interface, which obtains the partial weight file based on the file name and weight layer index information of the first weight file.

[0058] Furthermore, the offset and length of the file to be read can be calculated according to the location and size of the partial weight file to be obtained, and the partial weight file can be obtained through memory mapping.

[0059] Obviously, the above-described embodiments are only part of the embodiments of the present application, not all of the embodiments. In order to better understand the method for sending the above-mentioned weight file, the above-mentioned process is described below in conjunction with the embodiments, but it is not intended to limit the technical solution of the embodiments of the present application, specifically:

[0060] In related technologies, such as Figure 3 As shown, each process reads the model's weight file into the host memory and reads the weight file separately from the host memory. At this time, the memory requirement on the host side is relatively large.

[0061] The embodiment of the present application provides a set of high-performance, low-memory weight file reading interfaces that are not sensitive to the large model algorithm layer, are independent of the model parallel strategy, and can avoid repeated copying from the hard disk to the host memory. Figure 4 As shown in the figure, the model weight file is shared among multiple processes on the same host, and the life cycle of the weight file is managed by the interface. Algorithm engineers do not need to do extra work and can read and release the weight file according to existing habits.

[0062] The embodiment of the present application provides a method for sending a weight file. When multiple processes of the same host perform a weight file reading operation, the file reading interface is used to ensure that only one weight file is read into the host shared memory, and a file index table is established to record the file name, file location, file status and reference count. When different processes read the same weight file, it is no longer read from the hard disk again, but directly returns the file in the shared memory.

[0063] In the embodiment of the present application, the following two interfaces are defined:

[0064] 1. share_ckpt_l oad() weight file reading interface;

[0065] 2. share_ckpt_re l ease() weight file release interface.

[0066] When the user reads the weight file through the weight file reading interface, the share_ckpt_l oad() interface is called, the path and file name of the target weight file (equivalent to the first weight file in the above embodiment) are passed to the weight file reading interface, and the deserialized parameter tensor in the target weight file is obtained.

[0067] Optionally, when the user reads the weight file through the weight file reading interface, the file index table is retrieved through the share_ckpt_l oad() interface; if it is determined through the file index table that the target weight file does not exist, the file is read from the hard disk to the host memory, and the file is added to the file index list and the status is set to "reading".

[0068] After the file is successfully read into the host memory, the status is set to "read", the deserialized weight parameter is returned to the user, and the file reference count is increased by 1.

[0069] If the file status is "Reading", wait; when the file status is updated to "Read", return the deserialized weight parameter to the user and increase the file reference count by 1;

[0070] When the file status is "read", the deserialized weight parameter is returned to the user and the file reference count is increased by 1.

[0071] After the user reads the weight file through the weight file reading interface, when releasing the weight file, the corresponding share_ckpt_release() interface needs to be called to release the corresponding weight file. Specifically:

[0072] The share_ckpt_re l ease() interface first retrieves the file index list and decrements the reference count of the weight file; if the count is zero after decrement, the weight file is released in the shared memory and the record is deleted from the file index table.

[0073] It should be noted that there is a protection mechanism for the file index table in the embodiment of the present application. Specifically, the file index table, especially the reference count, is a critical resource and needs to be protected by mutual exclusive access between processes. The size of the shared memory can be set and modified according to demand, 70B large language model 200g.

[0074] Taking the training of 65B llama model on Cambrian chip as an example, if the traditional method is used, when training or fine-tuning on a single machine with 8 cards, reading the model weight file requires 65B×2Bytes host memory / parameters×8 cards, which requires 1040G host memory. Assuming that the host memory is 1T (1T=1024G<1040G), llama65B training cannot be performed. After passing this application, only 65B×2Bytes host memory / parameters 130G host memory is required.

[0075] That is, through the embodiments of the present application, the host memory overhead is greatly reduced and the transmission time of the weight file is shortened. At the same time, the interface user only needs to call the file reading interface and does not need to care about the file processing details, so that the user can focus on the large model itself. Compared with the model reading optimization method in the related art, the present application also breaks the limitation that reading the weight file requires using the same parallel strategy as the weight file.

[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0077] In this embodiment, a device for sending a weight file is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0078] Figure 5 is a structural block diagram of a device for sending a weight file according to an embodiment of the present application, the device comprising:

[0079] A determination module 52, configured to pass the file name of the first weight file into a file reading interface, and determine through the file reading interface whether there is a file corresponding to the file name in the shared memory;

[0080] The sending module 54 is used to obtain the first weight file in the shared memory and send the first weight file to the target acceleration chip when it is determined that a file corresponding to the file name exists in the shared memory.

[0081] Through the device in the embodiment of the present application, the file name of the first weight file is passed into the file reading interface, and the file reading interface is used to determine whether there is a file corresponding to the file name in the shared memory; when it is determined that there is a file corresponding to the file name in the shared memory, the first weight file is obtained in the shared memory, and the first weight file is sent to the target acceleration chip. That is, in the embodiment of the present application, when different processes (or acceleration chips) read the same weight file, it is no longer read from the hard disk again, but the file in the shared memory is directly returned through the file reading interface; the above technical solution is adopted to solve the problem that each acceleration chip in the related technology reads the weight file of the same model into the host memory respectively, resulting in a large consumption of the host memory, thereby avoiding the hard disk from copying the same file to the host.

[0082] In an exemplary embodiment, the determination module 52 is used to read the file index table based on the file reading interface; determine whether the file name exists in the file index table; if the file name exists in the file index table, determine that a file corresponding to the file name exists in the shared memory; if the file name does not exist in the file index table, determine that a file corresponding to the file name does not exist in the shared memory.

[0083] In an exemplary embodiment, the above-mentioned device also includes: an acquisition module, used to determine the status information of the first weight file according to the file index table; when the status information indicates a read state, pausing the acquisition of the first weight file in the shared memory; when the status information indicates a read state, acquiring the first weight file in the shared memory and updating the value of the file reference count field corresponding to the first weight file in the file index table.

[0084] In an exemplary embodiment, the acquisition module is also used to acquire a second weight file in the hard disk based on the file reading interface, wherein the second weight file stores a serialized parameter tensor of the target model; read the serialized parameter tensor in the second weight file, and deserialize the serialized parameter tensor to obtain a deserialized parameter tensor; store the deserialized parameter tensor in the first weight file; and add the file name corresponding to the first weight file in the file index table.

[0085] In an exemplary embodiment, an update module is used to call a file release interface when the target acceleration chip releases the first weight file, so as to read a file index table through the file release interface and update the value of a file reference count field corresponding to the first weight file in the file index table; determine whether the value of the file reference count field meets a preset condition; and when it is determined that the value of the file reference count field meets the preset condition, release the first weight file in the shared memory and delete the information corresponding to the first weight file in the file index table, wherein the information includes: the file name of the first weight file, the value of the file reference count field, and status information.

[0086] In an exemplary embodiment, a sending module is used to pass the file name and weight layer index information of a first weight file into a file reading interface to determine whether there is a file corresponding to the file name in the shared memory through the file reading interface, wherein the first weight file includes: parameter tensors corresponding to multiple weight layers; when it is determined that there is a file corresponding to the file name in the shared memory, the first weight file is obtained in the shared memory, and the parameter tensor corresponding to the weight layer index information is obtained in the first weight file; the parameter tensor corresponding to the weight layer index information is sent to the target acceleration chip.

[0087] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0088] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0089] S1: passing the file name of the first weight file into a file reading interface, and determining through the file reading interface whether there is a file corresponding to the file name in the shared memory;

[0090] S2: When it is determined that a file corresponding to the file name exists in the shared memory, the first weight file is obtained from the shared memory, and the first weight file is sent to the target acceleration chip.

[0091] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0092] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0093] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0094] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0095] S1: passing the file name of the first weight file into a file reading interface, and determining through the file reading interface whether there is a file corresponding to the file name in the shared memory;

[0096] S2: When it is determined that a file corresponding to the file name exists in the shared memory, the first weight file is obtained from the shared memory, and the first weight file is sent to the target acceleration chip.

[0097] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0098] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0099] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0100] An embodiment of the present application also provides a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0101] Optionally, in this embodiment, the computer program product may be configured to store a computer program for performing the following steps:

[0102] S1: passing the file name of the first weight file into a file reading interface, and determining through the file reading interface whether there is a file corresponding to the file name in the shared memory;

[0103] S2: When it is determined that a file corresponding to the file name exists in the shared memory, the first weight file is obtained from the shared memory, and the first weight file is sent to the target acceleration chip.

[0104] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0105] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0106] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for sending a weight file, characterized in that: include: The file name of the first weight file is passed into the file reading interface, and the file reading interface is used to determine whether there is a file corresponding to the file name in the shared memory; When it is determined that a file corresponding to the file name exists in the shared memory, the first weight file is obtained in the shared memory, and the first weight file is sent to the target acceleration chip.

2. The method for sending a weight file according to claim 1, characterized in that: Determining whether there is a file corresponding to the file name in the shared memory through the file reading interface includes: Reading a file index table based on the file reading interface; Determine whether the file name exists in the file index table; If the file name exists in the file index table, determining that a file corresponding to the file name exists in the shared memory; When the file name does not exist in the file index table, it is determined that the file corresponding to the file name does not exist in the shared memory.

3. The method for sending a weight file according to claim 2, characterized in that: After determining that a file corresponding to the file name exists in the shared memory, the method further includes: Determine the status information of the first weight file according to the file index table; When the state information indicates a read state, pausing obtaining the first weight file in the shared memory; When the status information indicates that the state has been read, the first weight file is obtained in the shared memory, and the value of the file reference count field corresponding to the first weight file in the file index table is updated.

4. The method for sending a weight file according to claim 2, characterized in that: After determining that there is no file corresponding to the file name in the shared memory, the method further includes: obtaining a second weight file corresponding to the file name in the hard disk based on the file reading interface, wherein the second weight file stores a serialized parameter tensor of the target model; reading the serialized parameter tensor in the second weight file, and deserializing the serialized parameter tensor to obtain a deserialized parameter tensor; Storing the deserialized parameter tensor to the first weight file; Add the file name corresponding to the first weight file in the file index table.

5. The method for sending a weight file according to claim 2, characterized in that: After sending the first weight file to the target acceleration chip, the method further includes: In the case where the target acceleration chip releases the first weight file, calling a file release interface to read a file index table through the file release interface, and updating a value of a file reference count field corresponding to the first weight file in the file index table; Determine whether the value of the file reference count field meets a preset condition; When it is determined that the value of the file reference count field meets the preset conditions, the first weight file is released in the shared memory, and the information corresponding to the first weight file in the file index table is deleted, wherein the information includes: the file name of the first weight file, the value of the file reference count field, and status information.

6. The method for sending a weight file according to claim 1, characterized in that: In the case where the target acceleration chip requests to obtain partial data of the first weight file, the method further includes: The file name and weight layer index information of the first weight file are passed into the file reading interface to determine whether there is a file corresponding to the file name in the shared memory through the file reading interface, wherein the first weight file includes: parameter tensors corresponding to multiple weight layers; when it is determined that there is a file corresponding to the file name in the shared memory, the first weight file is obtained in the shared memory, and the parameter tensor corresponding to the weight layer index information is obtained in the first weight file; The parameter tensor corresponding to the weight layer index information is sent to the target acceleration chip.

7. A device for sending a weight file, characterized in that: include: A determination module, used to pass the file name of the first weight file into the file reading interface, and determine whether there is a file corresponding to the file name in the shared memory through the file reading interface; A sending module is used to obtain the first weight file in the shared memory and send the first weight file to the target acceleration chip when it is determined that a file corresponding to the file name exists in the shared memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Full-duplex communication between double courses, and mutual protection method and system between double courses

    CN103441831A

  • Method and system for opening and closing file locally through client side based on catalogue authorization

    CN104021137A

  • Transparent encryption and decryption method and system based on multi-thread environment and storage medium

    CN111832054A

  • Memory sharing method and device

    CN112906075A

  • Video memory allocation processing method, device, equipment and system

    CN114168316A