Model acquisition method and device and electronic equipment
By using content identifier sets to pull and combine subfiles of large models in parallel in the local area network, the problem of slow transmission speed of large models in the local area network is solved, and rapid acquisition and verification are achieved, improving the efficiency and reliability of the system.
Patent Information
- Application Number
- CN202411319708.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2025-01-03
AI Technical Summary
The rapid deployment and access of large models in LANs is limited by the transmission speed caused by the huge data volume, so how to efficiently share large model data has become a challenge.
By connecting the central node and multiple edge nodes in the LAN, using the content identifier set to find sub-files of the big model in parallel, recombining the model, and verifying file integrity through the hash algorithm, achieving rapid acquisition and verification of the big model .
It significantly reduces the time when the client waits for the complete model to be transmitted, reduces network latency, enhances the reliability and scalability of the system, and ensures the continuity and availability of the system.
Smart Images

Figure CN119248723A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a model acquisition method, device and electronic equipment. Background Art
[0002] Since large models often have large data volumes, it is difficult to achieve fast transmission. This limits the rapid deployment and access of large models within the local area network. How to efficiently share large model data has become a challenge. Summary of the invention
[0003] The embodiments of the present application provide a model acquisition method, device and electronic device for improving the acquisition speed and efficiency of large models in a local area network, while optimizing the utilization of network bandwidth and enhancing the reliability and scalability of the system.
[0004] In a first aspect, an embodiment of the present application provides a model acquisition method, connecting a central node and multiple edge nodes in the same local area network, the method comprising:
[0005] When a client at any edge node receives a user request to obtain a target large model, it requests the central node to obtain a set of content identifiers of sub-files of the target large model. The central node is used to store content identifiers of sub-files of the large model in the local area network. Different content identifiers are used to identify different sub-files in the relevant files of the large model.
[0006] In the local area network, based on the content identifier set, find out whether other edge nodes have stored different sub-files of the target large model;
[0007] If there is at least one edge node storing different sub-files of the target large model, pull different sub-files of the target large model from each edge node in parallel;
[0008] The different sub-files of the target large model are reassembled to obtain the target large model.
[0009] Optionally, when any of the above edge nodes uploads relevant files of a new large model, it sends a content identifier set of the new large model to the central node, and the central node sends a publishing message of the new large model to other edge nodes, where the publishing message includes the content identifier set.
[0010] Optionally, after pulling different sub-files of the target large model from each edge node in parallel, the method further includes:
[0011] Verify that the subfile is complete based on the content identifier.
[0012] Optionally, the above-mentioned verification of whether the sub-file is complete based on the content identifier specifically includes:
[0013] Calculate a first hash value of the subfile using a preset hash algorithm;
[0014] comparing the first hash value to a second hash value in the content identifier;
[0015] If the first hash value is the same as the second hash value, it is determined that the subfile is complete.
[0016] Optionally, the above-mentioned recombining different sub-files of the target large model to obtain the target large model specifically includes:
[0017] According to the merging logic of the sub-files, the different sub-files of the target large model are recombined to obtain the target large model.
[0018] Optionally, after recombining the different sub-files of the target large model to obtain the target large model, the method further includes:
[0019] Whether the target big model is complete is verified based on the big model content identifier, where the big model content identifier is used to identify different big models.
[0020] Optionally, the above-mentioned verification of whether the target big model is complete based on the big model content identifier specifically includes:
[0021] Calculate the third hash value of the target large model using a preset hash algorithm;
[0022] comparing the third hash value to a fourth hash value in the large model content identifier;
[0023] If the third hash value is the same as the fourth hash value, it is determined that the target large model is complete.
[0024] In a second aspect, an embodiment of the present application provides a model acquisition device, which connects a central node and multiple edge nodes in the same local area network, and the device includes:
[0025] The transceiver module is used to request the central node to obtain the content identifier set of the sub-files of the target large model when receiving a user request to obtain the target large model. The central node is used to store the content identifiers of the sub-files of the large model in the local area network. Different content identifiers are used to identify different sub-files in the related files of the large model.
[0026] A processing module, used for searching in the local area network whether other edge nodes have stored different sub-files of the target large model based on the content identifier set;
[0027] The transceiver module is further used to pull the different sub-files of the target large model from each edge node in parallel if there is at least one edge node storing different sub-files of the target large model;
[0028] The processing module is also used to recombine different sub-files of the target large model to obtain the target large model.
[0029] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the processor implements any one of the model acquisition methods in the first aspect.
[0030] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the model acquisition method of any one of the first aspects is implemented.
[0031] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement a model acquisition method as described in any one of the first aspects above.
[0032] The technical effects brought about by any one of the implementation methods in the second to fifth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A flow chart of a model acquisition method provided in an embodiment of the present application;
[0034] Figure 2 An exemplary flow chart of a model acquisition provided in an embodiment of the present application;
[0035] Figure 3 A schematic diagram of a device for obtaining a model provided in an embodiment of the present application;
[0036] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0038] The application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.
[0039] Since large models often have large data volumes, it is difficult to achieve fast transmission. This limits the rapid deployment and access of large models in a local area network. How to efficiently share large model data has become a challenge. In order to solve the above problems, an embodiment of the present application provides a model acquisition method. The method is applied to a local area network connecting a central node and multiple edge nodes. When a client where any edge node is located receives a user request to obtain a target large model, it requests the central node to obtain a set of content identifiers of sub-files of the target large model. The central node is used to store the content identifiers of the sub-files of the large model in the local area network, and different content identifiers are used to identify different sub-files in the relevant files of the large model. In the local area network, based on the content identifier set, it is searched whether the edge node has stored different sub-files of the target large model. If there is at least one edge node that has stored different sub-files of the target large model, the different sub-files of the target large model are pulled in parallel from each edge node. The different sub-files of the target large model are recombined to obtain the target large model.
[0040] In this way, by pulling different sub-files of the target large model in parallel, the time the client waits for the full model to be transmitted can be significantly reduced. Since sub-files are usually much smaller than full models, they can be transmitted over the network more quickly, especially when transmitted directly between edge nodes, which can reduce latency. At the same time, by splitting large models into sub-files and storing them in multiple edge nodes, it is easier to adapt to network environments of different sizes and needs. As the number of edge nodes increases, the storage and computing power of the system can be expanded linearly, enhancing the flexibility of system use. At the same time, since the sub-files are stored in a decentralized manner, even if an edge node fails, other edge nodes can still provide the required sub-files, thereby ensuring the continuity and availability of the system. In addition, by searching and verifying through content identifiers, it can be ensured that the transmitted sub-files are correct, further enhancing the reliability of the system.
[0041] like Figure 1 As shown, a flow chart of a model acquisition method provided by an embodiment of the present application is applied to a local area network connecting a central node and multiple edge nodes, and specifically may include the following steps.
[0042] Step S101: When a client at any edge node receives a user request for obtaining a target large model, the client requests the central node to obtain a set of content identifiers of sub-files of the target large model.
[0043] The central node is used to store content identifiers of sub-files of the large model in the local area network. Different content identifiers are used to identify different sub-files in the related files of the large model.
[0044] It is understandable that the central node can serve as the startup node of the local area network, providing an initial connection point for new edge nodes that have just joined the network. After the new edge node is started, it will first try to establish a connection with this central node, and quickly integrate into the entire local area network through it. In this way, the network access process of new edge nodes is greatly simplified, and the accessibility and usability of the network are improved. At the same time. For edge nodes that are connected to the central node for the first time, the central node can send a table of the currently existing large model to the edge node, so that the edge node can quickly understand and obtain the required data resources later.
[0045] Optionally, when any edge node uploads a file related to a new large model, it sends a content identifier set of the new large model to the central node, and the central node sends a release message of the new large model to other edge nodes, wherein the release message includes the content identifier set.
[0046] It is understandable that the set of content identifiers generated by uploading the same large model multiple times is the same.
[0047] For example, when a new large model is ready to be deployed to the local area network, the edge node uploads the relevant files of the new large model to the central node. The relevant files may include model parameters, weights, architecture description, metadata and other information.
[0048] The edge node may also send a content identifier set to the central node. The content identifier set includes content identifiers of subfiles of the related files of the new large model. For example, the content identifier may include a hash value of a preset character length generated according to the file content using a preset hash algorithm.
[0049] It can be understood that, since the content identifier is generated based on the content of the sub-file, the content identifier can be used to uniquely identify the sub-file.
[0050] After receiving the content identifier set sent by the edge node, the central node can send a new large model publishing message to the edge node in the local area network. The publishing message is used to notify the edge node in the local area network that a new large model is available. The publishing message includes a content identifier set for identifying the sub-files of the new large model. The edge node receives the publishing message sent by the central node. The user can select the large model of interest on the client and click to download.
[0051] Optionally, edge nodes can use a preset encryption algorithm to protect model files and restrict access rights, thereby improving data security.
[0052] Step S102: In the local area network, based on the content identifier set, search other edge nodes to see whether they have stored different sub-files of the target large model.
[0053] For example, the edge node first checks whether the local database already contains the sub-files specified in the content identifier set. If the edge node does not find all the required sub-files in the local database, it initiates a query to other edge nodes in the local area network to see whether they store the required sub-files.
[0054] Step S103: If there is at least one edge node that has stored different sub-files of the target large model, pull different sub-files of the target large model from each edge node in parallel.
[0055] For example, the edge node initiates a data request to each edge node containing the required sub-file. The data requests may be sent simultaneously or not simultaneously. After receiving the data request, at least one edge node storing different sub-files of the target large model, each edge node storing different sub-files of the target large model sends the specified sub-file to the edge node.
[0056] In this way, pulling different sub-files of the target large model from each edge node in parallel can ensure that data pulling is done simultaneously without interfering with each other, which can significantly reduce the time the client has to wait for the complete model to be transmitted.
[0057] Optionally, after pulling different sub-files of the target large model from each edge node in parallel, the integrity and correctness of each sub-file can be verified to ensure that they match the expected version of the target large model.
[0058] In this way, by verifying whether the received sub-file is complete through the content identifier, the integrity and authenticity of the sub-file received during the download process can be ensured, the reliability of data transmission can be improved, and the overall security and credibility of the local area network can be enhanced.
[0059] For example, the integrity of the subfile can be verified based on the content identifier. Specifically, a preset hash algorithm is used to calculate a first hash value of the subfile. The first hash value is compared with a second hash value in the content identifier. If the first hash value is the same as the second hash value, it is determined that the subfile is intact.
[0060] Since the content identifier is a unique identifier of the content of the data block, only when the content of the data has not been tampered with at all, the first hash value calculated using the preset hash algorithm will match the hash value in the content identifier.
[0061] If the first hash value is the same as the second hash value, it means that the data has maintained integrity during the transmission process, and the subfile can continue to be processed or stored. If the first hash value is different from the second hash value, it means that the subfile may have been damaged during the transmission process, and the edge node can request the edge node storing the subfile to send the subfile again, or obtain the same subfile from other edge nodes storing the subfile.
[0062] Step S104: Recombining different sub-files of the target large model to obtain the target large model.
[0063] In an optional embodiment, after the integrity of the sub-files is determined, different sub-files of the target large model are recombined according to the merging logic of the sub-files to obtain the target large model.
[0064] For example, the merging logic between sub-files may be concatenation, which connects the contents of multiple sub-files in sequence to form a complete file.
[0065] For another example, the merging logic between sub-files may be merging fields, merging the same or related fields from different sub-files into a new data structure.
[0066] Optionally, in order to verify the integrity of the target big model and ensure that the target big model has not been tampered with or damaged after the sub-files are reassembled to obtain the target big model, the target big model can be verified to be complete based on the big model content identifier after the different sub-files of the target big model are reassembled to obtain the target big model, and the big model content identifier is used to identify different big models.
[0067] For example, a third hash value of the target large model is calculated using a preset hash algorithm. The third hash value is compared with the fourth hash value in the large model content identifier. If the third hash value is the same as the fourth hash value, the target large model is determined to be complete. If the hash values do not match, it indicates that there may be a problem in a certain link of the target large model, and the integrity of the sub-file or the merging logic needs to be rechecked.
[0068] Optionally, the local area network can use the InterPlanetary File System (IPFS) for network communication. IPFS is designed to create a network transmission protocol for persistent and distributed storage and sharing of files. Users can use the IPFS network to transmit their personalized large models without having to build and maintain storage services themselves.
[0069] Since the IPFS network is decentralized, distributed, and highly fault-tolerant, adding a large model file to the IPFS network can obtain a unique, content-based hash value, which makes it easier to retrieve and download the large model file through this hash value, and also increases the security and privacy of the data.
[0070] like Figure 2 As shown, an exemplary flow chart of a model acquisition method provided in an embodiment of the present application may include the following steps:
[0071] Step S201: When a client at any edge node receives a user request to obtain a target large model, it requests the central node to obtain a set of content identifiers of sub-files of the target large model;
[0072] Step S202: In the local area network, based on the content identifier set, find out whether other edge nodes have stored different sub-files of the target large model;
[0073] Step S203: If there is at least one edge node storing different sub-files of the target large model, pull different sub-files of the target large model from each edge node in parallel;
[0074] Step S204: Calculate a first hash value of the subfile using a preset hash algorithm, compare the first hash value with the second hash value in the content identifier, and if the first hash value is the same as the second hash value, determine that the subfile is complete;
[0075] Step S205: Recombine different sub-files of the target large model according to the merging logic of the sub-files to obtain the target large model;
[0076] Step S206: Use a preset hash algorithm to calculate the third hash value of the target large model, compare the third hash value with the fourth hash value in the large model content identifier, and if the third hash value is the same as the fourth hash value, determine that the target large model is complete.
[0077] Figure 3 The schematic diagram of the structure of a model acquisition device provided in an embodiment of the present application is shown in FIG. The device is applied to a local area network connecting a central node and multiple edge nodes. Figure 3 As shown, the device includes: a transceiver module 301 and a processing module 302.
[0078] The transceiver module 301 is used to request the central node to obtain the content identifier set of the sub-files of the target large model when receiving a user request to obtain the target large model. The central node is used to store the content identifiers of the sub-files of the large model in the local area network. Different content identifiers are used to identify different sub-files in the related files of the large model.
[0079] The processing module 302 is used to search other edge nodes in the local area network based on the content identifier set to see whether they have stored different sub-files of the target large model;
[0080] The transceiver module 301 is further configured to pull different sub-files of the target large model from each edge node in parallel if there is at least one edge node storing different sub-files of the target large model;
[0081] The processing module 302 is also used to recombine different sub-files of the target large model to obtain the target large model.
[0082] Optionally, when any of the above edge nodes uploads relevant files of a new large model, it sends a content identifier set of the new large model to the central node, and the central node sends a publishing message of the new large model to other edge nodes, where the publishing message includes the content identifier set.
[0083] Optionally, after pulling different sub-files of the target large model from each edge node in parallel, the processing module 302 is further used to:
[0084] Verify that the subfile is complete based on the content identifier.
[0085] Optionally, the above-mentioned verification of whether the subfile is complete based on the content identifier, the processing module 302 is specifically used for:
[0086] Calculate a first hash value of the subfile using a preset hash algorithm;
[0087] comparing the first hash value to a second hash value in the content identifier;
[0088] If the first hash value is the same as the second hash value, it is determined that the subfile is complete.
[0089] Optionally, different sub-files of the target large model are recombined to obtain the target large model, and the processing module 302 is specifically used for:
[0090] According to the merging logic of the sub-files, the different sub-files of the target large model are recombined to obtain the target large model.
[0091] Optionally, after the target large model is obtained by recombining different sub-files of the target large model, the processing module 302 is further used to:
[0092] Whether the target big model is complete is verified based on the big model content identifier, where the big model content identifier is used to identify different big models.
[0093] Optionally, based on the large model content identifier, verify whether the target large model is complete, and the processing module 302 is specifically used to:
[0094] Calculate the third hash value of the target large model using a preset hash algorithm;
[0095] comparing the third hash value to a fourth hash value in the large model content identifier;
[0096] If the third hash value is the same as the fourth hash value, it is determined that the target large model is complete.
[0097] Based on the same technical concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the functions of the aforementioned model acquisition device.
[0098] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0099] At least one processor 401, and a memory 402 connected to the at least one processor 401. The specific connection medium between the processor 401 and the memory 402 is not limited in the embodiment of the present application. Figure 4 In the example, the processor 401 and the memory 402 are connected via the bus 400. The bus 400 is Figure 4 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 400 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 401 can also be called a controller, and there is no limitation on the name.
[0100] In the embodiment of the present application, the memory 402 stores instructions that can be executed by at least one processor 401. The at least one processor 401 can execute a model acquisition method discussed above by executing the instructions stored in the memory 402. The processor 401 can implement Figure 3 The functions of each module in the device shown.
[0101] Among them, the processor 401 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 402 and calling the data stored in the memory 402, the various functions of the device and process data, the device can be monitored as a whole.
[0102] In one possible design, the processor 401 may include one or more processing units, and the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a driver interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.
[0103] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a model acquisition method disclosed in the embodiments of the present application can be directly embodied as a hardware processor execution, or a combination of hardware and software modules in the processor.
[0104] The memory 402 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 402 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 402 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 402 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0105] By programming the processor 401, the code corresponding to the model acquisition method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1A model acquisition method of the embodiment shown. How to design and program the processor 401 is a technology well known to those skilled in the art and will not be described in detail here.
[0106] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.
[0107] An embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a model acquisition method in the above embodiment.
[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 Flow or flows and / or boxes Figure 1 A device that performs the functions specified in a box or multiple boxes.
[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Flow or flows and / or boxes Figure 1 The functions specified in one or more boxes.
[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 Flow or flows and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0112] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A model acquisition method, characterized in that: Connecting a central node and multiple edge nodes in the same local area network, the method includes: When a client at any edge node receives a user request to obtain a target large model, it requests the central node to obtain a set of content identifiers of sub-files of the target large model. The central node is used to store content identifiers of sub-files of the large model in the local area network. Different content identifiers are used to identify different sub-files in the relevant files of the large model. In the local area network, searching other edge nodes based on the content identifier set whether they have stored different sub-files of the target large model; If there is at least one edge node storing different sub-files of the target large model, pulling different sub-files of the target large model from each edge node in parallel; The different sub-files of the target large model are reassembled to obtain the target large model.
2. The method according to claim 1, characterized in that When any edge node uploads the relevant files of the new large model, it sends the content identifier set of the new large model to the central node, and the central node sends a publishing message of the new large model to other edge nodes, and the publishing message includes the content identifier set.
3. The method according to claim 1, characterized in that After pulling different sub-files of the target large model from each edge node in parallel, the method further includes: The integrity of the subfile is verified based on the content identifier.
4. The method according to claim 3, characterized in that The verifying whether the subfile is complete based on the content identifier specifically includes: Calculating a first hash value of the subfile using a preset hash algorithm; comparing the first hash value to a second hash value in the content identifier; If the first hash value is the same as the second hash value, it is determined that the subfile is complete.
5. The method according to claim 1, characterized in that The step of recombining different sub-files of the target large model to obtain the target large model specifically includes: According to the merging logic of the sub-files, the different sub-files of the target large model are recombined to obtain the target large model.
6. The method according to claim 1, characterized in that After recombining the different sub-files of the target large model to obtain the target large model, the method further includes: Verifying whether the target big model is complete based on the big model content identifier, wherein the big model content identifier is used to identify different big models.
7. The method according to claim 3, characterized in that The verifying whether the target big model is complete based on the big model content identifier specifically includes: Calculating a third hash value of the target large model using a preset hash algorithm; comparing the third hash value to a fourth hash value in the large model content identifier; If the third Hash value is the same as the fourth Hash value, it is determined that the target large model is complete.
8. A model acquisition device, characterized in that: A central node is connected to multiple edge nodes in the same local area network, and the device includes: The transceiver module is used to request the central node to obtain the content identifier set of the sub-files of the target large model when receiving a user request to obtain the target large model. The central node is used to store the content identifiers of the sub-files of the large model in the local area network. Different content identifiers are used to identify different sub-files in the related files of the large model. A processing module, configured to search in the local area network whether other edge nodes have stored different sub-files of the target large model based on the content identifier set; The transceiver module is further configured to pull the different sub-files of the target large model from each edge node in parallel if there is at least one edge node storing the different sub-files of the target large model; The processing module is also used to recombine different sub-files of the target large model to obtain the target large model.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to cause the computer to execute the method according to any one of claims 1 to 7.
11. A computer program product, characterized in that When the computer program product is called by a computer, the computer is caused to execute the method according to any one of claims 1 to 7.
Citation Information
Cited By
Model file loading method and device and related equipment
CN119537029A
