File Compression Method and Apparatus, File Loading Method and Apparatus, and Electronic Device

Abstract

The present disclosure provides a file compression method and apparatus, a file loading method and apparatus, and an electronic device. The file compression method includes: obtaining a first type of file and a second type of file, where the first type of file includes a first binary file; obtaining a second binary file, which is obtained by converting the second type of file and only includes binary files; and obfuscating the first binary file and the second binary file to obtain a compressed third binary file, where the arrangement manners of the first binary file and the second binary file can be characterized by file location information.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technologies, and more particularly, to a file compression method and apparatus, a file loading method and apparatus, and an electronic device. Background Art

[0002] With the rapid development of Internet technologies and big data technologies, some data processing can be performed at the edge device side. Therefore, it is necessary to load model files and resource files at the edge device side for data processing.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: When loading model files and resource files based on the mainstream computing framework, the information security and information dissemination convenience of the model files and resource files cannot meet the user requirements. Summary of the Invention

[0004] In view of this, the present disclosure provides a file compression method and apparatus, a file loading method and apparatus, and an electronic device that can improve information security and information dissemination freedom.

[0005] One aspect of the present disclosure provides a file compression method executed by an edge device side. The cloud is respectively connected to a plurality of edge device sides. The method may include: obtaining a first type of file and a second type of file, where the first type of file includes a first binary file; obtaining a second binary file, where the second binary file is obtained by converting the second type of file, and the second binary file only includes binary files; and confusing the first binary file and the second binary file to obtain a compressed third binary file, where the arrangement manners of the first binary file and the second binary file can be characterized by file position information.

[0006] According to an embodiment of the present disclosure, the above method further includes: parsing the third binary file to determine the file position information; and writing the file position information into the third binary file.

[0007] According to an embodiment of the present disclosure, the file position information includes at least one of file identification information, file start position information, file end position information, file index information, and file length information.

[0008] According to an embodiment of the present disclosure, the first type of file is a model file, and the model file includes model topology information and model parameters; confusing the first binary file and the second binary file includes: determining the model topology information and at least part of the model parameters of at least one model from at least one first binary file; writing the confused model topology information and at least part of the model parameters of at least one model into a third binary file; and writing at least one first binary file and at least one second binary file into the third binary file in sequence.

[0009] According to an embodiment of the present disclosure, writing at least one first binary file and at least one second binary file into the third binary file in sequence includes: determining an offset, where the offset is used to change the file start position of the first binary file and / or the second binary file in the third binary file; and writing at least one first binary file and at least one second binary file into the third binary file in sequence based on the offset.

[0010] According to an embodiment of the present disclosure, the first confusion order of the model topology information and at least part of the model parameters of each first binary file in at least one first binary file may be the same as or different from the second confusion order of each first binary file in at least one first binary file, and the model topology information and at least part of the model parameters of at least one first binary file may be determined through file location information.

[0011] According to an embodiment of the present disclosure, the above method further includes: after obtaining the compressed third binary file, transmitting the compressed third binary file to at least one of a plurality of edge device terminals; and / or transmitting authorization information to at least one of a plurality of edge device terminals, where the authorization information is a string that enables at least one of a plurality of edge device terminals to resolve file location information.

[0012] Another aspect of the present disclosure provides a file loading method executed by an edge device terminal, including: obtaining a third binary file, where the third binary file includes a confused first binary file and a second binary file, and the arrangement manner of the first binary file and the second binary file can be characterized by file location information; and loading the file from the head of the third binary file to the tail of the third binary file based on the file location information in the manner of a memory-mapped file.

[0013] According to an embodiment of the present disclosure, loading a file from the head to the tail of a third binary file based on file location information in the manner of a memory-mapped file includes: mapping the third binary file to a specified storage space from the head to the tail of the third binary file; obtaining file location information; and loading a required first type of file and / or a second type of file from the specified storage space based on the file location information.

[0014] According to an embodiment of the present disclosure, loading a required first type of file and / or a second type of file from the specified storage space based on file location information includes: determining a required first binary file and / or a second binary file from the specified storage space based on the file location information; determining model topology information and at least part of model parameters corresponding to the required first binary file based on the file location information; and determining a required first type of file based on the required first binary file and the model topology information and at least part of the model parameters corresponding to the required first binary file, and / or determining a required second type of file based on the required second binary file.

[0015] According to an embodiment of the present disclosure, obtaining file location information includes: obtaining authorization information, where the authorization information is a string that enables at least one of a plurality of edge device ends to resolve the file location information; and determining the file location information from the specified storage space based on the authorization information.

[0016] Another aspect of the present disclosure provides a file compression device, which is set in the cloud, and the cloud is connected to at least one edge device end. The device includes: a file to be compressed acquisition module, a second binary file acquisition module, and a file obfuscation module. Among them, the file to be compressed acquisition module is used to acquire a first type of file and a second type of file, and the first type of file includes a first binary file; the second binary file acquisition module is used to acquire a second binary file, and the second binary file is obtained by converting the second type of file and only includes binary files; and the file obfuscation module is used to obfuscate the first binary file and the second binary file to obtain a compressed third binary file, where the arrangement manners of the first binary file and the second binary file can be characterized by file location information.

[0017] According to an embodiment of the present disclosure, the device further includes a file location information parsing module and a file location information writing module. The file location information parsing module is used to parse the third binary file to determine file location information; and the file location information writing module is used to write the file location information into the third binary file.

[0018] According to an embodiment of the present disclosure, the file location information includes at least one of file index information, file length information, file identification information, file start position information, and file end position information.

[0019] According to an embodiment of the present disclosure, the first type of file is a model file, and the model file includes model topology information and model parameters; the file obfuscation module includes: a model topology information determination sub-module, a model topology information writing sub-module, and a file writing sub-module. The model topology information determination sub-module is configured to determine the model topology information and at least part of the model parameters of each of at least one model from at least one first binary file; the model topology information writing sub-module is configured to write the obfuscated model topology information and at least part of the model parameters of each of at least one model into a third binary file; and the file writing sub-module is configured to sequentially write at least one first binary file and at least one second binary file into the third binary file.

[0020] According to an embodiment of the present disclosure, the file writing sub-module includes: an offset determination unit and a file writing unit. The offset determination unit is configured to determine an offset, and the offset is used to change the file start position of the first binary file and / or the second binary file in the third binary file; and the file writing unit is configured to sequentially write at least one first binary file and / or at least one second binary file into the third binary file based on the offset.

[0021] According to an embodiment of the present disclosure, the first obfuscation order of the model topology information and at least part of the model parameters of each first binary file in at least one first binary file may be the same as or different from the second obfuscation order of each first binary file in at least one first binary file, and the model topology information and at least part of the model parameters of each of at least one first binary file may be determined through the file location information.

[0022] According to an embodiment of the present disclosure, the cloud is connected to at least one edge device end, and the above device further includes at least one of the following modules: a file transmission module, configured to transmit the compressed third binary file to at least one of a plurality of edge device ends; an authorization information transmission module, configured to transmit authorization information to at least one of a plurality of edge device ends, where the authorization information is a string that enables at least one of a plurality of edge device ends to resolve the file location information.

[0023] Another aspect of the present disclosure provides a file loading device disposed at an edge device end. The device includes: a third binary file acquisition module and a file loading module. The third binary file acquisition module is configured to acquire a third binary file, where the third binary file includes a first binary file and a second binary file that are obfuscated, and the arrangement manner of the first binary file and the second binary file can be characterized by file location information; and the file loading module is configured to perform file loading from the head to the tail of the third binary file based on the file location information in the manner of a memory-mapped file.

[0024] According to an embodiment of the present disclosure, the file loading module includes: a memory mapping sub-module, a file location information acquisition sub-module, and a loading sub-module. The memory mapping sub-module is configured to map the third binary file to a specified storage space from the head to the tail of the third binary file; the file location information acquisition sub-module is configured to acquire the file location information; and the loading sub-module is configured to load a required first type of file and / or a second type of file from the specified storage space based on the file location information.

[0025] According to an embodiment of the present disclosure, the loading sub-module includes a binary file determination unit configured to determine a required first binary file and / or a second binary file from the specified storage space based on the file location information; a model topology determination unit configured to determine model topology information and at least part of model parameters corresponding to the required first binary file based on the file location information; and a file determination unit configured to determine a required first type of file based on the required first binary file and the model topology information and at least part of the model parameters corresponding to the required first binary file, and / or determine a required second type of file based on the required second binary file.

[0026] According to an embodiment of the present disclosure, the file location information acquisition sub-module includes: an authorization information unit and a file location information determination unit. The authorization information unit is configured to acquire authorization information, where the authorization information is a string that enables at least one of a plurality of edge device ends to resolve the file location information; and the file location information determination unit is configured to determine the file location information from the specified storage space based on the authorization information.

[0027] Another aspect of the present disclosure provides a computer system including one or more processors and a storage device, where the storage device is configured to store executable instructions, and when the executable instructions are executed by the processor, the above-described method is implemented.

[0028] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, and the instructions are configured to implement the above-described method when executed.

[0029] Another aspect of the present disclosure provides a computer program, which includes computer-executable instructions that, when executed, are used to implement the method described above.

[0030] According to an embodiment of the present disclosure, the resource file and the model file are obfuscated and compressed in the form of binary files, making it impossible to easily peek into and crack the resource file and the model file, effectively enhancing information security.

[0031] According to an embodiment of the present disclosure, multiple resource files and model files are arranged in a tiled manner, and by using the memory mapping method, it is convenient to achieve fast loading. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Through the following description of the embodiments of the present disclosure with reference to the drawings, the above and other objects, features, and advantages of the present disclosure will become clearer. In the drawings:

[0033] Figure 1 Schematically shows the application scenarios of the file compression method and device, the file loading method and device, and the electronic device according to an embodiment of the present disclosure;

[0034] Figure 2 Schematically shows the schematic diagram of the system architecture of the applicable file compression method and device, the file loading method and device, and the electronic device according to an embodiment of the present disclosure;

[0035] Figure 3 Schematically shows the flowchart of the file compression method according to an embodiment of the present disclosure;

[0036] Figure 4 Schematically shows the data flow diagram of the file compression method according to an embodiment of the present disclosure;

[0037] Figure 5 Schematically shows the flowchart of the file compression method according to another embodiment of the present disclosure;

[0038] Figure 6 Schematically shows the flowchart of the file loading method according to an embodiment of the present disclosure;

[0039] Figure 7 Schematically shows the data flow diagram of the file loading method according to an embodiment of the present disclosure;

[0040] Figure 8 Schematically shows the block diagram of the file compression device according to an embodiment of the present disclosure;

[0041] Figure 9 Schematically shows the block diagram of the file loading device according to another embodiment of the present disclosure; and

[0042] Figure 10A block diagram schematically showing an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0043] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, numerous specific details are set forth in order to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0044] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0045] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0046] In the case of using expressions such as "at least one of A, B, and C, etc.", generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, or C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0047] Embodiments of the present disclosure provide a file compression method executed by the cloud. The method includes a binary file acquisition process and a file obfuscation process. In the binary file acquisition process, first, a first type of file and a second type of file are acquired. The first type of file includes a first binary file. Then, a second binary file is acquired. The second binary file is obtained by converting the second type of file, and the second binary file only includes binary files. After the binary file acquisition process is completed, the file obfuscation process is entered, and the first binary file and the second binary file are obfuscated to obtain a compressed third binary file, where the arrangement modes of the first binary file and the second binary file can be characterized by file position information.

[0048] Embodiments of the present disclosure further provide a file loading method executed by an edge device. The method includes a binary file acquisition process and a file loading process. In the binary file acquisition process, a third binary file is acquired. The third binary file includes the obfuscated first binary file and second binary file, and the arrangement modes of the first binary file and the second binary file can be characterized by file position information. After the binary file acquisition process is completed, the file loading process is entered, and the file is loaded from the head to the tail of the third binary file based on the file position information in the manner of a memory-mapped file.

[0049] First, some concepts related to the present disclosure are exemplarily described below to better understand the embodiments of the present disclosure.

[0050] Edge computing refers to an open platform that integrates network, computing, storage, and application core capabilities on the side close to the object or data source to provide the nearest-end service nearby. Its application programs are initiated on the edge side, generating a faster network service response and meeting the basic requirements of the industry in aspects such as real-time services, application intelligence, security, and privacy protection. Edge computing is between physical entities and industrial connections or at the top of physical entities. Cloud computing can still access the historical data of edge computing.

[0051] For scenarios such as the Internet of Things, many controls will be implemented through local Internet of Things nodes without being handed over to the cloud, and the processing process will be completed at the local edge computing layer, reducing the load on the cloud and providing a faster response due to being closer to users.

[0052] Currently, models such as deep neural networks have begun to be applied to various edge computing devices. During the operation and deployment of edge computing devices, relevant resource files and model files need to be loaded.

[0053] Loading model files and resource files based on mainstream computing frameworks, where the format of the model files is output based on the format of mainstream popular computing frameworks, and the format of the resource files is also clear at a glance. It is not very friendly for confidentiality and free dissemination and deployment. In addition, it is also time-consuming to load the above files on edge computing devices. This results in the inability to guarantee the information security of resource files and model files. For example, the detailed details and content formats of sub-files of model files can be spied on based on popular parsing tools. Loading resource files and model files is very time-consuming and not conducive to deployment and implementation.

[0054] Figure 1 Schematically shows the application scenarios of a file compression method and apparatus, a file loading method and apparatus, and an electronic device according to an embodiment of the present disclosure.

[0055] As Figure 1 shown, taking a scenario including two types of files as an example for illustration, multiple first-type files and second-type files, etc. are respectively converted into binary files (such as binary file 1, n, etc. of the first-type files, binary file 1, n, etc. of the second-type files) to achieve compression. After confusing multiple binary files, they are written into a new binary file. The arrangement order of each type of file in the new binary file can be determined by parsing. The new binary file includes the key information of multiple files (such as the layer structure information and parameters of the model, etc.), as well as the binary files of multiple different types of files written in a confused manner, making it impossible to easily read the content of various types of files from the new binary file, etc., realizing information encryption. When file loading is required, the new binary file can be restored to the original first-type files and second-type files, etc., facilitating the completion of the file loading process. Among them, n can be a positive integer greater than 1. It should be noted that the above two types of files are only schematically shown and should not be understood as a limitation of the present disclosure. Compression and loading processes can also be performed for more types (such as 3 or more types) or fewer types (such as 1 type) of files.

[0056] Figure 2 Schematically shows a schematic diagram of the system architecture of a file compression method and apparatus, a file loading method and apparatus, and an electronic device according to an embodiment of the present disclosure. It should be noted that Figure 2 What is shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0057] As Figure 2As shown, the system architecture 200 according to this embodiment may include terminal devices 201, 202, 203, a network 204, and a server 205. The network 204 is used to provide a medium for communication links between the terminal devices 201, 202, 203 and the server 205. The network 204 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0058] Users can use the terminal devices 201, 202, 203 to interact with the server 205 through the network 204 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 201, 202, 203, such as shopping applications, web browser applications, operation and maintenance applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0059] The terminal devices 201, 202, 203 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, industrial control computers, terminal servers, etc.

[0060] The server 205 may be a server providing various services, such as a background management server that supports the websites browsed by users using the terminal devices 201, 202, 203 (for example only). The background management server may analyze and process data such as received user requests, etc., and feedback the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal devices.

[0061] It should be noted that the file compression method provided by the embodiments of the present disclosure can generally be executed by the server 205. Correspondingly, the file compression device provided by the embodiments of the present disclosure can generally be set in the server 205. The file loading method provided by the embodiments of the present disclosure can generally be executed by the terminal devices 201, 202, 203. Correspondingly, the file loading device provided by the embodiments of the present disclosure can generally be set in the terminal devices 201, 202, 203.

[0062] It should be understood that the numbers of terminal devices, networks, and servers are merely illustrative. According to actual needs, there may be any number of terminal devices, networks, and servers.

[0063] Figure 3 The flowchart of the file compression method according to the embodiments of the present disclosure is schematically shown.

[0064] As Figure 3 shown, this method includes operation S301 to operation S305.

[0065] In operation S301, obtain a first type of file and a second type of file, where the first type of file includes a first binary file.

[0066] In this embodiment, the first type of file and the second type of file may be different types of files. Specifically, the first type of file may be a binary file, and the second type of file may be a file including at least non-binary files. For example, the first type of file may be a model file, including but not limited to files of various neural network models, etc. The second type of file may be a resource file, including but not limited to text, picture files, etc. Among them, the neural network may include a network topology structure and network parameters. The network topology structure may include layer information, etc. The network parameters may include parameters such as weights and biases.

[0067] Among them, the model file may be stored as a binary file, and the binary file may include a data structure and is a formatted and serialized file.

[0068] In operation S303, obtain a second binary file, where the second binary file is obtained by converting the second type of file, and the second binary file only includes binary files.

[0069] In this embodiment, for the second type of file including non-binary files, it (or the non-binary files therein) can be converted into a binary file to facilitate encrypting the first type of file and the second type of file by means of obfuscation. It should be noted that if the second type of file itself only includes binary files, there is no need to perform the process of binarizing the file conversion.

[0070] In operation S305, obfuscate the first binary file and the second binary file to obtain a compressed third binary file, where the arrangement of the first binary file and the second binary file can be characterized by file position information.

[0071] Among them, the first binary file and the second binary file can be obfuscated randomly, or the first binary file and the second binary file can be obfuscated according to certain rules. If the first binary file and the second binary file are obfuscated according to certain rules, the above file position information is known.

[0072] Taking the first type of file as a model file, where the model file includes model topology information and model parameters as an example, obfuscating the first binary file and the second binary file may include the following operations.

[0073] First, determine the model topology information and at least some model parameters of each of at least one model from at least one first binary file. Among them, the model topology information can be, for example: the first layer is the input layer, the second layer is the fully connected layer, the third layer is the convolutional layer, the fourth layer is the pooling layer, the fifth layer is the convolutional layer, the sixth layer is the pooling layer, the seventh layer is the fully connected layer, the eighth layer is the output layer, etc. At least some model parameters can be the key parameters of the model, etc. For example, the model parameters can be the weights and biases of the layers, and can also be the weights and biases of the nodes, etc. The model can be a variety of artificial intelligence models, not limited to neural networks only. For example, it can also be a random tree, linear regression, decision tree, support vector machine, etc.

[0074] Then, write the model topology information and at least some model parameters of each of the at least one model after confusion into a third binary file. Specifically, the model topology information and at least some model parameters of each of the at least one model after confusion can be written to a specified location in the third binary file, such as the top, bottom, etc., to facilitate reading these key information.

[0075] Next, write the at least one first binary file and the at least one second binary file into the third binary file in sequence. Among them, the writing order is not limited. For example, first write a first binary file, then write a second binary file, and then repeat until all the first binary files and second binary files are written into the third binary file. In addition, it can also be writing two first binary files continuously, and then writing two second binary files, etc. Each of the multiple first binary files can be randomly selected or selected in a preset order.

[0076] In one embodiment, the first confusion order of the model topology information and at least some model parameters of each of the at least one first binary file is the same as or different from the second confusion order of each of the at least one first binary file. The model topology information and at least some model parameters of each of the at least one first binary file can be determined through the file location information. Among them, when the first confusion order is the same as the second confusion order, it is convenient to quickly determine the model parameters corresponding to the first binary file. When the first confusion order is different from the second confusion order, it is convenient to improve information security.

[0077] In one embodiment, sequentially writing at least one first binary file and at least one second binary file into a third binary file may include the following operations. First, determine an offset, which is used to change the file start position of the first binary file and / or the second binary file in the third binary file. Then, write at least one first binary file and / or at least one second binary file into the third binary file one by one based on the offset. Among them, the offset for each first binary file and second binary file can be fixed or random. The offset can also be stored at a specified position in the third binary file.

[0078] In one embodiment, the above method further includes: parsing the third binary file to determine file position information, and writing the file position information into the third binary file. In this way, it is possible to determine at which positions of the third binary file the first binary file and the second binary file are respectively stored by parsing.

[0079] For example, the file position information includes at least one of file index information, file length information, file identification information, file start position information, and file end position information.

[0080] Figure 4 A data flow diagram of a file compression method according to an embodiment of the present disclosure is schematically shown.

[0081] As Figure 4 shown, first, export the layer type and key calculation parameters of the model to a binary file to generate a new binary file, such as model.dat.

[0082] For the model file, obtain its key information (such as information info) and file length, perform an offset (Salt) to add additional confusion, and finally write the file position and file length.

[0083] For the resource file, if the resource file is text information, it needs to be compressed. If it is in binary format, it does not need to be compressed. Obtain the length of the resource file, add Salt to add additional confusion, and write it into the new binary file.

[0084] In another embodiment, to facilitate file loading on the device side, the cloud is connected to at least one edge device side.

[0085] Figure 5 A flowchart of a file compression method according to another embodiment of the present disclosure is schematically shown.

[0086] As Figure 5 shown, after performing operation S305 to obtain the compressed third binary file, the above method may further include operation S507.

[0087] In operation S507, the compressed third binary file and / or the authorization information are transmitted to at least one of multiple edge device terminals.

[0088] For example, the compressed third binary file is transmitted to at least one of multiple edge device terminals. For another example, the authorization information is transmitted to at least one of multiple edge device terminals, where the authorization information is a string that enables at least one of multiple edge device terminals to resolve the file location information. It should be noted that the compressed third binary file and / or the authorization information may also be pre-stored in the edge device terminals to facilitate file loading by the edge device terminals.

[0089] In the file compression method provided by the embodiments of the present disclosure, after the first type of file and the second type of file are converted into binary files, they are obfuscated and compressed, making it difficult for the outside to peek at and crack the content of the files, effectively improving information security.

[0090] On the other hand, the present disclosure also provides a file loading method.

[0091] Figure 6 Schematically shows a flowchart of the file loading method according to the embodiments of the present disclosure.

[0092] As Figure 6 shown, the file loading method executed by the edge device terminal may include operation S601 to operation S603

[0093] In operation S601, a third binary file is obtained. The third binary file includes the obfuscated first binary file and the second binary file, and the arrangement manner of the first binary file and the second binary file can be characterized by the file location information.

[0094] In operation S603, in the manner of memory mapping files, file loading is performed from the head of the third binary file to the tail of the third binary file based on the file location information.

[0095] Among them, for the first binary file, the second binary file, the third binary file, and the file location information, reference can be made to the relevant content of file compression, and details will not be listed one by one here.

[0096] Figure 7 Schematically shows a data flow diagram of the file loading method according to the embodiments of the present disclosure.

[0097] As Figure 7 shown, during the loading operation, obtain as Figure 4The new binary file shown. First, perform memory mapping and load it sequentially from the beginning to the end. Such a strategy will accelerate the loading speed, for example, facilitating the quick reading of relationship information, etc. Among them, for the second type of file converted from a binary file, it can be decompressed in the decompression space.

[0098] In one embodiment, loading a file from the head to the tail of a third binary file based on file position information in the manner of a memory-mapped file may include the following operations.

[0099] First, map the third binary file to a specified storage space from the head to the tail of the third binary file.

[0100] Then, obtain the file position information.

[0101] Next, load the required first type of file and / or second type of file from the specified storage space based on the file position information.

[0102] In one embodiment, loading the required first type of file and / or second type of file from the specified storage space based on the file position information may include the following operations.

[0103] First, determine the required first binary file and / or second binary file from the specified storage space based on the file position information.

[0104] Then, determine the model topology information and at least part of the model parameters corresponding to the required first binary file based on the file position information.

[0105] Next, determine the required first type of file based on the required first binary file and the model topology information and at least part of the model parameters corresponding to the required first binary file, and / or determine the required second type of file based on the required second binary file.

[0106] For example, obtaining the file position information may include the following operations.

[0107] First, obtain authorization information, where the authorization information is a string that enables at least one of multiple edge device ends to resolve the file position information. The authorization information can be obtained by means of being sent by the cloud, or can also be information such as text, pattern, and scan code printed on a physical object. This facilitates service providers to collect service fees, etc.

[0108] Then, determine the file position information from the specified storage space based on the authorization information.

[0109] The file loading method provided by the embodiments of the present disclosure arranges multiple resource files and model files in a tiled manner, and uses the memory mapping method, which helps to achieve fast loading.

[0110] Another aspect of the present disclosure provides a file compression device.

[0111] Figure 8 The block diagram of the file compression device according to the embodiments of the present disclosure is schematically shown.

[0112] As Figure 8 shown, the file compression device 800 is set in the cloud, and the cloud is connected to at least one edge device end. It may include: a file to be compressed acquisition module 810, a second binary file acquisition module 820, and a file obfuscation module 830.

[0113] Among them, the file to be compressed acquisition module 810 is used to acquire a first type of file and a second type of file, and the first type of file includes a first binary file.

[0114] The second binary file acquisition module 820 is used to acquire a second binary file, which is obtained by converting the second type of file, and the second binary file only includes binary files.

[0115] The file obfuscation module 830 is used to obfuscate the first binary file and the second binary file to obtain a compressed third binary file, wherein the arrangement manner of the first binary file and the second binary file can be characterized by file position information.

[0116] In one embodiment, the above device further includes a file position information parsing module and a file position information writing module.

[0117] Among them, the file position information parsing module is used to parse the third binary file to determine the file position information.

[0118] The file position information writing module is used to write the file position information into the third binary file.

[0119] In one embodiment, the file position information includes at least one of file index information, file length information, file identification information, file start position information, and file end position information.

[0120] In one embodiment, the first type of file is a model file, and the model file includes model topology information and model parameters. Correspondingly, the file obfuscation module 830 includes: a model topology information determination sub-module, a model topology information writing sub-module, and a file writing sub-module.

[0121] Among them, the model topology information determination sub-module is used to determine the model topology information and at least part of the model parameters of each of at least one model from at least one first binary file.

[0122] The model topology information writing sub-module is used to write the confused model topology information and at least part of the model parameters of each of at least one model into a third binary file.

[0123] The file writing sub-module is used to sequentially write at least one first binary file and at least one second binary file into a third binary file.

[0124] In one embodiment, the file writing sub-module includes: an offset determination unit and a file writing unit.

[0125] Among them, the offset determination unit is used to determine an offset, and the offset is used to change the file start position of the first binary file and / or the second binary file in the third binary file.

[0126] The file writing unit is used to sequentially write at least one first binary file and / or at least one second binary file into the third binary file based on the offset.

[0127] In one embodiment, the first confusion order of the model topology information and at least part of the model parameters of each first binary file in at least one first binary file is the same as or different from the second confusion order of each first binary file in at least one first binary file, and the model topology information and at least part of the model parameters of each of at least one first binary file can be determined through file position information.

[0128] In one embodiment, the cloud is connected to at least one edge device end, and the above device further includes at least one of the following modules: a file transmission module, an authorization information transmission module.

[0129] The file transmission module is used to transmit the compressed third binary file to at least one of multiple edge device ends.

[0130] The authorization information transmission module is used to transmit authorization information to at least one of multiple edge device ends, where the authorization information is a string that enables at least one of multiple edge device ends to parse out file position information.

[0131] Another aspect of the present disclosure provides a file compression device.

[0132] Figure 9 Schematically shows a block diagram of a file loading device according to another embodiment of the present disclosure.

[0133] Such as Figure 9As shown, the file loading device 900 may include a third binary file acquisition module 910 and a file loading module 920.

[0134] The third binary file acquisition module 910 is used to acquire a third binary file, which includes a first binary file and a second binary file that are confused, and the arrangement of the first binary file and the second binary file can be characterized by file location information.

[0135] The file loading module 920 is used to load the file from the head to the tail of the third binary file in the manner of a memory-mapped file based on the file location information.

[0136] In one embodiment, the file loading module 920 includes: a memory mapping sub-module, a file location information acquisition sub-module, and a loading sub-module.

[0137] The memory mapping sub-module is used to map the third binary file to a specified storage space from the head to the tail of the third binary file.

[0138] The file location information acquisition sub-module is used to acquire the file location information.

[0139] The loading sub-module is used to load the required first type of file and / or second type of file from the specified storage space based on the file location information.

[0140] In one embodiment, the loading sub-module includes a binary file determination unit,

[0141] The binary file determination unit is used to determine the required first binary file and / or second binary file from the specified storage space based on the file location information.

[0142] The model topology determination unit is used to determine the model topology information and at least part of the model parameters corresponding to the required first binary file based on the file location information.

[0143] The file determination unit is used to determine the required first type of file based on the required first binary file and the model topology information and at least part of the model parameters corresponding to the required first binary file, and / or determine the required second type of file based on the required second binary file.

[0144] In one embodiment, the file location information acquisition sub-module includes: an authorization information unit and a file location information determination unit.

[0145] The authorization information unit is used to acquire authorization information, where the authorization information is a string that enables at least one of multiple edge device ends to resolve the file location information.

[0146] The file location information determining unit is configured to determine file location information from a specified storage space based on authorization information.

[0147] Any multiple of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure, or at least part of the functions of any multiple of them, may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or may be implemented by any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be at least partially implemented as a computer program module, and when the computer program module runs, it may execute the corresponding functions.

[0148] For example, any multiple of the to-be-compressed file acquisition module 810, the second binary file acquisition module 820, and the file obfuscation module 830 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to the embodiments of the present disclosure, at least one of the to-be-compressed file acquisition module 810, the second binary file acquisition module 820, and the file obfuscation module 830 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or may be implemented by any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the to-be-compressed file acquisition module 810, the second binary file acquisition module 820, and the file obfuscation module 830 may be at least partially implemented as a computer program module, and when the computer program module runs, it may execute the corresponding functions.

[0149] Figure 10 A block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. Figure 10 The electronic device shown is only an example, and should not bring any limitation to the functions and usage scope of the embodiments of the present disclosure.

[0150] As shown Figure 10 in FIG. 1, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 1001 may also include on-board memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0151] In the RAM 1003, various programs and data required for the operation of the system 1000 are stored. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. The processor 1001 performs various operations of the method flow according to an embodiment of the present disclosure by executing the programs in the ROM 1002 and / or the RAM 1003. It should be noted that the programs may also be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flow according to an embodiment of the present disclosure by executing the programs stored in one or more memories.

[0152] According to an embodiment of the present disclosure, the system 1000 may further include an input / output (I / O) interface 1005, and the input / output (I / O) interface 1005 is also connected to the bus 1004. The system 1000 may further include one or more of the following components connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read from it can be installed into the storage section 1008 as needed.

[0153] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0154] The present disclosure also provides a computer-readable storage medium, which may be included in the device / device / system described in the above embodiments; or may exist separately without being assembled into the device / device / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0155] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 1002 and / or RAM 1003 and / or one or more memories other than ROM 1002 and RAM 1003.

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0157] Those skilled in the art will appreciate that the features recited in the various embodiments and / or claims of the present disclosure may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features recited in the various embodiments and / or claims of the present disclosure may be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0158] The embodiments of the present disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments have been described separately above, this does not mean that the measures in the respective embodiments cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A file compression method executed by the cloud, including: obtaining a first type of file and a second type of file, where the first type of file includes a first binary file, the first type of file is a model file, the model file includes model topology information and model parameters, and the second type of file is a resource file; obtaining a second binary file, which is obtained by converting the second type of file, and the second binary file only includes binary files; and confusing the first binary file and the second binary file to obtain a compressed third binary file, where the arrangement of the first binary file and the second binary file can be characterized by file position information; wherein, confusing the first binary file and the second binary file includes: determining the model topology information of at least one model and at least part of the model parameters of each of the at least one first binary file; writing the confused model topology information of at least one model and at least part of the model parameters into the third binary file; and writing at least one of the first binary files and at least one of the second binary files into the third binary file in sequence.

2. The method according to claim 1, further including: parsing the third binary file to determine the file position information; and writing the file position information into the third binary file.

3. The method according to claim 1 or 2, wherein, the file position information includes at least one of file index information, file length information, file identification information, file start position information, and file end position information.

4. The method according to claim 1, wherein, writing at least one of the first binary files and at least one of the second binary files into the third binary file in sequence includes: determining an offset, which is used to change the file start position of the first binary file and / or the second binary file in the third binary file; and writing at least one of the first binary files and / or at least one of the second binary files into the third binary file one by one based on the offset.

5. The method according to claim 1, wherein, the first confusion order of the model topology information and at least part of the model parameters of each first binary file in at least one of the first binary files is the same as or different from the second confusion order of each first binary file in at least one of the first binary files, and the model topology information and at least part of the model parameters of at least one first binary file can be determined by the file position information.

6. The method according to claim 1, where the cloud is connected to at least one edge device end, and the method further includes: after obtaining the compressed third binary file, transmitting the compressed third binary file to at least one of multiple edge device ends; and / or Transmit the authorization information to at least one of the multiple edge device terminals, where the authorization information is a string that enables at least one of the multiple edge device terminals to resolve the file location information.

7. A file loading method executed by an edge device terminal, comprising: Obtain a third binary file, where the third binary file includes a confused first binary file and a second binary file. The first binary file is a model file, the model file includes model topology information and model parameters, the second binary file is a resource file, and the arrangement of the first binary file and the second binary file can be characterized by file location information; and Perform file loading from the head to the tail of the third binary file based on the file location information in the manner of a memory-mapped file; where the third binary file is obtained through the following manner: Determine the model topology information and at least part of the model parameters of at least one model from at least one of the first binary files; Write the confused model topology information and at least part of the model parameters of at least one model into the third binary file; and Write at least one of the first binary files and at least one of the second binary files into the third binary file in sequence.

8. The method according to claim 7, wherein The performing file loading from the head to the tail of the third binary file based on the file location information in the manner of a memory-mapped file includes: From the head to the tail of the third binary file, map the third binary file to a specified storage space; Obtain the file location information; and Based on the file location information, load the required first type of file and / or second type of file from the specified storage space.

9. The method according to claim 8, wherein The loading the required first type of file and / or second type of file from the specified storage space based on the file location information includes: Based on the file location information, determine the required first binary file and / or second binary file from the specified storage space; Based on the file location information, determine the model topology information and at least part of the model parameters corresponding to the required first binary file; and Based on the required first binary file and the model topology information and at least part of the model parameters corresponding to the required first binary file, determine the required first type of file, and based on the required second binary file, determine the required second type of file.

10. The method according to claim 8, wherein The obtaining the file location information includes: Obtain authorization information, where the authorization information is a string that enables at least one of the multiple edge device terminals to resolve the file location information; and Based on the authorization information, determine the file location information from the specified storage space.

11. A file compression device, provided in the cloud, the device comprising: A module for obtaining files to be compressed, configured to obtain a first type of file and a second type of file. The first type of file includes a first binary file, which is a model file. The model file includes model topology information and model parameters. The second type of file is a resource file. A module for obtaining a second binary file, configured to obtain a second binary file, which is obtained by converting the second type of file, and the second binary file only includes binary files. And A file obfuscation module, configured to obfuscate the first binary file and the second binary file to obtain a compressed third binary file. Wherein, the arrangement of the first binary file and the second binary file can be characterized by file position information. Wherein, the file obfuscation module includes: A model topology information determination sub-module, configured to determine the model topology information and at least part of the model parameters of at least one model from at least one of the first binary files. A model topology information writing sub-module, configured to write the obfuscated model topology information and at least part of the model parameters of at least one model into the third binary file; and A file writing sub-module, configured to sequentially write at least one of the first binary files and at least one of the second binary files into the third binary file.

12. A file loading device, provided at the edge device side, the device includes: A module for obtaining a third binary file, configured to obtain a third binary file, which includes an obfuscated first binary file and a second binary file. The first binary file is a model file, which includes model topology information and model parameters. The second binary file is a resource file. The arrangement of the first binary file and the second binary file can be characterized by file position information. And A file loading module, configured to load files from the head to the tail of the third binary file in the manner of a memory-mapped file based on the file position information. Wherein, the third binary file is obtained by the following method: Determine the model topology information and at least part of the model parameters of at least one model from at least one of the first binary files; Write the obfuscated model topology information and at least part of the model parameters of at least one model into the third binary file; And Sequentially write at least one of the first binary files and at least one of the second binary files into the third binary file.

13. A computer system, includes: One or more processors; A storage device, configured to store executable instructions, which when executed by the processor, implement the method according to any one of claims 1 to 10.

14. A computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images