Data transmission method and apparatus, electronic device, storage medium, and product

By using a data compression model trained on sample data and a multi-level cloud architecture, the problem of low efficiency in streaming data compression is solved, achieving efficient data compression and transmission.

CN118540384BActive Publication Date: 2026-01-13INSPUR TIANYUAN COMM INFORMATION SYST CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410582480.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-11
Publication Date
2026-01-13
Estimated Expiration
2044-05-11

AI Technical Summary

Technical Problem

Existing data compression methods are not efficient in compressing and transmitting streaming data. The compression limit of existing data compression methods depends on the algorithm itself, resulting in high transmission costs.

Method used

A data compression model trained on sample transmission data and sample compressed data is adopted. The model is trained using the MLP algorithm and mean squared error loss function. Combined with a multi-level architecture of global model library and edge cloud server, data feature mining and compression are realized by introducing artificial intelligence model in the data compression process.

Benefits of technology

It improves data compression ratio and transmission efficiency, avoids the compression limits of mathematical and statistical methods, and achieves efficient compression and transmission of streaming data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118540384B_ABST
    Figure CN118540384B_ABST
Patent Text Reader

Abstract

The application provides a data transmission method and device, electronic equipment, storage medium and product, relates to the technical field of artificial intelligence, and comprises the following steps: obtaining to-be-transmitted data; inputting the to-be-transmitted data into a data compression model to obtain compressed data corresponding to the to-be-transmitted data output by the data compression model; and sending the compressed data to a data receiving end; wherein the data compression model is obtained based on sample transmission data and sample compressed data corresponding to the sample transmission data. In the foregoing manner, the to-be-transmitted data is input into the data compression model, the compressed data corresponding to the to-be-transmitted data output by the data compression model is obtained, an artificial intelligence model is introduced in the data compression process, data feature mining and data compression are realized by the model, the defect that the compression limit is limited by the algorithm itself when data compression is performed by using a mathematical statistical method can be avoided, the data compression ratio is improved, and the efficiency of data compression and data transmission is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a data transmission method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] The transmission and management of massive amounts of data, such as scientific computing data (e.g., meteorological data, earthquake data) and corpora (e.g., text, images) required for large-scale model training, consume a lot of network resources and are costly. Data compression is one of the main means to solve the problem of massive data transmission.

[0003] Existing data compression methods mainly employ mathematical and statistical methods, such as the EZW algorithm, ZIP algorithm, and data filtering method. The compression limit of these methods depends on the algorithm itself, and therefore their theoretical compression ratio has an upper limit.

[0004] Streaming data is a new data type. It is an ordered sequence of data items with the properties of being large in volume, continuous, fast, and non-reproducible. Examples include operator management data, financial credit data, web application data, and banking system data.

[0005] Because streaming data is characterized by its large volume, even with existing data compression methods, the efficiency of data compression and transmission remains low. Summary of the Invention

[0006] This invention provides a data transmission method, apparatus, electronic device, storage medium, and product to address the shortcomings of low data compression and transmission efficiency in the prior art.

[0007] This invention provides a data transmission method, comprising: acquiring data to be transmitted; inputting the data to be transmitted into a data compression model to obtain compressed data corresponding to the data to be transmitted output by the data compression model; and sending the compressed data to a data receiving end; wherein the data compression model is trained based on sample transmitted data and sample compressed data corresponding to the sample transmitted data.

[0008] According to a data transmission method provided by the present invention, before acquiring the data to be transmitted, the method further includes: acquiring sample transmission data and sample compressed data corresponding to the sample transmission data; training an initial model constructed based on the MLP algorithm based on the sample transmission data and the sample compressed data corresponding to the sample transmission data to obtain a data compression model; wherein the data compression model is trained using a preset function, and the preset function is the mean squared error loss function.

[0009] According to a data transmission method provided by the present invention, after training an initial model constructed based on the MLP algorithm based on sample transmission data and the corresponding compressed sample data to obtain a compressed data model, the method further includes: uploading the compressed data model to a global model library; deploying the global model library on a central cloud server; distributing the compressed data models in the global model library to sub-model libraries; and deploying the sub-model libraries on edge cloud servers.

[0010] According to a data transmission method provided by the present invention, after uploading the data compression model to the global model library, the method further includes: obtaining a test dataset; the test dataset includes multiple sample transmission data and sample compression data corresponding to the sample transmission data; based on the test dataset, the data compression model is verified to determine whether the data compression model can run correctly, and a judgment result is obtained.

[0011] According to a data transmission method provided by the present invention, after inputting the data to be transmitted into a data compression model and obtaining the compressed data corresponding to the data to be transmitted output by the data compression model, the method further includes: fine-tuning the data compression model based on the data to be transmitted and the compressed data corresponding to the data to be transmitted to obtain an updated data compression model.

[0012] According to a data transmission method provided by the present invention, the number of sub-model libraries is at least one; before inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted output by the data compression model, the method further includes: determining the sub-model library to be accessed; and downloading the data compression model from the sub-model library to be accessed.

[0013] The present invention also provides a data transmission device, comprising: an acquisition module for acquiring data to be transmitted; a compression module for inputting the data to be transmitted into a data compression model to obtain compressed data corresponding to the data to be transmitted output by the data compression model; and a sending module for sending the compressed data to a data receiving end; wherein the data compression model is trained based on sample transmitted data and sample compressed data corresponding to the sample transmitted data.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data transmission method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data transmission method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data transmission method as described above.

[0017] This invention provides a data transmission method, apparatus, electronic device, storage medium, and product. The method involves acquiring data to be transmitted; inputting the data to be transmitted into a data compression model to obtain compressed data corresponding to the data to be transmitted, output by the data compression model; and sending the compressed data to a data receiving end. The data compression model is trained based on sample transmitted data and corresponding sample compressed data. By inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted, and by introducing an artificial intelligence model into the data compression process, the model can perform data feature mining and data compression. This avoids the limitation of compression limits inherent in mathematical statistical methods when using data compression, thereby improving the data compression ratio and ultimately increasing the efficiency of data compression and data transmission. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the data transmission method provided by the present invention;

[0020] Figure 2 This is the second flowchart illustrating the data transmission method provided by the present invention;

[0021] Figure 3 This is a schematic diagram of the training process of the data compression model provided by the present invention;

[0022] Figure 4 This is the third flowchart illustrating the data transmission method provided by the present invention;

[0023] Figure 5 This is a schematic diagram of the data transmission device provided by the present invention;

[0024] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] Please see Figure 1 , Figure 1 This is one of the flowcharts illustrating the data transmission method provided by the present invention. In this embodiment, the data transmission method includes steps S110 to S130, each step being as follows:

[0027] S110: Obtain the data to be transmitted.

[0028] Specifically, the data to be transmitted is streaming data.

[0029] S120: Input the data to be transmitted into the data compression model and obtain the compressed data corresponding to the data to be transmitted output by the data compression model.

[0030] S130: Send compressed data to the data receiving end.

[0031] The data compression model is trained based on the sample transmission data and the corresponding compressed sample data.

[0032] Please see Figure 2 , Figure 2 This is the second flowchart illustrating the data transmission method provided by the present invention.

[0033] like Figure 2 As shown, in this embodiment, a data compression model is first trained using historical data, where the historical data includes sample transmission data and sample compression data corresponding to the sample transmission data.

[0034] Furthermore, after obtaining the trained data compression model, the data to be transmitted (i.e., Figure 2 The real-time data is input into the data compression model to obtain the compressed data corresponding to the data to be transmitted, which is output by the data compression model.

[0035] Furthermore, after obtaining the compressed data corresponding to the data to be transmitted, the compressed data can be sent to the data receiving end.

[0036] It should be noted that the process of compressing real-time data using a data compression model can be understood as a data encoding process; after receiving the compressed data at the data receiving end, the compressed data can be inferred and restored to obtain the real-time data, and the process of inferring and restoring the compressed data can be understood as a data decoding process.

[0037] This embodiment provides a data transmission method that involves acquiring data to be transmitted; inputting the data to be transmitted into a data compression model to obtain compressed data corresponding to the data to be transmitted, output by the data compression model; and sending the compressed data to a data receiving end. The data compression model is trained based on sample transmitted data and corresponding compressed sample data. By inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted, and by introducing an artificial intelligence model into the data compression process, the model can perform data feature mining and data compression. This avoids the limitation of compression limits inherent in mathematical statistical methods when using data compression, thereby improving the data compression ratio and ultimately increasing the efficiency of data compression and data transmission.

[0038] In some embodiments, before acquiring the data to be transmitted, the method further includes: acquiring sample transmission data and sample compressed data corresponding to the sample transmission data; training an initial model constructed based on the MLP algorithm based on the sample transmission data and the sample compressed data corresponding to the sample transmission data to obtain a data compression model; wherein the data compression model is trained using a preset function, which is the mean squared error loss function.

[0039] Please see Figure 3 , Figure 3 This is a schematic diagram of the training process of the data compression model provided by the present invention.

[0040] Historical data includes sample transmission data and corresponding compressed sample data. After obtaining historical data, the initial model built based on the MLP (Multi-Layer Perceptron) algorithm can be trained based on the historical data to obtain a data compressed model.

[0041] Specifically, such as Figure 3 As shown, the initial model built on the MLP algorithm is trained based on massive historical data. The input data are sample transmission data and the corresponding compressed sample data.

[0042] Optionally, the sample transmission data includes data index information, which is used to distinguish different sample transmission data and the corresponding compressed sample data. The data index information can be understood as a data ID.

[0043] For example, in three-dimensional data, sample transmission data needs to include three-dimensional subscripts of the form (x, y, z) as data IDs; the sample compressed data corresponding to the sample transmission data can be identified using the same three-dimensional subscripts.

[0044] It's important to note that the principle behind data compression using models is to mine data features through deep neural networks and then compress the data based on these mined features. Therefore, the better the model's ability to mine data features, the better the data compression performance of the ultimately trained model. Based on this, to improve the model's data feature mining capabilities, third-party knowledge bases can be introduced during model training.

[0045] Specifically, third-party knowledge bases contain knowledge about the distribution of different data (including but not limited to statistical data, streaming data, etc.), and this knowledge about data distribution can be used as features for model training.

[0046] Understandably, depending on the type of third-party knowledge base introduced, the input dimension of the sample transmission data is the data ID + knowledge base dimension. For example, in the case of three-dimensional data, if two types of knowledge bases are introduced, the input dimension of the model is 3 + 2 = 5.

[0047] Specifically, the data compression model is trained using a preset function until the model converges. The preset function is the Mean Square Error (MSE) loss function.

[0048] In some embodiments, after training an initial model based on the MLP algorithm to obtain a data compression model based on sample transmission data and corresponding sample compressed data, the method further includes: uploading the data compression model to a global model library; deploying the global model library on a central cloud server; distributing the data compression model in the global model library to sub-model libraries; and deploying the sub-model libraries on edge cloud servers.

[0049] Understandably, different data senders (i.e., data transmission points) can independently train and use data compression models for data compression at their respective senders. However, this limits the scope of application of the data compression model and also consumes a significant amount of local resources. Therefore, the data compression model can be uploaded to a cloud server. This reduces local resource consumption while allowing other data senders to download the required model from the cloud server as needed, achieving model sharing and expanding the scope of application of the data compression model.

[0050] Specifically, after obtaining the trained data compression model, the data compression model can be serialized and saved, and then encapsulated in a containerized form to obtain the model file. The model file is then uploaded to the global model library, which is deployed on a central cloud server. The global model library is used to save the data compression model and record its version.

[0051] Furthermore, the data compression models in the global model library are distributed over a wide area to sub-model libraries, which are then deployed on edge cloud servers.

[0052] Specifically, the sub-model library is deployed on an edge cloud server with limited storage resources but close proximity to the data sending end and low network latency. The data sending end can save and update models through the nearby sub-model library.

[0053] Optionally, the sub-model library can periodically compress information of the currently stored data model, such as version information, and report it to the global model library. This allows the global model library to distribute the models uploaded by different data senders to each sub-model library according to the information uploaded by each sub-model library and a preset update strategy.

[0054] Optionally, the global model library can update the cache of models in different sub-model libraries based on the historical access records of the sub-model libraries from different data senders. For example, models in the sub-model library can be updated according to the latest access or the most frequent access strategy.

[0055] Optionally, the global model library can record the storage location information of each model and the location information of the data sending end through the broadcast module, and broadcast it to all data sending ends to achieve model sharing; wherein, the storage location information of the model includes the location information of the sub-model library where the model is located.

[0056] Optionally, the broadcast module supports functions such as registration, management, and offline of data senders, and can be used to manage all data senders within the entire system.

[0057] Please see Figure 4 , Figure 4 This is the third flowchart illustrating the data transmission method provided by the present invention.

[0058] like Figure 4 As shown, after the data sending end 1 trains the data compression model using local historical data, it can upload the trained data compression model to the global model library deployed on the central cloud server to achieve model sharing.

[0059] Furthermore, the global model library can distribute the data compression model over a wide area, distributing the model to different sub-model libraries.

[0060] Furthermore, the global model library can also synchronize the model's storage location information (the location information of the sub-model library where the model is located) to different data sending terminals via broadcast.

[0061] For example Figure 4 In this process, the global model library distributes the compressed data model uploaded by data sender 1 across a wide area, distributing the model to different sub-model libraries. It also broadcasts the storage location information of the compressed data model uploaded by data sender 1 (the location information of the sub-model library where the model is located) to data sender 2, so that data sender 2 can download the compressed data model uploaded by data sender 1 from sub-model library 2 according to the principle of proximity based on the storage location information of the compressed data model, thereby realizing model sharing.

[0062] In some embodiments, after uploading the data compression model to the global model library, the method further includes: obtaining a test dataset; the test dataset includes multiple sample transmission data and sample compression data corresponding to the sample transmission data; based on the test dataset, the data compression model is verified to determine whether the data compression model can run correctly and to obtain the judgment result.

[0063] To ensure that the data compression model uploaded by the data sender can function properly, after uploading the data compression model to the global model library, the data compression model can be verified using a test dataset to determine whether the data compression model can run correctly and obtain the judgment result.

[0064] Specifically, after obtaining the trained data compression model, the data compression model can be serialized and saved, and then encapsulated in a containerized form to obtain the model file. The model file is then uploaded to the global model library, which is deployed on a central cloud server. The global model library is used to save the data compression model and record its version.

[0065] The global model library can call model files in a containerized manner and call test datasets; based on the test datasets, the data compression model is verified to determine whether the data compression model can run correctly and obtain the judgment results.

[0066] If the data compression model works correctly, then the data compression model is distributed widely, spreading the model to different sub-model libraries.

[0067] If the data compression model fails to function correctly, the data compression model uploaded by the data sender can be retrieved again.

[0068] Optionally, the global model library can use a preset function to verify the accuracy of the data compression model uploaded by the data sender, wherein the preset function is the mean squared error loss function.

[0069] In some embodiments, after inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted output by the data compression model, the method further includes: fine-tuning the data compression model based on the data to be transmitted and the compressed data corresponding to the data to be transmitted to obtain an updated data compression model.

[0070] Please continue reading. Figure 2 ,like Figure 2 As shown, after inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted output by the data compression model, the data to be transmitted and the compressed data corresponding to the data to be transmitted can be used as real-time data to fine-tune the data compression model and obtain an updated data compression model. Through fine-tuning, the model performance can be optimized and the model accuracy can be improved.

[0071] In some embodiments, the number of sub-model libraries is at least one.

[0072] Before inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted output by the data compression model, the process also includes: determining the sub-model library to be accessed; and downloading the data compression model from the sub-model library to be accessed.

[0073] Specifically, there are usually multiple sub-model libraries. After the global model library distributes the data compression model to each sub-model library, if the data sender needs to download the data compression model from the cloud, it needs to determine the sub-model library to be accessed and download the data compression model from the sub-model library to be accessed.

[0074] Generally, the sub-model library closest to the data sender is selected as the sub-model library to be accessed.

[0075] This embodiment provides a data transmission method that, based on data compression and transmission using deep learning algorithms, integrates the distribution patterns of massive historical data and real-time data characteristics. A model is trained based on this massive historical data and stored as system knowledge. Simultaneously, the model is fine-tuned based on real-time data to optimize performance and improve accuracy. Furthermore, by constructing a multi-level model library architecture with a central-periphery structure, global model distribution is achieved, facilitating model sharing and enabling efficient and high-precision compression of streaming data.

[0076] The present invention also provides a data transmission device; please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of the data transmission device provided by the present invention. In this embodiment, the data transmission device includes an acquisition module 510, a compression module 520, and a transmission module 530.

[0077] The acquisition module 510 is used to acquire the data to be transmitted.

[0078] The compression module 520 is used to input the data to be transmitted into the data compression model and obtain the compressed data corresponding to the data to be transmitted output by the data compression model.

[0079] The sending module 530 is used to send compressed data to the data receiving end.

[0080] The data compression model is trained based on the sample transmission data and the corresponding compressed sample data.

[0081] In some embodiments, the compression module 520 is used to acquire sample transmission data and sample compressed data corresponding to the sample transmission data; and to train an initial model constructed based on the MLP algorithm based on the sample transmission data and the sample compressed data corresponding to the sample transmission data to obtain a data compression model; wherein the data compression model is trained using a preset function, which is the mean squared error loss function.

[0082] In some embodiments, the compression module 520 is used to upload the data compression model to a global model library; the global model library is deployed on a central cloud server; the data compression model in the global model library is distributed to a sub-model library; the sub-model library is deployed on an edge cloud server.

[0083] In some embodiments, the compression module 520 is used to acquire a test dataset; the test dataset includes multiple sample transmission data and sample compressed data corresponding to the sample transmission data; based on the test dataset, the data compression model is verified to determine whether the data compression model can run correctly and to obtain the judgment result.

[0084] In some embodiments, the compression module 520 is used to fine-tune the data compression model based on the data to be transmitted and the compressed data corresponding to the data to be transmitted, so as to obtain an updated data compression model.

[0085] In some embodiments, the number of sub-model libraries is at least one.

[0086] The acquisition module 510 is used to determine the sub-model library to be accessed and to download the data compression model from the sub-model library to be accessed.

[0087] The present invention also provides an electronic device, Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute data transmission methods.

[0088] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the data transmission methods provided by the methods described above.

[0090] The present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the data transmission methods provided by the above methods.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data transmission method, characterized by, include: Get the data to be transmitted; The data to be transmitted is input into the data compression model to obtain the compressed data corresponding to the data to be transmitted output by the data compression model. The compressed data is sent to the data receiving end; The data compression model is trained based on the sample transmission data and the corresponding compressed sample data. Before acquiring the data to be transmitted, the process also includes: Obtain the sample transmission data and the corresponding compressed sample data; Based on the sample transmission data and the corresponding compressed sample data, the initial model constructed based on the MLP algorithm is trained to obtain the data compression model; The data compression model is trained using a preset function, which is the mean squared error loss function. After training the initial model built based on the MLP algorithm using the sample transmission data and the corresponding compressed sample data to obtain the data compression model, the process further includes: The data compression model is uploaded to the global model library; the global model library is deployed on a central cloud server. The data compression model in the global model library is distributed over a wide area to a sub-model library; the sub-model library is deployed on an edge cloud server. After uploading the data compression model to the global model library, the process further includes: Obtain the test dataset; the test dataset includes multiple sample transmission data and the corresponding compressed sample data; Based on the test dataset, the data compression model is verified to determine whether the data compression model can run correctly and to obtain the judgment result. If the data compression model fails to function correctly, the data compression model uploaded by the data sender can be retrieved again. The global model library uses the preset function to verify the accuracy of the data compression model uploaded by the data sending end.

2. The data transmission method of claim 1, wherein, After inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted output by the data compression model, the method further includes: Based on the data to be transmitted and the corresponding compressed data, the data compression model is fine-tuned to obtain an updated data compression model.

3. The data transmission method of claim 1, wherein, The number of sub-model libraries is at least one; Before inputting the data to be transmitted into the data compression model and obtaining the compressed data corresponding to the data to be transmitted output by the data compression model, the method further includes: Determine the sub-model library to be accessed; Download the data compression model from the desired sub-model library.

4. A data transmission apparatus characterized by comprising: include: The acquisition module is used to acquire the data to be transmitted. A compression module is used to input the data to be transmitted into a data compression model and obtain compressed data corresponding to the data to be transmitted output by the data compression model. The sending module is used to send the compressed data to the data receiving end; The data compression model is trained based on the sample transmission data and the corresponding compressed sample data. The compression module is used to acquire the sample transmission data and the corresponding compressed sample data; based on the sample transmission data and the corresponding compressed sample data, to train the initial model constructed based on the MLP algorithm to obtain the data compression model; wherein, the data compression model is trained using a preset function, and the preset function is the mean squared error loss function; The compression module is used to upload the data compression model to a global model library; the global model library is deployed on a central cloud server; the data compression model in the global model library is distributed to sub-model libraries; the sub-model libraries are deployed on edge cloud servers. The compression module is used to acquire a test dataset; the test dataset includes multiple sample transmission data and sample compressed data corresponding to the sample transmission data; based on the test dataset, the data compression model is verified to determine whether the data compression model can run correctly and to obtain a judgment result; if the data compression model cannot run correctly, the data compression model uploaded by the data sending end can be reacquired; the global model library uses the preset function to verify the accuracy of the data compression model uploaded by the data sending end.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the data transmission method as described in any one of claims 1 to 3. 6.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the data transmission method as described in any one of claims 1 to 3.

7. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the data transmission method as described in any one of claims 1 to 3.

Citation Information

Patent Citations

  • Object recognition method and device based on edge calculation, equipment and medium

    CN113688796A

  • Image compression method and device, electronic equipment and storage medium

    CN114979665A