Multimedia data transmission method, device, electronic device and storage medium
By segmenting and encoding the multimedia data packets, and classifying and encoding according to the similarity of standard media data samples, the problem of low multimedia data transmission efficiency is solved and more efficient data transmission is achieved.
Patent Information
- Application Number
- CN202410332445.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-22
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-03-22
AI Technical Summary
In the prior art, multimedia data transmission efficiency is low and the speed of data transmission cannot be effectively improved.
By obtaining multimedia data packets and dividing them into arrays according to their format, the arrays are divided into two categories according to the similarity of standard media data samples, encoded and arranged in position order and transmitted.
The transmission efficiency is improved because the encoded data volume is smaller than the original data volume, which significantly improves the transmission speed.
Smart Images

Figure CN118075210B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data transmission, and in particular to a multimedia data transmission method, device, electronic equipment and storage medium. Background Art
[0002] The information stored by file storage units is also changing with the times. At present, various digitally stored data have begun to completely replace paper data, and audio, images and videos have also been added to the database of stored data. The problem caused by this is that it takes a long time to upload.
[0003] In order to improve the efficiency of file transfer, people have come up with various methods, such as queuing data according to priority, dividing data into multiple segments and uploading them in parallel or in segments, etc. However, in essence, these methods only improve the security and reliability of data transmission, and the overall data transmission efficiency is not improved due to the lack of application of the above transmission means.
[0004] Based on this, it is necessary to develop and design a multimedia data transmission method. Summary of the invention
[0005] The embodiments of the present invention provide a multimedia data transmission method, device, electronic device and storage medium, which are used to solve the problem of low efficiency of multimedia data transmission in the prior art.
[0006] In a first aspect, an embodiment of the present invention provides a multimedia data transmission method, including:
[0007] Acquire multiple first arrays, wherein the first array is an array obtained by segmenting the multimedia data packet according to the format of the multimedia data packet;
[0008] Dividing the plurality of first arrays into a plurality of second arrays and a plurality of third arrays according to similarities with a plurality of standard media data samples, wherein the similarities between the second arrays and the plurality of standard media data samples are greater than a similarity threshold;
[0009] Inputting the plurality of second arrays into a media coding model in sequence to obtain a plurality of second transmission codes;
[0010] According to the positions of the plurality of second arrays and the plurality of third arrays in the multimedia data packet, the plurality of second transmission codes and the plurality of third arrays are sequentially arranged to obtain coded media, and the coded media is transmitted.
[0011] In a possible implementation manner, the plurality of standard media data samples are obtained according to a plurality of media data packet samples, including:
[0012] Performing data segmentation on the plurality of media data packet samples respectively to obtain a plurality of first sample arrays, and constructing a sample value according to the data in each first sample array, thereby obtaining a plurality of sample values;
[0013] Dividing the plurality of sample values into a plurality of intervals according to the magnitude of the values;
[0014] Counting the ratio of the number of sample values in each interval to the number of the multiple sample values as the interval proportion, thereby obtaining multiple interval proportions;
[0015] The interval whose interval proportion is greater than the proportion threshold is taken as the target interval, thereby obtaining multiple target intervals;
[0016] Using the source array of each target sample value as the second sample array, thereby obtaining a plurality of second sample arrays, wherein the target sample value is a sample value of the target interval;
[0017] Clustering the plurality of second sample arrays to obtain a plurality of media data classes;
[0018] The class centers of the multiple media data classes are used as the multiple standard media data samples.
[0019] In a possible implementation manner, clustering the plurality of second sample arrays to obtain a plurality of media data classes includes:
[0020] Get the distance threshold;
[0021] Randomly assign a sample array from the second sample array that has not been clustered as a cluster array;
[0022] Searching in the unclustered second sample array for an array whose distance to the cluster array is less than the distance threshold;
[0023] If there is an array whose distance to the clustering array is less than the distance threshold, then the array whose distance to the clustering array is less than the distance threshold is added to the multimedia data class of the clustering array, the array whose distance to the clustering array is less than the distance threshold is used as the clustering array, and the process jumps to the step of searching in the unclustered second sample array for an array whose distance to the clustering array is less than the distance threshold;
[0024] If there is no array whose distance from the cluster array is less than the distance threshold and there is an unclustered second sample array, jump to the step of randomly specifying a sample array from the unclustered second sample array as the cluster array;
[0025] Otherwise, extract the class centers of multiple multimedia data classes as multiple class centers, perform pairwise correlation calculations on the multiple class centers respectively, so as to obtain multiple correlation values. If there is a correlation value higher than the correlation threshold among the multiple correlation values, increase the distance threshold, and jump to the step of randomly specifying a sample array as the clustering array in the second sample array that has not been clustered.
[0026] In a possible implementation, the correlation value is determined according to a first formula, which is:
[0027]
[0028] In the formula, R(i,j) is the correlation value between the i-th class center and the j-th class center, CA i (dn) is the dnth data of the i-th class center, CA j (dn) is the dnth data of the jth class center, and dn_max is the total number of data in the first sample array.
[0029] In a possible implementation, the media coding model is constructed according to a plurality of second sample arrays, the second sample arrays are obtained by segmenting a media data packet, and the construction process of the media coding model includes:
[0030] Obtaining a media coding base model and a plurality of second sample arrays;
[0031] Dividing the plurality of second sample arrays into two parts to obtain a plurality of third sample arrays and a plurality of fourth sample arrays;
[0032] Inputting the plurality of third sample arrays into the first part of the media coding base model in sequence to obtain a plurality of first codes, wherein the first part of the media coding base model has one code output and a plurality of inputs having the same number of dimensions as the second sample array, and when the plurality of data of the third sample array are correspondingly input into the first part of the media coding base model, the first part of the media coding base model outputs the code;
[0033] Inputting the plurality of first codes sequentially into a second part of the media coding base model to obtain a plurality of fifth sample arrays, wherein the second part of the media coding base model has one code input and a plurality of outputs having the same number of dimensions as the second sample array, and when the first code is input into the second part of the media coding base model, a plurality of data of the sample array is outputted in the second part of the media coding base model;
[0034] Calculating deviations between the plurality of fifth sample arrays and the plurality of third sample arrays as first deviations;
[0035] If the absolute value of the first deviation is greater than the deviation threshold, adjusting multiple parameters of the media coding base model according to the first deviation, and jumping to the step of sequentially inputting the multiple third sample arrays into the first part of the media coding base model to obtain multiple first codes;
[0036] Otherwise, the plurality of fourth sample arrays are sequentially input into the first part of the media coding base model to obtain a plurality of second codes, the plurality of second codes are sequentially input into the second part of the media coding base model to obtain a plurality of sixth sample arrays, deviations between the plurality of sixth sample arrays and the plurality of fourth sample arrays are calculated as second deviations, and if the absolute value of the second deviation is greater than the deviation threshold, the structure of the media coding base model is simplified, and the process is jumped to the step of sequentially inputting the plurality of third sample arrays into the first part of the media coding base model to obtain a plurality of first codes;
[0037] The media coding base model is used as the media coding model.
[0038] In a possible implementation, the media coding basic model is:
[0039]
[0040] Where Y1 is the output of the first part of the media coding basic model, index_max is the total index number, X1 is the joint input, and x inx is the inxth input of the first part of the media coding basic model, X2 is the input of the second part of the media coding basic model, Y2 is the joint output, y outnx is the outnxth output of the second part of the media coding basic model, α1, α2, α3 and α4 are the first constant, the second constant, the third constant and the fourth constant respectively, win index 、wx inx 、wout index and wy outnx They are the indexth first parameter, the inxth second parameter, the indexth third parameter, and the outnxth fourth parameter respectively.
[0041] In a possible implementation manner, the transmitting the encoded media includes:
[0042] Acquire the type and format of the multimedia data packet according to the multiple first arrays;
[0043] The type and format of transmitting the multimedia data packet;
[0044] Transport coded media.
[0045] In a second aspect, an embodiment of the present invention provides a multimedia data transmission device, which is used to implement the multimedia data transmission method described in the first aspect or any possible implementation of the first aspect, wherein the multimedia data transmission device includes:
[0046] A data acquisition module, used to acquire a plurality of first arrays, wherein the first array is an array obtained by segmenting the multimedia data packet according to the format of the multimedia data packet;
[0047] a data classification module, configured to classify the plurality of first arrays into a plurality of second arrays and a plurality of third arrays according to similarities with a plurality of standard media data samples, wherein the similarities between the second arrays and the plurality of standard media data samples are greater than a similarity threshold;
[0048] A coding acquisition module, used for inputting the plurality of second arrays into a media coding model in sequence to obtain a plurality of second codes;
[0049] as well as,
[0050] The data transmission module is used to sequentially arrange the multiple second codes and the multiple third arrays according to the positions of the multiple second arrays and the multiple third arrays in the multimedia data packet, obtain the coded media, and transmit the coded media.
[0051] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the steps of the method described in the first aspect or any possible implementation method of the first aspect.
[0052] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.
[0053] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0054] The embodiment of the present invention discloses a multimedia data transmission method, which first obtains a plurality of first arrays, wherein the first array is an array obtained by segmenting the multimedia data packet according to the format of the multimedia data packet; then, according to the similarity with a plurality of standard media data samples, the plurality of first arrays are divided into a plurality of second arrays and a plurality of third arrays, wherein the similarity between the second array and the plurality of standard media data samples is greater than a similarity threshold; then, the plurality of second arrays are sequentially input into a media coding model to obtain a plurality of second codes; finally, according to the positions of the plurality of second arrays and the plurality of third arrays in the multimedia data packet, the plurality of second codes and the plurality of third arrays are sequentially arranged to obtain coded media, and the coded media is transmitted. The embodiment of the present invention classifies arrays, performs coding processing on arrays with a higher frequency of occurrence, and transmits them through coding. Since the amount of coded data is less than the amount of array data, the transmission efficiency is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0056] Figure 1 is a flow chart of a multimedia data transmission method provided by an embodiment of the present invention;
[0057] Figure 2 is a functional block diagram of a media coding model provided by an embodiment of the present invention;
[0058] Figure 3 is a functional block diagram of a multimedia data transmission device provided by an embodiment of the present invention;
[0059] Figure 4 It is a functional block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0060] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted so as not to obscure the description of the present invention with unnecessary details.
[0061] In order to make the purpose, technical solutions and advantages of the present invention more clear, a specific implementation method will be described below in conjunction with the accompanying drawings.
[0062] The following is a detailed description of an embodiment of the present invention. This example is implemented based on the technical solution of the present invention, and provides a detailed implementation method and a specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0063] Figure 1 The present invention provides a flowchart of a multimedia data transmission method.
[0064] like Figure 1 As shown, it shows a flowchart of the implementation method of the multimedia data transmission method provided by the embodiment of the present invention, which is described in detail as follows:
[0065] In step 101, a plurality of first arrays are obtained, wherein the first arrays are arrays obtained by segmenting the multimedia data packets according to the formats of the multimedia data packets.
[0066] In step 102, the plurality of first arrays are divided into a plurality of second arrays and a plurality of third arrays according to similarities with a plurality of standard media data samples, wherein the similarity between the second arrays and the plurality of standard media data samples is greater than a similarity threshold.
[0067] In some implementations, the plurality of standard media data samples are obtained based on a plurality of media data packet samples, including:
[0068] Performing data segmentation on the plurality of media data packet samples respectively to obtain a plurality of first sample arrays, and constructing a sample value according to the data in each first sample array, thereby obtaining a plurality of sample values;
[0069] Dividing the plurality of sample values into a plurality of intervals according to the magnitude of the values;
[0070] Counting the ratio of the number of sample values in each interval to the number of the multiple sample values as the interval proportion, thereby obtaining multiple interval proportions;
[0071] The interval whose interval proportion is greater than the proportion threshold is taken as the target interval, thereby obtaining multiple target intervals;
[0072] Using the source array of each target sample value as the second sample array, thereby obtaining a plurality of second sample arrays, wherein the target sample value is a sample value of the target interval;
[0073] Clustering the plurality of second sample arrays to obtain a plurality of media data classes;
[0074] The class centers of the multiple media data classes are used as the multiple standard media data samples.
[0075] In some implementations, clustering the plurality of second sample arrays to obtain a plurality of media data classes includes:
[0076] Get the distance threshold;
[0077] Randomly assign a sample array from the second sample array that has not been clustered as a cluster array;
[0078] Searching in the unclustered second sample array for an array whose distance to the cluster array is less than the distance threshold;
[0079] If there is an array whose distance to the clustering array is less than the distance threshold, then the array whose distance to the clustering array is less than the distance threshold is added to the multimedia data class of the clustering array, the array whose distance to the clustering array is less than the distance threshold is used as the clustering array, and the process jumps to the step of searching in the unclustered second sample array for an array whose distance to the clustering array is less than the distance threshold;
[0080] If there is no array whose distance from the cluster array is less than the distance threshold and there is an unclustered second sample array, jump to the step of randomly specifying a sample array from the unclustered second sample array as the cluster array;
[0081] Otherwise, extract the class centers of multiple multimedia data classes as multiple class centers, perform pairwise correlation calculations on the multiple class centers respectively, so as to obtain multiple correlation values. If there is a correlation value higher than the correlation threshold among the multiple correlation values, increase the distance threshold, and jump to the step of randomly specifying a sample array as the clustering array in the second sample array that has not been clustered.
[0082] In some implementations, the correlation value is determined according to a first formula, wherein the first formula is:
[0083]
[0084] In the formula, R(i,j) is the correlation value between the i-th class center and the j-th class center, CA i (dn) is the dnth data of the i-th class center, CA j (dn) is the dnth data of the jth class center, and dn_max is the total number of data in the first sample array.
[0085] Exemplarily, multimedia in an embodiment of the present invention includes multiple forms, such as: images, audio, video, etc., and multiple arrays are obtained by segmenting these multimedia data packets. It should be noted that in some application scenarios, the array sizes obtained by segmenting different multimedia data packet formats are different. For example, the array obtained by segmenting a picture in JPG format is smaller, while the array obtained by segmenting a video in AVI format is larger. The purpose of this setting is to balance the transmission efficiency and the accuracy of the transmitted data.
[0086] As we know, a small deviation in image, audio or video data has little effect on the expression of the image, audio or video. Based on this, the embodiment of the present invention divides the multimedia data packet into multiple arrays (first array), encodes the array with a higher frequency in the multiple arrays, and transmits it through encoding. The arrays with a lower frequency of occurrence are transmitted using the original code, and the encoding received at the receiving end is restored through the decoding model. The array obtained after restoration is recombined with the array transmitted by the original code to complete the transmission of the multimedia data packet.
[0087] To achieve the above-mentioned purpose, after acquiring multiple first arrays, the implementation manner of the present invention classifies these first arrays, one of which is an array with a higher frequency of occurrence, and the other is an array with a lower frequency of occurrence. The implementation manner of the present invention judges the similarity between the first array and multiple standard media data samples. When the similarity between the first array and at least one of the multiple standard media data samples is higher than a similarity threshold, the first array is determined to be an array with a higher frequency of occurrence (second array), and when the similarities with multiple standard media data samples are all low, the first array is determined to be an array with a lower frequency of occurrence (third array).
[0088] Here, we have to explain the origin of multiple standard media data samples. In some application scenarios, standard media data samples are selected from the sample arrays obtained after segmenting multiple media data packets, and those sample arrays with higher occurrence frequencies are selected, and then clustered and constructed according to the cluster center.
[0089] Specifically, after obtaining multiple first sample arrays, a sample value is constructed according to each sample array. For example, a sample array contains data: N1, N1, N3, and the values of these data are 35, 42, and 75 respectively. Then the sample value constructed according to this sample array is 354275. The multiple sample values are arranged in sequence and divided into equal intervals according to the value intervals (for example, the value intervals are all 256). We can find that some intervals have more values and other intervals have fewer values. At this time, these relatively small times are removed and the remaining intervals are used as target intervals. According to the values in the target interval, the corresponding first sample array (second sample array) is found from multiple first sample arrays, the second sample array is clustered, and the cluster center is extracted as the standard media data template sample.
[0090] In some application scenarios, the clustering method is to set a distance threshold, and search the distance clustering array for the second sample array within the distance threshold radius according to the distance threshold, until all the second sample arrays are clustered to obtain multiple media data classes.
[0091] At this time, the centers of multiple media data classes are extracted to perform pairwise correlation calculations. If there is a value higher than the correlation threshold among the obtained correlation values, the distance threshold is adjusted. The embodiment of the present invention adopts a method of increasing the distance threshold and clustering is performed again until the pairwise correlation coefficients of the centers of multiple media data classes are all higher than the correlation threshold.
[0092] In some application scenarios, the pairwise correlation is determined according to the first formula:
[0093]
[0094] In the formula, R(i,j) is the correlation value between the i-th class center and the j-th class center, CA i (dn) is the dnth data of the i-th class center, CA j (dn) is the dnth data of the jth class center, and dn_max is the total number of data in the first sample array.
[0095] The implementation manner of the present invention classifies arrays, encodes arrays with higher occurrence frequencies, and transmits them through encoding. Since the amount of encoded data is smaller than the amount of array data, the transmission efficiency is greatly improved.
[0096] In step 103, the multiple second arrays are sequentially input into a media coding model to obtain multiple second transmission codes.
[0097] In some implementations, the media coding model is constructed according to a plurality of second sample arrays, the second sample arrays are obtained by segmenting a media data packet, and the construction process of the media coding model includes:
[0098] Obtaining a media coding base model and a plurality of second sample arrays;
[0099] Dividing the plurality of second sample arrays into two parts to obtain a plurality of third sample arrays and a plurality of fourth sample arrays;
[0100] Inputting the plurality of third sample arrays into the first part of the media coding base model in sequence to obtain a plurality of first codes, wherein the first part of the media coding base model has one code output and a plurality of inputs having the same number of dimensions as the second sample array, and when the plurality of data of the third sample array are correspondingly input into the first part of the media coding base model, the first part of the media coding base model outputs the code;
[0101] Inputting the plurality of first codes sequentially into a second part of the media coding base model to obtain a plurality of fifth sample arrays, wherein the second part of the media coding base model has one code input and a plurality of outputs having the same number of dimensions as the second sample array, and when the first code is input into the second part of the media coding base model, a plurality of data of the sample array is outputted in the second part of the media coding base model;
[0102] Calculating deviations between the plurality of fifth sample arrays and the plurality of third sample arrays as first deviations;
[0103] If the absolute value of the first deviation is greater than the deviation threshold, adjusting multiple parameters of the media coding base model according to the first deviation, and jumping to the step of sequentially inputting the multiple third sample arrays into the first part of the media coding base model to obtain multiple first codes;
[0104] Otherwise, the plurality of fourth sample arrays are sequentially input into the first part of the media coding base model to obtain a plurality of second codes, the plurality of second codes are sequentially input into the second part of the media coding base model to obtain a plurality of sixth sample arrays, deviations between the plurality of sixth sample arrays and the plurality of fourth sample arrays are calculated as second deviations, and if the absolute value of the second deviation is greater than the deviation threshold, the structure of the media coding base model is simplified, and the process is jumped to the step of sequentially inputting the plurality of third sample arrays into the first part of the media coding base model to obtain a plurality of first codes;
[0105] The media coding base model is used as the media coding model.
[0106] In some implementations, the media coding base model is:
[0107]
[0108] Where Y1 is the output of the first part of the media coding basic model, index_max is the total index number, X1 is the joint input, and x inx is the inxth input of the first part of the media coding basic model, X2 is the input of the second part of the media coding basic model, Y2 is the joint output, y outnx is the outnxth output of the second part of the media coding basic model, α1, α2, α3 and α4 are the first constant, the second constant, the third constant and the fourth constant respectively, win index 、wx inx 、wout index and wy outnx They are the indexth first parameter, the inxth second parameter, the indexth third parameter, and the outnxth fourth parameter respectively.
[0109] Exemplarily, as mentioned above, the embodiment of the present invention inputs the array into the media coding model for coding. In fact, the media coding model is constructed according to the second sample array.
[0110] like Figure 2 As shown, this model includes two parts, wherein the first part 201 is used to generate a code after inputting multiple data of an array, and the second part 202 is used to generate multiple data of an array according to the code. For the data sending end, only the first part 201 is needed, and the second part 202 can be set at the receiving end. The sending end encodes the data through the first part 201 of the model, and the receiving end decodes the data through the second part 202 of the model.
[0111] In terms of model construction, the implementation method of the present invention first obtains a basic model, which is:
[0112]
[0113] Where Y1 is the output of the first part of the media coding basic model, index_max is the total index number, X1 is the joint input, and x inx is the inxth input of the first part of the media coding basic model, X2 is the input of the second part of the media coding basic model, Y2 is the joint output, y outnx is the outnxth output of the second part of the media coding basic model, α1, α2, α3 and α4 are the first constant, the second constant, the third constant and the fourth constant respectively, win index 、wx inx 、wout index and wy outnxThey are the indexth first parameter, the inxth second parameter, the indexth third parameter, and the outnxth fourth parameter respectively.
[0114] Then, the multiple second sample arrays are divided into two parts, one of which is a combination of multiple third sample numbers, and the other is a plurality of fourth sample arrays. The multiple third sample arrays are input into the basic model to obtain multiple first codes, and then the multiple first codes are input into the second part of the basic model to generate multiple fifth sample arrays, and the deviations between the multiple fifth sample arrays and the multiple third sample arrays are calculated, for example, the distance between the fifth sample array and the third sample array is calculated using the Euclidean distance, and the multiple distances are further accumulated to obtain the deviations. If the absolute value of the deviation is greater than the deviation threshold, multiple parameters of the model are adjusted according to the deviation, and then the step of inputting the third sample array is jumped to.
[0115] If the absolute value of the deviation is less than the deviation threshold, the fourth sample array is input into the basic model to obtain the second code, and then the second code is input into the basic model to obtain the sixth sample array, and the deviation between the sixth sample array and the fourth sample array is calculated. If the deviation is greater than the threshold, the number of total indexes in the basic model is reduced, and the step of inputting the third sample array into the basic model is jumped to.
[0116] If the deviation obtained after inputting the fourth sample array into the basic model is less than the deviation threshold, it means that the model is successfully constructed, and the media coding basic model is used as the media coding model.
[0117] The embodiment of the present invention constructs a media coding model with two parts through a sample array, wherein the first part is used to encode the array, and the second part is used to generate the array according to the code. Since the two parts are constructed synchronously, the first part is used to generate the code and send it, and the second part is used to receive and decode to obtain the array, the synchronous construction ensures the correspondence between the code and the array, and ensures the accuracy of decoding and restoring the array.
[0118] In step 104, the plurality of second transmission codes and the plurality of third arrays are sequentially arranged according to the positions of the plurality of second arrays and the plurality of third arrays in the multimedia data packet to obtain coded media, and the coded media is transmitted.
[0119] In some implementations, transmitting the encoded media includes:
[0120] Acquire the type and format of the multimedia data packet according to the multiple first arrays;
[0121] The type and format of transmitting the multimedia data packet;
[0122] Transport coded media.
[0123] Exemplarily, in an embodiment of the present invention, the second coding and the third data array are sorted according to the positions of the second data array and the third data array in the multimedia data packet to obtain the coded media, and then the coded media is transmitted.
[0124] In some application scenarios, the type and format of the multimedia data packet are also obtained. For example, if the type of the multimedia data packet obtained is video and the format is AVI, the video and AVI are sent as the data headers, and the encoded media is sent after the sending is completed.
[0125] At the receiving end, after receiving the media type and format, the media coding model corresponding to the media type and format is selected, and the coding is extracted from the coded media and input into the media coding model, the array is restored, and after restoration, it is spliced with the existing array to complete the restoration of the multimedia data packet.
[0126] In addition, in some scenarios that require accurate transmission, after the multimedia data packet data is restored, the multimedia data packet check value must be calculated. When the received check value deviates from the check value of the sender, the multimedia data packet at the receiving end is divided into two segments, and the check value of each segment is calculated and compared with the check values of the two segments of the sender to find the data segment with deviation. The data segment with deviation is divided again. When the array with deviation is located back and forth, according to the position of the array, the sender is requested to send the array at this position again and replace it, so as to ensure the accuracy of the transmitted multimedia data packet.
[0127] The multimedia data transmission method implementation method of the present invention first obtains a plurality of first arrays, wherein the first array is an array obtained by segmenting the multimedia data packet according to the format of the multimedia data packet; then, according to the similarity with a plurality of standard media data samples, the plurality of first arrays are divided into a plurality of second arrays and a plurality of third arrays, wherein the similarity between the second array and the plurality of standard media data samples is greater than a similarity threshold; then, the plurality of second arrays are sequentially input into the media coding model to obtain a plurality of second codes; finally, according to the positions of the plurality of second arrays and the plurality of third arrays in the multimedia data packet, the plurality of second codes and the plurality of third arrays are sequentially arranged to obtain coded media, and the coded media is transmitted. The implementation method of the present invention classifies arrays, performs coding processing on arrays with a higher frequency of occurrence, and transmits them through coding. Since the amount of coded data is less than the amount of array data, the transmission efficiency is greatly improved.
[0128] It should be understood that the size of the serial numbers of the steps in the above implementation does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation method of the present invention.
[0129] The following is an implementation of the device of the present invention. For details not described in detail, reference may be made to the corresponding method implementation described above.
[0130] Figure 3 is a functional block diagram of a multimedia data transmission device provided by an embodiment of the present invention, referring to Figure 3 The multimedia data transmission device includes: a data acquisition module 301, a data classification module 302, a coding acquisition module 303 and a data transmission module 304, wherein:
[0131] The data acquisition module 301 is used to acquire a plurality of first arrays, wherein the first array is an array obtained by segmenting the multimedia data packet according to the format of the multimedia data packet;
[0132] A data classification module 302, configured to classify the plurality of first arrays into a plurality of second arrays and a plurality of third arrays according to similarities with a plurality of standard media data samples, wherein the similarity of the second arrays with the plurality of standard media data samples is greater than a similarity threshold;
[0133] A code acquisition module 303, used for sequentially inputting the plurality of second arrays into a media coding model to obtain a plurality of second codes;
[0134] The data transmission module 304 is used to sequentially arrange the multiple second codes and the multiple third arrays according to the positions of the multiple second arrays and the multiple third arrays in the multimedia data packet, obtain the coded media, and transmit the coded media.
[0135] Figure 4 is a functional block diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 400 and a memory 401, wherein the memory 401 stores a computer program 402 that can be run on the processor 400. When the processor 400 executes the computer program 402, the steps in the above-mentioned various multimedia data transmission methods and embodiments are implemented, such as Figure 1 Steps 101 to 104 are shown.
[0136] Exemplarily, the computer program 402 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 401 and executed by the processor 400 to implement the present invention.
[0137] The electronic device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 4 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 4 may also include input and output devices, network access devices, buses, etc.
[0138] The processor 400 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0139] The memory 401 may be an internal storage unit of the electronic device 4, such as a hard disk or memory of the electronic device 4. The memory 401 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. Further, the memory 401 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 401 is used to store the computer program 402 and other programs and data required by the electronic device 4. The memory 401 may also be used to temporarily store data that has been output or is to be output.
[0140] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the implementation method can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method implementation method, which will not be repeated here.
[0141] In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0142] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0143] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0145] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0146] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned implementation method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various methods and device implementation methods. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.
[0147] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A multimedia data transmission method, It is characterized in that include: Acquire multiple first arrays, wherein the first array is an array obtained by segmenting the multimedia data packet according to the format of the multimedia data packet; Dividing the plurality of first arrays into a plurality of second arrays and a plurality of third arrays according to similarities with a plurality of standard media data samples, wherein the similarities between the second arrays and the plurality of standard media data samples are greater than a similarity threshold; Inputting the plurality of second arrays into a media coding model in sequence to obtain a plurality of second transmission codes; According to the positions of the plurality of second arrays and the plurality of third arrays in the multimedia data packet, the plurality of second transmission codes and the plurality of third arrays are sequentially arranged to obtain coded media, and the coded media is transmitted; in, The plurality of standard media data samples are obtained according to a plurality of media data packet samples, including: Performing data segmentation on the plurality of media data packet samples respectively to obtain a plurality of first sample arrays, and constructing a sample value according to the data in each first sample array, thereby obtaining a plurality of sample values; Dividing the plurality of sample values into a plurality of intervals according to the magnitude of the values; Counting the ratio of the number of sample values in each interval to the number of the multiple sample values as the interval proportion, thereby obtaining multiple interval proportions; The interval whose interval proportion is greater than the proportion threshold is taken as the target interval, thereby obtaining multiple target intervals; Using the source array of each target sample value as the second sample array, thereby obtaining a plurality of second sample arrays, wherein the target sample value is a sample value of the target interval; Clustering the plurality of second sample arrays to obtain a plurality of media data classes; The class centers of the multiple media data classes are used as the multiple standard media data samples.
2. The multimedia data transmission method according to claim 1, It is characterized in that The step of clustering the plurality of second sample arrays to obtain a plurality of media data classes includes: Get the distance threshold; Randomly assign a sample array from the second sample array that has not been clustered as a cluster array; Searching in the unclustered second sample array for an array whose distance to the cluster array is less than the distance threshold; If there is an array whose distance to the clustering array is less than the distance threshold, then the array whose distance to the clustering array is less than the distance threshold is added to the multimedia data class of the clustering array, the array whose distance to the clustering array is less than the distance threshold is used as the clustering array, and the process jumps to the step of searching in the unclustered second sample array for an array whose distance to the clustering array is less than the distance threshold; If there is no array whose distance from the cluster array is less than the distance threshold and there is an unclustered second sample array, jumping to the step of randomly specifying a sample array from the unclustered second sample array as the cluster array; Otherwise, extract the class centers of multiple multimedia data classes as multiple class centers, perform pairwise correlation calculations on the multiple class centers respectively, so as to obtain multiple correlation values. If there is a correlation value higher than the correlation threshold among the multiple correlation values, increase the distance threshold, and jump to the step of randomly specifying a sample array as the clustering array in the second sample array that has not been clustered.
3. The multimedia data transmission method according to claim 2, It is characterized in that The correlation value is determined according to a first formula, which is: In the formula, For the Class Center and The correlation value of the cluster center, For the The first data, For the The first data, is the total number of data in the first sample array.
4. The multimedia data transmission method according to claim 1, It is characterized in that The media coding model is constructed according to a plurality of second sample arrays, the second sample arrays are obtained by segmenting the media data packets, and the construction process of the media coding model includes: Obtaining a media coding base model and a plurality of second sample arrays; Dividing the plurality of second sample arrays into two parts to obtain a plurality of third sample arrays and a plurality of fourth sample arrays; Inputting the plurality of third sample arrays into the first part of the media coding base model in sequence to obtain a plurality of first codes, wherein the first part of the media coding base model has one code output and a plurality of inputs having the same number of dimensions as the second sample array, and when the plurality of data of the third sample array are correspondingly input into the first part of the media coding base model, the first part of the media coding base model outputs the code; Inputting the plurality of first codes sequentially into a second part of the media coding base model to obtain a plurality of fifth sample arrays, wherein the second part of the media coding base model has one code input and a plurality of outputs having the same number of dimensions as the second sample array, and when the first code is input into the second part of the media coding base model, a plurality of data of the sample array is outputted in the second part of the media coding base model; Calculating deviations between the plurality of fifth sample arrays and the plurality of third sample arrays as first deviations; If the absolute value of the first deviation is greater than the deviation threshold, adjusting multiple parameters of the media coding base model according to the first deviation, and jumping to the step of sequentially inputting the multiple third sample arrays into the first part of the media coding base model to obtain multiple first codes; Otherwise, the plurality of fourth sample arrays are sequentially input into the first part of the media coding base model to obtain a plurality of second codes, the plurality of second codes are sequentially input into the second part of the media coding base model to obtain a plurality of sixth sample arrays, deviations between the plurality of sixth sample arrays and the plurality of fourth sample arrays are calculated as second deviations, and if the absolute value of the second deviation is greater than the deviation threshold, the structure of the media coding base model is simplified, and the process is jumped to the step of sequentially inputting the plurality of third sample arrays into the first part of the media coding base model to obtain a plurality of first codes; The media coding base model is used as the media coding model.
5. The multimedia data transmission method according to claim 4, It is characterized in that The media coding basic model is: In the formula, is the output of the first part of the media coding basic model, is the total number of exponentials, is the combined input quantity, This is the first part of the Media Coding Foundation Model. Inputs, The input to the second part of the media coding base model, is the combined output, The second part of the Media Coding Foundation Model outputs, , , as well as are the first constant, the second constant, the third constant and the fourth constant respectively, , , as well as Respectively The first parameter, The second parameter, The third parameter and The fourth parameter, The total number of input nodes for the first part of the media coding infrastructure model, The total number of output nodes of the second part of the media coding base model.
6. The multimedia data transmission method according to any one of claims 1 to 5, It is characterized in that The transmitting of the encoded media comprises: Acquire the type and format of the multimedia data packet according to the multiple first arrays; The type and format of transmitting the multimedia data packet; Transport coded media.
7. A multimedia data transmission device, It is characterized in that Used to implement the multimedia data transmission method according to any one of claims 1 to 6, the multimedia data transmission device comprises: A data acquisition module, used to acquire a plurality of first arrays, wherein the first array is an array obtained by segmenting the multimedia data packet according to the format of the multimedia data packet; a data classification module, configured to classify the plurality of first arrays into a plurality of second arrays and a plurality of third arrays according to similarities with a plurality of standard media data samples, wherein the similarities between the second arrays and the plurality of standard media data samples are greater than a similarity threshold; A coding acquisition module, used for inputting the plurality of second arrays into a media coding model in sequence to obtain a plurality of second codes; as well as, The data transmission module is used to sequentially arrange the multiple second codes and the multiple third arrays according to the positions of the multiple second arrays and the multiple third arrays in the multimedia data packet, obtain the coded media, and transmit the coded media.
8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
