Audio and video coding method and device based on adaptive filtering, equipment and storage medium

Abstract

The invention discloses an audio and video coding method and device based on adaptive filtering, equipment and a storage medium, and relates to the technical field of audio and video coding, and the method comprises the steps: carrying out the filtering processing of an audio and video file based on an improved adaptive filtering algorithm and a coding level, and obtaining an audio and video filtering file, segmenting the audio and video filtering file according to the coding level to obtain audio and video segmented files, carrying out clustering analysis on the audio and video segmented files to obtain a plurality of segmented file sets, and respectively determining core segmented files in the segmented file sets, according to the core segmented file and the file transformation relation between other segmented files in the segmented file set and the core segmented file, audio and video coding is carried out based on the core segmented file and the file transformation relation to obtain a coded file, redundant files in the audio and video files can be greatly reduced, and the size of the audio and video files is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of audio and video coding, and in particular to an audio and video coding method, apparatus, device and storage medium based on adaptive filtering. Background Art

[0002] In the current digital media era, the transmission efficiency and storage cost of audio and video files are two important considerations. With the rapid development of the Internet and mobile devices, the transmission and processing of audio and video data has become more and more common and important. Audio and video files are usually large in size, which requires a lot of bandwidth and time for data transmission. At present, in order to reduce the file size of audio and video files, high-compression encoding methods such as H.264 / AVC and H.265 / HEVC are usually used. However, the data after encoding and compression is still relatively large, which will still cause a heavy burden on data transmission and storage, which is not conducive to data transmission and data storage.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present application is to provide an audio and video encoding method, device, equipment and storage medium based on adaptive filtering, aiming to solve the technical problem of large file size after audio and video file encoding in the prior art.

[0005] To achieve the above object, the present application proposes an audio and video encoding method based on adaptive filtering, and the audio and video encoding method based on adaptive filtering includes: The audio and video files are filtered based on the improved adaptive filtering algorithm and the coding level to obtain audio and video filtering files; Segmenting the audio and video filter file according to the encoding level to obtain an audio and video segmentation file; Performing cluster analysis on the audio and video segmented files to obtain a plurality of segmented file sets, and respectively determining core segmented files in the segmented file sets, and file transformation relationships between other segmented files in the segmented file sets and the core segmented files; Audio and video encoding is performed based on the core segmented files and the file transformation relationship to obtain an encoded file.

[0006] In one embodiment, the step of filtering the audio and video files based on the improved adaptive filtering algorithm and the coding level to obtain the audio and video filtering files includes: Convert the audio and video files into audio and video file signals, and determine the initial filter vector according to the encoding level; Obtaining a filtering signal according to the audio and video file signal and the initial filtering vector; Calculating an estimation error based on the expected signal and the filtered signal; When the estimated error is greater than a preset error, iterating based on the estimated error, updating the filter weight vector until the estimated error is less than the preset error, and outputting a target filtered signal stream; The target filter signal stream is compiled into an audio and video filter file.

[0007] In one embodiment, the step of obtaining a filtered signal according to the audio and video file signal and the initial filtering vector comprises: Generate an initial population according to the audio and video file signal; Determining individual positions in the audio and video file signals in the initial population; Updating the flight speed loudness and pulse emission rate of each individual in the initial population; Generate a uniform random number, and when the uniform random number is greater than the pulse emission rate, perturb the individual position based on the initial filter vector, and calculate the fitness of the individuals in the initial population; Determine the best individual in the initial population based on the fitness, and determine the best individual position corresponding to the best individual; The audio file signal corresponding to the optimal individual position is determined as a filtering reference signal, and filtering is performed according to the audio and video file signal and the filtering reference signal to obtain a filtered signal.

[0008] In one embodiment, the step of segmenting the audio and video filter file according to the encoding level to obtain the audio and video segmentation file includes: Determine a segmentation level according to the encoding level, and determine a segmentation ratio according to the segmentation level; Analyzing the audio and video filter file, and when the audio and video filter file is a video filter file, separating the image and audio of the audio and video filter file to obtain an audio file and an image file; According to the segmentation ratio, the image file and the audio file are segmented to obtain an image segmentation file and an audio segmentation file respectively; When the audio and video filter file is an audio filter file, the audio filter file is segmented according to the segmentation ratio to obtain an audio filter file.

[0009] In one embodiment, the steps of performing cluster analysis on the audio and video segmented files to obtain a plurality of segmented file sets, and respectively determining a core segmented file in the segmented file set, and a file transformation relationship between other segmented files in the segmented file set and the core segmented file include: Performing cluster analysis on the audio and video segmentation files to obtain multiple cluster sets; Determine the cluster center and cluster point in the cluster set respectively, determine the audio and video segmentation file corresponding to the cluster center as the core audio and video segmentation file, and determine the cluster point as other audio and video segmentation files; Determine an offset distance between the other audio and video segmentation files and the core audio and video segmentation file, and obtain a change vector according to the offset distance; The file transformation relationship between the other audio and video segmentation files and the core audio and video segmentation file is obtained according to the change vector.

[0010] In one embodiment, the step of performing audio and video encoding based on the core segmented files and the file transformation relationship to obtain the encoded file includes: Generate a material library based on the core segmentation file; Generate an index file using the file transformation relationship and the material library; Audio and video encoding is performed based on the index file to obtain an encoded file.

[0011] In one embodiment, the step of performing audio and video encoding based on the index file to obtain an encoded file includes: When the audio and video segmentation file is an audio segmentation file, generating a precompiled file according to the audio timestamp information and the index file, and performing audio and video encoding on the precompiled file to obtain an encoded file; When the audio and video segmentation file is a video segmentation file, determining the time correspondence between the image and the audio in the video to obtain a time matching file; respectively generating an audio precompiled file from the audio timestamp information and the index file, and generating an image precompiled file from the image timestamp information and the index file; The time matching file, the audio precompiled file and the image precompiled file are subjected to audio and video encoding to obtain an encoded file.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes an audio and video encoding device based on adaptive filtering, and the audio and video encoding device based on adaptive filtering includes: A file filtering module, used for filtering audio and video files based on an improved adaptive filtering algorithm and coding level to obtain audio and video filtering files; A file segmentation module, used for segmenting the audio and video filter file according to the encoding level to obtain audio and video segmentation files; A file indexing module is used to perform cluster analysis on the audio and video segmented files to obtain multiple segmented file sets, and respectively determine the core segmented files in the segmented file sets, and the file transformation relationship between other segmented files in the segmented file set and the core segmented files; The file encoding module is used to perform audio and video encoding based on the core segmented files and the file transformation relationship to obtain an encoded file.

[0013] In addition, to achieve the above-mentioned objectives, the present application also proposes an audio and video encoding device based on adaptive filtering, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the audio and video encoding method based on adaptive filtering as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the audio and video encoding method based on adaptive filtering as described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the audio and video encoding method based on adaptive filtering as described above are implemented.

[0016] One or more technical solutions proposed in the present application have at least the following technical effects: filtering the audio and video files based on an improved adaptive filtering algorithm and a coding level to obtain audio and video filter files, segmenting the audio and video filter files according to the coding level to obtain audio and video segmentation files, clustering the audio and video segmentation files to obtain multiple segmentation file sets, and respectively determining the core segmentation files in the segmentation file sets, as well as the file transformation relationship between other segmentation files in the segmentation file set and the core segmentation files, performing audio and video encoding based on the core segmentation files and the file transformation relationship to obtain encoded files, which can significantly reduce redundant files in audio and video files and reduce the size of audio and video files. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of the first embodiment of the audio and video encoding method based on adaptive filtering provided by the present application; Figure 2 A schematic diagram of an adaptive filter provided in accordance with an embodiment of an audio and video encoding method based on adaptive filtering of the present application; Figure 3 A structural diagram of an FIR filter provided in an embodiment of an audio and video encoding method based on adaptive filtering of the present application; Figure 4 A schematic diagram of a bat algorithm flow chart provided for an embodiment of an audio and video encoding method based on adaptive filtering of the present application; Figure 5 A schematic diagram of clustering results provided in an embodiment of an audio and video encoding method based on adaptive filtering of the present application; Figure 6 This is a schematic diagram of the module structure of an audio and video encoding device based on adaptive filtering according to an embodiment of the present application; Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the audio and video encoding method based on adaptive filtering in the embodiment of the present application.

[0020] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0022] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of the embodiment of the present application is: filtering the audio and video files based on the improved adaptive filtering algorithm and the coding level to obtain the audio and video filtering files; segmenting the audio and video filtering files according to the coding level to obtain the audio and video segmentation files; clustering the audio and video segmentation files to obtain multiple segmentation file sets, and respectively determining the core segmentation files in the segmentation file sets, as well as the file transformation relationship between other segmentation files in the segmentation file set and the core segmentation files; performing audio and video encoding based on the core segmentation files and the file transformation relationship to obtain the encoded files.

[0024] In this embodiment, for ease of description, the following description is made by taking the identification of an audio and video encoding device based on adaptive filtering as the execution subject.

[0025] Since audio and video files in the prior art are usually large in size, a lot of bandwidth and time are required for data transmission. In order to reduce the file size of audio and video files, high-compression encoding methods such as H.264 / AVC and H.265 / HEVC are usually used. However, the data after encoding and compression is still relatively large, which still puts a heavy burden on data transmission and storage, and is not conducive to data transmission and data storage.

[0026] The present application provides a solution that can segment audio and video files, segment large audio and video files, obtain core files in the audio and video files, and store the core files as a material library, and generate index files for encoding based on the transformation relationship between the materials and materials constituted by the core files, which can greatly reduce the size of audio and video files and facilitate storage and transmission.

[0027] It can be seen from the above embodiments that the present application obtains an audio and video filter file by filtering the audio and video file based on an improved adaptive filtering algorithm and a coding level, and then segmenting the audio and video filter file according to the coding level to obtain an audio and video segmentation file. The audio and video segmentation file is clustered and analyzed to obtain a plurality of segmentation file sets, and the core segmentation file in the segmentation file set and the file transformation relationship between other segmentation files in the segmentation file set and the core segmentation file are respectively determined. The audio and video encoding is performed based on the core segmentation file and the file transformation relationship to obtain the encoded file, which can significantly reduce the redundant files in the audio and video files and reduce the size of the audio and video files.

[0028] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an audio and video encoding device based on adaptive filtering, etc. The following takes an audio and video encoding device based on adaptive filtering as an example to illustrate this embodiment and the following embodiments.

[0029] Based on this, the embodiment of the present application provides an audio and video encoding method based on adaptive filtering, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the audio and video encoding method based on adaptive filtering of the present application.

[0030] In this embodiment, the audio and video encoding method based on adaptive filtering includes steps S10 to S40: Step S10, filtering the audio and video files based on the improved adaptive filtering algorithm and the coding level to obtain audio and video filtering files.

[0031] It should be noted that audio and video files refer to audio files and video files, which are a general term for these two types of files. Video files include audio files and image files. The improved adaptive filtering algorithm is based on the LMS filtering algorithm (Least Mean Square, adaptive filtering algorithm) combined with the bat algorithm to improve the LMS algorithm. The encoding level can be related to the expected compression degree of the audio and video. The higher the encoding level, the smaller the audio and video file size will be compressed.

[0032] In a specific implementation, the audio and video file is converted into an audio and video file signal, and an initial filter vector is determined according to the encoding level, a filter signal is obtained according to the audio and video file signal and the initial filter vector, an estimated error is calculated according to the expected signal and the filter signal, and when the estimated error is greater than a preset error, iteration is performed based on the estimated error to update the filter weight vector until the estimated error is less than the preset error, and a target filter signal stream is output, and the target filter signal stream is compiled into an audio and video filter file. Specifically, the adaptive filtering algorithm and the digital filter can constitute the adaptive filter involved in this embodiment, wherein the digital filter includes a finite impulse filter (FiniteImpulse Response, FIR) and an infinite impulse filter (InfiniteImpulse Response, IIR), and the schematic diagram of the adaptive filter is shown as follows Figure 2 As shown. Among them, is the input, i.e. the audio and video file signal, is the output of the filter, is the noise signal, is the expected signal, is the error signal. When processing audio and video signals, the adaptive filter can include two stages, namely, the filtering stage and the adaptive stage.

[0033] Filtering stage: The filter outputs the corresponding data based on the input signal, and calculates the estimated error by comparing the expected signal and the output signal.

[0034] Adaptive stage: Through multiple iterations, the filter vector is updated to make the output signal Infinitely close to the expected signal , which makes the error signal Infinitely approaching 0, so as to achieve the best filtering effect. In the adaptive filter, considering that the IIR filter is easily affected by the quantization error, the FIR filter is used for adaptive filtering. The structure diagram of the FIR filter is as follows: Figure 3 The filtering process can be expressed as follows: The input of the time-adaptive filter is , filter vector , where L is the order of the filter, which is directly related to the coding level, for The filter vector at time instant, the output of the filter for:

[0035] The expected signal is:

[0036] The error signal is:

[0037] When filtering, the LMS algorithm is used for filtering. According to the minimum error criterion of the LMS algorithm, the cost function is determined as:

[0038] in, represents the expected value. According to the relationship between the error signal, the expected signal and the output signal, the cost function can be rewritten as:

[0039] for and The cross-correlation matrix of for The autocorrelation matrix is:

[0040]

[0041] Then, the cost function can be expressed as:

[0042] The filter vector Taking the derivative, we get:

[0043] when When is a non-singular matrix, let ,but: .

[0044] When iterating, you can use the formula:

[0045] in, is the step size factor of adaptive filtering, is the gradient operator, and then the updated filter vector can be expressed as:

[0046] The error at the moment is: .

[0047] Based on the adaptive filtering process, the audio and video files are filtered to obtain audio and video filtering files.

[0048] In a feasible implementation manner, the step of obtaining a filtered signal according to the audio and video file signal and the initial filtering vector includes: Generate an initial population according to the audio and video file signal; Determining individual positions in the audio and video file signals in the initial population; Updating the flight speed loudness and pulse emission rate of each individual in the initial population; Generate a uniform random number, and when the uniform random number is greater than the pulse emission rate, perturb the individual position based on the initial filter vector, and calculate the fitness of the individuals in the initial population; Determine the best individual in the initial population based on the fitness, and determine the best individual position corresponding to the best individual; The audio file signal corresponding to the optimal individual position is determined as a filtering reference signal, and filtering is performed according to the audio and video file signal and the filtering reference signal to obtain a filtered signal.

[0049] In a specific implementation, when a filter signal is obtained according to the audio and video file signal and the initial filter vector, the optimal filter signal can be determined according to the BA algorithm. In the BA algorithm, the bat uses ultrasound to locate when obtaining food location information. In the specific process, each bat uses ultrasound to obtain the distance information between the food and the obstacle, and the bat's position is used as the Indicates that bats and fixed frequency During random flight, the ultrasonic sound emitted by bats changes from the maximum Reduce to minimum , there is a coding level to determine the search range of the bat, and the number of dimensions is dimension, in each iteration, the speed of each bat and location The update rules are:

[0050]

[0051]

[0052] Among them, the frequency of ultrasonic waves emitted by bats is , the frequency adjustment range is , is a random vector that follows a uniform distribution and has a range of , the global optimal position is The optimal position is obtained by comparing and calculating the distance information between each bat and the food.

[0053] Bats search for food within a certain range. Once the target is determined, a local new solution is generated by each bat's random walk. The process of generating a local new solution is:

[0054] in, is a random number in the range of [-1,1], is the average loudness of the entire bat population in this time step.

[0055] When bats are foraging, they constantly adjust the frequency and loudness of the ultrasonic waves to increase their success rate in capturing prey. When prey is found, the loudness is lowered, and the number of sound wave pulses increases, at which point the loudness and emission rate of the sound waves need to be updated. The loudness of the ultrasonic waves emitted by a bat and emissivity to update.

[0056]

[0057]

[0058] in, For the The initial pulse firing rate of the bats, is the number of times the loudness is attenuated, ranging from , is a constant whose range is In this embodiment, and Set to The emission rate and loudness of the ultrasonic pulses are updated when the bat's position is optimized, indicating that the bat is moving toward the best hunting position. Figure 4As shown. First, the audio and video file signals are simulated into individual bats to obtain the initialized bat population. According to the initial position of the bats, a set of initial solutions is obtained, and the current optimal solution is calculated. The bat's flight speed, position, loudness and pulse emission rate are updated, and a uniformly distributed random number rand is generated. It is judged whether rand is greater than the pulse emission frequency. If not, the bat is in a stationary state. If so, random perturbations are performed to randomly form a bat individual near the optimal solution, and the fitness values ​​of all bats are calculated to obtain the optimal solution. It is also judged whether the termination condition of the algorithm is met. If so, the optimal position is output, and the filtered audio and video filter file signal is obtained according to the optimal position. If not, the bat's flight speed, position, loudness and pulse emission rate are updated again, and the process is continuously repeated.

[0059] Step S20, segmenting the audio and video filter file according to the encoding level to obtain an audio and video segmentation file.

[0060] It should be noted that the audio and video segmentation file refers to the fragmented file obtained after the audio and video file is segmented to a certain extent. That is to say, the current audio and video segmentation file is one of the components of the original audio and video filter file. The encoding level determines the degree of segmentation. The higher the encoding level, the higher the degree of segmentation. For the same audio and video file, more audio and video segmentation files will be obtained after segmentation.

[0061] In a specific implementation, when the audio and video filter files are segmented according to the encoding level, the segmentation level can be determined according to the encoding level, and the segmentation ratio can be determined according to the segmentation level, that is, the number of parts into which the audio and video filter files are segmented can be determined according to the encoding level. When segmenting, a suitable segmentation method can be selected according to the actual situation. One method is to evenly segment the audio and video files into The first is a number of copies, n is the current coding level, and the second is to determine the number of files after uniform segmentation in the form of a table lookup according to the generated coding segmentation table. When segmenting the audio and video filter file, the audio and video filter file can be analyzed. The audio and video filter file includes a video filter file and an audio filter file, wherein the video filter file includes an image filter file and an audio filter file. When the audio and video filter file is a video filter file, the image and audio of the audio and video filter file are separated to obtain an audio file and an image file. According to the segmentation ratio, the image file and the audio file are segmented respectively to obtain an image segmentation file and an audio segmentation file. When the audio and video filter file is an audio filter file, the audio filter file is segmented according to the segmentation ratio to obtain an audio filter file.

[0062] Step S30, cluster analysis is performed on the audio and video segmented files to obtain multiple segmented file sets, and core segmented files in the segmented file sets and file transformation relationships between other segmented files in the segmented file set and the core segmented files are determined respectively.

[0063] It should be noted that the segmented file set refers to a file set obtained by clustering the segmented files according to their similarity, and the segmented files in the same segmented file set have a high degree of similarity. In addition, in the segmented file set, the core segmented files are determined according to the repetition frequency of each segmented file, and the other segmented files are collectively referred to as other segmented files. The file transformation relationship between the other segmented files and the core segmented files refers to the change mode between the other segmented files and the core segmented files as a file transformation relationship, that is, describing the change process from the core segmented files to the other segmented files.

[0064] In a specific implementation, firstly, the audio and video segmentation files can be preprocessed, and the audio and video files can be converted into files of a format suitable for analysis. According to the audio and video segmentation files, the audio and image files can be unified into the same format, and the preprocessed image files and audio files can be feature extracted to obtain audio features and image features. Then, clustering operations are performed based on audio features and image features, and cluster analysis is performed based on audio features and image features to obtain multiple segmentation file sets, and the cluster center of each file set is determined, and the file corresponding to the cluster center is determined as the core segmentation file. When determining the core segmentation file, the core segmentation file can be determined according to the frequency of repetition, and the file with the highest frequency of repetition is determined as the core segmentation file, and other segmentation files are determined as other segmentation files. In addition, the transformation relationship between each type of segmentation file and the core segmentation file of the current file set is determined, that is, it is determined what kind of transformation is required to achieve the change mode from the core segmentation file, and each change mode is determined as a file transformation relationship.

[0065] In a feasible implementation manner, the steps of performing cluster analysis on the audio and video segmented files to obtain multiple segmented file sets, and respectively determining the core segmented files in the segmented file sets, and the file transformation relationship between other segmented files in the segmented file sets and the core segmented files include: Performing cluster analysis on the audio and video segmentation files to obtain multiple cluster sets; Determine the cluster center and cluster point in the cluster set respectively, determine the audio and video segmentation file corresponding to the cluster center as the core audio and video segmentation file, and determine the cluster point as other audio and video segmentation files; Determine an offset distance between the other audio and video segmentation files and the core audio and video segmentation file, and obtain a change vector according to the offset distance; The file transformation relationship between the other audio and video segmentation files and the core audio and video segmentation file is obtained according to the change vector.

[0066] In a specific implementation, before clustering analysis is performed on the audio and video segmentation files to obtain multiple clustering sets, the audio and video segmentation files can be tested for repeatability, the audio and video segmentation files are traversed one by one, and each audio and video segmentation file is marked. A clustering file is generated with the audio and video segmentation file that appears for the first time, and in the subsequent traversal process, when a determined clustering file appears, the number of its corresponding clustering files is increased by 1. Therefore, in the process of clustering file processing, a large amount of redundant data can be reduced, and the time consumption and resource consumption of clustering can be reduced. Before clustering analysis is performed on the audio and video segmentation files, a clustering file set is generated. ,in is the cluster file of the i-th file. After clustering analysis is performed on the cluster files, multiple cluster sets are obtained. When clustering, the K-means clustering algorithm can be used to cluster the cluster file sets. Divide into k categories ,in First, cluster the files together and repeat the frequency to determine the cluster center, so that the other segmentation files can converge and make the error local minimum. Figure 5 , Figure 5 The figure is a schematic diagram of the clustering result. In each cluster set, the similarity between the segmented files is high. Taking any cluster set as an example, in a cluster set, a transformation network is generated with the core segmented file as the cluster center, and the position of each other segmented file in the transformation network is determined. In the transformation network, each dimension represents a file attribute and represents the position of other segmented files in the transformation network. and the location of the core split files ,Sure and The previous offset distance is used to obtain the file transformation relationship between the other audio and video segmentation files and the core audio and video segmentation files according to the change vector. .

[0067] Step S40: performing audio and video encoding based on the core segmented files and the file transformation relationship to obtain an encoded file.

[0068] It should be noted that the encoded file refers to a file that is obtained after encoding the audio and video file and is used to describe the content of the audio and video file. The file size of this file will be greatly reduced compared to the original audio and video file.

[0069] In a specific implementation, the core segmented files are extracted separately to form a material library, and combined with the material library according to the file transformation relationship to generate an index file. The index file includes the segmentation position in the original position of the audio and video file, the generation method of the segmented file, including which core segmented file is used as the benchmark, which file transformation relationship is used for file transformation, and the core segmented file and the file transformation relationship are used to form an index value of a segmented file block, and multiple index values ​​are combined in the order of file composition to obtain a coded file. If the audio and video encoding is video encoding, timestamps can be added to the audio encoding and image encoding to establish a time correspondence between the audio encoding and the video encoding.

[0070] In a feasible implementation manner, the step of performing audio and video encoding based on the index file to obtain an encoded file includes: When the audio and video segmentation file is an audio segmentation file, generating a precompiled file according to the audio timestamp information and the index file, and performing audio and video encoding on the precompiled file to obtain an encoded file; When the audio and video segmentation file is a video segmentation file, determining the time correspondence between the image and the audio in the video to obtain a time matching file; respectively generating an audio precompiled file from the audio timestamp information and the index file, and generating an image precompiled file from the image timestamp information and the index file; The time matching file, the audio precompiled file and the image precompiled file are subjected to audio and video encoding to obtain an encoded file.

[0071] In a specific implementation, when the audio and video segmentation file is an audio segmentation file, a precompiled file is generated according to the audio timestamp information and the index file, and the precompiled file is encoded with audio and video to obtain an encoded file; when the audio and video segmentation file is a video segmentation file, the time correspondence between the image and the audio in the video is determined to obtain a time matching file; the audio timestamp information and the index file are respectively used to generate an audio precompiled file, and the image timestamp information and the index file are used to generate an image precompiled file; the time matching file, the audio precompiled file and the image precompiled file are encoded with audio and video to obtain an encoded file, and after obtaining the encoded file, It can be transmitted and stored. For the encoded files, there is a material library with non-repetitive content. The content of the audio and video files is greatly reduced, and the redundant parts are described by referring to the materials and file transformation relationships in the material library in an indexed manner to describe the entire audio and video file. Therefore, a large number of audio and video files are converted into the form of material plus text. Therefore, whether it is storage or transmission, the transmission and storage purposes can be achieved at a lower cost. As for after the transmission, the receiving device can restore the audio and video files according to the material library, file transformation relationships and index files. This method can also maintain the original quality of the original audio and video and achieve lossless transmission.

[0072] The present embodiment provides an audio and video encoding method based on adaptive filtering, which obtains an audio and video filter file by filtering the audio and video file based on an improved adaptive filtering algorithm and a coding level, and then segments the audio and video filter file according to the coding level to obtain an audio and video segmentation file, and then clusters the audio and video segmentation file to obtain a plurality of segmentation file sets, and then respectively determines a core segmentation file in the segmentation file set, and a file transformation relationship between other segmentation files in the segmentation file set and the core segmentation file, and then performs audio and video encoding based on the core segmentation file and the file transformation relationship to obtain an encoded file, which can significantly reduce redundant files in audio and video files and reduce the size of audio and video files.

[0073] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the audio and video encoding method based on adaptive filtering of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0074] The present application also provides an audio and video encoding device based on adaptive filtering, please refer to Figure 6 , the audio and video encoding device based on adaptive filtering includes: The file filtering module 10 is used to filter the audio and video files based on the improved adaptive filtering algorithm and the coding level to obtain the audio and video filtering files; A file segmentation module 20, used to segment the audio and video filter file according to the encoding level to obtain audio and video segmentation files; A file indexing module 30 is used to perform cluster analysis on the audio and video segmented files to obtain a plurality of segmented file sets, and to respectively determine a core segmented file in the segmented file set, and a file conversion relationship between other segmented files in the segmented file set and the core segmented file; The file encoding module 40 is used to perform audio and video encoding based on the core segmented files and the file transformation relationship to obtain an encoded file.

[0075] In one embodiment, the file filtering module 10 is also used to convert the audio and video file into an audio and video file signal, and determine an initial filter vector according to the encoding level; obtain a filter signal according to the audio and video file signal and the initial filter vector; calculate an estimated error according to an expected signal and the filtered signal; when the estimated error is greater than a preset error, iterate based on the estimated error, update the filter weight vector until the estimated error is less than the preset error, and output a target filter signal stream; compile the target filter signal stream into an audio and video filter file.

[0076] In one embodiment, the file filtering module 10 is also used to generate an initial population based on the audio and video file signal; determine the individual position in the audio and video file signal in the initial population; update the flight speed loudness and pulse emission rate of each individual in the initial population; generate a uniform random number, and when the uniform random number is greater than the pulse emission rate, perturb the individual position based on the initial filtering vector, and calculate the fitness of the individuals in the initial population; determine the optimal individual in the initial population based on the fitness, and determine the optimal individual position corresponding to the optimal individual; determine the audio file signal corresponding to the optimal individual position as a filtering reference signal, and filter according to the audio and video file signal and the filtering reference signal to obtain a filtered signal.

[0077] In one embodiment, the file segmentation module 20 is further used to determine the segmentation level according to the encoding level, and determine the segmentation ratio according to the segmentation level; analyze the audio and video filter file, and when the audio and video filter file is a video filter file, separate the image and audio of the audio and video filter file to obtain an audio file and an image file; according to the segmentation ratio, segment the image file and the audio file respectively to obtain an image segmentation file and an audio segmentation file; when the audio and video filter file is an audio filter file, segment the audio filter file according to the segmentation ratio to obtain an audio filter file.

[0078] In one embodiment, the file index module 30 is also used to perform cluster analysis on the audio and video segmentation files to obtain multiple cluster sets; respectively determine the cluster center and cluster points in the cluster set, determine the audio and video segmentation files corresponding to the cluster center as the core audio and video segmentation files, and determine the cluster points as other audio and video segmentation files; determine the offset distance between the other audio and video segmentation files and the core audio and video segmentation files, and obtain a change vector based on the offset distance; and obtain a file transformation relationship between the other audio and video segmentation files and the core audio and video segmentation files based on the change vector.

[0079] In one embodiment, the file encoding module 40 is further used to generate a material library based on the core segmented file; generate an index file by combining the file transformation relationship with the material library; and perform audio and video encoding based on the index file to obtain an encoded file.

[0080] In one embodiment, the file encoding module 40 is further used to generate a precompiled file according to the audio timestamp information and the index file when the audio and video segmentation file is an audio segmentation file, and perform audio and video encoding on the precompiled file to obtain an encoded file; when the audio and video segmentation file is a video segmentation file, determine the time correspondence between the image and the audio in the video to obtain a time matching file; generate an audio precompiled file using the audio timestamp information and the index file, and generate an image precompiled file using the image timestamp information and the index file; perform audio and video encoding on the time matching file, the audio precompiled file, and the image precompiled file to obtain an encoded file.

[0081] The audio and video encoding device based on adaptive filtering provided by the present application adopts the audio and video encoding method based on adaptive filtering in the above embodiment, which can solve the technical problem of large file size after audio and video files are encoded in the prior art. Compared with the prior art, the beneficial effects of the audio and video encoding device based on adaptive filtering provided by the present application are the same as the beneficial effects of the audio and video encoding method based on adaptive filtering provided by the above embodiment, and other technical features in the audio and video encoding device based on adaptive filtering are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0082] The present application provides an audio and video encoding device based on adaptive filtering, and the audio and video encoding device based on adaptive filtering includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the audio and video encoding method based on adaptive filtering in the above-mentioned embodiment one.

[0083] Reference below Figure 7 , which shows a schematic diagram of the structure of an audio and video encoding device based on adaptive filtering suitable for implementing the embodiment of the present application. The audio and video encoding device based on adaptive filtering in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The audio and video encoding device based on adaptive filtering shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0084] like Figure 7 As shown, the audio and video encoding device based on adaptive filtering may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the audio and video encoding device based on adaptive filtering are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the audio and video encoding device based on adaptive filtering to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an audio and video encoding device based on adaptive filtering with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0085] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0086] The audio and video encoding device based on adaptive filtering provided by the present application adopts the audio and video encoding method based on adaptive filtering in the above embodiment, which can solve the technical problem of large file size after audio and video files are encoded in the prior art. Compared with the prior art, the beneficial effects of the audio and video encoding device based on adaptive filtering provided by the present application are the same as the beneficial effects of the audio and video encoding method based on adaptive filtering provided by the above embodiment, and other technical features in the audio and video encoding device based on adaptive filtering are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0087] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0088] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0089] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the audio and video encoding method based on adaptive filtering in the above-mentioned embodiment.

[0090] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0091] The computer-readable storage medium may be included in the audio and video encoding device based on adaptive filtering; or may exist independently without being assembled into the audio and video encoding device based on adaptive filtering.

[0092] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the audio and video encoding device based on adaptive filtering, the audio and video encoding device based on adaptive filtering: The audio and video files are filtered based on the improved adaptive filtering algorithm and the coding level to obtain audio and video filtering files; Segmenting the audio and video filter file according to the encoding level to obtain an audio and video segmentation file; Performing cluster analysis on the audio and video segmented files to obtain a plurality of segmented file sets, and respectively determining core segmented files in the segmented file sets, and file transformation relationships between other segmented files in the segmented file sets and the core segmented files; Audio and video encoding is performed based on the core segmented file and the file transformation relationship to obtain an encoded file. The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming language includes an object-oriented programming language-such as Java, Smalltalk, C++, and also includes a conventional procedural programming language-such as "C" language or similar programming language. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network-including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0093] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0094] The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0095] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned audio and video encoding method based on adaptive filtering, and can solve the technical problem of large file size after audio and video file encoding in the prior art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the audio and video encoding method based on adaptive filtering provided by the above-mentioned embodiment, and will not be described in detail here.

[0096] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned audio and video encoding method based on adaptive filtering when executed by a processor.

[0097] The computer program product provided by the present application can solve the technical problem of large file size after audio and video file encoding in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the audio and video encoding method based on adaptive filtering provided by the above embodiment, which will not be repeated here.

[0098] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An audio and video encoding method based on adaptive filtering, characterized in that: The audio and video encoding method based on adaptive filtering includes: The audio and video files are filtered based on the improved adaptive filtering algorithm and the coding level to obtain audio and video filtering files; Segmenting the audio and video filter file according to the encoding level to obtain an audio and video segmentation file; Performing cluster analysis on the audio and video segmented files to obtain a plurality of segmented file sets, and respectively determining core segmented files in the segmented file sets, and file transformation relationships between other segmented files in the segmented file sets and the core segmented files; Audio and video encoding is performed based on the core segmented files and the file transformation relationship to obtain an encoded file.

2. The method according to claim 1, characterized in that The step of filtering the audio and video files based on the improved adaptive filtering algorithm and the coding level to obtain the audio and video filtering files comprises: Convert the audio and video files into audio and video file signals, and determine the initial filter vector according to the encoding level; Obtaining a filtering signal according to the audio and video file signal and the initial filtering vector; Calculating an estimation error based on the expected signal and the filtered signal; When the estimated error is greater than a preset error, iterating based on the estimated error, updating the filter weight vector until the estimated error is less than the preset error, and outputting a target filtered signal stream; The target filter signal stream is compiled into an audio and video filter file.

3. The method according to claim 2, characterized in that The step of obtaining a filter signal according to the audio and video file signal and the initial filter vector comprises: Generate an initial population according to the audio and video file signal; Determining individual positions in the audio and video file signals in the initial population; Updating the flight speed loudness and pulse emission rate of each individual in the initial population; Generate a uniform random number, and when the uniform random number is greater than the pulse emission rate, perturb the individual position based on the initial filter vector, and calculate the fitness of the individuals in the initial population; Determine the best individual in the initial population based on the fitness, and determine the best individual position corresponding to the best individual; The audio file signal corresponding to the optimal individual position is determined as a filtering reference signal, and filtering is performed according to the audio and video file signal and the filtering reference signal to obtain a filtered signal.

4. The method according to claim 1, characterized in that The step of segmenting the audio and video filter file according to the encoding level to obtain the audio and video segmentation file comprises: Determine a segmentation level according to the encoding level, and determine a segmentation ratio according to the segmentation level; Analyzing the audio and video filter file, and when the audio and video filter file is a video filter file, separating the image and audio of the audio and video filter file to obtain an audio file and an image file; According to the segmentation ratio, the image file and the audio file are segmented to obtain an image segmentation file and an audio segmentation file respectively; When the audio and video filter file is an audio filter file, the audio filter file is segmented according to the segmentation ratio to obtain an audio filter file.

5. The method according to claim 1, characterized in that The steps of performing cluster analysis on the audio and video segmented files to obtain a plurality of segmented file sets, and respectively determining the core segmented files in the segmented file sets, and the file transformation relationship between other segmented files in the segmented file sets and the core segmented files include: Performing cluster analysis on the audio and video segmentation files to obtain multiple cluster sets; Determine the cluster center and cluster point in the cluster set respectively, determine the audio and video segmentation file corresponding to the cluster center as the core audio and video segmentation file, and determine the cluster point as other audio and video segmentation files; Determine an offset distance between the other audio and video segmentation files and the core audio and video segmentation file, and obtain a change vector according to the offset distance; The file transformation relationship between the other audio and video segmentation files and the core audio and video segmentation file is obtained according to the change vector.

6. The method according to claim 1, characterized in that The step of performing audio and video encoding based on the core segmented file and the file transformation relationship to obtain the encoded file comprises: Generate a material library based on the core segmentation file; Generate an index file using the file transformation relationship and the material library; Audio and video encoding is performed based on the index file to obtain an encoded file.

7. The method according to claim 6, characterized in that The step of performing audio and video encoding based on the index file to obtain an encoded file comprises: When the audio and video segmentation file is an audio segmentation file, generating a precompiled file according to the audio timestamp information and the index file, and performing audio and video encoding on the precompiled file to obtain an encoded file; When the audio and video segmentation file is a video segmentation file, determining the time correspondence between the image and the audio in the video to obtain a time matching file; respectively generating an audio precompiled file from the audio timestamp information and the index file, and generating an image precompiled file from the image timestamp information and the index file; The time matching file, the audio precompiled file and the image precompiled file are subjected to audio and video encoding to obtain an encoded file.

8. An audio and video encoding device based on adaptive filtering, characterized in that: The device comprises: A file filtering module, used for filtering audio and video files based on an improved adaptive filtering algorithm and coding level to obtain audio and video filtering files; A file segmentation module, used for segmenting the audio and video filter file according to the encoding level to obtain audio and video segmentation files; A file indexing module is used to perform cluster analysis on the audio and video segmented files to obtain multiple segmented file sets, and respectively determine the core segmented files in the segmented file sets, and the file transformation relationship between other segmented files in the segmented file set and the core segmented files; The file encoding module is used to perform audio and video encoding based on the core segmented files and the file transformation relationship to obtain an encoded file.

9. An audio and video encoding device based on adaptive filtering, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio and video encoding method based on adaptive filtering as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio and video encoding method based on adaptive filtering as described in any one of claims 1 to 7 are implemented.

Images