Image encoding optimization method and optimization system based on deep learning

Abstract

The application provides a deep learning-based image coding optimization method and an optimization system thereof, and belongs to the technical field of image processing. Through a highly adaptive parameter adjustment model, the parameters of the encoder can be automatically adjusted according to the coding habits of different users, the processing priorities of image features and real-time network states, the dual optimization of coding efficiency and quality is realized, and through the pre-processing step to extract image key information and the accurate deployment of the parameter adaptive adjustment model on the coding parameters, not only the high-quality image coding effect is ensured, but also the stable performance in the complex network environment is maintained. In addition, the continuous learning and optimization mechanism can continuously adapt to new data and environment, provide the user with a more smooth and personalized image coding experience, and significantly improve the overall coding efficiency and user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image coding optimization method based on deep learning and an optimization system thereof. BACKGROUND

[0002] Image coding, also known as image compression, is a technical process that reduces the amount of image data while maintaining the quality of the image as much as possible. It is crucial for the storage and transmission of digital images, especially in the fields of network communication, video streaming, cloud computing services, etc. The basic principle of image coding is to utilize the characteristics of the human visual system to remove redundant information and invisible details in the image, and to represent the image data in an efficient manner.

[0003] Traditional image coding methods often struggle to find an ideal balance between high compression rate and high image quality, especially in resource-constrained or unstable network conditions. Fixed coding parameter settings often result in either compromised image quality or low transmission efficiency. Furthermore, fluctuations in network environment such as bandwidth limitations and delay changes directly affect the quality and speed of image transmission. Traditional coding systems lack effective mechanisms to respond to these changes in real-time, making it difficult to maintain good transmission performance under various network conditions. Moreover, existing systems often lack the ability to learn and optimize themselves, making it difficult to continuously improve coding performance to adapt to new image content and user habits.

[0004] Therefore, it is necessary to provide an image coding optimization method based on deep learning and an optimization system thereof to solve the above technical problems. SUMMARY

[0005] To solve the above technical problems, the present application provides an image coding optimization method based on deep learning and an optimization system thereof, which achieves dual optimization of coding efficiency and quality, not only guarantees high-quality image coding effect, but also maintains stable performance in complex network environment and improves overall coding efficiency and user experience.

[0006] The image coding optimization method based on deep learning provided by the present application specifically includes the following steps:

[0007] The optimization method specifically includes the following steps:

[0008] S1: Collect historical image coding processing data, user historical data, and network state monitoring data and analyze them, and train and construct a parameter self-adaptive adjustment model according to the analysis results;

[0009] S2: Collect image data and pre-process the image data to extract and obtain image processing data, wherein the pre-processing specifically includes color space conversion, image denoising, size adjustment, and block processing;

[0010] S3: input the image processed data into the parameter adaptive adjustment model, obtain the parameter adjustment result, and determine the balance point of controlling the adjustment quality and compression ratio according to the parameter adjustment result, automatically adjust the parameters of the encoder, and obtain the parameter adjusted encoder;

[0011] S4: encode the image processed data through the obtained parameter adjusted encoder, continuously record and evaluate the encoding process, feed back according to the evaluation result, and continuously optimize through deep learning.

[0012] Preferably, step S1 specifically comprises the following steps:

[0013] The step S1 specifically comprises the following steps:

[0014] S101: The collected historical image encoding processing data includes original images, encoding settings and final encoding results; the collected user historical data includes the user's preferred and selected image quality level and compression ratio; and the collected network state monitoring data includes monitored network bandwidth and delay data, and is constructed into a data set;

[0015] S102: analyze the encoding habits, feature processing priorities in images and the association between network and encoding in the data set, and obtain an analysis result;

[0016] S103: select a deep learning model architecture, train the deep learning model architecture through the analysis result, and construct a parameter adaptive adjustment model.

[0017] Preferably, the step S3 specifically comprises the following steps:

[0018] S301: input the image processed data into the constructed parameter adaptive adjustment model, and the parameter adaptive adjustment model performs forward propagation calculation based on the encoding habits, feature processing priorities and the association between network and encoding, and outputs a parameter adjustment result;

[0019] S302: determine whether the output parameter adjustment result meets the expected target, and obtain a determination result;

[0020] S303: determine the balance point of controlling the adjustment quality and compression ratio based on the determination result, automatically adjust the parameters of the encoder, and obtain the parameter adjusted encoder.

[0021] Preferably, the step S302 specifically comprises the following steps:

[0022] S3021: determine the expected target of the lowest acceptable image quality, the largest allowed file size and the expected loading time;

[0023] S3022: compare the parameter adjustment result output by the parameter adaptive adjustment model with the expected target to obtain a comparison result;

[0024] S3023: analyze the comparison result, specifically, if the comparison result indicates that the parameter adjustment result meets the expected target, the next step is executed, if the comparison result indicates that the parameter adjustment result does not meet the expected target, the reason for not meeting the expected target is analyzed, and the model is retrained according to the reason.

[0025] Preferably, step S4 specifically comprises the following steps:

[0026] S401: input the image processing data into the configured encoder for encoding operation;

[0027] S402: during the encoding operation of the image processing data, the key indicators are continuously recorded, wherein the key indicators include the encoding speed, the compression ratio and the image quality indicators;

[0028] S403: after the encoding of the image processing data is completed, the encoding result is evaluated, and the evaluation result and the recorded key indicators are fed back to the parameter adaptive adjustment model to obtain the feedback result of the quantitative and qualitative information of the encoding quality;

[0029] S404: based on the feedback result, the parameter adaptive adjustment model is continuously trained by deep learning technology to optimize the parameter adaptive adjustment model to accurately predict the optimal encoding parameter combination.

[0030] The application also provides an image encoding optimization system based on deep learning, the optimization system comprising:

[0031] A model construction module is used to collect historical image encoding processing data, user historical data and network state monitoring data and analyze them, and train and construct a parameter adaptive adjustment model according to the analysis result;

[0032] A data processing module is used to collect image data and pre-process the image data to extract and obtain image processing data, wherein the pre-processing specifically includes color space conversion, image denoising, size adjustment and block processing;

[0033] A scheduling module is used to input the image processing data into the parameter adaptive adjustment model to obtain a parameter adjustment result, and determine a balance point of the quality and compression ratio according to the parameter adjustment result, automatically adjust the parameters of the encoder to obtain a parameter-adjusted encoder;

[0034] An encoding and optimization module is configured to encode the processed data of the image by the obtained parameter-adjusted encoder, continuously record and evaluate the encoding process, feed back according to the evaluation result, and continuously optimize by deep learning.

[0035] Preferably, the model construction module comprises:

[0036] The acquisition module is configured to acquire historical image encoding processing data, including original images, encoding settings and final encoding results; acquire user historical data, including user's preferred image quality level and compression ratio; acquire network state monitoring data, including monitored network bandwidth and delay data, and construct the data into a data set.

[0037] The analysis module is configured to analyze encoding habits, feature processing priorities in the image and the correlation between the network and the encoding in the data set, and obtain an analysis result.

[0038] The training module is configured to select a deep learning model architecture, train the deep learning model architecture by the analysis result, and construct a parameter self-adaptive adjustment model.

[0039] Preferably, the scheduling module comprises:

[0040] The calculation module is configured to input the processed data of the image into the constructed parameter self-adaptive adjustment model, perform forward propagation calculation based on the encoding habits, the feature processing priorities and the correlation between the network and the encoding, and output a parameter adjustment result.

[0041] The judgment module is configured to judge whether the output parameter adjustment result meets the expected target, and obtain a judgment result.

[0042] The deployment module is configured to determine a balance point of the quality and the compression ratio based on the judgment result, automatically adjust the parameters of the encoder, and obtain a parameter-adjusted encoder.

[0043] Preferably, the judgment module comprises:

[0044] The selection module is configured to determine the expected target of the lowest acceptable image quality, the largest allowed file size and the expected loading time.

[0045] The comparison module is configured to compare the output parameter adjustment result of the parameter self-adaptive adjustment with the expected target, and obtain a comparison result.

[0046] The parameter evaluation module is configured to analyze the comparison result, specifically, if the comparison result shows that the parameter adjustment result meets the expected target, the next step is executed, and if the comparison result shows that the parameter adjustment result does not meet the expected target, the reason for not meeting the expected target is analyzed, and the model is retrained according to the reason.

[0047] Preferably, the encoding and optimization module comprises:

[0048] The encoding processing module is configured to input the image-processed data into the configured encoder for encoding operation.

[0049] The recording module is configured to record key indicators during the encoding operation of the image-processed data, wherein the key indicators specifically include encoding speed, compression ratio, and image quality indicators.

[0050] The encoding evaluation module is configured to evaluate the encoding result after the encoding of the image-processed data is completed, and feed back the evaluation result and the recorded key indicators to the parameter self-adaptive adjustment model to obtain the feedback result of the quantitative and qualitative information of the encoding quality.

[0051] The feedback optimization module is configured to continuously train the parameter self-adaptive adjustment model based on the feedback result through deep learning technology to optimize the parameter self-adaptive adjustment model to accurately predict the optimal combination of encoding parameters.

[0052] Compared with the related art, the image encoding optimization method and system based on deep learning provided by the present application have the following beneficial effects:

[0053] The highly adaptive parameter adjustment model constructed by the present application can automatically adjust the parameters of the encoder according to the encoding habits of different users, the processing priority of image features, and the real-time network state, realize the dual optimization of encoding efficiency and quality, and through the pre-processing step of extracting image key information and the accurate deployment of the parameter self-adaptive adjustment model, not only ensure the high-quality image encoding effect, but also maintain stable performance in complex network environment. In addition, the continuous learning and optimization mechanism can continuously adapt to new data and environment, provide users with more smooth and personalized image encoding experience, and significantly improve the overall encoding efficiency and user experience. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 The flowchart of the image encoding optimization method based on deep learning provided by the present application is provided.

[0055] Figure 2 The module structure diagram of the image encoding optimization system based on deep learning provided by the present application is provided. DETAILED DESCRIPTION

[0056] The present application will be further described below in conjunction with the drawings and embodiments.

[0057] Embodiment One

[0058] The image encoding optimization method based on deep learning, for reference Figure 1As shown, comprising the following steps:

[0059] S1: Collect historical image encoding processing data, user historical data and network state monitoring data and analyze, and train and build parameter adaptive adjustment model according to the analysis result.

[0060] Specifically, collect historical image encoding processing data, user historical data and network state monitoring data, and analyze its encoding habit, processing priority of features in image and the association between network and encoding, obtain the analysis result, train and build parameter adaptive adjustment model according to the analysis result, understand the user's encoding habit and network condition, can provide the most suitable image quality for different scenes, improve user experience.

[0061] S2: Collect image data, and pre-process the image data, extract and obtain image processing data, wherein the pre-processing specifically includes color space conversion, image denoising, size adjustment and block processing.

[0062] Specifically, according to the application scenario and encoding requirement, select the appropriate color space for conversion, and use the image processing library, such as the function in OpenCV, to convert the original image from RGB color space to the target color space; analyze the noise of the image, determine the type and degree of the noise, use the denoising function in the image processing library to denoise the image, to reduce the noise in the image and improve the image quality; according to the specific content of the image and the application requirement, select the appropriate adjustment method, use the size adjustment function in the image processing library to adjust the size of the image; use the block function in the image processing library to divide the image into multiple small blocks, each small block can be independently processed in the subsequent encoding process, improve the encoding efficiency.

[0063] S3: Input the image processing data to the parameter adaptive adjustment model, obtain the parameter deployment result, and determine the balance point of controlling the adjustment quality and compression ratio according to the parameter deployment result, automatically adjust the parameters of the encoder, obtain the parameter adjusted encoder.

[0064] Specifically, input the pre-processed image data into the parameter adaptive adjustment model, the parameter adaptive adjustment model automatically adjusts the parameters of the encoder according to the input data, including quantization step, filter setting, etc., determines the balance point of controlling the adjustment quality and compression ratio according to the parameter deployment result, ensures that the quality requirement is met while achieving a high compression ratio, can automatically adjust the parameters of the encoder according to the characteristics of the image and user demand, realize personalized encoding optimization, at the same time can realize high compression ratio while ensuring image quality, save storage space and network bandwidth.

[0065] S4: The obtained parameters are used to adjust the encoder, the data processed by the image is encoded by the adjusted encoder, and the encoding process is continuously recorded and evaluated, feedback is given according to the evaluation results, and continuous optimization is performed through deep learning.

[0066] Specifically, the preprocessed image data is encoded using the parameter-adjusted encoder, key indicators in the encoding process are recorded, the key indicators recorded in this embodiment are encoding speed, compression ratio and image quality indicators, and are compared with expected targets, and deep learning feedback is performed to continuously adjust and optimize the model, forming a closed-loop feedback mechanism.

[0067] In the specific implementation process, step S1 specifically includes the following steps:

[0068] S101: The collected historical image encoding processing data includes original images, encoding settings and final encoding results; the collected user historical data includes the user's preferred and commonly selected image quality level and compression ratio. The collected network state monitoring data includes monitoring network bandwidth and delay data, and is constructed into a data set.

[0069] Specifically, different types of original images are collected, and the encoding settings of each image are collected, including resolution, compression algorithm, compression level, etc.; the image quality level and preferred compression ratio selected by the user in different scenarios are collected through logs or user feedback, and the change of the user's image quality preference in a specific network environment such as Wi-Fi: mobile data is analyzed; network monitoring tools are used to continuously record the fluctuation of network bandwidth, including uplink and downlink rates, and collect monitoring network delay data, especially in high traffic periods and different geographic locations; the data is integrated to obtain a data set.

[0070] S102: Analyze the encoding habits, processing priorities of features in the image, and the association between the network and the encoding in the data set, and obtain the analysis results.

[0071] Specifically, the historical coding habits of the user are analyzed, specifically the user group in the data set, the coding mode selection, and the coding quality preference. In this embodiment, most users (about 70%) select the default coding parameter setting, and the remaining users (about 30%) adjust the coding parameters according to their own needs. About 30% of the total number of users prefer high quality, and tend to select higher coding quality, while about 50% of the total number of users prefer speed, and pay more attention to coding speed. The processing priority of the features in the image is analyzed, specifically the edge and texture information. In this embodiment, about 80% of the images will preferentially retain these key features during the encoding process. The analysis of the association between the network and the coding specifically includes the analysis of the influence of the network state on the coding and the mapping of the network state and the coding parameters. In this embodiment, it is analyzed that when the network bandwidth is low (such as less than 2 Mbps), the user tends to select a lower coding quality to reduce the transmission time, and when the network bandwidth is sufficient (such as more than 10 Mbps), the analysis result is obtained.

[0072] S103: Select a deep learning model architecture, and train the deep learning model architecture through the analysis result to construct a parameter adaptive adjustment model.

[0073] Specifically, according to the requirements of the analysis result, a suitable deep learning model architecture is selected. In this embodiment, a convolutional neural network (CNN) is used for automatic extraction of image features. The selected model architecture is trained using the data set. During the training process, the model learns how to predict the optimal coding parameters based on the input image features, user preferences, and network conditions. After the training is completed, the parameter adaptive adjustment model is obtained.

[0074] In the specific implementation process, step S3 specifically includes the following steps:

[0075] S301: Input the image processed data into the constructed parameter adaptive adjustment model. The parameter adaptive adjustment model performs forward propagation calculation based on the coding habits, feature processing priority, and the association between the network and the coding, and outputs the parameter deployment result.

[0076] Specifically, the image processed data is input into the constructed parameter adaptive adjustment model. After receiving the input, the parameter adaptive adjustment model processes the features of the coding habit processing layer, the image feature processing layer, and the network state processing layer in this embodiment. The features related to the user coding habits and the association between the network and the coding are extracted, and the features from the coding habit processing layer, the image feature processing layer, and the network state processing layer are fused. The forward propagation calculation is performed, the input data is transformed to extract the features related to the coding optimization, and after multiple transformations, the parameter adaptive adjustment model finally outputs a set of parameters reflecting the optimal coding settings under the current image content, user preferences, and network environment.

[0077] S302: Determine whether the output parameter adjustment result meets the expected target, and obtain the determination result.

[0078] Specifically, the model output parameter adjustment result is compared with the pre-set expected target, wherein the expected target includes the minimum acceptable image quality, the maximum allowed file size, and the expected loading time.

[0079] In this embodiment, the image quality PSNR threshold is set to 30 dB to ensure that the encoded image quality is not lower than this standard; the maximum allowed file size is set, the target is to compress the video file to not more than 100 MB, and the expected loading time is set: for real-time applications, ensure that the loading time of the video stream does not exceed 2 seconds.

[0080] S303: Based on the determination result, determine the balance point of controlling the quality and compression ratio, automatically adjust the parameters of the encoder, and obtain the parameter-adjusted encoder.

[0081] Specifically, based on the set PSNR threshold of 30 dB to ensure that the encoded image quality is not lower than this standard; set the maximum allowed file size, the target is to compress the video file to not more than 100 MB, set the expected loading time: for real-time applications, ensure that the loading time of the video stream does not exceed 2 seconds, adjust the internal settings of the encoder, including changing the bit rate allocation, selecting the appropriate encoding algorithm or adjusting the frame rate, etc. to achieve the best balance between quality and compression ratio, in this embodiment, by reducing the bit rate allocation of part of the frame to reduce the file size, while maintaining the high quality of the key frame, and for videos containing a large amount of details and dynamic content, select an encoding algorithm with higher compression efficiency but sacrifice a little quality, for video content that needs to capture fast action or needs high frame rate experience, maintain a higher frame rate; then after parameter adjustment, a customized encoder for the current image and environment is formed, ready for actual image encoding work.

[0082] In the specific implementation process, step S302 specifically includes the following steps:

[0083] S3021: Determine the expected target, including the minimum acceptable image quality, the maximum allowed file size, and the expected loading time.

[0084] S3022: Compare the output parameter adjustment result of the parameter self-adaptive adjustment with the expected target, and obtain the comparison result.

[0085] S3023: Analyze the comparison result, specifically, if the comparison result shows that the parameter adjustment result meets the expected target, proceed to the next step, if the comparison result shows that the parameter adjustment result does not meet the expected target, further analysis and feedback are needed.

[0086] Specifically, when the analysis and comparison result shows that the parameter deployment result does not meet the expected target, in-depth analysis is performed to determine whether the model prediction deviation, input data quality problem, or unreasonable target setting is the cause, and the analysis result is fed back to the model training or parameter adjustment process. For the model prediction deviation, more sample data is collected to retrain the model; if it is an input data problem, the preprocessing step is optimized; if the target setting is unreasonable, the application scenario requirement is reexamined, and the expected target is adjusted; if the result of the judgment shows that the parameter deployment result meets the expected target, the image-processed data is input to the configured encoder.

[0087] In the specific implementation process, step S4 specifically includes the following steps:

[0088] S401: input the image-processed data into the configured encoder for encoding operation.

[0089] Specifically, the image-processed data is taken as input, and the parameter-configured encoder API or library function is called to perform actual encoding operation. During the encoding process, the data is converted into a compressed format to reduce the file size.

[0090] S402: during the encoding operation of the image-processed data, key indicators are continuously recorded, specifically including the encoding speed, compression ratio, and image quality indicators.

[0091] Specifically, during the encoding process, the interfaces in the API or SDK provided by the encoder are used to regularly (e.g., every second) query the amount of data processed by the encoder, so as to calculate the encoding speed, compression ratio (the ratio of the size of the original data to the size of the compressed data), and image quality indicators (e.g., PSNR and SSIM), and record these indicators in a log file or a database in real time for subsequent analysis and optimization.

[0092] In this embodiment, the monitoring record time point is 1 second: 100 KB of data is processed, and between the 1st second and the 2nd second, the encoding speed is (220 KB-100 KB)) / 1 second=120 KB / s, at 10 seconds after the start of encoding: the size of the compressed data is 2 MB, and after the completion of encoding, the compression ratio is 10 MB / 5 MB=2:1, at 30 seconds after the start of encoding: PSNR=38 dB, and at 30 seconds after the start of encoding (the same key frame as PSNR): SSIM=0.95.

[0093] S403: after the encoding of the image-processed data is completed, the encoding result is evaluated, and the evaluation result and the recorded key indicators are fed back to the parameter self-adaptive adjustment model to obtain a feedback result.

[0094] Specifically, the data of indicators such as encoding speed, compression ratio and image quality indicators are summarized, compared with the expected target, analyzed and objectively evaluated, and then the evaluation results and recorded key indicators are fed back to the parameter adaptive adjustment model to provide learning samples for the model and indicate the pros and cons of the current configuration.

[0095] In this embodiment, the comparative analysis shows that the encoding speed reaches 150fps, which exceeds the expected 120fps, and performs well; the compression ratio is 5:1, which falls within the expected range of 4:1 to 6:1, and meets the requirements; the PSNR is 40dB, which exceeds the expected 38dB, and the image quality is high; the SSIM is 0.98, which also exceeds the expected 0.95, and the image quality is well maintained. The evaluation results and recorded key indicator data are fed back to the parameter adaptive adjustment model.

[0096] S404: Based on the feedback results, the parameter adaptive adjustment model is continuously optimized through deep learning technology.

[0097] Specifically, based on the feedback results, the parameter adaptive adjustment model is trained using deep learning algorithms such as neural networks, and through multiple iterations, the model weights are continuously adjusted until the model can accurately predict or recommend the optimal encoding parameter combination under specific conditions. Then, the performance of the optimized model is tested on the validation set, and after confirming the improvement, it is deployed back to the encoding process to achieve automatic parameter adjustment.

[0098] Embodiment Two

[0099] The image encoding optimization system based on deep learning, as shown in Figure 2 The optimization system includes:

[0100] A model construction module is configured to collect historical image encoding processing data, user historical data and network state monitoring data and analyze them, and train and construct a parameter adaptive adjustment model according to the analysis results.

[0101] Specifically, the model construction module is configured to collect historical image encoding processing data, user historical data and network state monitoring data, analyze the encoding habits, processing priorities of features in images and the correlation between the network and encoding, obtain analysis results, train and construct a parameter adaptive adjustment model according to the analysis results, and provide the most suitable image quality for different scenarios by understanding the encoding habits of users and network conditions, thereby improving user experience.

[0102] A data processing module is configured to collect image data and pre-process the image data to extract and obtain image processing data, wherein the pre-processing specifically includes color space conversion, image denoising, size adjustment and block processing.

[0103] Specifically, the data processing module is configured to select a suitable color space for conversion according to an application scenario and coding requirements, and convert an original image from an RGB color space to a target color space by using functions in an image processing library such as OpenCV; perform noise analysis on the image to determine the type and degree of noise, and perform noise reduction processing on the image by using a noise reduction function in the image processing library, so as to reduce noise in the image and improve image quality; select a suitable adjustment mode according to the specific content of the image and application requirements, and perform size adjustment on the image by using a size adjustment function in the image processing library; and divide the image into a plurality of small blocks by using a block division function in the image processing library, so that each small block can be independently subjected to subsequent encoding processing, thereby improving encoding efficiency.

[0104] The scheduling module is configured to input the image-processed data into the parameter adaptive adjustment model, obtain a parameter adjustment result, determine a balance point of adjusting quality and compression ratio according to the parameter adjustment result, automatically adjust parameters of the encoder, and obtain the parameter-adjusted encoder.

[0105] Specifically, the scheduling module is configured to input the preprocessed image data into the parameter adaptive adjustment model, and the parameter adaptive adjustment model automatically adjusts parameters of the encoder, including a quantization step and a filter setting, according to the input data, determines a balance point of adjusting quality and compression ratio according to a parameter adjustment result, ensures that a high compression ratio is achieved while a quality requirement is met, automatically adjusts parameters of the encoder according to characteristics of the image and user requirements, realizes personalized encoding optimization, and achieves a high compression ratio while ensuring image quality, thereby saving storage space and network bandwidth.

[0106] The encoding and optimization module is configured to encode the image-processed data by using the obtained parameter-adjusted encoder, continuously record and evaluate an encoding process, feed back a result of the evaluation, and continuously optimize by using deep learning.

[0107] Specifically, the encoding and optimization module is configured to encode the preprocessed image data using the parameter-adjusted encoder, record key indicators in the encoding process, and compare the recorded key indicators, such as encoding speed, compression ratio, and image quality indicators, with expected targets, and perform deep learning feedback to continuously adjust and optimize the model, thereby forming a closed-loop feedback mechanism.

[0108] In the specific implementation process, the model construction module includes:

[0109] The collection module is configured to collect historical image encoding processing data including original images, encoding settings, and final encoding results, collect user historical data including user preferred image quality levels and compression ratios, and collect network state monitoring data including monitored network bandwidth and delay data, and construct a data set.

[0110] Specifically, the collection module is configured to collect different types of original images, and collect encoding settings of each image, including resolution, compression algorithm, and compression level; collect image quality levels and preferred compression ratios selected by users in different scenarios through logs or user feedback, and analyze image quality preference changes of users in specific network environments such as Wi-Fi and mobile data; use network monitoring tools to continuously record network bandwidth fluctuations, including uplink and downlink rates, and collect monitoring network delay data, especially in high traffic periods and different geographic locations; and integrate the data to obtain a data set.

[0111] The analysis module is configured to analyze encoding habits, processing priorities of features in images, and the association between network and encoding in the data set, and obtain an analysis result.

[0112] Specifically, the analysis module is configured to analyze user historical encoding habits, specifically analyze user groups, encoding method selection, and encoding quality preference in the data set, and obtain an analysis result. In this embodiment, it is analyzed that most users (about 70%) in the user group select default encoding parameter settings, the remaining users (about 30%) adjust the encoding parameters according to their own needs, and high-quality preference users account for about 30% of the total number of users, and tend to select higher encoding quality, while speed priority users account for about 50% of the total number of users, and pay more attention to encoding speed; analyze the processing priority of features in the image, specifically analyze edge and texture information, and obtain an analysis result. In this embodiment, it is analyzed that about 80% of the images will preferentially retain these key features in the encoding process; and analyze the association between network and encoding, specifically including analyzing the influence of network state on encoding and the mapping between network state and encoding parameters, and obtain an analysis result.

[0113] The training module is configured to select a deep learning model architecture and train the deep learning model architecture by analyzing the results to build a parameter self-adaptive adjustment model.

[0114] Specifically, the training module is configured to select a suitable deep learning model architecture according to the requirements of the analysis results, and in this embodiment, a convolutional neural network (CNN) is selected for automatic extraction of image features. The selected model architecture is trained using a data set. During the training process, the model learns how to predict the optimal encoding parameters based on the input image features, user preferences, and network conditions. After the training is completed, the parameter self-adaptive adjustment model is obtained.

[0115] In the specific implementation process, the scheduling module includes:

[0116] The calculation module is configured to input the image-processed data into the built parameter self-adaptive adjustment model. The parameter self-adaptive adjustment model performs forward propagation calculation based on the encoding habits, feature processing priorities, and the association between the network and the encoding, and outputs the parameter adjustment result.

[0117] Specifically, the calculation module is configured to input the image-processed data into the built parameter self-adaptive adjustment model. After receiving the input, the parameter self-adaptive adjustment model performs forward propagation calculation through a series of layers (such as convolutional layers, fully connected layers, etc.) to transform the input data and extract features related to encoding optimization. After multiple layers of transformation, the parameter self-adaptive adjustment model finally outputs a set of parameters reflecting the optimal encoding settings under the current image content, user preferences, and network environment.

[0118] The judgment module is configured to judge whether the output parameter adjustment result meets the expected target and obtain a judgment result.

[0119] Specifically, the judgment module is configured to compare the parameter adjustment result output by the model with the pre-set expected target, wherein the expected target includes the minimum acceptable image quality, the maximum allowed file size, and the expected loading time.

[0120] In this embodiment, the image quality PSNR threshold is set to 30 dB to ensure that the image quality after encoding is not lower than this standard. The maximum allowed file size is set to compress the video file to not more than 100 MB. The expected loading time is set to ensure that the loading time of the video stream is not more than 2 seconds for real-time applications.

[0121] The deployment module is configured to determine the balance point of controlling the adjustment quality and the compression ratio based on the judgment result, automatically adjust the parameters of the encoder, and obtain the parameter-adjusted encoder.

[0122] Specifically, the deployment module is configured to set a PSNR threshold of 30 dB to ensure that the quality of the encoded image does not fall below this standard, set a maximum allowed file size, with the goal of compressing the video file to no more than 100 MB, set an expected loading time, for real-time applications, ensure that the loading time of the video stream does not exceed 2 seconds, adjust the internal settings of the encoder, including changing the bit rate allocation, selecting a suitable encoding algorithm, or adjusting the frame rate, to achieve the best balance between quality and compression ratio, in the embodiment, by reducing the bit rate allocation of some frames to reduce the file size while maintaining the high quality of key frames, and for videos containing a large amount of details and dynamic content, selecting an encoding algorithm with higher compression efficiency but sacrificing a little quality, for video content that needs to capture fast action or needs a high frame rate experience, maintaining a higher frame rate, and then after completing the parameter adjustment, forming an encoder customized for the current image and environment, ready for actual image encoding work.

[0123] In the specific implementation process, the judgment module comprises:

[0124] The selection module is configured to determine the expected target, including reducing the compression ratio while ensuring a certain image quality: achieving the expected target of fast encoding under certain network conditions.

[0125] The comparison module is configured to compare the parameter adjustment result of the parameter adjustment module with the expected target to obtain a comparison result.

[0126] The parameter evaluation module is configured to analyze the comparison result, specifically, if the comparison result shows that the parameter adjustment result meets the expected target, the next step is executed, if the comparison result shows that the parameter adjustment result does not meet the expected target, further analysis is needed and feedback is given.

[0127] Specifically, the parameter evaluation module is configured to analyze the comparison result that the parameter adjustment result does not meet the expected target, and further analyze whether it is caused by model prediction deviation, input data quality problem, or unreasonable target setting, and analyze the problem and feed back the analysis result to the model training or parameter adjustment process, for model prediction deviation, collect more sample data to retrain the model, if it is an input data problem, optimize the preprocessing step, if the target setting is unreasonable, the application scenario demand needs to be reexamined and the expected target needs to be adjusted, if the judgment result shows that the parameter adjustment result meets the expected target, the image processing data is input to the configured encoder.

[0128] In the specific implementation process, the encoding and optimization module comprises:

[0129] The encoding processing module is configured to input the image processing data into the configured encoder for encoding operation.

[0130] Specifically, the encoding processing module is configured to take the image-processed data as input, call an encoder API or a library function with configured parameters to perform actual encoding operation, and convert the data into a compressed format during the encoding process to reduce file size.

[0131] The recording module is configured to record key indicators during the encoding operation of the image-processed data, including encoding speed, compression ratio, and image quality indicators.

[0132] Specifically, the recording module is configured to use an interface in an API or SDK provided by the encoder to query the amount of data processed by the encoder during the encoding process at regular intervals (e.g., every second), calculate the encoding speed, compression ratio (the ratio of the size of the original data to the size of the compressed data), and image quality indicators (e.g., PSNR and SSIM), and record these indicators in a log file or a database in real time for subsequent analysis and optimization.

[0133] In this embodiment, the monitoring and recording time point is 1 second: 100KB of data is processed, and the encoding speed is (220KB-100KB) / 1s=120KB / s between the 1st second and the 2nd second. At 10 seconds after the encoding starts, the size of the compressed data is 2MB. After the encoding is completed, the compression ratio is 10MB / 5MB=2:1. At 30 seconds after the encoding starts, the PSNR is 38dB, and at 30 seconds after the encoding starts (the same key frame as the PSNR), the SSIM is 0.95.

[0134] The encoding evaluation module is configured to evaluate the encoding result after the encoding of the image-processed data is completed, and feed back the evaluation result and the recorded key indicators to the parameter adaptive adjustment model to obtain a feedback result.

[0135] Specifically, the encoding evaluation module is configured to compare and analyze the data of the encoding speed, compression ratio, and image quality indicators with expected targets, perform objective quality evaluation, and then feed back the evaluation result and the recorded key indicators to the parameter adaptive adjustment model to provide learning samples for the model and indicate the pros and cons of the current configuration.

[0136] In this embodiment, the comparison and analysis show that the encoding speed reaches 150fps, which exceeds the expected 120fps and performs excellently. The compression ratio is 5:1, which falls within the expected range of 4:1 to 6:1 and meets the requirements. The PSNR is 40dB, which exceeds the expected 38dB and the image quality is high. The SSIM is 0.98, which also exceeds the expected 0.95 and the image quality is well maintained. The evaluation result and the recorded key indicator data are fed back to the parameter adaptive adjustment model.

[0137] The feedback optimization module is configured to continuously optimize the parameter self-adaptive adjustment model based on the feedback results by using deep learning techniques.

[0138] Specifically, the feedback optimization module is configured to train the parameter self-adaptive adjustment model based on the feedback results by using deep learning algorithms (such as neural networks), and continuously adjust the model weights through multiple iterations until the model can accurately predict or recommend the optimal encoding parameter combination under specific conditions. Then, the performance of the optimized model is tested on a validation set, and after confirming the improvement, the model is deployed back to the encoding process to realize automatic parameter adjustment.

[0139] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of the flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0140] A person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer readable medium capable of carrying or storing data.

[0141] It is also to be noted that the terms "comprising", "comprises" or "comprised of" when used in this specification are not to be interpreted in an excluding nature, such that processes, methods, articles or devices comprising a list of elements are not to exclude other elements not explicitly listed, or other elements inherent to such processes, methods, articles or devices. An element procured in the phrase "comprising a... " does not exclude the presence of additional identical elements in the process, method, article or device comprising the element, unless more limited by context.

Claims

1. A method for image coding optimization based on deep learning, characterized in that, The optimization method comprises the following steps: S1: Collect historical image encoding processing data, user historical data and network state monitoring data and analyze them, and train and construct a parameter adaptive adjustment model according to the analysis result; S2: Collect image data and pre-process the image data to extract and obtain image processing data, wherein the pre-processing specifically comprises color space conversion, image denoising, size adjustment and block processing; S3: input the image processing data into the parameter adaptive adjustment model to obtain the parameter adjustment result, and determine the balance point of controlling the adjustment quality and the compression ratio according to the parameter adjustment result, automatically adjust the parameters of the encoder, and obtain the parameter-adjusted encoder; Wherein, step S3 specifically comprises the following steps: S301: input the image processing data into the constructed parameter adaptive adjustment model, the parameter adaptive adjustment model extracts features through the encoding habit processing layer, the image feature processing layer and the network state processing layer, fuses the features extracted by each layer, performs forward propagation calculation, and outputs the parameter adjustment result; Wherein, the encoding habit is specifically the selection preference of the encoding parameters corresponding to different user groups, and the encoding parameters include quantization step parameters and filter setting parameters; S302: determine whether the output parameter adjustment result meets the expected target, and obtain a judgment result; S303: based on the judgment result, determine the balance point of controlling the adjustment quality and the compression ratio, automatically adjust the encoding parameters of the encoder, and obtain the parameter-adjusted encoder; S4: encode the image processing data through the obtained parameter-adjusted encoder, continuously record and evaluate the encoding process, feed back according to the evaluation result, and continuously optimize through deep learning. 2.The deep learning-based image encoding optimization method of claim 1, wherein, Step S1 specifically comprises the following steps: S101: collect historical image encoding processing data, specifically including collecting original images, encoding settings and final encoding results; collect user historical data, specifically including collecting image quality and compression ratio selected by different user groups; collect network state monitoring data, specifically including collecting network bandwidth and delay data, and construct a data set; S102: analyze the encoding habit, the processing priority of the features in the image and the association between the network and the encoding in the data set, and obtain an analysis result; S103: select a deep learning model architecture, and train the deep learning model architecture through the analysis result to construct a parameter adaptive adjustment model. 3.The deep learning-based image encoding optimization method of claim 2, wherein, Step S302 specifically comprises the following steps: S3021: determine the expected target of the lowest acceptable image quality, the maximum allowed file size and the expected loading time; S3022: compare the parameter adjustment result output by the parameter adaptive adjustment model with the expected target to obtain a comparison result; S3023: analyze the comparison result, specifically, if the comparison result shows that the parameter adjustment result meets the expected target, proceed to the next step, if the comparison result shows that the parameter adjustment result does not meet the expected target, analyze the reason why it does not meet the expected target, and retrain the model according to the reason.

4. The deep learning-based image encoding optimization method of claim 3, wherein, Step S4 specifically comprises the following steps: S401: input the image processing data into the configured encoder to perform encoding operation; S402: In the process of encoding the data after image processing, the key indicators are continuously recorded, wherein the key indicators include encoding speed, compression ratio and image quality indicators; S403: After the encoding of the data after image processing is completed, the encoding result is evaluated, and the evaluation result and the recorded key indicators are fed back to the parameter adaptive adjustment model to obtain the feedback result of the encoding parameters; S404: Based on the feedback result, the parameter adaptive adjustment model is continuously trained by deep learning technology to optimize the parameter adaptive adjustment model to accurately predict the optimal combination of encoding parameters.

5. The deep learning based image coding optimization system applied to the deep learning based image coding optimization method according to any one of claims 1-4, characterized in that, The optimization system comprises: A model construction module for collecting historical image encoding processing data, user historical data and network state monitoring data and analyzing, and training and constructing a parameter adaptive adjustment model according to the analysis result; A data processing module for collecting image data and pre-processing the image data to extract and obtain the data after image processing, wherein the pre-processing specifically includes color space conversion, image denoising, size adjustment and block processing; A scheduling module for inputting the data after image processing to the parameter adaptive adjustment model to obtain a parameter deployment result, and determining a balance point of controlling the quality and compression ratio according to the parameter deployment result, automatically adjusting the parameters of the encoder to obtain a parameter-adjusted encoder; The scheduling module comprises: A calculation module for inputting the data after image processing into the constructed parameter adaptive adjustment model, the parameter adaptive adjustment model extracting features through an encoding habit processing layer, an image feature processing layer and a network state processing layer, fusing the features extracted by each layer, performing forward propagation calculation, and outputting a parameter deployment result; Wherein, the encoding habit is specifically the selection preference of the encoding parameters corresponding to different user groups, and the encoding parameters include quantization step parameters and filter setting parameters; A judgment module for judging whether the output parameter deployment result meets the expected target to obtain a judgment result; A deployment module for determining a balance point of controlling the quality and compression ratio based on the judgment result, automatically adjusting the encoding parameters of the encoder to obtain a parameter-adjusted encoder; An encoding and optimization module for encoding the data after image processing by the obtained parameter-adjusted encoder, continuously recording and evaluating the encoding process, feeding back the evaluation result, and continuously optimizing by deep learning.

6. The deep learning based image encoding optimization system of claim 5, wherein, The model construction module comprises: An acquisition module for acquiring historical image encoding processing data, specifically including acquiring original images, encoding settings and final encoding results; acquiring user historical data, specifically including acquiring image quality and compression ratio selected by different user groups; and acquiring network state monitoring data, specifically including acquiring monitoring network bandwidth and delay data, and constructing a data set; An analysis module for analyzing the encoding habit, the processing priority of the features in the image and the association between the network and the encoding in the data set to obtain an analysis result; A training module for selecting a deep learning model architecture and training the deep learning model architecture by the analysis result to construct a parameter adaptive adjustment model.

7. The deep learning based image coding optimization system of claim 6, wherein, The judgment module comprises: The selected module is configured to determine the expected target of the lowest acceptable image quality, the maximum allowed file size, and the desired loading time, wherein the image quality is a specific value of PSNR (peak signal-to-noise ratio); The comparison module is configured to compare the parameter adjustment result of the parameter adaptive adjustment output with the expected target to obtain a comparison result; The parameter evaluation module is configured to analyze the comparison result, specifically, if the comparison result indicates that the parameter adjustment result meets the expected target, the next step is performed, and if the comparison result indicates that the parameter adjustment result does not meet the expected target, the reason for not meeting the expected target is analyzed, and the model is retrained according to the reason.

8. The deep learning based image coding optimization system of claim 7, wherein, The encoding and optimization module comprises: The encoding processing module is configured to input the image-processed data into the configured encoder to perform encoding operation; The recording module is configured to continuously record key indicators during the encoding operation of the image-processed data, wherein the key indicators specifically include encoding speed, compression ratio, and image quality indicators; The encoding evaluation module is configured to evaluate the encoding result after the encoding of the image-processed data is completed, and feed back the evaluation result and the recorded key indicators to the parameter adaptive adjustment model to obtain a feedback result of the encoding parameters; The feedback optimization module is configured to continuously train the parameter adaptive adjustment model based on the feedback result through deep learning technology to optimize the parameter adaptive adjustment model to accurately predict the optimal encoding parameter combination.

Images