Deep learning-based image intelligent compression method, system and medium

Abstract

The application relates to an image intelligent compression method, system and medium based on deep learning. The method comprises the following steps: performing cooperative analysis on the fluctuation degree and the autocorrelation degree of a historical bandwidth sequence corresponding to each period of a network transmission layer, to obtain a predicted bandwidth of a next period; calculating a safe offset of each period according to the deviation degree of the predicted bandwidth and the actual bandwidth of each period, and determining a safe lower limit of each period according to the difference between the safe offset of each period and the predicted bandwidth of the next period; calculating an effective receiving code rate of each period according to the negative correlation degree between the packet loss rate and the actual bandwidth of the network transmission layer in each period; determining a code rate safe upper limit of each period according to the comparison result of the safe lower limit of each period and the effective receiving code rate of each period, and encoding a to-be-compressed image according to the code rate safe upper limit of each period. The application can improve the visual quality of the receiving end under the premise of ensuring smooth transmission.

Description

Technical Field

[0001] This application relates to the field of image communication technology, specifically to an image intelligent compression method, system, and medium based on deep learning. Background Technology

[0002] In real-time image transmission scenarios such as video conferencing, cloud gaming, and remote control, image compression is a core element in balancing transmission bandwidth and visual quality. Deep learning-based image compression methods are gradually replacing traditional hybrid coding frameworks due to their end-to-end optimization capabilities, but bitrate control in dynamic network environments still faces significant challenges.

[0003] Current deep learning image compression systems rely entirely on existing network conditions for bitrate control decisions. When bandwidth fluctuates drastically and briefly, bitrate adjustments consistently lag behind these changes. When bandwidth drops sharply, encoded high-bitrate data packets accumulate at bottlenecks, and bitrate reduction only takes effect after a feedback signal. This time lag, with continuously transmitted data packets exacerbating congestion, eventually leads to buffer exhaustion and image stuttering at the receiver. Similarly, when bandwidth recovers, bitrate increase is delayed, wasting available bandwidth. Therefore, current technologies cannot anticipate future network condition changes before bandwidth fluctuations occur, nor can they establish differentiated protection mechanisms against the asymmetric impact of prediction errors, resulting in an inability to maximize visual quality at the receiver while ensuring smooth transmission. Summary of the Invention

[0004] To address the aforementioned technical problems, the purpose of this application is to provide a deep learning-based intelligent image compression method, system, and medium. The specific technical solution adopted is as follows: Firstly, a deep learning-based intelligent image compression method is provided, the method comprising: In multiple cycles, the fluctuation and autocorrelation of the historical bandwidth sequence corresponding to each cycle of the network transport layer are analyzed in a coordinated manner to obtain the predicted bandwidth for the next cycle of each cycle. In the plurality of cycles, the safety offset of each cycle is calculated based on the degree of deviation between the predicted bandwidth and the actual bandwidth of each cycle, and the safety lower bound of each cycle is determined based on the difference between the safety offset of each cycle and the predicted bandwidth of the next cycle; the safety lower bound is used to constrain the coding rate of the next cycle of the corresponding cycle. The effective received bit rate for each cycle is calculated based on the negative correlation between the packet loss rate and the actual bandwidth of the network transport layer in each cycle; the effective received bit rate indicates the maximum bit rate that the network transport layer can effectively transmit in the current cycle. Based on the comparison between the lower bound of the security for each cycle and the effective received bit rate for each cycle, the upper bound of the bit rate for each cycle is determined, and the image to be compressed is encoded according to the upper bound of the bit rate for each cycle.

[0005] Optionally, in multiple periods, a collaborative analysis is performed on the fluctuation and autocorrelation of the historical bandwidth sequence corresponding to each period of the network transport layer to obtain the predicted bandwidth for the next period, including: In multiple cycles, the ratio of the standard deviation to the mean of all actual bandwidths in the historical bandwidth sequence corresponding to each cycle of the network transport layer is calculated to obtain the fluctuation coefficient of the next cycle of each cycle; multiple sequence points in the historical bandwidth sequence correspond one-to-one with multiple cycles, and multiple cycles include each cycle and multiple consecutive historical cycles before it. The autocorrelation decay distance for the next period is determined based on the number of lag steps corresponding to the first time the autocorrelation function value of the historical bandwidth sequence falls below the preset autocorrelation threshold. The fluctuation coefficient and autocorrelation decay distance of the next period are fused to obtain the prediction coefficient of the next period. The actual bandwidth of each period, the predicted bandwidth of each period, and the prediction coefficient of the next period are fused to obtain the predicted bandwidth of the next period.

[0006] Optionally, the fluctuation coefficient and the autocorrelation decay distance of the next period are fused to obtain the prediction coefficient for the next period, including: The initial coefficient value for the next period is obtained by combining the product of the fluctuation coefficient of the next period with the preset first constant weight and the ratio of the preset second constant weight to the autocorrelation value; the autocorrelation value indicates the larger value between the preset value and the autocorrelation decay distance of the next period. The prediction coefficients for the next period are determined by comparing the initial coefficient values ​​with the preset values.

[0007] Optionally, in the plurality of periods, a safety offset for each period is calculated based on the degree of deviation between the predicted bandwidth and the actual bandwidth for each period, including: If the predicted bandwidth of each period is greater than the actual bandwidth in multiple periods, the difference between the actual bandwidth and the predicted bandwidth in each period is calculated to obtain the error magnitude of each period. Obtain the error magnitude of each period and multiple historical periods before it to obtain multiple error magnitudes, and sort the multiple error magnitudes from smallest to largest to obtain the set of historical prediction errors corresponding to each period; The target quantile for each cycle is determined based on the buffer occupancy rate at the receiving end in each cycle; the target quantile is inversely proportional to the buffer occupancy rate. Based on the historical prediction error set corresponding to each period and the target quantile for each period, the safety offset for each period is determined; the safety offset is used to offset the risk of prediction bandwidth.

[0008] Optionally, based on the historical prediction error set corresponding to each period and the target quantile for each period, a safety offset for each period is determined, including: From the historical prediction error set corresponding to each period, the error magnitude corresponding to the target quantile is extracted to obtain the safety offset for each period.

[0009] Optionally, a lower bound for each period is determined based on the difference between the security offset of each period and the predicted bandwidth of the next period, including: The difference between the predicted bandwidth for the next cycle and the safety offset for each cycle is calculated to obtain the safety lower bound for each cycle.

[0010] Optionally, the effective received code rate for each cycle is calculated based on the negative correlation between the packet loss rate and the actual bandwidth at the network transport layer in each cycle, including: The effective received bit rate for each period is obtained by multiplying the actual bandwidth of each period by the packet loss difference of each period; the packet loss difference of each period indicates the difference between the preset value and the packet loss rate of each period.

[0011] Optionally, based on the comparison between the lower bound of each cycle and the effective received bit rate of each cycle, a safe upper bound of the bit rate for each cycle is determined, and the image to be compressed is encoded according to the safe upper bound of the bit rate for each cycle, including: The lower bound of the security for each cycle is determined by the smaller of the lower bound of the security for each cycle and the effective received code rate for each cycle. The basic bitrate of the content is determined based on the image characteristics of the image to be compressed; A feasible bitrate range is constructed with the basic bitrate of the content as the lower limit and the safe upper limit of the bitrate for each period as the upper limit. The bitrate to be compressed is then selected within the feasible bitrate range for encoding.

[0012] Secondly, a deep learning-based intelligent image compression system is provided. The system includes an analysis module, which is used to perform collaborative analysis on the fluctuation and autocorrelation of the historical bandwidth sequence of the network transport layer in each cycle to obtain the predicted bandwidth for the next cycle of each cycle. The determination module is used to calculate the safety offset of each period based on the degree of deviation between the predicted bandwidth and the actual bandwidth of each period, and to determine the safety lower bound of each period based on the difference between the safety offset of each period and the predicted bandwidth of the next period; the safety lower bound is used to constrain the coding rate of the next period of the corresponding period. The calculation module is used to calculate the effective received bit rate for each cycle based on the negative correlation between the packet loss rate and the actual bandwidth of the network transport layer in each cycle; the effective received bit rate indicates the maximum bit rate that the network transport layer can effectively transmit in the current cycle. The encoding module is used to determine the upper limit of the bit rate for each cycle based on the comparison between the lower limit of the security for each cycle and the effective received bit rate for each cycle, so as to encode the image to be compressed according to the upper limit of the bit rate for each cycle.

[0013] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0014] This application offers the following advantages: It collaboratively analyzes the fluctuation and autocorrelation of historical bandwidth sequences in the network transport layer across multiple periods to obtain the predicted bandwidth for the next period. It leverages the inertial characteristics of network bandwidth over short timescales to capture changing trends and quantify its uncertainty from historical bandwidth sequences, thus constructing a safe bitrate boundary in advance. A safety offset is calculated based on the deviation between the predicted and actual bandwidth for each period. This safety offset is then used to offset the risk of the predicted bandwidth for the next period, resulting in a safe lower bound. Since the safety offset is negatively correlated with buffer occupancy, the encoder can detect the decreasing trend before the actual network bandwidth begins to decline and reduce the output bitrate in advance, avoiding sudden data packet accumulation at bottlenecks and increasing the bitrate before bandwidth recovery to synchronize image quality recovery. The effective received bitrate is calculated based on the negative correlation between packet loss rate and actual bandwidth as a physical hard constraint for the current period. The smaller of the safe lower bound and the effective received bitrate is used as the upper bound for bitrate safety, subjecting coding decisions to both future prediction constraints and current physical constraints. Encoding the image to be compressed according to the safe upper limit of the bit rate for each cycle eliminates the inherent response delay of feedback adjustment, thereby improving the visual quality of the receiving end while ensuring smooth transmission. Attached Figure Description

[0015] Figure 1 This is a flowchart of a deep learning-based intelligent image compression method in one embodiment; Figure 2 This is a schematic diagram of the structure of a deep learning-based intelligent image compression system in one embodiment. Detailed Implementation

[0016] The following description, in conjunction with the accompanying drawings, details a specific scheme for a deep learning-based intelligent image compression method provided in this application. For example... Figure 1 As shown, the method includes: S11. In multiple cycles, the fluctuation and autocorrelation of the historical bandwidth sequence corresponding to each cycle of the network transport layer are analyzed collaboratively to obtain the predicted bandwidth for the next cycle of each cycle.

[0017] This application deploys a network monitoring module at the sending end in the network transport layer to send probe data packets to the receiving end at set intervals, records the round-trip time from sending to receiving the acknowledgment reply, and calculates the actual bandwidth for the current period. The total number of data packets sent and the number of data packets acknowledged by the receiver within each measurement window are simultaneously counted to calculate the packet loss rate. The receiving end sends back the ratio of the number of frames currently cached in the playback buffer to the total capacity of the buffer at the same interval, which is also known as the buffer occupancy rate. For each current period, a historical bandwidth sequence is constructed that traces back multiple consecutive historical periods from the current period as the end of the time window. This yields the historical bandwidth sequence corresponding to the current period. Each historical bandwidth sequence contains the actual bandwidth of each period and multiple consecutive historical periods preceding it. Multiple sequence points in the historical bandwidth sequence correspond one-to-one with multiple periods, and the value of each sequence point is the actual bandwidth of each period. In this embodiment, the multiple periods analyzed include the current period and multiple consecutive historical periods preceding it.

[0018] In real-time image transmission scenarios, network bandwidth changes are not entirely random, but exhibit a certain short-term inertia and variation pattern. To introduce foresight into bitrate decisions, it is necessary to predict future network bandwidth.

[0019] In one embodiment, across multiple periods, a collaborative analysis is performed on the fluctuation and autocorrelation of the historical bandwidth sequence corresponding to each period of the network transport layer to obtain the predicted bandwidth for the next period, including: In multiple cycles, the ratio of the standard deviation to the mean of all actual bandwidths in the historical bandwidth sequence corresponding to each cycle of the network transport layer is calculated to obtain the fluctuation coefficient of the next cycle of each cycle; multiple sequence points in the historical bandwidth sequence correspond one-to-one with multiple cycles, and multiple cycles include each cycle and multiple consecutive historical cycles before it. The autocorrelation decay distance for the next period is determined based on the number of lag steps corresponding to the first time the autocorrelation function value of the historical bandwidth sequence falls below the preset autocorrelation threshold. The fluctuation coefficient and autocorrelation decay distance of the next period are fused to obtain the prediction coefficient of the next period. The actual bandwidth of each period, the predicted bandwidth of each period, and the prediction coefficient of the next period are fused to obtain the predicted bandwidth of the next period.

[0020] Specifically, the fluctuation coefficient and autocorrelation decay distance of the next period are fused to obtain the prediction coefficient for the next period, including: The initial coefficient value for the next period is obtained by combining the product of the fluctuation coefficient of the next period with the preset first constant weight and the ratio of the preset second constant weight to the autocorrelation value; the autocorrelation value indicates the larger value between the preset value and the autocorrelation decay distance of the next period. The prediction coefficients for the next period are determined by comparing the initial coefficient values ​​with the preset values.

[0021] In multiple consecutive cycles, the ratio of the standard deviation to the mean of all actual bandwidths in the historical bandwidth sequence corresponding to each cycle of the network transport layer is calculated to obtain the fluctuation coefficient of the next cycle. Specifically, the historical bandwidth sequence corresponding to the i-th cycle is obtained, and the mean of the historical bandwidth sequence corresponding to the i-th cycle is... Reflects the center level and standard deviation of bandwidth This reflects the fluctuation range and calculates the standard deviation. and mean The ratio of these values ​​yields the fluctuation coefficient for the (i+1)th period. Volatility coefficient The degree of network volatility was quantified: The larger the value, the more violently the network is oscillating, and the less valuable the historical data becomes. The smaller the value, the more stable the network, and the higher the reference value of historical data.

[0022] In some scenarios, only That's not enough. Take a rapidly moving device as an example: the bandwidth may remain at a high level but change extremely frequently, and the correlation between adjacent cycles is inherently weak. To quantify the strength of this correlation, we can calculate the autocorrelation degree of the historical bandwidth sequence, specifically the autocorrelation function value of the historical bandwidth sequence. The calculation formula is: ; in, The lag step is k cycles, and E is the expected value. Let be the actual bandwidth of the i-th cycle. This represents the actual bandwidth in the (i+k)th cycle. Let be the mean of the historical bandwidth sequence corresponding to the i-th period. Let be the standard deviation of the historical bandwidth sequence corresponding to the i-th period. For extremely small values, such as The autocorrelation decay distance in the (i+1)th period for The first time it falls below the autocorrelation threshold (e.g., 0.5) corresponds to value, Used to measure the similarity of bandwidth sequences at different time points, the autocorrelation threshold can be configured according to the fluctuation sensitivity of the actual network environment. A higher value is taken when the network is sensitive to fluctuations so that the prediction is more dependent on the current observation, and a lower value is taken when the network has a higher tolerance for fluctuations so as to make more use of historical trends. It measures the effective memory length of the actual bandwidth. The smaller the value, the weaker the correlation between actual bandwidths at adjacent moments, meaning historical data has almost no reference value for the future. This weak inertia affects prediction by making the component in exponential smoothing that relies on historical trends unreliable, necessitating a further increase in the weight of current observations.

[0023] The fluctuation coefficient and autocorrelation decay distance of the next period are fused to obtain the prediction coefficient for the next period. Specifically, the prediction coefficient for the (i+1)th period is... It can be calculated using the following linear formula: ; The preset value can be 1. The first constant weight, The second constant weight, and The values ​​used to balance the contribution of fluctuation coefficient and autocorrelation decay distance to the prediction coefficient can be obtained through offline training in typical network scenarios via grid search or Bayesian optimization, or through online adaptive adjustment based on actual transmission performance after system deployment. Let be the fluctuation coefficient for the (i+1)th period. The autocorrelation decay distance in the (i+1)th period is... Let be the autocorrelation value, and min() is the function to minimize it.

[0024] Based on volatility coefficient and autocorrelation decay distance Two scenario metrics that can be dynamically determined The value of is chosen to match the prediction strategy with the fluctuation range and inertia characteristics of the current network. A large value indicates that the network is in a state of violent fluctuation, and the reference value of historical data is reduced. A larger value should be chosen to ensure the prediction closely follows the current bandwidth; When the network is smaller, it is relatively stable. A smaller value can be chosen to fully utilize historical trends and smooth out transient noise.

[0025] The actual bandwidth of each period, the predicted bandwidth of each period, and the prediction coefficient for the next period are fused to obtain the predicted bandwidth for the next period and the predicted bandwidth for the (i+1)th period. The calculation formula is: ,in, Let be the actual bandwidth of the i-th cycle. Let be the predicted bandwidth for the i-th period. This determines whether the prediction bandwidth follows current observations more closely or relies more on historical trends.

[0026] S12. In the plurality of cycles, the safety offset of each cycle is calculated based on the degree of deviation between the predicted bandwidth and the actual bandwidth of each cycle, and the safety lower bound of each cycle is determined based on the difference between the safety offset of each cycle and the predicted bandwidth of the next cycle.

[0027] The safety lower bound is used to constrain the coding rate of the next cycle of the corresponding cycle.

[0028] Prediction always involves errors, and in real-time image transmission, the consequences of prediction errors are severely unequal in two directions: when the predicted bandwidth of the same period is higher than the actual bandwidth (negative error), it may lead to excessive data transmission, causing congestion and image stuttering; while when the predicted bandwidth of the same period is lower than the actual bandwidth (positive error), it only results in suboptimal image quality, but smooth transmission. Therefore, an asymmetric safety boundary can be constructed, focusing on preventing negative errors, with the lower safety bound used to constrain the coding bitrate of the next period.

[0029] In one embodiment, within the plurality of periods, a security offset for each period is calculated based on the degree of deviation between the predicted bandwidth and the actual bandwidth for each period, including: If the predicted bandwidth of each period is greater than the actual bandwidth in multiple periods, the difference between the actual bandwidth and the predicted bandwidth in each period is calculated to obtain the error magnitude of each period. Obtain the error magnitude of each period and multiple historical periods before it to obtain multiple error magnitudes, and sort the multiple error magnitudes from smallest to largest to obtain the set of historical prediction errors corresponding to each period; The target quantile for each cycle is determined based on the buffer occupancy rate at the receiving end in each cycle; the target quantile is inversely proportional to the buffer occupancy rate. Based on the historical prediction error set corresponding to each period and the target quantile for each period, the safety offset for each period is determined; the safety offset is used to offset the risk of prediction bandwidth.

[0030] Specifically, based on the historical prediction error set corresponding to each period and the target quantile for each period, the safety offset for each period is determined, including: From the historical prediction error set corresponding to each period, the error magnitude corresponding to the target quantile is extracted to obtain the safety offset for each period.

[0031] For the i-th period, if the predicted bandwidth of the i-th period is greater than its actual bandwidth, then the difference between the actual bandwidth and the predicted bandwidth is calculated to obtain the error magnitude of the i-th period. That is, the error range Obtain the error magnitudes of the i-th period and several previous historical periods, for example, 10 historical periods. Sort these error magnitudes from smallest to largest to obtain the historical prediction error set corresponding to the i-th period. ,in, , Let be the predicted bandwidth for the i-th period. is the actual bandwidth of the i-th cycle.

[0032] Get the buffer occupancy of the receiver in each cycle. (That is, the ratio of the number of cached frames to the total buffer capacity). Buffer occupancy rate. This reflects the system's fault tolerance margin against network fluctuations. The smaller the value, the lower the system's tolerance for negative bias. Therefore, according to... Dynamically determine a target quantile Target quantile and buffer occupancy They show an inverse relationship. Target quantile It can be calculated using the following linear formula: ,in, and These are the preset upper and lower limits for quantiles. For example, when At lower (close to 0), near (For example, 85%); when When it is relatively high (close to 1), near (e.g., 70%).

[0033] Determine if the content of the historical prediction error set corresponding to the current period is empty; if so, forcibly assign a preset default value to the safety offset of the current period; if not empty, extract the error magnitude corresponding to the target quantile from the historical prediction error set to obtain the safety offset of the current period. Based on the historical prediction error set corresponding to each period... and the target quantile for each period Determine the safe offset for each cycle. Specifically, from the historical prediction error set In the process, target quantiles are extracted. Corresponding error range: ,in, The function is used to retrieve exist The value at that location. Safety offset. Used to offset the risk of predicted bandwidth. The larger, The larger the value, the greater the safety margin.

[0034] In one embodiment, the lower bound of a security period is determined based on the difference between the security offset of each period and the predicted bandwidth of the next period, including: The difference between the predicted bandwidth for the next cycle and the safety offset for each cycle is calculated to obtain the safety lower bound for each cycle.

[0035] The safe lower bound for the i-th period The calculation formula is: Safe lower bound Based on historical prediction experience, under the current network fluctuation pattern, the probability that the actual bandwidth is lower than this value does not exceed [a certain percentage]. Furthermore, this probability threshold is adaptively adjusted according to the buffer margin.

[0036] S13. Calculate the effective received code rate for each cycle based on the negative correlation between the packet loss rate and the actual bandwidth of the network transport layer in each cycle.

[0037] The effective received bit rate indicates the maximum bit rate that the network transport layer can effectively transmit in the current cycle.

[0038] In one embodiment, the effective received code rate for each cycle is calculated based on the negative correlation between the packet loss rate and the actual bandwidth at the network transport layer in each cycle, including: The effective received bit rate for each period is obtained by multiplying the actual bandwidth of each period by the packet loss difference of each period; the packet loss difference of each period indicates the difference between the preset value and the packet loss rate of each period.

[0039] Effective received code rate in the i-th period The calculation formula is: , Let be the actual bandwidth of the i-th cycle. Let be the packet loss rate in the i-th period. Let $\frac{i}{i}$ be the packet loss difference in the i-th period. If the encoder output bitrate exceeds $\frac{i}{i}$, then $\frac{i}{i}$ is the packet loss difference in the i-th period. Excess data packets will likely be lost during transmission, which not only does not improve image quality but also exacerbates network congestion.

[0040] S14. Based on the comparison between the lower bound of the security for each period and the effective received bit rate for each period, determine the upper bound of the bit rate for each period, and encode the image to be compressed according to the upper bound of the bit rate for each period.

[0041] Safe lower bound Represents the constraints of future projections, effective received bit rate This represents the current physical constraints. Both have their own focuses under different network conditions, and using either alone has blind spots; therefore, they can be combined.

[0042] In one embodiment, a code rate safety upper limit for each cycle is determined based on a comparison between the safety lower bound and the effective received code rate for each cycle, and the image to be compressed is encoded according to the code rate safety upper limit for each cycle, including: The lower bound of the security for each cycle is determined by the smaller of the lower bound of the security for each cycle and the effective received code rate for each cycle. The basic bitrate of the content is determined based on the image characteristics of the image to be compressed; A feasible bitrate range is constructed with the basic bitrate of the content as the lower limit and the safe upper limit of the bitrate for each period as the upper limit. The bitrate to be compressed is then selected within the feasible bitrate range for encoding.

[0043] When the internet is in a downturn It's already very stringent. Because the bandwidth value during off-peak periods is inherently lower, the prediction model gives... It is also at a low level. It may actually be higher If at this point only... If the upper limit is set too high, the bit rate may be amplified before the channel has truly recovered, causing a new round of congestion. Therefore, in this scenario, the upper limit should be set to... As the standard.

[0044] When the network is at its peak but shows signs of slowing down—for example, when users begin to move outwards from the edge of Wi-Fi coverage—the frequency and magnitude of negative biases in recent prediction errors increase as bandwidth begins to decline from its high point. The distribution widens, leading to The price dropped to below 100000 in advance. If at this point only... Setting the upper limit as such would be tantamount to ignoring the signals that downside risks are accumulating; it should be based on... To ensure this, a buffer is reserved in advance for upcoming bandwidth slumps. This means setting a safe bitrate limit for each cycle. The calculation formula is: .

[0045] After obtaining the bitrate safety upper limit, the encoder determines the final encoding bitrate by combining the image features of the image to be compressed. In one embodiment, a basic content bitrate is first determined based on image features such as spatial entropy and inter-frame residual energy of the image to be compressed. Further, the step of determining the basic content bitrate can be specifically configured as follows: First, obtain the spatial entropy and inter-frame residual energy of the image to be compressed. Then, process it using a preset weighted fusion rule: using a set minimum guaranteed bitrate constant as the base, accumulate the product of the spatial entropy and a first set conversion coefficient, and accumulate the product of the inter-frame residual energy and a second set conversion coefficient. The first and second set conversion coefficients are obtained by querying a preset mapping table. The mapping table stores weighted coefficient pairs of spatial entropy and inter-frame residual energy corresponding to different image complexity levels (e.g., flat, medium, rich detail). The mapping table is pre-constructed through offline regression training on a public image dataset. Finally, the calculated final integrated value is used as the basic content bitrate with bitrate measurement units. Then, a feasible bitrate range is constructed, with the content-based bitrate as the lower limit and the bitrate safety upper limit as the upper limit. Finally, within the feasible bitrate range, the bitrate that achieves optimal encoded image quality and whose encoding latency does not exceed the end-to-end latency budget is selected. This bitrate is then mapped to the encoder's quantization parameters and encoding structure configuration, controlling the encoder to complete image compression of the image to be compressed at the target bitrate.

[0046] This application also provides a deep learning-based intelligent image compression system, such as... Figure 2 As shown, the system includes: Analysis module 21 is used to perform collaborative analysis on the fluctuation and autocorrelation of the historical bandwidth sequence of the network transport layer in each cycle, and to obtain the predicted bandwidth for the next cycle of each cycle. The determining module 22 is used to calculate the safety offset of each period based on the degree of deviation between the predicted bandwidth and the actual bandwidth of each period in the plurality of periods, and to determine the safety lower bound of each period based on the difference between the safety offset of each period and the predicted bandwidth of the next period; the safety lower bound is used to constrain the coding rate of the next period of the corresponding period. Calculation module 23 is used to calculate the effective received bit rate for each cycle based on the negative correlation between the packet loss rate and the actual bandwidth of the network transport layer in each cycle; the effective received bit rate indicates the maximum bit rate that the network transport layer can effectively transmit in the current cycle. The encoding module 24 is used to determine the upper limit of the bit rate for each period based on the comparison between the lower limit of the security for each period and the effective received bit rate for each period, so as to encode the image to be compressed according to the upper limit of the bit rate for each period.

[0047] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0048] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a computer-readable storage medium.

[0049] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

Claims

1. A deep learning-based intelligent image compression method, characterized in that, The method includes: In multiple cycles, the fluctuation and autocorrelation of the historical bandwidth sequence corresponding to each cycle of the network transport layer are analyzed in a coordinated manner to obtain the predicted bandwidth for the next cycle of each cycle. In the plurality of cycles, the safety offset of each cycle is calculated based on the degree of deviation between the predicted bandwidth and the actual bandwidth of each cycle, and the safety lower bound of each cycle is determined based on the difference between the safety offset of each cycle and the predicted bandwidth of the next cycle; the safety lower bound is used to constrain the coding rate of the next cycle of the corresponding cycle. The effective received bit rate for each cycle is calculated based on the negative correlation between the packet loss rate and the actual bandwidth of the network transport layer in each cycle; the effective received bit rate indicates the maximum bit rate that the network transport layer can effectively transmit in the current cycle. Based on the comparison between the lower bound of the security for each cycle and the effective received bit rate for each cycle, the upper bound of the bit rate for each cycle is determined, and the image to be compressed is encoded according to the upper bound of the bit rate for each cycle. The method involves a collaborative analysis of the fluctuation and autocorrelation of the historical bandwidth sequence of the network transport layer in each of multiple periods to obtain the predicted bandwidth for the next period, including: In multiple cycles, the ratio of the standard deviation to the mean of all actual bandwidths in the historical bandwidth sequence corresponding to each cycle of the network transport layer is calculated to obtain the fluctuation coefficient of the next cycle of each cycle; multiple sequence points in the historical bandwidth sequence correspond one-to-one with multiple cycles, and multiple cycles include each cycle and multiple consecutive historical cycles before it. The autocorrelation decay distance for the next period is determined based on the number of lag steps corresponding to the first time the autocorrelation function value of the historical bandwidth sequence falls below the preset autocorrelation threshold. The fluctuation coefficient and autocorrelation decay distance of the next period are fused to obtain the prediction coefficient of the next period. The actual bandwidth of each period, the predicted bandwidth of each period, and the prediction coefficient of the next period are fused to obtain the predicted bandwidth of the next period. In the plurality of cycles, the safety offset for each cycle is calculated based on the degree of deviation between the predicted bandwidth and the actual bandwidth in each cycle, including: If the predicted bandwidth of each period is greater than the actual bandwidth in multiple periods, the difference between the actual bandwidth and the predicted bandwidth in each period is calculated to obtain the error magnitude of each period. Obtain the error magnitude of each period and multiple historical periods before it to obtain multiple error magnitudes, and sort the multiple error magnitudes from smallest to largest to obtain the set of historical prediction errors corresponding to each period; The target quantile for each cycle is determined based on the buffer occupancy rate at the receiving end in each cycle; the target quantile is inversely proportional to the buffer occupancy rate. Based on the historical prediction error set corresponding to each period and the target quantile for each period, the safety offset for each period is determined; the safety offset is used to offset the risk of prediction bandwidth.

2. The image intelligent compression method based on deep learning as described in claim 1, characterized in that, The process of fusing the fluctuation coefficient and the autocorrelation decay distance of the next period to obtain the prediction coefficient for the next period includes: The initial coefficient value for the next period is obtained by combining the product of the fluctuation coefficient of the next period with the preset first constant weight and the ratio of the preset second constant weight to the autocorrelation value; the autocorrelation value indicates the larger value between the preset value and the autocorrelation decay distance of the next period. The prediction coefficients for the next period are determined by comparing the initial coefficient values ​​with the preset values.

3. The image intelligent compression method based on deep learning as described in claim 1, characterized in that, The step of determining the safety offset for each period based on the historical prediction error set corresponding to each period and the target quantile for each period includes: From the historical prediction error set corresponding to each period, the error magnitude corresponding to the target quantile is extracted to obtain the safety offset for each period.

4. The image intelligent compression method based on deep learning as described in claim 1, characterized in that, The step of determining the lower bound of security for each period based on the difference between the security offset of each period and the predicted bandwidth of the next period includes: The difference between the predicted bandwidth for the next cycle and the safety offset for each cycle is calculated to obtain the safety lower bound for each cycle.

5. The image intelligent compression method based on deep learning as described in claim 1, characterized in that, The calculation of the effective received code rate for each cycle, based on the negative correlation between the packet loss rate and the actual bandwidth at the network transport layer in each cycle, includes: The effective received bit rate for each period is obtained by multiplying the actual bandwidth of each period by the packet loss difference of each period; the packet loss difference of each period indicates the difference between the preset value and the packet loss rate of each period.

6. The image intelligent compression method based on deep learning as described in claim 1, characterized in that, The step of determining the upper code rate safety limit for each cycle based on the comparison between the lower code rate safety limit and the effective received code rate for each cycle, and then encoding the image to be compressed according to the upper code rate safety limit for each cycle, includes: The lower bound of the security for each cycle is determined by the smaller of the lower bound of the security for each cycle and the effective received code rate for each cycle. The basic bitrate of the content is determined based on the image characteristics of the image to be compressed; A feasible bitrate range is constructed with the basic bitrate of the content as the lower limit and the safe upper limit of the bitrate for each period as the upper limit. The bitrate to be compressed is then selected within the feasible bitrate range for encoding.

7. A deep learning-based intelligent image compression system, characterized in that, The system includes: The analysis module is used to perform a collaborative analysis of the fluctuation and autocorrelation of the historical bandwidth sequence of the network transport layer in each cycle, and to obtain the predicted bandwidth for the next cycle. The determination module is used to calculate the safety offset of each period based on the degree of deviation between the predicted bandwidth and the actual bandwidth of each period, and to determine the safety lower bound of each period based on the difference between the safety offset of each period and the predicted bandwidth of the next period; the safety lower bound is used to constrain the coding rate of the next period of the corresponding period. The calculation module is used to calculate the effective received bit rate for each cycle based on the negative correlation between the packet loss rate and the actual bandwidth of the network transport layer in each cycle; the effective received bit rate indicates the maximum bit rate that the network transport layer can effectively transmit in the current cycle. The encoding module is used to determine the upper limit of the bit rate for each cycle based on the comparison between the lower limit of the security for each cycle and the effective received bit rate for each cycle, so as to encode the image to be compressed according to the upper limit of the bit rate for each cycle. In multiple cycles, a collaborative analysis is performed on the fluctuation and autocorrelation of the historical bandwidth sequence corresponding to each cycle of the network transport layer to obtain the predicted bandwidth for the next cycle, including: In multiple cycles, the ratio of the standard deviation to the mean of all actual bandwidths in the historical bandwidth sequence corresponding to each cycle of the network transport layer is calculated to obtain the fluctuation coefficient of the next cycle of each cycle; multiple sequence points in the historical bandwidth sequence correspond one-to-one with multiple cycles, and multiple cycles include each cycle and multiple consecutive historical cycles before it. The autocorrelation decay distance for the next period is determined based on the number of lag steps corresponding to the first time the autocorrelation function value of the historical bandwidth sequence falls below the preset autocorrelation threshold. The fluctuation coefficient and autocorrelation decay distance of the next period are fused to obtain the prediction coefficient of the next period. The actual bandwidth of each period, the predicted bandwidth of each period, and the prediction coefficient of the next period are fused to obtain the predicted bandwidth of the next period. In the plurality of cycles, based on the degree of deviation between the predicted bandwidth and the actual bandwidth in each cycle, a safety offset for each cycle is calculated, including: If the predicted bandwidth of each period is greater than the actual bandwidth in multiple periods, the difference between the actual bandwidth and the predicted bandwidth in each period is calculated to obtain the error magnitude of each period. Obtain the error magnitude of each period and multiple historical periods before it to obtain multiple error magnitudes, and sort the multiple error magnitudes from smallest to largest to obtain the set of historical prediction errors corresponding to each period; The target quantile for each cycle is determined based on the buffer occupancy rate at the receiving end in each cycle; the target quantile is inversely proportional to the buffer occupancy rate. Based on the historical prediction error set corresponding to each period and the target quantile for each period, the safety offset for each period is determined; the safety offset is used to offset the risk of prediction bandwidth.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images