Communication method and system for delay-free ultra-high-definition image transmission IP core software

Abstract

The present invention provides a communication method and system for delay-free ultra-high-definition image transmission IP core software, which relates to the field of image communication technology, including: analyzing image content, identifying static scenes and dynamic scenes; evaluating scene complexity, performing adaptive analysis of coding parameters, and obtaining coding strategies; connecting a display device of the IP core software and identifying display parameters; performing pattern matching adjustment and determining an adjustment coding strategy; optimizing the allocation of transmission network bandwidth with the goal of zero transmission delay according to the adjustment coding strategy, establishing a transmission channel, and including a transmission mandatory timestamp, which is used to control the data transmission time sequence and transmission time node. The present invention solves the technical problem that the prior art lacks effective bandwidth management and parameter optimization strategies for dynamic image content, and it is difficult to meet the requirements of delay-free transmission in different scenarios, resulting in poor real-time performance and efficiency of ultra-high-definition image transmission.

Description

Technical Field

[0001] The present invention relates to the field of image communication technology, and in particular to a communication method and system for delay-free ultra-high-definition image transmission IP core software. Background Art

[0002] Ultra-high-definition image transmission without delay is mainly used in the field of high-demand real-time video transmission, which has extremely high requirements for the real-time and clarity of the video and needs to ensure that the image can be transmitted in the shortest time with the highest quality. However, changes in the complexity of the image content will affect the encoding efficiency and video quality. The existing technology cannot adjust the encoding parameters in real time to adapt to these changes, resulting in fluctuations in image quality or low encoding efficiency; and different display devices have different parameters. The existing technology is difficult to adjust the encoding strategy in real time according to the characteristics of the specific display device, resulting in poor image quality or unsatisfactory display effects. Summary of the invention

[0003] This application provides a communication method and system for delay-free ultra-high-definition image transmission IP core software, aiming to solve the technical problem that the existing technology lacks effective bandwidth management and parameter optimization strategies for dynamic image content, making it difficult to meet the requirements of delay-free transmission in different scenarios, resulting in poor real-time performance and efficiency of ultra-high-definition image transmission.

[0004] The first aspect disclosed in the present application provides a communication method for delay-free ultra-high-definition image transmission IP core software, the method comprising: analyzing image content to identify static scenes and dynamic scenes; evaluating scene complexity according to the static scenes and dynamic scenes, and adaptively analyzing encoding parameters based on the scene complexity to obtain an encoding strategy; connecting a display device of the IP core software to identify display parameters; performing pattern matching adjustment on the encoding strategy according to the display parameters to determine an adjusted encoding strategy; according to the adjusted encoding strategy, optimizing the allocation of transmission network bandwidth with the goal of zero transmission delay, and establishing a transmission channel, the transmission channel comprising a transmission mandatory timestamp, and the transmission mandatory timestamp is used to control the data transmission time sequence and transmission time node.

[0005] The second aspect disclosed in the present application provides a communication system for delay-free ultra-high-definition image transmission IP core software, and the system is used for the above-mentioned communication method for delay-free ultra-high-definition image transmission IP core software, and the system includes: an image content analysis module, the image content analysis module is used to analyze image content and identify static scenes and dynamic scenes; a coding strategy acquisition module, the coding strategy acquisition module is used to evaluate scene complexity according to the static scenes and dynamic scenes, and perform adaptive analysis of coding parameters based on the scene complexity to obtain a coding strategy; a display parameter identification module, the display parameter identification module is used to connect a display device of the IP core software and identify display parameters; a pattern matching adjustment module, the pattern matching adjustment module is used to perform pattern matching adjustment on the coding strategy according to the display parameters to determine the adjusted coding strategy; a transmission channel establishment module, the transmission channel establishment module is used to optimize the allocation of transmission network bandwidth according to the adjusted coding strategy with the goal of zero transmission delay, and establish a transmission channel, the transmission channel includes a transmission forced timestamp, and the transmission forced timestamp is used to control the data transmission time sequence and transmission time node.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] By analyzing the image content, the static scenes and dynamic scenes in the image can be accurately identified. The inter-frame pixel change of the static scene is zero, and the inter-frame pixel change of the dynamic scene is non-zero. By identifying these scenes, the most appropriate encoding method can be formulated for each scene, laying the foundation for subsequent complexity evaluation and encoding strategy adjustment; based on the scene complexity evaluation of static scenes and dynamic scenes, the encoding difficulty of the scene can be quantified. This evaluation helps to decide how to adjust the encoding parameters to better adapt to different scene complexities. Dynamically adjusting the encoding parameters according to the scene complexity can ensure that the encoding strategy can achieve the best compression effect in different scenarios, which helps to balance image quality and bandwidth requirements and improve transmission efficiency and quality; the display device connected to the IP core software, By identifying the display parameters, the encoding strategy can be adjusted to meet the requirements of the display device, optimize the display effect, and avoid image distortion or quality degradation; by pattern matching the display parameters and the encoding strategy, it can be ensured that the encoding strategy adapts to the actual display conditions to the greatest extent, which enables the encoding strategy to effectively control bandwidth usage while ensuring image quality; according to the adjustment of the encoding strategy, the transmission network bandwidth is optimized and allocated, which can ensure that there is no delay in the transmission of image data, thereby realizing real-time transmission. By establishing a transmission channel including a mandatory timestamp, the time sequence and time nodes of data transmission are strictly controlled. Introducing mandatory timestamps during the transmission process ensures that the transmission sequence and time nodes of data packets are strictly controlled, thereby achieving a delay-free transmission effect.

[0008] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flow chart of a communication method for delay-free ultra-high-definition image transmission IP core software provided in an embodiment of the present application;

[0010] Figure 2 A schematic diagram of the communication system structure for the delay-free ultra-high-definition image transmission IP core software provided in an embodiment of the present application.

[0011] Explanation of the reference numerals: image content analysis module 10 , encoding strategy acquisition module 20 , display parameter identification module 30 , pattern matching adjustment module 40 , transmission channel establishment module 50 . DETAILED DESCRIPTION

[0012] The embodiments of the present application provide a communication method and system for delay-free ultra-high-definition image transmission IP core software, thereby solving the technical problem that the prior art lacks effective bandwidth management and parameter optimization strategies for dynamic image content, making it difficult to meet the requirements of delay-free transmission in different scenarios, resulting in poor real-time performance and efficiency of ultra-high-definition image transmission.

[0013] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically introduced below in conjunction with the accompanying drawings of the specification. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0014] Embodiment 1, as Figure 1 As shown, the embodiment of the present application provides a communication method for delay-free ultra-high-definition image transmission IP core software, the method comprising:

[0015] Analyze image content and identify static and dynamic scenes.

[0016] The entire image sequence is processed by frame division, that is, the continuous video stream or image sequence is divided into several independent frames, each frame represents a static image at a time point, and these independent frames are arranged in chronological order to form a frame sequence. For two adjacent frames, each pixel point is compared to identify the pixel changes between frames. If the pixel changes between two adjacent frames are large, it means that there is obvious movement or change in the scene, which is a dynamic scene; conversely, if the pixel changes between frames are small or there is no change, it means that the scene is relatively still and is a static scene.

[0017] A scene complexity evaluation is performed according to the static scene and the dynamic scene, and an encoding parameter adaptive analysis is performed based on the scene complexity to obtain an encoding strategy.

[0018] Based on the identified pixel changes between frames, the proportion of moving pixels in the image is determined. This proportion reflects how much of the image has undergone motion changes. The motion complexity coefficient is calculated based on the motion pixel proportion. This coefficient is used to quantify the dynamic complexity of the image. The edge features and texture features of each frame of the image are identified. The edge features refer to the boundaries of objects in the image, and the texture features refer to the surface details in the image. Based on the identified edge features and texture features, the picture complexity of each frame is evaluated and the picture complexity coefficient is calculated. The motion complexity coefficient and the picture complexity coefficient are combined to determine the complexity of the entire scene.

[0019] According to the complexity of the scene, the motion complexity coefficient and the picture complexity coefficient are analyzed. These coefficients are used as input variables of the adaptive model to guide the adjustment of the encoding parameters. A multi-layer adaptive model is used to analyze the encoding parameters. The model includes multiple analysis layers, including a quantization step analysis layer, a bit rate analysis layer, a frame rate analysis layer, a prediction mode selection layer, etc. These analysis layers are trained and tested with historical sample data. The model finally outputs a coding strategy, which is a combination of coding parameters that achieves optimal compression rate and minimum delay while ensuring image quality.

[0020] Connect the display device of the IP core software and identify the display parameters.

[0021] The display device connected to the IP core software is a device used for image display. These display devices can be of various types, including but not limited to monitors, TVs, projectors, etc. It automatically identifies the type of connected device and obtains the display parameters of the device, including resolution, refresh rate, color depth, bandwidth, etc. Among them, the resolution of the display device defines the number of pixels that the device can display, which affects the selection of encoding resolution. The encoding resolution cannot exceed the highest resolution of the display device, otherwise it will not display correctly; the refresh rate of the display device determines the number of times the screen is updated per second, which affects the frame rate selection in encoding. If the encoding frame rate exceeds the refresh rate of the device, it will cause frame rate mismatch and affect the viewing experience; the color depth of the display device determines the color accuracy that each pixel can display. Devices with high color depth need to be encoded to retain more color information, which will affect the selection of quantization step size and bit rate; the input bandwidth supported by the display device limits the bit rate that can be transmitted and decoded, affecting the setting of the bit rate during encoding.

[0022] If multiple display devices are connected at the same time, the independent parameters of each device are identified. By identifying the parameters of the display device, the display capabilities of the device can be taken into account during encoding to avoid image quality degradation or transmission failure due to the device not supporting certain high-end display features.

[0023] The encoding strategy is pattern-matched and adjusted according to the display parameters to determine the adjusted encoding strategy.

[0024] Based on the analysis results of scene complexity, basic constraints of the image are defined. These constraints reflect the minimum encoding parameters required under the scene complexity, such as the minimum bit rate, minimum frame rate and minimum quantization step size. These parameters ensure that the encoding strategy can meet the basic requirements of image clarity and smoothness even in the most complex scenes.

[0025] When only one display device is connected, the display parameters of the device, including resolution, refresh rate, and color depth, are used as the only display constraint; when multiple display devices are connected, the display parameters of each device are identified and the smallest parameter value, i.e., the least ideal parameter, is selected as the display constraint. This configuration ensures that when playing on multiple devices, the image quality differences between different devices are minimized.

[0026] Establish the configuration influence relationship between display parameters and encoding parameters, and determine how different display parameters affect the encoding strategy. For example, higher resolution requires higher bit rate and finer quantization step size.

[0027] Under the image constraints, based on the configuration influence relationship, the encoding strategy is adjusted with the goal of maximizing image clarity to obtain the first adjusted encoding strategy; based on the display constraints of the device, the encoding strategy is adjusted again. The goal of this adjustment is to ensure that the best image quality can be obtained on each display device, especially in a monitor or multi-monitor environment, to obtain the second adjusted encoding strategy. The two are integrated through the benefit balance process, and the strategy is adjusted comprehensively. The final generated adjusted encoding strategy balances clarity and transmission efficiency, ensuring a consistent and high-quality viewing experience on different display devices.

[0028] According to the adjustment coding strategy, the transmission network bandwidth is optimally allocated with the goal of zero transmission delay, and a transmission channel is established. The transmission channel includes a transmission mandatory timestamp, and the transmission mandatory timestamp is used to control the data transmission time sequence and transmission time node.

[0029] Comprehensively monitor the currently used transmission networks, collect the bandwidth conditions and transmission quality characteristics of each network, including packet loss rate, delay, jitter, etc. Based on the monitoring data, predict the transmission bandwidth and transmission quality label of each network. The transmission bandwidth reflects the data transmission rate that the network can support, and the transmission quality label is an assessment of the stability and reliability of network transmission.

[0030] According to the adjusted encoding strategy, the required bandwidth requirement is analyzed through the calculation formula of bandwidth requirement. The bandwidth requirement is directly related to encoding parameters such as quantization step size, frame rate, prediction mode, etc., ensuring that these parameters will not affect the image quality due to insufficient bandwidth during transmission.

[0031] Taking bandwidth demand as a constraint, it is matched with the predicted transmission bandwidth of each transmission network, and the network resources that best meet the demand are selected for data transmission. Based on the transmission quality label, the channel matching relationship is further determined, and the channel with the highest transmission quality and the lowest delay is selected for data transmission.

[0032] According to the channel matching relationship, the deviation nodes in the network are identified, that is, the nodes where delays or data loss may occur during the transmission process. In order to avoid the impact of these deviations on the overall transmission, a buffer area is inserted at the deviation node. The transmission path is compensated and configured through the buffer area to ensure the integrity of the data and the accuracy of the transmission order. The transmission path is constructed based on the compensation configuration of the deviation node.

[0033] After the transmission path is built, a clock synchronization mechanism is deployed at the data sending and receiving ends. Clock synchronization ensures the consistency of time nodes when data packets are transmitted between different devices, thereby avoiding data loss or misordering due to time differences. The time series relationship of data transmission is parsed according to the transmission path, and a mandatory timestamp is established. The mandatory timestamp is used to mark the sending and receiving time points of the data packet to ensure that the data is transmitted in the predetermined time sequence. Through comprehensive bandwidth optimization and timestamp control, an efficient and stable transmission channel is built, which can not only cope with highly complex image data transmission, but also ensure delay-free transmission under various network conditions.

[0034] Among them, the mandatory timestamp also has a bandwidth redundancy switching function. When the timestamp of a certain transmission channel does not meet the transmission time requirements, it will automatically switch to the redundant network channel to continue to complete the transmission of image data. This mechanism ensures that even when there is a problem with the main network channel, data transmission can still be carried out in a delay-free manner.

[0035] Furthermore, analyzing the image content and identifying static scenes and dynamic scenes includes:

[0036] The image content is divided into frames to establish a frame sequence; according to the continuous relationship of the frame sequence, the pixel changes between frames are identified; according to the pixel changes between frames, the static scene and the dynamic scene are identified, wherein the static scene is an image with zero pixel changes between frames, and the dynamic scene is an image with non-zero pixel changes between frames.

[0037] Extract continuous image frames from a video stream or image sequence, each frame represents image data at a time point, store the extracted frames in a frame sequence in chronological order, and establish a frame sequence. Select any pair of adjacent frames from the frame sequence, compare each pixel of the adjacent frames, calculate the change in pixel value, for example, use the absolute difference in pixel value to calculate the pixel change, and obtain the pixel change between frames.

[0038] If the pixel change between adjacent frames is zero, it means that the image content has not changed, and such images are marked as static scenes. Static scenes refer to image content that has not changed significantly on the timeline, such as a fixed landscape or a non-moving object. If the pixel change between adjacent frames is not zero, it means that the image content has changed, and such images are marked as dynamic scenes. Dynamic scenes refer to image content that has changed significantly on the timeline, such as character movement, camera switching, etc.

[0039] Furthermore, the scene complexity evaluation is performed according to the static scene and the dynamic scene, including:

[0040] According to the inter-frame pixel changes, the motion pixel ratio is determined to obtain a motion complexity coefficient; the edge features and texture features of each frame are identified; the picture complexity is evaluated according to the edge features and texture features to obtain a picture complexity coefficient; the motion complexity coefficient and the picture complexity coefficient are merged to determine the scene complexity.

[0041] Compare the pixel changes between consecutive frames. Specifically, detect the changes in consecutive frames one by one. For example, between frame A and frame B, if the position or color of a pixel changes, this pixel is marked as a moving pixel. If there is no change, it is marked as a static pixel. After identifying the moving pixels in each frame, calculate the total number of all moving pixels in the entire frame. This total number represents the intensity of motion in the frame. Compare the total number of moving pixels in each frame with the total number of pixels in the frame to calculate the moving pixel ratio. This ratio indicates how many pixels in the entire frame are in motion. According to the moving pixel ratio, calculate the motion complexity coefficient, which is proportional to the moving pixel ratio. The obtained motion complexity coefficient directly reflects the complexity of the dynamic scene. If the moving pixel ratio of a frame is high, it means that the frame contains a lot of motion information, and the motion complexity coefficient will be correspondingly high.

[0042] Apply edge detection algorithms to each frame of the image. For example, the Canny edge detection algorithm is used to accurately identify the edges in the image through steps such as smoothing, calculating gradients, non-maximum suppression, and double thresholding. The main purpose of edge detection is to find areas with strong brightness changes in the image, which usually represent the boundaries of objects.

[0043] Texture feature recognition is performed on each frame of the image. For example, the gray-level co-occurrence matrix method is used to quantify the texture's roughness, contrast, homogeneity and other properties by calculating the joint distribution of gray values ​​between pixels. Gabor filters are used to perform multi-scale and multi-directional frequency domain analysis on the image through a filter bank to capture texture features of different directions and frequencies. The identified texture features are quantified into a set of statistical values, such as mean, variance, contrast, homogeneity, etc., to describe the overall texture complexity of the image.

[0044] By calculating the ratio of edge pixels to total pixels in each frame, i.e. edge density, the edge complexity of the image can be evaluated. The higher the edge density, the more details and object boundaries the image contains, and the higher the complexity. By analyzing the statistical values ​​of texture features, the texture complexity of the image can be evaluated. Complex textures usually have high contrast and low homogeneity, which means that there are various details in the image. Combining edge complexity and texture complexity, a comprehensive picture complexity coefficient is calculated. This coefficient reflects the overall static complexity of the image and is a quantitative representation of various types of detail information in the image.

[0045] The motion complexity coefficient and the picture complexity coefficient are fused. For example, a weighted average method is used to assign different weights to the two coefficients according to the needs of the scene. The weights can be adjusted according to the focus in the actual application. For example, in a dynamic scene, the motion complexity coefficient is dominant, while in a static scene, the picture complexity coefficient has a greater weight. The scene complexity is obtained based on the fusion result. The scene complexity combines the dynamic and static complexity of the image. It is a comprehensive quantification of the overall characteristics of the scene, which can directly reflect the encoding difficulty of the scene and provide an important basis for the subsequent adaptive analysis of encoding parameters.

[0046] Furthermore, performing adaptive analysis of encoding parameters based on the scene complexity to obtain an encoding strategy includes:

[0047] The motion complexity coefficient and the picture complexity coefficient are parsed according to the scene complexity; the motion complexity coefficient and the picture complexity coefficient are used as input variables, encoding parameters are parsed through an adaptive model, and the encoding strategy is output; wherein the adaptive model includes a quantization step length parsing layer, a bit rate parsing layer, a frame rate parsing layer, a prediction mode selection layer, and a fusion connection layer, and the quantization step length parsing layer, the bit rate parsing layer, the frame rate parsing layer, the prediction mode selection layer, and the fusion connection layer are all obtained through historical sample data training and testing, and the historical sample data includes the motion complexity coefficient, the picture complexity coefficient and the corresponding quantization step length, bit rate, frame rate, and prediction mode selection parsing relationship, the quantization step length parsing layer is used for quantization step length adaptive parsing, the bit rate parsing layer is used for bit rate adaptive parsing, the frame rate parsing layer is used for frame rate adaptive parsing, the prediction mode selection layer is used for prediction mode adaptive selection, and the fusion connection layer is used to optimize and fuse the output results of the quantization step length parsing layer, the bit rate parsing layer, the frame rate parsing layer, and the prediction mode selection layer with the goal of maximizing complexity matching.

[0048] According to the complexity of the scene, the motion complexity coefficient is analyzed and the contribution of the motion complexity coefficient in the scene is identified. For example, in a dynamic scene, the level of the motion complexity coefficient directly affects the required bit rate and frame rate configuration. The motion complexity coefficient is analyzed in detail to identify the detailed features of the moving object such as speed, direction change, motion mode, etc., so as to determine the optimization direction of the encoding parameters; according to the complexity of the scene, the picture complexity coefficient is analyzed and attention is paid to the static features in the scene, such as edge clarity, texture complexity, detail richness, etc. These factors determine the degree of detail retention that needs to be paid attention to during the encoding process. The picture complexity coefficient is further analyzed to identify possible high-frequency or low-frequency details in the image, so as to guide the selection of quantization step size and compression ratio setting during encoding.

[0049] The architecture of the adaptive model includes multiple parsing layers, including the quantization step size parsing layer, the bit rate parsing layer, the frame rate parsing layer, the prediction mode selection layer, and the final fusion connection layer. Each layer focuses on specific coding parameters and makes corresponding adjustments based on the input complex coefficients. The adaptive model is trained with a large amount of historical data, which contains the experience of selecting coding parameters in scenarios of different complexity, such as frame rate, bit rate, quantization step size, etc. The model learns the relationship between each parameter and scene features through these data, so that it can quickly and accurately output optimized coding strategies in practical applications.

[0050] Specifically, based on the input complexity coefficient, the appropriate quantization step size is determined in the quantization step analysis layer. For scenes with high motion complexity, a smaller quantization step size is selected to maintain the clarity of details; for scenes with low complexity, a larger quantization step size is selected to compress the amount of data; in the bit rate analysis layer, the bit rate is dynamically adjusted according to the scene complexity. Scenes with high motion complexity coefficients require higher bit rates to ensure smooth motion performance; in static scenes, a lower bit rate may be sufficient; in the frame rate analysis layer, the frame rate is adjusted according to the motion complexity coefficient. For scenes with fast motion, a high frame rate helps to reduce motion blur and enhance the viewing experience; in scenes with low motion complexity, reducing the frame rate can save bandwidth; in the prediction mode selection layer, according to the picture complexity coefficient, it is decided whether to use a more complex prediction mode, such as bidirectional prediction, or a simpler mode. Complex prediction modes are usually suitable for scenes with high picture complexity and can compress data more effectively; finally, in the fusion connection layer, the output results of the quantization step analysis layer, the bit rate analysis layer, the frame rate analysis layer, and the prediction mode selection layer are optimized and fused with the goal of maximizing complexity matching, and the optimized encoding strategy is output.

[0051] Further, performing pattern matching adjustment on the encoding strategy according to the display parameter to determine the adjusted encoding strategy includes:

[0052] According to the scene complexity, determine the picture constraint condition, the picture constraint condition is the minimum encoding parameter of the scene complexity; perform display parameter constraint configuration on the display device to obtain the display constraint condition, wherein the display device includes one or more, when the display device is one, the display constraint condition is the display parameter of the display device, and when the display device is multiple, the display constraint condition is the minimum value of the display parameter among multiple devices; establish the configuration influence relationship between the display parameters and the encoding parameters; under the picture constraint condition, based on the configuration influence relationship, perform clarity maximum target matching adjustment on the encoding parameters in the encoding strategy according to the display parameters, to obtain a first adjusted encoding strategy; under the display constraint condition, based on the configuration influence relationship, perform clarity maximum target matching adjustment on the encoding parameters in the encoding strategy according to the display parameters, to obtain a second adjusted encoding strategy; perform interest balancing processing according to the first adjusted encoding strategy and the second adjusted encoding strategy to obtain the adjusted encoding strategy.

[0053] The image constraint condition is to ensure the basic encoding quality of the image in scenes of different complexity. This condition sets the minimum encoding parameter threshold of scene complexity. Even in the simplest scene, the image quality can be guaranteed not to be lower than this threshold, including setting the minimum quantization step to ensure the preservation of image details; setting the minimum bit rate to ensure sufficient bandwidth to transmit image data to avoid image quality degradation due to excessive compression; setting the minimum frame rate to ensure the smoothness of dynamic scenes. This helps to avoid the degradation of encoding quality due to low scene complexity, thereby ensuring the image effect during transmission.

[0054] The purpose of display parameter constraint configuration is to determine the impact of the display device on the encoding strategy, to ensure that the encoding strategy can adapt to the capabilities of the display device to achieve the best image display effect. The display parameter constraints are set according to the characteristics of the display device to ensure that the encoded image can obtain the best presentation effect on the display device. When there is only one display device, the display constraints come directly from the display parameters of the device; when there are multiple display devices, the display constraints are the minimum values ​​of the display parameters of all display devices. This is to ensure that all devices can adapt to the encoding strategy, to avoid the problem of poor display effect on some devices, to ensure the display effect of each device, and to balance the display of each device.

[0055] Establish the configuration influence relationship between display parameters and encoding parameters, where the resolution of the display device defines the number of pixels that the device can display, which affects the selection of encoding resolution. The encoding resolution cannot exceed the maximum resolution of the display device, otherwise it will not display correctly; the refresh rate of the display device determines the number of times the screen is updated per second, which affects the frame rate selection in encoding. If the encoding frame rate exceeds the refresh rate of the device, it will cause frame rate mismatch and affect the viewing experience; the color depth of the display device determines the color accuracy that each pixel can display. Devices with high color depth need to encode to retain more color information, which will affect the selection of quantization step and bit rate; the input bandwidth supported by the display device limits the bit rate that can be transmitted and decoded, affecting the setting of the bit rate during encoding.

[0056] The image constraint condition is to ensure that the image can meet the minimum requirements of image encoding on the display device under the given scene complexity, and the image clarity will not be affected by too low encoding settings. Based on the configuration impact relationship, the encoding parameters are adjusted to maximize the clarity of the image. Specifically, resolution matching is performed to ensure that the encoding resolution at least meets the resolution requirements of the display device. For example, if the display device is 4K resolution, then the encoding resolution should also be 4K; frame rate adjustment is performed to ensure that the encoding frame rate matches the refresh rate of the display device to avoid image freezes or jitters. For example, if the refresh rate of the display device is 120Hz, the encoding frame rate should also be set to 120fps; color depth optimization is performed to adjust the encoding color depth to match the color depth requirements of the display device. High color depth can provide richer color information. After adjustment, the first adjustment encoding strategy is obtained.

[0057] Display constraints are constraints that must be followed by encoding settings in the actual display device environment, ensuring that the encoding strategy matches the actual capabilities of the display device to avoid affecting image quality due to improper settings. According to the display constraints and configuration impact relationships, the encoding parameters are adjusted to maximize image clarity. Specifically, resolution matching is performed. If the resolution of the display device is lower than the encoding resolution, the encoding resolution needs to be lowered to adapt to the display device to ensure that the image is not scaled or distorted. Conversely, if the resolution of the display device is higher than the encoding resolution, the encoding resolution needs to be increased to ensure image details; frame rate optimization is performed, and the encoding frame rate is adjusted according to the refresh rate of the display device to ensure that the image is smooth and there is no stuttering. If the display device supports a high refresh rate, the encoding frame rate should also be set to a high value to match the capabilities of the device; color depth adjustment is performed, and the encoding color depth is set to match the color depth requirements of the display device to ensure color accuracy and richness. For high color depth display devices, the encoding color depth also needs to be increased accordingly. After adjustment, the second adjustment encoding strategy is obtained.

[0058] The first adjustment coding strategy and the second adjustment coding strategy are used to balance the interests, with the aim of finding the best balance between the clarity required by the image complexity and the actual transmission and display requirements of the display device to obtain the final adjustment coding strategy. This step ensures that the coding strategy can meet the high complexity requirements of the image and adapt to the performance limitations of the display device.

[0059] Specifically, the advantages of the first adjustment encoding strategy in terms of image clarity and the advantages of the second adjustment encoding strategy in terms of device adaptation are analyzed, and the conflict points between clarity and device performance are identified. For example, high resolution may require high bandwidth, but the bandwidth limitation of the device may not be able to support it. In the balancing process, the resolution, frame rate, color depth, etc. are optimized. The advantages of the first adjustment encoding strategy and the second adjustment encoding strategy are combined to ensure that the encoding settings can ensure image clarity while also adapting to the actual capabilities of the display device. The optimal balance point between clarity and device performance is found, the final encoding parameter settings are formulated, and the adjustment encoding strategy is obtained.

[0060] Further, according to the adjustment coding strategy, the transmission network bandwidth is optimally allocated with the goal of no transmission delay, and a transmission channel is established, wherein the transmission channel includes a transmission mandatory timestamp, including:

[0061] Monitor the bandwidth and transmission quality characteristics of each transmission network to obtain the predicted transmission bandwidth and transmission quality label of each transmission network; perform bandwidth transmission demand analysis according to the adjustment coding strategy to determine the bandwidth demand; perform bandwidth matching with the predicted transmission bandwidth based on the bandwidth demand, and determine the channel matching relationship based on the transmission quality label; determine the deviation node according to the channel matching relationship, insert the deviation node into the buffer area to perform transmission path compensation configuration, and build the transmission path; deploy a clock synchronization mechanism at the sending end and the receiving end, analyze the time series relationship according to the transmission path, establish a forced timestamp, and complete the construction of the transmission channel, wherein the forced timestamp fits the bandwidth redundancy switching mechanism, and when the forced timestamp does not meet the transmission time, the redundant network channel is switched to transmit the image data.

[0062] The bandwidth utilization and idle bandwidth of the transmission network are monitored through SNMP, traffic detection and other methods. Based on the current bandwidth utilization and idle bandwidth data, future bandwidth demand and available bandwidth are predicted to generate predicted transmission bandwidth.

[0063] Use RTT (Round-Trip Time) testing to monitor the round-trip delay in the network path; calculate the proportion of lost packets by sending test data packets and estimate transmission stability; monitor fluctuations in network delay and evaluate transmission stability. Combine the results of delay monitoring, packet loss rate detection, and jitter detection to form an overall view of network performance and generate transmission quality labels based on the level of network performance.

[0064] The bandwidth requirement is calculated using the bandwidth requirement calculation formula to obtain the bandwidth requirement.

[0065] Match the determined bandwidth demand with the predicted transmission bandwidth to ensure that the bandwidth demand is met, classify the network according to the transmission quality label, and select the most suitable transmission channel according to the bandwidth demand and transmission quality label. Specifically, if the predicted bandwidth and quality label show that the network quality is high, a high-bandwidth, low-latency channel can be selected; if the network quality is medium, a channel with moderate bandwidth can be selected, which may have some delays or packet loss; if the network quality is low, the bandwidth demand and tolerance need to be optimized, and redundant network channels may need to be used. By determining the best channel matching relationship, a balance between bandwidth demand and transmission quality can be achieved.

[0066] Determine the deviation nodes based on the channel matching relationship. Specifically, based on the monitored bandwidth data, find out the nodes whose bandwidth is lower than the demand. Use the delay monitoring data to identify the nodes with higher delay. Combine the bandwidth and delay data to determine the deviation nodes that affect the data transmission quality.

[0067] Inserting a cache area at the deviation node can temporarily store data packets to reduce data loss or delay caused by insufficient bandwidth or delay. On the other hand, it can help smooth the data flow and reduce the impact caused by insufficient instantaneous bandwidth or network jitter.

[0068] Configure the cache area. Specifically, configure the cache area of ​​appropriate size according to the delay and bandwidth requirements. The size of the cache area should be large enough to accommodate the data backlog caused by the delay; set the cache strategy, such as first-in-first-out or priority cache, to effectively manage the data flow. Based on the channel matching relationship and cache area configuration, optimize the data transmission path, build the transmission path, and ensure that the data can pass through the compensation node and reach the target smoothly.

[0069] A clock synchronization mechanism is deployed at the sending and receiving ends. The synchronization protocol can use the Network Time Protocol (NTP), which is used for clock synchronization in ordinary networks and is suitable for applications with lower precision; or the Precision Time Protocol (PTP), which is used for applications with high precision requirements and provides nanosecond-level synchronization. It is suitable for scenarios where ultra-high-definition image transmission requires a high-precision clock.

[0070] Analyze the data transmission time series through the transmission path to ensure that the data arrives within the specified time window, calculate the delay from sending to receiving, including network delay and cache delay, to determine the transmission time of the data packet, insert a mandatory timestamp in the data packet, record the actual transmission time and expected arrival time of the data packet, ensure that the timestamps of the data packet can be aligned at the sending and receiving ends to reduce time differences and delays, and complete the construction of the transmission channel.

[0071] Fitting bandwidth redundant switching mechanism, that is, setting one or more backup network channels. When problems occur in the main channel, switch to the backup channel for data transmission. Real-time monitoring of mandatory timestamps to ensure that data packets arrive within the specified time. When the timestamp of the main channel does not meet the transmission time requirements, it automatically switches to the redundant network channel to reduce the impact of transmission interruptions.

[0072] Furthermore, performing bandwidth transmission demand analysis according to the adjusted encoding strategy to determine the bandwidth demand includes:

[0073] Use the bandwidth demand calculation formula: Calculate and obtain, where is the bandwidth demand, in Mbps, is the influencing factor of the quantization step size, For frame rate, is the prediction model impact factor, is a coefficient less than 1, is the base bit rate, , and is the fitting parameter, is the quantization step size, is the base of natural logarithms.

[0074] Use the bandwidth demand calculation formula: Calculate the bandwidth requirement. In this formula, the bandwidth requirement The bandwidth required to ensure smooth transmission of video data; the base bit rate It is the benchmark value of video data transmission rate under standard or unchanged conditions; the influencing factor of quantization step size It is usually a nonlinear function, indicating that as the quantization step size QP increases, the bandwidth requirement decreases. The specific function form can be fitted through experimental data. , and For fitting parameters, obtained through experimental data, the quantization step size A parameter that controls image quality and data volume. Higher QP values ​​typically reduce image quality, thereby reducing data volume and bandwidth requirements. Frame rate The number of image frames displayed per second. The higher the frame rate, the larger the amount of data transmitted, because more image frames need to be transmitted per second. Prediction mode influencing factor is a coefficient less than 1. Inter-frame prediction (P frame, B frame) requires lower bandwidth than intra-frame prediction (I frame). This formula helps determine the actual bandwidth requirements to ensure that the transmission of video data can meet the requirements of no delay under a given quantization step size, frame rate and prediction mode.

[0075] In summary, the communication method for delay-free ultra-high-definition image transmission IP core software provided in the embodiment of the present application has the following technical effects:

[0076] By analyzing the image content, the static scenes and dynamic scenes in the image can be accurately identified. The inter-frame pixel change of the static scene is zero, and the inter-frame pixel change of the dynamic scene is non-zero. By identifying these scenes, the most appropriate encoding method can be formulated for each scene, laying the foundation for subsequent complexity evaluation and encoding strategy adjustment; based on the scene complexity evaluation of static scenes and dynamic scenes, the encoding difficulty of the scene can be quantified. This evaluation helps to decide how to adjust the encoding parameters to better adapt to different scene complexities. Dynamically adjusting the encoding parameters according to the scene complexity can ensure that the encoding strategy can achieve the best compression effect in different scenarios, which helps to balance image quality and bandwidth requirements and improve transmission efficiency and quality; the display device connected to the IP core software, By identifying the display parameters, the encoding strategy can be adjusted to meet the requirements of the display device, optimize the display effect, and avoid image distortion or quality degradation; by pattern matching the display parameters and the encoding strategy, it can be ensured that the encoding strategy adapts to the actual display conditions to the greatest extent, which enables the encoding strategy to effectively control bandwidth usage while ensuring image quality; according to the adjustment of the encoding strategy, the transmission network bandwidth is optimized and allocated, which can ensure that there is no delay in the transmission of image data, thereby realizing real-time transmission. By establishing a transmission channel including a mandatory timestamp, the time sequence and time nodes of data transmission are strictly controlled. Introducing mandatory timestamps during the transmission process ensures that the transmission sequence and time nodes of data packets are strictly controlled, thereby achieving a delay-free transmission effect.

[0077] Embodiment 2 is based on the same inventive concept as the communication method for delay-free ultra-high-definition image transmission IP core software in the aforementioned embodiment. Figure 2 As shown, the embodiment of the present application provides a communication system for delay-free ultra-high-definition image transmission IP core software, and the system includes:

[0078] An image content analysis module 10, the image content analysis module 10 is used to analyze image content and identify static scenes and dynamic scenes; a coding strategy acquisition module 20, the coding strategy acquisition module 20 is used to evaluate scene complexity according to the static scenes and dynamic scenes, and to perform adaptive analysis of coding parameters based on the scene complexity to obtain a coding strategy; a display parameter identification module 30, the display parameter identification module 30 is used to connect a display device of the IP core software and identify display parameters; a pattern matching adjustment module 40, the pattern matching adjustment module 40 is used to perform pattern matching adjustment on the coding strategy according to the display parameters and determine the adjusted coding strategy; a transmission channel establishment module 50, the transmission channel establishment module 50 is used to optimize the allocation of transmission network bandwidth according to the adjusted coding strategy with the goal of zero transmission delay, and to establish a transmission channel, wherein the transmission channel includes a transmission mandatory timestamp, and the transmission mandatory timestamp is used to control the data transmission time sequence and transmission time node.

[0079] Furthermore, the system further includes a static and dynamic scene recognition module to perform the following operation steps:

[0080] The image content is divided into frames to establish a frame sequence; according to the continuous relationship of the frame sequence, the pixel changes between frames are identified; according to the pixel changes between frames, the static scene and the dynamic scene are identified, wherein the static scene is an image with zero pixel changes between frames, and the dynamic scene is an image with non-zero pixel changes between frames.

[0081] Furthermore, the system further includes a scene complexity determination module to perform the following operation steps:

[0082] According to the inter-frame pixel changes, the motion pixel ratio is determined to obtain a motion complexity coefficient; the edge features and texture features of each frame are identified; the picture complexity is evaluated according to the edge features and texture features to obtain a picture complexity coefficient; the motion complexity coefficient and the picture complexity coefficient are merged to determine the scene complexity.

[0083] Furthermore, the system further includes a coding strategy output module to perform the following operation steps:

[0084] The motion complexity coefficient and the picture complexity coefficient are parsed according to the scene complexity; the motion complexity coefficient and the picture complexity coefficient are used as input variables, encoding parameters are parsed through an adaptive model, and the encoding strategy is output; wherein the adaptive model includes a quantization step length parsing layer, a bit rate parsing layer, a frame rate parsing layer, a prediction mode selection layer, and a fusion connection layer, and the quantization step length parsing layer, the bit rate parsing layer, the frame rate parsing layer, the prediction mode selection layer, and the fusion connection layer are all obtained through historical sample data training and testing, and the historical sample data includes the motion complexity coefficient, the picture complexity coefficient and the corresponding quantization step length, bit rate, frame rate, and prediction mode selection parsing relationship, the quantization step length parsing layer is used for quantization step length adaptive parsing, the bit rate parsing layer is used for bit rate adaptive parsing, the frame rate parsing layer is used for frame rate adaptive parsing, the prediction mode selection layer is used for prediction mode adaptive selection, and the fusion connection layer is used to optimize and fuse the output results of the quantization step length parsing layer, the bit rate parsing layer, the frame rate parsing layer, and the prediction mode selection layer with the goal of maximizing complexity matching.

[0085] Furthermore, the system further includes an adjustment coding strategy acquisition module to perform the following operation steps:

[0086] According to the scene complexity, determine the picture constraint condition, the picture constraint condition is the minimum encoding parameter of the scene complexity; perform display parameter constraint configuration on the display device to obtain the display constraint condition, wherein the display device includes one or more, when the display device is one, the display constraint condition is the display parameter of the display device, and when the display device is multiple, the display constraint condition is the minimum value of the display parameter among multiple devices; establish the configuration influence relationship between the display parameters and the encoding parameters; under the picture constraint condition, based on the configuration influence relationship, perform clarity maximum target matching adjustment on the encoding parameters in the encoding strategy according to the display parameters, to obtain a first adjusted encoding strategy; under the display constraint condition, based on the configuration influence relationship, perform clarity maximum target matching adjustment on the encoding parameters in the encoding strategy according to the display parameters, to obtain a second adjusted encoding strategy; perform interest balancing processing according to the first adjusted encoding strategy and the second adjusted encoding strategy to obtain the adjusted encoding strategy.

[0087] Furthermore, the system further includes an image data transmission module to perform the following operation steps:

[0088] Monitor the bandwidth and transmission quality characteristics of each transmission network to obtain the predicted transmission bandwidth and transmission quality label of each transmission network; perform bandwidth transmission demand analysis according to the adjustment coding strategy to determine the bandwidth demand; perform bandwidth matching with the predicted transmission bandwidth based on the bandwidth demand, and determine the channel matching relationship based on the transmission quality label; determine the deviation node according to the channel matching relationship, insert the deviation node into the buffer area to perform transmission path compensation configuration, and build the transmission path; deploy a clock synchronization mechanism at the sending end and the receiving end, analyze the time series relationship according to the transmission path, establish a forced timestamp, and complete the construction of the transmission channel, wherein the forced timestamp fits the bandwidth redundancy switching mechanism, and when the forced timestamp does not meet the transmission time, the redundant network channel is switched to transmit the image data.

[0089] Furthermore, the system further includes a module for determining bandwidth demand to perform the following operation steps:

[0090] Use the bandwidth demand calculation formula: Calculate and obtain, where is the bandwidth demand, in Mbps, is the influencing factor of the quantization step size, For frame rate, is the prediction model impact factor, is a coefficient less than 1, is the base bit rate, , and is the fitting parameter, is the quantization step size, is the base of natural logarithms.

[0091] Through the aforementioned detailed description of the communication method for the delay-free ultra-high-definition image transmission IP core software in this specification, technical personnel in this field can clearly understand the communication system for the delay-free ultra-high-definition image transmission IP core software in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0092] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A communication method for non-delay ultra-high-definition image transmission IP core software, characterized in that: The method comprises: Analyze image content and identify static and dynamic scenes; Performing scene complexity evaluation according to the static scene and the dynamic scene, and performing adaptive analysis of encoding parameters based on the scene complexity to obtain an encoding strategy; Connect the display device of the IP core software and identify the display parameters; Performing pattern matching adjustment on the encoding strategy according to the display parameters to determine an adjusted encoding strategy; According to the adjustment coding strategy, the transmission network bandwidth is optimally allocated with the goal of zero transmission delay, and a transmission channel is established, wherein the transmission channel includes a transmission mandatory timestamp, and the transmission mandatory timestamp is used to control the data transmission time sequence and the transmission time node; The step of performing pattern matching adjustment on the encoding strategy according to the display parameters to determine the adjusted encoding strategy includes: Determining a picture constraint condition according to the scene complexity, wherein the picture constraint condition is a minimum encoding parameter of the scene complexity; Performing display parameter constraint configuration on the display device to obtain display constraint conditions, wherein the display device includes one or more display devices, when there is one display device, the display constraint conditions are display parameters of the display device, and when there are multiple display devices, the display constraint conditions are minimum values ​​of display parameters among the multiple devices; Establishing a configuration influence relationship between the display parameters and the encoding parameters; Under the picture constraint condition, based on the configuration influence relationship, and according to the display parameters, the encoding parameters in the encoding strategy are adjusted for maximum definition target matching to obtain a first adjusted encoding strategy; Under the display constraint condition, based on the configuration influence relationship, and according to the display parameters, the encoding parameters in the encoding strategy are adjusted for maximum definition target matching to obtain a second adjusted encoding strategy; The interest balancing process is performed according to the first adjustment coding strategy and the second adjustment coding strategy to obtain the adjustment coding strategy.

2. The communication method for non-delay ultra-high-definition image transmission IP core software according to claim 1, characterized in that: The analyzing of image content and identifying static scenes and dynamic scenes includes: Divide the image content into frames and establish a frame sequence; According to the continuous relationship of the frame sequence, identify the pixel changes between frames; According to the inter-frame pixel change, the static scene and the dynamic scene are identified, wherein the static scene is an image with zero inter-frame pixel change, and the dynamic scene is an image with non-zero inter-frame pixel change.

3. The communication method for non-delay ultra-high-definition image transmission IP core software according to claim 2, characterized in that: The scene complexity evaluation is performed according to the static scene and the dynamic scene, including: According to the pixel change between frames, the motion pixel ratio is determined to obtain a motion complexity coefficient; Identify the edge features and texture features of each frame of image; Evaluate the picture complexity according to the edge features and texture features to obtain a picture complexity coefficient; The motion complexity coefficient and the picture complexity coefficient are integrated to determine the scene complexity.

4. The communication method for non-delay ultra-high-definition image transmission IP core software according to claim 3, characterized in that: Performing adaptive analysis of encoding parameters based on the scene complexity to obtain an encoding strategy includes: Analyze the motion complexity coefficient and the picture complexity coefficient according to the scene complexity; Taking the motion complexity coefficient and the picture complexity coefficient as input variables, performing encoding parameter analysis through an adaptive model, and outputting the encoding strategy; Among them, the adaptive model includes a quantization step length analysis layer, a bit rate analysis layer, a frame rate analysis layer, a prediction mode selection layer, and a fusion connection layer. The quantization step length analysis layer, the bit rate analysis layer, the frame rate analysis layer, the prediction mode selection layer, and the fusion connection layer are all obtained through training and testing of historical sample data. The historical sample data includes the motion complexity coefficient, the picture complexity coefficient and the corresponding quantization step length, bit rate, frame rate, and prediction mode selection analysis relationship. The quantization step length analysis layer is used for quantization step length adaptive analysis, the bit rate analysis layer is used for bit rate adaptive analysis, the frame rate analysis layer is used for frame rate adaptive analysis, the prediction mode selection layer is used for prediction mode adaptive selection, and the fusion connection layer is used to optimize and fuse the output results of the quantization step length analysis layer, the bit rate analysis layer, the frame rate analysis layer, and the prediction mode selection layer with the goal of maximizing complexity matching.

5. The communication method for non-delay ultra-high-definition image transmission IP core software according to claim 1, characterized in that: According to the adjustment coding strategy, the transmission network bandwidth is optimally allocated with the goal of zero transmission delay, and a transmission channel is established, wherein the transmission channel includes a transmission mandatory timestamp, including: Monitor the bandwidth and transmission quality characteristics of each transmission network, and obtain the predicted transmission bandwidth and transmission quality label of each transmission network; Analyze the bandwidth transmission demand according to the adjustment coding strategy to determine the bandwidth demand; Performing bandwidth matching with the predicted transmission bandwidth based on the bandwidth demand as a constraint, and determining a channel matching relationship based on the transmission quality label; Determine the deviation node according to the channel matching relationship, insert the deviation node into the buffer area to perform transmission path compensation configuration, and construct the transmission path; A clock synchronization mechanism is deployed at the sending end and the receiving end, and the time series relationship is parsed according to the transmission path to establish a mandatory timestamp to complete the construction of the transmission channel, wherein the mandatory timestamp fits the bandwidth redundancy switching mechanism, and when the mandatory timestamp does not meet the transmission time, the redundant network channel is switched to transmit the image data.

6. The communication method for non-delay ultra-high-definition image transmission IP core software according to claim 5, characterized in that: The bandwidth transmission requirement is analyzed according to the adjusted encoding strategy to determine the bandwidth requirement, including: Use the bandwidth demand calculation formula: Calculate and obtain, where is the bandwidth demand, in Mbps, is the influencing factor of the quantization step size, For frame rate, is the prediction model impact factor, is a coefficient less than 1, is the base bit rate, , and is the fitting parameter, is the quantization step size, is the base of natural logarithms.

7. A communication system for non-delay ultra-high-definition image transmission IP core software, characterized in that: A communication method for implementing the IP core software for delay-free ultra-high-definition image transmission according to any one of claims 1 to 6, the system comprising: An image content analysis module, which is used to analyze image content and identify static scenes and dynamic scenes; A coding strategy acquisition module, the coding strategy acquisition module is used to evaluate the scene complexity according to the static scene and the dynamic scene, perform adaptive analysis of coding parameters based on the scene complexity, and obtain a coding strategy; A display parameter identification module, which is used to connect to the display device of the IP core software and identify display parameters; A pattern matching adjustment module, the pattern matching adjustment module is used to perform pattern matching adjustment on the encoding strategy according to the display parameters to determine the adjusted encoding strategy; A transmission channel establishment module, wherein the transmission channel establishment module is used to optimize the allocation of transmission network bandwidth according to the adjustment coding strategy with the goal of zero transmission delay, and establish a transmission channel, wherein the transmission channel includes a transmission mandatory timestamp, and the transmission mandatory timestamp is used to control the data transmission time sequence and transmission time node.

Images