Digital image transmission system based on ultra-low time delay

Abstract

The invention belongs to the technical field of digital image transmission, and particularly relates to a digital image transmission system based on ultra-low time delay, which comprises the following steps: firstly, acquiring digital image data in a scene in real time by using a high-frame-rate and low-noise image sensor, and transmitting the acquired original image data; then, carrying out preprocessing operation on the collected original image data; then, a deep neural network model is utilized to adaptively adjust coding parameters and a coding mode according to the content characteristics of the image and the real-time transmission bandwidth condition; on the basis, the adaptive lightweight coding module is arranged, a complex inter-frame prediction algorithm in a traditional coding standard is abandoned, a block-level differential coding mode is adopted, and coding parameters and modes are adaptively adjusted according to image content characteristics and real-time transmission bandwidth; the method has the advantages that the coding delay is effectively reduced, and the high-real-time scene coding requirement is met.

Description

Technical Field

[0001] This invention relates to the field of digital image transmission technology, specifically to a digital image transmission system based on ultra-low latency. Background Technology

[0002] With the rapid development of digital technology, digital image transmission is playing an increasingly important role in many fields. In telemedicine, doctors need to use real-time transmitted high-definition images to remotely diagnose patients and guide surgical procedures; the latency of image transmission directly affects the accuracy of diagnosis and the safety of surgery. In autonomous driving, vehicles need to receive real-time image information of their surroundings to make rapid driving decisions and avoid traffic accidents. In industrial real-time monitoring, enterprises need to monitor the operating status of production equipment and production processes through real-time images to promptly detect and handle abnormalities, ensuring smooth production.

[0003] Currently, digital image transmission technology is constantly developing, but in scenarios with stringent latency requirements, existing technologies still struggle to meet practical needs. To achieve efficient image transmission, images are typically compressed and encoded, and corresponding transmission protocols and frame processing methods are employed. However, these steps often suffer from high latency, hindering the application of digital image transmission in high real-time scenarios. Specifically, the following problems exist:

[0004] 1. Excessive Encoding Latency: Existing digital image transmission systems mostly use traditional encoding standards such as H.264 and H.265. These standards employ complex algorithms such as inter-frame prediction, transform coding, and entropy coding to achieve high compression rates. The encoding process requires complex calculations on a large amount of image data, resulting in long encoding latency. For example, in remote surgery scenarios, when using the H.265 encoding standard to encode images of the surgical area, a single encoding latency can reach 50-80ms. When doctors perform surgical operations based on the transmitted images, the encoding latency causes significant operational delays, potentially leading to surgical errors and endangering the patient's life.

[0005] 2. Poor Transmission Protocol Adaptability and High Latency: Currently, the commonly used image transmission protocols are mainly TCP and UDP. TCP is characterized by reliable transmission, but its three-way handshake and retransmission mechanism can lead to significant transmission delays when network congestion or packet loss occurs. For example, in autonomous driving scenarios, when a vehicle is driving in an area with unstable network signals, if a 10% packet loss rate occurs when transmitting images of the road ahead using TCP, the transmission delay will increase from the normal 20-30ms to 100-150ms. The vehicle may not be able to obtain road information in time, potentially missing the optimal opportunity to avoid obstacles and causing traffic accidents. While UDP offers fast transmission speeds, it lacks effective packet loss retransmission and flow control mechanisms. In poor network environments, the reliability of image transmission cannot be guaranteed, easily leading to image distortion, stuttering, and other problems, similarly failing to meet the demands of high real-time scenarios.

[0006] 3. Inappropriate frame processing methods lead to increased latency: Most existing systems use a method of receiving, processing, and then transmitting entire frames during image frame processing. When the image resolution is high, the data volume of a single frame is large, and receiving and processing the entire frame takes considerable time, thus increasing the overall latency of image transmission. For example, in industrial real-time monitoring, when transmitting 4K resolution images of production equipment, using the whole-frame processing method, receiving and processing just one frame takes 30-40ms. Adding subsequent encoding and transmission delays, the total latency exceeds 100ms. Monitoring personnel may be unable to detect abnormalities in the production equipment in a timely manner, potentially leading to escalating production failures and significant economic losses. Summary of the Invention

[0007] To address the aforementioned technical problems of excessive encoding latency, poor compatibility with transmission protocols, and increased latency due to unreasonable frame processing methods, this invention provides the following technical solution:

[0008] A digital image transmission method based on ultra-low latency includes the following specific steps:

[0009] S1: Real-time acquisition of digital image data in the scene using a high frame rate, low noise image sensor, and transmission of the acquired raw image data;

[0010] S2: Perform preprocessing operations on the acquired raw image data;

[0011] S3: Utilizes a deep neural network model to adaptively adjust encoding parameters and encoding methods based on the content features of the image and the real-time transmission bandwidth.

[0012] S4: Automatically switches between TCP, UDP, and custom lightweight protocols based on network conditions; simultaneously monitors bandwidth changes in real time and provides feedback, adjusting the encoding bitrate accordingly.

[0013] S5: Based on the pre-stored scene-based region priority rules and combined with region of interest markers, priority is assigned to the encoded image data;

[0014] S6: Use the corresponding decoding algorithm to quickly decode the received encoded image data;

[0015] S7: Under the premise of ultra-low latency, repair the image quality loss caused by simplified coding; first, a lightweight layered repair model is adopted to quickly detect the distortion area and level of the decoded image; then, pixel interpolation, local feature mapping and neighborhood information completion repair methods are used to repair mild, moderate and severe distortion respectively.

[0016] S8: Utilizes a high refresh rate display device to display the decoded image data in real time.

[0017] As a preferred embodiment of the ultra-low latency digital image transmission method described in this invention, the specific steps of step S2 are as follows:

[0018] S21: Based on actual application requirements, remove irrelevant areas from the image and retain only the image data of the key areas;

[0019] S22: An adaptive median filtering algorithm is used to remove noise from the image while avoiding excessive blurring of image details;

[0020] S23: By using edge detection and region growing algorithms, the region of interest in the image is automatically identified and extracted, reducing the amount of data for subsequent encoding and transmission.

[0021] As a preferred embodiment of the ultra-low latency digital image transmission method described in this invention, the specific steps of S4 are as follows:

[0022] S41: Utilizes built-in TCP, UDP, and custom lightweight transport protocols to automatically select the optimal transport protocol based on the current network conditions.

[0023] S42: Monitors network bandwidth changes in real time and provides feedback on bandwidth information to adjust the encoding bitrate based on the bandwidth information, ensuring that the encoded image data can be transmitted stably under the current network bandwidth, and avoiding image stuttering and increased latency due to insufficient bandwidth.

[0024] As a preferred embodiment of the ultra-low latency digital image transmission method described in this invention, the specific steps of S5 are as follows:

[0025] S51 utilizes a built-in region priority mapping library to pre-store image region priority rules for different application scenarios, while simultaneously receiving transmitted region of interest (ROI) marker information.

[0026] S52, when receiving transmitted encoded image data, first parses the region priority label in the data, and then dynamically adjusts the transmission slice order based on the current remaining network bandwidth.

[0027] S53 synchronously sends priority scheduling instructions to enable the decoding of high-priority area data, ensuring that critical image information is displayed before non-critical information.

[0028] As a preferred embodiment of the ultra-low latency digital image transmission method described in this invention, the specific steps of S7 are as follows:

[0029] S71: Based on the MobileNetV2 lightweight network architecture, pre-train lightweight hierarchical restoration model for image distortion restoration parameters in different scenarios;

[0030] S72: During operation, the output image data is subjected to distortion detection to identify blurred and color block distortion areas in the image caused by simplified encoding through a pixel gradient difference algorithm, and the distortion level is marked, including mild, moderate and severe.

[0031] S73: The repair intensity is dynamically adjusted according to the distortion level; pixel interpolation is used to repair slightly distorted areas, only optimizing edge smoothness; local feature mapping is used to repair moderately distorted areas, restoring key details; and neighborhood information is used to complete and repair severely distorted areas, generating reasonable content based on surrounding pixels.

[0032] A digital image transmission system based on ultra-low latency includes:

[0033] The image acquisition module is used to acquire digital image data of the scene in real time using a high frame rate, low noise image sensor, and to transmit the acquired raw image data.

[0034] An edge preprocessing module is set at the edge nodes near the image acquisition module to perform preprocessing operations on the acquired raw image data;

[0035] The adaptive lightweight encoding module is used to adaptively adjust the encoding parameters and encoding method based on the content features of the image and the real-time transmission bandwidth using a deep neural network model.

[0036] The intelligent transmission module automatically switches between TCP, UDP, and custom lightweight protocols based on network conditions; it also monitors bandwidth changes in real time and feeds them back to the adaptive lightweight encoding module to adjust the encoding bitrate accordingly.

[0037] The dynamic priority scheduling module is used to prioritize coded image data based on pre-stored scene-based region priority rules and the region of interest markers from the edge preprocessing module.

[0038] Fast decoding module: used to quickly decode the received encoded image data using the corresponding decoding algorithm;

[0039] The adaptive quality restoration module is used to restore image quality loss caused by simplified encoding under ultra-low latency conditions. First, a lightweight layered restoration model is used to quickly detect the distortion areas and levels of the decoded image. Then, pixel interpolation, local feature mapping, and neighborhood information completion restoration methods are used to restore mild, moderate, and severe distortions, respectively.

[0040] The image display module is used to display the decoded image data in real time using a high refresh rate display device.

[0041] As a preferred embodiment of the ultra-low latency digital image transmission system described in this invention, the edge preprocessing module includes:

[0042] The image cropping unit is used to remove irrelevant areas from the image and retain only the image data of the key areas according to the actual application requirements.

[0043] The noise filtering unit is used to remove noise from the image using an adaptive median filtering algorithm, while avoiding excessive blurring of image details;

[0044] The Region of Interest (ROI) extraction unit is used to automatically identify and extract the region of interest in an image through edge detection and region growing algorithms, thereby reducing the amount of data required for subsequent encoding and transmission.

[0045] As a preferred embodiment of the ultra-low latency digital image transmission system described in this invention, the intelligent transmission module includes:

[0046] The transport protocol selection unit is used to automatically select the optimal transport protocol based on the current network status, utilizing the built-in TCP protocol, UDP protocol, and custom lightweight transport protocols.

[0047] The dynamic bandwidth adaptation unit is used to monitor network bandwidth changes in real time and feed the bandwidth information back to the adaptive lightweight coding module. This allows the adaptive lightweight coding module to adjust the coding rate according to the bandwidth information, ensuring that the encoded image data can be transmitted stably under the current network bandwidth and avoiding image stuttering and increased latency due to insufficient bandwidth.

[0048] As a preferred embodiment of the ultra-low latency digital image transmission system described in this invention, the dynamic priority scheduling module includes:

[0049] The pre-storage unit is used to pre-store image region priority rules for different application scenarios using the built-in region priority mapping library, and at the same time receive region of interest marking information transmitted by the edge preprocessing module.

[0050] The priority parsing unit is used to first parse the region priority label in the data when receiving the encoded image data transmitted by the intelligent transmission module, and then dynamically adjust the transmission slice order in combination with the current remaining network bandwidth.

[0051] The scheduling instruction sending unit is used to synchronously send priority scheduling instructions so as to decode high-priority area data first and ensure that critical image information is displayed before non-critical information.

[0052] As a preferred embodiment of the ultra-low latency digital image transmission system described in this invention, the adaptive quality restoration module includes:

[0053] The model training unit is used to pre-train the lightweight hierarchical restoration model for image distortion restoration parameters in different scenarios based on the MobileNetV2 lightweight network architecture.

[0054] The distortion detection unit is used to perform distortion detection on the image data output by the fast decoding module during operation. It identifies blurred and color block distortion areas in the image caused by simplified encoding through a pixel gradient difference algorithm and marks the distortion level, which includes mild, moderate and severe.

[0055] The repair unit is used to dynamically adjust the repair intensity according to the distortion level; pixel interpolation is used to repair slightly distorted areas, only optimizing edge smoothness; local feature mapping is used to repair moderately distorted areas, restoring key details; and neighborhood information is used to complete and repair severely distorted areas, generating reasonable content based on surrounding pixels.

[0056] Compared with existing technologies:

[0057] By setting up an edge preprocessing module close to the image acquisition module, image cropping, noise filtering, and region of interest extraction are performed on the original image data. This eliminates the need for the traditional method of receiving and processing the entire frame before transmission. Simultaneously, an adaptive lightweight coding module is used, employing block-level differential coding. The coding parameters and methods are adaptively adjusted based on image content features and transmission bandwidth. Furthermore, an intelligent transmission module containing a transmission protocol selection unit and a dynamic bandwidth adaptation unit automatically selects the optimal transmission protocol based on network conditions and adjusts the coding rate in real time based on bandwidth information. This approach achieves reduced latency in frame processing, effective control of coding latency, and improved transmission protocol adaptability, while ensuring transmission reliability and the integrity of critical image information. It comprehensively solves the latency problems in encoding, transmission, and frame processing in existing technologies, meeting the core requirements of digital image transmission in high real-time scenarios. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the overall framework of the present invention;

[0059] Figure 2 This is a schematic diagram of the edge preprocessing module framework of the present invention;

[0060] Figure 3 This is a schematic diagram of the intelligent transmission module framework of the present invention;

[0061] Figure 4 This is a schematic diagram of the dynamic priority scheduling module framework of the present invention;

[0062] Figure 5 This is a schematic diagram of the adaptive quality repair module framework of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0064] This invention provides a digital image transmission system based on ultra-low latency, such as... Figure 1 As shown, it includes:

[0065] The image acquisition module is used to acquire digital image data of the scene in real time using a high frame rate, low noise image sensor, and transmit the acquired raw image data; the frame rate of the image sensor can be adjusted according to the actual application scenario requirements, with a maximum frame rate of 240fps, ensuring that it can capture rapidly changing information in the scene;

[0066] An edge preprocessing module is set at the edge nodes near the image acquisition module to perform preprocessing operations on the acquired raw image data;

[0067] The adaptive lightweight encoding module utilizes a deep neural network model (trained on a large amount of image data) to adaptively adjust encoding parameters and methods based on image content features and real-time transmission bandwidth. Specifically, for regions with simple textures and slow changes, a simplified encoding algorithm is used to reduce computational load; for regions with complex textures and rapid changes, an optimized encoding algorithm is employed to minimize encoding latency while maintaining image quality. Furthermore, this module abandons the complex inter-frame prediction algorithms of traditional encoding standards, employing block-level differential coding to encode only the differences between adjacent image blocks, further reducing the amount of encoded data and encoding latency. The encoded image data is then transmitted to the intelligent transmission module.

[0068] The intelligent transmission module automatically switches between TCP, UDP, and custom lightweight protocols based on network conditions; it also monitors bandwidth changes in real time and feeds them back to the adaptive lightweight encoding module to adjust the encoding bitrate accordingly.

[0069] The dynamic priority scheduling module is used to prioritize coded image data based on pre-stored scene-based region priority rules and the region of interest markers from the edge preprocessing module.

[0070] Fast decoding module: used to quickly decode the received encoded image data using the corresponding decoding algorithm; since the adaptive lightweight encoding module adopts a simplified encoding algorithm and block-level differential encoding method, the decoding process of the fast decoding module is also simplified accordingly, without the need for complex calculations, which greatly reduces the decoding latency;

[0071] The adaptive quality restoration module is used to restore image quality loss caused by simplified encoding under ultra-low latency conditions. First, a lightweight layered restoration model is used to quickly detect the distortion areas and levels of the decoded image. Then, pixel interpolation, local feature mapping, and neighborhood information completion restoration methods are used to restore mild, moderate, and severe distortions, respectively.

[0072] The image display module is used to display the decoded image data in real time using a high refresh rate display device. The refresh rate of this display device can reach up to 240Hz, which can match the frame rate of the image acquisition module, avoiding image stuttering and delay caused by insufficient refresh rate of the display device, and ensuring that users can observe the image information in the scene in real time.

[0073] like Figure 2 As shown, the edge preprocessing module includes:

[0074] The image cropping unit is used to remove irrelevant areas from the image and retain only the image data of the key areas according to the actual application requirements.

[0075] Image cropping is primarily based on "scenario-based requirement definition + key area algorithm localization + device parameter collaborative adaptation," with the specific logic as follows:

[0076] Scenario-based requirements are defined to provide target-oriented tailoring: This unit serves latency-sensitive scenarios such as telemedicine, autonomous driving, and industrial real-time monitoring. The core areas of concern differ significantly across scenarios, which is the primary basis for tailoring. For example, in telemedicine, the focus is on the surgical operation area (such as incisions and instrument interactions), while irrelevant areas (such as the surrounding environment of the operating table) need to be tailored. In autonomous driving, the focus is on the road ahead (lane lines, obstacles, pedestrians), while irrelevant areas such as the vehicle's interior and distant views need to be removed. In industrial real-time monitoring, the focus is on core components of production equipment (such as gears and dashboards), while open areas of the factory and areas not monitored by the equipment need to be discarded. By defining the "critical areas of concern" in advance for each application scenario, the target boundaries of tailoring are clearly defined, avoiding the waste of subsequent processing resources by invalid data.

[0077] Key region localization algorithms provide technical support for cropping: Based on scenario-specific requirements, the module combines edge detection + region growing algorithms (in conjunction with subsequent region of interest extraction algorithms) to automatically identify and locate key regions in the image, providing precise technical basis for cropping. First, edge detection algorithms (such as the Canny algorithm) extract the contours of key objects in the image (such as the edges of surgical instruments or equipment components), initially determining the approximate range of the key region. Then, using a region growing algorithm, starting from seed pixels within the contour, it gradually expands to the complete key region, accurately dividing the "key region to be retained" and the "irrelevant region to be cropped." For example, in industrial monitoring, the algorithm can automatically locate the contour of a gear in a 4K image and grow it into a complete key region. Subsequent cropping only around this region removes more than 80% of the surrounding irrelevant pixel data.

[0078] Device parameter adaptation provides a dynamic adjustment basis for cropping: The cropping process also needs to adapt to the device parameters of the image acquisition module (such as frame rate and resolution) to ensure that the amount of data after cropping matches the processing capabilities of subsequent encoding and transmission stages. When the image acquisition module adjusts the frame rate (e.g., increasing to 240fps in high-speed autonomous driving scenarios) or resolution (e.g., switching to 4K when high definition is required for remote medical care), the edge preprocessing module will synchronously adjust the cropping parameters—appropriately reducing the cropping range when the frame rate is increased (reducing the amount of data per frame to adapt to high-frequency processing requirements), and maintaining the pixel density of key areas when the resolution is increased (ensuring the clarity of key information), avoiding fluctuations in the amount of data after cropping due to changes in device parameters, which would affect the overall latency control.

[0079] In summary, image cropping achieves precise simplification of the original image through three layers of criteria: "defining the target based on requirements, locating the range of the algorithm, and dynamically adapting parameters." This reduces the data processing pressure for subsequent encoding and transmission, helping the system achieve its goal of ultra-low latency.

[0080] The noise filtering unit is used to remove noise from the image using an adaptive median filtering algorithm, while avoiding excessive blurring of image details;

[0081] The Region of Interest (ROI) extraction unit is used to automatically identify and extract the region of interest in an image through edge detection and region growing algorithms, thereby reducing the amount of data required for subsequent encoding and transmission.

[0082] like Figure 3 As shown, the intelligent transmission module includes:

[0083] The transmission protocol selection unit automatically selects the optimal transmission protocol based on the current network conditions (such as network bandwidth, packet loss rate, and latency jitter) using the built-in TCP, UDP, and custom lightweight transmission protocols. When the network conditions are good (packet loss rate less than 1%, latency jitter less than 10ms), UDP is selected for transmission to achieve the lowest transmission latency. When the network conditions are poor (packet loss rate greater than 5%, latency jitter greater than 30ms), the custom lightweight transmission protocol is selected. This protocol integrates a simple packet loss retransmission mechanism and flow control mechanism to reduce transmission latency while ensuring transmission reliability. When the network conditions are at an intermediate level, TCP is selected for transmission.

[0084] The dynamic bandwidth adaptation unit is used to monitor network bandwidth changes in real time and feed the bandwidth information back to the adaptive lightweight coding module. This allows the adaptive lightweight coding module to adjust the coding rate according to the bandwidth information, ensuring that the encoded image data can be transmitted stably under the current network bandwidth and avoiding image stuttering and increased latency due to insufficient bandwidth.

[0085] like Figure 4 As shown, the dynamic priority scheduling module includes:

[0086] The pre-storage unit is used to pre-store image region priority rules for different application scenarios using the built-in region priority mapping library (e.g., in telemedicine scenarios: surgical instrument region > surgical field background region > environmental region; in autonomous driving scenarios: obstacle region > lane line region > roadside vegetation region). At the same time, it receives region of interest labeling information transmitted by the edge preprocessing module (the original edge preprocessing module has already extracted the region of interest and can output the region priority label synchronously).

[0087] The priority parsing unit is used to first parse the region priority tags in the received encoded image data from the intelligent transmission module, and then dynamically adjust the transmission slice order based on the current remaining network bandwidth. For encoded data in high-priority regions, a dedicated transmission channel is allocated (bandwidth ratio increased by 30%-50%) and marked as a "priority decoding packet". For encoded data in low-priority regions, compressed slice transmission is used (slice size reduced to 60%-80% of the original size) to avoid occupying critical resources.

[0088] The scheduling instruction sending unit is used to synchronously send priority scheduling instructions so as to decode high-priority area data first and ensure that critical image information is displayed 1-3ms earlier than non-critical information. For example, in autonomous driving scenarios, obstacle area images can be decoded and displayed 0.8-1.5ms earlier, giving the vehicle more reaction time to make decisions.

[0089] like Figure 5 As shown, the adaptive quality repair module includes:

[0090] The model training unit is used to pre-train the lightweight hierarchical restoration model for image distortion restoration parameters in different scenarios (such as equipment texture distortion in industrial monitoring scenarios and blurred tissue details in remote medical scenarios) based on the MobileNetV2 lightweight network architecture (the number of parameters is only 15%-20% of that in traditional restoration models).

[0091] The distortion detection unit is used to perform distortion detection on the image data output by the fast decoding module during operation. It identifies blurred and color block distortion areas in the image caused by simplified encoding within 2-3ms using a pixel gradient difference algorithm, and marks the distortion level, which includes mild, moderate and severe.

[0092] The repair unit dynamically adjusts the repair intensity based on the distortion level. For slightly distorted areas (such as slightly blurred backgrounds), pixel interpolation repair is used (processing time < 1ms), which only optimizes edge smoothness. For moderately distorted areas (such as blurred device textures), local feature mapping repair is used (processing time 1-2ms), which restores key details. For severely distorted areas (such as color blocks caused by temporary packet loss), neighborhood information completion repair is used (processing time 2-3ms), which generates reasonable content based on surrounding pixels. The total time for all repair processes is controlled within 5ms, without exceeding the overall 20ms latency limit.

[0093] It also includes a digital image transmission method based on ultra-low latency, the specific steps of which are as follows:

[0094] S1: Utilizes a high frame rate, low noise image sensor to acquire digital image data of the scene in real time and transmits the acquired raw image data; the frame rate of this image sensor can be adjusted according to the actual application scenario requirements, with a maximum frame rate of up to 240fps, ensuring that it can capture rapidly changing information in the scene;

[0095] S2: Perform preprocessing operations on the acquired raw image data;

[0096] The specific steps of S2 are as follows:

[0097] S21: Based on actual application requirements, remove irrelevant areas from the image and retain only the image data of the key areas;

[0098] S22: An adaptive median filtering algorithm is used to remove noise from the image while avoiding excessive blurring of image details;

[0099] S23: By using edge detection and region growing algorithms, the region of interest in the image is automatically identified and extracted, reducing the amount of data for subsequent encoding and transmission;

[0100] S3: Utilizing a deep neural network model (trained on a large amount of image data), the encoding parameters and encoding method are adaptively adjusted based on the image's content features and real-time transmission bandwidth. Specifically, for regions with simple textures and slow changes, a simplified encoding algorithm is used to reduce the amount of computation; for regions with complex textures and rapid changes, an optimized encoding algorithm is used to minimize encoding latency while maintaining image quality. Simultaneously, the complex inter-frame prediction algorithm found in traditional encoding standards is abandoned, and a block-level differential coding method is adopted, encoding only the difference data between adjacent image blocks, further reducing the amount of encoded data and encoding latency.

[0101] S4: Automatically switches between TCP, UDP, and custom lightweight protocols based on network conditions; simultaneously monitors bandwidth changes in real time and provides feedback, adjusting the encoding bitrate accordingly.

[0102] The specific steps of S4 are as follows:

[0103] S41: Utilizing built-in TCP, UDP, and a custom lightweight transport protocol, it automatically selects the optimal transport protocol based on the current network conditions (such as network bandwidth, packet loss rate, and latency jitter). When the network conditions are good (packet loss rate less than 1%, latency jitter less than 10ms), it selects the UDP protocol for transmission to achieve the lowest transmission latency. When the network conditions are poor (packet loss rate greater than 5%, latency jitter greater than 30ms), it selects the custom lightweight transport protocol, which integrates a simple packet loss retransmission mechanism and flow control mechanism to reduce transmission latency while ensuring transmission reliability. When the network conditions are at an intermediate level, it selects the TCP protocol for transmission.

[0104] S42: Monitor network bandwidth changes in real time and provide feedback on bandwidth information to adjust the encoding bitrate based on the bandwidth information, ensuring that the encoded image data can be transmitted stably under the current network bandwidth, and avoiding image stuttering and increased latency due to insufficient bandwidth;

[0105] S5: Based on the pre-stored scene-based region priority rules and combined with region of interest markers, priority is assigned to the encoded image data;

[0106] The specific steps of S5 are as follows:

[0107] S51 utilizes a built-in region priority mapping library to pre-store image region priority rules for different application scenarios (e.g., in telemedicine scenarios: surgical instrument region > surgical field background region > environmental region; in autonomous driving scenarios: obstacle region > lane line region > roadside vegetation region), while simultaneously receiving transmitted region of interest marking information.

[0108] S52, when receiving transmitted encoded image data, first parses the region priority label in the data, and then dynamically adjusts the transmission slice order based on the current remaining network bandwidth; for encoded data in high-priority regions, a dedicated transmission channel is allocated (bandwidth ratio increased by 30%-50%), and it is marked as a "priority decoding packet"; for encoded data in low-priority regions, compressed slice transmission is used (slice size reduced to 60%-80% of the original size) to avoid occupying critical resources;

[0109] S53 synchronously sends priority scheduling instructions to enable the decoding of high-priority area data, ensuring that critical image information is displayed 1-3ms earlier than non-critical information. For example, in autonomous driving scenarios, obstacle area images can be decoded and displayed 0.8-1.5ms earlier, giving the vehicle more reaction time for decision-making.

[0110] S6: The received encoded image data is quickly decoded using the corresponding decoding algorithm. Due to the simplified encoding algorithm and block-level differential coding, the decoding process is also simplified, eliminating the need for complex calculations and significantly reducing decoding latency. The simplified encoding algorithm is a lightweight encoding scheme that abandons the complex inter-frame / intra-frame prediction and transform quantization processes of traditional image encoding standards (such as JPEG and HEVC). It achieves efficient compression of image data through a five-step pipeline of "block partitioning - region prediction - differential calculation - simplified quantization - entropy coding," focusing on optimizing decoding speed and resource consumption, and is suitable for low-latency transmission requirements in high real-time scenarios. The block-level differential coding is a technique that divides the image into fixed-size blocks (such as 4×4, 8×8, 16×16 pixels) and encodes the difference between each pixel within a block and a reference value (the mean value within the block, pixels in adjacent blocks, or blocks at the same position in historical frames), rather than directly encoding the pixel value itself. By utilizing local image correlation, it significantly reduces the amount of data while supporting fast decoding and error recovery.

[0111] S7: Under the premise of ultra-low latency, repair the image quality loss caused by simplified coding; first, a lightweight layered repair model is adopted to quickly detect the distortion area and level of the decoded image; then, pixel interpolation, local feature mapping and neighborhood information completion repair methods are used to repair mild, moderate and severe distortion respectively.

[0112] The specific steps of S7 are as follows:

[0113] S71: Based on the MobileNetV2 lightweight network architecture (the number of parameters is only 15%-20% of that of traditional inpainting models), pre-train lightweight hierarchical inpainting models for image distortion inpainting parameters in different scenarios (such as equipment texture distortion in industrial monitoring scenarios and blurred tissue details in remote medical scenarios).

[0114] S72: During operation, the output image data is subjected to distortion detection. By using a pixel gradient difference algorithm, the blurred and color block distortion areas caused by simplified encoding in the image are identified within 2-3ms, and the distortion level is marked, including mild, moderate and severe.

[0115] S73: The repair intensity is dynamically adjusted according to the distortion level; for slightly distorted areas (such as slightly blurred background), pixel interpolation repair is used (processing time < 1ms), only optimizing edge smoothness; for moderately distorted areas (such as blurred device texture), local feature mapping repair is used (processing time 1-2ms), restoring key details; for severely distorted areas (such as color blocks caused by temporary packet loss), neighborhood information completion repair is used (processing time 2-3ms), generating reasonable content based on surrounding pixels, and the total time of all repair processes is controlled within 5ms, without exceeding the overall 20ms delay limit;

[0116] S8: Utilizes a high refresh rate display device to display the decoded image data in real time; the refresh rate of this display device can reach up to 240Hz, which can match the frame rate of image acquisition, avoiding image stuttering and delay caused by insufficient refresh rate of the display device, and ensuring that users can observe the image information in the scene in real time.

[0117] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for transmitting digital images based on ultra-low latency, characterized in that, The specific steps include the following: S1: Real-time acquisition of digital image data in the scene using a high-frame-rate, low-noise image sensor, and transmission of the acquired original image data; S2: Preprocessing operation on the acquired original image data; S3: Using a deep neural network model, adaptively adjusting the encoding parameters and encoding mode according to the content features of the image and the real-time transmission bandwidth; S4: Automatically switching between TCP, UDP, and a custom lightweight protocol according to the network state; simultaneously, real-time monitoring of bandwidth changes and feedback, and linkage adjustment of the encoding rate; S5: Relying on pre-stored scene-based regional priority rules and combining region-of-interest markers, priority division of the encoded image data; S6: Using the corresponding decoding algorithm to quickly decode the received encoded image data; S7: Under the premise of ultra-low latency, repairing the image quality loss caused by simplified encoding; first, using a lightweight hierarchical repair model to quickly detect the distortion area and level of the decoded image, and then using pixel interpolation, local feature mapping, and neighborhood information completion repair methods for repair according to the degree of distortion; S8: Using a high-refresh-rate display device to display the decoded image data in real time.

2. The method of claim 1, wherein, The specific steps of S2 are as follows: S21: According to the actual application requirements, remove irrelevant areas in the image and only keep the image data of the key areas; S22: Using an adaptive median filter algorithm to remove noise in the image while avoiding excessive blurring of image details; S23: Automatically identifying and extracting the region of interest in the image through edge detection and region growing algorithms, reducing the amount of data for subsequent encoding and transmission.

3. The method of claim 1, wherein, The specific steps of S4 are as follows: S41: Using built-in TCP protocol, UDP protocol, and custom lightweight transmission protocol, automatically selecting the optimal transmission protocol according to the current network state; S42: Real-time monitoring of network bandwidth changes and feedback on bandwidth information to adjust the encoding rate according to the bandwidth information, ensuring that the encoded image data can be stably transmitted under the current network bandwidth, avoiding image stuttering and increased latency due to insufficient bandwidth.

4. The method of claim 1, wherein, The specific steps of S5 are as follows: S51: Using the built-in regional priority mapping library, pre-storing image regional priority rules for different application scenarios, and receiving the region-of-interest marker information transmitted; S52: When receiving the transmitted encoded image data, first parse the regional priority tags in the data, and then dynamically adjust the transmission slice order in combination with the current network bandwidth remaining; S53: Synchronized sending of priority scheduling instructions to enable preferential decoding of high-priority area data and ensure that key image information is displayed ahead of non-key information.

5. The method of claim 1, wherein, The specific steps of S7 are as follows: S71: According to the MobileNetV2 lightweight network architecture, pre-training the lightweight hierarchical repair model with different scene image distortion repair parameters; S72: In working time, the output image data is distorted to identify the blur and color block distortion area caused by simplified encoding in the image by pixel gradient difference algorithm, and the distortion level is marked, including light, medium and heavy; S73: According to the distortion level, the repair strength is dynamically adjusted; the pixel interpolation repair is used for light distortion area, and only the edge smoothness is optimized; the local feature mapping repair is used for medium distortion area, and the key details are restored; the neighborhood information completion repair is used for heavy distortion area, and reasonable content is generated based on the surrounding pixels.

6. A digital image transmission system based on ultra-low latency, characterized in that, It comprises: An image acquisition module is used for acquiring digital image data in a scene in real time by using a high frame rate and low noise image sensor, and transmitting the acquired original image data; An edge preprocessing module is arranged at an edge node close to the image acquisition module to perform preprocessing operation on the acquired original image data; An adaptive lightweight encoding module is used for adaptively adjusting encoding parameters and encoding mode according to the content features of the image and the real-time transmission bandwidth by using a deep neural network model; An intelligent transmission module is used for automatically switching among TCP, UDP and a self-defined lightweight protocol according to the network state; at the same time, the bandwidth change is monitored in real time and fed back to the adaptive lightweight encoding module to link and adjust the encoding rate; A dynamic priority scheduling module is used for dividing the priority of the encoded image data according to the pre-stored scene-based regional priority rules and the region of interest marked by the edge preprocessing module; A fast decoding module is used for fast decoding the received encoded image data by using a corresponding decoding algorithm; An adaptive quality repair module is used for repairing the image quality loss caused by simplified encoding under the premise of ultra-low delay; firstly, a lightweight layered repair model is used to quickly detect the distortion area and level of the decoded image, and then pixel interpolation, local feature mapping and neighborhood information completion repair methods are used for repairing the light, medium and heavy distortion respectively; An image display module is used for displaying the decoded image data in real time by using a high refresh rate display device.

7. A digital image transmission system based on ultra-low latency according to claim 6, characterized in that, The edge preprocessing module comprises: An image cropping unit is used for removing irrelevant areas in the image and retaining only the image data of the key area according to actual application requirements; A noise filtering unit is used for removing noise in the image by using an adaptive median filtering algorithm, while avoiding excessive blurring of image details; An interesting region extraction unit is used for automatically identifying and extracting the interesting region in the image by edge detection and region growing algorithm, reducing the data amount of subsequent encoding and transmission.

8. The ultra-low latency based digital image transmission system as claimed in claim 6, wherein, The intelligent transmission module comprises: A transmission protocol selection unit is used for automatically selecting the optimal transmission protocol according to the current network state by using the built-in TCP protocol, UDP protocol and self-defined lightweight transmission protocol; The dynamic bandwidth adaptation unit is used for monitoring network bandwidth changes in real time and feeding back bandwidth information to the adaptive lightweight coding module, so that the adaptive lightweight coding module adjusts the coding code rate according to the bandwidth information, ensures that the coded image data can be stably transmitted under the current network bandwidth, and avoids image lag and increased delay caused by insufficient bandwidth.

9. The ultra-low latency based digital image transmission system as claimed in claim 6, wherein, The dynamic priority scheduling module comprises: A pre-storage unit is configured to pre-store image region priority rules of different application scenarios by using a built-in regional priority mapping library, and receive region-of-interest marking information transmitted by the edge preprocessing module; A priority analysis unit is configured to analyze the region priority label in the data first when receiving the encoded image data transmitted by the intelligent transmission module, and then dynamically adjust the transmission slice order in combination with the current network bandwidth remaining amount; A scheduling instruction sending unit is configured to synchronously send the priority scheduling instruction, so as to preferentially decode high-priority region data and ensure that key image information is displayed earlier than non-key information.

10. The ultra-low latency based digital image transmission system as claimed in claim 6, wherein, The adaptive quality repair module comprises: A model training unit is configured to pre-train image distortion repair parameters of different scenes of a lightweight layered repair model according to a MobileNetV2 lightweight network architecture; A distortion detection unit is configured to detect the image data output by the fast decoding module when working, to identify blurred and color block distortion regions in the image caused by simplified coding by using a pixel gradient difference algorithm, and mark a distortion level, which includes light, medium and heavy; A repair unit is configured to dynamically adjust the repair intensity according to the distortion level; the light distortion region is repaired by pixel interpolation, only optimizing the edge smoothness; the medium distortion region is repaired by local feature mapping, restoring key details; and the heavy distortion region is repaired by neighborhood information completion, generating reasonable content based on surrounding pixels.

Images