Media picture rendering optimization method and system based on artificial intelligence

Abstract

The invention relates to the technical field of picture rendering, in particular to a media picture rendering optimization method and system based on artificial intelligence. The method comprises the following steps: acquiring a terminal video stream; performing RGB format conversion on the terminal video stream to generate an RGB type frame image set; pixel partitioning is carried out on the RGB type frame image set, and RGB pixel matrix blocks are generated; performing color space acceleration conversion on the RGB pixel matrix block to generate a YUV cache data matrix; the YUV storage data matrix is sent to a DSP internal processing unit for key frame selection, and YUV key video frame data and YUV non-key video frame data are obtained; and performing intra-frame coding on the YUV key video frame data so as to generate YUV key frame coded data. According to the method, the media picture rendering efficiency is improved by optimizing data processing, transmission and rendering and combining user visual focus analysis.

Description

Technical Field

[0001] The present invention relates to the field of picture rendering technology, and in particular to an artificial intelligence-based media picture rendering optimization method and system. Background Art

[0002] The application of artificial intelligence in the field of media image rendering is developing rapidly, especially the introduction of technologies such as deep learning and convolutional neural networks (CNN), which have greatly improved the rendering quality and efficiency. In traditional image rendering, although methods such as ray tracing, texture mapping, and geometric calculation can generate realistic images, they have high computational costs and are inefficient when processing complex scenes. With the development of computer hardware and graphics processing units (GPUs), the rendering speed has increased, but it still faces the bottleneck of real-time processing capabilities. The development of artificial intelligence technology has brought new opportunities for rendering optimization. Early rule-based rendering optimization techniques have gradually been replaced by more adaptive and intelligent deep learning models. Network architectures represented by U-Net and generative adversarial networks (GANs) can efficiently reconstruct high-quality images by learning texture and light and shadow characteristics in large-scale data sets. However, in the current existing technologies, high-quality rendering of video streams often requires a lot of bandwidth, especially the processing differences between key frames and non-key frames affect the transmission efficiency. At the same time, traditional rendering methods usually process the entire picture evenly, fail to fully consider the user's visual focus, resulting in a waste of computing resources, and then lead to low efficiency of media image rendering. Summary of the invention

[0003] Based on this, it is necessary to provide an artificial intelligence-based media image rendering optimization method and system to solve at least one of the above technical problems.

[0004] To achieve the above object, a media picture rendering optimization method based on artificial intelligence is provided, the method comprising the following steps:

[0005] Step S1: obtaining a terminal video stream; performing RGB format conversion on the terminal video stream to generate an RGB type frame image set; performing pixel block division on the RGB type frame image set to generate an RGB pixel matrix block; performing color space acceleration conversion on the RGB pixel matrix block to generate a YUV cache data matrix;

[0006] Step S2: Send the YUV storage data matrix to the DSP internal processing unit for key frame selection to obtain YUV key video frame data and YUV non-key video frame data; perform intra-frame encoding on the YUV key video frame data to generate YUV key frame encoded data; perform inter-frame encoding on the YUV non-key video frame data to generate YUV non-key frame encoded data; perform network packaging and transmission on the YUV key frame encoded data and the YUV non-key frame encoded data to generate an RTP network transmission data packet;

[0007] Step S3: reorganizing data segments of data packets sent over the RTP network based on the remote device to generate a video reorganized bit stream; performing packet loss compensation on the video reorganized bit stream to generate an unpacked video stream; performing component decoding on the unpacked video stream to generate RGB data to be rendered;

[0008] Step S4: perform initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; perform user visual focus detection on the unpacked video stream to generate user sight track data; map the user sight track data to the media rendering core screen for layered rendering settings to generate local fine rendering data and global fuzzy rendering data; perform mixed rendering and execution adjustment on the initial media rendering screen according to the local fine rendering data and the global fuzzy rendering data, thereby generating rendering adjustment screen data to execute the media screen optimization rendering operation.

[0009] The present invention can make subsequent processing more efficient by converting the terminal video stream into an RGB type frame image set, especially in multimedia processing, the RGB format is a basic image processing format, which is easy to operate and optimize. The RGB frame image set is pixel-blocked, and the large-scale image data is divided into small pixel matrix blocks, which is conducive to parallel processing, improves processing speed and resolution, and reduces computational complexity. The RGB pixel matrix block is converted into a YUV cache data matrix, which is a format suitable for video compression and transmission, reduces data redundancy, reduces bandwidth consumption, and provides a basis for subsequent compression and encoding. By processing the YUV cache data matrix in the internal processing unit of the DSP, key frames and non-key frames are distinguished, and the compression efficiency of the video stream is improved. The key frame saves the complete picture information, and the non-key frame only saves the changed part. This division of labor optimizes the video encoding efficiency. The key frame is intra-frame encoded and the non-key frame is inter-frame encoded respectively, which greatly reduces the amount of transmitted data, reduces the network transmission pressure, and ensures that high-quality images can still be maintained under low bandwidth. By packaging the encoded YUV data into RTP network transmission data packets, the burden of data transmission is further reduced, and the reliable transmission of data in the network is guaranteed, which enhances the transmission stability of the system. The remote device reassembles the RTP data packets and recovers the video bitstream, ensuring the integrity and consistency of the data, and can effectively cope with the instability of the network environment. The packet loss compensation mechanism allows the missing data to be supplemented by the algorithm even if data is lost during network transmission, avoiding screen freeze or loss, and improving the playback quality of the video stream. The reassembled video bitstream is decoded into RGB data to ensure the high-quality data source required for video rendering, provide support for subsequent rendering optimization, and ensure the accuracy of the picture presentation. The decoded video stream is initially rendered to generate a basic picture to ensure that the user can see the picture quickly and improve the response speed. This step provides a rough picture and lays the foundation for further optimization of rendering. By detecting the user's visual focus and generating user sight trajectory data, the system can understand the area of ​​user concern, realize reasonable resource allocation, and contribute to personalized rendering optimization. The user's sight trajectory data is mapped to the picture, and local fine rendering and global fuzzy rendering are performed according to the user's visual focus, which not only improves the picture quality of the key area, but also saves computing resources in non-key areas. The local fine rendering and global fuzzy rendering are combined to further optimize the initial picture and generate the final rendering adjustment data. Through this process, not only the picture quality is improved, but also the rendering efficiency is greatly improved, the user experience is optimized, especially the efficient use of resources in real-time rendering. Therefore, the present invention improves the efficiency of media picture rendering by optimizing data processing, transmission and rendering, combined with user visual focus analysis.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S11: using a camera to capture a video stream of a terminal media device to obtain a terminal video stream;

[0012] Step S12: Decomposing the terminal video stream into video frame images to obtain a terminal video frame image set; performing RGB format conversion on the terminal video frame image set to generate an RGB type frame image set;

[0013] Step S13: performing image preprocessing on the RGB type frame image set to generate a standard RGB type frame image, wherein the image preprocessing includes image filtering, image brightness enhancement and image geometric neighborhood transformation;

[0014] Step S14: pixel-blocking the standard RGB type frame image set to generate RGB pixel matrix blocks; performing color space acceleration conversion on the RGB pixel matrix blocks to generate a YUV cache data matrix.

[0015] The present invention ensures efficient acquisition of terminal media device video through real-time video stream capture function, which not only ensures the real-time processing, but also improves the response speed of the system. It is suitable for scenarios that require fast video processing, such as video calls, video monitoring, etc. By decoupling the video stream into a set of video frame images, the complexity of video continuity processing is avoided. This step provides the advantage of frame-by-frame image processing, so that the quality of each frame image is improved, and lays the foundation for subsequent calculation steps, especially for improving processing accuracy in computer vision applications. By converting the image from RGB format to RGB format, it is more suitable for current mainstream image processing algorithms (such as convolutional neural networks in deep learning) and hardware acceleration devices. This conversion improves the compatibility of the system and greatly improves the image processing speed and result accuracy. Image preprocessing not only removes image noise, but also improves the visibility of the image by brightness enhancement and other means, ensures image clarity in low-light environments, and enhances the visual experience of the video source. In addition, geometric neighborhood transformation can correct image distortion and ensure the accuracy of geometric information in subsequent analysis. Pixel blocking technology can divide large images into smaller areas for parallel processing, effectively reducing processing time and improving system throughput. This process is very suitable for scenarios that require processing a large number of frame images, such as video editing and streaming media processing. By converting the RGB color space to the YUV color space, the redundancy of color information is reduced, which helps compression processing. The YUV space separates brightness and chrominance information, significantly improves encoding efficiency in video compression algorithms, and reduces bandwidth consumption. It is especially suitable for real-time video transmission scenarios. Generating a YUV cache data matrix can not only improve the system's response speed when processing and transmitting videos, but also greatly compress storage requirements through color space optimization, thereby saving storage space and transmission resources and improving the overall performance of the system. The conversion from RGB to YUV ensures compatibility with video coding standards (such as H.264, HEVC, etc.), and provides a convenient way for video compression, transmission and playback, which enables terminal devices to seamlessly connect with different video formats, decoders and playback platforms.

[0016] Preferably, performing color space accelerated conversion on the RGB pixel matrix block includes:

[0017] The floating point conversion formula is defined for the RGB pixel matrix block to obtain the floating point conversion calculation formula, where the floating point conversion calculation formula is as follows:

[0018] Y=0.299·R+0.587·G+0.114·B;

[0019] U=-0.14713·R-0.28886·G+0.436·B;

[0020] V=0.615·R-0.51499·G-0.10001·B;

[0021] Where Y represents brightness, U represents blue chromaticity component, V represents red chromaticity component, R represents the red component of each pixel in the image, G represents the green component of each pixel in the image, and B represents the blue component of each pixel in the image.

[0022] The component scale transformation is performed on the RGB pixel matrix block to generate an RGB pixel digital matrix; the RGB pixel digital matrix is ​​parallelized to an integer color space conversion using a floating point conversion calculation formula to generate YUV component data after parallel processing; the YUV component data after parallel processing is cached and stored to generate a YUV cache data matrix.

[0023] The present invention effectively converts the RGB color space into the YUV color space through the definition of the floating point conversion formula (YUV conversion). This conversion can decompose the three components of RGB into brightness (Y) and two chromaticity information (U and V), so as to better separate the brightness and color information of the image and improve the compression and encoding efficiency in subsequent video processing. The use of floating point conversion calculation formulas (such as Y=0.299·R+0.587·G+0.114·B, etc.) can more accurately calculate the brightness and chromaticity information of each pixel. This accuracy is essential for maintaining high-quality image conversion, especially in scenes with high color expression requirements, and can ensure less distortion. Component scaling transformation of RGB pixel matrix blocks helps to standardize the color value range of different images, so that the processed data is more suitable for subsequent compression, storage or transmission. This standardization ensures the consistency of image color information, especially when multi-source data is synthesized. Through parallel integer color space conversion, the conversion process from RGB to YUV is greatly accelerated, especially suitable for real-time processing scenarios of large-scale image or video data. Parallel processing makes full use of multi-core or GPU resources, significantly improves processing efficiency, and reduces system latency. The YUV color space separates the brightness and chrominance information, which is conducive to more efficient data compression. The brightness information Y is usually more important than the chrominance information U and V, so the U and V components can be compressed at a lower bit rate without significantly affecting the image quality, which is crucial for video coding standards (such as H.264, HEVC, etc.), thereby greatly reducing bandwidth usage. After converting the RGB pixel matrix to YUV, the storage space required is usually less because the chrominance information can be compressed or stored at a lower sampling rate (such as 4:2:0 sampling mode), which helps to reduce the storage overhead of the data and improve the storage efficiency of the data, especially in large-scale video processing or real-time video transmission, reducing the burden on the system. The YUV format is the basis of many video coding standards. The YUV cache data matrix generated by parallel processing is easy to seamlessly connect with the existing video compression algorithm, which not only improves the compatibility of terminal devices in the video processing process, but also provides a stable foundation for video compression, transmission and playback. The design of caching and storing YUV component data is very useful when processing large-scale video data in real time. The cache mechanism can temporarily store the intermediate processing results, reduce the direct occupation of system memory, and improve the data transmission efficiency between the processor and the storage device. Through precise color space conversion and parallel processing, the color distortion problem in traditional color conversion algorithms is avoided and the color fidelity of the image is maintained, which is particularly important for high-demand application scenarios such as video surveillance and film and television production. Parallel processing and cache storage mechanisms provide scalability for high-performance computing devices (such as GPUs, FPGAs, etc.), and can support a variety of terminal media devices from mobile devices to high-performance computing servers.

[0024] Preferably, step S2 comprises the following steps:

[0025] Step S21: Sending the YUV storage data matrix to the DSP internal processing unit through the DSP hardware interface to initialize the encoder and generate encoder initialization parameter setting data, wherein the encoder initialization includes resolution initialization, bit rate control mode initialization, frame rate initialization and key frame interval initialization;

[0026] Step S22: based on the encoder initialization parameter setting data, the YUV storage data matrix is ​​subjected to key frame selection to obtain YUV key video frame data and YUV non-key video frame data; the YUV key video frame data is subjected to intra-frame encoding to generate YUV key frame encoded data; the YUV non-key video frame data is subjected to inter-frame encoding to generate YUV non-key frame encoded data;

[0027] Step S23: transmitting the YUV key frame coded data and the YUV non-key frame coded data to the upper layer CPU data, thereby generating YUV upper layer transmission data;

[0028] Step S24: Slice the YUV upper layer transmission data to generate YUV upper layer slice data; add RTP packet header information to the YUV upper layer slice data to generate RTP packet header information data; control the packing sequence of the YUV upper layer slice data according to the RTP packet header information data to generate an RTP data packet; send the RTP data packet to the network to generate an RTP network sending data packet.

[0029] The present invention realizes the transmission of YUV data matrix and the initialization of encoder through the DSP hardware interface of step S21. The encoder initialization parameter setting includes the optimized configuration of parameters such as resolution, rate control mode, frame rate and key frame interval, which ensures that the encoder is dynamically adjusted according to different application scenarios and optimizes the video encoding quality and processing efficiency. Resolution initialization can adapt to different video quality requirements, the rate control mode effectively balances the video quality and bandwidth utilization, and the frame rate and key frame interval control ensure the smoothness and fast playback of the video. Key frame selection is performed on the YUV video stream based on the data of the initialization parameters to ensure that the high quality of the key frame is maintained during the video compression process, while the non-key frame is effectively compressed. The encoding within the key frame (I frame) retains the complete picture information and guarantees the visual effect during video decoding, while the non-key frame (P frame and B frame) is compressed based on the information of the previous and next frames through inter-frame coding, which greatly reduces data redundancy, effectively improves the video compression rate, and reduces the bandwidth required for storage and transmission. Inter-frame coding technology achieves more efficient compression by analyzing redundant information between video frames. This technology reduces the transmission of repeated information between consecutive frames, greatly reduces the network bandwidth requirement, and enables smooth transmission of video in low-bandwidth environments, which is particularly suitable for real-time video application scenarios. By uploading YUV key frame and non-key frame encoding data to the upper CPU, the computing pressure of the DSP processing unit is reduced. Through the distributed processing architecture, complex encoding tasks are divided between the DSP and the CPU to improve the overall processing efficiency, which can achieve fast task allocation and parallel processing, reduce delays, and improve processing performance. By slicing the encoded data and adding RTP (Real-time Transport Protocol) headers, step S24 ensures the structuring and management of data packets in network transmission. Data slicing technology can split large blocks of data into small fragments suitable for transmission, reduce transmission delays and improve network utilization. The addition of information in the RTP header helps to maintain the order and integrity of the data packet and ensure the reliability of video data during network transmission. The RTP protocol provides key support for the transmission of real-time data. It can ensure the orderly and timely transmission of video data in an unreliable network environment (such as the Internet or wireless network). The addition of RTP header information makes each data packet carry a timestamp and sequence number, ensuring low latency and continuity of the video stream in network transmission, and avoiding problems such as video jitter and freeze. By configuring the bit rate control mode in the initialization phase, the bit rate in the encoding process can be dynamically adjusted according to the network bandwidth and video quality requirements. This not only ensures the smoothness of the video, but also avoids excessive bandwidth consumption, and can still provide a higher viewing experience in a network environment with limited bandwidth. Through the packet sequence control and RTP data packet sending in step S24, the order and integrity of the data during network transmission are ensured.Even if some data packets are lost during transmission, the receiving end can re-sort and restore the data through the sequence number and timestamp information in the RTP header, thereby improving the fault tolerance of the video stream. The modular design of the entire process makes the system highly flexible and scalable. The distributed processing structure of DSP and CPU, as well as the standardized data segmentation and RTP packaging transmission mechanism, enable the system to easily cope with different video resolutions, frame rates and bandwidth requirements, and also facilitate future hardware upgrades or system expansions.

[0030] Preferably, step S3 comprises the following steps:

[0031] Step S31: receiving data packets sent from the RTP network based on the remote device, generating a local cache RTP data packet; performing RTP packet header parsing on the local cache RTP data packet, generating RTP packet header parsing data;

[0032] Step S32: reorganizing the data segments of the locally cached RTP data packets according to the RTP packet header parsing data to generate a video reorganized bit stream;

[0033] Step S33: performing packet loss rate detection on the video reassembled bit stream, and when the packet loss rate of the video reassembled bit stream is greater than or equal to a preset packet loss rate threshold, a data packet retransmission request is made; when the packet loss rate of the video reassembled bit stream is less than the preset packet loss rate threshold, packet loss compensation is performed on the video reassembled bit stream to generate a depacketized video stream;

[0034] Step S34: component decoding is performed on the unpacked video stream to generate RGB data to be rendered.

[0035] The present invention receives data packets sent by the RTP network through a remote device and generates a local cache, thereby ensuring the data integrity in real-time video streaming transmission. By locally caching the received data packets, the impact of network fluctuations can be alleviated, and the continuity and reliability of data transmission can be ensured. At the same time, the analysis of the RTP header can quickly obtain key information such as timestamps and sequence numbers, providing a basis for subsequent data processing. The data fragments are reorganized based on the RTP header analysis data, which effectively solves the problem of incomplete data caused by disordered data packets or fragment loss during network transmission. The reorganized video bit stream can restore the original video data structure, ensure that no information is lost or repeated during the transmission process, and ensure the restoration quality of the video content. A packet loss rate detection mechanism is introduced to ensure the quality of video transmission. If the packet loss rate of the reorganized bit stream exceeds the preset threshold, the system will initiate a packet retransmission request, actively complete the lost data packets, and ensure the integrity and reliability of the video stream; when the packet loss rate is lower than the threshold, the system uses packet loss compensation technology to predict and fill the lost data through an algorithm, reduce the interruption of video playback or the degradation of picture quality, and improve user experience. The system dynamically adjusts the packet loss rate threshold according to the real-time network situation. When the network condition is poor, the threshold is appropriately increased to reduce the number of retransmission requests and avoid excessive consumption of network bandwidth; when the network condition is good, the threshold is lowered to ensure the high quality and stability of the video stream. This adaptive control strategy balances the video quality and the utilization efficiency of network resources. When the packet loss rate exceeds the threshold, the system can quickly detect and initiate a retransmission request. Through this real-time retransmission mechanism, data loss caused by network jitter or insufficient bandwidth can be effectively reduced, further ensuring the complete transmission of the video stream, which is suitable for scenarios with high real-time requirements, such as video conferencing and distance education. When the packet loss rate is lower than the threshold, the system compensates for the lost data through an intelligent algorithm to avoid frequent retransmission requests, thereby improving the efficiency of video transmission. The packet loss compensation algorithm can infer and fill in the lost information based on the similarity of the previous and next frames or adjacent data, thereby maintaining the smoothness of the video stream without affecting the user's viewing experience. By component decoding the unpacked video stream, the YUV data in the video stream is accurately converted into RGB data to be rendered. Through an efficient decoding process, the color, brightness and detail information of the video can be maintained to the greatest extent, ensuring the quality of the rendered image. The generation of RGB data provides high-quality input for subsequent video display or image processing. The system's component decoding not only supports efficient hardware acceleration, but also ensures data consistency during the decoding process. Whether it is RGB or YUV color space, the decoding process can ensure that the video image is presented consistently on different devices and under different playback conditions, and is suitable for adaptation to a variety of terminal devices.

[0036] Preferably, step S33 includes the following steps:

[0037] Step S331: marking the RTP packet sequence of the video reassembled bit stream to obtain RTP packet sequence data; performing continuity detection on the RTP packet sequence data, and when the RTP packet sequence data is discontinuous, calculating the packet loss rate of the video reassembled bit stream using a packet loss rate detection formula to obtain the packet loss rate of the video reassembled bit stream;

[0038] The formula for calculating the packet loss rate is as follows:

[0039] L = (T ÷ K) × 100%;

[0040] In the formula, L represents the packet loss rate, T represents the number of lost packets, and K represents the expected number of packets;

[0041] Step S332: when the packet loss rate of the video reassembled bit stream is greater than or equal to the preset packet loss rate threshold, a data packet retransmission request is made; when the packet loss rate of the video reassembled bit stream is less than the preset packet loss rate threshold, a packet loss sequence interval is marked on the video reassembled bit stream to obtain a packet loss sequence interval;

[0042] Step S333: extracting forward error correction code from the video reconstructed bit stream according to the packet loss sequence interval to obtain a video forward error correction code; inserting a packet loss sequence into the packet loss sequence interval according to the video forward error correction code to generate a packet loss sequence insertion interval;

[0043] Step S334: reconstructing video frames of the video reconstructed bit stream by inserting a packet loss sequence into an interval, thereby generating a reconstructed video bit stream; and depacketizing the reconstructed video bit stream, thereby generating a depacketized video stream.

[0044] The present invention effectively identifies lost data packets by marking the RTP packet sequence and detecting the continuity of the video recombined bit stream. This process ensures that the order of data packets in transmission is not disrupted, and can timely detect data packet loss or transmission errors, thereby ensuring the continuity and accuracy of the video stream. A specific packet loss rate calculation formula is adopted, namely L=(T÷K)×100%, where T represents the number of lost packets and K represents the expected number of packets. The formula can quickly and accurately evaluate the packet loss rate of the video stream, providing a scientific basis for the system to take corresponding remedial measures. The detection of the packet loss rate can help the system to grasp the quality status of network transmission in real time. The video stream is judged by a preset packet loss rate threshold. When the packet loss rate exceeds the threshold, the system can immediately initiate a retransmission request, thereby effectively reducing the data loss caused by packet loss; when the packet loss rate is lower than the threshold, compensation is performed according to the packet loss sequence. Through this threshold judgment mechanism, the integrity of the data can be maintained while reducing the number of retransmissions, thereby improving the transmission efficiency. The lost data packets are compensated by the forward error correction code technology. Even in the case of a low packet loss rate, the system can still intelligently mark the packet loss sequence interval and compensate for it. The forward error correction code extraction and insertion process improves the fault tolerance of the system, can effectively recover some lost data, and ensure the continuous playback of the video stream. By extracting the video forward error correction code and using these error correction codes to insert the packet loss sequence interval, the system can repair some lost data without relying on retransmission, avoiding unnecessary network overhead and transmission delay. This mechanism improves the efficiency and quality of video transmission. The video stream is reconstructed by inserting the packet loss sequence interval to ensure that the reconstructed video bitstream maintains the quality of the original video as much as possible. Reasonable speculation and filling of lost data during frame reconstruction can effectively reduce the phenomenon of image quality degradation or playback jamming, and improve the user's viewing experience. Combined with the packet loss sequence insertion interval and video frame reconstruction technology, the system can not only compensate for the lost data, but also effectively reconstruct the video stream to ensure that the output video stream has a high consistency with the original video stream. Through this process, the reconstructed video bitstream can restore the video details and continuity to the greatest extent. The combination of packet loss rate detection and error correction code insertion enables the system to maintain high transmission quality and reduce screen freezes or video interruptions in high packet loss or complex network environments. This feature is particularly suitable for real-time video transmission scenarios, such as video conferencing and online live broadcasts. The efficient generation and unpacking process of the reassembled video bitstream ensures the final presentation quality of the video stream after transmission. Even in an undesirable network environment, the system can still maintain the smoothness and clarity of the video through compensation and reassembly technologies, and users can hardly detect packet loss or transmission problems. Forward error correction code technology allows the system to quickly repair lost data without waiting for retransmission of packets, thereby reducing transmission delays. This mechanism is particularly important for application scenarios that require low latency, such as online games and real-time video surveillance.

[0045] Preferably, step S4 comprises the following steps:

[0046] Step S41: performing initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; performing boundary box segmentation on the initial media rendering screen to obtain a media rendering core screen;

[0047] Step S42: Perform user visual focus detection on the media rendering core picture to generate user sight track data; map the user sight track data to the media rendering core picture to identify the key picture area, and generate media rendering key picture area data and media rendering non-key picture area data;

[0048] Step S43: performing local fine rendering setting on the media rendering key picture area data to generate local fine rendering data; performing global fuzzy rendering setting on the media rendering non-key picture area data to generate global fuzzy rendering data; performing mixed rendering on the initial media rendering picture according to the local fine rendering data and the global fuzzy rendering data, thereby generating mixed rendering picture data;

[0049] Step S44: adjusting the rendering effect of the mixed rendering picture data to generate rendering adjustment picture data to perform a media picture optimization rendering operation.

[0050] The present invention divides the initial media rendering screen into core screen and non-core screen areas through the bounding box segmentation technology, thereby providing a basis for subsequent fine and fuzzy rendering. By finely rendering only the core area of ​​user concern, the high-precision rendering requirements for the entire screen are reduced, the rendering efficiency is significantly improved, and the system resource consumption is reduced. Through the user visual focus detection technology, the user's sight trajectory data is generated in real time, and these data are mapped to the rendering screen, and the key screen areas of user concern are accurately identified, which enables the rendering process to dynamically adapt to the user's visual needs, thereby providing users with a more personalized and immersive viewing experience. Combining local fine rendering and global fuzzy rendering, high-precision rendering is performed on the key areas of user concern, and fuzzy rendering is performed on non-key areas, which can not only ensure the clarity of the user's visual focus area, but also effectively reduce the resource overhead of non-essential areas, and further optimize the rendering efficiency and video quality. Through the fine rendering of the key areas, the visual effect of the user when watching is more prominent, the details are clear and vivid, and the overall picture is more fascinating. At the same time, the fuzzy processing of the non-key areas will not interfere with the user's viewing experience, but will help to highlight the key screen areas and enhance the layering and focus effect of the picture. Unnecessary calculations and resource waste are avoided through the reasonable allocation of fine rendering and fuzzy rendering. With limited hardware resources, the system can achieve the highest image quality with the lowest performance overhead, which is particularly suitable for video playback on mobile devices or in low-bandwidth environments. Based on changes in the user's visual focus, the system can adjust the rendering strategy in real time to achieve dynamic adaptation of picture rendering. No matter how the user's line of sight moves, the system can respond in time and concentrate rendering resources on the area where the user's line of sight stays, providing a smooth and natural viewing experience. Global fuzzy rendering technology reduces unnecessary high-precision data during transmission, especially when network bandwidth is limited. By reducing the need to transmit details in non-critical areas, bandwidth usage is significantly reduced, which allows the video to maintain high picture quality and smoothness in low-bandwidth or unstable network environments.

[0051] Preferably, step S42 includes the following steps:

[0052] Step S421: using a camera to collect the user gaze position on the media rendering core screen to obtain user gaze position information data; performing time series analysis on the user gaze position information data to generate user gaze position time series data; constructing a user gaze trajectory according to the user gaze position information data and the user gaze position time series data to obtain user gaze trajectory data;

[0053] Step S422: Calculate the pixel density of the user's sight trajectory data to obtain pixel density distribution data; generate a heat map based on the pixel density distribution data to obtain a user's sight heat map; identify and mark the sight focus area of ​​the user's sight heat map based on a preset density threshold to generate focus area marking data;

[0054] Step S423: performing screen coordinate conversion on the focus area mark data to generate focus area screen coordinate conversion data; mapping the focus area screen coordinate conversion data to the media rendering core screen to perform screen key area classification to generate media rendering key screen area data;

[0055] Step S424: using the media rendering key picture area data to divide the media rendering core picture into non-key picture areas, thereby generating media rendering non-key picture area data.

[0056] The present invention captures the user's gaze position in real time and generates an accurate user's sight trajectory, which provides a basis for subsequent focus area identification. By dynamically tracking the user's gaze position, the system can adjust the rendering effect in real time during video playback, synchronize the picture with the user's sight, and enhance the sense of immersion. Through the user's sight density data and heat map, the system can automatically and accurately identify the user's visual focus area, so that the rendering of the key area is more in line with the user's visual habits. Using the user's sight heat map, the system can focus resources on the area of ​​user attention, reduce unnecessary global rendering consumption, and thus optimize system performance. Through coordinate conversion and identification of the focus area, the system can accurately locate the user's focus point in the picture, make the rendering of the key area more accurate, and thus improve the visual effect. Through the fine rendering of the key area, the user can obtain a clearer and more realistic visual effect when watching. By dividing the non-critical areas, the system can perform low-priority processing on these areas, such as global fuzzy rendering, to reduce the system's consumption of rendering resources for non-important areas. The division of non-critical areas makes the overall rendering task more lightweight, and the system can concentrate computing resources on the key areas, thereby improving the speed and efficiency of rendering. By capturing the user's line of sight and focus area, the system can provide customized rendering effects for each user, improving the overall viewing experience. By reasonably dividing key and non-key areas, the system can effectively reduce unnecessary resource consumption, thereby providing high-quality video rendering under hardware constraints. The rendering strategy is dynamically adjusted based on the user's visual focus to ensure that the user is always focused on a clear and detailed picture area, greatly improving the viewing immersion and visual effects.

[0057] Preferably, step S43 includes the following steps:

[0058] Step S431: performing local fine rendering settings on the media rendering key picture area data, thereby generating a local fine rendering setting parameter data set, wherein the local fine rendering settings include local anti-aliasing and local material map detail enhancement; performing GPU accelerated rendering on the media rendering key picture area data using the local fine rendering setting parameter data set, thereby generating local fine rendering data;

[0059] Step S432: performing global blur rendering settings on the media rendering non-critical picture area data, thereby generating a global fine rendering setting parameter data set, wherein the global fine rendering settings include global dynamic blur and global resolution reduction; performing parallel blur accelerated rendering on the media rendering non-critical picture area data using the global fine rendering setting parameter data set, thereby generating global fine rendering data;

[0060] Step S433: Integrate the local fine rendering data and the global fuzzy rendering data into the initial media rendering picture to perform rendering area image fusion, thereby generating mixed rendering picture data.

[0061] The present invention can effectively improve the details and realism of key areas and enhance the user's visual experience through local fine rendering. GPU accelerated rendering is used to improve processing efficiency, so that complex rendering tasks can be completed in a short time and the smoothness of the picture is maintained. Through global fuzzy rendering, the system can reduce the rendering details of non-key areas, thereby saving computing resources and improving the overall rendering efficiency. The application of global dynamic blur can increase the vividness of the picture, so that users can feel a stronger sense of presence during the viewing process. By fusing the data of local fine rendering and global fuzzy rendering, the consistency of the overall style and quality of the picture is ensured, and the visual smoothness is improved. Hybrid rendering effectively combines details and blur effects, so that the picture has both focus and dynamics, providing a viewing experience that is more in line with user expectations. By targetedly enhancing the rendering details of key areas and blurring non-key areas at the same time, the system can provide personalized visual performance to meet the viewing needs of users. Effectively utilize computing resources, reduce the processing requirements of non-key areas through targeted rendering settings, thereby improving the overall rendering efficiency and adapting to the limitations of hardware performance. Through the combination of motion blur and detailed rendering, users can get a more realistic and immersive viewing experience, enhancing their immersion in media content.

[0062] In this specification, a media picture rendering optimization system based on artificial intelligence is provided, which is used to execute the above-mentioned media picture rendering optimization method based on artificial intelligence. The media picture rendering optimization system based on artificial intelligence includes:

[0063] The color gamut conversion module is used to obtain the terminal video stream; perform RGB format conversion on the terminal video stream to generate an RGB type frame image set; perform pixel block division on the RGB type frame image set to generate an RGB pixel matrix block; perform color space acceleration conversion on the RGB pixel matrix block to generate a YUV cache data matrix;

[0064] The video frame transmission module is used to send the YUV storage data matrix to the DSP internal processing unit for key frame selection to obtain YUV key video frame data and YUV non-key video frame data; perform intra-frame encoding on the YUV key video frame data to generate YUV key frame encoded data; perform inter-frame encoding on the YUV non-key video frame data to generate YUV non-key frame encoded data; package the YUV key frame encoded data and the YUV non-key frame encoded data for network transmission to generate RTP network transmission data packets;

[0065] The packet loss compensation module is used to reassemble the data segments of the data packets sent by the RTP network based on the remote device to generate a video reassembled bit stream; perform packet loss compensation on the video reassembled bit stream to generate an unpacked video stream; perform component decoding on the unpacked video stream to generate RGB data to be rendered;

[0066] The rendering acceleration module is used to perform initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; perform user visual focus detection on the unpacked video stream to generate user sight track data; map the user sight track data to the media rendering core screen for layered rendering settings to generate local fine rendering data and global fuzzy rendering data; perform mixed rendering and execution adjustment on the initial media rendering screen according to the local fine rendering data and the global fuzzy rendering data, thereby generating rendering adjustment screen data to execute media screen optimization rendering operations.

[0067] The beneficial effect of the present invention is that through pixel segmentation and accelerated conversion, efficient format conversion of video frames is achieved, providing a fast-response data basis for subsequent processing. The conversion from RGB to YUV reduces data redundancy and saves bandwidth for transmission and processing. The combination of intra-frame and inter-frame coding can significantly reduce the data volume of the video stream while maintaining the high quality of the key frames. The packaged data can be efficiently transmitted in the network, improving the transmission performance of the real-time video stream. Through packet loss compensation, the data loss occurring during video transmission is effectively reduced, ensuring the integrity of the video stream. Component decoding ensures accurate recovery of RGB data, laying the foundation for subsequent rendering. Through user visual focus detection technology, the system can automatically identify the user's gaze area, optimize the rendering effect of the key area, and reduce the processing burden of the non-key area. Global fuzzy rendering reduces the calculation pressure of the non-key area and improves the overall system performance. Hybrid rendering makes the entire picture have both clear key areas and smooth visual transitions, which improves the user's perception. Using intra-frame coding, inter-frame coding and RTP package transmission can reduce bandwidth consumption and improve the reliability of network transmission. Through focus detection and local rendering optimization, the clarity of key areas during user viewing is ensured, enhancing the interactive experience. Through color space accelerated conversion and GPU accelerated rendering, computing resources are effectively utilized to ensure efficient video stream processing and rendering operations. Therefore, the present invention improves the efficiency of media screen rendering by optimizing data processing, transmission and rendering, combined with user visual focus analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of the steps of a media image rendering optimization method based on artificial intelligence;

[0069] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0070] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0071] Figure 4 for Figure 1 Detailed implementation steps of step S4 in FIG.

[0072] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0073] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0074] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0075] It should be understood that although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, and the term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0076] To achieve this, please refer to Figures 1 to 4 , a media picture rendering optimization method based on artificial intelligence, the method comprising the following steps:

[0077] Step S1: obtaining a terminal video stream; performing RGB format conversion on the terminal video stream to generate an RGB type frame image set; performing pixel block division on the RGB type frame image set to generate an RGB pixel matrix block; performing color space acceleration conversion on the RGB pixel matrix block to generate a YUV cache data matrix;

[0078] Step S2: Send the YUV storage data matrix to the DSP internal processing unit for key frame selection to obtain YUV key video frame data and YUV non-key video frame data; perform intra-frame encoding on the YUV key video frame data to generate YUV key frame encoded data; perform inter-frame encoding on the YUV non-key video frame data to generate YUV non-key frame encoded data; perform network packaging and transmission on the YUV key frame encoded data and the YUV non-key frame encoded data to generate an RTP network transmission data packet;

[0079] Step S3: reorganizing data segments of data packets sent over the RTP network based on the remote device to generate a video reorganized bit stream; performing packet loss compensation on the video reorganized bit stream to generate an unpacked video stream; performing component decoding on the unpacked video stream to generate RGB data to be rendered;

[0080] Step S4: perform initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; perform user visual focus detection on the unpacked video stream to generate user sight track data; map the user sight track data to the media rendering core screen for layered rendering settings to generate local fine rendering data and global fuzzy rendering data; perform mixed rendering and execution adjustment on the initial media rendering screen according to the local fine rendering data and the global fuzzy rendering data, thereby generating rendering adjustment screen data to execute the media screen optimization rendering operation.

[0081] The present invention can make subsequent processing more efficient by converting the terminal video stream into an RGB type frame image set, especially in multimedia processing, the RGB format is a basic image processing format, which is easy to operate and optimize. The RGB frame image set is pixel-blocked, and the large-scale image data is divided into small pixel matrix blocks, which is conducive to parallel processing, improves processing speed and resolution, and reduces computational complexity. The RGB pixel matrix block is converted into a YUV cache data matrix, which is a format suitable for video compression and transmission, reduces data redundancy, reduces bandwidth consumption, and provides a basis for subsequent compression and encoding. By processing the YUV cache data matrix in the internal processing unit of the DSP, key frames and non-key frames are distinguished, and the compression efficiency of the video stream is improved. The key frame saves the complete picture information, and the non-key frame only saves the changed part. This division of labor optimizes the video encoding efficiency. The key frame is intra-frame encoded and the non-key frame is inter-frame encoded respectively, which greatly reduces the amount of transmitted data, reduces the network transmission pressure, and ensures that high-quality images can still be maintained under low bandwidth. By packaging the encoded YUV data into RTP network transmission data packets, the burden of data transmission is further reduced, and the reliable transmission of data in the network is guaranteed, which enhances the transmission stability of the system. The remote device reassembles the RTP data packets and recovers the video bitstream, ensuring the integrity and consistency of the data, and can effectively cope with the instability of the network environment. The packet loss compensation mechanism allows the missing data to be supplemented by the algorithm even if data is lost during network transmission, avoiding screen freeze or loss, and improving the playback quality of the video stream. The reassembled video bitstream is decoded into RGB data to ensure the high-quality data source required for video rendering, provide support for subsequent rendering optimization, and ensure the accuracy of the picture presentation. The decoded video stream is initially rendered to generate a basic picture to ensure that the user can see the picture quickly and improve the response speed. This step provides a rough picture and lays the foundation for further optimization of rendering. By detecting the user's visual focus and generating user sight trajectory data, the system can understand the area of ​​user concern, realize reasonable resource allocation, and contribute to personalized rendering optimization. The user's sight trajectory data is mapped to the picture, and local fine rendering and global fuzzy rendering are performed according to the user's visual focus, which not only improves the picture quality of the key area, but also saves computing resources in non-key areas. The local fine rendering and global fuzzy rendering are combined to further optimize the initial picture and generate the final rendering adjustment data. Through this process, not only the picture quality is improved, but also the rendering efficiency is greatly improved, the user experience is optimized, especially the efficient use of resources in real-time rendering. Therefore, the present invention improves the efficiency of media picture rendering by optimizing data processing, transmission and rendering, combined with user visual focus analysis.

[0082] In the embodiment of the present invention, reference Figure 1The above is a schematic diagram of the steps of a media picture rendering optimization method based on artificial intelligence of the present invention. In this example, the media picture rendering optimization method based on artificial intelligence includes the following steps:

[0083] Step S1: obtaining a terminal video stream; performing RGB format conversion on the terminal video stream to generate an RGB type frame image set; performing pixel block division on the RGB type frame image set to generate an RGB pixel matrix block; performing color space acceleration conversion on the RGB pixel matrix block to generate a YUV cache data matrix;

[0084] In an embodiment of the present invention, a video stream is obtained from a terminal device through a system, which is an input from a camera, a video file or a real-time streaming media. A video stream is usually composed of continuous frames, each of which is an image represented in RGB format. The obtained video stream frame is format converted to RGB format. If the input video stream is not in RGB format (such as YUV or other formats), the system will first convert it to RGB format for subsequent processing. The converted frame image set will be represented in RGB color mode, that is, each pixel is composed of three color components: red (R), green (G), and blue (B). After each frame is format converted, a corresponding RGB type frame image set is generated, which contains RGB data of all frames in the video stream, and these image sets will be used as input for subsequent processing. The RGB type frame image set is pixel-blocked, and each frame image is blocked according to a fixed size. Each block is composed of a number of adjacent pixels, and the system divides the entire frame image according to the set resolution or pixel block size. The pixel data after blocking will form RGB pixel matrix blocks, each matrix block contains RGB data of multiple pixels, and these matrix blocks will be stored and used as input for subsequent color space conversion. Use hardware acceleration (such as GPU or dedicated video processing chip) to convert the RGB pixel matrix block from RGB color space to YUV color space. Y = 0.299·R + 0.587·G + 0.114·B; U = -0.14713·R -0.28886·G + 0.436·B; V = 0.615·R -0.51499·G -0.10001·B. The advantage of YUV color space is that it separates brightness (Y) and chrominance (U, V), which is convenient for compression and processing. The converted YUV data will be stored in matrix form, becoming a YUV cache data matrix, which can be used for video encoding, compression or subsequent rendering and transmission processing.

[0085] Step S2: Send the YUV storage data matrix to the DSP internal processing unit for key frame selection to obtain YUV key video frame data and YUV non-key video frame data; perform intra-frame encoding on the YUV key video frame data to generate YUV key frame encoded data; perform inter-frame encoding on the YUV non-key video frame data to generate YUV non-key frame encoded data; perform network packaging and transmission on the YUV key frame encoded data and the YUV non-key frame encoded data to generate an RTP network transmission data packet;

[0086] In an embodiment of the present invention, the YUV buffer data matrix generated in step S1 is sent to the internal processing module of the digital signal processing unit (DSP). The DSP unit has a special video processing capability and can accelerate the selection and encoding process of key frames. The internal processing unit of the DSP determines which frames are key frames (I frames) and which are non-key frames (P frames or B frames) by analyzing the content changes of the video frames. Key frames are usually frames with large content changes and can be decoded independently. Non-key frames are encoded based on the difference between key frames or previous and next frames. In this way, the YUV data matrix is ​​divided into two categories: YUV key video frame data and YUV non-key video frame data. The selected YUV key frame data is intra-frame encoded to generate YUV key frame encoded data. Intra-frame coding only compresses and encodes the current frame, does not depend on other frames, and is usually used in the basic encoding process of image compression. Standard compression technology (such as H.264 or H.265) can be used in the encoding process to reduce the data volume. Inter-frame encoding is performed on the YUV non-key frame data to generate YUV non-key frame encoded data. Inter-frame coding encodes the motion vector and difference data between frames by comparing the difference between the current frame and the previous and next frames. This coding method can further compress the amount of data because non-key frames only need to record the change information with the previous and next frames. Encapsulate the YUV key frame coded data after intra-frame coding and the YUV non-key frame coded data after inter-frame coding to generate RTP network data packets for transmission. RTP (Real-time Transport Protocol) is a protocol for multimedia data stream transmission, which can provide timing and synchronization services for video and audio. The encapsulation process includes the generation of RTP header information, which is used to indicate important parameters such as sequence number, timestamp, data type, etc. The generated RTP data packet will contain the coded frame of the video and is ready to be transmitted over the network. The encapsulated RTP network data packet is sent to the specified transmission network or server for decoding and playback by the receiving end.

[0087] Step S3: reorganizing data segments of data packets sent over the RTP network based on the remote device to generate a video reorganized bit stream; performing packet loss compensation on the video reorganized bit stream to generate an unpacked video stream; performing component decoding on the unpacked video stream to generate RGB data to be rendered;

[0088] In an embodiment of the present invention, a remote device sends RTP data packets through a network, and a terminal device receives these data packets. The RTP data packets contain coded frames (key frames and non-key frames) of a video stream, which are divided into multiple data packets for transmission. The receiving end sorts and reassembles these scattered RTP data packets to restore the complete video stream fragments. The sequence number and timestamp in the RTP header information are used to ensure the correct order of the data packets. A video reassembled bit stream is generated through the reassembly process, and the bit stream is an encoded version of the received original video stream. During the network transmission process, data packets may be lost or damaged. In order to ensure the integrity of the video stream, the system performs packet loss detection on the video reassembled bit stream. If some packets are found to be lost, the system will perform packet loss compensation, which can recover the lost data by using forward error correction code (FEC) or other compensation mechanisms. The packet loss compensation process can minimize the degradation of video quality caused by packet loss. After compensation, an unpacked video stream is generated, which is a complete video stream after packet loss compensation. The unpacked video stream is decoded to restore the original color and brightness information of the video. The video stream uses YUV color space encoding, so the YUV video data needs to be decoded into RGB format. The component decoding step includes extracting the brightness (Y) and chrominance (U, V) components from the YUV data and converting them into the RGB color space. After component decoding, RGB rendering data is generated, which represents the RGB pixel data of each frame in the video stream. RGB data is a standard color format suitable for display and rendering, and can be used directly for image display or further image processing (such as rendering, special effects, etc.).

[0089] Step S4: perform initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; perform user visual focus detection on the unpacked video stream to generate user sight track data; map the user sight track data to the media rendering core screen for layered rendering settings to generate local fine rendering data and global fuzzy rendering data; perform mixed rendering and execution adjustment on the initial media rendering screen according to the local fine rendering data and the global fuzzy rendering data, thereby generating rendering adjustment screen data to execute the media screen optimization rendering operation.

[0090] In an embodiment of the present invention, by using the unpacked video stream, preliminary rendering is performed to generate an initial media rendering screen, and this process can be completed by a graphics processing unit (GPU). The initial rendering may include basic settings for color, brightness, and contrast to ensure the clarity of the visual image. Computer vision algorithms are used to analyze the visual focus of users when watching video streams. This can obtain the user's eye movement data through a camera or sensor, identify the area they focus on, and generate user sight trajectory data, which records the movement and stay time of the user's sight during the viewing process. The user's sight trajectory data is mapped to the media rendering core screen to identify which areas are the user's key focus areas. By analyzing the sight trajectory, the "key area" and "non-key area" of the screen are divided. Local fine rendering settings are performed on the key areas of user focus to generate local fine rendering data. This process includes: applying anti-aliasing technology and material detail enhancement to make these areas look clearer and more vivid. GPU acceleration is used to perform rendering to improve rendering efficiency. Global fuzzy rendering settings are performed on non-key areas to generate global fuzzy rendering data. This process includes: using dynamic blur and resolution reduction technology to make these areas less prominent visually and reduce interference with user attention. The GPU is also used for parallel rendering to speed up the rendering. The local fine rendering data and the global fuzzy rendering data are integrated into the initial media rendering screen to perform image fusion of the rendering area. This process can use the weighted mixing method to reasonably combine the rendering effects of the area that the user is concerned about with other areas. According to the results of the mixed rendering, the rendering effect is further adjusted to optimize the visual effect of the media screen. Finally, the rendering adjustment screen data is generated to provide a basis for the subsequent media screen optimization rendering operation.

[0091] Preferably, step S1 comprises the following steps:

[0092] Step S11: using a camera to capture a video stream of a terminal media device to obtain a terminal video stream;

[0093] Step S12: Decomposing the terminal video stream into video frame images to obtain a terminal video frame image set; performing RGB format conversion on the terminal video frame image set to generate an RGB type frame image set;

[0094] Step S13: performing image preprocessing on the RGB type frame image set to generate a standard RGB type frame image, wherein the image preprocessing includes image filtering, image brightness enhancement and image geometric neighborhood transformation;

[0095] Step S14: pixel-blocking the standard RGB type frame image set to generate RGB pixel matrix blocks; performing color space acceleration conversion on the RGB pixel matrix blocks to generate a YUV cache data matrix.

[0096] In an embodiment of the present invention, a high-resolution camera is used to capture a video stream of a terminal media device. Ensure that the camera has a suitable frame rate and resolution to meet application requirements. Use a video encoding format (such as H.264) for real-time compression and transmission. Extract each frame of the image from the captured video stream to generate a terminal video frame image set. Perform RGB (red-green-blue-transparent) format conversion on each frame of the image to a standard RGB (red-green-blue) type frame image set to ensure the versatility and compatibility of the image data. Perform the following image preprocessing on the RGB type frame image set: Apply techniques such as Gaussian filtering or median filtering to remove image noise. Use histogram equalization or gamma correction methods to enhance image brightness to enhance image details. Perform geometric transformations such as affine transformation or perspective transformation to meet subsequent processing requirements, generate standard RGB type frame images, and ensure that the image maintains high quality after processing. Perform pixel block division on the standard RGB type frame image set to generate RGB pixel matrix blocks. A uniform block division method can be used to ensure that the size of each block is consistent (for example, 8x8 or 16x16 pixels). Perform color space accelerated conversion on RGB pixel matrix blocks to YUV (luminance-chrominance-chrominance) format to generate a YUV buffer data matrix. This conversion can use common conversion formulas or library functions, such as the cvtColor function in OpenCV.

[0097] Preferably, performing color space accelerated conversion on the RGB pixel matrix block includes:

[0098] The floating point conversion formula is defined for the RGB pixel matrix block to obtain the floating point conversion calculation formula, where the floating point conversion calculation formula is as follows:

[0099] Y=0.299·R+0.587·G+0.114·B;

[0100] U=-0.14713·R-0.28886·G+0.436·B;

[0101] V=0.615·R-0.51499·G-0.10001·B;

[0102] Where Y represents brightness, U represents blue chromaticity component, V represents red chromaticity component, R represents the red component of each pixel in the image, G represents the green component of each pixel in the image, and B represents the blue component of each pixel in the image.

[0103] The component scale transformation is performed on the RGB pixel matrix block to generate an RGB pixel digital matrix; the RGB pixel digital matrix is ​​parallelized to an integer color space conversion using a floating point conversion calculation formula to generate YUV component data after parallel processing; the YUV component data after parallel processing is cached and stored to generate a YUV cache data matrix.

[0104] In the embodiment of the present invention, the following floating-point conversion calculation formula is defined according to the requirements of color space conversion: brightness (Y) calculation: Y=0.299·R+0.587·G+0.114·B; blue chromaticity component (U) calculation: U=-0.14713·R-0.28886·G+0.436·B; red chromaticity component (V) calculation: V=0.615·R-0.51499·G-0.10001·B; wherein YYY represents brightness, UUU represents blue chromaticity component, VVV represents red chromaticity component, RRR, GGG and BBB respectively represent the red, green and blue components of each pixel in the image. Perform component scaling transformation on the RGB pixel matrix block, normalize the RGB value of each pixel to a floating point number (for example, map the integer range of 0-255 to the floating point range of 0.0-1.0), and generate an RGB pixel digital matrix, in which each element is a color component represented by a floating point number. Use a parallel computing framework (such as OpenMP, CUDA, or OpenCL) to parallelize the RGB pixel digital matrix. Each processing unit is responsible for calculating the Y, U, and V values ​​of the pixel it is responsible for, reducing the calculation time. Use the floating point conversion calculation formula to parallelize the RGB pixel digital matrix to convert the integer color space to generate a YUV component data matrix. Store the parallel processed YUV component data in a cache for subsequent processing or transmission. Cache storage can be performed in the form of a memory mapped file, a cache array, or a data stream to generate a YUV cache data matrix, in which each element corresponds to the Y, U, and V components of each pixel in the image.

[0105] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0106] Step S21: Sending the YUV storage data matrix to the DSP internal processing unit through the DSP hardware interface to initialize the encoder and generate encoder initialization parameter setting data, wherein the encoder initialization includes resolution initialization, bit rate control mode initialization, frame rate initialization and key frame interval initialization;

[0107] Step S22: based on the encoder initialization parameter setting data, the YUV storage data matrix is ​​subjected to key frame selection to obtain YUV key video frame data and YUV non-key video frame data; the YUV key video frame data is subjected to intra-frame encoding to generate YUV key frame encoded data; the YUV non-key video frame data is subjected to inter-frame encoding to generate YUV non-key frame encoded data;

[0108] Step S23: transmitting the YUV key frame coded data and the YUV non-key frame coded data to the upper layer CPU data, thereby generating YUV upper layer transmission data;

[0109] Step S24: Slice the YUV upper layer transmission data to generate YUV upper layer slice data; add RTP packet header information to the YUV upper layer slice data to generate RTP packet header information data; control the packing sequence of the YUV upper layer slice data according to the RTP packet header information data to generate an RTP data packet; send the RTP data packet to the network to generate an RTP network sending data packet.

[0110] In an embodiment of the present invention, the YUV storage data matrix is ​​sent to the internal processing unit of the DSP through the DSP (digital signal processor) hardware interface. Data transmission can be carried out using interface protocols such as SPI, I2C or UART to ensure efficient and reliable data transmission. After the transmission is completed, the DSP receives the data and performs initialization processing. According to the size of the input YUV data, the resolution parameter of the encoder is set. Select an appropriate bit rate control mode (such as CBR, VBR) and set the corresponding target bit rate. According to application requirements, the frame rate parameters of the encoder are set (for example, 30fps or 60fps). Set the interval of the key frame (for example, one key frame every 30 frames) to control the compression efficiency and quality of the video, and generate encoder initialization parameter setting data for subsequent processing. Based on the encoder initialization parameter setting data, the YUV storage data matrix is ​​selected for key frames. Analysis can be performed based on the degree of change between frames, motion estimation or image complexity. The selected frames are divided into YUV key video frame data and YUV non-key video frame data. The YUV key video frame data is intra-encoded, and the encoding standard (such as H.264 or H.265) is used to generate YUV key frame encoded data. Intra-frame encoding uses the pixel information in the same frame to reduce redundancy. The YUV non-key video frame data is inter-encoded, and the difference between the previous and next frames is used to generate YUV non-key frame encoded data. Inter-frame encoding effectively reduces the bit rate. The YUV key frame encoded data and the YUV non-key frame encoded data are transmitted to the upper CPU data layer. DMA (direct memory access) or interrupt mode can be used for efficient transmission to ensure the rapid transfer of data and generate YUV upper layer transmission data for use in subsequent steps. The YUV upper layer transmission data is data fragmented to ensure that the size of each data slice meets the network transmission requirements (for example, the size of each slice does not exceed 1500 bytes), and the YUV upper layer fragmented data is generated for subsequent packaging. According to the RTP (Real-time Transport Protocol) specification, RTP packet header information is added to the YUV upper layer fragmented data. The header information includes timestamp, sequence number, load type, etc. to ensure the effective transmission and decoding of data and generate RTP header information data. According to the RTP header information data, the YUV upper layer fragment data is packaged and sequenced to ensure the order and integrity of the data packet, prevent packet loss or retransmission, generate RTP data packets, and prepare for network transmission. The RTP data packet is sent through a network interface (such as Ethernet, Wi-Fi, etc.) and transmitted using the UDP protocol to ensure low latency, generate RTP network transmission data packets, and complete real-time transmission of video data.

[0111] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0112] Step S31: receiving data packets sent from the RTP network based on the remote device, generating a local cache RTP data packet; performing RTP packet header parsing on the local cache RTP data packet, generating RTP packet header parsing data;

[0113] Step S32: reorganizing the data segments of the locally cached RTP data packets according to the RTP packet header parsing data to generate a video reorganized bit stream;

[0114] Step S33: performing packet loss rate detection on the video reassembled bit stream, and when the packet loss rate of the video reassembled bit stream is greater than or equal to a preset packet loss rate threshold, a data packet retransmission request is made; when the packet loss rate of the video reassembled bit stream is less than the preset packet loss rate threshold, packet loss compensation is performed on the video reassembled bit stream to generate a depacketized video stream;

[0115] Step S34: component decoding is performed on the unpacked video stream to generate RGB data to be rendered.

[0116] In an embodiment of the present invention, a data packet sent by an RTP network is received by using a network interface (such as Ethernet or Wi-Fi) based on a remote device. Socket programming (such as BSD socket) can be used to receive data packets. The received data packet is stored in a local cache to generate a local cache RTP data packet. The RTP packet header of the locally cached RTP data packet is parsed to extract key information, including timestamp, sequence number, load type, synchronization source identifier (SSRC), etc., and RTP packet header parsing data is generated for subsequent use. According to the RTP packet header parsing data, each data fragment is identified and sorted. Ensure that the order of the fragments is consistent with the original sending order. Merge the reassembled data fragments to generate a video reassembled bitstream. This step requires processing lost or disordered fragments to ensure that the generated bitstream is complete. The packet loss rate of the video reassembled bitstream is detected. The calculation formula is as follows: packet loss rate = number of lost packets / total number of sent packets × 100%, set a preset packet loss rate threshold (such as 5%), and determine whether the packet loss rate meets the conditions. When the packet loss rate of the video reassembled bitstream is greater than or equal to the preset packet loss rate threshold, a packet retransmission request is sent to the remote device. The request contains the sequence number information of the lost packet so that the remote device can resend the missing data packet. When the packet loss rate of the video reassembled bitstream is less than the preset packet loss rate threshold, packet loss compensation is performed. The following techniques can be used: Recover lost data through redundant data. Use the existing frame data to estimate the content of the lost frame and generate an unpacked video stream to ensure smooth playback. Perform component decoding on the unpacked video stream, use the corresponding decoding algorithm (such as H.264, H.265) to decode the compressed video data into RGB format, and generate RGB data to be rendered for subsequent image processing and display.

[0117] Preferably, step S33 includes the following steps:

[0118] Step S331: marking the RTP packet sequence of the video reassembled bit stream to obtain RTP packet sequence data; performing continuity detection on the RTP packet sequence data, and when the RTP packet sequence data is discontinuous, calculating the packet loss rate of the video reassembled bit stream using a packet loss rate detection formula to obtain the packet loss rate of the video reassembled bit stream;

[0119] The formula for calculating the packet loss rate is as follows:

[0120] L = (T ÷ K) × 100%;

[0121] In the formula, L represents the packet loss rate, T represents the number of lost packets, and K represents the expected number of packets;

[0122] Step S332: when the packet loss rate of the video reassembled bit stream is greater than or equal to the preset packet loss rate threshold, a data packet retransmission request is made; when the packet loss rate of the video reassembled bit stream is less than the preset packet loss rate threshold, a packet loss sequence interval is marked on the video reassembled bit stream to obtain a packet loss sequence interval;

[0123] Step S333: extracting forward error correction code from the video reconstructed bit stream according to the packet loss sequence interval to obtain a video forward error correction code; inserting a packet loss sequence into the packet loss sequence interval according to the video forward error correction code to generate a packet loss sequence insertion interval;

[0124] Step S334: reconstructing video frames of the video reconstructed bit stream by inserting a packet loss sequence into an interval, thereby generating a reconstructed video bit stream; and depacketizing the reconstructed video bit stream, thereby generating a depacketized video stream.

[0125] In an embodiment of the present invention, RTP packet sequence data is generated by extracting the sequence number of the RTP data packet in the video reassembled bit stream. This data includes the sequence number of each data packet for subsequent continuity detection. The continuity of the sequence number is checked by traversing the RTP packet sequence data. When it is found that the sequence number is missing (i.e., discontinuous), the number of missing packets is recorded. The packet loss rate is calculated using the packet loss rate calculation formula: L = (T ÷ K) × 100%; where L represents the packet loss rate, T represents the number of lost packets, and K represents the expected number of packets; it is determined whether the calculated packet loss rate L is greater than or equal to the preset packet loss rate threshold. When the packet loss rate is greater than or equal to the threshold, a data packet retransmission request is made to request the missing data packet from the transmitter. When the packet loss rate is less than the preset threshold, based on the result of the continuity detection, the interval of the lost packet sequence is marked, which can be achieved by recording the start and end positions of the missing sequence number to generate a lost packet sequence interval. According to the information of the packet loss sequence interval, the corresponding forward error correction code (FEC) is extracted from the video reconstructed bit stream. These FECs are usually redundant information added when the data packet is sent, which is used to recover the lost data packet. The extracted forward error correction code is used to insert the marked packet loss sequence interval. The specific process can be to insert redundant data to reconstruct the lost frame and generate a packet loss sequence insertion interval. The video reconstructed bit stream is processed through the packet loss sequence insertion interval to reconstruct the video frame, which usually involves encoding and combining the inserted frame data to generate a continuous video stream. The reconstructed video bit stream is unpacked, and the corresponding unpacking algorithm (such as parsing the RTP packet) is used to extract the final video frame data to generate an unpacked video stream.

[0126] As an example of the present invention, refer to Figure 4 As shown, in this example, step S4 includes:

[0127] Step S41: performing initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; performing boundary box segmentation on the initial media rendering screen to obtain a media rendering core screen;

[0128] Step S42: Perform user visual focus detection on the media rendering core picture to generate user sight track data; map the user sight track data to the media rendering core picture to identify the key picture area, and generate media rendering key picture area data and media rendering non-key picture area data;

[0129] Step S43: performing local fine rendering setting on the media rendering key picture area data to generate local fine rendering data; performing global fuzzy rendering setting on the media rendering non-key picture area data to generate global fuzzy rendering data; performing mixed rendering on the initial media rendering picture according to the local fine rendering data and the global fuzzy rendering data, thereby generating mixed rendering picture data;

[0130] Step S44: adjusting the rendering effect of the mixed rendering picture data to generate rendering adjustment picture data to perform a media picture optimization rendering operation.

[0131] In an embodiment of the present invention, an initial media rendering screen is generated by initially rendering the unpacked video stream using real-time rendering technology (such as OpenGL or DirectX). The screen is the first frame of the video stream and presents the basic content of the video. In the initial media rendering screen, a bounding box detection algorithm (such as a target detection algorithm based on a convolutional neural network) is applied to identify different objects or areas in the screen, generate a bounding box, and divide the screen into multiple areas, from which a media rendering core screen is extracted, which is usually the main content area that the user is concerned about. Eye tracking technology is used to detect the user's visual focus by analyzing the user's eye movement data (such as gaze point, gaze time, etc.), and generate user sight track data, which records the key points that the user is looking at during the viewing process. The user's sight track data is mapped to the media rendering core screen, and the user's attention focus area is visualized using heat map technology. Identify the key areas and non-key areas in media rendering, and generate media rendering key screen area data and media rendering non-key screen area data for subsequent rendering settings. For the data of the key picture area of ​​media rendering, local fine rendering settings are performed, and advanced rendering techniques (such as anti-aliasing, lighting, and shadows) are used to enhance the details and clarity of the area, and local fine rendering data is generated to ensure that the user's visual focus area presents the best effect. For the data of the non-key picture area of ​​media rendering, global fuzzy rendering settings are performed, and a fuzzy algorithm (such as Gaussian blur) is applied to reduce the details of the area to highlight the core picture that the user is concerned about, and global fuzzy rendering data is generated to ensure that the user's attention is not disturbed. Based on the local fine rendering data and the global fuzzy rendering data, the initial media rendering picture is mixed and rendered. Using a weighted mixing algorithm, the two rendering data are synthesized to generate the final mixed rendering picture data. The rendering effect of the mixed rendering picture data is adjusted, and color correction and contrast enhancement technology are used to improve the overall effect of the picture to generate rendering adjustment picture data. After this data is optimized, it is ready to perform the media picture optimization rendering job.

[0132] Preferably, step S42 includes the following steps:

[0133] Step S421: using a camera to collect the user gaze position on the media rendering core screen to obtain user gaze position information data; performing time series analysis on the user gaze position information data to generate user gaze position time series data; constructing a user gaze trajectory according to the user gaze position information data and the user gaze position time series data to obtain user gaze trajectory data;

[0134] Step S422: Calculate the pixel density of the user's sight trajectory data to obtain pixel density distribution data; generate a heat map based on the pixel density distribution data to obtain a user's sight heat map; identify and mark the sight focus area of ​​the user's sight heat map based on a preset density threshold to generate focus area marking data;

[0135] Step S423: performing screen coordinate conversion on the focus area mark data to generate focus area screen coordinate conversion data; mapping the focus area screen coordinate conversion data to the media rendering core screen to perform screen key area classification to generate media rendering key screen area data;

[0136] Step S424: using the media rendering key picture area data to divide the media rendering core picture into non-key picture areas, thereby generating media rendering non-key picture area data.

[0137] In an embodiment of the present invention, a camera (such as a high-resolution camera or an eye tracker) is used to capture the gaze position of a user on a core screen of media rendering. The user's gaze point in each frame is recorded to generate user gaze position information data, including a timestamp and corresponding screen coordinates. The user's gaze position information data is subjected to a time series analysis, and the user's gaze position time series data is generated by calculating the time interval between the gaze points. The data provides a time context for the subsequent eye track construction and helps analyze the user's gaze changes. The user's eye track is constructed according to the user's gaze position information data and the user's gaze position time series data. An interpolation algorithm is used to smooth the connection between the gaze points to form continuous eye track data to characterize the user's viewing path. The user's eye track data is subjected to a pixel density calculation, and the number of gaze points in each screen area is counted to generate pixel density distribution data. The screen area is divided into rectangular grids, and the density of gaze points in each area is calculated. Based on the pixel density distribution data, a user's eye hotspot map is generated by a color mapping algorithm (such as a heat map algorithm). The hotspot map uses color depth to indicate the concentration of the gaze points, and the dark area indicates the part that the user pays more attention to. According to the preset density threshold, analyze the user's sightline heat map, identify areas with density higher than the threshold, mark them, and generate focus area marking data, which represents the area of ​​the screen that the user is most interested in. Perform screen coordinate conversion on the focus area marking data and convert it into a coordinate system consistent with the media rendering core screen. Ensure that the focus area can be accurately mapped to the corresponding media screen for subsequent processing. Map the focus area screen coordinate conversion data to the media rendering core screen, classify the screen into key areas according to the marking information, and generate media rendering key screen area data. These areas are considered to be the main focus of the user. Using the media rendering key screen area data, divide other areas in the media rendering core screen into non-key areas and generate media rendering non-key screen area data. These areas are usually the areas that users pay less attention to.

[0138] Preferably, step S43 includes the following steps:

[0139] Step S431: performing local fine rendering settings on the media rendering key picture area data, thereby generating a local fine rendering setting parameter data set, wherein the local fine rendering settings include local anti-aliasing and local material map detail enhancement; performing GPU accelerated rendering on the media rendering key picture area data using the local fine rendering setting parameter data set, thereby generating local fine rendering data;

[0140] Step S432: performing global blur rendering settings on the media rendering non-critical picture area data, thereby generating a global fine rendering setting parameter data set, wherein the global fine rendering settings include global dynamic blur and global resolution reduction; performing parallel blur accelerated rendering on the media rendering non-critical picture area data using the global fine rendering setting parameter data set, thereby generating global fine rendering data;

[0141] Step S433: Integrate the local fine rendering data and the global fuzzy rendering data into the initial media rendering picture to perform rendering area image fusion, thereby generating mixed rendering picture data.

[0142] In an embodiment of the present invention, local fine rendering settings are performed according to the media rendering key picture area data, and these settings include: reducing edge jaggedness and improving image quality through multi-sampling anti-aliasing (MSAA) or fast approximate anti-aliasing (FXAA) algorithms. High-resolution normal maps and displacement maps are applied to enhance surface details and make the material performance more realistic. The above rendering setting parameters are integrated into a local fine rendering setting parameter data set, which includes all required settings and parameters. The local fine rendering setting parameter data set is used to accelerate rendering through a graphics processing unit (GPU). During rendering, the GPU will process multiple pixels in parallel to improve rendering efficiency and generate high-quality local fine rendering data. According to the media rendering non-key picture area data, global blur rendering settings are performed, and these settings include: applying motion blur technology to blur dynamic scenes according to the moving speed of objects to enhance the sense of motion. The resolution of non-key picture areas is reduced to reduce the computational burden while maintaining the smoothness of visual effects. The global blur rendering settings are integrated into a global fine rendering setting parameter data set, which includes all parameters required for global blur. Using the global fine rendering setting parameter data set, parallel blur accelerated rendering is performed on the non-critical area data of media rendering. GPU is used for parallel processing to accelerate the rendering process, generate global fine rendering data, and optimize rendering performance while maintaining the blur effect. The generated local fine rendering data and global blur rendering data are integrated into the initial media rendering picture. Using image fusion technology, the two rendering results are combined to ensure that local details are prominent and non-critical areas maintain a natural blur transition. Through the fusion operation, the final mixed rendering picture data is generated. This data will serve as the basis for subsequent rendering effect adjustments to provide a high-quality final picture.

[0143] In this specification, a media picture rendering optimization system based on artificial intelligence is provided, which is used to execute the above-mentioned media picture rendering optimization method based on artificial intelligence. The media picture rendering optimization system based on artificial intelligence includes:

[0144] The color gamut conversion module is used to obtain the terminal video stream; perform RGB format conversion on the terminal video stream to generate an RGB type frame image set; perform pixel block division on the RGB type frame image set to generate an RGB pixel matrix block; perform color space acceleration conversion on the RGB pixel matrix block to generate a YUV cache data matrix;

[0145] The video frame transmission module is used to send the YUV storage data matrix to the DSP internal processing unit for key frame selection to obtain YUV key video frame data and YUV non-key video frame data; perform intra-frame encoding on the YUV key video frame data to generate YUV key frame encoded data; perform inter-frame encoding on the YUV non-key video frame data to generate YUV non-key frame encoded data; package the YUV key frame encoded data and the YUV non-key frame encoded data for network transmission to generate RTP network transmission data packets;

[0146] The packet loss compensation module is used to reassemble the data segments of the data packets sent by the RTP network based on the remote device to generate a video reassembled bit stream; perform packet loss compensation on the video reassembled bit stream to generate an unpacked video stream; perform component decoding on the unpacked video stream to generate RGB data to be rendered;

[0147] The rendering acceleration module is used to perform initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; perform user visual focus detection on the unpacked video stream to generate user sight track data; map the user sight track data to the media rendering core screen for layered rendering settings to generate local fine rendering data and global fuzzy rendering data; perform mixed rendering and execution adjustment on the initial media rendering screen according to the local fine rendering data and the global fuzzy rendering data, thereby generating rendering adjustment screen data to execute media screen optimization rendering operations.

[0148] The beneficial effect of the present invention is that through pixel segmentation and accelerated conversion, efficient format conversion of video frames is achieved, providing a fast-response data basis for subsequent processing. The conversion from RGB to YUV reduces data redundancy and saves bandwidth for transmission and processing. The combination of intra-frame and inter-frame coding can significantly reduce the data volume of the video stream while maintaining the high quality of the key frames. The packaged data can be efficiently transmitted in the network, improving the transmission performance of the real-time video stream. Through packet loss compensation, the data loss occurring during video transmission is effectively reduced, ensuring the integrity of the video stream. Component decoding ensures accurate recovery of RGB data, laying the foundation for subsequent rendering. Through user visual focus detection technology, the system can automatically identify the user's gaze area, optimize the rendering effect of the key area, and reduce the processing burden of the non-key area. Global fuzzy rendering reduces the calculation pressure of the non-key area and improves the overall system performance. Hybrid rendering makes the entire picture have both clear key areas and smooth visual transitions, which improves the user's perception. Using intra-frame coding, inter-frame coding and RTP package transmission can reduce bandwidth consumption and improve the reliability of network transmission. Through focus detection and local rendering optimization, the clarity of key areas during user viewing is ensured, enhancing the interactive experience. Through color space accelerated conversion and GPU accelerated rendering, computing resources are effectively utilized to ensure efficient video stream processing and rendering operations. Therefore, the present invention improves the efficiency of media screen rendering by optimizing data processing, transmission and rendering, combined with user visual focus analysis.

[0149] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0150] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A media image rendering optimization method based on artificial intelligence, characterized in that: The following steps are involved: Step S1: Obtain terminal video stream; Convert the terminal video stream into RGB format to generate an RGB type frame image set; Divide the RGB type frame image set into pixels and generate RGB pixel matrix blocks; Perform color space acceleration conversion on RGB pixel matrix blocks to generate YUV buffer data matrix; Step S2: Send the YUV storage data matrix to the DSP internal processing unit for key frame selection to obtain YUV key video frame data and YUV non-key video frame data; perform intra-frame encoding on the YUV key video frame data to generate YUV key frame encoded data; perform inter-frame encoding on the YUV non-key video frame data to generate YUV non-key frame encoded data; perform network packaging and transmission on the YUV key frame encoded data and the YUV non-key frame encoded data to generate an RTP network transmission data packet; Step S3: reassembling data segments of data packets sent over the RTP network based on the remote device to generate a video reassembled bit stream; Perform packet loss compensation on the video reassembled bit stream to generate a depacketized video stream; Decode the unpacked video stream component-wise to generate RGB data to be rendered; Step S4: performing initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; Perform user visual focus detection on the unpacked video stream to generate user eye tracking data; Map the user's sight trajectory data to the media rendering core screen for layered rendering settings to generate local fine rendering data and global fuzzy rendering data; The initial media rendering picture is adjusted by hybrid rendering according to the local fine rendering data and the global blurred rendering data, so as to generate rendering adjustment picture data to perform the media picture optimization rendering operation.

2. The method for optimizing media image rendering based on artificial intelligence according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: using a camera to capture a video stream of a terminal media device to obtain a terminal video stream; Step S12: Decomposing the terminal video stream into video frame images to obtain a terminal video frame image set; performing RGB format conversion on the terminal video frame image set to generate an RGB type frame image set; Step S13: performing image preprocessing on the RGB type frame image set to generate a standard RGB type frame image, wherein the image preprocessing includes image filtering, image brightness enhancement and image geometric neighborhood transformation; Step S14: pixel-blocking the standard RGB type frame image set to generate RGB pixel matrix blocks; performing color space acceleration conversion on the RGB pixel matrix blocks to generate a YUV cache data matrix.

3. The method for optimizing media image rendering based on artificial intelligence according to claim 2, characterized in that: The color space acceleration conversion of RGB pixel matrix blocks includes: The floating point conversion formula is defined for the RGB pixel matrix block to obtain the floating point conversion calculation formula, where the floating point conversion calculation formula is as follows: Y=0.299·R+0.587·G+0.114·B; U=-0.14713·R-0.28886·G+0.436·B; V=0.615·R-0.51499·G-0.10001·B; Where Y represents brightness, U represents blue chromaticity component, V represents red chromaticity component, R represents the red component of each pixel in the image, G represents the green component of each pixel in the image, and B represents the blue component of each pixel in the image. The component scale transformation is performed on the RGB pixel matrix block to generate an RGB pixel digital matrix; the RGB pixel digital matrix is ​​parallelized to an integer color space conversion using a floating point conversion calculation formula to generate YUV component data after parallel processing; the YUV component data after parallel processing is cached and stored to generate a YUV cache data matrix.

4. The method for optimizing media image rendering based on artificial intelligence according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: Sending the YUV storage data matrix to the DSP internal processing unit through the DSP hardware interface to initialize the encoder and generate encoder initialization parameter setting data, wherein the encoder initialization includes resolution initialization, bit rate control mode initialization, frame rate initialization and key frame interval initialization; Step S22: based on the encoder initialization parameter setting data, the YUV storage data matrix is ​​subjected to key frame selection to obtain YUV key video frame data and YUV non-key video frame data; the YUV key video frame data is subjected to intra-frame encoding to generate YUV key frame encoded data; the YUV non-key video frame data is subjected to inter-frame encoding to generate YUV non-key frame encoded data; Step S23: transmitting the YUV key frame coded data and the YUV non-key frame coded data to the upper layer CPU data, thereby generating YUV upper layer transmission data; Step S24: Slice the YUV upper layer transmission data to generate YUV upper layer slice data; add RTP packet header information to the YUV upper layer slice data to generate RTP packet header information data; control the packing sequence of the YUV upper layer slice data according to the RTP packet header information data to generate an RTP data packet; send the RTP data packet to the network to generate an RTP network sending data packet.

5. The method for optimizing media image rendering based on artificial intelligence according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: receiving data packets sent from the RTP network based on the remote device, generating a local cache RTP data packet; performing RTP packet header parsing on the local cache RTP data packet, generating RTP packet header parsing data; Step S32: reorganizing the data segments of the locally cached RTP data packets according to the RTP packet header parsing data to generate a video reorganized bit stream; Step S33: performing packet loss rate detection on the video reassembled bit stream, and when the packet loss rate of the video reassembled bit stream is greater than or equal to a preset packet loss rate threshold, a data packet retransmission request is made; when the packet loss rate of the video reassembled bit stream is less than the preset packet loss rate threshold, packet loss compensation is performed on the video reassembled bit stream to generate a depacketized video stream; Step S34: component decoding is performed on the unpacked video stream to generate RGB data to be rendered.

6. The method for optimizing media image rendering based on artificial intelligence according to claim 5, characterized in that: Step S33 includes the following steps: Step S331: marking the RTP packet sequence of the video reassembled bit stream to obtain RTP packet sequence data; performing continuity detection on the RTP packet sequence data, and when the RTP packet sequence data is discontinuous, calculating the packet loss rate of the video reassembled bit stream using a packet loss rate detection formula to obtain the packet loss rate of the video reassembled bit stream; The formula for calculating the packet loss rate is as follows: L = (T ÷ K) × 100%; In the formula, L represents the packet loss rate, T represents the number of lost packets, and K represents the expected number of packets; Step S332: when the packet loss rate of the video reassembled bit stream is greater than or equal to the preset packet loss rate threshold, a data packet retransmission request is made; when the packet loss rate of the video reassembled bit stream is less than the preset packet loss rate threshold, a packet loss sequence interval is marked on the video reassembled bit stream to obtain a packet loss sequence interval; Step S333: extracting forward error correction code from the video reconstructed bit stream according to the packet loss sequence interval to obtain a video forward error correction code; inserting a packet loss sequence into the packet loss sequence interval according to the video forward error correction code to generate a packet loss sequence insertion interval; Step S334: reconstructing video frames of the video reconstructed bit stream by inserting a packet loss sequence into an interval, thereby generating a reconstructed video bit stream; and depacketizing the reconstructed video bit stream, thereby generating a depacketized video stream.

7. The method for optimizing media image rendering based on artificial intelligence according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; performing boundary box segmentation on the initial media rendering screen to obtain a media rendering core screen; Step S42: Perform user visual focus detection on the media rendering core picture to generate user sight track data; map the user sight track data to the media rendering core picture to identify the key picture area, and generate media rendering key picture area data and media rendering non-key picture area data; Step S43: performing local fine rendering setting on the media rendering key picture area data to generate local fine rendering data; performing global fuzzy rendering setting on the media rendering non-key picture area data to generate global fuzzy rendering data; performing mixed rendering on the initial media rendering picture according to the local fine rendering data and the global fuzzy rendering data, thereby generating mixed rendering picture data; Step S44: adjusting the rendering effect of the mixed rendering picture data to generate rendering adjustment picture data to perform a media picture optimization rendering operation.

8. The method for optimizing media image rendering based on artificial intelligence according to claim 7, characterized in that: Step S42 includes the following steps: Step S421: using a camera to collect the user gaze position on the media rendering core screen to obtain user gaze position information data; performing time series analysis on the user gaze position information data to generate user gaze position time series data; constructing a user gaze trajectory according to the user gaze position information data and the user gaze position time series data to obtain user gaze trajectory data; Step S422: Calculate the pixel density of the user's sight trajectory data to obtain pixel density distribution data; generate a heat map based on the pixel density distribution data to obtain a user's sight heat map; identify and mark the sight focus area of ​​the user's sight heat map based on a preset density threshold to generate focus area marking data; Step S423: performing screen coordinate conversion on the focus area mark data to generate focus area screen coordinate conversion data; mapping the focus area screen coordinate conversion data to the media rendering core screen to perform screen key area classification to generate media rendering key screen area data; Step S424: using the media rendering key picture area data to divide the media rendering core picture into non-key picture areas, thereby generating media rendering non-key picture area data.

9. The method for optimizing media image rendering based on artificial intelligence according to claim 7, characterized in that: Step S43 includes the following steps: Step S431: performing local fine rendering settings on the media rendering key picture area data, thereby generating a local fine rendering setting parameter data set, wherein the local fine rendering settings include local anti-aliasing and local material map detail enhancement; performing GPU accelerated rendering on the media rendering key picture area data using the local fine rendering setting parameter data set, thereby generating local fine rendering data; Step S432: performing global blur rendering settings on the media rendering non-critical picture area data, thereby generating a global fine rendering setting parameter data set, wherein the global fine rendering settings include global dynamic blur and global resolution reduction; performing parallel blur accelerated rendering on the media rendering non-critical picture area data using the global fine rendering setting parameter data set, thereby generating global fine rendering data; Step S433: Integrate the local fine rendering data and the global fuzzy rendering data into the initial media rendering picture to perform rendering area image fusion, thereby generating mixed rendering picture data.

10. A media image rendering optimization system based on artificial intelligence, characterized in that: Used to execute the artificial intelligence-based media picture rendering optimization method as claimed in claim 1, the artificial intelligence-based media picture rendering optimization system comprises: The color gamut conversion module is used to obtain the terminal video stream; perform RGB format conversion on the terminal video stream to generate an RGB type frame image set; perform pixel block division on the RGB type frame image set to generate an RGB pixel matrix block; perform color space acceleration conversion on the RGB pixel matrix block to generate a YUV cache data matrix; The video frame transmission module is used to send the YUV storage data matrix to the DSP internal processing unit for key frame selection to obtain YUV key video frame data and YUV non-key video frame data; perform intra-frame encoding on the YUV key video frame data to generate YUV key frame encoded data; perform inter-frame encoding on the YUV non-key video frame data to generate YUV non-key frame encoded data; package the YUV key frame encoded data and the YUV non-key frame encoded data for network transmission to generate RTP network transmission data packets; The packet loss compensation module is used to reassemble the data segments of the data packets sent by the RTP network based on the remote device to generate a video reassembled bit stream; perform packet loss compensation on the video reassembled bit stream to generate an unpacked video stream; perform component decoding on the unpacked video stream to generate RGB data to be rendered; The rendering acceleration module is used to perform initial screen rendering on the unpacked video stream to obtain an initial media rendering screen; perform user visual focus detection on the unpacked video stream to generate user sight track data; map the user sight track data to the media rendering core screen for layered rendering settings to generate local fine rendering data and global fuzzy rendering data; perform mixed rendering and execution adjustment on the initial media rendering screen according to the local fine rendering data and the global fuzzy rendering data, thereby generating rendering adjustment screen data to execute media screen optimization rendering operations.

Images