A computer network communication data processing method and system based on artificial intelligence

Abstract

The present invention relates to the field of image communication technology, and specifically provides a computer network communication data processing method and system based on artificial intelligence, including the following steps: performing continuous frame difference calculation on the data of real-time video frames, collecting pixel-level change data between two frames, and obtaining a difference data set. By performing continuous frame difference calculation on real-time video frames, the present invention captures pixel-level change data between two frames, reduces the storage and transmission occupancy of useless data, and alleviates the data processing pressure. The compression of difference data further optimizes the size of data packets, improves the transmission efficiency, and at the same time ensures the integrity of key video content. Analyzing the compressed difference data packets and extracting key visual elements, by focusing on visual key points such as faces and actions, allocating limited transmission resources to important information parts, enhancing the pertinence of the transmission process.

Description

Technical Field

[0001] The present invention relates to the technical field of image communication, and in particular to a method and system for processing computer network communication data based on artificial intelligence. Background Art

[0002] The field of image communication technology is an important branch of computer network communication, mainly studying how to efficiently transmit and process image and video data through computer networks. This field covers the complete chain from image acquisition, encoding and compression, transmission to decoding and display, and involves many aspects such as image compression algorithms, channel transmission optimization, network protocol design, and dynamic adaptation technology.

[0003] However, the existing data processing methods lack a mechanism for judging the importance of image content, resulting in no obvious distinction in transmission priorities between high-value visual elements and background information. This may cause key information to be delayed or discarded under resource-constrained conditions. For the real-time changes in network status, it is difficult for the existing technologies to dynamically adjust the data transmission strategy. Usually, a static transmission order is adopted, and the efficiency decreases when the network delay increases or the bandwidth decreases, affecting the reliability of transmission. In addition, the image resolution adjustment methods are mostly fixed solutions and are not adapted according to the device display capabilities and network conditions. This may cause a processing burden on low-performance devices for high-resolution data, or image detail loss on high-performance devices. These deficiencies limit the adaptability of the existing technologies in complex network scenarios. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies existing in the prior art, and to propose a method and system for processing computer network communication data based on artificial intelligence.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions. A method for processing computer network communication data based on artificial intelligence includes the following steps:

[0006] Perform continuous frame difference calculation on the data of real-time video frames, collect pixel-level change data between two frames, and obtain a difference data set; compress the difference data set to generate compressed difference data packets;

[0007] Based on the compressed difference data packets, analyze the image content, identify key visual elements, including faces or key frames of actions, and generate visual key element identifiers; sort the data packets according to the visual key element identifiers to obtain sorted data packets with priorities;

[0008] Collect real-time network status data, including bandwidth utilization and latency, to obtain network status monitoring results; adjust the data packet transmission strategy according to the network status monitoring results, dynamically determine the transmission order of the sorted data packets with priorities, and generate optimized scheduling data packets;

[0009] The resolution of the scheduling optimization data packet is adjusted, the image resolution is dynamically modified according to the current network conditions and the device display capability, the image size is adjusted, and an adjusted image data packet is generated; the adjusted image data packet is sent to the target device to complete the image data transmission process.

[0010] Preferably, the step of acquiring the difference data set is:

[0011] Collect two frames of continuous image data in the real-time video stream, compare the grayscale value of each pixel one by one, calculate the pixel grayscale difference between the two frames of images, and record the difference change results of all pixels to generate pixel-level difference data;

[0012] Based on the pixel-level difference data, grouping the pixel difference change results, screening the pixel points whose change values ​​exceed the set threshold, and integrating the positions of the pixel points exceeding the set threshold with the difference values ​​to form pixel change data;

[0013] Based on the pixel change data, the change records are stored in a structured manner according to the frame number and the change range, and redundant data are removed to generate a difference data set.

[0014] Preferably, the steps of obtaining the compressed difference data packet are:

[0015] Based on the difference data set, the pixel change data of each frame is partitioned according to a preset block rule, the pixel change characteristics in each block are counted one by one, and the pixel value of each block is coded and analyzed in combination with the change range to generate a block pixel change coding result;

[0016] Based on the block pixel change coding result, the repeated or redundant coding information is removed from the coding content of each block and the data representation structure is readjusted to generate optimized block coding content;

[0017] Based on the optimized block coding content, the coding data of all blocks are integrated, all block coding is rearranged according to the data structure of the difference data set, and stored in a compressed storage form to generate a compressed difference data packet.

[0018] Preferably, the steps of obtaining the visual key element identification are:

[0019] Based on the compressed difference data packet, the frame features in the data packet are extracted frame by frame, a feature vector set of each frame is constructed, and potential geometric transformation information in the image frame is extracted, and a frame feature vector and a geometric transformation matrix are generated by comparing the local offset of pixel features between adjacent frames;

[0020] Calculate the discriminant value of the key visual elements according to the frame feature vector and the geometric transformation matrix. The expression is as follows:

[0021]

[0022] where is the discriminant value of the key visual elements, is the translation value of the th row in the geometric transformation matrix, is the rotation value of the th row in the geometric transformation matrix, is the eigenvalue of the th pixel point in the frame feature vector, is the offset of the corresponding pixel point in the adjacent frame, is the number of rows of the geometric transformation matrix, is the dimension of the feature vector, is the number of offset pixel points in the frame;

[0023] Based on the discriminant value of the key visual elements, analyze all frames, screen the frame with the highest discriminant value, and determine the position of the key visual elements by combining the positioning information in the frame feature vector and the geometric transformation matrix, and generate the visual key element identifier.

[0024] Preferably, the step of obtaining the priority sorting data packet is as follows:

[0025] Based on the visual key element identifier, parse the image frame information in each data packet, extract the corresponding frame number, visual key element type and position data, and obtain the initial priority data of the frame;

[0026] According to the initial priority data of the frame, calculate the priority sorting score of the frame. The expression is as follows:

[0027]

[0028] where is the priority sorting score of the th frame, is the type characteristic value of the visual key element, is the spatial distribution coefficient of the corresponding visual key element, is the eigenvalue of the pixel point in the th frame, is the total number of visual key elements, is the number of pixel points in the frame;

[0029] Based on the priority sorting scores of the frames, all the frames in the data packet are rearranged from high to low in priority, and the frame transmission order of the data packet is reconstructed to obtain a priority sorted data packet.

[0030] Preferably, the step of obtaining the network status monitoring result is as follows:

[0031] Real-time collect the network bandwidth utilization data, calculate the average value of the collected bandwidth utilization within a continuous time window, and record the bandwidth change trend of each time window to generate bandwidth utilization record data;

[0032] Real-time collect the network transmission delay situation, mark the timestamp for each transmission delay data and calculate the delay distribution characteristics within the time window to generate network delay record data;

[0033] Based on the bandwidth utilization record data and the network delay record data, analyze the dynamic change relationship and combine the network operation status at the current time point to perform status determination to obtain the network status monitoring result.

[0034] Preferably, the step of obtaining the scheduling optimization data packet is as follows:

[0035] Based on the network status monitoring result, extract the bandwidth utilization and delay data within the current time window, analyze the transmission requirements of the priority sorted data packet one by one, and combine the size, transmission delay requirements of each data packet and the current network bandwidth status to generate an initial sorting value for data packet transmission;

[0036] According to the initial sorting value of data packet transmission, calculate the transmission priority of the data packet, and the expression is:

[0037]

[0038] Wherein, is the transmission priority of the th data packet, is the bandwidth requirement of the th data packet, is the transmission delay requirement of the th data packet, is the data packet size of the th data packet, is the stability coefficient related to transmission in the current network state, is the bandwidth utilization of the current network state, is the total transmission delay of the current network, is the total number of priority sorted data packets, is the number of parameters of the stability coefficient in the current network state;

[0039] Based on the transmission priority of the data packets, sort the priority-sorted data packets from high to low in terms of priority. At the same time, in combination with the transmission priority of the data packets, reallocate and adjust the transmission order of the data packets with restricted transmission conditions to generate scheduled and optimized data packets.

[0040] Preferably, the step of obtaining the adjusted image data packets is as follows:

[0041] Based on the scheduled and optimized data packets, extract the resolution parameters of each image frame, including the width and height resolutions, and at the same time extract the bandwidth utilization rate in the current network condition and the upper limit value of the resolution of the device display capability to generate an image resolution adaptation parameter set;

[0042] According to the image resolution adaptation parameter set, calculate the resolution adjustment value of each image frame. The expression is:

[0043]

[0044] Where is the resolution adjustment value of the image frame, is the width resolution of the current image frame, is the height resolution of the current image frame, is the maximum supported width resolution of the device display capability, is the maximum supported height resolution of the device display capability, is the current network bandwidth utilization rate, is the refresh rate parameter of the device display capability, is the total number of pixels in the current frame, is the delay correction value in the network condition;

[0045] Based on the resolution adjustment value of the image frame, dynamically adjust the width and height resolutions of each image frame to obtain the adjusted image data packets.

[0046] The present invention provides a computer network communication data processing system, including:

[0047] A video frame differentiation module, which extracts two consecutive frames from a real-time video frame, calculates the pixel changes between the two frames, and obtains a pixel difference data set;

[0048] A data compression and key element recognition module, which performs compression processing on the pixel difference data set to form a compressed difference data packet; based on the compressed difference data packet, recognizes the key visual elements in the image, records the positions and attributes of the key visual elements, and generates visual key element identifiers;

[0049] A data packet priority sorting module, which uses the visual key element identifiers to perform priority sorting on the data packets to generate priority-sorted data packets;

[0050] The network status adaptation and scheduling optimization module monitors the bandwidth and latency of the current network and generates network status monitoring results. Combining the network status monitoring results, it dynamically adjusts the sending order of priority-sorted data packets and creates scheduled optimization data packets.

[0051] The image resolution dynamic adjustment module adjusts the resolution of image data according to the instructions in the scheduled optimization data packet, combines the network conditions and the display capabilities of the target device, and generates adjusted image data packets. It is sent to the target device to complete the image data transmission process.

[0052] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0053] The present invention calculates the continuous frame differences of real-time video frames, captures the pixel-level change data between two frames, reduces the storage and transmission occupancy of useless data, and alleviates the data processing pressure. The compression of the difference data further optimizes the size of the data packets, improves the transmission efficiency, and at the same time ensures the integrity of the key content of the video. Analyzing and extracting key visual elements from the compressed difference data packets, by focusing on visual key points such as faces and actions, the limited transmission resources are concentrated on important information parts, enhancing the pertinence of the transmission process. Prioritizing the data packets with key visual element identifiers enables high-value data to be transmitted preferentially under limited resources, reducing the impact of latency on key content. Real-time collecting and analyzing the network status, dynamically adjusting the data transmission order according to the bandwidth utilization rate and latency conditions, and achieving flexible adaptation to complex network environments. The resolution adjustment step combines the current network conditions and the device display capabilities to dynamically modify the image resolution, which not only reduces the bandwidth requirements but also ensures the display effect on the device side. In this way, data redundancy is reduced, the transmission of key information is more efficient, the adaptive ability to complex network environments is enhanced, and the accuracy of data transmission and the compatibility of display are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a step schematic diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] Please refer to Figure 1 , the present invention provides a technical solution, a computer network communication data processing method based on artificial intelligence, including the following steps:

[0057] Perform continuous frame difference calculation on the data of real-time video frames, collect pixel-level change data between two frames, and obtain a difference data set; compress the difference data set to generate a compressed difference data packet;

[0058] Based on the compressed difference data packets, the image content is analyzed, key visual elements, including face or action key frames, are identified, and visual key element identifiers are generated; the data packets are prioritized according to the visual key element identifiers to obtain prioritized data packets;

[0059] Collect real-time network status data, including bandwidth utilization and latency, to obtain network status monitoring results; adjust data packet transmission strategies based on network status monitoring results, dynamically determine the transmission order of priority-sorted data packets, and generate scheduling-optimized data packets;

[0060] The resolution of the scheduling optimization data packet is adjusted, the image resolution is dynamically modified according to the current network conditions and device display capabilities, the image size is adjusted, and the adjusted image data packet is generated; the adjusted image data packet is sent to the target device to complete the image data transmission process.

[0061] The steps to obtain the difference data set are:

[0062] Collect two frames of continuous image data in the real-time video stream, compare the grayscale value of each pixel one by one, calculate the pixel grayscale difference between the two frames of images, and record the difference change results of all pixels to generate pixel-level difference data;

[0063] Based on the pixel-level difference data, the results of the pixel difference change are grouped and processed, the pixels whose change values ​​exceed the set threshold are screened, and the positions of the pixels exceeding the set threshold and the difference values ​​are integrated to form the pixel change data;

[0064] Based on the pixel change data, the change records are stored in a structured manner according to the frame number and change range, and redundant data are removed to generate a difference data set.

[0065] Specifically, based on the two frames of continuous image data obtained previously, the pixel grayscale value of each frame of the image is first read and compared one by one, and then the grayscale difference of each pixel position is stored in the form of absolute value in the software environment, and a comparison table is established for different ambient lighting conditions and shooting equipment specifications to clarify the grayscale value range. to The actual grayscale value of each pixel is matched with the range one by one. If there is data outside this range, it is marked. Then, all pixel difference results are summarized through a unified operation process and the correlation analysis between adjacent pixels is performed. For example, when the difference between adjacent pixel indexes does not exceed When available, a pre-established local region comparison function can be called to calculate the local mean and variance, and the time difference information of consecutive frames can be combined to form a grayscale difference matrix. To ensure accuracy, image data marked in a known scenario can be used in the training phase to train a simple supervised classification model. The training process of the model obtains true labels by collecting thousands of video clips captured under diverse lighting conditions. For the difference value of each pixel, the least mean square error method is used to calculate the loss of the model and the weights are continuously adjusted through gradient descent, thereby forming a final model that can identify abnormal difference values. Subsequently, during the application phase, the real-time collected grayscale difference data is compared with the parameters of the model, unreasonable errors are screened out, and an intermediate difference matrix record is generated. After unified integration, summary data reflecting the pixel difference distribution between frames is obtained, and finally pixel-level difference data is generated.

[0066] Based on the pixel-level difference data obtained previously, first, the difference values of each pixel point are sorted and the position indexes are marked, and then they are arranged in ascending order of the difference magnitude calculated point by point. A reference line is set for each pixel difference value, and the initial value of the reference line is set based on a large amount of experimental data. For example, about thirty thousand frames of images are collected in the test phase, the difference distribution of each pixel in a typical motion scenario is statistically analyzed, and the average difference mean and variance are calculated. The result obtained by adding twice the variance to the mean is defined as the threshold. If the difference value of a certain pixel point exceeds the threshold, it is regarded as a significant change point. At this time, its position and difference value will also be integrated into a special data table. The integration process includes coordinate mapping and grayscale difference analysis. For the determined pixel points exceeding the threshold, their position changes on the time axis will be additionally retrieved. If a high difference continuously appears in multiple frames, it is recorded as a stable change point, otherwise it is regarded as a random jitter point. In this way, the position information and difference value summaries of multiple batches of stable change points and random jitter points are obtained respectively, and finally pixel change data is formed.

[0067] Based on the pixel change data obtained previously, a corresponding relationship is established between the frame numbers and change ranges of all pixel points, and they are batch-recorded. During the recording process, a redundancy determination flag is set for each pixel. If it is found that the pixel point has repeated marks and similar difference magnitudes in adjacent frames, a repeatability determination is made for the pixel before storage. If the number of repetitions exceeds three, the information with the same position and similar difference magnitudes is merged into a single record, thereby compressing redundant numerical items. Subsequently, the data is structured and configured in the software in the order of frame numbers. The remaining pixel change entries after redundancy merging are arranged into a retrievable list, and on this basis, an associable frame index and difference value mapping relationship are provided for subsequent image analysis or processing, and finally a difference data set is generated.

[0068] The steps for obtaining the compressed difference data packet are as follows:

[0069] Based on the difference data set, partition the pixel change data of each frame according to a preset block division rule, statistically analyze the pixel change characteristics in each block one by one, and combine the change range to perform coding analysis on the pixel values of each block to generate a block pixel change coding result;

[0070] Based on the block pixel change coding result, remove duplicate or redundant coding information from the coding content of each block and readjust the data representation structure to generate optimized block coding content;

[0071] Based on the optimized block coding content, integrate the coding data of all blocks, rearrange all block codings according to the data structure of the difference data set, and store them in a compressed storage form to generate a compressed difference data packet.

[0072] Specifically, based on the previously obtained difference data set, partition the pixel change data of each frame using a pre-defined block division rule. First, read the pixel positions and corresponding gray level differences involved in each frame and mark the frame number and coordinate range in the data structure. Then, divide the entire frame image into several adjacent or overlapping sub-regions according to the pre-determined block dimension. Subsequently, retrieve the change amplitude of all pixels within each sub-region and establish a matching list to compare whether it conforms to the change distribution characteristics within the block. To determine the overall change situation of the block, multiple sets of sample frames will be collected during the test phase and the pixel change mean and variance of each sub-region will be calculated. These statistical values will be set as the benchmark range. If the pixel fluctuation during operation exceeds the benchmark range and exceeds an empirically set threshold T1, for example, T1 is established by adding three times the variance to the average change amount obtained from the statistical results in a large-scale motion picture scene. When the change amplitude of any pixel is greater than T1, the block where it is located will be marked as a high-change area. Subsequently, perform multi-level division according to the average change intensity of the pixels within the block and analyze the gray level change pattern point by point. For example, count the frequency of all pixels exceeding the threshold within each block and compare this information with known motion or lighting scene labels. If the corresponding block shows a large range of differences in multiple frames, it will be recorded as a stable high-change block; otherwise, it will be independently coded as an instantaneous fluctuation block. Then, during the summary phase, perform coding analysis on the pixel values according to the change range of each block. This analysis process first performs unsigned processing on the pixel gray level differences and rearranges them in the form of short integers or fixed-length bit widths, and then generates coding identifiers one by one, so as to quickly locate the high-change area position and match its difference information during subsequent retrieval. After the above processing, obtain the block pixel change coding data of each sub-region, and finally generate a block pixel change coding result.

[0073] Based on the block pixel change coding results generated previously, check each coding record of each sub-region one by one to see if there are duplicate codings or highly similar codings, and clarify the correspondence between the pixel coordinate index and the gray-scale change amplitude during the checking process. For example, if both the coordinate index and the gray-scale difference of two adjacent records in the coding list are within a range where the difference does not exceed 3, it is determined as duplicate or redundant information. The value of the corresponding threshold can be determined according to the statistical results of the coding overlap rate in the initial experiment. For example, when observing that 3 is the best superposition range in a motion scenario of 10,000 frames, this 3 is used as the threshold T2 for judgment. Once redundancy is confirmed, it is immediately merged into a single coding identifier, and the overlapping segment is marked as merged in the internal record, so as to reduce the number of duplicate codings and shrink the data scale. Subsequently, when adjusting the data representation structure, the sub-region boundaries during block division will be referred to. Within the same sub-region, all coding entries after redundancy merging are sorted according to the gray-scale difference from small to large or according to the occurrence frequency. During this period, a hierarchical table can be loaded to place pixels with a difference amplitude within a certain range into the same sub-category, so as to maintain the hierarchy and retrievability of the data structure. For single-pixel differences with extremely low occurrence frequencies, independent annotation is performed. After sorting, the new coding order is written into the unified block coding overview list, and necessary information such as block index and pixel difference interval is retained in this list. Finally, an encoding summary after removing redundancy and re-adjusting the representation structure is obtained, and optimized block coding content is generated.

[0074] Based on the optimized block coding content obtained previously, summarize the coding records of all sub-regions and perform a secondary arrangement according to the original frame order and pixel index of the difference data set. During this process, a new external index will be added to each block coding to indicate its spatial position and block serial number in the frame, and check whether there is intersection or overlap between each block according to the overall data structure preset by the difference data set. If it is found that two adjacent blocks contain similar coding positions in the same frame and the pixel change amplitude difference does not exceed the threshold T3 set based on previous tests, then these two blocks are merged in sections and the subsequent coding records are postponed, so as to reduce the possibility of duplicate block calls. To obtain a relatively compact coding arrangement, a set of block coding mapping tables will be generated uniformly after all blocks are merged. In this mapping table, the indexes, pixel difference intervals, and corresponding frame numbers of the merged blocks are stored to ensure that the coding position corresponding to each differential pixel can be accurately traced during subsequent data decoding and restoration. Finally, the integrated block coding information is packaged and stored in a compressed manner to generate a compressed difference data packet.

[0075] The steps for obtaining the visual key element identifier are as follows:

[0076] Based on the compressed differential data packet, extract the frame features in the data packet frame by frame, construct the feature vector set of each frame, and at the same time extract the potential geometric transformation information in the image frame. Generate the frame feature vector and the geometric transformation matrix by comparing the local offsets of the pixel features between adjacent frames;

[0077] According to the frame feature vector and the geometric transformation matrix, calculate the discriminant value of the key visual element. The expression is:

[0078]

[0079] where, is the discriminant value of the key visual element, is the translation value of the th row in the geometric transformation matrix, is the rotation value of the th row in the geometric transformation matrix, is the feature value of the rd pixel point in the frame feature vector, is the offset of the corresponding pixel point in the adjacent frame, is the number of rows of the geometric transformation matrix, is the dimension of the feature vector, is the number of offset pixel points in the frame;

[0080] Based on the discriminant value of the key visual element, analyze all frames, select the frame with the highest discriminant value, and determine the position of the key visual element by combining the positioning information in the frame feature vector and the geometric transformation matrix to generate the visual key element identifier.

[0081] Specifically, based on the compressed differential data packets obtained previously, the frame features contained therein are sequentially read and identified frame by frame in the order of image frames. First, a frame number is set for each frame in the internal structure, and the corresponding relationship between the number and the frame features is recorded in the software. Then, in combination with the block pixel change coding information obtained previously, the block indexes of the current frame in the horizontal and vertical directions are determined, so as to retrieve the pixel sets corresponding to each block. On this basis, local elements such as pixel gray level, texture, and contour edge are extracted. Then, during the process of comparing adjacent frames, the position differences of each pixel in the time direction are statistically calculated. If the change amount of the pixel coordinates reaches an offset threshold T1 obtained based on preliminary observation data in consecutive frames, for example, when recording the average time offset of each pixel in about two thousand frame test samples as 3 and adding twice the standard deviation to determine the offset threshold T1 as 8, the pixels with larger coordinate changes are regarded as the focus points and independently marked. Then, by combining and comparing the local gray level and texture around the marked pixels, more refined frame feature elements are obtained. If it is found that the similarity of the local features with the previous frame is within a certain numerical range, the local features in the current frame are mapped one-to-one with the previous frame and the mapping relationship is written into the frame mapping record. Subsequently, the mapping information of consecutive frames is summarized to construct a feature sequence. In this feature sequence, the evolution trend of each frame feature over time can be viewed, and then the spatial change law of the local texture or gray level as the frame number increases is summarized. Through this process, a feature vector set for each frame is obtained and local information such as potential illumination changes and structural changes is recorded. Finally, a geometric transformation matrix is generated based on the local offset results of adjacent frame pixels for subsequent comprehensive analysis.

[0082] The advantage of the formula lies in integrating the translation and rotation information of the geometric transformation matrix, as well as the frame feature vector and the pixel offset amount. 、 、 、 and other parameters jointly participate in the operation and establish a unified discrimination framework with the number of rows 、dimension 、the number of pixel offsets to associate spatial transformation with local offset data in the same calculation expression;

[0083] The acquisition steps of are as follows: combining the row vectors of the geometric transformation matrix obtained previously, recording the horizontal translation amount and the vertical translation amount recorded in each row vector as independent values respectively. Then, 1200 frames of images containing translational movements are collected from the actual scene, the translation amplitude distribution of each frame is statistically calculated, the median and the interquartile range are calculated and averaged to obtain a translation reference value of 8. The part higher than 8 is statistically calculated separately and a weighted average translation amount of 12 is obtained through weighted calculation. This weighted average translation amount is used as the translation value in The acquisition steps are as follows: According to the rotation angle distribution in each row vector by counting the same number of images, record the angle peaks and valleys in the detection process and obtain the average rotation angle of 5 degrees by arithmetic mean. For cases higher than twice the variance of this average rotation angle, measure again and take its smoothed value of 7 degrees as the rotation value of The acquisition steps are as follows: Quantize the grayscale and texture information for each pixel point from the frame feature vector, statistically segment by the grayscale range from 0 to 255, divide the texture direction into four levels, form a feature value after combining different levels and grayscale segments through mapping, combine the statistical results of large-scale samples, determine the average value of this feature value as 23 and limit the fluctuation range between 15 and 30. Analyze the feature values outside this interval additionally to obtain the corrected value, which is finally agreed to be 27 as the higher grayscale texture combination value. The acquisition steps are as follows: Based on the previously recorded adjacent frame pixel position offsets, compare the coordinate changes of each pixel in the time direction one by one and perform absolute value processing, and then use the method of numerical integration to obtain the average offset amplitude of 6 as the main offset. represents the number of rows of the geometric transformation matrix. Here, 4 rows are taken to represent four main transformation combinations. represents the dimension of the frame feature vector. Refer to the grayscale and texture combination mapping done previously and set it to 2 and add an extra column to store local contour features to form 3 dimensions. represents the total number of pixels regarded as significantly offset pixel points. Approximately 55 pixel points are statistically obtained under the above conditions;

[0084] Calculation process:

[0085] The first step is to calculate the numerator part , where 、 、 、 , 、 、 、 , so:

[0086]

[0087] The second step is to calculate the denominator part , where take , assume the eigenvalue 、 、 , then:

[0088]

[0089] The third step is to calculate and divide by , take here , based on the previously obtained pixel average offset amplitude 6 and adding the cumulative value of the per-pixel residual which is approximately 50, the total result is approximately 330. Therefore:

[0090]

[0091] Step 4, substitute the above result into the formula:

[0092]

[0093] Step 5, first calculate the square root part , then calculate , so: ;

[0094] This result indicates that when the value is 1.5616, the discriminant value obtained by combining the geometric transformation information and the translation and rotation data in the current frame is at a relatively high level. If further compare its calculated value with that of other frames and find the frame with the most prominent value among them, it means that the pixel changes contained in this frame are more likely to form a significant focus visually, which has important reference significance for subsequent determination of the position of key visual elements.

[0095] Based on the previously obtained discriminant value data of key visual elements, scan the discriminant values of all frames one by one and mark the differences in the high and low values. First, establish a discriminant value index for each frame and put the frame number and the corresponding discriminant value into a sorted list. Then, in this sorted list, view the discriminant value distribution of each frame in order from largest to smallest. If the discriminant value exceeds a threshold T2 set based on test experience, for example, after statistically analyzing about 20,000 video frames, the average value of the discriminant value is about 0.9 and adding 1.5 times the variance to establish T2 as 1.5, then determine that this frame is a high discriminant value frame and extract the corresponding frame feature vector and the coordinate position data recorded in the geometric transformation matrix. Compare this data with the similar content of subsequent frames to verify whether there are consecutive high value frames. If the number of consecutive high value frames is not less than 3 frames, then regard them as target frames that can be preferentially analyzed and include them in the scope of fine detection. Then, mine the local area coordinates in the corresponding target frames and register the specific pixel positions, gray scale changes, texture detection, etc. information. After summarizing these information, the specific distribution positions of key visual elements in the scene can be obtained and the differences in adjacent frames can be recorded. Finally, select the area with the most definite positioning information to generate the visual key element identifier.

[0096] The steps for obtaining the priority sorted data packet are as follows:

[0097] Based on the visual key element identifier, parse the image frame information in each data packet, extract the corresponding frame number, visual key element type and position data, and obtain the initial priority data of the frame;

[0098] Calculate the priority ranking score of the frame according to the initial priority data of the frame. The expression is:

[0099]

[0100] where, is the priority ranking score of the frame, is the type characteristic value of the visual key element, is the spatial distribution coefficient of the corresponding visual key element, is the characteristic value of the pixel points in this frame, is the data packet transmission delay time corresponding to this frame, is the total number of visual key elements, is the number of pixel points in the frame;

[0101] Based on the priority ranking score of the frame, all the frames in the data packet are rearranged from high to low according to the priority, and the frame transmission order of the data packet is reconstructed to obtain the priority ranking data packet.

[0102] Specifically, based on the visual key element identifiers obtained previously, parse the image frame information in each data packet and read the frame numbers in sequence. Subsequently, in the application environment, combine the types of visual key elements obtained previously to indicate the specific content of each key element. For example, in a sports scene, "racket" or "player's body part" can be regarded as different types. By collecting about three hundred groups of actual game videos and retrieving frame by frame, record the timing and position of each key element when it appears. Then, configure a temporary record for each frame during the parsing process that can reflect the type of key element and the coordinate range. Each record contains the frame number, the key element classification code, and the two-dimensional coordinate points. If a key element has multiple positions, expand the record content to collect them all. Then, filter all the key elements and establish a mapping with the corresponding frames. According to the importance of different key elements to the scene, establish a spatial distribution table for these mappings. In this spatial distribution table, mark the pixel area and the center coordinates occupied by each key element, and perform a local check on it in the supporting software. During the check process, compare the color uniformity and shape characteristics of the pixels. If the color deviation is within the previously determined range of 0 to 15 and the shape parameters match the known characteristics, it is considered the same key element. After multiple rounds of scanning, obtain the type and position data of the visual key elements in each frame. Then, set up several priority factors according to the scene requirements, such as considering the area ratio of the element in the picture and the relative importance of its location, and configure these factors together with the frame number of this frame into an initial priority table, and summarize to form the complete initial priority data of the frame.

[0103] The benefit of the formula is to combine the type characteristic values of visual key elements and the corresponding spatial distribution coefficients, while integrating the sum of the squares of the characteristic values of all pixel points in this frame, and then superimposing the logarithmic quantization of the data packet transmission delay time, so as to take into account the frame importance of different dimensions in one expression;

[0104] The acquisition steps of are as follows. First, based on the type information of visual key elements obtained previously, the occurrence frequency and average attention of each element are statistically counted in about 500 frames of scenes with people and moving objects. After dividing the attention into several segments, the characteristic values of each type are determined in combination with experimental observations. For example, the characteristic value of the human face is set to 7, and the characteristic value of a high-speed moving object is set to 10. The acquisition steps of are as follows. Use the established scene spatial distribution table to record the occupied range and central coordinates of elements in the frame image. According to the proportion of the range occupying the image and the superposition relationship with adjacent elements, a distribution coefficient with an average value of about 2.5 is sorted out through segmented statistics, and then additional measurements are carried out on items deviating from this average value to obtain a range of about 1.0 up and down. Therefore, when applying, values around 2.5 can be assigned to the distribution coefficients of different key elements. The acquisition steps of are as follows. Numerically convert the texture or color characteristics corresponding to all pixel points in the frame. The RGB values in the range of 0 to 255 are split and represented by the average brightness of each channel, and then a comprehensive characteristic value is obtained by combining texture direction quantization. Statistics show that its range is about 10 to 40 in general high-definition motion scenes, and the characteristic values exceeding this interval are re-evaluated later. The acquisition steps of are as follows. Use transmission monitoring means to observe the delay time of each frame in the network environment, take the median or average value of multiple delay acquisition results. During actual measurement, the delay can be measured to be about 30 milliseconds to 60 milliseconds under stable bandwidth conditions. After recording it in millisecond values, add 1 to it for operation in the logarithmic function. represents the total number of visual key elements, which is accumulated sequentially when multiple key elements are detected in the frame. represents the number of pixel points in the frame. When the resolution is 1920×1080, about 2.0736 million pixels can be obtained, and it will increase correspondingly when higher resolutions are adopted.

[0105] Calculation process:

[0106] The first step: Calculate , when 2 key elements are detected in the frame, take , respectively, then there is:

[0107]

[0108] The second step: Calculate , at a resolution of 1920×1080, if only 5000 pixels in a partial area are intercepted, then according to the aforementioned eigenvalue range of 10 to 40, the value of each pixel varies with the acquisition result. After frequency statistics, it can be assumed that the average value is about 25 with slight fluctuations up and down. When calculating the sum of squares of pixel features, it can be carried out according to:

[0109]

[0110] Step 3: Calculate the denominator ;

[0111] Step 4: For , for example, if the measured value of the frame delay time is 40 milliseconds, then , so ;

[0112] Step 5: Substitute the above results into the formula:

[0113]

[0114] The result shows that when is 3.7405, it means that the current frame has a relatively high priority sorting score after weighing multiple indicators such as visual key elements, eigenvalue, and delay time. When the value is greater than 3, it indicates that the key elements are relatively concentrated and the transmission delay remains within the common range. When the value is less than 2, it often indicates that there are fewer elements or the network delay is too large, and different degrees of sorting arrangements can be made in the subsequent frame transmission order.

[0115] Based on the priority sorting score of the frames obtained previously, assign a sorting label to each frame and arrange them in descending order of scores. First, combine the frame numbers recorded in the obtained priority score table with the corresponding scores. If the frames with scores in the top 10% are regarded as having extremely high priority, then compare them in turn. The frames in the range of 10% to 30% are defined as medium priority, and the frames below 30% are regarded as ordinary or low priority in practice. During the sorting process, the number of each frame should also be corresponded to the data packet it belongs to to ensure that the frame order after grouping will not be confused with the content of other data packets. Then, rearrange the frame list in the software environment according to the above priority order. If conflicts are found between frame numbers during this period, they should be corrected. For example, when it is detected that the same number is accidentally repeated, the frame content should be cross-compared and the only valid record should be selected. After confirming that there is no conflict in sorting, the entries with higher priority in the frame list are concentrated to generate a data packet queue with higher priority, and the remaining frames are compiled into subsequent queues according to the subsequent data throughput and network status requirements. The entire list is finally merged into a rearranged frame transmission order queue to obtain priority sorted data packets.

[0116] The steps to obtain the network status monitoring results are as follows:

[0117] Collect real-time network bandwidth utilization data, calculate the average value of the collected bandwidth utilization within a continuous time window, record the bandwidth change trend of each time window, and generate bandwidth utilization record data;

[0118] Collect real-time network transmission delay conditions, mark each transmission delay data with a timestamp, calculate the delay distribution characteristics within the time window, and generate network delay record data;

[0119] Based on the bandwidth utilization record data and network delay record data, analyze the dynamic change relationship, and combine the network operation status at the current time point to make a status determination to obtain the network status monitoring result.

[0120] Specifically, based on the previously planned bandwidth utilization collection scheme, first set a group of bandwidth real-time readers at the network exit or network core node and control the reading frequency between 1 time per minute and 60 times per minute. Then, mark the collected bandwidth utilization data in chronological order, record it in the collection sequence in seconds or milliseconds. Referring to the fluctuation range of the actual business traffic, select a time window in the software environment, such as an interval of 60 seconds to 120 seconds. Statistically calculate the average value of all bandwidth readings within this time window, and at the same time calculate the difference amplitude between the peak and trough of the bandwidth value from the start to the end of the window. Compare this difference with the average value within the window. If the difference value exceeds the threshold T1 determined based on historical measured experience, for example, when the average bandwidth is about 500Mbps in the bandwidth usage records within two weeks and T1 is 100Mbps in combination with the peak and trough distribution, mark this window significantly in the record. Then, continue to the next time window. After completing the rolling analysis of consecutive windows, record the trend of the average value sequence of each window. Identify possible sudden increases or fluctuations by comparing the bandwidth mean and difference amplitude of adjacent windows, and register the corresponding time points and the floating range inside. If the floating amplitude exceeds twice the variance, add a further inspection link to exclude abnormal or temporary event interference. After all windows are processed, summarize all bandwidth means and change trend information to finally obtain the bandwidth utilization record data.

[0121] Based on the bandwidth utilization collection mode obtained previously, additionally monitor the network transmission delay. Measure the round-trip time of each network data packet sent and received, and label each delay record according to the same timestamp rule. For example, conduct 30 delay tests per minute, calculate the delay value as the difference between the packet reception time and the packet sending time, and then collect these delay values in the test order and divide them into continuous time windows that match the bandwidth. Calculate the average value and distribution characteristics of all delay values in each window. For example, determine whether the delay values are mainly distributed in the range of 10 milliseconds to 50 milliseconds. If a batch of delay values is scattered above 300 milliseconds, mark it as an abnormal point. Then, form a delay distribution curve at the statistical level. To clarify the stability of this distribution, calculate the upper and lower quartiles of the distribution range and record the peak situation. If the peak significantly deviates from the threshold T2 obtained based on the previous data statistics, for example, if the average delay is 25 milliseconds in the peak concurrent scenario within a month and a certain multiple of the standard deviation is added to determine T2 as 80 milliseconds, then separately mark such abnormal windows and pay attention to their duration to distinguish instantaneous delay jitter from persistent delay. After analyzing all time windows, store the sorted delay mean, peak, and upper and lower distribution intervals as a set of delay record data, and finally generate network delay record data.

[0122] Based on the previously obtained bandwidth utilization record data and network delay record data, first align the average bandwidth utilization sequence and the corresponding delay mean sequence in the application environment and determine their corresponding relationship under the same time window. Then, compare the fluctuations of the bandwidth sequence and the delay sequence. If the bandwidth utilization rises to a significant level in a certain time window and the delay also exceeds the previously determined empirical threshold or floating range, then combine and mark this window as an abnormal fluctuation window. Then, comprehensively judge the network status by combining basic monitoring information such as the number of network devices in use and the request queuing situation at the current time point. For example, first count the device occupancy rate. If it exceeds the predetermined standard calculated based on test experience in the range of 30% to 70%, it is classified as a busy state. If the delay is marked as too high or the bandwidth is too low, it will also be set to an overloaded state. After summarizing all windows and states, correspond each monitoring index to the time point for unified inspection, so as to identify whether the network is in a high-load, lightly congested, or normal stable state, and finally record the corresponding conclusion internally to form the network status monitoring result.

[0123] The steps for scheduling and optimizing the acquisition of data packets are as follows:

[0124] Based on the network status monitoring result, extract the bandwidth utilization and delay data within the current time window, analyze the transmission requirements of data packets with priority sorting one by one, and generate an initial sorting value for data packet transmission in combination with the size of each data packet, the transmission delay requirement, and the current network bandwidth situation.

[0125] Calculate the transmission priority of the data packet according to the initial sorting value of the data packet transmission. The expression is:

[0126]

[0127] Where is the transmission priority of the th data packet, is the bandwidth requirement of the th data packet, is the transmission delay requirement of the th data packet, is the data packet size of the th data packet, is the stability coefficient related to transmission in the current network state, is the bandwidth utilization rate of the current network state, is the total transmission delay of the current network, is the total number of data packets sorted by priority, is the number of parameters of the stability coefficient in the current network state;

[0128] Based on the transmission priority of the data packet, sort the data packets sorted by priority from high to low. At the same time, reallocate and adjust the transmission order of the data packets with restricted transmission conditions in combination with the transmission priority of the data packets to generate scheduled optimized data packets.

[0129] Specifically, based on the previously obtained network status monitoring results, first read and map the bandwidth utilization rate and latency data within the current time window. Define the time window as a relatively short fixed length, such as 30 seconds or 60 seconds for convenient statistics. Then, retrieve the basic information of each packet one by one according to the obtained priority sorting of packets, including its size and the upper limit range of the required transmission latency. Combine the bandwidth utilization rate to numerically record the possible bandwidth occupied by each packet. Then, perform the same quantization process on the latency requirements of each packet. For example, clarify the latency requirement range for different types of packets through the latency distribution curve obtained in large-scale tests before. If it is found that the latency threshold of some packets is below 10 milliseconds, they are regarded as extremely short latency requirement packets. If it is in the range of 10 milliseconds to 100 milliseconds, they are regarded as medium and short latency requirement packets. If it exceeds 100 milliseconds, it can be classified into the ordinary transmission category. When comparing these requirements with the current network bandwidth, the floating range of the bandwidth utilization rate should also be considered. For example, if the bandwidth utilization rate has approached the empirical range of 60% to 80%, it is regarded as the network tending to be tense. At this time, strengthen the record of the size and latency requirements of each packet and set the corresponding initial sorting identifier to reflect the possible increase in transmission latency of large packets in the high bandwidth occupancy scenario. If the volume of a certain packet exceeds the previously defined threshold T1, for example, in the real online environment comparison, it is most common that the size of a single packet is in the range of 5MB to 20MB. When it exceeds 20MB, an additional mark is made for this packet in the record to remind that it may occupy higher bandwidth or cause local congestion. After integrating the bandwidth requirements and latency requirements of all packets, an incremental scanning method will be used to give a preliminary sorting value. Packets with higher demand and relatively smaller volume usually have more advantages in the sorting value, while larger packets with low latency requirements have relatively lower sorting values. Then, this initial sorting value is centrally recorded to form the initial sorting value for packet transmission.

[0130] The benefit of the formula is that it comprehensively considers the packet bandwidth requirement, latency requirement, packet size, and the stability coefficient in the network status, and incorporates the current bandwidth utilization rate and the total transmission latency into the same denominator structure, connecting the packet requirements with the real-time network environment through this form;

[0131] The obtaining steps of are as follows: segment and statistically analyze the previously obtained packet bandwidth requirements within multiple time windows to determine relatively constant or fluctuating bandwidth requirement intervals. Then, conduct multiple transmission tests on the same type of packets, record the bandwidth application amount each time and average the results. For example, after about 50 tests, obtain an average bandwidth requirement of about 2.5Mbps and set a fluctuation range of 1.0Mbps to 3.5Mbps in combination with the actual traffic volume. The acquisition steps are as follows: Use delayed acquisition means to quantify the shortest round-trip time, the longest round-trip time, and the main distribution range of each data packet, and find out the ideal delay requirement of the data packet. If it is a cross-node data packet, the delay distribution may be between 30 milliseconds and 60 milliseconds. At this time, the average delay requirement obtained by combining multiple actual measurements is about 45 milliseconds. The acquisition steps are as follows: Read the actual volume size of the data packet. If it is between 3MB and 10MB, write it into a variable according to its number of bytes. If it exceeds the 10MB range, it also needs to be split and recorded. The acquisition steps are as follows: Multiply the parameters related to network stability one by one. For example, in the monitoring, the wireless network signal fluctuation coefficient is about 1.1 to 1.3, and the network routing hop count stability coefficient is about 1.05 to 1.2, etc. Finally, multiply multiple coefficients to obtain , The acquisition steps are as follows: Through the summary result of the previous bandwidth utilization record data in the current time window, if the average bandwidth utilization is 60%, it is recorded as 0.6. The acquisition steps are as follows: Make an accumulation or average statistical method for the previous network delay record data in the current time window. If the measured mean value of the total transmission delay distribution is 40 milliseconds, it is recorded as 40. is the total number of packets sorted by priority. Based on the previous summary of the number of packets, a statistical value such as 10 can be obtained. is the number of stability coefficients. According to the subdivision of wireless signal fluctuation, network routing, etc. adopted, the number of parameters can range from 2 to 4;

[0132] Calculation process:

[0133] The first step is to calculate , for example , read in sequence Mbps, milliseconds, MB; Mbps, milliseconds, MB; Mbps, milliseconds, MB. Note that the units need to be unified. For example, the bandwidth Mbps can be converted into a more microscopic value or the data packet volume can be converted into bytes for consistency processing. Here, it is simplified to numerical multiplication to obtain:

[0134]

[0135] The second step is to calculate , when , for example, if the wireless signal fluctuation coefficient is taken as 1.2 and the network routing hop count stability coefficient is taken as 1.1, then:

[0136]

[0137] In the third step, add the results of the above two parts:

[0138]

[0139] In the fourth step, take the denominator , if the current bandwidth utilization rate = 0.6 and the total transmission delay , then:

[0140]

[0141] In the fifth step, substitute the result into the formula:

[0142]

[0143] This result indicates that when the transmission priority is 13.0411, it shows that the result of combining the current data packet with the network state parameters in terms of bandwidth requirements, delay requirements, and volume size is relatively large. A value higher than 10 generally means that this data packet needs to be arranged for transmission in a relatively front position. If it is found in subsequent steps that the priorities of other data packets are lower than 5 or even lower, their transmission order can be moved back, so as to make the utilization of transmission resources more in line with the requirements of this data packet.

[0144] Based on the obtained data packet transmission priorities, first extract the sorted data packet list in the software environment and perform a quick sort on it from high priority to low priority. For several data packets with similar priority scores, additionally check the detailed records of bandwidth requirements and delay requirements. For example, if the difference in priority values between two data packets is less than 1.0, it is determined that their levels are similar and they can be grouped similarly. Separate groups are made for data packets with restricted transmission conditions or whose size has exceeded the previously set threshold T1, and these groups are processed again during subsequent transmission gaps or when the network state is relatively idle. If a tightening situation where the bandwidth utilization rate approaches 70% to 80% again is encountered, the transmission activities of this group can be suspended and a prompt can be given. Finally, after completing all inspections, the sequential list is renumbered to obtain the scheduled and optimized data packets.

[0145] The steps for obtaining the adjusted image data packets are as follows:

[0146] Based on the scheduled and optimized data packets, extract the resolution parameters of each image frame, including the width and height resolutions, and at the same time extract the bandwidth utilization rate in the current network conditions and the upper limit value of the resolution of the device display ability to generate an image resolution adaptation parameter set;

[0147] According to the image resolution adaptation parameter set, calculate the resolution adjustment value of each image frame. The expression is:

[0148]

[0149] where, is the resolution adjustment value of the image frame, is the width resolution of the current image frame, is the height resolution of the current image frame, is the maximum supported width resolution of the device display capability, is the maximum supported height resolution of the device display capability, is the current network bandwidth utilization rate, is the refresh rate parameter of the device display capability, is the total number of pixels within the current frame, is the delay correction value in the network condition;

[0150] Based on the resolution adjustment value of the image frame, dynamically adjust the width and height resolutions of each image frame to obtain the adjusted image data packet.

[0151] Specifically, based on the scheduling optimization data packet obtained previously, first read the resolution parameters of each image frame and analyze the actual values of the width and height. To ensure the reliability of the collected resolution information, in practical applications, a dedicated detection step is required to record the resolution combination of each frame. At the same time, retrieve the bandwidth utilization rate under the current network conditions in the scenario and extract the display capability parameters of the visual device. For example, in some display terminals, the maximum supported resolution is 1920×1080, while for more advanced terminals, it may support 3840×2160. If the detected bandwidth utilization rate is in the range of 60% to 80%, it is regarded as a state of relatively tight network, and the dynamic adjustment of the width and height resolutions may be further triggered. If the upper limit value of the resolution of the device display capability is approximately 1920×1080, but the current sampled frame has reached 2560×1440, it is necessary to confirm whether the actual network transmission rate of the frame can meet the continuous transmission requirements of this resolution. If not, an over-limit mark is added to the record for this frame, and then combined with the device refresh rate and network delay information to determine whether there is still a possibility of frame loss or stuttering. During the process, the delay is compared with the parameters obtained previously from network status monitoring. If the delay value exceeds a threshold T1 established by test statistics continuously for multiple times. For example, if the average conventional delay is monitored to be about 40 milliseconds within a month, and T1 is established as 100 milliseconds after adding twice the standard deviation, then this frame is registered internally as a high-delay risk, and strategies such as reducing the resolution or reducing the frame size occupancy are considered during subsequent adjustments. After summarizing all this information, the resolution and device support of different frames are distinguished in tabular form, and elements such as bandwidth utilization rate and delay value are merged, and finally an image resolution adaptation parameter set is generated.

[0152] The advantage of the formula is that it incorporates the width and height resolutions of the current image frame, as well as the device display capability, network bandwidth utilization rate, the number of pixels within the frame, and the network delay correction value at the same time, and balances the coupling relationship between the frame resolution and network and device conditions through the same expression;

[0153] The acquisition step of is to record the actual width resolution of each frame frame by frame in the application, and select the target frame width value to be adjusted currently according to the scene requirements during the processing stage. For example, it is statistically found that the widths of a certain group of frames are mostly at different standards such as 1920, 2560, 3840, etc. The acquisition step of is the same as but for the height resolution, it is managed uniformly. Extract it frame by frame from the actual shooting or rendering stage and classify and register the current height. The acquisition step of is to detect the specifications of the display device. For example, some monitors support a maximum of 3840 (horizontal pixel count). If the reading is higher than this value, it is recorded as a super-resolution situation, and 3840 is recorded in , The acquisition steps are as follows. Similarly to , if the display device claims a maximum of 2160 (vertical pixel count), then write this data into , The acquisition steps are as follows. Record the refresh rate of the device through multiple automatic detections and synthesize a complete display refresh sampling data. For example, if the measured average refresh rate is around 75 Hz, then write 75 into and make appropriate fluctuations according to the frame interpolation and actual refresh capabilities of the monitor during program execution. The acquisition steps are as follows. Obtain the bandwidth utilization rate within the current time window through bandwidth monitoring. If the measured value is 0.65, then record it as 0.65. The acquisition steps are as follows. Statistically calculate the total number of all pixels in the current frame. For example, for 1920×1080, it is approximately 2,073,600. The acquisition steps are as follows. The delay correction amount obtained by comparing the cumulative value or correction number of each frame delay in the network status monitoring. If the monitoring shows that some frames have jitter, then L may be set to a correction value of 10 or 20 milliseconds.

[0154] Calculation process:

[0155] First step, substitute , . The resolution of this frame is higher than the conventional 1080P. , because the maximum supported specification of the detected device is 4K. comes from the statistically calculated refresh rate. represents the current bandwidth utilization rate of 65%. , represents a correction amount of 12 milliseconds evaluated according to recent network jitter.

[0156] Second step, calculate the three parts in the numerator respectively:

[0157]

[0158]

[0159]

[0160] Add the three to get the numerator:

[0161]

[0162] Third step, calculate the denominator , where , then add , to get:

[0163]

[0164] Step 4, substitute the result into the formula:

[0165]

[0166] The result shows that when the resolution adjustment value of the current image frame is 109.40, it indicates that under the combined action of multiple factors such as the 4K display ability, bandwidth utilization, and network latency correction of the device, this frame is expected to have a certain reduction or maintenance in the horizontal and vertical resolutions to match the resolution level corresponding to this value. If it is greater than 120, it means that the resolution can be moderately maintained at a relatively high level. If it is lower than 80, it means that the resolution may need to be reduced for transmission.

[0167] Based on the obtained resolution adjustment value of the image frame, first compare the resolution adjustment value of each frame with its initial width and height in the record. If it is found during detection that the current resolution adjustment value is less than 100 while the maximum supported resolution of the device is 1920×1080, it means it is necessary to reduce the original frame resolution of 2560×1440 to a range close to 1920×1080. If the calculated resolution adjustment value is higher than a certain empirical threshold T1, such as observing that the picture is clearer and the frame rate does not drop significantly when the adjustment value is greater than 150 in multiple tests, then make a smaller reduction in the width and height resolutions according to the adjustment value, then mark the new resolution combination in the record and re - estimate the occupied bandwidth and check the latency, and uniformly summarize all the processed frame information into a new frame data list, and finally form an adjusted image data packet.

[0168] The present invention provides a computer network communication data processing system, including:

[0169] A video frame differentiation module that extracts two consecutive frames from a real - time video frame, calculates the pixel changes between the two frames, and obtains a pixel difference data set;

[0170] A data compression and key element recognition module that compresses the pixel difference data set to form a compressed difference data packet; based on the compressed difference data packet, identifies the key visual elements in the image, records the positions and attributes of the key visual elements, and generates visual key element identifiers;

[0171] A data packet priority sorting module that uses the visual key element identifiers to perform priority sorting on the data packets and generates a priority - sorted data packet;

[0172] A network state adaptation and scheduling optimization module that monitors the bandwidth and latency of the current network and generates a network state monitoring result; combines the network state monitoring result to dynamically adjust the sending order of the priority - sorted data packets and creates a scheduling - optimized data packet;

[0173] The image resolution dynamic adjustment module adjusts the resolution of the image data according to the instructions in the scheduling-optimized data packet, combines the network conditions and the display capabilities of the target device, generates an adjusted image data packet, and sends it to the target device to complete the image data transmission process.

[0174] The above are only the preferred embodiments of the present invention, and the present invention is not limited in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A computer network communication data processing method based on artificial intelligence, characterized in that: The following steps are involved: Performing continuous frame difference calculation on the data of real-time video frames, collecting pixel-level change data between two frames, and obtaining a difference data set; compressing the difference data set to generate a compressed difference data packet; Based on the compressed difference data packet, analyzing the image content, identifying key visual elements, including face or action key frames, and generating visual key element identifiers; prioritizing the data packets according to the visual key element identifiers to obtain priority-ordered data packets; Collecting real-time network status data, including bandwidth utilization and delay, to obtain network status monitoring results; adjusting the data packet transmission strategy according to the network status monitoring results, dynamically determining the transmission order of the priority-sorted data packets, and generating scheduling optimization data packets; Adjusting the resolution of the scheduling optimization data packet, dynamically modifying the image resolution according to the current network conditions and the device display capability, performing image size adjustment, and generating an adjusted image data packet; sending the adjusted image data packet to the target device to complete the image data transmission process; The steps of obtaining the difference data set are: Collect two frames of continuous image data in the real-time video stream, compare the grayscale value of each pixel one by one, calculate the pixel grayscale difference between the two frames of images, and record the difference change results of all pixels to generate pixel-level difference data; Based on the pixel-level difference data, grouping the pixel difference change results, screening the pixel points whose change values ​​exceed the set threshold, and integrating the positions of the pixel points exceeding the set threshold with the difference values ​​to form pixel change data; Based on the pixel change data, the change records are stored in a structured manner according to the frame number and the change range, and redundant data is removed to generate a difference data set; The steps of acquiring the adjusted image data packet are: Based on the scheduling optimization data packet, the resolution parameters of each image frame are extracted, including width and height resolution, and the bandwidth utilization rate in the current network conditions and the upper limit of the resolution of the device display capability are extracted to generate an image resolution adaptation parameter set; According to the image resolution adaptation parameter set, the resolution adjustment value of each image frame is calculated, and the expression is: in, is the resolution adjustment value of the image frame, is the width resolution of the current image frame, is the height resolution of the current image frame, The maximum supported width resolution of the device display capability. The maximum supported height resolution of the device display capability. is the current network bandwidth utilization, The refresh rate parameter of the device display capability. is the total number of pixels in the current frame, is the delay correction value in network conditions; Based on the resolution adjustment value of the image frame, the width and height resolution of each image frame are dynamically adjusted to obtain an adjusted image data packet.

2. The method for processing computer network communication data based on artificial intelligence according to claim 1, characterized in that: The steps of obtaining the compressed difference data packet are as follows: Based on the difference data set, the pixel change data of each frame is partitioned according to a preset block rule, the pixel change characteristics in each block are counted one by one, and the pixel value of each block is coded and analyzed in combination with the change range to generate a block pixel change coding result; Based on the block pixel change coding result, the repeated or redundant coding information is removed from the coding content of each block and the data representation structure is readjusted to generate optimized block coding content; Based on the optimized block coding content, the coding data of all blocks are integrated, all block coding is rearranged according to the data structure of the difference data set, and stored in a compressed storage form to generate a compressed difference data packet.

3. The computer network communication data processing method based on artificial intelligence according to claim 1 is characterized in that: The steps for obtaining the visual key element identification are: Based on the compressed difference data packet, the frame features in the data packet are extracted frame by frame, a feature vector set of each frame is constructed, and potential geometric transformation information in the image frame is extracted, and a frame feature vector and a geometric transformation matrix are generated by comparing the local offset of pixel features between adjacent frames; According to the frame feature vector and geometric transformation matrix, the discriminant value of the key visual element is calculated, and the expression is: in, is the discriminant value of the key visual element, is the geometric transformation matrix The translation value of the row, is the geometric transformation matrix The rotation value of the row, is the first The feature value of each pixel, is the offset of the corresponding pixel points in adjacent frames, is the number of rows of the geometric transformation matrix, is the dimension of the feature vector, is the number of offset pixels in the frame; Based on the discrimination value of the key visual element, all frames are analyzed, the frame with the highest discrimination value is selected, and the position of the key visual element is determined by combining the frame feature vector and the positioning information in the geometric transformation matrix to generate a visual key element identifier.

4. The computer network communication data processing method based on artificial intelligence according to claim 1 is characterized in that: The steps of obtaining the priority sorted data packets are as follows: Based on the visual key element identifier, the image frame information in each data packet is parsed, the corresponding frame number, visual key element type and position data are extracted, and the initial priority data of the frame is obtained; According to the initial priority data of the frame, the priority ranking score of the frame is calculated, and the expression is: in, For the The frame's prioritization score, is the type attribute value of the visual key element, is the spatial distribution coefficient of the corresponding visual key elements, is the characteristic value of the w-th pixel in the current frame, is the data packet transmission delay time corresponding to the u-th frame, is the total number of key visual elements, is the number of pixels in the frame; Based on the priority ranking scores of the frames, all frames in the data packet are rearranged from high to low priority, and the frame transmission order of the data packet is reconstructed to obtain a priority ranking data packet.

5. The computer network communication data processing method based on artificial intelligence according to claim 1 is characterized in that: The steps for obtaining the network status monitoring result are: Collect network bandwidth utilization data in real time, calculate the average value of the collected bandwidth utilization within a continuous time window, record the bandwidth change trend of each time window, and generate bandwidth utilization record data; Collect network transmission delay in real time, timestamp each transmission delay data, calculate the delay distribution characteristics within the time window, and generate network delay record data; Based on the bandwidth utilization record data and the network delay record data, the dynamic change relationship is analyzed and the status is determined in combination with the network operation status at the current time point to obtain the network status monitoring result.

6. The method for processing computer network communication data based on artificial intelligence according to claim 1, characterized in that: The steps of obtaining the scheduling optimization data packet are as follows: Based on the network status monitoring results, the bandwidth utilization and delay data in the current time window are extracted, the transmission requirements of the priority-sorted data packets are analyzed one by one, and the initial sorting value of the data packet transmission is generated by combining the size of each data packet, the transmission delay requirement and the current bandwidth status of the network; According to the initial sorting value of the data packet transmission, the transmission priority of the data packet is calculated, and the expression is: in, For the The transmission priority of each data packet, For the The bandwidth requirement of each packet is For the The transmission delay requirement of each data packet is For the The packet size of the packets, is the stability coefficient related to transmission in the current network state, is the bandwidth utilization of the current network status, is the total transmission delay of the current network, The total number of packets sorted for priority, is the number of parameters of the stability coefficient in the current network state; Based on the transmission priority of the data packet, the priority-sorted data packets are sorted from high to low according to priority, and at the same time, the data packets with limited transmission conditions are reallocated and the transmission order is adjusted in combination with the transmission priority of the data packet to generate a scheduling optimized data packet.

7. A computer network communication data processing system according to any one of claims 1 to 6, characterized in that: include: The video frame differentiation module extracts two consecutive frames from the real-time video frame, calculates the pixel changes between the two frames, and obtains the pixel difference data set; The data compression and key element identification module compresses the pixel difference data set to form a compressed difference data packet; based on the compressed difference data packet, it identifies the key visual elements in the image, records the location and attributes of the key visual elements, and generates visual key element identifiers; a data packet prioritization module, which uses the visual key element identification to perform priority sorting on the data packets and generate a prioritized data packet; The network status adaptation and scheduling optimization module monitors the bandwidth and delay of the current network and generates network status monitoring results. It dynamically adjusts the sending order of priority-sorted data packets based on the network status monitoring results and creates scheduling optimization data packets. The image resolution dynamic adjustment module adjusts the resolution of the image data according to the instructions in the scheduling optimization data packet, combined with the network conditions and the display capability of the target device, and generates an adjusted image data packet; Send to the target device to complete the image data transmission process.

Images