Computer network communication data processing method and system based on artificial intelligence

Abstract

The invention relates to the technical field of image communication, in particular to a computer network communication data processing method and system based on artificial intelligence, and the method comprises the following steps: carrying out the continuous frame difference calculation of the data of real-time video frames, collecting the pixel-level change data between two frames, and obtaining a difference data set; according to the method, continuous frame difference calculation is carried out on real-time video frames, pixel-level change data between two frames is captured, storage and transmission occupation of useless data is reduced, and data processing pressure is relieved. And the compression of the difference data further optimizes the size of the data packet, so that the transmission efficiency is improved, and the integrity of video key content is guaranteed at the same time. The compressed difference data packets are analyzed, key visual elements are extracted, and limited transmission resources are distributed to important information parts in a centralized manner by focusing on visual key points such as human faces and actions, so that the pertinence of the transmission process is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of image communication technology, and in particular to a computer network communication data processing method and system based on artificial intelligence. Background Art

[0002] The field of image communication technology is an important branch of computer network communication, and mainly studies how to efficiently transmit and process image and video data through computer networks. This field covers the complete chain from image acquisition, encoding and compression, transmission to decoding and display, involving image compression algorithms, channel transmission optimization, network protocol design, dynamic adaptation technology and many other aspects.

[0003] However, existing data processing methods lack a mechanism for judging the importance of image content, resulting in no clear distinction between high-value visual elements and background information in terms of transmission priority, which may cause key information to be delayed or discarded under resource-constrained conditions. Existing technologies find it difficult to dynamically adjust data transmission strategies for real-time changes in network status, and usually adopt a static transmission order. When network delays increase or bandwidth decreases, efficiency decreases, affecting transmission reliability. In addition, image resolution adjustment methods are mostly fixed solutions that fail to adapt to device display capabilities and network conditions, which may cause high-resolution data to cause a processing burden on low-performance devices, or image details to be lost on high-performance devices. These deficiencies limit the adaptability of existing technologies in complex network scenarios. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a computer network communication data processing method and system based on artificial intelligence.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme, a computer network communication data processing method based on artificial intelligence, comprising the following steps:

[0006] Performing continuous frame difference calculation on the data of real-time video frames, collecting pixel-level change data between two frames, and obtaining a difference data set; compressing the difference data set to generate a compressed difference data packet;

[0007] Based on the compressed difference data packet, analyzing the image content, identifying key visual elements, including face or action key frames, and generating visual key element identifiers; prioritizing the data packets according to the visual key element identifiers to obtain priority-ordered data packets;

[0008] Collecting real-time network status data, including bandwidth utilization and delay, to obtain network status monitoring results; adjusting the data packet transmission strategy according to the network status monitoring results, dynamically determining the transmission order of the priority-sorted data packets, and generating scheduling optimization data packets;

[0009] The resolution of the scheduling optimization data packet is adjusted, the image resolution is dynamically modified according to the current network conditions and the device display capability, the image size is adjusted, and an adjusted image data packet is generated; the adjusted image data packet is sent to the target device to complete the image data transmission process.

[0010] Preferably, the step of acquiring the difference data set is:

[0011] Collect two frames of continuous image data in the real-time video stream, compare the grayscale value of each pixel one by one, calculate the pixel grayscale difference between the two frames of images, and record the difference change results of all pixels to generate pixel-level difference data;

[0012] Based on the pixel-level difference data, grouping the pixel difference change results, screening the pixel points whose change values ​​exceed the set threshold, and integrating the positions of the pixel points exceeding the set threshold with the difference values ​​to form pixel change data;

[0013] Based on the pixel change data, the change records are stored in a structured manner according to the frame number and the change range, and redundant data are removed to generate a difference data set.

[0014] Preferably, the steps of obtaining the compressed difference data packet are:

[0015] Based on the difference data set, the pixel change data of each frame is partitioned according to a preset block rule, the pixel change characteristics in each block are counted one by one, and the pixel value of each block is coded and analyzed in combination with the change range to generate a block pixel change coding result;

[0016] Based on the block pixel change coding result, the repeated or redundant coding information is removed from the coding content of each block and the data representation structure is readjusted to generate optimized block coding content;

[0017] Based on the optimized block coding content, the coding data of all blocks are integrated, all block coding is rearranged according to the data structure of the difference data set, and stored in a compressed storage form to generate a compressed difference data packet.

[0018] Preferably, the steps of obtaining the visual key element identification are:

[0019] Based on the compressed difference data packet, the frame features in the data packet are extracted frame by frame, a feature vector set of each frame is constructed, and potential geometric transformation information in the image frame is extracted, and a frame feature vector and a geometric transformation matrix are generated by comparing the local offset of pixel features between adjacent frames;

[0020] According to the frame feature vector and geometric transformation matrix, the discriminant value of the key visual element is calculated, and the expression is:

[0021]

[0022] in, is the discriminant value of the key visual element, is the geometric transformation matrix The translation value of the row, is the geometric transformation matrix The rotation value of the row, is the first The feature value of each pixel, is the offset of the corresponding pixel points in adjacent frames, is the number of rows of the geometric transformation matrix, is the dimension of the feature vector, is the number of offset pixels in the frame;

[0023] Based on the discrimination value of the key visual element, all frames are analyzed, the frame with the highest discrimination value is selected, and the position of the key visual element is determined by combining the frame feature vector and the positioning information in the geometric transformation matrix to generate a visual key element identifier.

[0024] Preferably, the steps of obtaining the priority sorted data packets are:

[0025] Based on the visual key element identifier, the image frame information in each data packet is parsed, the corresponding frame number, visual key element type and position data are extracted, and the initial priority data of the frame is obtained;

[0026] According to the initial priority data of the frame, the priority ranking score of the frame is calculated, and the expression is:

[0027]

[0028] in, For the The frame's prioritization score, is the type attribute value of the visual key element, is the spatial distribution coefficient corresponding to the key visual elements, is the characteristic value of the pixel in the jth frame, is the data packet transmission delay time corresponding to the jth frame, is the total number of key visual elements, is the number of pixels in the frame;

[0029] Based on the priority ranking scores of the frames, all frames in the data packet are rearranged from high to low priority, and the frame transmission order of the data packet is reconstructed to obtain a priority ranking data packet.

[0030] Preferably, the steps of obtaining the network status monitoring result are:

[0031] Collect network bandwidth utilization data in real time, calculate the average value of the collected bandwidth utilization within a continuous time window, record the bandwidth change trend of each time window, and generate bandwidth utilization record data;

[0032] Collect network transmission delay in real time, timestamp each transmission delay data, calculate the delay distribution characteristics within the time window, and generate network delay record data;

[0033] Based on the bandwidth utilization record data and the network delay record data, the dynamic change relationship is analyzed and the status is determined in combination with the network operation status at the current time point to obtain the network status monitoring result.

[0034] Preferably, the steps of acquiring the scheduling optimization data packet are:

[0035] Based on the network status monitoring results, the bandwidth utilization and delay data in the current time window are extracted, the transmission requirements of the priority-sorted data packets are analyzed one by one, and the initial sorting value of the data packet transmission is generated by combining the size of each data packet, the transmission delay requirement and the current bandwidth status of the network;

[0036] According to the initial sorting value of the data packet transmission, the transmission priority of the data packet is calculated, and the expression is:

[0037]

[0038] in, For the The transmission priority of each data packet, For the The bandwidth requirement of each packet is For the The transmission delay requirement of each data packet is For the The packet size of the packets, is the stability coefficient related to transmission in the current network state, is the bandwidth utilization of the current network status, is the total transmission delay of the current network, The total number of packets sorted for priority, is the number of parameters of the stability coefficient in the current network state;

[0039] Based on the transmission priority of the data packet, the priority-sorted data packets are sorted from high to low according to priority, and at the same time, the data packets with limited transmission conditions are reallocated and the transmission order is adjusted in combination with the transmission priority of the data packet to generate a scheduling optimized data packet.

[0040] Preferably, the step of acquiring the adjusted image data packet is:

[0041] Based on the scheduling optimization data packet, the resolution parameters of each image frame are extracted, including width and height resolution, and the bandwidth utilization rate in the current network conditions and the upper limit of the resolution of the device display capability are extracted to generate an image resolution adaptation parameter set;

[0042] According to the image resolution adaptation parameter set, the resolution adjustment value of each image frame is calculated, and the expression is:

[0043]

[0044] in, is the resolution adjustment value of the image frame, is the width resolution of the current image frame, is the height resolution of the current image frame, The maximum supported width resolution of the device display capability. The maximum supported height resolution of the device display capability. is the current network bandwidth utilization, The refresh rate parameter of the device display capability. is the total number of pixels in the current frame, is the delay correction value in network conditions;

[0045] Based on the resolution adjustment value of the image frame, the width and height resolution of each image frame are dynamically adjusted to obtain an adjusted image data packet.

[0046] The present invention provides a computer network communication data processing system, comprising:

[0047] The video frame differentiation module extracts two consecutive frames from the real-time video frame, calculates the pixel changes between the two frames, and obtains the pixel difference data set;

[0048] The data compression and key element identification module compresses the pixel difference data set to form a compressed difference data packet; based on the compressed difference data packet, it identifies the key visual elements in the image, records the location and attributes of the key visual elements, and generates visual key element identifiers;

[0049] a data packet prioritization module, which uses the visual key element identification to perform priority sorting on the data packets and generate a prioritized data packet;

[0050] The network status adaptation and scheduling optimization module monitors the bandwidth and delay of the current network and generates network status monitoring results. It dynamically adjusts the sending order of priority-sorted data packets based on the network status monitoring results and creates scheduling optimization data packets.

[0051] The image resolution dynamic adjustment module adjusts the resolution of the image data according to the instructions in the scheduling optimization data packet, combined with the network conditions and the display capabilities of the target device, and generates an adjusted image data packet; it is sent to the target device to complete the image data transmission process.

[0052] Compared with the prior art, the advantages and positive effects of the present invention are:

[0053] The present invention performs continuous frame difference calculation on real-time video frames to capture pixel-level change data between two frames, thereby reducing the storage and transmission occupancy of useless data and alleviating data processing pressure. The compression of difference data further optimizes the data packet size, improves transmission efficiency, and ensures the integrity of key video content. The compressed difference data packet is analyzed and key visual elements are extracted. By focusing on visual highlights such as faces and actions, limited transmission resources are concentratedly allocated to important information parts, thereby enhancing the pertinence of the transmission process. Data packets marked with key visual elements are prioritized so that high-value data can be transmitted preferentially under limited resources, thereby reducing the impact of delay on key content. The network status is collected and analyzed in real time, and the data transmission order is dynamically adjusted according to bandwidth utilization and delay conditions to achieve flexible adaptation to complex network environments. The resolution adjustment link combines the current network conditions and the display capabilities of the device to dynamically modify the image resolution, thereby reducing bandwidth requirements and ensuring the display effect of the device end. In this way, data redundancy is reduced, key information transmission is more efficient, the adaptive ability to complex network environments is enhanced, and the accuracy of data transmission and the compatibility of display are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] See also Figure 1 The present invention provides a technical solution, a computer network communication data processing method based on artificial intelligence, comprising the following steps:

[0057] Perform continuous frame difference calculation on the data of real-time video frames, collect pixel-level change data between two frames, and obtain a difference data set; compress the difference data set to generate a compressed difference data packet;

[0058] Based on the compressed difference data packets, the image content is analyzed, key visual elements, including face or action key frames, are identified, and visual key element identifiers are generated; the data packets are prioritized according to the visual key element identifiers to obtain prioritized data packets;

[0059] Collect real-time network status data, including bandwidth utilization and latency, to obtain network status monitoring results; adjust data packet transmission strategies based on network status monitoring results, dynamically determine the transmission order of priority-sorted data packets, and generate scheduling-optimized data packets;

[0060] The resolution of the scheduling optimization data packet is adjusted, the image resolution is dynamically modified according to the current network conditions and device display capabilities, the image size is adjusted, and the adjusted image data packet is generated; the adjusted image data packet is sent to the target device to complete the image data transmission process.

[0061] The steps to obtain the difference data set are:

[0062] Collect two frames of continuous image data in the real-time video stream, compare the grayscale value of each pixel one by one, calculate the pixel grayscale difference between the two frames of images, and record the difference change results of all pixels to generate pixel-level difference data;

[0063] Based on the pixel-level difference data, the results of the pixel difference change are grouped and processed, the pixels whose change values ​​exceed the set threshold are screened, and the positions of the pixels exceeding the set threshold and the difference values ​​are integrated to form the pixel change data;

[0064] Based on the pixel change data, the change records are stored in a structured manner according to the frame number and change range, and redundant data are removed to generate a difference data set.

[0065] Specifically, based on the two frames of continuous image data obtained previously, the pixel grayscale value of each frame of the image is first read and compared one by one, and then the grayscale difference of each pixel position is stored in the form of absolute value in the software environment, and a comparison table is established for different ambient lighting conditions and shooting equipment specifications to clarify the grayscale value range. to The actual grayscale value of each pixel is matched with the range one by one. If there is data outside this range, it is marked. Then, all pixel difference results are summarized through a unified operation process and the correlation analysis between adjacent pixels is performed. For example, when the difference between adjacent pixel indexes does not exceed When , the pre-established local area comparison function can be called to calculate the local mean and variance, and the grayscale difference matrix can be formed by combining the time difference information of consecutive frames. In order to ensure accuracy, a simple supervised classification model can be trained using labeled image data in known scenes during the training phase. The model training process obtains the real label by collecting thousands of frames of video clips shot under diverse lighting conditions. For the difference value of each pixel, the minimum mean square error method is used to calculate the loss of the model and the weights are continuously adjusted through gradient descent to form a final model that can identify abnormal difference values. Subsequently, in the application phase, the grayscale difference data collected in real time will be compared with the parameters of the model to screen out unreasonable errors and generate intermediate difference matrix records. After unified integration, the summary data that can reflect the distribution of pixel differences between frames is obtained, and finally the pixel-level difference data is generated.

[0066] Based on the pixel-level difference data obtained above, the difference values ​​of each pixel are first sorted and the position index is marked. Then, they are arranged from small to large according to the difference amplitude calculated point by point, and a reference line is set for each pixel difference value. The initial value of the reference line is set based on a large amount of experimental data. For example, about 30,000 frames of images were collected during the test phase, and the difference distribution of each pixel in a typical motion scene was statistically calculated, and the average difference mean and variance were calculated. The result obtained by adding twice the variance to the mean is defined as the threshold. If the difference value of a pixel exceeds the threshold, it is regarded as a significant change point. At this time, its position and difference value are also integrated into a special data table. The integration process includes coordinate mapping and grayscale difference analysis. For the determined super-threshold pixel point, its position change on the time axis will be additionally retrieved. If a high difference continues to appear in multiple frames, it is recorded as a stable change point, otherwise it is regarded as a random jitter point. In this way, the position information and difference value summary of multiple batches of stable change points and random jitter points are obtained respectively, and finally the pixel change data is formed.

[0067] Based on the pixel change data obtained previously, a corresponding relationship is established between the frame numbers and change ranges of all pixels, and they are recorded in batches and a redundant determination mark is set for each pixel during the recording process. If it is found that the pixel has repeated marks in adjacent frames and the difference amplitude is similar, the pixel is first determined to be repeatable before storage. If the number of repetitions exceeds three times, the information of the same position and similar difference amplitude is merged into a single record to compress redundant numerical items. Subsequently, the structured configuration of the data is completed in the software according to the frame number sequence, and the remaining pixel change entries after the redundant merger are arranged into a searchable list. On this basis, a mapping relationship between associative frame indexes and difference values ​​is provided for subsequent image analysis or processing, and finally a difference data set is generated.

[0068] The steps to obtain the compressed difference data packet are:

[0069] Based on the difference data set, the pixel change data of each frame is partitioned according to the preset block rules, the pixel change characteristics in each block are counted one by one, and the pixel value of each block is coded and analyzed in combination with the change range to generate the block pixel change coding result;

[0070] Based on the block pixel change coding results, the repeated or redundant coding information of each block coding content is removed and the data representation structure is readjusted to generate optimized block coding content;

[0071] Based on the optimized block coding content, the coding data of all blocks are integrated, all block coding is rearranged according to the data structure of the difference data set, and stored in a compressed storage form to generate a compressed difference data packet.

[0072] Specifically, based on the difference data set obtained previously, the pixel change data of each frame is partitioned using a pre-defined block rule. First, the pixel positions and corresponding grayscale differences involved in each frame are read and the frame number and coordinate range are marked in the data structure. The entire frame image is then divided into several adjacent or overlapping sub-regions according to the pre-defined block dimension. Subsequently, the change amplitudes of all pixels in each sub-region are retrieved and a matching list is established to compare whether they conform to the change distribution characteristics within the block. In order to determine the overall change of the block, multiple groups of sample frames are collected during the test phase and the pixel change mean and variance of each sub-region are calculated. These statistical values ​​are set as the benchmark range. If the runtime data has pixel fluctuations that exceed the benchmark range and exceed a threshold T1 set based on experience, for example, the average change plus three times the variance is established using the statistical results in large-scale motion picture scenes. When any image When the pixel change amplitude is greater than T1, the block where it is located is marked as a high change area. Subsequently, it is divided into multiple levels according to the average change intensity of the pixels in the block and its grayscale change pattern is analyzed point by point. For example, the frequency of all super-threshold pixels in each block is counted and the information is compared with the known motion or illumination scene labels. If the corresponding block shows a large range of differences in multiple frames, it is recorded as a stable high-change block. Otherwise, it is independently encoded as an instantaneous fluctuation block. Then, in the summary stage, the pixel value is encoded and analyzed according to the change range of each block. The analysis process first unsigns the pixel grayscale difference and rearranges it in the form of short integer or fixed length and width, and then generates coding identifiers one by one, so that the position of the high-change area can be quickly found and its difference information can be matched during subsequent retrieval. After the above processing, the block pixel change coding data of each sub-area is obtained, and finally the block pixel change coding result is generated.

[0073] Based on the previously generated block pixel change coding results, the coding records of each sub-area are checked one by one to see if there is repeated coding or high-similarity coding, and the corresponding relationship between the pixel coordinate index and the grayscale change amplitude is clarified during the checking process. For example, in the coding list, if the coordinate index and grayscale difference of two adjacent records are within the range of no more than 3, it is determined to be repeated or redundant information. The value of the corresponding threshold can be determined based on the statistical results of the coding overlap rate in the initial experiment. For example, in a motion scene of 10,000 frames, when 3 is observed to be the best superposition range, 3 is used as the threshold T2 for judgment. Once redundancy is confirmed, it is immediately merged into a single coding identifier and the overlapping segments are marked as merged in the internal record to reduce duplication. The number of codes is increased and the data size is reduced. When adjusting the data representation structure later, the sub-region boundaries during block division will be referred to. In the same sub-region, all redundant merged coding entries will be sorted from small to large according to the grayscale difference or according to the frequency of occurrence. During this period, a set of hierarchical tables can be loaded to put pixels with a certain range of difference amplitudes in the same sub-category, so as to maintain the hierarchy and searchability of the data structure. For single-pixel differences with extremely low frequency of occurrence, independent annotation will be performed. After the sorting is completed, the new coding order will be written into a unified block coding overview list, and necessary information such as block index and pixel difference interval will be retained in the list. Finally, the coding summary after eliminating redundancy and readjusting the representation structure is obtained to generate optimized block coding content.

[0074] Based on the optimized block coding content obtained above, the coding records of all sub-areas are summarized and re-arranged according to the original frame order and pixel index of the difference data set. In this process, an external index is added to each block code to indicate its spatial position and block sequence number in the frame, and the blocks are checked for intersection or overlap according to the overall data structure preset by the difference data set. If two adjacent blocks are found to contain similar coding positions in the same frame and the difference in pixel change amplitude does not exceed the threshold T3 set based on the previous test, the two blocks are segmented and the subsequent coding records are postponed, thereby reducing the possibility of repeated block calls. In order to obtain a more compact coding arrangement, a set of block coding mapping tables will be uniformly generated after all blocks are merged. The index, pixel difference interval and corresponding frame number of each merged block are stored in the mapping table to ensure that the coding position corresponding to each difference pixel can be accurately traced during subsequent data decoding and restoration. Finally, the integrated block coding information is packaged and stored in a compressed manner to generate a compressed difference data packet.

[0075] The steps to obtain the visual key element identification are:

[0076] Based on the compressed difference data packet, the frame features in the data packet are extracted frame by frame, and the feature vector set of each frame is constructed. At the same time, the potential geometric transformation information in the image frame is extracted, and the frame feature vector and geometric transformation matrix are generated by comparing the local offset of the pixel features between adjacent frames.

[0077] According to the frame feature vector and geometric transformation matrix, the discriminant value of the key visual element is calculated, and the expression is:

[0078]

[0079] in, is the discriminant value of the key visual element, is the geometric transformation matrix The translation value of the row, is the geometric transformation matrix The rotation value of the row, is the first The feature value of each pixel, is the offset of the corresponding pixel points in adjacent frames, is the number of rows of the geometric transformation matrix, is the dimension of the feature vector, is the number of offset pixels in the frame;

[0080] Based on the discrimination value of key visual elements, all frames are analyzed, the frames with the highest discrimination value are selected, and the positions of key visual elements are determined by combining the positioning information in the frame feature vector and the geometric transformation matrix to generate visual key element identification.

[0081] Specifically, based on the compressed difference data packet obtained previously, the frame features contained therein are read in sequence and marked frame by frame in the order of the image frames. First, a frame number is set in the internal structure for each frame and the correspondence between the number and the frame feature is recorded in the software. Then, the block index of the current frame in the horizontal and vertical directions is determined in combination with the previously obtained block pixel change encoding information, so as to retrieve the pixel set corresponding to each block, and on this basis, extract local elements such as pixel grayscale, texture and contour edge. Then, in the process of comparing adjacent frames, the position difference of each pixel in the time direction is counted. If the pixel coordinate change in continuous frames reaches an offset threshold T1 obtained based on preliminary observation data, for example, in about two thousand test frames, the time offset mean of each pixel is recorded as 3 and twice the standard deviation is added to determine the offset threshold T1 as 8 , these pixels with large coordinate changes are regarded as focus points and marked independently, and then the local grayscale and texture around the marked pixels are combined and compared to obtain more refined frame feature elements. If it is found that the similarity between the local feature and the previous frame is within a certain numerical range, the local feature will be mapped one-to-one with the previous frame in the current frame and the mapping relationship will be written into the frame mapping record. Subsequently, the mapping information of consecutive frames will be summarized to construct a feature sequence, in which the evolution trend of each frame feature over time can be viewed, and then the spatial change law of local texture or grayscale with increasing frame number can be summarized. Through this process, a set of feature vectors for each frame is obtained and potential local information such as illumination changes and structural changes is recorded. Finally, a geometric transformation matrix is ​​generated according to the local offset results of pixels in adjacent frames for subsequent comprehensive analysis.

[0082] The formula is beneficial in that it fuses the translation and rotation information of the geometric transformation matrix with the frame feature vector and pixel offset. , , , Parameters such as , Dimension , pixel offset amount Establish a unified discriminant framework to associate spatial transformation and local offset data in the same computational expression;

[0083] The acquisition steps are as follows: combining the row vectors of the geometric transformation matrix obtained above, the horizontal translation and the vertical translation recorded in each row vector are recorded as independent values, and then 1200 frames of images containing translation actions are collected from the actual scene, and the translation amplitude distribution of each frame is counted, and the median and quartile difference are calculated and averaged to obtain the translation reference value 8. The part higher than 8 is counted separately and weighted to obtain the weighted average translation 12, and the weighted average translation is used as The translation value in , The acquisition steps are as follows: according to the same number of images, the distribution of rotation angles in each row of vectors is counted. In the detection phase, the angle peaks and valleys are recorded and the average rotation angle of 5 degrees is obtained by arithmetic averaging. For the case where the variance is higher than twice the average rotation angle, the measurement is performed again and the smoothed value of 7 degrees is taken as The rotation value of The acquisition steps are as follows: the grayscale and texture information of each pixel in the frame feature vector are quantized, the grayscale range is 0 to 255, and the texture direction is divided into four levels. Different levels and grayscale segments are combined and mapped to form a feature value. Combined with the statistical results of large-scale samples, the average value of the feature value is determined to be 23 and the fluctuation range is limited to between 15 and 30. The feature values ​​beyond this interval are additionally analyzed to obtain the correction value, and finally agreed to be 27 as the higher grayscale texture combination value. The acquisition steps are as follows: based on the pixel position offsets of adjacent frames recorded previously, the coordinate changes of each pixel in the time direction are compared one by one and the absolute value is processed, and then the average offset amplitude 6 is obtained as the main offset by the numerical integration method. Indicates the number of rows of the geometric transformation matrix. Here, 4 rows are taken to represent the four main transformation combinations. Represents the dimension of the frame feature vector. Refer to the grayscale and texture combination mapping done previously. Set it to 2 and add an extra column to store local contour features to form a 3-dimensional, Indicates the total number of pixels that are considered to be significantly shifted. Under the above conditions, the total number of pixels is about 55.

[0084] Calculation process:

[0085] The first step is to calculate the molecular part ,in , , , , , , , ,therefore:

[0086]

[0087] The second step is to calculate the denominator , among which , assuming the eigenvalue , , ,but:

[0088]

[0089] Step 3: Calculate and divided by , here take , based on the previously obtained average pixel offset of 6 and the cumulative value of the pixel-by-pixel residual of about 50, the total result is about 330, so:

[0090]

[0091] Step 4: Substitute the above results into the formula:

[0092]

[0093] Step 5: Calculate the square root first , and then calculate ,therefore: ;

[0094] This result shows that when When the value is 1.5616, the discrimination value obtained by combining the geometric transformation information and the translation and rotation data in the current frame is at a relatively high level. If it is further compared with the calculated values ​​of other frames and the frame with the most prominent value is found, it means that the pixel changes contained in the frame are more likely to form a significant focus visually, which is of great reference significance for the subsequent determination of the position of key visual elements.

[0095] Based on the key visual element discrimination value data obtained previously, the discrimination values ​​of all frames are scanned one by one and the difference in values ​​is marked. A discrimination value index is first established for each frame and the frame number and the corresponding discrimination value are put into a sorted list. Then, the discrimination value distribution of each frame is checked in the sorted list in descending order. If the discrimination value exceeds a threshold T2 set based on test experience, for example, after counting about 20,000 video frames, the average value of the discrimination value is about 0.9 and 1.5 times the variance is added to establish T2 as 1.5, then the frame is judged as a high discrimination value frame and its corresponding frame feature is extracted. The coordinate position data recorded in the eigenvector and the geometric transformation matrix are compared with the similar content of the subsequent frames to verify whether there are continuous high-value frames. If the number of continuous high-value frames is not less than 3 frames, they are regarded as target frames that can be analyzed first and included in the fine detection range. Then, the local area coordinates are mined in the corresponding target frame and its specific pixel position, grayscale change, texture detection and other information are registered. After summarizing this information, the specific distribution position of the key visual elements in the scene can be obtained and the differences in adjacent frames can be recorded. Finally, the area with the clearest positioning information is selected to generate the visual key element identification.

[0096] The steps to obtain the priority data packet are:

[0097] Based on the visual key element identification, the image frame information in each data packet is parsed, the corresponding frame number, visual key element type and position data are extracted, and the initial priority data of the frame is obtained;

[0098] According to the initial priority data of the frame, the priority ranking score of the frame is calculated, and the expression is:

[0099]

[0100] in, For the The frame's prioritization score, is the type attribute value of the visual key element, is the spatial distribution coefficient corresponding to the key visual elements, is the characteristic value of the pixel in the frame, is the data packet transmission delay time corresponding to the frame, is the total number of key visual elements, is the number of pixels in the frame;

[0101] Based on the priority ranking score of the frame, all frames in the data packet are rearranged from high to low according to the priority, and the frame transmission order of the data packet is reconstructed to obtain the priority ranking data packet.

[0102] Specifically, based on the visual key element identification obtained previously, the image frame information in each data packet is parsed and the frame numbers are read in sequence. Then, in the application environment, the specific content of each key element is indicated in combination with the visual key element type obtained previously. For example, in a sports scene, "racket" or "player's body part" can be regarded as different types. By collecting about three hundred sets of actual game videos and searching them frame by frame, the timing and position of each key element are recorded. Then, during the parsing process, a temporary record that can reflect the type and coordinate range of the key element is configured for each frame. Each record contains a frame number, a key element classification code, and a two-dimensional coordinate point. If a key element has multiple positions, the record content is expanded to collect them together. Then, all key elements are screened and mapped to the corresponding frames. According to the importance of different key elements to the scene, a spatial distribution table is established for these mappings. The pixel area and center coordinates occupied by each key element are marked in the spatial distribution table, and a local inspection is performed on it in the supporting software. During the inspection, the color uniformity and shape characteristics of the pixels are compared. If the color deviation is within the previously determined range of 0 to 15 and the shape parameters match the known characteristics, they are identified as the same key element. After multiple rounds of scanning, the type and position data of the visual key elements in each frame are obtained, and then several priority factors are established in combination with the scene requirements. For example, the area ratio of the element in the picture and the relative importance of the location are considered, and these factors are configured together with the frame number into an initial priority table, which is summarized to form the initial priority data of the complete frame.

[0103] The formula is beneficial in that it combines the type characteristic value of the key visual element and the corresponding spatial distribution coefficient, integrates the square sum of the characteristic values ​​of all pixels in the frame, and then superimposes the logarithmic quantization of the data packet transmission delay time, thereby taking into account the importance of frames in different dimensions in one expression;

[0104] The acquisition steps are as follows: first, based on the previously obtained visual key element type information, the occurrence frequency and average attention of each element in about 500 frames of scenes with people and moving objects are counted, and the attention is divided into several sections and then the characteristic values ​​of each type are determined based on experimental observations. For example, the characteristic value for the face of a person is set to 7, and the characteristic value for a high-speed moving object is set to 10. The acquisition steps are as follows: using the established scene space distribution table to record the occupied range and center coordinates of the elements in the frame, and according to the proportion of the range occupied by the frame and the superposition relationship of the adjacent elements, a distribution coefficient with a mean of about 2.5 is sorted out by means of segmented statistics. Then, additional measurements are made on the items that deviate from the mean to obtain a range of about 1.0 above and below. Therefore, when applying, the distribution coefficients of different key elements can be assigned values ​​of about 2.5. The acquisition steps are to convert the texture or color features corresponding to all pixels in the frame into numerical values, split the RGB values ​​from 0 to 255 and represent them as the average brightness of each channel, and then combine the texture direction quantization to obtain a comprehensive feature value. Statistics show that in general high-definition motion scenes, its range is about 10 to 40. Feature values ​​beyond this range will be evaluated later. The acquisition steps are: use transmission monitoring means to observe the delay time of each frame in the network environment, take the median or mean of multiple delay acquisition results, and in actual measurement, under stable bandwidth conditions, the delay can be measured to be about 30 milliseconds to 60 milliseconds. After recording it in milliseconds, it is added to 1 for calculation in the logarithmic function. Indicates the total number of visual key elements. When multiple key elements are identified in a frame, they are accumulated in sequence. Indicates the number of pixels in a frame. For a resolution of 1920×1080, it can be approximately 2.0736 million pixels. When a higher resolution is used, the number increases accordingly.

[0105] Calculation process:

[0106] Step 1: Calculation , when two key elements are detected in the frame, they are taken , , then:

[0107]

[0108] Step 2: Calculation , if only a part of the area with a total of 5000 pixels is intercepted at a resolution of 1920×1080, then according to the aforementioned characteristic value range of 10 to 40, the value of each pixel point varies with the acquisition result. After frequency statistics, it can be assumed that the average value is about 25 with slight fluctuations. When calculating the square sum of pixel features, the following can be used:

[0109]

[0110] Step 3: Calculate the denominator ;

[0111] Step 4: Target For example, if the actual measured value of the frame delay is 40 milliseconds, then ,therefore ;

[0112] Step 5: Substitute the above results into the formula:

[0113]

[0114] This result shows that when When it is 3.7405, it means that the current frame has a higher priority ranking score after weighing multiple indicators such as visual key elements, feature values ​​and delay time. When the value is greater than 3, it means that the key elements are more concentrated and the transmission delay remains within the common range. When the value is less than 2, it often means that there are fewer elements or the network delay is too large. Different degrees of sorting can be performed in the subsequent frame transmission order.

[0115] Based on the priority ranking scores of the frames obtained above, a ranking label is assigned to each frame and the frames are arranged in order from large to small according to the scores. First, the frame numbers and corresponding scores recorded in the priority score table are combined. If the scores are in the top 10%, the frames are regarded as extremely high priority. Then, the frames between 10% and 30% are defined as medium priority. Frames below 30% are regarded as ordinary or low priority in practice. In the sorting process, the number of each frame should be matched with the data packet to which it belongs to ensure that the frame sequence after grouping will not be confused with the content of other data packets. Then, the frame list is rearranged in the software environment according to the above priority order. If conflicts are found between frame numbers during the process, corrections are made. For example, when the same number is detected to be accidentally repeated, the frame content should be cross-checked and the only valid record should be selected. After confirming that there is no conflict in the sorting, the entries in the front of the frame list are concentrated to generate a higher priority data packet queue. Then, according to the subsequent data throughput and network status requirements, the remaining frames are compiled into subsequent queues. The entire list is finally merged into a rearranged frame transmission order queue to obtain a priority-ranked data packet.

[0116] The steps to obtain network status monitoring results are as follows:

[0117] Collect network bandwidth utilization data in real time, calculate the average value of the collected bandwidth utilization within a continuous time window, record the bandwidth change trend of each time window, and generate bandwidth utilization record data;

[0118] Collect network transmission delay in real time, timestamp each transmission delay data, calculate the delay distribution characteristics within the time window, and generate network delay record data;

[0119] Based on the bandwidth utilization record data and network delay record data, the dynamic change relationship is analyzed and the status is determined in combination with the network operation status at the current time point to obtain the network status monitoring result.

[0120] Specifically, based on the previously planned bandwidth utilization collection plan, a group of bandwidth real-time readers are first set at the network exit or network core node and the reading frequency is controlled between 1 and 60 times per minute. Then, the bandwidth utilization data collected each time are marked in chronological order and recorded in the collection sequence in seconds or milliseconds. Then, referring to the fluctuation range of actual business traffic, a time window is selected in the software environment, such as the interval of 60 seconds to 120 seconds. All bandwidth readings within the time window are counted and the average value is obtained. At the same time, the difference between the peak and the trough of the bandwidth value from the beginning to the end of the window is calculated, and this difference is compared with the average value within the window. If the difference value exceeds the threshold value T determined based on historical measured experience, the average value is calculated. At 1, for example, in the bandwidth usage records within two weeks, the average bandwidth is about 500Mbps, and combined with the distribution of peaks and low peaks, T1 is 100Mbps. In this case, the window is marked significantly in the record, and then the next time window is continuously advanced. After the rolling analysis of the continuous windows is completed, the trend of the average value sequence of each window is recorded. By comparing the bandwidth means and difference amplitudes of adjacent windows, possible sudden increases or fluctuations can be identified, and the corresponding time points and the range of fluctuations are registered internally. If the fluctuation amplitude exceeds twice the variance, further inspection links are added to eliminate interference from abnormalities or temporary events. After all windows are processed, all bandwidth means and change trend information are summarized to finally obtain bandwidth utilization record data.

[0121] Based on the bandwidth utilization collection mode obtained above, additional monitoring is performed on the network transmission delay, and the round-trip time of each sent and received network data packet is measured. Each delay record is marked according to the same timestamp rule. For example, 30 delay tests are performed every minute, and the delay value is calculated by the difference between the packet return time and the packet sending time. These delay values ​​are then collected in the order of the test and divided into continuous time windows to match the bandwidth. The average and distribution characteristics of all delay values ​​in each window are calculated. For example, it is determined whether the delay values ​​are mainly distributed in the range of 10 milliseconds to 50 milliseconds. If a batch of delay values ​​is dispersed to more than 300 milliseconds, it is recorded as Abnormal points are found, and then a delay distribution curve is formed at the statistical level. In order to clarify the stability of this distribution, the upper and lower quartiles of the distribution range can be calculated and the peak value can be recorded. If the peak value deviates significantly from the threshold T2 obtained based on the previous data statistics, for example, based on the peak concurrency scenario within a month, the average delay is 25 milliseconds and T2 is determined to be 80 milliseconds with a certain multiple of the standard deviation, then such abnormal windows will be additionally marked and their duration will be paid attention to, so as to distinguish instantaneous delay jitter from continuous delay. After the analysis of all time windows is completed, the sorted delay mean, peak value, and upper and lower distribution ranges will be stored as a set of delay record data, and finally the network delay record data will be generated.

[0122] Based on the bandwidth utilization record data and network delay record data obtained above, first align the bandwidth utilization average value sequence and the corresponding delay average value sequence in the application environment, and determine the corresponding relationship between the two in the same time window, and then compare the fluctuations of the bandwidth sequence and the delay sequence. If the bandwidth utilization rises to a significant level in a certain time window and the delay also appears to be higher than the empirical threshold or floating range determined above, then the window is merged and marked as an abnormal fluctuation window, and then the network status is comprehensively judged in combination with the basic monitoring information such as the number of network devices used and request queuing at the current time point. For example, if the device occupancy rate exceeds the predetermined standard range of 30% to 70% calculated based on test experience, it is classified as a busy state. If the delay is marked too high or the bandwidth is too low, it will be set to an overload state. After summarizing all windows and states, each monitoring indicator is matched with the time point for unified inspection, so as to identify whether the network is in a high load, light congestion or normal stability, and finally record the corresponding conclusion internally to form a network status monitoring result.

[0123] The steps to obtain the scheduling optimization data packet are:

[0124] Based on the network status monitoring results, the bandwidth utilization and delay data in the current time window are extracted, and the transmission requirements of the priority-sorted data packets are analyzed one by one. Combined with the size of each data packet, the transmission delay requirement and the current bandwidth status of the network, the initial sorting value of the data packet transmission is generated;

[0125] According to the initial sorting value of the data packet transmission, the transmission priority of the data packet is calculated, and the expression is:

[0126]

[0127] in, For the The transmission priority of each data packet, For the The bandwidth requirement of each packet is For the The transmission delay requirement of each data packet is For the The packet size of the packets, is the stability coefficient related to transmission in the current network state, is the bandwidth utilization of the current network status, is the total transmission delay of the current network, The total number of packets sorted for priority, is the number of parameters of the stability coefficient in the current network state;

[0128] Based on the transmission priority of the data packet, the priority-sorted data packets are sorted from high to low according to the priority. At the same time, the data packets with limited transmission conditions are reallocated and the transmission order is adjusted in combination with the transmission priority of the data packet to generate scheduling optimized data packets.

[0129] Specifically, based on the previously obtained network status monitoring results, the bandwidth utilization and delay data in the current time window are first read and mapped accordingly, and the time window is defined as a shorter fixed length such as 30 seconds or 60 seconds for easy statistics. Then, the basic information of each data packet is retrieved one by one according to the priority-sorted data packets that have been obtained, including its size and the required upper limit of the transmission delay. The amount of bandwidth that each data packet may occupy is numerically recorded in combination with the bandwidth utilization, and then the delay requirement of each data packet is quantified in the same way. For example, the delay distribution curve obtained in the previous large-scale test is used to clarify the delay requirement range for different types of data packets. If it is found that the delay threshold of some data packets is below 10 milliseconds, it is regarded as an extremely short delay requirement data packet. If it is in the range of 10 milliseconds to 100 milliseconds, it is regarded as a medium-short delay requirement data packet. If it exceeds 100 milliseconds, it can be classified as a normal transmission category. When comparing these requirements with the current network bandwidth, bandwidth utilization must also be considered. For example, if the bandwidth utilization rate is close to the empirical range of 60% to 80%, the network is considered to be strained. At this time, the size and delay requirements of each data packet will be recorded and the corresponding initial sorting mark will be set to reflect the increase in transmission delay of large data packets in high bandwidth occupancy scenarios. If the size of a data packet exceeds the previously defined threshold T1, for example, in the comparison of real online environments, it is found that the size of a single data packet is most common when it is in the range of 5MB to 20MB. When it exceeds 20MB, the data packet will be additionally marked in the record to remind that it may occupy higher bandwidth or cause local congestion. After the bandwidth requirements and delay requirements of all data packets are integrated, an incremental scanning method will be used to give a preliminary sorting value. Data packets with higher requirements and relatively smaller volumes often have more advantages in sorting values, while larger data packets with lower delay requirements have correspondingly later sorting values. After that, this initial sorting value is recorded centrally to form the initial sorting value for data packet transmission.

[0130] The usefulness of the formula lies in that it combines the packet bandwidth requirement, delay requirement, packet size, and stability coefficient in the network state, and incorporates the current bandwidth utilization and total transmission delay into the same denominator structure, thus connecting the packet demand with the real-time network environment;

[0131] The acquisition step is to perform segmented statistics on the bandwidth requirements of the data packets obtained above in multiple time windows to determine a relatively constant or fluctuating bandwidth requirement interval, and then perform multiple transmission tests on the same type of data packets, record the bandwidth application amount each time and average the results. For example, after about 50 tests, the average bandwidth requirement is about 2.5Mbps and the fluctuation range is set from 1.0Mbps to 3.5Mbps based on the actual business volume. The acquisition steps are to use delay collection methods to quantify the shortest round-trip time, the longest round-trip time and the main distribution interval of each data packet, and find out the ideal delay requirement of the data packet. If it is a cross-node data packet, the delay distribution may be between 30 milliseconds and 60 milliseconds. At this time, the average delay requirement can be obtained by combining multiple consecutive actual measurements to be around 45 milliseconds. The acquisition steps are: read the actual size of the data packet. If it is between 3MB and 10MB, write it into the variable according to its number of bytes. If the range exceeds 10MB, the records must be split. The acquisition step is to collect the parameters related to network stability one by one and then multiply them. For example, in the monitoring, the wireless network signal fluctuation coefficient is about 1.1 to 1.3, the network routing hop stability coefficient is about 1.05 to 1.2, etc. Finally, the various coefficients are multiplied to obtain , The acquisition step is to summarize the previous bandwidth utilization record data in the current time window. If the average bandwidth utilization is 60%, it is recorded as 0.6. The acquisition step is to accumulate or average the previously obtained network delay record data in the current time window. If the total transmission delay distribution measurement mean is 40 milliseconds, it is recorded as 40. The total number of priority-sorted packets. Based on the previous summary of the number of packets, a statistical value such as 10 can be obtained. is the number of stability coefficients, which can range from 2 to 4 according to the adopted wireless signal fluctuations, network routing, etc.

[0132] Calculation process:

[0133] The first step is to calculate ,For example , read in sequence Mbps, millisecond, MB; Mbps, millisecond, MB; Mbps, millisecond, MB. Note that a unified unit is required. For example, bandwidth Mbps can be converted into a more microscopic value or the data packet size can be converted into bytes for consistency processing. Here, it is simplified to multiply the values ​​to get:

[0134]

[0135] The second step is to calculate ,when For example, if the wireless signal fluctuation coefficient is 1.2 and the network routing hop stability coefficient is 1.1, then:

[0136]

[0137] The third step is to add the above two results:

[0138]

[0139] Step 4: Take the denominator , if the current bandwidth utilization =0.6, total transmission delay ,but:

[0140]

[0141] Step 5: Substitute the result into the formula:

[0142]

[0143] This result shows that when the transmission priority When it is 13.0411, it means that the bandwidth requirement, delay requirement and volume of the current data packet combined with the network status parameters are relatively large. A value higher than 10 generally means that the data packet needs to be arranged at a higher position for transmission. If it is found in subsequent steps that the priority of other data packets is lower than 5 or lower, their transmission order can be moved back to make the transmission resource utilization more in line with the needs of the data packet.

[0144] Based on the data packet transmission priority obtained previously, the sorted data packet list is first extracted in the software environment and quickly sorted from high priority to low priority. For several data packets with similar priority scores, the detailed records of bandwidth requirements and delay requirements are additionally checked. For example, if the priority values ​​of two data packets differ by less than 1.0, they are determined to be of similar levels and can be allocated in the same category. Data packets with limited transmission conditions or whose sizes exceed the previously set threshold T1 are grouped separately and processed in subsequent transmission gaps or when the network status is relatively idle. If the bandwidth utilization rate approaches 70% to 80% again, the transmission activity of the group can be suspended and a prompt can be given. Finally, after all inspections are completed, the sequence list is renumbered to obtain the scheduling optimization data packet.

[0145] The steps for obtaining the adjusted image data packet are:

[0146] Based on the scheduling optimization data packet, the resolution parameters of each image frame are extracted, including width and height resolution. At the same time, the bandwidth utilization rate in the current network conditions and the upper limit of the resolution of the device display capability are extracted to generate an image resolution adaptation parameter set.

[0147] According to the image resolution adaptation parameter set, the resolution adjustment value of each image frame is calculated, and the expression is:

[0148]

[0149] in, is the resolution adjustment value of the image frame, is the width resolution of the current image frame, is the height resolution of the current image frame, The maximum supported width resolution of the device display capability. The maximum supported height resolution of the device display capability. is the current network bandwidth utilization, The refresh rate parameter of the device display capability. is the total number of pixels in the current frame, is the delay correction value in network conditions;

[0150] Based on the resolution adjustment value of the image frame, the width and height resolution of each image frame are dynamically adjusted to obtain an adjusted image data packet.

[0151] Specifically, based on the previously acquired scheduling optimization data packet, the resolution parameters of each image frame are first read and the actual values ​​of width and height are analyzed. To ensure that the collected resolution information is reliable, a special detection step is required in actual applications to record the resolution combination of each frame. At the same time, the bandwidth utilization under the current network conditions is retrieved in the scene and the display capability parameters of the visual device are extracted. For example, the maximum supported resolution in some display terminals is 1920×1080, while more advanced terminals may support 3840×2160. If it is detected that the bandwidth utilization is already in the range of 60% to 80%, it is considered that the network is in a tight state. Dynamic adjustment of the width and height resolution may be further triggered. If the upper limit of the resolution of the device display capability is approximately 1920×1080 but the current sampling frame has reached 2560×14 40, it is necessary to confirm whether the actual network transmission rate of the frame can meet the continuous transmission requirements of this resolution. If not, add an over-limit mark to the frame in the record, and then combine the device refresh rate and network delay information to determine whether there is still a possibility of frame loss or freeze. In the process, the delay is compared with the parameters previously obtained by network status monitoring. If the delay value exceeds a threshold T1 established by test statistics for multiple consecutive times, for example, within a month, the average regular delay is monitored to be about 40 milliseconds, and T1 is established as 100 milliseconds after adding two times the standard deviation, then the frame is registered internally as a high delay risk, and in subsequent adjustments, consider strategies to reduce resolution or reduce frame size occupancy. All this information is summarized in a table to distinguish the resolution and device support of different frames, and combine factors such as bandwidth utilization and delay value, and finally generate an image resolution adaptation parameter set.

[0152] The formula is useful in that it incorporates the width and height resolution of the current image frame, as well as the device display capability, network bandwidth utilization, number of pixels in the frame, and network delay correction value, balancing the coupling relationship between frame resolution and network and device conditions through the same expression;

[0153] The acquisition steps are as follows: the actual width resolution of each frame is recorded frame by frame in the application, and the target frame width value to be adjusted is selected according to the scene requirements in the processing stage. For example, statistics show that the width of a certain group of frames is mostly in different standards such as 1920, 2560, and 3840. The steps to obtain are as follows: Similar but consistent management of height resolution, extracting and classifying the current height frame by frame from the actual shooting or rendering stage, The acquisition step is to detect the specifications of the display device. For example, some displays support a maximum of 3840 (horizontal pixels). If the reading is higher than this value, it is recorded as super resolution and 3840 is recorded in , The steps to obtain are as follows. If the display device claims that it can reach up to 2160 (vertical pixels), write this data to , The acquisition steps are to record the refresh rate of the device through multiple automatic detections and synthesize a complete display refresh sampling data. For example, if the average refresh rate is around 75Hz, then write 75 When the program is executed, it will be adjusted appropriately according to the display's interpolation and actual refresh capabilities. The acquisition steps are: obtain the bandwidth utilization in the current time window through bandwidth monitoring. If the measured value is 0.65, it is recorded as 0.65. The acquisition steps are to count the total number of pixels in the current frame, such as 1920×1080, which is about 2073600. The acquisition step is to compare the delay cumulative value or correction number of each frame in the network status monitoring to obtain the delay correction value. If the monitoring shows that some frames have jitter, L may be set to a correction value of 10 or 20 milliseconds.

[0154] Calculation process:

[0155] The first step is to , Substitute, this frame resolution is higher than the conventional 1080P, , The maximum supported specification of the device is 4K. From the refresh rate statistics, Indicates that the current bandwidth utilization is 65%. , It indicates a 12 millisecond correction value estimated based on recent network jitter;

[0156] The second step is to calculate the three parts of the numerator separately:

[0157]

[0158]

[0159]

[0160] Add the three together to get the numerator:

[0161]

[0162] Step 3: Calculate the denominator ,in , then add ,get:

[0163]

[0164] Step 4: Substitute the result into the formula:

[0165]

[0166] The result shows that when the resolution adjustment value of the current image frame is 109.40, it means that under the combined effect of multiple factors such as the device's 4K display capability, bandwidth utilization, and network delay correction, the frame is expected to need to be reduced or maintained in horizontal and vertical resolution to a certain extent to match the resolution level corresponding to this value. If it is greater than 120, it means that the resolution can be maintained at a moderately high level. If it is lower than 80, it means that the resolution may need to be reduced for transmission.

[0167] Based on the image frame resolution adjustment value obtained previously, the resolution adjustment value of each frame is first compared with its initial width and height in the record. If it is found during detection that the current resolution adjustment value is less than 100 and the maximum supported resolution of the device is 1920×1080, it means that it is necessary to reduce the original 2560×1440 frame resolution to a range close to 1920×1080. If the resolution adjustment value obtained after calculation is higher than a certain empirical threshold T1, such as observing in multiple tests that the picture is clearer and the frame rate does not drop significantly when the adjustment value is greater than 150, the width and height resolutions are reduced by a smaller amount according to the adjustment value, and then the new resolution combination is marked in the record and the occupied bandwidth estimation and delay check are re-performed. All processed frame information is unified and summarized into a new frame data list to finally form an adjusted image data packet.

[0168] The present invention provides a computer network communication data processing system, comprising:

[0169] The video frame differentiation module extracts two consecutive frames from the real-time video frame, calculates the pixel changes between the two frames, and obtains the pixel difference data set;

[0170] The data compression and key element identification module compresses the pixel difference data set to form a compressed difference data packet; based on the compressed difference data packet, it identifies the key visual elements in the image, records the location and attributes of the key visual elements, and generates visual key element identifiers;

[0171] a data packet prioritization module, which uses the visual key element identification to perform priority sorting on the data packets and generate a prioritized data packet;

[0172] The network status adaptation and scheduling optimization module monitors the bandwidth and delay of the current network and generates network status monitoring results. It dynamically adjusts the sending order of priority-sorted data packets based on the network status monitoring results and creates scheduling optimization data packets.

[0173] The image resolution dynamic adjustment module adjusts the resolution of the image data according to the instructions in the scheduling optimization data packet, combined with the network conditions and the display capabilities of the target device, and generates an adjusted image data packet; it is sent to the target device to complete the image data transmission process.

[0174] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A computer network communication data processing method based on artificial intelligence, characterized in that: The following steps are involved: Performing continuous frame difference calculation on the data of real-time video frames, collecting pixel-level change data between two frames, and obtaining a difference data set; compressing the difference data set to generate a compressed difference data packet; Based on the compressed difference data packet, analyzing the image content, identifying key visual elements, including face or action key frames, and generating visual key element identifiers; prioritizing the data packets according to the visual key element identifiers to obtain priority-ordered data packets; Collecting real-time network status data, including bandwidth utilization and delay, to obtain network status monitoring results; adjusting the data packet transmission strategy according to the network status monitoring results, dynamically determining the transmission order of the priority-sorted data packets, and generating scheduling optimization data packets; The resolution of the scheduling optimization data packet is adjusted, the image resolution is dynamically modified according to the current network conditions and the device display capability, the image size is adjusted, and an adjusted image data packet is generated; the adjusted image data packet is sent to the target device to complete the image data transmission process.

2. The method for processing computer network communication data based on artificial intelligence according to claim 1, characterized in that: The steps of obtaining the difference data set are: Collect two frames of continuous image data in the real-time video stream, compare the grayscale value of each pixel one by one, calculate the pixel grayscale difference between the two frames of images, and record the difference change results of all pixels to generate pixel-level difference data; Based on the pixel-level difference data, grouping the pixel difference change results, screening the pixel points whose change values ​​exceed the set threshold, and integrating the positions of the pixel points exceeding the set threshold with the difference values ​​to form pixel change data; Based on the pixel change data, the change records are stored in a structured manner according to the frame number and the change range, and redundant data are removed to generate a difference data set.

3. The computer network communication data processing method based on artificial intelligence according to claim 1 is characterized in that: The steps of obtaining the compressed difference data packet are as follows: Based on the difference data set, the pixel change data of each frame is partitioned according to a preset block rule, the pixel change characteristics in each block are counted one by one, and the pixel value of each block is coded and analyzed in combination with the change range to generate a block pixel change coding result; Based on the block pixel change coding result, the repeated or redundant coding information is removed from the coding content of each block and the data representation structure is readjusted to generate optimized block coding content; Based on the optimized block coding content, the coding data of all blocks are integrated, all block coding is rearranged according to the data structure of the difference data set, and stored in a compressed storage form to generate a compressed difference data packet.

4. The computer network communication data processing method based on artificial intelligence according to claim 1 is characterized in that: The steps for obtaining the visual key element identification are: Based on the compressed difference data packet, the frame features in the data packet are extracted frame by frame, a feature vector set of each frame is constructed, and potential geometric transformation information in the image frame is extracted, and a frame feature vector and a geometric transformation matrix are generated by comparing the local offset of pixel features between adjacent frames; According to the frame feature vector and geometric transformation matrix, the discriminant value of the key visual element is calculated, and the expression is: in, is the discriminant value of the key visual element, is the geometric transformation matrix The translation value of the row, is the geometric transformation matrix The rotation value of the row, is the first The feature value of each pixel, is the offset of the corresponding pixel points in adjacent frames, is the number of rows of the geometric transformation matrix, is the dimension of the feature vector, is the number of offset pixels in the frame; Based on the discrimination value of the key visual element, all frames are analyzed, the frame with the highest discrimination value is selected, and the position of the key visual element is determined by combining the frame feature vector and the positioning information in the geometric transformation matrix to generate a visual key element identifier.

5. The computer network communication data processing method based on artificial intelligence according to claim 1 is characterized in that: The steps of obtaining the priority sorted data packets are as follows: Based on the visual key element identifier, the image frame information in each data packet is parsed, the corresponding frame number, visual key element type and position data are extracted, and the initial priority data of the frame is obtained; According to the initial priority data of the frame, the priority ranking score of the frame is calculated, and the expression is: in, For the The frame's prioritization score, is the type attribute value of the visual key element, is the spatial distribution coefficient of the corresponding visual key elements, is the characteristic value of the pixel in the jth frame, is the data packet transmission delay time corresponding to the jth frame, is the total number of key visual elements, is the number of pixels in the frame; Based on the priority ranking scores of the frames, all frames in the data packet are rearranged from high to low priority, and the frame transmission order of the data packet is reconstructed to obtain a priority ranking data packet.

6. The method for processing computer network communication data based on artificial intelligence according to claim 1, characterized in that: The steps for obtaining the network status monitoring result are: Collect network bandwidth utilization data in real time, calculate the average value of the collected bandwidth utilization within a continuous time window, record the bandwidth change trend of each time window, and generate bandwidth utilization record data; Collect network transmission delay in real time, timestamp each transmission delay data, calculate the delay distribution characteristics within the time window, and generate network delay record data; Based on the bandwidth utilization record data and the network delay record data, the dynamic change relationship is analyzed and the status is determined in combination with the network operation status at the current time point to obtain the network status monitoring result.

7. The computer network communication data processing method based on artificial intelligence according to claim 1 is characterized in that: The steps of obtaining the scheduling optimization data packet are as follows: Based on the network status monitoring results, the bandwidth utilization and delay data in the current time window are extracted, the transmission requirements of the priority-sorted data packets are analyzed one by one, and the initial sorting value of the data packet transmission is generated by combining the size of each data packet, the transmission delay requirement and the current bandwidth status of the network; According to the initial sorting value of the data packet transmission, the transmission priority of the data packet is calculated, and the expression is: in, For the The transmission priority of each data packet, For the The bandwidth requirement of each packet is For the The transmission delay requirement of each data packet is For the The packet size of the packets, is the stability coefficient related to transmission in the current network state, is the bandwidth utilization of the current network status, is the total transmission delay of the current network, The total number of packets sorted for priority, is the number of parameters of the stability coefficient in the current network state; Based on the transmission priority of the data packet, the priority-sorted data packets are sorted from high to low according to priority, and at the same time, the data packets with limited transmission conditions are reallocated and the transmission order is adjusted in combination with the transmission priority of the data packet to generate a scheduling optimized data packet.

8. The method for processing computer network communication data based on artificial intelligence according to claim 1, characterized in that: The steps of acquiring the adjusted image data packet are: Based on the scheduling optimization data packet, the resolution parameters of each image frame are extracted, including width and height resolution, and the bandwidth utilization rate in the current network conditions and the upper limit of the resolution of the device display capability are extracted to generate an image resolution adaptation parameter set; According to the image resolution adaptation parameter set, the resolution adjustment value of each image frame is calculated, and the expression is: in, Adjust the resolution of the image frame. is the width resolution of the current image frame, is the height resolution of the current image frame, The maximum supported width resolution of the device display capability. The maximum supported height resolution of the device display capability. is the current network bandwidth utilization, The refresh rate parameter of the device display capability. is the total number of pixels in the current frame, is the delay correction value in network conditions; Based on the resolution adjustment value of the image frame, the width and height resolution of each image frame are dynamically adjusted to obtain an adjusted image data packet.

9. A computer network communication data processing system according to any one of claims 1 to 8, characterized in that: include: The video frame differentiation module extracts two consecutive frames from the real-time video frame, calculates the pixel changes between the two frames, and obtains the pixel difference data set; The data compression and key element identification module compresses the pixel difference data set to form a compressed difference data packet; based on the compressed difference data packet, it identifies the key visual elements in the image, records the location and attributes of the key visual elements, and generates visual key element identifiers; a data packet prioritization module, which uses the visual key element identification to perform priority sorting on the data packets and generate a prioritized data packet; The network status adaptation and scheduling optimization module monitors the bandwidth and delay of the current network and generates network status monitoring results. It dynamically adjusts the sending order of priority-sorted data packets based on the network status monitoring results and creates scheduling optimization data packets. The image resolution dynamic adjustment module adjusts the resolution of the image data according to the instructions in the scheduling optimization data packet, combined with the network conditions and the display capability of the target device, and generates an adjusted image data packet; Send to the target device to complete the image data transmission process.

Images