Video transmission processing method based on HDMI2.0

Abstract

The invention discloses a video transmission processing method based on HDMI2.0, relates to the technical field of video processing, and solves the technical problems of transmission channel blockage and influence on transmission efficiency caused by simultaneous transmission of a plurality of pieces of image data when the image data in a shared memory is transmitted through a network processor in the prior art. The method comprises the following steps: segmenting an original video to obtain an original video image and an original video audio, and respectively segmenting the original video image and the original video audio; respectively coding the image segment and the audio segment to obtain an image identifier and an audio identifier; respectively calculating the priorities of the image segments and the audio segments; sequentially transmitting the image segments and the audio segments according to the priorities of the image segments and the audio segments; the client side sorts and recombines the received image segments and audio segments, detects whether sound and picture synchronization is carried out on the sorted and recombined video or not, and corrects the asynchronous video; the transmission channel can be prevented from being blocked, and the transmission efficiency can be improved.

Description

Technical Field

[0001] The invention belongs to the field of video processing, and in particular is a video transmission processing method based on HDMI2.0. Background Art

[0002] The bandwidth of HDMI2.0 version is 18Gbps, which supports higher resolution and frame rate. It can also support flicker-free playback of 4K resolution video at a frame rate of 60Hz, while supporting multiple color modes and audio formats. SoC integrates multiple functions on a single chip, including HDMI signal reception, decoding, processing and other possible audio and video processing functions. This high degree of integration reduces the number of components required for the system and reduces the complexity of the system. The SoC integrates a dedicated HDMI signal processor that can process HDMI signals efficiently and accurately, ensuring high-quality audio and video transmission while reducing energy consumption. In addition, the use of SoC integrated processing of HDMI signals can provide clear and realistic audio and video effects, enhancing the user experience.

[0003] The prior art (invention patent with application number 2017800053782) discloses a video transmission method for a video processor (10). The video transmission method comprises: (S11) acquiring a frame of image, the image comprising at least one slice; (S12) acquiring a first-level memory unit from a shared memory, the first-level memory unit comprising at least one second-level memory unit; (S13) encoding the slice; (S14) storing the encoded slice in the second-level memory unit to form image data stored in the shared memory; (S15) notifying a network processor (20) to send the image data in the shared memory; in the prior art, when the image data in the shared memory is sent through the network processor, multiple image data may be sent simultaneously during the process of sending the image data, resulting in a problem of blocking the transmission channel, thereby affecting the transmission efficiency.

[0004] The present invention provides a video transmission processing method based on HDMI2.0 to solve the above technical problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a video transmission processing method based on HDMI2.0, which is used to solve the technical problem that in the prior art, when image data in a shared memory is sent through a network processor, multiple image data may be sent simultaneously during the process of sending the image data, resulting in a transmission channel blockage, thereby affecting the transmission efficiency.

[0006] To achieve the above object, a first aspect of the present invention provides a video transmission processing method based on HDMI2.0, comprising:

[0007] Step S1: obtaining the transmitted original video, segmenting the image and audio in the original video to obtain the original video image and the original video audio; cutting and segmenting the original video image and the original video audio respectively to obtain a plurality of image segments and audio segments;

[0008] Step S2: encoding a plurality of image segments to obtain a plurality of image segment image identifiers; determining the image segments to be transmitted according to the image identifiers; encoding a plurality of audio segments to obtain audio identifiers of the plurality of audio segments;

[0009] Step S3: acquiring basic parameters of the image segment, calculating the image priority of the image segment according to the basic parameters, and transmitting the image segment according to the image priority of the image segment; acquiring audio parameters of the audio segment, calculating the audio priority of the audio segment according to the audio parameters, and transmitting the audio segment based on the audio priority;

[0010] Step S4: After receiving the image segments, the client sorts and reorganizes the image segments according to the image identifiers of the image segments to obtain the video images; after receiving the audio segments, the client sorts and reorganizes the audio segments according to the audio identifiers of the audio segments to obtain the video audio;

[0011] Step S5: Analyze the time of the video image and video audio to determine whether the received video is synchronized with the sound and picture. If yes, integrate the video image and video audio according to the time to obtain the final video; if not, correct the video image or video audio.

[0012] Preferably, the segmenting of the image and the audio in the original video includes:

[0013] Retrieving the original video, analyzing the signal in the original video, and obtaining the image signal and the sound signal;

[0014] The image signal and the sound signal are segmented to obtain the corresponding original video image and original video audio.

[0015] The present invention analyzes the signal in the original video to obtain the image signal and the sound signal; segment the image signal and the sound signal, thereby completing the segmentation of the image and the audio in the original video; provides a basis for subsequent processing of the image and the audio, and can process the image and the audio respectively according to the different characteristics of the image and the audio.

[0016] Preferably, the cutting and segmenting of the original video image to obtain a plurality of image segments includes:

[0017] Retrieving the original video image, obtaining the duration of the original video image and the cutting duration; wherein the cutting duration is set according to actual experience;

[0018] The original video image is evenly cut into several image segments according to the cutting duration.

[0019] When the present invention cuts and segments the original video image, the original video image is cut evenly according to the duration of the original video image and the set cutting duration; it can ensure that the length of each image segment is the same, and lays a foundation for the subsequent encoding of the image segments, which is conducive to improving the transmission of the original video image.

[0020] Preferably, the original video audio is cut and segmented to obtain a plurality of audio clips, including:

[0021] Retrieving the original video and audio, analyzing the signal of the original video and audio, cutting the original video and audio at the point where the signal is 0, and obtaining a number of initial audio clips;

[0022] Obtain the duration of several initial audio segments; determine whether the duration of several initial audio segments is not greater than a duration threshold; if yes, mark the initial audio segments as audio segments; if no, perform secondary cutting on the initial audio segments according to the duration threshold to obtain audio segments.

[0023] It should be noted that when the initial audio segment is cut for the second time according to the duration threshold, the initial audio segment is cut into a plurality of segments evenly, and the duration of each segment is equal and not greater than the duration threshold.

[0024] The present invention cuts the original video and audio at the point where the signal is 0 according to the signal of the original video and audio; when the duration of the initial audio segment is not less than the duration threshold, the initial audio is cut twice according to the duration threshold; the efficiency of the original video and audio transmission can be ensured, which is conducive to achieving efficient transmission.

[0025] Preferably, the encoding of the plurality of image segments to obtain the plurality of image segment image identifiers includes:

[0026] Retrieving a number of image segments and obtaining image parameters of the number of image segments; wherein the image parameters include image pixels, feature points, colors and corresponding color quantities;

[0027] Image segments with the same image parameters are divided into the same group, and each group is encoded to obtain an initial code; a number of image segments are sorted according to the original video to obtain a sequence of the image segments; the initial code is improved according to the sequence of the image segments to obtain image identifiers of the image segments.

[0028] It should be noted that the last N digits of the identifier of the image segment in each group represent the position of the image segment in the original video. The identifiers except the last N digits are consistent, and N is determined according to the number of image segments.

[0029] The present invention divides the image fragments into several groups according to their image parameters, determines the serial numbers of the image fragments in the original video, and improves the encoding of the several images to obtain the identifications of the image fragments; it can avoid the occurrence of image fragment sequence errors that lead to differences from the original video, and lays a foundation for the subsequent transmission of the image fragments.

[0030] Preferably, the step of determining the image segment to be transmitted according to the image identifier includes:

[0031] Retrieving image identifiers of a plurality of image segments, extracting initial codes of the plurality of image segments from the image identifiers, and comparing the initial codes of the plurality of image segments;

[0032] It is determined whether the initial codes of several image segments are consistent. If yes, one of the image segments is retained; if no, it is determined that the corresponding image segment needs to be transmitted.

[0033] The present invention determines the image segments to be transmitted according to the initial coding of several image segments; only one repeated image segment needs to be transmitted, which can reduce the amount of transmitted data and is conducive to improving the transmission efficiency.

[0034] Preferably, encoding the plurality of audio clips to obtain audio identifiers of the plurality of audio clips includes:

[0035] Retrieving a plurality of audio clips, and sorting the plurality of audio clips according to the order of the plurality of audio clips in the original video to obtain an audio clip sorting table;

[0036] The plurality of audio segments are encoded according to the audio segment sorting table to obtain audio identifiers of the plurality of audio segments.

[0037] The present invention sorts a plurality of audio clips according to the order of the audio clips in the original video, and encodes the sorted plurality of audio clips to obtain audio identifiers of a plurality of audio and video clips; the order of the audio clips can be sorted, which provides a basis for the subsequent integration of the audio clips, and is conducive to ensuring that the integrated clips are consistent with the original audio.

[0038] Preferably, calculating the image priority of the image segment according to the basic parameters includes:

[0039] Retrieving basic parameters of the image segment; wherein the basic parameters include required transmission bandwidth, same number and data volume;

[0040] The required transmission bandwidth, the same number and the data volume are marked as TK, XS, TL respectively; the image priority of the image segment is calculated by the formula TYJ=α×TK+β×ln(XS+1)+γ×TL; wherein α, β and γ are proportional coefficients greater than 0.

[0041] Preferably, the transmitting of the image fragments according to the image priorities of the image fragments comprises:

[0042] Retrieving image priorities of image segments, sorting the image segments according to the image priorities, and obtaining an image transmission sorting table; and transmitting the image segments in sequence according to the image transmission sorting table;

[0043] The image identifiers corresponding to the image segments are retrieved, the image identifiers corresponding to the image segments are packaged into data packets, and the data packets and the corresponding image segments are transmitted simultaneously.

[0044] It should be noted that the image identifiers corresponding to the image segments are integrated. For example, if there are two identifiers corresponding to the image segments, the two image identifiers are integrated, packaged into a data packet, and transmitted simultaneously with the corresponding image segments.

[0045] The present invention transmits image fragments according to their image priorities, and packs corresponding image identifiers into data packets, which are transmitted simultaneously with the image fragments; this can avoid transmission congestion caused by excessive data, is beneficial to improving transmission efficiency, and provides a basis for subsequent integration of image fragments.

[0046] Preferably, the step of calculating the audio priority of the audio segment according to the audio parameter comprises:

[0047] Retrieving audio parameters of several audio clips; wherein the audio parameters include audio duration, required transmission bandwidth and data volume;

[0048] The audio duration, the required transmission bandwidth and the data volume are marked as YT, YK, and YL respectively; the audio priority of the audio segment is calculated by the formula YYJ=θ×YT+δ×lg(YK+1)+μ×YL; wherein θ, δ and μ are proportional coefficients greater than 0.

[0049] Preferably, the transmitting of the audio clip based on the audio priority comprises:

[0050] Retrieve the audio priority of the audio clip and the audio identifier of the audio clip; sort the audio clips according to the audio priority from large to small to obtain an audio transmission sorting table; transmit the audio clips and the corresponding audio identifier according to the audio transmission sorting table.

[0051] The present invention sorts the audio clips according to their priorities, and transmits the audio clips and corresponding audio identification performances according to the sorting results, so that the audio clips can be transmitted in sequence, avoiding clogging of the transmission channel, which is beneficial to ensuring the transmission rate.

[0052] Preferably, the step of sorting and reorganizing the image segments according to the image identifiers of the image segments comprises:

[0053] Retrieving data packets corresponding to the image fragments, disassembling the data packets to obtain corresponding image identifiers, sorting and reorganizing the image fragments according to the image identifiers; and analyzing the integrity of the sorted and reorganized image fragments;

[0054] It is determined whether there are any missing image identifiers. If yes, the missing image identifiers and the corresponding image segments are retransmitted; if no, the sorted and reorganized image segments are marked as video images.

[0055] The present invention sorts and reorganizes image fragments according to their image identifiers, and analyzes the integrity of the reorganized image fragments; it can ensure that the transmitted image fragments correspond to the original video, which is beneficial to ensuring the integrity and continuity of the video picture.

[0056] Preferably, the step of sorting and reorganizing the audio segments according to the audio identifiers of the audio segments comprises:

[0057] Retrieving audio identifiers of a plurality of audio clips, sorting and reorganizing the plurality of audio clips according to the audio identifiers, and analyzing the integrity of the sorted and reorganized audio clips;

[0058] Determine whether the audio identifier is missing. If yes, retransmit the missing audio identifier and the corresponding audio segment; if no, mark the sorted and reorganized audio segment as video audio.

[0059] The present invention sorts and reorganizes a plurality of audio segments according to their audio identifiers, analyzes the integrity of the sorted and reorganized audio segments, and retransmits the missing audio segments, thereby ensuring the integrity of the received audio and facilitating the quality of the transmitted video.

[0060] Preferably, the step of determining whether the received video has audio and video synchronization includes:

[0061] Retrieving the duration of the video image and the video audio, and comparing the duration of the video image with the duration of the video audio;

[0062] Determine whether the duration of the video image is consistent with the duration of the video audio; if yes, determine that the received video audio and picture are synchronized; if no, determine that the received video audio and picture are not synchronized.

[0063] The present invention analyzes the duration of the video image and the video audio to determine whether the durations of the two are consistent; and detects the synchronization of the integrated video sound and picture, which can ensure the quality of the received video and is conducive to improving the viewer's experience.

[0064] Preferably, the correction of the video image or video audio includes:

[0065] Retrieving the video image and video audio that are out of sync with the sound and picture, and identifying the video image; aligning the corresponding video sound with the video image to obtain a corrected video;

[0066] The redundant video images and video audio in the corrected video are extracted, and the redundant video images and video audio are cut and deleted to obtain the final video.

[0067] The present invention corrects video images and video audios with unsynchronized sound and picture, aligns the video audios with the video images according to the video images to obtain a corrected video, and cuts and deletes redundant video images and video audios in the corrected video to obtain a final video; this is beneficial to ensuring the quality of the final video and improving the viewer's experience.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The present invention obtains the original video to be transmitted, divides the image and audio in the original video, and cuts and segments the original video image and the original video audio to obtain a plurality of image segments and audio segments; divides the image segments into a plurality of groups according to the image parameters of the plurality of image segments, determines the numbers of the plurality of image segments in the original video, and improves the encoding of the plurality of images to obtain the identification of the plurality of image segments; it can avoid the occurrence of image segment sequence errors resulting in differences from the original video; according to the initial encoding of the plurality of image segments, the image segments to be transmitted are determined; for repeated image segments, only one needs to be transmitted, which can reduce the amount of transmitted data and is conducive to improving the transmission efficiency; sorts the plurality of audio segments according to the order of the audio segments in the original video, and encodes the sorted plurality of audio segments to obtain the audio identification of the plurality of audio and video; it can sort the order of the audio segments, which provides a basis for the subsequent integration of the audio segments, which is conducive to ensuring that the integrated segments are consistent with the original audio; and the image segments and the audio segments are transmitted according to the priorities of the image segments and the audio segments, respectively, which can avoid blocking the transmission channel and is conducive to ensuring the transmission rate.

[0070] 2. The present invention sorts and reorganizes the image segments according to their image identifiers, and analyzes the integrity of the reorganized image segments; it can ensure that the image segments correspond to the original video after transmission, which is conducive to ensuring the integrity and continuity of the video picture; sorts and reorganizes a number of audio segments according to their audio identifiers, and analyzes the integrity of the sorted and reorganized audio segments, and retransmits the missing audio segments, which can ensure the integrity of the received audio, which is conducive to ensuring the quality of the video after transmission; analyzes the duration of the video image and the video audio to determine whether the duration of the two is consistent; detects the synchronization of the integrated video sound and picture, which can ensure the quality of the received video, which is conducive to improving the viewer's experience; corrects the video image and video audio with out-of-sync sound and picture, aligns the video audio with the video image according to the video image, obtains the corrected video, and cuts and deletes the redundant video image and video audio in the corrected video to obtain the final video; it is conducive to ensuring the quality of the final video and improving the viewer's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 A schematic diagram of the steps of the method of the present invention;

[0073] Figure 2 A schematic diagram of the steps of processing and transmitting the original video image of the present invention;

[0074] Figure 3 A schematic diagram of the original video and audio processing and transmission steps of the present invention;

[0075] Figure 4 It is a schematic diagram of the image and audio integration steps of the present invention. DETAILED DESCRIPTION

[0076] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] See also Figure 1 The first aspect of the present invention provides a video transmission processing method based on HDMI2.0, comprising:

[0078] Step S1: obtaining the transmitted original video, segmenting the image and audio in the original video to obtain the original video image and the original video audio; cutting and segmenting the original video image and the original video audio respectively to obtain a plurality of image segments and audio segments;

[0079] Step S2: encoding a plurality of image segments to obtain a plurality of image segment image identifiers; determining the image segments to be transmitted according to the image identifiers; encoding a plurality of audio segments to obtain audio identifiers of the plurality of audio segments;

[0080] Step S3: acquiring basic parameters of the image segment, calculating the image priority of the image segment according to the basic parameters, and transmitting the image segment according to the image priority of the image segment; acquiring audio parameters of the audio segment, calculating the audio priority of the audio segment according to the audio parameters, and transmitting the audio segment based on the audio priority;

[0081] Step S4: After receiving the image segments, the client sorts and reorganizes the image segments according to the image identifiers of the image segments to obtain the video images; after receiving the audio segments, the client sorts and reorganizes the audio segments according to the audio identifiers of the audio segments to obtain the video audio;

[0082] Step S5: Analyze the time of the video image and video audio to determine whether the received video is synchronized with the sound and picture. If yes, integrate the video image and video audio according to the time to obtain the final video; if not, correct the video image or video audio.

[0083] See also Figure 2 , obtain the transmitted original video, analyze the signal in the original video, and obtain the image signal and the sound signal; segment the image signal and the sound signal to obtain the corresponding original video image and original video audio; retrieve the original video image, obtain the duration of the original video image and the cutting duration; wherein the cutting duration is set according to actual experience; and cut the original video image into a plurality of image segments on average according to the cutting duration.

[0084] For example: assuming that an original video is obtained, the signal in the original video is analyzed to obtain an image signal and a sound signal; the image signal and the sound signal are segmented to obtain the corresponding original video image and original video audio; the original video image is processed;

[0085] The original video has a duration of 20 seconds. If the cutting duration is set to 1 second, the original video image will be cut into 20 image segments according to the cutting duration.

[0086] Obtain image parameters of several image fragments; wherein the image parameters include image pixels, feature points, colors and the corresponding number of colors; divide the image fragments with the same image parameters into the same group, encode each group to obtain an initial code; sort the several image fragments according to the original video to obtain the order of the several image fragments; improve the initial code according to the order of the several image fragments to obtain image identifiers of the several image fragments; extract the initial codes of the several image fragments from the image identifiers, and compare the initial codes of the several image fragments; determine whether the initial codes of the several image fragments are consistent, if yes, retain one of the image fragments; if not, determine that the corresponding image fragment needs to be transmitted.

[0087] For example: suppose that image parameters of 20 image fragments are obtained, and the image parameters of the 20 image fragments are compared respectively, the image fragments with the same image parameters are divided into the same small groups, and the image fragments in each group are encoded to obtain the initial encoding; among them, there are 5 image fragments in the first group, and the initial encoding of the 5 image fragments is YSTPA; the image fragments are sorted according to the order of the original video, and the initial encoding is improved to obtain the corresponding image identification; the order of the 5 image fragments is YSTPA09, YSTPA02, YSTPA12, YSTPA16, and YSTPA20 respectively.

[0088] One image segment in each group is selected for transmission, and the corresponding image identifier is packaged into a data packet, which is transmitted simultaneously with the image segment; the first image segment in the first group is packaged into a data packet with the corresponding 5 image identifiers and transmitted simultaneously.

[0089] Retrieve basic parameters of the image fragment; wherein the basic parameters include the required transmission bandwidth, the same number and the data volume; mark the required transmission bandwidth, the same number and the data volume as TK, XS, TL respectively; calculate the image priority of the image fragment by the formula TYJ=α×TK+β×ln(XS+1)+γ×TL; wherein α, β and γ are proportional coefficients greater than 0; sort the image fragments according to the image priority to obtain an image transmission sorting table; transmit the image fragments in sequence according to the image transmission sorting table; retrieve the image identifier corresponding to the image fragment, pack the image identifier corresponding to the image fragment into a data packet, and transmit the data packet and the corresponding image fragment simultaneously.

[0090] It should be noted that α, β and γ are set manually based on experience, and their value range is (0,100].

[0091] For example, suppose that the image priorities of three image segments to be transmitted are calculated, the basic parameters of the three image segments are obtained, and α, β and γ are set to 1, 10 and 0.1 respectively; the specific steps are as follows:

[0092] Image segment A: required transmission bandwidth TK = 2Mbps; same number XS = 5; data volume TL = 500MB; then image priority TYJ = 69.91;

[0093] Image segment B: required transmission bandwidth TK = 1Mbps; same number XS = 3; data volume TL = 250MB; then image priority TYJ = 39.86;

[0094] Image segment C: required transmission bandwidth TK = 3Mbps; same number XS = 10; data volume TL = 700MB; then image priority TYJ = 96.98;

[0095] The transmission order of the three image segments is image segment C, image segment A, and image segment B; the image segments are transmitted in sequence according to the image transmission sorting table.

[0096] See also Figure 3 , retrieve the original video and audio, analyze the signal of the original video and audio, cut the original video and audio at the point where the signal is 0 to obtain a number of initial audio clips; obtain the duration of the several initial audio clips; determine whether the duration of the several initial audio clips is not greater than the duration threshold; if yes, mark the initial audio clip as an audio clip; if not, perform secondary cutting on the initial audio clip according to the duration threshold to obtain an audio clip; sort the several audio clips according to the order of the several audio clips in the original video to obtain an audio clip sorting table; encode the several audio clips according to the audio clip sorting table to obtain audio identifiers of the several audio clips.

[0097] For example, the original video audio is cut into 4 segments, the duration of the 4 initial audio segments is analyzed, and it is found that the duration of the third initial audio segment is greater than the duration threshold, and the third initial audio segment is cut again to obtain the final 5 audio segments;

[0098] The audio clips are sorted according to the order in the original video, and the audio clips are sorted and encoded, and the audio identifiers of the 14 audio clips are obtained as YSYP01, ..., YSYP05.

[0099] Retrieve audio parameters of several audio clips; wherein the audio parameters include audio duration, required transmission bandwidth and data volume; mark the audio duration, required transmission bandwidth and data volume as YT, YK and YL respectively; calculate the audio priority of the audio clip by the formula YYJ=θ×YT+δ×lg(YK+1)+μ×YL; wherein θ, δ and μ are proportional coefficients greater than 0. Sort the audio clips according to the audio priority from large to small, and obtain an audio transmission sorting table; transmit the audio clips and the corresponding audio identifiers according to the audio transmission sorting table.

[0100] For example, suppose the audio priorities of three audio clips are calculated, the audio parameters of the three audio clips are obtained, and θ, δ, and μ are set to 1, 10, and 1 respectively; the specific steps are as follows:

[0101] Audio segment A: audio duration YT = 5S; required transmission bandwidth YK = 2Mbps; data volume YL = 30MB; audio priority YYJ = 39.77;

[0102] Audio segment B: audio duration YT = 8S; required transmission bandwidth YK = 4Mbps; data volume YL = 50NB; audio priority YYJ = 64.99;

[0103] Audio segment C: audio duration YT = 3S; required transmission bandwidth YK = 1Mbps; data volume YL = 20MB; audio priority YYJ = 26.01;

[0104] The audio clips are sorted according to the audio priorities to obtain a corresponding transmission order of audio clip B, audio clip A, and audio clip C; the audio clips are transmitted in sequence according to the order.

[0105] See also Figure 4 , retrieve the data packet corresponding to the image fragment, disassemble the data packet to obtain the corresponding image identifier, sort and reorganize the image fragment according to the image identifier; analyze the integrity of the sorted and reorganized image fragment; determine whether there is a missing image identifier, if so, retransmit the missing image identifier and the corresponding image fragment; if not, mark the sorted and reorganized image fragment as a video image.

[0106] Retrieve audio identifiers of several audio clips, sort and reorganize the several audio clips according to the audio identifiers, and analyze the integrity of the sorted and reorganized audio clips; determine whether the audio identifiers are missing, if yes, retransmit the missing audio identifiers and the corresponding audio clips; if no, mark the sorted and reorganized audio clips as video audio.

[0107] For example: assuming that the transmitted image fragments and audio fragments are received, the image fragments and audio fragments are sorted and reorganized according to the image identifier and the audio identifier, and the integrity of the reorganized image fragments and audio fragments is analyzed. The image fragment is missing the 7th and the audio fragment is missing the 2nd segment. Then, the missing image fragments and audio fragments are retransmitted and then sorted and reorganized to obtain the video image and video audio.

[0108] Retrieve the duration of the video image and video audio, and compare the duration of the video image with the duration of the video audio; determine whether the duration of the video image is consistent with that of the video audio; if yes, determine that the received video sound and picture are synchronized; if not, determine that the received video sound and picture are not synchronized; retrieve the video image and video audio with out-of-sync sound and picture, and identify the video image; align the corresponding video sound with the video image to obtain a corrected video; extract redundant video images and video audio in the corrected video, cut and delete the redundant video images and video audio to obtain a final video.

[0109] For example, if the duration of the video image and the video audio is counted and it is found that the lengths of the two are the same, then the received video audio and video are synchronized;

[0110] If the durations of the two are different, the video image is identified; the corresponding video sound is aligned with the video image to obtain a corrected video; redundant video images and video audio in the corrected video are extracted, and the redundant video images and video audio are cut and deleted to obtain a final video.

[0111] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0112] The working principle of the present invention is as follows: the present invention obtains the original video to be transmitted, separates the image and the audio in the original video, and obtains the original video image and the original video audio; cuts and segments the original video image and the original video audio respectively, and obtains a plurality of image segments and audio segments; encodes the plurality of image segments, and obtains a plurality of image segment image identifiers; determines the image segments to be transmitted according to the image identifiers; encodes the plurality of audio segments, and obtains the audio identifiers of the plurality of audio segments; obtains the basic parameters of the image segments, calculates the image priority of the image segments according to the basic parameters, and transmits the image segments according to the image priority of the image segments; obtains the audio parameters of the audio segments, calculates the audio priority of the audio segments according to the audio parameters, and transmits the audio segments based on the audio priority;

[0113] After receiving the image fragments, the client sorts and reorganizes the image fragments according to the image identifiers of the image fragments to obtain the video image; after receiving the audio fragments, the client sorts and reorganizes the audio fragments according to the audio identifiers of the audio fragments to obtain the video audio; the time of the video image and the video audio is analyzed to determine whether the received video is synchronized with the sound and picture. If so, the video image and the video audio are integrated according to the time to obtain the final video; if not, the video image or the video audio is corrected.

[0114] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A video transmission processing method based on HDMI2.0, characterized in that: include: Step S1: obtaining the transmitted original video, segmenting the image and audio in the original video to obtain the original video image and the original video audio; cutting and segmenting the original video image and the original video audio respectively to obtain a plurality of image segments and audio segments; Step S2: encoding a plurality of image segments to obtain a plurality of image segment image identifiers; determining the image segments to be transmitted according to the image identifiers; encoding a plurality of audio segments to obtain audio identifiers of the plurality of audio segments; Step S3: acquiring basic parameters of the image segment, calculating the image priority of the image segment according to the basic parameters, and transmitting the image segment according to the image priority of the image segment; acquiring audio parameters of the audio segment, calculating the audio priority of the audio segment according to the audio parameters, and transmitting the audio segment based on the audio priority; Step S4: After receiving the image segments, the client sorts and reorganizes the image segments according to the image identifiers of the image segments to obtain the video images; after receiving the audio segments, the client sorts and reorganizes the audio segments according to the audio identifiers of the audio segments to obtain the video audio; Step S5: Analyze the time of the video image and the video audio to determine whether the received video is synchronized with the sound and picture. If yes, integrate the video image and the video audio according to the time to obtain the final video; If not, the video image or video audio is corrected.

2. The video transmission processing method based on HDMI 2.0 according to claim 1, characterized in that: The step of segmenting the image and audio in the original video includes: Retrieving the original video, analyzing the signal in the original video, and obtaining the image signal and the sound signal; The image signal and the sound signal are segmented to obtain the corresponding original video image and original video audio.

3. The video transmission processing method based on HDMI2.0 according to claim 1, characterized in that: The original video image is cut and segmented to obtain a plurality of image segments, including: Retrieving the original video image, obtaining the duration of the original video image and the cutting duration; wherein the cutting duration is set according to actual experience; The original video image is evenly cut into several image segments according to the cutting duration.

4. The video transmission processing method based on HDMI2.0 according to claim 1, characterized in that: The original video audio is cut and segmented to obtain a number of audio clips, including: Retrieving the original video and audio, analyzing the signal of the original video and audio, cutting the original video and audio at the point where the signal is 0, and obtaining a number of initial audio clips; Obtain the duration of several initial audio segments; determine whether the duration of several initial audio segments is not greater than a duration threshold; if yes, mark the initial audio segments as audio segments; if no, perform secondary cutting on the initial audio segments according to the duration threshold to obtain audio segments.

5. The video transmission processing method based on HDMI2.0 according to claim 1, characterized in that: The encoding of the plurality of image segments to obtain the plurality of image segment image identifiers includes: Retrieving a number of image segments and obtaining image parameters of the number of image segments; wherein the image parameters include image pixels, feature points, colors and corresponding color quantities; Image segments with the same image parameters are divided into the same group, and each group is encoded to obtain an initial code; a number of image segments are sorted according to the original video to obtain a sequence of the image segments; the initial code is improved according to the sequence of the image segments to obtain image identifiers of the image segments.

6. The video transmission processing method based on HDMI 2.0 according to claim 1, characterized in that: The step of determining the image segment to be transmitted according to the image identifier includes: Retrieving image identifiers of a plurality of image segments, extracting initial codes of the plurality of image segments from the image identifiers, and comparing the initial codes of the plurality of image segments; It is determined whether the initial codes of several image segments are consistent. If yes, one of the image segments is retained; if no, it is determined that the corresponding image segment needs to be transmitted.

7. The video transmission processing method based on HDMI2.0 according to claim 1, characterized in that: The encoding of the plurality of audio segments to obtain audio identifiers of the plurality of audio segments includes: Retrieving a plurality of audio clips, and sorting the plurality of audio clips according to the order of the plurality of audio clips in the original video to obtain an audio clip sorting table; The plurality of audio segments are encoded according to the audio segment sorting table to obtain audio identifiers of the plurality of audio segments.

8. The video transmission processing method based on HDMI 2.0 according to claim 1, characterized in that: The calculating the image priority of the image segment according to the basic parameters includes: Retrieving basic parameters of the image segment; wherein the basic parameters include required transmission bandwidth, same number and data volume; The required transmission bandwidth, the same number and the data volume are marked as TK, XS, TL respectively; the image priority of the image segment is calculated by the formula TYJ=α×TK+β×ln(XS+1)+γ×TL; wherein α, β and γ are proportional coefficients greater than 0.

9. The video transmission processing method based on HDMI2.0 according to claim 8, characterized in that: The transmitting of the image fragments according to the image priorities of the image fragments comprises: Retrieving image priorities of image segments, sorting the image segments according to the image priorities, and obtaining an image transmission sorting table; and transmitting the image segments in sequence according to the image transmission sorting table; The image identifier corresponding to the image fragment is retrieved, the image identifier corresponding to the image fragment is packaged into a data packet, and the data packet and the corresponding image fragment are transmitted simultaneously.

10. The video transmission processing method based on HDMI2.0 according to claim 1, characterized in that: The step of calculating the audio priority of the audio segment according to the audio parameter includes: Retrieving audio parameters of several audio clips; wherein the audio parameters include audio duration, required transmission bandwidth and data volume; The audio duration, the required transmission bandwidth and the data volume are marked as YT, YK, and YL respectively; the audio priority of the audio segment is calculated by the formula YYJ=θ×YT+δ×lg(YK+1)+μ×YL; wherein θ, δ and μ are proportional coefficients greater than 0.

11. The video transmission processing method based on HDMI2.0 according to claim 10, characterized in that: The transmitting of the audio segment based on the audio priority comprises: Retrieve the audio priority of the audio clip and the audio identifier of the audio clip; sort the audio clips according to the audio priority from large to small to obtain an audio transmission sorting table; transmit the audio clips and the corresponding audio identifier according to the audio transmission sorting table.

12. The video transmission processing method based on HDMI2.0 according to claim 1, characterized in that: The step of sorting and reorganizing the image segments according to the image identifiers of the image segments comprises: Retrieving data packets corresponding to the image fragments, disassembling the data packets to obtain corresponding image identifiers, sorting and reorganizing the image fragments according to the image identifiers; and analyzing the integrity of the sorted and reorganized image fragments; It is determined whether there are any missing image identifiers. If yes, the missing image identifiers and the corresponding image segments are retransmitted; if no, the sorted and reorganized image segments are marked as video images.

13. The video transmission processing method based on HDMI 2.0 according to claim 1, characterized in that: The step of sorting and reorganizing the audio segments according to the audio identifiers of the audio segments comprises: Retrieving audio identifiers of a plurality of audio clips, sorting and reorganizing the plurality of audio clips according to the audio identifiers, and analyzing the integrity of the sorted and reorganized audio clips; Determine whether the audio identifier is missing. If yes, retransmit the missing audio identifier and the corresponding audio segment; if no, mark the sorted and reorganized audio segment as video audio.

14. The video transmission processing method based on HDMI2.0 according to claim 1, characterized in that: The step of determining whether the received video has audio and video synchronization includes: Retrieving the duration of the video image and the video audio, and comparing the duration of the video image with the duration of the video audio; Determine whether the duration of the video image is consistent with the duration of the video audio; if yes, determine that the received video audio and picture are synchronized; if no, determine that the received video audio and picture are not synchronized.

15. The video transmission processing method based on HDMI2.0 according to claim 14, characterized in that: The video image or video audio is corrected, including: Retrieving the video image and video audio that are out of sync with the sound and picture, and identifying the video image; aligning the corresponding video sound with the video image to obtain a corrected video; The redundant video images and video audio in the corrected video are extracted, and the redundant video images and video audio are cut and deleted to obtain the final video.

Images