Video conference processing method, system and device
By obtaining the target video keyframes and data packets in preset sending timing in video conferencing, the problem of video out of synchronization in video conferencing is solved, the synchronization of video packets is achieved, and the user experience and development efficiency are improved.
Patent Information
- Application Number
- CN202110800651.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-07-15
AI Technical Summary
In the case of unreliable network, the video sent by the sending end in the video conference is easily out of sync with the video received by the receiving end, resulting in decoding failure.
The target video keyframe and the corresponding target video data packet are obtained through the preset transmission timing. Starting from the target video data packet, the video data packet continues to send to the receiving end according to the preset transmission timing to ensure the synchronization of the video data packets.
The synchronization of videos between the sending and receiving ends is achieved, the stability and user experience of video conferencing are improved, the development and testing process is simplified, and the development cost of multi-point access equipment is reduced.
Smart Images

Figure CN113612962B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of video conferencing technology, and in particular to a video conferencing processing method, system and device. Background Art
[0002] With the continuous advancement of communication technology, the development of video communication has also advanced by leaps and bounds. Especially in the context of public health events such as epidemics, various industries need to use online platforms to conduct real-time video communications with multiple user terminals, such as live video broadcasts and video conferences. In the process of video conferencing, encryption technology is usually required to encrypt video data packets to avoid information leakage.
[0003] In traditional video conferencing, the sender encrypts the video data packets before sending them to the receiver to achieve video conferencing. However, in the case of unreliable network, the video sent by the sender of the video conferencing will be out of sync with the video received by the receiver. Summary of the invention
[0004] Based on this, it is necessary to provide a video conference processing method, system and device to address the above technical issues.
[0005] In a first aspect, a video conference processing method is provided, the method comprising:
[0006] Sending a plurality of video data packets contained in a video frame to a receiving end according to a preset sending timing;
[0007] If the decoding failure information sent by the receiving end is received, then obtaining a target video key frame located after the currently sent video frame in the preset sending timing, and obtaining a target video data packet corresponding to the target video key frame;
[0008] Starting from the target video data packet, the video data packets are continuously sent to the receiving end according to the preset sending timing.
[0009] In one of the embodiments, the step of obtaining a target video key frame located after a currently transmitted video frame in the preset transmission timing, and obtaining a target video data packet corresponding to the target video key frame, includes:
[0010] Acquire a video packet extension sequence number of the target video key frame, where the video packet extension sequence number is used to indicate the order of the video frame in the preset sending timing;
[0011] According to the video packet extension sequence number, searching for a video packet position index corresponding to the video packet extension sequence number from a pre-acquired fault-tolerant jump table;
[0012] The target video data packet is read according to the video packet position index.
[0013] In one of the embodiments, obtaining the video packet extended sequence number of the target video key frame includes:
[0014] Reading sending count information, where the sending count information is used to record the number of video data packets sent by the sending end to the receiving end;
[0015] According to the sending count information, a video packet extension sequence number corresponding to the target video key frame is obtained.
[0016] In one of the embodiments, obtaining the video packet extended sequence number corresponding to the target video key frame according to the sending count information includes:
[0017] The first video packet extension sequence number greater than the sending count information is searched in the fault-tolerant jump table, and the searched video packet extension sequence number is used as the video packet extension sequence number corresponding to the target video key frame.
[0018] In one of the embodiments, the error-tolerant jump table includes the multiple video key frames, and the video key frames carry the video packet sequence number, the video packet extension sequence number, and the video packet position index.
[0019] In one of the embodiments, the method further comprises: reassigning fixed headers and extended headers of the plurality of video data packets.
[0020] In a second aspect, a video conference processing method is provided, the method comprising:
[0021] receiving a plurality of video data packets included in a video frame sent by a transmitter according to a preset sending timing;
[0022] Sending decoding failure information to the sending end, wherein the decoding failure information is used to instruct the sending end to search for a target video key frame located after a currently sent video frame in the preset sending timing and a target video data packet corresponding to the target video key frame;
[0023] The receiving end receives the video data packets sent from the target video data packet according to the preset sending timing, continues to combine the received video data packets to obtain combined video frames, and decodes the combined video frames.
[0024] In one embodiment, the method further comprises:
[0025] Performing inverse processing on the received video data packet to obtain an inversely processed video data packet, wherein the inversely processed video data packet includes a timestamp and a synchronization source field before the video data packet is reassigned;
[0026] The inversely processed video data packet is decoded.
[0027] In a third aspect, a video conference processing system is provided, the system comprising: a transmitting end and a receiving end;
[0028] in,
[0029] The sending end is used to execute the steps of the method described in the embodiment of the first aspect above;
[0030] The receiving end is used to execute the steps of the method described in the above-mentioned second aspect embodiment.
[0031] In a fourth aspect, a video conference processing device is provided, the device comprising:
[0032] A data sending module, used for sending a plurality of video data packets contained in a video frame to a receiving end according to a preset sending timing;
[0033] A data acquisition module, configured to acquire a target video key frame located after a currently transmitted video frame in the preset transmission timing when receiving the decoding failure information transmitted by the receiving end, and acquire a target video data packet corresponding to the target video key frame;
[0034] The data resending module is used to continue sending the video data packets to the receiving end according to the preset sending timing starting from the target video data packet.
[0035] In a fifth aspect, a video conference processing device is provided, the device comprising:
[0036] A data receiving module, used for receiving a plurality of video data packets contained in a video frame sent by a sending end according to a preset sending timing;
[0037] A decoding information sending module, used for sending decoding failure information to the sending end, wherein the decoding failure information is used for instructing the sending end to search for a target video key frame located after a currently sent video frame in the preset sending timing and a target video data packet corresponding to the target video key frame;
[0038] The data re-receiving module is used to receive the video data packets sent by the sending end starting from the target video data packet according to the preset sending timing, and continue to combine the received video data packets to obtain combined video frames, and decode the combined video frames.
[0039] In the above-mentioned video conference processing method, system and device, the sending end sends multiple video data packets contained in the video frame to the receiving end according to a preset sending timing. If the decoding failure information sent by the receiving end is received, the target video key frame located after the currently sent video frame in the preset sending timing and the target video data packet corresponding to the target video key frame are obtained, and the video data packets are continued to be sent to the receiving end starting from the target video data packet according to the preset sending timing; when the decoding at the receiving end fails, the method can re-acquire a new target video key frame and a corresponding new video data packet, and then combine the new video data packets to obtain a combined video frame, and then re-decode the combined video frame to achieve normal decoding, thereby achieving synchronization of the video sent by the sending end of the video conference and the video received by the receiving end at the test end. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 A diagram of an application environment of a video conference processing method in an embodiment;
[0041] Figure 2 A schematic diagram of a flow chart of a video conference processing method in one embodiment;
[0042] Figure 3 A schematic diagram of a specific process of obtaining a video data packet in an embodiment;
[0043] Figure 4 A schematic diagram of the structure of an extended sequence number in a target video key frame in another embodiment;
[0044] Figure 5 A schematic diagram of a flow chart of obtaining a video packet extended sequence number of a target video key frame in another embodiment;
[0045] Figure 6 It is a specific structural diagram of the RTP fixed header of the original RTP data packet in another embodiment;
[0046] Figure 7 A specific structural diagram of an extended header of an original RTP data packet in another embodiment;
[0047] Figure 8 It is a specific structural diagram of the RTP data packet extension header after reassignment in another embodiment;
[0048] Fig. 9 is another flowchart of a video conference processing method in another embodiment;
[0049] Fig.10 A schematic diagram of a flow chart of a method for decoding a data packet in another embodiment;
[0050] Fig.11is a structural block diagram of a video conference processing device in one embodiment;
[0051] Fig.12 is a structural block diagram of a video conference processing device in another embodiment;
[0052] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0054] The video conference processing method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the sending end 102 and the receiving end 104 communicate through the network. In this embodiment, the sending end 102 and the receiving end 104 are both virtual ends, which can use the existing analog data, that is, the video data packet in this embodiment, to process the video code stream of the multi-point video conference to achieve the synchronization of the audio and video received by the receiving end of the video conference with the audio and video sent by the sending end, and the sending end 102 and the receiving end 104 can both be terminal emulators that comply with the H.323 function. The terminal emulator can integrate the H.323 protocol stack and the session initiation protocol stack, and can initiate calls and be called in video conferences H.323 / sessions. In this embodiment, the above-mentioned sending end 102 can collect several video data packets in each video frame contained in a video code stream generated during a video conference through software such as Wireshark, tcpdump, or hardware such as a switch with a network mirroring function.
[0055] The sending end 102 and the receiving end 104 may be implemented by an independent server or a server cluster consisting of multiple servers, and may also be, but not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices.
[0056] In one embodiment, Figure 2 As shown, a video conference processing method is provided, which is applied to Figure 1 Taking the sending end 102 in the example as an example, the following steps are included:
[0057] S100: Send multiple video data packets included in a video frame to a receiving end according to a preset sending timing.
[0058] Specifically, the above-mentioned preset sending timing can be understood as the corresponding sequential positions of multiple video data packets corresponding to each video frame in each video frame in the same conversation of the video conference, and the position is fixed. It can also be understood as the sequential order of each video frame in the conversation in the same conversation of the video conference. The video frame can include a video key frame, a forward prediction frame and / or a bidirectional prediction frame. When collecting the video code stream, the position of the video key frame, the forward prediction frame and / or the bidirectional prediction frame can be encoded. The data volume of the bidirectional prediction frame and the forward prediction frame is less than the data volume of the video key frame; each video code stream includes multiple video frames, and multiple forward prediction frames and / or bidirectional prediction frames can be included between two video key frames. If a video frame can be composed of 3 video data packets, corresponding to video data packet 1 at position 1, video data packet 2 at position 2, and video data packet 3 at position 3, then the preset sending timing can be 1, 2, 3, that is, the sequence of the video data packets sent by the transmitting end 102 to the receiving end 104 can be video data packet 1, video data packet 2, and video data packet 3. In this embodiment, after sending all video data packets corresponding to a video frame, the transmitter 102 sends all video data packets corresponding to the next video frame, and the order in which these video frames are sent can also be a preset sending sequence. Optionally, each video frame can be divided into multiple video data packets.
[0059] In this embodiment, the video data packet may be an encrypted video data packet corresponding to an encrypted video conference. After the encrypted video conference function is enabled for the physical device, the video code stream collected by the transmitting end 102 may correspond to multiple video frames, and each video frame may be divided into several video data packets, that is, the video data packet may be directly processed at the transmitting end 102 as a sample data packet to achieve video synchronization between the transmitting end and the receiving end. Optionally, the physical device may be a multipoint access unit device and an endpoint device. The endpoint device may initiate a call request for an encrypted video conference to hundreds of multipoint access units. After the call request is responded, an encrypted encrypted video code stream may be generated. The call request carries a preset protocol. The video code stream may include a video code stream corresponding to a video signal in the same session of the video conference and an audio code stream corresponding to an audio signal in the same session. That is, the video data packet may include a video data packet corresponding to a video signal in the same session of the video conference and an audio data packet corresponding to an audio signal; correspondingly, the video signal may be a segment of a dynamic image with coherent content, and the audio signal may be a segment of music or a segment of speech, which is not limited thereto.
[0060] The video code stream collected by the transmitter 102 may be data with different protocols, formats, frame rates and other attributes, but in this embodiment, the collected video code stream complies with the Real-time Transport Protocol (RTP), and the time interval between collecting two adjacent video code streams may be determined according to actual needs. The RTP includes a standard data packet format for transmitting audio and video on the Internet, so in this embodiment, the video data packet may be referred to as an RTP data packet.
[0061] It is understandable that the receiving end 104 can receive multiple video data packets corresponding to each video frame according to the preset sending timing, and after receiving the video data packets, combine these video data packets together to form a video frame, and then decode the composed video frame. In the decoding process, if the receiving end 104 can receive multiple combined video frames and decode each combined video frame in turn, when decoding the first combined video frame, the first combined video frame can be decoded based on the initialization data; when decoding the second combined video frame, the second combined video frame can be decoded based on the first combined video frame; and so on, when decoding the next combined video frame, it can be decoded based on the previous combined video frame.
[0062] If packet loss occurs during the process of sending multiple video data packets corresponding to the current video frame to the receiving end 104, the receiving end 104 will fail to decode. If packet loss does not occur, the receiving end 104 will successfully decode and decode the currently received video frame. In this embodiment, the sending end 102 can send the video data packet to the network through the user datagram protocol and the transmission control protocol to the receiving end 104. The above decoding process can be understood as decryption processing, decompression processing, etc., but in this embodiment, the decoding process can be decryption processing.
[0063] It can also be understood that the sending end 102 can send multiple video data packets to the receiving end 104 in sequence according to a preset sending timing, wherein one video data packet is sent each time, the time interval between two sending operations can be a preset time interval, and the time intervals between two sending operations can be the same or different, as long as the last video data packet can be sent out before the next sending operation.
[0064] S200: If decoding failure information sent by the receiving end is received, a target video key frame located after the currently sent video frame in a preset sending sequence is obtained, and a target video data packet corresponding to the target video key frame is obtained.
[0065] Specifically, after receiving the decoding failure information sent by the receiving end 104, the transmitting end 102 can obtain the target video key frame located after the currently transmitted video frame in the preset transmission timing, and simultaneously obtain the target video data packet corresponding to the target video key frame. Optionally, the decoding failure information can be indicated by a related identifier of the decoding failure, which can be a number, a letter, or other special symbols. Optionally, the decoding failure information can instruct the transmitting end to search for the next video key frame after the currently transmitted video frame, that is, the target video key frame, and the target video data packet corresponding to the target video key frame.
[0066] The target video key frame can be a key frame extracted from all video frames of a bitstream according to the type of video, and can be the first frame image of all video frames, or can be an intermediate frame image or a last frame image of all video frames. In video compression standards such as H.264, the target video key frame carries a compressed image that can initialize decoder parameters. After receiving this frame image, the decoder can decode the original image, as well as the compressed image encoded based on the target video key frame. The above H.264 is a digital video compression format jointly proposed by the International Organization for Standardization and the International Telecommunication Union.
[0067] S300 , starting from the target video data packet, continue to send the video data packets to the receiving end according to the preset sending timing.
[0068] Specifically, after the sending end 102 obtains the target video data packet, it can start from the target video data packet and continue to send the target video data packet and all video data packets after the target video data packet to the receiving end 104 according to the preset sending timing. That is, the receiving end 104 can receive the new video data packet, and then combine the new video data packets to obtain a combined video frame, and then decode the combined video frame. If the decoding fails, steps S200 to S300 can be continued until the decoding is successful.
[0069] In the above-mentioned video conference processing method, the sending end sends multiple video data packets contained in the video frame to the receiving end according to the preset sending sequence. If the decoding failure information sent by the receiving end is received, the target video key frame located after the currently sent video frame in the preset sending sequence and the target video data packet corresponding to the target video key frame are obtained, and the video data packet is continued to be sent to the receiving end according to the preset sending sequence starting from the target video data packet. This method can use pre-encrypted video data packets to effectively reduce the cost of setting up a test environment for developing and docking encrypted code streams for multi-point access unit devices and endpoint devices for video conferencing, and can achieve synchronization of the video sent by the sending end of the video conference with the video received by the receiving end at the test end, achieving the effect of synchronous testing, improving the user experience of using the video conferencing function, and also improving the test efficiency of synchronizing the video sent by the sending end with the video received by the receiving end; at the same time, it is convenient for developers to discover problems in the development and testing process.
[0070] In one scenario, in order to synchronize the video sent by the sending end 102 of the video conference with the video received by the receiving end 104, the sending end 102 needs to reacquire new video key frames and corresponding video data packets to achieve a successful decoding state; Figure 3 As shown, the step of obtaining the target video key frame located after the currently transmitted video frame in the preset transmission timing in the above S200, and obtaining the target video data packet corresponding to the target video key frame, can be specifically implemented by the following steps:
[0071] S210, obtaining a video packet extension sequence number of a target video key frame, where the video packet extension sequence number is used to indicate the order of the video frame in a preset sending timing, wherein the target video key frame carries a video packet sequence number, a video packet extension sequence number, and a video packet position index.
[0072] Specifically, the above-mentioned target video key frame can carry a video packet sequence number, a video packet extension sequence number and a video packet position index. Since the video data packet includes a video data packet corresponding to the video signal and an audio data packet corresponding to the audio signal, the target video key frame can specifically carry a video packet sequence number, a video packet extension sequence number, a video packet position index, an audio packet sequence number, an audio packet extension sequence number and an audio packet position index.
[0073] It can also be understood that the transmitting end 102 can find the target video key frame according to the video frame corresponding to the currently transmitted video data packet, and then obtain the video packet extension sequence number corresponding to the target video key frame, and the video packet extension sequence number corresponding to the currently transmitted video frame can be smaller than the video packet extension sequence number corresponding to the target video key frame. Several video data packets contained in each video frame all have corresponding video packet extension sequence numbers, and the video packet extension sequence number completes encoding according to the sequence of collection in the video data packet collection process. The currently transmitted video frame can be understood as the video frame corresponding to the currently transmitted video data packet.
[0074] In this embodiment, the transmitting end 102 can obtain each video frame according to a certain preset period, and each time the video data packet corresponding to the video frame is sent to the receiving end 102, the video packet sequence number, video packet extension sequence number, video packet position index, audio packet sequence number, audio packet extension sequence number and audio packet position index carried by the video frame will be recorded. Optionally, the video packet extension sequence number and the audio packet extension sequence number are both 32 bits, and their composition is as follows Figure 4 As shown, ROC (rollover counter) represents the sequence number rollover count, the sequence number seq is 16 bits, and ROC starts from 0. When the sequence number increases from 0 to 65535 and then returns to 0, ROC can increase by 1.
[0075] Among them, Figure 5 As shown, the step of obtaining the video packet extended sequence number of the target video key frame in the above S210 may specifically include the following steps:
[0076] S211. Read sending count information, where the sending count information is used to record the number of video data packets sent by the sending end to the receiving end.
[0077] Optionally, the sending count information may represent the number of video data packets that the sending end 102 has sent to the receiving end 104, but it does not matter whether the receiving end 104 has received the video data packet, as long as the sending end 102 has sent it. The sending count information may be represented by a numerical value or by a letter; however, in this embodiment, the sending count information is represented by a numerical value, the initial value of which is 0, and each time the sending end 102 sends a video data packet to the receiving end 104, the value increases by 1.
[0078] S212: Obtain a video packet extension sequence number corresponding to the target video key frame according to the sending count information.
[0079] It is understandable that the transmitting end 102 can find the video packet extended sequence number equal to the sending count information from the fault-tolerant jump table according to the recorded sending count information, and can also find the video packet extended sequence number greater than the sending count information from the fault-tolerant jump table according to the recorded sending count information. Of course, it is also understandable that the transmitting end 102 can find all the video packet extended sequence numbers greater than the sending count information in the fault-tolerant jump table, and use one of the video packet extended sequence numbers as the video packet extended sequence number corresponding to the target video key frame.
[0080] Among them, the step of searching the video packet extension sequence number corresponding to the target video key frame from the fault-tolerant jump table according to the sending count information in the above S212 may include: searching the first video packet extension sequence number greater than the sending count information in the fault-tolerant jump table, and using the found video packet extension sequence number as the video packet extension sequence number corresponding to the target video key frame.
[0081] In this embodiment, the transmitter 102 can search in the fault-tolerant jump table in the order of video key frames from 1 to N, find the first video packet extension sequence number that is greater than the sending count information, and use the video packet extension sequence number as the video packet extension sequence number corresponding to the target video key frame.
[0082] S220: According to the video packet extension sequence number, search for the video packet position index corresponding to the video packet extension sequence number in a pre-acquired error-tolerant jump table, wherein the error-tolerant jump table includes a plurality of video key frames.
[0083] In fact, the video packet sequence number, video packet extended sequence number, video packet position index, audio packet sequence number, audio packet extended sequence number and audio packet position index contained in the fault-tolerant jump table are one-to-one corresponding. The transmitting end 102 can search the video packet position index and audio packet position index corresponding to the next video packet extended sequence number adjacent to the video packet extended sequence number from the fault-tolerant jump table obtained in advance according to the video packet extended sequence number, and can also search the video packet position index and audio packet position index corresponding to the previous video packet extended sequence number adjacent to the video packet extended sequence number from the fault-tolerant jump table obtained in advance according to the video packet extended sequence number. Of course, the video packet position index and audio packet position index corresponding to the video packet extended sequence number can also be searched from the fault-tolerant jump table obtained in advance according to the video packet extended sequence number. But in the present embodiment, the transmitting end 102 can specifically search the video packet position index and audio packet position index corresponding to the video packet extended sequence number from the fault-tolerant jump table obtained in advance according to the video packet extended sequence number. Table 1 below is an example of a fault-tolerant jump table, which includes N video key frames. The video key frames can be determined during the process of collecting video data packets. There are several forward prediction frames and / or bidirectional prediction frames between any two video key frames.
[0084] Table 1
[0085]
[0086] S230 , reading a target video data packet according to the video packet position index.
[0087] Specifically, the transmitting end 102 can read the target video data packet corresponding to the video packet position index and the audio packet position index according to the found video packet position index and the audio packet position index, and can also read the target video data packet corresponding to the adjacent next video packet position index and the next audio packet position index according to the found video packet position index and the audio packet position index, and of course, can also read the target video data packet corresponding to the adjacent previous video packet position index and the previous audio packet position index according to the found video packet position index and the audio packet position index. However, in the present embodiment, the transmitting end 102 can specifically read the target video data packet corresponding to the video packet position index and the audio packet position index according to the found video packet position index and the audio packet position index.
[0088] When the receiving end is in a decoding failure state, the above-mentioned video conferencing processing method needs to re-acquire the next video key frame and the corresponding video data packet corresponding to the currently sent video frame, so that the receiving end can re-decode the newly acquired combined video frame to achieve normal decoding, so that the video sent by the sending end and the video received by the receiving end can be synchronized, thereby improving the user experience of using the video conferencing function, and adding a processing strategy for achieving synchronization after responding to the decoding failure of the receiving end.
[0089] In one scenario, in order not to affect the processing of non-encrypted links such as statistical packet loss counts and frame rates of video data packets received by the receiving end 104, this embodiment requires the sending end 102 to reassign the video data packets before sending the video data packets to the receiving end 104. Therefore, in this embodiment, the above-mentioned video conferencing processing method may also include: reassigning fixed headers and extended headers of multiple video data packets.
[0090] In this embodiment, the transmitting end 102 performs re-assignment processing before sending the video data packets, and the video data packets received by the receiving end 104 can all be the re-assigned video data packets.
[0091] Among them, Figure 6 The figure shows the specific structure of the RTP fixed header of the RTP data packet before reassignment. Figure 6 Where V represents the version number, which is fixed to 2 (2 bits), P represents padding (1 bit), X represents extension (1 bit), CC represents CSRC count (4 bits), M represents flag (1 bit), Payload type (PT) represents payload type (7 bits), sequence number (seq) represents sequence number (16 bits), timestamp represents timestamp (32 bits), synchronization source (SSRC) identifier represents synchronization source flag, and contributing source (CSRC) identifiers represents contributing source list. The sequence number field in the video data packet (RTP data packet) can represent the count of the RTP data packet, and its value cycles between 0 and 65535. The sequence number field information can be used by the receiving end to determine whether there is packet loss, calculate the packet loss rate, and reorder after disorder caused by network transmission. The step of the sequence number field is 1; the timestamp field in the video data packet (RTP data packet) can represent the range of 0 to 4294967295, and the step of the timestamp field is basically not equal to the step of the sequence number field; packet loss can actually indicate that the video data packet is not successfully sent from the sender to the receiver. The sequence number value of the RTP fixed header in the video data packet (RTP data packet) can start from 0. The timestamp field in the video data packet can represent the sampling time value of the audio code stream and video code stream carried by the RTP data packet, which can be used for synchronization and jitter calculation. Its value is generally related to the RTP payload or a special definition.
[0092] like Figure 7 The figure shows the specific structure of the extended header of the RTP data packet before reassignment. Figure 8The structure diagram of the extended header obtained after re-assigning the RTP data packet. In this embodiment, the specific structure of the new RTP data packet obtained after re-assignment consists of the RTP data packet fixed header, the RTP data packet extended header and the RTP payload. In the process of re-assignment, the X field in the RTP data packet fixed header can be set to 1, and then the extended header is filled after the RTP data packet fixed header, and the Srctimestamp and Srcseq fields in the extended header can be the original timestamp and seq in the video data packet before re-assignment.
[0093] In this embodiment, the timestamp field can indicate the display time between the current frame and the next frame of the video, and one step is added for each additional frame. For example, for video, the clock frequency is generally set to 90kHz, and the frame rate is 30 frames. Between frames, the timestamp step used is 90000 / 30=3000, that is, if the timestamp value of the current frame of the video is 6666 (the initial value can be random), then the timestamp of all RTP data packets of the next frame of the video is 3000+6666=9666. For audio, in audio with a fixed sampling rate, a video data packet is sent at each fixed sampling period, and its timestamp = sampling rate / number of samples per second. For example, at a sampling rate of 16000Hz, 20ms is read from the sender each time, so the timestamp step can be 320. Among them, different audio and video protocols in video conferencing have different RTP packet definitions, but most of them use the above calculation method to calculate the timestamp, such as H.263, H.264, H.265 for video, AAC, G.711 for audio, etc.
[0094] In the above-mentioned video conferencing processing method, before the sending end sends the video data packet to the receiving end, the video data packet is first reassigned to avoid affecting the statistical packet loss count, frame rate and other non-encrypted processing of the video data packet received by the receiving end 104. That is, the method can use a pre-encrypted video data packet, and reassign the encrypted video data packet, and then send the repeated assignment result to the receiving end, so that the video data packets received by the receiving end can be combined together and decoded normally.
[0095] In one scenario, in order to synchronize the video received by the receiving end 104 with the video sent by the sending end 102, the receiving end 104 needs to combine the received video data packets to generate a combined video frame, and decode the combined video frame. After the decoding is successful, the synchronization effect is achieved. Therefore, in one embodiment, a video conference processing method is provided, such as Fig. 9 As shown, this method can be implemented by:
[0096] S400: Receive multiple video data packets included in a video frame sent by a transmitter according to a preset sending timing.
[0097] Specifically, for the receiving end 104, the transmitting end 102 sends multiple video data packets contained in the video frame to the receiving end 104 in sequence according to the preset sending timing. The receiving end 104 can receive the multiple video data packets sent by the transmitting end 102 to the receiving end 104 in sequence according to the preset sending timing. However, when the network is unstable, there will be data packet loss. At this time, the receiving end 104 may not receive one or more video data packets in the multiple video data packets in the same video frame. The above-mentioned preset sending timing can be understood as the corresponding sequential positions of the multiple video data packets corresponding to each video frame in the same conversation of the video conference, and the position is fixed. It can also be understood as the sequential order of each video frame in the conversation in the same conversation of the video conference.
[0098] In this embodiment, the video data packets received by the receiving end 104 are all video data packets that have been re-assigned, that is, the fixed header and the extended header of the video data packet are data that have been re-assigned. The specific process of re-assignment can be as follows: Figure 8 The embodiments shown are described and will not be repeated here. After receiving all the video data packets corresponding to each video frame, the receiving end 104 combines these video data packets together to form a combined video frame, and then decodes the combined video frame. When the receiving end 104 decodes the first combined video frame, it can decode the first combined video frame based on the initialization data; when decoding the second combined video frame, it can decode the second combined video frame based on the first combined video frame; and so on, when decoding the next combined video frame, it can be decoded based on the previous combined video frame.
[0099] S500, sending decoding failure information to the sending end, where the decoding failure information is used to instruct the sending end to search for a target video key frame and a target video data packet corresponding to the target video key frame in a preset sending sequence that is located after a currently sent video frame.
[0100] Specifically, the receiving end 104 may send decoding failure information to the transmitting end 102. When the transmitting end 102 receives the decoding failure information, it will search for the next video key frame after the current video frame, that is, the target video key frame, and the target video data packet corresponding to the target video key frame. Optionally, the decoding failure information may be indicated by a relevant identifier of the decoding failure, which may be a number, a letter, or other special symbols.
[0101] S600: The receiving end sends video data packets starting from the target video data packet according to the preset sending timing, and continues to combine the received video data packets to obtain combined video frames, and decodes the combined video frames.
[0102] It is understandable that, after the transmitting end 102 finds the target video data packet, it can continue to send the target video data packet and all video data packets after the target video data packet to the receiving end 104. Optionally, the receiving end 104 can combine the received new video data packets to obtain a combined video frame, decode the combined video frame, and obtain a decoding result, and the decoding result in this case can be a decoding failure message or a decoding success message. In the present embodiment, if the receiving end 104 combines the new video data packets to obtain a combined video frame, decodes the combined video frame, and after the decoding fails, the steps of S500 to S600 can be continued until the decoding is successful. In the present embodiment, the combined video data packet is a video data packet contained in the same video frame sent by the transmitting end 102.
[0103] It is also understandable that after each decoding failure of the receiving end 104 and a new video data packet is received again, the receiving end 104 may delete all previously stored video data packets to reduce the memory of the receiving end 104 .
[0104] In some scenarios, the video data packets received by the receiving end 104 are the result of re-assignment processing. Therefore, when the receiving end 104 combines the video data packets to obtain the combined video frames and decodes the combined video frames, it is necessary to obtain the original code stream of the combined video frames for correct decoding. As one of the embodiments, Fig.10 As shown, the above video conference processing method may further include the following steps:
[0105] S700: Perform inverse processing on the received video data packet to obtain the inversely processed video data packet, where the inversely processed video data packet includes the timestamp and synchronization source field before the video data packet is reassigned.
[0106] Among them, after receiving the video data packet, the receiving end 104 can first perform inverse processing on the received video data packet to obtain the inversely processed video data packet. Optionally, the inverse processing can be understood as data restoration processing, and the processing method adopted can be a deconvolution restoration algorithm, a data recovery algorithm, etc. In this embodiment, when the sending end 102 sends the original video data packet to the receiving end 104, if the serial number field of the original video data packet is only 101 to 65530, the sending end 102 assumes that the original video data packet is not reassigned before sending, then, after sending 65530, it starts to send 101 again. At this time, the receiving end 104 does not know whether there is data packet loss, but after reassigning the original video data packet, the receiving end 104 can normally determine whether the serial number of the video data packet is continuous; the same is true for the reassignment of the timestamp field.
[0107] In this embodiment, the above inverse processing may specifically be to restore the timestamp and sequence number fields of the extended header of the video data packet to the timestamp and sequence number fields of the fixed header of the video data packet before reassignment.
[0108] S800: Combine the inversely processed video data packets to generate combined video frames, and decode the combined video frames.
[0109] Specifically, the receiving end 104 may combine the inversely processed video data packets to generate combined video frames, and decode the combined video frames. Optionally, the decoding method may be a traditional audio and video decoding method, which will not be described in detail.
[0110] In the above-mentioned video conference processing method, the receiving end receives the video data packets sent by the sending end starting from the target video data packet according to the preset sending timing, and continues to decode the received video data packets, reversely processes the received video data packets, obtains the reversely processed video data packets, combines the reversely processed video data packets to generate combined video frames, and decodes the combined video frames; this method can use pre-encrypted video data packets to effectively reduce the cost of setting up a test environment for developing and docking encrypted code streams for multi-point access unit devices and endpoint devices for video conferencing, and can achieve synchronization of the video sent by the sending end of the video conference with the video received by the receiving end at the test end, achieving the effect of synchronous testing, improving the user experience of using the video conferencing function, and also improving the test efficiency of the synchronization of the video sent by the sending end with the video received by the receiving end; at the same time, it is convenient for developers to discover problems in the development and testing process.
[0111] Continue to see Figure 1, another embodiment provides a specific structural diagram of a video conference processing system; the video conference processing system includes: a transmitting end 102 and a receiving end 104.
[0112] The sending end 102 is used to execute the above Figure 2 , Figure 3 and Figure 5 The method in any corresponding embodiment;
[0113] The receiving end 104 is used to execute the above Figure 9-10 The method in the corresponding embodiment.
[0114] Specifically, the sending end 102 can send multiple video data packets contained in the video frame and multiple target video key frames corresponding to the multiple video data packets to the receiving end according to the preset sending timing. If the decoding failure information sent by the receiving end is received, the target video key frame located after the currently sent target video key frame in the preset sending timing and the target video data packet corresponding to the target video key frame are obtained, and starting from the target video key frame and the target video data packet, the video data packets and the target video key frame are continued to be sent to the receiving end according to the preset sending timing.
[0115] In addition, the receiving end 104 can receive the video data packets sent by the sending end starting from the target video data packet according to the preset sending timing, and continue to combine the received video data packets to obtain combined video frames, decode the combined video frames, inversely process the received video data packets, obtain the inversely processed video data packets, and decode the inversely processed video data packets.
[0116] The video conference processing system provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.
[0117] It should be understood that although Figure 2 , 3 The steps in the flowcharts of , 5, 9 and 10 are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 , 3 At least part of the steps in , 5, 9 and 10 may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0118] In one embodiment, Fig.11 As shown, a video conference processing device is provided, including: a data sending module 11, a data acquisition module 12 and a data resending module 13, wherein:
[0119] The data sending module 11 is used to send a plurality of video data packets contained in a video frame to a receiving end according to a preset sending timing;
[0120] The data acquisition module 12 is used to acquire a target video key frame located after the currently transmitted video frame in the preset transmission timing when receiving the decoding failure information transmitted by the receiving end, and acquire a target video data packet corresponding to the target video key frame;
[0121] The data resending module 13 is used to continue sending video data packets to the receiving end according to a preset sending timing starting from the target video data packet.
[0122] The video conference processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0123] In one embodiment, the data acquisition module 12 includes: a serial number acquisition unit, a search unit and a data reading unit, wherein:
[0124] A sequence number acquisition unit, used to acquire a video packet extension sequence number of the target video key frame, wherein the video packet extension sequence number is used to indicate the order of the video frame in the preset sending timing, and the target video key frame carries a video packet sequence number, a video packet extension sequence number, and a video packet position index;
[0125] A search unit, configured to search, according to the video packet extension sequence number, for a video packet position index corresponding to the video packet extension sequence number from a pre-acquired fault-tolerant jump table;
[0126] The data reading unit is used to read the target video data packet according to the video packet position index.
[0127] The video conference processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0128] In one embodiment, the serial number acquisition unit includes: a counting information reading subunit and a serial number acquisition subunit, wherein:
[0129] The counting information reading subunit is used to read the sending counting information, and the sending counting information is used to record the number of video data packets sent by the sending end to the receiving end;
[0130] The sequence number acquisition subunit is used to acquire the video packet extended sequence number corresponding to the target video key frame according to the sending count information.
[0131] The video conference processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0132] In one embodiment, the sequence number acquisition subunit is specifically used to search the first video packet extended sequence number greater than the sending count information in the fault-tolerant jump table, and use the found video packet extended sequence number as the video packet extended sequence number corresponding to the target video key frame.
[0133] The fault-tolerant jump table includes a plurality of video key frames, and the video key frames carry a video packet sequence number, a video packet extension sequence number, and a video packet position index.
[0134] The video conference processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0135] In one of the embodiments, the video conference processing device further includes: a re-assignment module, the re-assignment module being used to re-assign fixed headers and extended headers of a plurality of video data packets.
[0136] The video conference processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0137] In one embodiment, Fig.12 As shown, a video conference processing device is provided, comprising: a data receiving module 14, a decoding information sending module 15 and a data re-receiving module 16, wherein:
[0138] The data receiving module 14 is used to receive a plurality of video data packets contained in the video frame sent by the sending end according to a preset sending timing;
[0139] A decoding information sending module 15 is used to send decoding failure information to the sending end, wherein the decoding failure information is used to instruct the sending end to search for a target video key frame located after a currently sent video frame in the preset sending timing and a target video data packet corresponding to the target video key frame;
[0140] The data re-receiving module 16 is used to receive the video data packets sent by the sending end starting from the target video data packet according to the preset sending timing, and continue to combine the received video data packets to obtain combined video frames, and decode the combined video frames.
[0141] The video conference processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0142] In one embodiment, the video conference processing device further includes: an inverse processing module and a decoding module, wherein:
[0143] The inverse processing module is used to perform inverse processing on the received video data packet to obtain the inversely processed video data packet, wherein the inversely processed video data packet includes the timestamp and synchronization source field before the video data packet is reassigned;
[0144] The decoding module is used to decode the video data packet after the inverse processing.
[0145] The video conference processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0146] For the specific definition of the video conference processing device, please refer to the definition of the video conference processing method above, which will not be repeated here. Each module in the above video conference processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0147] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video data packets and video frames. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video conference processing method is implemented.
[0148] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0150] Sending a plurality of video data packets contained in a video frame to a receiving end according to a preset sending timing;
[0151] If the decoding failure information sent by the receiving end is received, a target video key frame located after the currently sent video frame in the preset sending timing is obtained, and a target video data packet corresponding to the target video key frame is obtained;
[0152] Starting from the target video data packet, the video data packets continue to be sent to the receiving end according to the preset sending timing.
[0153] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are further implemented:
[0154] receiving a plurality of video data packets included in a video frame sent by a transmitter according to a preset sending timing;
[0155] Sending decoding failure information to the sending end, where the decoding failure information is used to instruct the sending end to search for a target video key frame and a target video data packet corresponding to the target video key frame in a preset sending timing sequence that is located after the currently sent video frame;
[0156] The receiving and transmitting end starts from the target video data packet, sends the video data packet according to the preset sending timing, continues to combine the received video data packets to obtain combined video frames, and decodes the combined video frames.
[0157] In one embodiment, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0158] Sending a plurality of video data packets contained in a video frame to a receiving end according to a preset sending timing;
[0159] If the decoding failure information sent by the receiving end is received, a target video key frame located after the currently sent video frame in the preset sending timing is obtained, and a target video data packet corresponding to the target video key frame is obtained;
[0160] Starting from the target video data packet, the video data packets continue to be sent to the receiving end according to the preset sending timing.
[0161] In one embodiment, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are further implemented:
[0162] receiving a plurality of video data packets contained in a video frame sent by a transmitter according to a preset sending timing;
[0163] Sending decoding failure information to the sending end, where the decoding failure information is used to instruct the sending end to search for a target video key frame and a target video data packet corresponding to the target video key frame in a preset sending timing sequence that is located after the currently sent video frame;
[0164] The receiving and transmitting end starts from the target video data packet, sends the video data packet according to the preset sending timing, continues to combine the received video data packets to obtain combined video frames, and decodes the combined video frames.
[0165] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0166] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0167] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A video conference processing method, It is characterized in that The method comprises: Sending a plurality of video data packets contained in a video frame to a receiving end according to a preset sending timing; the preset sending timing is the corresponding order of each of the video data packets in the video frame; If the decoding failure information sent by the receiving end is received, a target video key frame located after the currently sent video frame in the preset sending timing is obtained, and a video packet extension sequence number of the target video key frame is obtained, where the video packet extension sequence number is used to indicate the order of the target video key frame in the preset sending timing, and the target video key frame carries a video packet sequence number, a video packet extension sequence number, and a video packet position index; According to the video packet extended sequence number, the video packet position index corresponding to the video packet extended sequence number is searched from the pre-acquired fault-tolerant jump table, and the target video data packet is read according to the video packet position index; the fixed header and the extended header in the video data packet and the target video data packet are both reassigned data; in the reassignment process, the X field in the fixed header in the video data packet before reassignment and the target video data packet before reassignment is set to 1, and the extended header is filled after the reassigned fixed header, and the Srctimestamp and Srcseq fields in the extended header are the original timestamp and seq of the fixed header in the video data packet before reassignment and the target video data packet before reassignment; the video data packet and the target video data packet are both encrypted data packets obtained after the sender turns on the encrypted video conferencing function; the X represents extension, the Srctimestamp and the timestamp both represent timestamps, and the Srcseq and the seq both represent sequence numbers; Starting from the target video data packet, the target video data packet and other video data packets after the target video data packet are continuously sent to the receiving end according to the preset sending timing.
2. The method according to claim 1, It is characterized in that The step of obtaining the video packet extended sequence number of the target video key frame includes: Reading sending count information, where the sending count information is used to record the number of video data packets sent by the sending end to the receiving end; According to the sending count information, a video packet extension sequence number corresponding to the target video key frame is obtained.
3. The method according to claim 2, It is characterized in that The acquiring, according to the sending count information, a video packet extension sequence number corresponding to the target video key frame comprises: Searching the first video packet extension sequence number greater than the sending count information in the fault-tolerant jump table, and using the found video packet extension sequence number as the video packet extension sequence number corresponding to the target video key frame; The fault-tolerant jump table includes the multiple video key frames, and the video key frames carry the video packet sequence number, the video packet extension sequence number, and the video packet position index.
4. A video conference processing method, It is characterized in that The method comprises: receiving a plurality of video data packets contained in a video frame sent by a transmitter according to a preset sending timing; A decoding failure message is sent to the sending end, wherein the decoding failure message is used to instruct the sending end to search for a target video key frame located after a currently sent video frame in the preset sending timing and a target video data packet corresponding to the target video key frame; the preset sending timing is the corresponding order of each of the video data packets in the video frame; the fixed header and the extended header in the video data packet and the target video data packet are both reassigned data; wherein the process of searching for the target video data packet includes obtaining a video packet extended sequence number of the target video key frame, searching for a video packet position index corresponding to the video packet extended sequence number from a pre-acquired fault-tolerant jump table according to the video packet extended sequence number, and reading the target video data packet according to the video packet position index; the video packet extended sequence number is used to indicate that the target video key frame is located after the currently sent video frame in the preset sending timing. The target video key frame carries a video packet sequence number, a video packet extension sequence number and a video packet position index; in the reassignment process, the X field in the fixed header of the video data packet before reassignment and the target video data packet before reassignment is set to 1, and the extended header is filled after the reassigned fixed header, and the Srctimestamp and Srcseq fields in the extended header are the original timestamp and seq of the fixed header of the video data packet before reassignment and the target video data packet before reassignment; the video data packet and the target video data packet are both encrypted data packets obtained after the sender turns on the encrypted video conferencing function; the X represents extension, the Srctimestamp and the timestamp both represent timestamps, and the Srcseq and the seq both represent sequence numbers; The receiving end starts from the target video data packet, sends the target video data packet and other video data packets after the target video data packet according to the preset sending timing, continues to combine the received target video data packet and the other video data packets to obtain a combined video frame, and decodes the combined video frame.
5. The method according to claim 4, It is characterized in that The method further comprises: Performing inverse processing on the received video data packet to obtain an inversely processed video data packet, wherein the inversely processed video data packet includes a timestamp and a synchronization source field before the video data packet is reassigned; The inversely processed video data packet is decoded.
6. A video conference processing system, It is characterized in that The system comprises: a transmitting end and a receiving end; wherein, The sending end is used to perform the steps of the method described in claims 1-3 above; The receiving end is used to execute the steps of the method described in claims 4-5 above.
7. A video conference processing device, It is characterized in that The device comprises: A data sending module, used for sending a plurality of video data packets contained in a video frame to a receiving end according to a preset sending timing; the preset sending timing is the corresponding order of each of the video data packets in the video frame; A data acquisition module, configured to, upon receiving the decoding failure information sent by the receiving end, acquire a target video key frame located after the currently sent video frame in the preset sending sequence, and acquire a video packet extension sequence number of the target video key frame, wherein the video packet extension sequence number is used to indicate the order of the target video key frame in the preset sending sequence, and the target video key frame carries a video packet sequence number, a video packet extension sequence number, and a video packet position index; According to the video packet extended sequence number, the video packet position index corresponding to the video packet extended sequence number is searched from the pre-acquired fault-tolerant jump table, and the target video data packet is read according to the video packet position index; the fixed header and the extended header in the video data packet and the target video data packet are both reassigned data; in the reassignment process, the X field in the fixed header in the video data packet before reassignment and the target video data packet before reassignment is set to 1, and the extended header is filled after the reassigned fixed header, and the Srctimestamp and Srcseq fields in the extended header are the original timestamp and seq of the fixed header in the video data packet before reassignment and the target video data packet before reassignment; the video data packet and the target video data packet are both encrypted data packets obtained after the sender turns on the encrypted video conferencing function; the X represents extension, the Srctimestamp and the timestamp both represent timestamps, and the Srcseq and the seq both represent sequence numbers; The data retransmission module is used to start from the target video data packet and continue to send the target video data packet and other video data packets after the target video data packet to the receiving end according to the preset sending timing.
8. A video conference processing device, It is characterized in that The device comprises: A data receiving module, used for receiving a plurality of video data packets contained in a video frame sent by a sending end according to a preset sending timing; A decoding information sending module, used for sending decoding failure information to the sending end, the decoding failure information is used to instruct the sending end to search for a target video key frame and a target video data packet corresponding to the target video key frame in the preset sending timing; the preset sending timing is the corresponding sequence of each of the video data packets in the video frame; the fixed header and the extended header in the video data packet and the target video data packet are both reassigned data; wherein, the process of searching for the target video data packet includes obtaining a video packet extension sequence number of the target video key frame, searching for a video packet position index corresponding to the video packet extension sequence number from a pre-acquired fault-tolerant jump table according to the video packet extension sequence number, and reading the target video data packet according to the video packet position index; the video packet extension sequence number is used to indicate that the target video key frame is in the The order in the preset sending timing is that the target video key frame carries a video packet sequence number, a video packet extension sequence number and a video packet position index; in the process of reassignment, the X field in the fixed header of the video data packet before reassignment and the target video data packet before reassignment is set to 1, and the extended header is filled after the reassigned fixed header, and the Srctimestamp and Srcseq fields in the extended header are the original timestamp and seq of the fixed header in the video data packet before reassignment and the target video data packet before reassignment; the video data packet and the target video data packet are both encrypted data packets obtained after the sender turns on the encrypted video conferencing function; the X represents extension, the Srctimestamp and the timestamp both represent timestamps, and the Srcseq and the seq both represent sequence numbers; The data re-receiving module is used to receive the target video data packet and other video data packets after the target video data packet sent by the sending end starting from the target video data packet according to the preset sending timing, and continue to combine the received target video data packet and the other video data packets to obtain a combined video frame, and decode the combined video frame.
9. The device according to claim 8, It is characterized in that The device also includes: An inverse processing module, used for inversely processing the received video data packet to obtain an inversely processed video data packet, wherein the inversely processed video data packet includes a timestamp and a synchronization source field before the video data packet is reassigned; A decoding module is used to decode the video data packet after inverse processing.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.
