Data processing method and device, equipment, chip and storage medium

Abstract

The application discloses a data processing method, device, equipment, chip and storage medium, and belongs to the technical field of audio and video. The method comprises the following steps: acquiring a video data stream from an application layer; performing a packetizing operation on the video data stream with a coding frame in the video data stream as a packetizing granularity to obtain a plurality of video data packets; and sending the plurality of video data packets to a receiving end, wherein each video data packet in the plurality of video data packets contains at least one coding frame. For a large video data stream, each coding frame is usually greater than MTU, and each RTP packet obtained by performing the packetizing operation on the video data stream according to the RTP protocol in the related art is not greater than MTU. Therefore, compared with the RTP protocol in the related art, the application finally divides fewer video data packets, and the transmission efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio and video technology, and in particular to a data processing method and device, equipment, chip and storage medium. BACKGROUND

[0002] With the development of technology, it is a common scenario that a sending device sends real-time data to a receiving device. The related technology provides an RTP (Real-time Transport Protocol) protocol, which is used to provide end-to-end real-time transport services for multimedia data such as voice, image, and fax that need real-time transmission.

[0003] In the related technology, the RTP protocol is a transmission protocol for a network card, and the MTU (Maximum Transmission Unit) of the network card transmission is usually 1500 bytes. The RTP protocol in the transmission layer of the sending device will perform a packeting operation on the received real-time data based on the size of the MTU.

[0004] In the related technology, the RTP packet divided by the RTP protocol cannot be larger than the MTU. For a large video data stream, the RTP protocol will divide a large number of RTP packets, and the transmission rate is slow. SUMMARY

[0005] The present application provides a data processing method, device, equipment, chip and storage medium, which improves the transmission efficiency. The technical solution includes the following contents.

[0006] According to one aspect of the present application, a data processing method is provided, which is executed by a sending end, and the method comprises:

[0007] Obtaining a video data stream from an application layer;

[0008] Performing a packeting operation on the video data stream with a coding frame in the video data stream as a packeting granularity to obtain a plurality of video data packets;

[0009] Sending the plurality of video data packets to a receiving end;

[0010] Among them, each video data packet in the plurality of video data packets contains at least one coding frame.

[0011] According to one aspect of the present application, a data processing method is provided, which is executed by a receiving end, and the method comprises:

[0012] receive a plurality of video data packets corresponding to the video data stream, the plurality of video data packets being obtained by the sending end performing a packetizing operation on the video data stream with a packetizing granularity of encoded frames, each of the plurality of video data packets containing at least one encoded frame;

[0013] extract payload data in the plurality of video data packets.

[0014] According to an aspect of the present application, a data processing apparatus is provided, the apparatus operating at a sending end, the apparatus comprising:

[0015] an obtaining module configured to obtain a video data stream from an application layer;

[0016] a packetizing module configured to perform a packetizing operation on the video data stream with a packetizing granularity of encoded frames in the video data stream to obtain a plurality of video data packets;

[0017] a sending module configured to send the plurality of video data packets to a receiving end;

[0018] wherein each of the plurality of video data packets contains at least one encoded frame.

[0019] According to another aspect of the present application, a data processing apparatus is provided, the apparatus operating at a receiving end, the apparatus comprising:

[0020] a receiving module configured to receive a plurality of video data packets corresponding to the video data stream, the plurality of video data packets being obtained by the sending end performing a packetizing operation on the video data stream with a packetizing granularity of encoded frames, each of the plurality of video data packets containing at least one encoded frame;

[0021] an extracting module configured to extract payload data in the plurality of video data packets.

[0022] According to another aspect of the present application, a communication device is provided, the communication device comprising a memory and a processor, the memory being configured to store a computer program, and the processor being configured to load and execute the computer program to implement the data processing method as above.

[0023] According to another aspect of the present application, a computer readable storage medium is provided, the storage medium storing a computer program, the computer program being loaded and executed by a processor to implement the data processing method as above.

[0024] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of a communication device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the communication device to perform the data processing method as above.

[0025] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0026] In the packet splitting mode of the related art using the RTP protocol, each RTP packet obtained by splitting does not exceed the MTU size, and thus, for a large video data stream, a large number of RTP packets will be split. The present application performs packet splitting operation in the granularity of encoded frames to obtain a plurality of video data packets. For a large video data stream, each encoded frame is usually larger than the MTU, which makes the final video data packets to be split fewer, and the number of times of checking for the fewer video data packets is smaller, thereby facilitating the improvement of transmission efficiency, that is, the present application provides a packet splitting mode suitable for a large video data stream.

[0027] Furthermore, the present application takes the encoded frame as the packet splitting granularity. When the receiving end cannot restore the video image of a certain encoded frame, the receiving end can feed back to the sending end, and the sending end retransmits the data packet containing the encoded frame. At this time, packet splitting in the granularity of encoded frames facilitates retransmission in the granularity of encoded frames when an error occurs. For an encoded frame I frame, P frame, P frame, P frame, …, the decoding of a plurality of P frames depends on the same I frame. In the present application, if one of the P frames is in error, the P frame only needs to be retransmitted, and the decoding of other P frames can be normally performed. In the related art, packet splitting is not performed in the granularity of encoded frames. If one data packet is in error, the content of the I frame may be retransmitted additionally (the P frame data is actually in error), and at this time, the decoding of other P frames cannot be normally performed. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical scheme in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0029] Figure 1 is a flowchart of a data processing method of a sending end provided by an embodiment of the present application.

[0030] Figure 2 is a schematic diagram of a packet splitting mode of a video data stream provided by an embodiment of the present application.

[0031] Figure 3 is a flowchart of a data processing method of a sending end provided by an embodiment of the present application.

[0032] Figure 4 is a schematic diagram of a packet splitting mode of an audio data stream provided by an embodiment of the present application.

[0033] Figure 5is a schematic diagram of a packeting manner of first streaming media data provided by an embodiment of the present application.

[0034] Figure 6 is a schematic diagram of interaction between a sending end and a receiving end provided by an embodiment of the present application.

[0035] Figure 7 is a schematic diagram of interaction between a sending end and a receiving end provided by an embodiment of the present application.

[0036] Figure 8 is a flow chart of a data processing method of a receiving end provided by an embodiment of the present application.

[0037] Figure 9 is a schematic diagram of interaction between a sending end and a receiving end provided by an embodiment of the present application.

[0038] Figure 10 is a schematic diagram of distributed transmission provided by an embodiment of the present application.

[0039] Figure 11 is a schematic diagram of data scatter phenomenon provided by an embodiment of the present application.

[0040] Figure 12 is a structural block diagram of a data processing apparatus provided by an embodiment of the present application.

[0041] Figure 13 is a structural block diagram of a data processing apparatus provided by an embodiment of the present application.

[0042] Figure 14 is a structural block diagram of a data processing apparatus provided by an embodiment of the present application.

[0043] Figure 15 is a structural block diagram of a communication device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0045] First, the terms involved in the embodiments of the present application are briefly introduced.

[0046] RTP (Real-time Transport Protocol) protocol: used to provide end-to-end real-time transmission service for voice, image, facsimile and other multimedia data requiring real-time transmission.

[0047] In a typical application, the RTP protocol and the UDP (User Datagram Protocol) protocol both complete the protocol transmission of the transport layer, and the RTP protocol is created on the UDP protocol. The RTP protocol is used to provide time stamp and other technologies to ensure the time correctness of the streaming media in real-time transmission, and the UDP protocol is responsible for transmitting data packets.

[0048] The packet mode of the RTP protocol includes the following three modes: single packet mode, combined packet mode and fragmentation packet mode.

[0049] Single packet mode: for the received NALU (Network Abstract Layer Unit) unit of the upper layer transmission, if the length of the NALU unit is less than MTU, the NALU is encapsulated into a data packet.

[0050] Combined packet mode: if the received NALU unit of the upper layer transmission is particularly small, multiple NALU units are encapsulated into a data packet, and the encapsulated data packet is not greater than MTU.

[0051] Fragmentation packet mode: if the received NALU unit of the upper layer transmission exceeds MTU, the NALU unit is split and encapsulated into multiple data packets.

[0052] It can be seen that the RTP protocol is based on MTU for packeting, and the size of the split RTP packet will not exceed MTU. For large real-time data, if the RTP protocol is used for packeting, more data packets will be split, which leads to the network layer data packet being abnormally large, and also leads to the sending end and the receiving end processing the packet and the packet more frequently. The transmission efficiency of large real-time data is low. When the frequency of the CPU (Central Processing Unit) is not high, the phenomena of lag, packet loss and screen tearing often occur.

[0053] Application layer: the application layer is the highest layer in the network structure, and the application layer directly faces users and application programs, and provides various network services and application functions. The application layer will format and analyze the received streaming media data, including data encoding, decoding, parsing and presentation operations.

[0054] Transport layer: in the network structure, it is the next layer of the application layer, and is used to provide logical communication between processes. After receiving the real-time data sent by the application layer, the transport layer will perform packeting operation on the real-time data.

[0055] Figure 1A flow chart of a data processing method provided by an example embodiment of the present application is shown, which is performed by a sending terminal device, and the method comprises the following steps:

[0056] In step 120, a video stream is acquired from an application layer.

[0057] The video stream refers to streaming media data of a multimedia type of video. In the present application, the application layer of the sending terminal performs a series of operations such as encoding on original video data to obtain a video stream, and transmits the video stream to a transmission layer.

[0058] Optionally, the application layer encodes using H264 protocol or H265 protocol to obtain the video stream. The encoded video stream contains a plurality of encoded frames, and the plurality of encoded frames contain I frames and P frames (there is no B frame in real-time streaming media data), and if H264 protocol is used for encoding, each I frame contains SPS (Sequence Parameter Set, sequence parameter set) and PPS (Picture Parameter Set, picture parameter set) in front. If H265 protocol is used for encoding, each I frame contains SPS, PPS and VPS (Video Parameter Set, video parameter set) in front.

[0059] In step 140, the video stream is packetized with the encoded frames in the video stream as the packetization granularity to obtain a plurality of video data packets.

[0060] Each video data packet in the plurality of video data packets contains at least one encoded frame.

[0061] In an embodiment, the transmission layer of the sending terminal divides the video stream data transmitted by the application layer according to the encoded frames to obtain I frames, P frames, P frames, P frames, P frames, etc. Optionally, each encoded frame is divided into a data packet, i.e., each divided data packet contains only one encoded frame. Illustratively, an I frame is divided into a first data packet, and a P frame is divided into a second data packet. For large video stream data, the sizes of the I frame and the P frame are both large, both of which are larger than MTU (commonly 1500 bytes). Illustratively, reference is made to Figure 2 , Figure 2 It is shown that the video stream 201 contains key frames and a plurality of non-key frames. For a non-key frame (P frame) 202, the non-key frame 202 is divided into a payload field in a data packet 203, and a data packet 203 includes an interval field, a header field and a payload field.

[0062] In the related art, each RTP packet is divided by the RTP protocol, and the size of each RTP packet does not exceed the MTU. Therefore, for a large video data stream, a large number of RTP packets are divided. In the embodiment, the I frame is divided into a data packet, and the size of the I frame is much larger than the MTU, for example, the size of the I frame is 1M. For a large video data stream, the P frame is also large and larger than the MTU. Therefore, the application divides each I frame and each P frame into a data packet. For a large video data stream, a small number of data packets are finally divided, and the number of times of checking the small number of data packets is small, thereby improving the transmission efficiency. That is, the application provides a packet division method suitable for a large video data stream.

[0063] Optionally, for the case that the encoder in the application layer is close to the limit, the P frame of the large video data stream can also be small. Optionally, a plurality of P frames with a total size not exceeding a second threshold are divided into a data packet, and a P frame with a size larger than the second threshold is divided into a data packet. Optionally, the size of the second threshold is 10M, and the second threshold is much larger than the MTU (the common MTU is 1500 bytes).

[0064] Optionally, the plurality of P frames with a total size not exceeding the second threshold refers to a plurality of P frames in a critical state, that is, if one more P frame is added, the total size will exceed the second threshold. The second threshold is optionally 10M, 1M, etc.

[0065] Optionally, for the video stream data encoded by the H264 protocol, the SPS, PPS and I frame are divided into a data packet. Optionally, for the video stream data encoded by the H265 protocol, the SPS, PPS, VPS and I frame are divided into a data packet.

[0066] Step 160, sending a plurality of video data packets to the receiving end.

[0067] The sending end performs packet division on the video data stream to obtain a plurality of video data packets, and sends the plurality of video data packets to the receiving end. Optionally, the lower network layer of the transmission layer also processes the plurality of video data packets, and then the sending end sends the processed data packets to the receiving end. Optionally, the transmission layer of the receiving end performs packet disassembly and packet assembly on the plurality of video data packets.

[0068] Figure 3 A flowchart of a data processing method provided by an example embodiment of the application is shown, and the method is executed by a sending end device. The method comprises:

[0069] Step 320, obtaining first stream media data from the application layer;

[0070] The streaming media refers to a media format played on the Internet in a streaming manner. The first streaming media data can be a video data stream or an audio data stream. In the present application, the application layer of the sending end performs a series of operations such as encoding to obtain the first streaming media data, and sends the first streaming media data to the transport layer.

[0071] Optionally, the application layer encodes by using an H264 protocol or an H265 protocol to obtain the first streaming media data, and the first streaming media data is a video data stream.

[0072] Optionally, the application layer encodes by using an AAC (Advanced Audio Coding) to obtain the first streaming media data, and the first streaming media data is an audio data stream. Each frame in the encoded audio data stream is an AAC structure, and the encoded audio data stream has a plurality of AAC structures, and each AAC structure includes an ADTS header field and an audio basic code stream.

[0073] In step 340, a first multimedia type corresponding to the first streaming media data is determined.

[0074] The transport layer of the sending end device determines the multimedia type of the received first streaming media data to obtain the first multimedia type. The first multimedia type can be a video or an audio.

[0075] In step 360, a first packeting manner corresponding to the first multimedia type is used to perform packeting operation on the first streaming media data to obtain a plurality of data packets.

[0076] For different types of first streaming media data, the transport layer will use different packeting manners for packeting.

[0077] For the case that the first multimedia type is a video and the first streaming media data is a video data stream, packeting operation is performed on the video data stream to obtain a plurality of video data packets. The above has been introduced, and thus will not be repeated here.

[0078] For the case that the first multimedia type is an audio and the first streaming media data is an audio data stream, the transport layer takes the AAC structure in the audio data stream as the packeting granularity, performs packeting operation on the audio data stream to obtain a plurality of audio data packets, and each of the plurality of audio data packets contains at least one AAC structure.

[0079] The audio data stream transmitted in the application layer has multiple AAC structures, each of which includes an ADTS header field and an audio elementary stream. In general, the AAC structures are small, and in order to meet the real-time requirement of the audio (the user is more sensitive to the real-time of the audio), multiple AAC structures with a total size not exceeding a first threshold are divided into a third data packet, and an AAC structure with a single size greater than the first threshold is divided into a fourth data packet. Optionally, the first threshold is MTU, and the MTU is 1500 bytes.

[0080] Optionally, the multiple AAC structures with a total size not exceeding the first threshold refer to multiple AAC structures with a total size in a critical state, i.e., if one more AAC structure is added, the total size will exceed the first threshold. The first threshold is optionally MTU, and the MTU is optionally 1500 bytes.

[0081] In the embodiment of the present application, an AAC structure greater than MTU is allowed to be divided into a fourth data packet alone, and in the related art RTP protocol, the packetized RTP packet cannot be greater than MTU. Therefore, for the audio data stream, the packetizing manner provided in the present application makes the capacity of each data packet larger, and ultimately fewer data packets are divided, and the number of times of checking for the fewer data packets is less, thereby facilitating the improvement of transmission efficiency.

[0082] For illustration, reference is made to Figure 4 , Figure 4 An audio data stream 401 is shown, which includes multiple AAC structures 402 Figure 4 only one AAC structure is shown in the figure, Figure 4 The AAC structure 402 is shown divided into a payload field in a data packet 403, an AAC structure includes an ADTS header structure and an audio elementary stream, and a data packet 403 includes an interval field, a header field, and a payload field.

[0083] In one embodiment, multiple poor-quality AAC structures are filtered out from the audio data stream by a neural network model, and the multiple poor-quality AAC structures are divided into the same audio data packet. The poor-quality AAC structure refers to an AAC structure that is easy to be lost in the transmission process.

[0084] In one training round of the neural network model, multiple sample AAC structures from the same audio are obtained, the multiple sample AAC structures are input into the neural network model, the neural network model predicts the sample AAC structures with poor quality, and transmission tests are performed on the multiple sample AAC structures, and the sample AAC structure with the most loss times in the transmission test is determined as the real sample AAC structure with poor quality.

[0085] According to the gap between the real poor-quality sample AAC structure and the predicted poor-quality sample AAC structure, the model parameters of the neural network model are adjusted. For example, the real poor-quality sample AAC structure includes sample AAC structures 1, 2, and 5, and the predicted poor-quality sample AAC structure includes sample AAC structures 1, 2, and 3. The model parameters of the neural network model are optimized by a gradient descent method.

[0086] A plurality of training rounds are performed until the model parameters of the neural network model converge.

[0087] The above has introduced the segmentation method of the first stream media data. The first stream media data can be a video data stream or an audio data stream.

[0088] Next, the encapsulation method of the data packet will be introduced.

[0089] Figure 5 The transmission layer of the sending end device is shown in the schematic diagram of the encapsulation of the segmented first stream media data. After the transmission layer of the sending end device obtains the original data (first stream media data) 501, the original data 501 is cut into a plurality of small block data 502, which is the data obtained by segmenting the first stream media data as described above. The transmission layer encapsulates each small block data 502 to obtain a plurality of data packets 503.

[0090] Any one of the plurality of data packets 503 includes a gap field, a header field, and a payload field.

[0091] The gap field (Startcode) is used to separate the current data packet in the plurality of data packets. The gap field can be configured as any data string. For example, the gap field is 0x01 0x02 0x03 0x04, which is 4 bytes and 32 bits, indicating the start position of a data packet.

[0092] The header field (Header) is used to describe the multimedia type corresponding to the payload data of the current data packet, and is used to describe the length of the payload field. The payload data is the data (i.e. small block data) segmented from the first stream media data. The header field includes a type field and a length field. The type field includes 1 byte, which is used to identify whether the current data packet is an audio data packet, a video data packet, or other data packets. Optionally, the character type of the type field is Char (character type). The length field includes 4 bytes, which is used to identify the length of the payload field. Optionally, the character type of the length field is Unsigned Int (unsigned integer type).

[0093] Payload, used to carry payload data, and used to describe the offset position of the current data packet in multiple data packets. Illustratively, the first eight bits of the payload field are the offset field, used to describe the position of the current data packet in multiple data packets. The payload field is used to carry the payload data after the shell, i.e. to prevent the payload data from having the same data string as the interval field, a special character is inserted in the payload data containing the same data string as the interval field, for example, the original payload data has "0x01 0x02 0x03 0x04", which is the same as the data string of the interval field, after the shell, it will be changed to "0x01 0x02 0x00 0x03 0x04". The length field in the header field is used to describe the actual length of the payload data after the shell. Optionally, to prevent data tampering, the payload field is also encrypted as a whole, or the payload field is added with a sha1sum hash value after negotiation with the receiving end.

[0094] In the packaging manner provided in the present application, the data packet carries description information (interval field, type, length and offset position), when transmission error occurs, the description information can be used for repair, improving transmission reliability, reducing the use of ACK to ensure transmission reliability, thereby improving transmission rate and reducing transmission delay.

[0095] Regarding the sending process:

[0096] Figure 6 A schematic diagram of the sending process of multiple data packets provided by an exemplary embodiment of the present application is shown. The sending end 601 will send a status request 603 (Request Status) to the receiving end 602 at the initial time, i.e. an inquiry message, the inquiry message is used to inquire the status of the receiving end 602. The receiving end 602 will return a status response 604 to the sending end 601.

[0097] In the case where the inquiry message carries the SSRC (Synchronization Source) parameter of the first stream media data, the receiving end 602 determines the latest offset position from the local offset record, and carries the latest offset position in the status response 604 returned to the sending end 601, the latest offset position is the offset position corresponding to the data packet of the first stream media data recently received by the receiving end 602.

[0098] In the case where the inquiry message does not carry the SSRC parameter, the receiving end 602 generates the SSRC parameter corresponding to the first stream media data, and carries the SSRC parameter in the status response 604 returned to the sending end 601.

[0099] After the sending end 601 receives the latest offset position, if the data packet corresponding to the latest offset position exists in the local cache 605, the sending end 601 sends the data packet 606 corresponding to the next offset position of the latest offset position to the receiving end 602. After the receiving end 602 receives the data packet 606, the receiving end 602 records the offset position of the data packet 606 in the local offset record.

[0100] After the sending end 601 receives the latest offset position, if the data packet corresponding to the latest offset position does not exist in the local cache 605, the sending end 601 prompts the service layer to supplement the data packet between the latest offset position and the current offset position of the local cache 605, and sends the data packet 606 corresponding to the next offset position of the latest offset position to the receiving end 602. After the receiving end 602 receives the data packet 606, the receiving end 602 records the offset position of the data packet 606 in the local offset record.

[0101] The local cache 605 can delete the data packet corresponding to the latest offset position due to timeout or too large data packet. If the data packet corresponding to the latest offset position is deleted, the application layer of the sending end 601 calls the callback function to prompt the service layer to supplement the data packet between the latest offset position and the current offset position of the local cache 605.

[0102] The sending end 601 continuously obtains the next data packet 607 and the data packet 608 from the local cache 605 and sends them to the receiving end 602. After the receiving end 602 receives the data packet 607 and the data packet 608, the receiving end 602 records the offset positions of the data packet 607 and the data packet 608 in the local offset record.

[0103] In summary, in the sending process provided in the present application, by aligning the offset positions of the data packets of the sending and receiving ends, the sending end can ensure that the data packets sent later are correct, the sending end can continuously send the next data packet, and the receiving end can continuously receive the data packets, so that the receiving end does not need to feed back an ACK to the sending end after receiving each data packet. The present application can reduce the use of ACK, thereby improving the transmission rate and reducing the transmission delay.

[0104] Regarding transmission errors:

[0105] After the data packets are obtained by the method of the present application, the data packets are transmitted through the TCP network layer. The errors of the TCP network layer include blocking. Therefore, in the present application, the receiving end also reserves a cache space for caching backup data to facilitate resuming transmission.

[0106] For the first stream media data of the type of video, the receiving end caches a first number of GOPs (Group of Pictures), and the first number is a preset value, i.e., the cache space of the receiving end has a size of the first number of GOPs. For the first stream media data of the type of audio, the receiving end caches audio data corresponding to the first number of GOPs, i.e., the cache space of the receiving end has a size of the audio data corresponding to the first number of GOPs. Optionally, the first number is three, and two GOPs are separated by an I frame.

[0107] Figure 7 A retransmission flow of a transmission error is shown. The sending end 701 finds a next data packet 704 to be sent from the local cache 703, and sends the data packet 704 to the receiving end 702.

[0108] The receiving end 702 receives and parses the data packet 704, and if the receiving end 702 cannot parse a correct offset position from the data packet 704 (i.e., the offset position does not match), the receiving end 702 sends a Rebase message (carrying a correct offset position) 705 to the sending end 701. After receiving the correct offset position, the sending end 701 addresses and re-sends in the local cache 703, and sends a data packet corresponding to the correct offset position to the receiving end 702. The receiving end 702 receives and parses the data packet 706, confirms that the offset position of the data packet 706 is correct (i.e., the offset position of the data packet matches), and records the offset position of the data packet 706 in the local offset record.

[0109] In an embodiment, the receiving end 702 does not store an error data packet (i.e., a data packet with a mismatched offset position) in the cache space, so the receiving end 702 can tolerate a certain number of error data packets.

[0110] In summary, for the case of a transmission error, the receiving end sends the offset position of a correct data packet to the sending end, and the sending end sends a correct data packet. In this application, the sending and receiving ends can correct a transmission error through the offset position of a data packet, the receiving end does not need to feed back an ACK message to the sending end for each received data packet, and thus the use of ACK can be reduced, the transmission rate can be improved, and the transmission delay can be reduced.

[0111] Next, the content of the receiving end side will be introduced.

[0112] Figure 8 A flowchart of a data processing method is shown, which is performed by a receiving end device, and the method includes:

[0113] Step 810, continuously reading a plurality of data packets;

[0114] The plurality of data packets are obtained by performing a packetizing operation on the first stream media data according to a first multimedia type corresponding to the first stream media data. Optionally, the first stream media data is a video data stream, and the plurality of data packets are a plurality of video data packets. Optionally, the first stream media data is an audio data stream, and the plurality of data packets are a plurality of audio data packets. Any one of the plurality of data packets includes an interval field, a header field, and a payload field. For details of the plurality of data packets, refer to the content of the sending end side described above.

[0115] The transport layer of the receiving end device continuously reads data from the first socket interface. After reading the data, the receiving end device merges the data into a local cache space. In an embodiment, for the first stream media data of the video type, the receiving end caches a first number of GOPs (Group of Pictures), and the first number is a preset value, i.e., the size of the cache space is the size of the first number of GOPs. For the first stream media data of the audio type, the receiving end caches audio data corresponding to the first number of GOPs. That is, the size of the cache space is the size of the audio data corresponding to the first number of GOPs. Optionally, the first number is three, and the two GOPs are separated by an I frame.

[0116] The receiving end searches for the interval field in the local cache space. The interval field is a start flag of a data packet. For example, the interval field is configured as 0x01 0x02 0x03 0x04, and the receiving end continuously searches for the data string of 0x01 0x02 0x03 0x04.

[0117] Step 820, in the case of reading the interval field, continue to read the header field;

[0118] In the case of reading the interval field, the receiving end continues to read the header field. The length of the header field is fixed, and is optionally 5 bytes. Optionally, the header field includes a type field (1 byte) and a length field (4 bytes). If the header field cannot be completely read, the receiving end continues to read the first socket interface until the complete header field is read.

[0119] Step 830, continue to read the payload field according to the length of the payload field recorded in the header field;

[0120] The receiving end continues to read the payload field according to the length of the payload field recorded in the header field.

[0121] Step 840, in the case of matching the offset position recorded in the payload field, extract the payload data carried in the payload field;

[0122] The load field includes an offset field and load data, the length of the offset field is fixed, and is optionally 8 bits. The receiving end reads the offset position recorded in the offset field, and in the case of matching the offset position, extracts the load data carried in the load field. The load data is part of the first stream media data, and the offset position is used to describe the position of the current data packet in multiple data packets.

[0123] In one embodiment, in the case of matching the offset position, the receiving end also needs to perform a de-casing operation on the cased load data. The de-casing operation is opposite to the casing operation, that is, to find a target data string and remove the special character inserted in advance from the target data string. The target data string is a data string in which a special character is inserted in the data string in the interval field. For example, the target data string is 0x01 0x02 0x00 0x03 0x04, and the special character inserted by the sending end is 0x00. Therefore, after the de-casing operation, 0x01 0x02 0x03 0x04 is obtained.

[0124] Step 850, in the case that the offset position recorded in the load field does not match, a rebase message is sent to the sending end, and the rebase message is used to instruct the sending end to continue transmitting the data packet.

[0125] The receiving end reads the offset position recorded in the offset field, and in the case that the offset position does not match, a rebase message (Rebase message) is sent to the sending end, and the rebase message is used to instruct the sending end to continue transmitting the data packet. Optionally, the rebase message carries a correct offset position, and the sending end continues transmitting the data packet corresponding to the correct offset position according to the indicated correct offset position.

[0126] In summary, the receiving end device will continue to read the head field according to the received interval field, and continue to read the load field according to the length of the load field indicated in the head field, that is, the application provides a way of reading a data packet. The packet structure of the data packet is the packet structure adopted by the peer device (sending end device) when encapsulating.

[0127] Moreover, in the application, the receiving end device sends a rebase message to the sending end device only in the case that the offset position does not match, rather than feeding back an ACK message for each received data packet in the related art. The way of repairing errors (offset position does not match) provided by the application can effectively reduce the use of ACK, and thus can reduce the transmission delay, and the transmission reliability is ensured by the way of reading the data packet provided by the application, rather than by feeding back an ACK message for each received data packet to ensure the reliability.

[0128] In one embodiment, if the receiving end reads the current data packet from the two interval fields and finds that there is useless data and / or empty packet data, the receiving end sends a PING message to the sending end using the PING-PONG mechanism. If the receiving end receives the PONG message returned by the sending end, the receiving end confirms that the current data packet is legal and continues to read data. If the receiving end does not receive the PONG message sent by the sending end, the receiving end disconnects the connection with the sending end to protect the system.

[0129] The PING-PONG mechanism is shown in the schematic diagram. Figure 9 The PING-PONG mechanism is shown in the schematic diagram.

[0130] The sending end 901 reads the next data packet 904 from the local cache 903 and sends the data packet 904 to the receiving end 902. The receiving end 902 reads the data packet 904 and finds that there is abnormal data (useless data and / or empty packet data) in the data packet 904. The receiving end 902 sends a PING message 905 to the sending end 901, and the sending end 901 returns a PONG message 906 to the receiving end within a specified time. According to the received PONG message 906, the receiving end 902 confirms that the data packet 904 is legal and continues to read data.

[0131] In this application, there is a PING-PONG sniffing mechanism between the sending end device and the receiving end device. The PING-PONG sniffing mechanism is used to ensure the reliability of transmission, rather than using an ACK message to ensure the reliability of transmission, which can effectively reduce the use of ACK and further reduce the transmission delay.

[0132] In one embodiment, a plurality of data packets are sent in a distributed sending manner, that is, a plurality of sending ends distribute the sending of a plurality of data packets of the first stream media data to the receiving end. Figure 10 As shown in the figure, the plurality of sending ends include a master sending end 1001 and at least one slave sending end 1002.

[0133] In the process of distributed sending, initially, the master sending end 1001 sends a state request 1004, that is, an inquiry message, to the receiving end 1003 to inquire about the state of the receiving end 1003. The receiving end 1003 returns a state response 1005 to the master sending end 1001. Optionally, the state request 1004 includes distributed prompt information for prompting the receiving end 1003 that this sending is a distributed sending task. The receiving end 1003 will open a cache space to temporarily store each data packet sent by the subsequent sending end in the cache space. Optionally, the size of the cache space is (first number*GOP*number of sending ends). Optionally, the first number is three. Figure 10In this case, only two sending ends are provided, and the size of the cache space is (3*GOP*2).

[0134] After receiving the status response 1005, the master sending end 1001 coordinates the slave sending end 1002 to perform parallel data sending (distributed data sending). The master sending end 1001 sends a master synchronization task 1009 to the slave sending end 1002, instructing the slave sending end 1002 to send a data packet 1010 in a plurality of data packets, and the master sending end 1001 itself also finds a data packet 1007 to be sent to the receiving end 1003 from the local cache 1006 and sends the data packet 1007. After receiving the master synchronization task 1009, the slave sending end 1002 sends the data packet 1010 to the receiving end 1003 according to the instruction of the master synchronization task 1009, and returns a slave synchronization task 1011 to the master sending end 1001, informing the master sending end 1001 that the corresponding data packet has been sent, and waiting for the next master synchronization task 1012.

[0135] Since it is asynchronous sending, the offset positions of the data packets sent by different sending ends are different, and there may be a data scatter phenomenon. The data scatter phenomenon refers to the phenomenon that the received data packets are discontinuous.

[0136] In the process of receiving a plurality of data packets, if the data scatter phenomenon persists in the received data packets, the receiving end 1003 sends a padding data message to the master sending end 1001. The padding data message is used for the master sending end 1001 to send or instruct at least one slave sending end 1002 to send the missing data packet in the data scatter phenomenon.

[0137] The schematic diagram is shown in FIG. 1. Figure 11 The change process of the cache space of the receiving end is shown. Figure 11 Part (A) of FIG. 1 shows that in the initial state, the cache space of the receiving end stores a data packet with an offset position 0 and a length 1024, and the receiving end receives a new data packet with an offset position 2048 and a length 1024.

[0138] In the initial state, the receiving end has received the data packet with the offset position 2048 and the length 1024, but has not received the data packet with the offset position 1024 and the length 1024. Figure 11 In part (B) of FIG. 1, the receiving end stores the new data packet in the cache space, and removes the data packet with the offset position 0 and the length 1024. At this time, the receiving end finds that the data packet with the offset position 2048 and the length 1024 has been received, but the data packet with the offset position 1024 and the length 1024 has not been received. After a period of time, if the receiving end still does not receive the data packet with the offset position 1024 and the length 1024, the receiving end sends a padding data message to the master sending end to pad the data packet with the offset position 1024 and the length 1024. The master sending end sends or instructs the slave sending end to send the data packet with the offset position 1024 and the length 1024.

[0139] In one case, due to asynchronous processing, in the process of waiting for the missing data packets corresponding to the filling data message, the missing data packets that are originally to be sent can be received, and the receiving end device can also perform a de-duplication operation after receiving two missing data packets to remove one of the data packets.

[0140] In summary, the present application provides a distributed sending method, and the distributed sending (asynchronous sending) can improve the transmission efficiency of the multiple data packets corresponding to the first stream media data.

[0141] In one embodiment, in the case of a distributed task, the receiving end device continuously reads the data packets sent by the multiple sending ends from the first socket interface. The receiving end device merges the data into the local cache space after reading the data, and the receiving end device reads the data packets from the local cache space.

[0142] In one embodiment, in the case of a non-distributed task, the receiving end continuously reads the multiple data packets sent by the first sending end from the first socket interface, and binds the first socket interface and the first sending end, so that when other sending ends attempt to resume transmission of the multiple data packets (corresponding to the first stream media data), the first socket interface is blocked. When the first sending end completes the sending of the multiple data packets (corresponding to the first stream media data), the binding relationship between the first sending end and the first socket interface is unbound.

[0143] Figure 12 A data processing apparatus 1200 provided by one exemplary embodiment of the present application is shown, the data processing apparatus 1200 runs on a sending end, and the data processing apparatus 1200 includes:

[0144] An acquisition module 1201 is configured to acquire a video data stream from an application layer;

[0145] A packetizing module 1202 is configured to perform a packetizing operation on the video data stream by taking an encoded frame in the video data stream as a packetizing granularity, to obtain multiple video data packets;

[0146] A sending module 1203 is configured to send the multiple video data packets to a receiving end;

[0147] Each of the multiple data packets includes at least one encoded frame.

[0148] In one optional embodiment, the encoded frame includes an I frame or a P frame; the packetizing module 1202 is further configured to separately divide each I frame in the video data stream into a first data packet, and separately divide each P frame in the video data stream into a second data packet.

[0149] In an optional embodiment, the obtaining module 1201 is further configured to obtain the audio data stream from an application layer.

[0150] The packetizing module 1202 is further configured to perform a packetizing operation on the audio data stream with an advanced audio coding (AAC) structure in the audio data stream as a packetizing granularity, to obtain a plurality of audio data packets, wherein each of the plurality of audio data packets contains at least one AAC structure.

[0151] In an optional embodiment, the packetizing module 1202 is further configured to, for the AAC structure in the audio data stream, divide a plurality of AAC structures with a total size not exceeding a first threshold to one third data packet, and divide an AAC structure with a size greater than the first threshold to one fourth data packet.

[0152] In an optional embodiment, any one of the plurality of video data packets comprises an interval field, a header field, and a payload field.

[0153] The interval field is configured to separate a current video data packet from the plurality of video data packets.

[0154] The header field is configured to describe a multimedia type to which payload data of the current video data packet corresponds, and to describe a length of the payload field, the payload data being data obtained by performing the packetizing operation on the video data stream.

[0155] The payload field is configured to carry the payload data, and to describe an offset position of the current video data packet in the plurality of video data packets.

[0156] In an optional embodiment, the sending module 1203 is further configured to send an inquiry message to the receiving end.

[0157] The obtaining module 1201 is further configured to, in a case where the inquiry message carries a synchronization source (SSRC) parameter corresponding to the video data stream, obtain a latest offset position determined by the receiving end from the offset record, the latest offset position being an offset position corresponding to a video data packet most recently received by the receiving end.

[0158] The sending module 1203 is further configured to send, based on the latest offset position, a video data packet corresponding to a next offset position to the receiving end.

[0159] In an optional embodiment, the sending module 1203 is further configured to, in a case where the video data packet of the latest offset position exists in a local cache of the sending end, send, to the receiving end, a video data packet corresponding to a next offset position of the latest offset position.

[0160] Combined reference Figure 13In an optional embodiment, the apparatus further comprises a prompting module 1204 configured to prompt the service layer to supplement the video data packets between the latest offset position and the current offset position in the local cache of the sending end in the case that the video data packet of the latest offset position does not exist in the local cache; and the sending module 1203 is further configured to send the video data packet corresponding to the next offset position of the latest offset position to the receiving end.

[0161] In an optional embodiment, the sending module 1203 is further configured to send the inquiry message to the receiving end. The apparatus further comprises a receiving module 1205 configured to acquire the SSRC parameter corresponding to the first stream media data generated by the receiving end in the case that the inquiry message does not carry the SSR C parameter.

[0162] In an optional embodiment, the receiving module 1205 is further configured to receive the message carrying the correct offset position, which is sent by the receiving end in the case that the receiving end fails to parse the correct offset position from the error video data packet sent by the sending end.

[0163] The sending module 1203 is further configured to send the video data packet corresponding to the correct offset position to the receiving end.

[0164] In an optional embodiment, the receiving end caches a first number of groups of pictures (GOPs), and the first number is a preset value.

[0165] In summary, in the packetization manner of the related art using the RTP protocol, each RTP packet is divided to be no more than the MTU size, and thus, for a large video data stream, a large number of RTP packets will be divided. The present application performs packetization operation in the granularity of encoding frames to obtain a plurality of video data packets. For a large video data stream, each encoding frame is usually larger than the MTU, which makes the final video data packets to be divided to be less, and the video data packets to be verified less frequently, thereby facilitating the improvement of transmission efficiency, i.e., the present application provides a packetization manner suitable for large video data streams.

[0166] Furthermore, the present application takes the encoding frame as the packetization granularity, and when the receiving end fails to restore the video image of a certain encoding frame, the receiving end can feed back to the sending end to retransmit the data packet containing the encoding frame by the sending end. At this time, packetization in the granularity of encoding frames is beneficial to retransmission in the granularity of encoding frames when errors occur.

[0167] Figure 14 FIG. 1 is a structural block diagram of a data processing apparatus 1400 provided by an exemplary embodiment of the present application, which runs in a receiving end, and the apparatus comprises:

[0168] The receiving module 1401 is configured to receive a plurality of video data packets corresponding to a video data stream, the plurality of video data packets being obtained by performing packetization on the video data stream with an encoding frame as a packetization granularity, and each video data packet in the plurality of video data packets containing at least one encoding frame.

[0169] The extraction module 1402 is configured to extract payload data in the plurality of video data packets.

[0170] In an optional embodiment, any video data packet in the plurality of video data packets includes an interval field, a header field, and a payload field. The apparatus further includes a reading module 1403 configured to continuously read the plurality of video data packets.

[0171] In a case where the interval field is read, the header field is continuously read.

[0172] The payload field is continuously read according to a length of the payload field recorded in the header field.

[0173] The extraction module 1402 is further configured to extract payload data carried in the payload field in a case where an offset position recorded in the payload field matches, the payload data being part of the video data stream, and the offset position being used to describe a position of a current video data packet in the plurality of video data packets.

[0174] The apparatus further includes a sending module 1404 configured to send a base change message to the sending end in a case where the offset position recorded in the payload field does not match, the base change message being used to instruct the sending end to resume transmission of the video data packet.

[0175] In an optional embodiment, the sending module 1404 is further configured to send a PING message to the sending end in a case where there is useless data and / or empty packet data in the current video data packet read.

[0176] The data processing apparatus 1400 further includes an acknowledgement module 1405. The acknowledgement module 1405 is configured to acknowledge that the current video data packet is legal in a case where a PONG message sent by the sending end is received.

[0177] The processing module 1406 is configured to disconnect in a case where the PONG message sent by the sending end is not received.

[0178] In an optional embodiment, the receiving end supports a plurality of sending ends distributedly and jointly sending first stream media data, the plurality of sending ends including a master sending end and at least one slave sending end.

[0179] The sending module 1404 is further configured to send a padding data message to the master sending end if the received video data packets still have the data scatter phenomenon during the process of receiving the video data packets, and the padding data message is used to instruct the master sending end or at least one slave sending end to send the missing video data packets in the data scatter phenomenon.

[0180] In an optional embodiment, the data processing apparatus 1400 further comprises a binding module 1407. The reading module 1403 is further configured to continuously read the video data packets sent by the first sending end from the first socket interface in the case of non-distributed tasks, and the binding module 1407 is further configured to bind the first socket interface and the first sending end.

[0181] In summary, the receiving end device will continue to read the head field according to the received interval field, and continue to read the payload field according to the length of the payload field indicated in the head field, that is, the application provides a way of reading video data packets. The packet structure of the video data packet is the packet structure adopted by the peer device (sending end device) when encapsulating.

[0182] Moreover, in the application, the receiving end device sends the base change message to the sending end only in the case of mismatch of the offset position, rather than feeding back an ACK message for each received video data packet in the related art. The way of reading video data packets provided by the receiving end device in the application can effectively reduce the use of ACK, thereby reducing the transmission delay, and the way of reading video data packets provided by the application can ensure the reliability of transmission, rather than using the ACK message to ensure the reliability.

[0183] Figure 15 Fig. 1 shows a structural schematic diagram of a communication device provided by an exemplary embodiment of the application. The communication device can be a sending end device or a receiving end device, and the communication device comprises a processor 1501, a receiver 1502, a transmitter 1503, a memory 1504 and a bus 1505.

[0184] The processor 1501 comprises one or more than one processing core, and the processor 1501 performs various functional applications and information processing by running software programs and modules.

[0185] The receiver 1502 and the transmitter 1503 can be implemented as a communication component, which can be a communication chip.

[0186] The memory 1504 is connected to the processor 1501 through the bus 1505.

[0187] The memory 1504 can be used to store at least one instruction, and the processor 1501 is configured to execute the at least one instruction to implement the various steps in the above method embodiments.

[0188] In addition, the memory 1504 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, including but not limited to: a magnetic disk or an optical disk, an electrically erasable programmable read only memory (EEPROM, Electrically Erasable Programmable Read Only Memory), an erasable programmable read only memory (EPROM, Erasable Programmable Read Only Memory), a static random access memory (SRAM, Static Random-Access Memory), a read only memory (ROM, Read Only Memory), a magnetic storage, a flash memory, a programmable read only memory (PROM, Programmable Read Only Memory).

[0189] In an exemplary embodiment, a non-transitory computer readable storage medium including instructions, such as a memory including instructions, is also provided, which can be executed by a processor of a communication device to complete the above-mentioned determination method of coverage enhancement level. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM, Random-Access Memory), a compact disc read only memory (CD-ROM, Compact Disc Read Only Memory), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0190] A non-transitory computer readable storage medium, when instructions in the non-transitory computer storage medium are executed by a processor of a communication device, enables the communication device to perform the above-mentioned data processing method.

[0191] The present application also provides a communication device, which includes a processor and a memory, and the memory stores a computer program, which is loaded and executed by the processor to implement the data processing method provided by the above-mentioned method embodiments.

[0192] The present application also provides a computer readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, which is loaded and executed by a processor to implement the data processing method provided by the above-mentioned method embodiments.

[0193] The application provides a computer program product or a computer program, the computer program product or the computer program comprising computer instructions stored in a computer readable storage medium. A processor of a communication device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the communication device to perform the data processing method provided by the above method embodiments.

[0194] The above embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0195] Those skilled in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk. The above is only an optional embodiment of the application, and does not limit the application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A data processing method, characterized by, The method is executed by a sending end, and the method comprises: obtaining a video data stream from an application layer; for each I frame in the video data stream with a size greater than a maximum transmission unit (MTU), separately dividing one first data packet in a plurality of video data packets; for each P frame in the video data stream with a size greater than the MTU, separately dividing one second data packet in the plurality of video data packets; sending the plurality of video data packets to a receiving end.

2. The method of claim 1, wherein, The method further comprises: obtaining an audio data stream from the application layer; performing a packetizing operation on the audio data stream with an advanced audio coding (AAC) structure in the audio data stream as a packetizing granularity to obtain a plurality of audio data packets; wherein each audio data packet in the plurality of audio data packets comprises at least one AAC structure.

3. The method of claim 2, wherein, The performing of the packetizing operation on the audio data stream with the AAC structure in the audio data stream as the packetizing granularity to obtain the plurality of audio data packets comprises: for each AAC structure in the audio data stream, dividing a plurality of AAC structures with a total size not exceeding a first threshold into one third data packet, and separately dividing an AAC structure with a single size greater than the first threshold into one fourth data packet.

4. The method of claim 1, wherein, Any one video data packet in the plurality of video data packets comprises a gap field, a header field, and a payload field; the gap field is used to separate a current video data packet in the plurality of video data packets; the header field is used to describe a multimedia type to which payload data of the current video data packet corresponds, and is used to describe a length of the payload field, the payload data being data obtained by performing the packetizing operation on the video data stream; the payload field is used to carry the payload data, and is used to describe an offset position of the current video data packet in the plurality of video data packets.

5. The method of claim 1, wherein, The method further comprises: sending an inquiry message to the receiving end; in a case where the inquiry message carries a synchronization source (SSRC) parameter corresponding to the video data stream, obtaining a latest offset position determined by the receiving end from an offset record, the latest offset position being an offset position corresponding to a video data packet most recently received by the receiving end; based on the latest offset position, sending a video data packet corresponding to a next offset position to the receiving end.

6. The method of claim 5, wherein, The sending of the video data packet corresponding to the next offset position to the receiving end based on the latest offset position comprises: in a case where the video data packet of the latest offset position exists in a local cache of the sending end, sending the video data packet corresponding to the next offset position of the latest offset position to the receiving end; in a case where the video data packet of the latest offset position does not exist in the local cache of the sending end, prompting a service layer to supplement video data packets between the latest offset position and a current offset position of the local cache, and sending the video data packet corresponding to the next offset position of the latest offset position to the receiving end.

7. The method of claim 1, wherein, The method further comprises: sending an inquiry message to the receiving end; In the case that the query message does not carry the SSRC parameter, the SSRC parameter corresponding to the video data stream generated by the receiving end is acquired.

8. The method of claim 1, wherein, The method further comprises: receiving a message carrying a correct offset position, the message carrying the correct offset position being sent by the receiving end in the case that the receiving end cannot parse a correct offset position from the error video data packet sent by the sending end; sending a video data packet corresponding to the correct offset position to the receiving end.

9. A data processing method, characterized by, The method is executed by a receiving end, and the method comprises: receiving a plurality of video data packets corresponding to a video data stream, the plurality of video data packets being obtained by separately dividing, by a sending end, each I frame with a size greater than MTU in the video data stream into a first data packet in the plurality of video data packets, and each P frame with a size greater than MTU in the video data stream into a second data packet in the plurality of video data packets; extracting payload data in the plurality of video data packets.

10. The method of claim 9, wherein, Any one of the plurality of video data packets comprises an interval field, a header field, and a payload field; the method further comprises: continuously reading the plurality of video data packets corresponding to the video data stream; in the case that the interval field is read, continuously reading the header field; continuously reading the payload field according to the length of the payload field recorded in the header field; the extracting of the payload data in the plurality of video data packets comprises: in the case that an offset position recorded in the payload field matches, extracting payload data carried in the payload field, the payload data being part of data in the video data stream, and the offset position being used to describe the position of the current video data packet in the plurality of video data packets.

11. The method of claim 10, wherein, The method further comprises: in the case that the offset position recorded in the payload field does not match, sending a rebase message to the sending end, the rebase message being used to instruct the sending end to continue transmitting a video data packet.

12. The method of claim 10, wherein, The method further comprises: in the case that there is useless data and / or empty packet data in the current video data packet read, sending a PING message to the sending end; in the case that a PONG message sent by the sending end is received, confirming that the current video data packet is legal; in the case that the PONG message sent by the sending end is not received, disconnecting.

13. The method of claim 10, wherein, The receiving end supports multiple sending ends distributedly sending the video data stream together, the multiple sending ends comprising a master sending end and at least one slave sending end, and the method further comprises: in the process of receiving the plurality of video data packets, if the received video data packet continuously has a data scatter phenomenon, a padding data message is sent to the master sending end, the padding data message being used for the master sending end to send or instruct the at least one slave sending end to send a video data packet missing in the data scatter phenomenon.

14. The method of claim 10, wherein, The continuously reading of the plurality of video data packets comprises: In the case of non-distributed tasks, continuously reading the plurality of video data packets sent by the first sender from the first Socket interface, and binding the first Socket interface and the first sender.

15. A data processing apparatus, characterized by: The device runs at a sender, and the device comprises: an obtaining module, configured to obtain a video data stream from an application layer; a packetizing module, configured to individually divide each I frame with a size greater than a maximum transmission unit (MTU) in the video data stream into a first data packet in a plurality of video data packets, and individually divide each P frame with a size greater than the MTU in the video data stream into a second data packet in the plurality of video data packets; a sending module, configured to send the plurality of video data packets to a receiver.

16. A data processing apparatus, characterized by The device runs at a receiver, and the device comprises: a receiving module, configured to receive a plurality of video data packets corresponding to a video data stream, the plurality of video data packets being obtained by individually dividing each I frame with a size greater than a maximum transmission unit (MTU) in the video data stream into a first data packet in the plurality of video data packets, and individually dividing each P frame with a size greater than the MTU in the video data stream into a second data packet in the plurality of video data packets; an extracting module, configured to extract payload data in the plurality of video data packets.

17. A communication device, characterized by The communication device comprises a processor and a memory, the memory is configured to store a computer program, and the processor is configured to execute the computer program to implement the data processing method in any one of claims 1 to 14.

18. A chip, characterized by The chip comprises a processor, configured to read and execute a computer program from a memory to implement the data processing method in any one of claims 1 to 14.

19. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the data processing method in any one of claims 1 to 14.

Images