Data transmission method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a data transmission method and device, electronic equipment, a storage medium and a program product. The method comprises the steps that a first video code stream is acquired, and the first video code stream comprises at least one first video frame; sequentially writing to-be-transmitted user data into the application mark area of the at least one first video frame to obtain a second video frame; generating a second video code stream based on the second video frame; and sending the second video code stream. Through the technical scheme of the invention, a new transmission link is prevented from being established for user data, and the cost and complexity of a hardware circuit are reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of video coding, and in particular to a data transmission method and device, an electronic device, a storage medium, and a program product. Background Art

[0002] MJPEG (Motion Joint Photographic Experts Group), also known as "motion still image (or frame-by-frame) compression technology", is a type of video coding. MJPEG is a dynamic image compression technology developed based on the static image compression technology JPEG (Joint Photographic Experts Group).

[0003] Therefore, when used for simultaneous transmission of MJPEG video code stream and user data, how to reduce the cost and complexity of hardware circuits is an urgent problem to be solved. Summary of the invention

[0004] According to the present disclosure, a data transmission method and device, an electronic device, a storage medium and a program product are provided, which reduce the cost and complexity of the hardware circuit for simultaneous transmission of video code streams and user data.

[0005] In a first aspect, the present disclosure provides a data transmission method. The method includes: obtaining a first video code stream, wherein the first video code stream includes at least one first video frame; writing user data to be transmitted into an application mark area of ​​at least one first video frame in sequence to obtain a second video frame; generating a second video code stream based on the second video frame; and sending the second video code stream.

[0006] In some embodiments, the step of sequentially writing user data to be transmitted into the application mark area of ​​at least one first video frame to obtain a second video frame may include: writing user data into the application mark area of ​​the i-th first video frame; determining whether the user data is completely written into at least one first video frame; if the user data is not completely written, setting the value of i to i+1, and returning to the step of writing user data into the application mark area of ​​the i-th first video frame; if the user data is completely written, obtaining a second video frame; wherein i is a positive integer.

[0007] In some embodiments, the i-th first video frame includes n application mark areas; wherein the step of writing the user data into the application mark area of ​​the i-th first video frame may include: for the j-th application mark area in the i-th first video frame, obtaining the data length of the first part of the data that has not been written into the user data; determining whether the data length of the first part of the data is greater than the maximum user data amount of the j-th application mark area; when the data length of the first part of the data is greater than the maximum user data amount of the j-th application mark area, sequentially performing the following steps: writing the second part of the data in the first part of the data into the j-th application mark area, wherein the data length of the second part of the data is the maximum user data amount of the j-th application mark area; setting the value of j to j+1, and returning to the step of obtaining the data length of the first part of the data that has not been written into the user data; when the data length of the first part of the data is equal to or less than the maximum user data amount of the j-th application mark area, writing the first part of the data into the j-th application mark area; wherein j and n are both positive integers, 1<j≤n.

[0008] In some embodiments, the operation of writing the second part of data into the jth application mark area may also include: writing relevant information of the jth application mark area into the jth application mark area; wherein the relevant information includes at least one of the following: mark code, area data length, user data type, video frame rate, retention information, user data length.

[0009] In some embodiments, the data length of the first part of the data can be represented by the first data length; wherein, the operation of writing the user data to the application marker area of ​​the i-th first video frame can also include: when writing the second part of the data to the j-th application marker area, updating the first data length to the difference between the data length of the first part of the data and the data length of the second part of the data; when writing the first part of the data to the j-th application marker area, updating the first data length to 0.

[0010] In some embodiments, the operation of determining whether the user data is completely written into at least one first video frame may include: when the first data length is equal to 0, determining that the user data is completely written; when the first data length is greater than 0, determining that the user data is not completely written.

[0011] In some implementations, the user data includes at least one of the following: a timestamp, subtitle data, audio data, and a third video code stream.

[0012] In a second aspect, the present disclosure provides a data transmission device. The device includes: a code stream acquisition module, an encapsulation module, a code stream generation module, and a sending module. The code stream acquisition module is configured to acquire a first video code stream, wherein the first video code stream includes at least one first video frame. The code stream encapsulation module is configured to sequentially write user data to be transmitted into an application marker area of ​​at least one first video frame to obtain a second video frame. The code stream generation module is configured to generate a second video code stream based on the second video frame. The sending module is configured to send the second video code stream.

[0013] In some embodiments, the encapsulation module can be configured to: write user data to the application mark area of ​​the i-th first video frame; determine whether the user data is completely written into at least one first video frame; if the user data is not completely written, set the value of i to i+1, and return to the step of writing user data to the application mark area of ​​the i-th first video frame; if the user data is completely written, obtain a second video frame; wherein i is a positive integer.

[0014] In some embodiments, the i-th first video frame includes n application mark areas; wherein the encapsulation module can be configured to: for the j-th application mark area in the i-th first video frame, obtain the data length of the first part of the data that has not been written in the user data; determine whether the data length of the first part of the data is greater than the maximum user data amount of the j-th application mark area; when the data length of the first part of the data is greater than the maximum user data amount of the j-th application mark area, perform the following steps in sequence: write the second part of the data in the first part of the data to the j-th application mark area, wherein the data length of the second part of the data is the maximum user data amount of the j-th application mark area; set the value of j to j+1, and return to the step of obtaining the data length of the first part of the data that has not been written in the user data; when the data length of the first part of the data is equal to or less than the maximum user data amount of the j-th application mark area, write the first part of the data to the j-th application mark area; wherein j and n are both positive integers, 1<j≤n.

[0015] In some embodiments, the encapsulation module can be configured to: write relevant information of the jth application marking area into the jth application marking area; wherein the relevant information includes at least one of the following: marking code, area data length, user data type, video frame rate, reservation information, user data length.

[0016] In some embodiments, the data length of the first part of data can be represented by the first data length; wherein the encapsulation module can be configured to: when the second part of data is written to the j-th application marker area, the first data length is updated to the difference between the data length of the first part of data and the data length of the second part of data; when the first part of data is written to the j-th application marker area, the first data length is updated to 0.

[0017] In some embodiments, the encapsulation module may be configured to: when the first data length is equal to 0, determine that the user data is completely written; and when the first data length is greater than 0, determine that the user data is not completely written.

[0018] In some implementations, the user data includes at least one of the following: a timestamp, subtitle data, audio data, and a third video code stream.

[0019] In a third aspect, the present disclosure provides an electronic device. The electronic device includes a processor and a memory. The memory is connected to the processor and is used to store executable instructions. The processor is configured to execute the executable instructions in the memory to implement the data transmission method as described in any one of the first aspect and possible implementations thereof.

[0020] In a fourth aspect, the present disclosure provides a storage medium having executable instructions stored thereon. When the executable instructions are executed by a processor, the processor implements the data transmission method as described in any one of the first aspect and possible implementations thereof.

[0021] In a fifth aspect, the present disclosure provides a computer program product. The computer program product includes computer instructions or codes. When the computer instructions or codes are executed by a processor, the data transmission method as described in any one of the first aspect and possible implementations thereof is implemented.

[0022] According to the present disclosure, a first video code stream is obtained, wherein the first video code stream includes at least one first video frame; user data to be transmitted is sequentially written into the application mark area of ​​at least one first video frame to obtain a second video frame; based on the second video frame, a second video code stream is generated; and the second video code stream is sent. In this way, when it is necessary to transmit the video code stream and user data simultaneously, the user data is written into the application mark area of ​​the video frame of the video code stream to generate a new video code stream. The new video code stream uses the video coding of the original video code stream, and the new video code stream can be transmitted through the transmission link corresponding to the video code stream. In this way, the transmission of the video code stream and the user data can be realized simultaneously only through the link used for the video code stream, without the need to establish a new transmission link for the user data. The hardware circuits of the receiving end and the transmitting end in the electronic device only need to support the link of the video code stream, thereby reducing the design and manufacturing cost of the hardware circuit and reducing the complexity of the hardware circuit.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] Figure 1 It is a schematic diagram of the MJPEG video code stream in the related art.

[0026] Figure 2 It is a schematic diagram of a transmission scenario of an MJPEG video code stream in the related art.

[0027] Figure 3 It is a schematic diagram of a transmission scenario of an MJPEG video code stream provided by an embodiment of the present disclosure.

[0028] Figure 4 It is a flowchart of the data transmission method provided by an embodiment of the present disclosure.

[0029] Figure 5 It is a flowchart of step S420 in the data transmission method provided in an embodiment of the present disclosure.

[0030] Figure 6 It is a flowchart of step S510 in the data transmission method provided in an embodiment of the present disclosure.

[0031] Figure 7 It is a flowchart of an exemplary embodiment of the data transmission method provided by the embodiment of the present disclosure.

[0032] Figure 8It is a structural diagram of a data transmission device provided in an embodiment of the present disclosure.

[0033] Fig. 9 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig.10 A schematic diagram of the structure of a data transmission system provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0035] Video coding refers to the process of converting a video format file into another video format file through compression technology. Through video coding, information redundancy in the spatial dimension and / or time dimension can be removed, thereby achieving compression of video data.

[0036] MJPEG, also known as "motion still image (or frame-by-frame) compression technology", is a type of video coding. MJPEG is a dynamic image compression technology developed from JPEG.

[0037] Figure 1 FIG. 1 is a schematic diagram of an MJPEG video code stream in the related art. Figure 1 As shown in the figure, an MJPEG video stream consists of multiple JPEG images. The main feature of MJPEG is intra-frame compression. That is, in an MJPEG video stream, each frame is compressed completely and independently, and the changes between different frames in the MJPEG video stream are basically not considered.

[0038] JPEG images consist of an identification code and compressed data. A JPEG image can contain one or more identification codes. Each identification code is used to mark the corresponding field. Common identification codes in JPEG images can be found in Table 1.

[0039] Table 1

[0040] Marker code name Mark code content SOI 0xFFD8 APP 0xFFE0~0xFFEF DQ 0xFFDB SOF 0xFFC0 DHT 0xFFC4 DRI 0xFFDD SOS 0xFFDA EOI 0xFFD9

[0041] As shown in Table 1, the marker codes in a JPEG image may include: SOI (Start Of Image), APPn (APPlication marker n), DQT (Define Quantization Table), SOF (Start Of Frame), DHT (Define Huffman Table), DRI (Define Restart Interval), SOS (Start Of Scan), and EOI (End Of Image). The marker code SOI is used to mark the start of a JPEG image. The marker code APPn is used to mark application-specific user data contained in an image. The marker code APP0 is used to identify a JPEG file that uses the JFIF (JPEG File Interchange Format) specification. The marker codes APP1 to APP15 are marker codes for identifying an extended marker area. The marker code DQT is used to mark one or more quantization tables required for decoding an image. The marker code SOF is used to mark the start of a frame of an image. The marker code DHT is used to mark one or more Huffman tables required for decoding an image. The marker code DRI is used to mark the interval of resetting the accumulated scan code. The marker code SOS is used to mark the beginning of the scan line. The marker code EOI is used to mark the end of the JPEG image.

[0042] In practical applications, MJPEG video code streams encoded using MJPEG are transmitted between different devices. Figure 2 FIG. 1 is a schematic diagram of a transmission scenario of an MJPEG video code stream in the related art. Figure 2 As shown, the MJPEG video code stream is transmitted between the first device 201, the second device 202, and the third device 203.

[0043] The transmitter 202Tx of the second device 202 sends an MJPEG video code stream to the receiver 201Rx of the first device 201 through the link A. After receiving the MJPEG video code stream through the receiver 201Rx, the first device 201 may need to send it together with user data through the transmitter 201Tx to the third device 203. There are multiple links between the transmitter 201Tx of the first device 201 and the receiver 203Rx of the third device 203. Link A is used to implement the transmission of the MJPEG video code stream. In addition, in order to implement the transmission of user data, a new link, for example, link B and link C, can be established between the transmitter 201Tx of the first device 201 and the receiver 203Rx of the third device 203. Link B is used to implement the transmission of audio data. Link C is used to implement the transmission of a code stream using the H.26x encoding method. The audio data transmitted on link B and the code stream transmitted on link C are both additional user data. For example, in scenarios such as smart doorbells and door locks, network cameras, etc., MJPEG video streams are displayed on the screen, and H.264 video streams or audio and other user data are sent to the cloud for storage. Among them, MJPG has higher image quality and is suitable for display, and H.264 has a higher compression rate. The same video is compressed to a smaller size and is more suitable for storage. In this case, the second device 202 can be a camera, a streaming media server, or an MJPEG encoder integrated on the first device 201. The first device 201 can be an embedded device. The third device 203 can be an embedded device that can be connected to an external screen for display, and connected to the network to send user data to the cloud.

[0044] In the above transmission scenario, the establishment of a new link requires hardware support. Specifically, the hardware circuits of the transmitter 201Tx of the first device 201 and the receiver 203Rx of the third device 203 should support the simultaneous existence of a link for transmitting an MJPEG video stream and a link for transmitting user data. This will increase the design cost and manufacturing cost of the hardware circuit, as well as the complexity of the hardware circuit.

[0045] Therefore, when used for simultaneous transmission of MJPEG video code stream and user data, how to reduce the cost and complexity of hardware circuits is an urgent problem to be solved.

[0046] Figure 3 Schematic diagram of the transmission scenario of the MJPEG video code stream provided by the embodiment of the present disclosure. Figure 3 As shown, the MJPEG video code stream is transmitted between the first device 201, the second device 202, and the third device 203.

[0047] The transmitter 202Tx of the second device 202 sends a first video code stream to the receiver 201Rx of the first device 201 through the link A. The first video code stream is an MJPEG video code stream. After receiving the first video code stream through the receiver 201Rx, the first device 201 can add user data to the first video code stream to obtain a second video code stream. The transmitter 201Tx of the first device 201 sends a second video code stream to the receiver 203Rx of the third device 203 through the link A. The second video code stream transmitted between the first device 201 and the third device 203 through the link A not only includes the MJPEG video code stream, but also includes the user data added by the first device 201.

[0048] In some embodiments, the first device 201 may be an application server, the second device 202 may be a data server storing an MJPEG video code stream, and the third device 203 may be a terminal device. For example, a user may control a network player on a terminal device to play an MJPEG video. In this case, the data server may send the MJPEG video code stream to the application server; the application server may add user data to the MJPEG video code stream, and send the MJPEG video code stream with the added user data to the terminal device.

[0049] In some embodiments, the first device 201 may be a terminal device, the second device 202 may be a data server storing an MJPEG video code stream, and the third device 203 may be an attached device of the terminal device. For example, a user may control the terminal device to send an MJPEG video to an attached device for playback. In this case, the data server may send the MJPEG video code stream to the terminal device; the terminal device may add user data to the MJPEG video code stream, and send the MJPEG video code stream with the added user data to the attached device.

[0050] In some embodiments, the first device 201 may be a terminal device, the second device 202 may be an attached device of the terminal device, and the third device 203 may be an application server. For example, the terminal device may send the MJPEG video captured by the attached device to the application server. In this case, the attached device may send the MJPEG video code stream to the terminal device; the terminal device may add user data to the MJPEG video code stream, and send the MJPEG video code stream with the added user data to the application server.

[0051] In some embodiments, the second device 202 may be omitted. For example, the first device 201 may obtain an MJPEG video stream based on a locally stored MJPEG video file, add user data to the MJPEG video stream, and send the MJPEG video stream with the added user data to the third device 203.

[0052] Figure 4 2 is a flow chart of a data transmission method provided by an embodiment of the present disclosure. In some embodiments, the data transmission method of the embodiment of the present disclosure may be executed by the first device 201. Figure 4 As shown, the data transmission method of the embodiment of the present disclosure may include steps S410 to S440.

[0053] In step S410, a first video code stream is obtained.

[0054] Here, the first video code stream includes at least one first video frame.

[0055] In some embodiments, the first video code stream may be an MJPEG video code stream, and each first video frame in the first video code stream may be a JPEG image.

[0056] In some embodiments, the first video code stream may be obtained by the first device 201 from another device. In one example, the first device 201 may receive the first video code stream sent by the second device 202.

[0057] In some embodiments, the first video code stream may be obtained locally by the first device 201. In one example, the first device 201 may process a locally stored video file to obtain the first video code stream. The first video code stream may be encapsulated in a variety of video containers. The file formats of video files corresponding to different video containers may be different. For example, the file formats of video files may include MJPG, AVI (Audio Video Interleaved), MOV (Movie Format), MP4 (Moving Picture experts group 4), etc.

[0058] In some embodiments, the first video code stream may be obtained through a first link. For example, the first link may be a dedicated link for transmitting the first video code stream.

[0059] In step S420, user data to be transmitted is written into the application mark area of ​​at least one first video frame in sequence to obtain a second video frame.

[0060] Here, the first device 201 may determine that additional user data needs to be sent while sending the first video code stream. Then, the first device 201 may write the user data to be transmitted into the application mark area of ​​at least one first video frame of the first video code stream. The first video frame to which the user data to be transmitted is added may be referred to as a second video frame.

[0061] In some embodiments, the user data may include at least one of the following: a timestamp, subtitle data, audio data, and a third video code stream.

[0062] The timestamp may be used to indicate time information of the data. In some embodiments, the timestamp may be associated with the first video frame. Through the timestamp, the time order between the multiple first video frames in the first video code stream may be managed. In some embodiments, a timestamp may be used to indicate the reception time of the video frame corresponding to the timestamp in the first video code stream at the first device 201. The timestamp may be determined by the first device 201, for example. In some embodiments, the timestamp may be associated with at least one of the subtitle data, the audio data, and the third video code stream.

[0063] The subtitle data may include subtitles associated with the first video stream. In one example, the subtitle data may be received by the first device 201 from another device. For example, the first device 201 may obtain subtitle data corresponding to the first video stream through a network based on relevant information of the first video stream. In one example, the subtitle data may be obtained locally by the first device 201. For example, the first device 201 may obtain subtitle data corresponding to the first video stream from a local video file or subtitle file. In one example, the subtitle data may be generated by the first device 201. For example, the first device 201 may generate subtitle data based on the first video stream, and / or the third video stream, and / or audio data.

[0064] The audio data may include audio related to the first video stream. In one example, the audio data may be received by the first device 201 from another device. For example, the first device 201 may obtain audio data corresponding to the first video stream through a network based on the relevant information of the first video stream. In one example, the audio data may be obtained locally by the first device 201. For example, the first device 201 may obtain audio data corresponding to the first video stream from a local video file or audio file. In one example, the audio data may be collected and generated by the first device 201. For example, the first device 201 may collect sound signals in the device environment through a microphone and generate audio data.

[0065] The third video code stream may be a video code stream different from the first video code stream. The third video code stream may be encoded in a different manner from the first video code stream. For example, the third video code stream may be a video code stream encoded in H.26x. In one example, the third video code stream may be generated by the first device 201. For example, the third video code stream may be captured and generated by the first device 201 through a camera. For example, the third video code stream may be generated by the first device 201 through screen recording. In one example, the third video code stream may be obtained by the first device 201 based on a locally stored video file.

[0066] The user data may include data related to the first video code stream, and may also include data unrelated to the first video code stream. For example, one or more of the timestamp, subtitle data, audio data, and the third video code stream may be related to the first video code stream. It is understandable that one or more of the timestamp, subtitle data, audio data, and the third video code stream may be unrelated to the first video code stream.

[0067] It should be noted that user data may also include other data, which is not specifically limited in the embodiments of the present disclosure.

[0068] In some embodiments, when there is user data to be transmitted, the user data may be sequentially written into the application mark area of ​​at least one first video frame. At least a portion of all the user data may be written into a first video frame. When the user data is written into multiple first video frames, the multiple first video frames may be consecutive first video frames in the first video code stream.

[0069] It is understandable that user data can be written starting from any first video frame in the first video code stream. In some embodiments, the acquisition of user data can be earlier than the acquisition of the first video code stream, and the writing of user data can start from the first first video frame in the first video code stream. In some embodiments, the acquisition of user data can be later than the acquisition of the first video code stream, and the writing of user data can start from a first video frame in the middle of the first video code stream. For example, at a certain time in the process of the first device 201 receiving the first video code stream and sending it, the first device 201 can obtain user data. At this time, the first device 201 can write user data starting from the first video frame that has not yet been sent.

[0070] After writing user data in a first video frame in the first video stream, if the user data has not been completely written (i.e., there is still user data that has not been written), the user data can continue to be written in the next first video frame in the first video stream, and so on, until the user data to be transmitted is completely written into one or more first video frames of the first video stream.

[0071] Figure 5 FIG. 4 is a flow chart of step S420 in the data transmission method provided in the embodiment of the present disclosure. Figure 5 As shown, in some embodiments, step S420 may include steps S510 to S540.

[0072] In step S510, user data is written into the application mark area of ​​the i-th first video frame.

[0073] Here, the user data may be written into at least one first video frame. For the i-th first video frame in the at least one first video frame, the first device 201 may write the user data to obtain a second video frame corresponding to the i-th first video frame. Here, i is a positive integer.

[0074] It can be understood that the i-th first video frame can be any first video frame among the at least one first video frame in which the user data is written. Therefore, the value of i can be equal to 1, 2, 3, . . . .

[0075] In some embodiments, the first device 201 may include a memory. The memory may have at least one storage area for implementing storage of data. The storage areas of the memory may include: a first storage area, a second storage area, and a third storage area. The first storage area is used to store a first video frame. The size of the first storage area may be greater than or equal to the data volume of a first video frame. The second storage area is used to store user data. The size of the second storage area may be greater than or equal to the data volume of all user data. The third storage area is used to store a second video frame. The size of the third storage area may be greater than or equal to the data volume of the second video frame. The first storage area, the second storage area, and the third storage area may be pre-allocated.

[0076] The first storage area, the second storage area, and the third storage area may be used to implement writing of user data in the i-th first video frame. In some embodiments, the first device 201 may obtain the i-th first video frame from the first video code stream and store it in the first storage area. The user data obtained by the first device 201 may be stored in the second storage area. Afterwards, the first device 201 may write the user data into the application mark area of ​​the i-th first video frame and store the obtained second video frame in the third storage area.

[0077] In some embodiments, the storage area of ​​the memory may further include a fourth storage area. The fourth storage area is used to store temporary data during the user data write operation. In some embodiments, the fourth storage area may be pre-allocated or temporarily allocated.

[0078] In some embodiments, the first video frame may be a JPEG image, and the first video frame may include n application marker areas. Here, the value of n may be 15. User data may be written into the application marker area corresponding to marker codes APP1 to APP15. Therefore, user data may be written into the application marker area corresponding to at least one of marker codes APP1 to APP15 of the i-th first video frame. It should be noted that the first video frame may also include an application marker area marked by marker code APP0. However, user data is generally not written into the application marker area corresponding to marker code APP0.

[0079] In some embodiments, the implementation of step S510 may include: sequentially writing user data in n application mark areas in the i-th first video frame. Specifically, for the j-th application mark area among the n application mark areas in the i-th first video frame, the writing of user data may be performed. Here, j is a positive integer, and 1<j≤n.

[0080] Figure 6 FIG. 5 is a flow chart of step S510 in the data transmission method provided in the embodiment of the present disclosure. Figure 6 As shown, in some embodiments, step S510 may include steps S610 to S640.

[0081] In step S610, the data length of the first portion of the user data that has not been written is obtained.

[0082] Here, the first portion of data includes a portion of the user data that has not been written into the first video code stream. The first device 201 may obtain the data length of the first portion of data.

[0083] In some embodiments, the data length of the first portion of data may be represented by a first data length. In the case where the user data has not yet been written, the first data length may be equal to the data amount of all the user data. For example, the first device 201 may obtain the data amount of the user data in the second storage area as the first data length. In the case where part of the user data is written into the first video code stream, the first data length may be equal to the data amount of the remaining user data.

[0084] In some embodiments, the first data length may be stored in a specific storage area. For example, the first device 201 may allocate a storage area and store the first data length in the storage area. Thus, the first data length may be obtained from the storage area.

[0085] In some embodiments, the first device 201 may store the second data length in a specific storage area. The second data length may represent the data length of the portion of the user data that has been written into the first video code stream. For example, the first device 201 may allocate a storage area and store the second data length in the storage area. In this way, the second data length may be obtained from the storage area, and the first data length may be determined based on the second data length and the amount of user data.

[0086] In some embodiments, the storage area allocated for the first data length and / or the second data length may be a part of the second storage area or may be independent of the second storage area.

[0087] In step S620, it is determined whether the data length of the first portion of data is greater than the maximum user data size of the jth application marker area.

[0088] After obtaining the data length of the first portion of data, the first device 201 may compare the data length of the first portion of data with the maximum user data amount of the j-th application marker area.

[0089] The maximum user data amount refers to the maximum amount of user data that can be accommodated in the j-th application marker area. The data length of the user data written in the j-th application marker area should be less than or equal to the maximum user data amount. It is understandable that the application marker area of ​​each first video frame in the first video code stream can have the same maximum user data amount. Of course, the maximum user data amount of different first video frames and / or different application marker areas can be different.

[0090] In step S630, when the data length of the first part of the data is greater than the maximum user data size of the jth application marker area, the second part of the data is written to the jth application marker area, the value of j is set to j+1, and the process returns to step S610.

[0091] The comparison result obtained by step S620 may be that the data length of the first part of the data is greater than the maximum user data amount of the j-th application mark area. This means that the data length of the user data that has not yet been written is greater than the maximum data amount of the user data that can be accommodated in the j-th application mark area. In this case, the first device 201 may write the second part of the data to the j-th application mark area. The second part of the data is a part of the first part of the data, and the data length of the second part of the data is the maximum user data amount of the j-th application mark area.

[0092] In some embodiments, the process of writing the second portion of data to the jth application mark area may include: reading the second portion of data from the second storage area; and writing the second portion of data into the third storage area.

[0093] In some embodiments, the process of writing the second portion of data to the jth application tag area may include: reading the second portion of data from the second storage area; writing the second portion of data to the fourth storage area; and copying all data in the fourth storage area to the third storage area.

[0094] In some embodiments, step S630 may further include: writing relevant information of the jth application mark area into the jth application mark area. Here, the relevant information may include at least one of the following: mark code, area data length, user data type, video frame rate, reserved information, and user data length.

[0095] In some embodiments, in the jth application tag area, an example of the arrangement of user data and related information can be seen in Table 2 below.

[0096] Table 2

[0097]

[0098] The marker code is used to indicate the jth application marker area. The marker code may occupy 2 bytes and may range from 0xFFE1 to 0xFFEF. 0xFFE1 indicates the first application marker area in the first video frame, 0xFFE2 indicates the second application marker area in the first video frame, and so on.

[0099] The area data length is the data length of the jth application tag area. The area data length can occupy 2 bytes. The maximum value of the area data length can be 0xFFFF, that is, the maximum data length of the jth application tag area can be 65535.

[0100] The user data type is the type of user data written in the jth application tag area. The user data type can occupy 1 byte. 0x01 indicates that the user data type is H.264. 0x02 indicates that the user data type is H.265. 0x03 indicates that the user data type is H.266. 0x04 indicates that the user data type is ACC (Advanced Audio Coding). 0x06 indicates that the user data type is timestamp.

[0101] The video frame rate is used to indicate the frame rate of the first video code stream, that is, the number of video frames per second. The video frame rate can occupy 1 byte.

[0102] The reserved information may be reserved invalid information. The reserved information field may be filled with 0. The reserved information may occupy 6 bytes.

[0103] The user data length is the data length of the user data contained in the jth application mark area. The user data length may occupy 2 bytes. In one example, the user data length may be equal to the data length of the user data written in the jth application mark area plus 2 bytes. For example, the user data length may be equal to k+2. In one example, the user data length may be equal to the data length of the user data written in the jth application mark area. For example, the user data length may be equal to k.

[0104] In one example, the user data and related information in the jth application marking area may be arranged in the following order: marking code, area data length, user data type, video frame rate, reserved information, user data length, and user data.

[0105] It can be understood that the number of bytes, values, and arrangement order of the marking code, area data length, user data type, video frame rate, retention information, user data length, user data, etc. in the jth application marking area can be set according to actual needs, and the embodiments of the present disclosure do not make specific limitations on this.

[0106] In some embodiments, the process of writing the second portion of data to the jth application tag area may include: writing the area data length, user data type, video frame rate, reservation information, and user data length in the fourth storage area in sequence; reading the second portion of data from the second storage area and writing it to the fourth storage area; copying all the data in the fourth storage area to the third storage area. For example, writing 0xFFEj, 0xFFFF, 0x01, 0xF9, 0x000, and 0xFFF5 in sequence in the fourth storage area; then, reading 65523 bytes of user data from the second storage area and writing it to the fourth storage area; and copying all the data in the fourth storage area to the third storage area.

[0107] After the second part of the data is written, the value of j is updated. In one example, the value of j is set to j+1, thereby achieving the accumulation of j. After the value of j is accumulated, the process can return to step S610. In this way, user data can be written to the application mark area of ​​the updated j index.

[0108] In step S640, when the data length of the first partial data is equal to or smaller than the maximum user data amount of the jth application marker area, the first partial data is written to the jth application marker area.

[0109] The comparison result obtained by step S620 may be that the data length of the first part of the data is less than or equal to the maximum amount of user data of the j-th application mark area. This means that the data length of the user data that has not yet been written does not exceed the maximum amount of user data that can be accommodated in the j-th application mark area. In this case, the first device 201 may write the first part of the data to the j-th application mark area.

[0110] The specific implementation process of step S640 is similar to step S620, which will not be described in detail here. It can be understood that because the data length of the first part of the data is less than or equal to the maximum user data amount of the j-th application mark area, the data length of the user data written in the j-th application mark area can be less than or equal to the maximum user data amount. For example, the data length of the user data read from the second storage area can be less than or equal to 65523 bytes. Correspondingly, the value of the user data length can be less than or equal to 0xFFF5.

[0111] Through steps S610, S620 and S640, the last part of the user data can be written into the first video code stream. Through steps S610, S620 and S630, the rest of the user data except the last part can be written into the first video code stream. It can be understood that steps S610 to S640 are performed for the jth application mark area in the i-th first video frame. For each application mark area in the i-th first video frame, steps S610 to S640 can be performed once until all application mark areas are written with user data or all user data is written.

[0112] In some embodiments, when the second portion of data is written in the j=nth application mark area, user data is written in all application mark areas. In some embodiments, when the first data length is equal to 0, all user data is written.

[0113] In some embodiments, step S510 may further include step S650. In step S650, the first data length is updated.

[0114] Because the first data length indicates the amount of unwritten user data in the user data in the second storage area, after writing the user data in the jth application mark area, the first data length may be updated to indicate that the amount of unwritten user data has changed.

[0115] In some embodiments, when step S630 is performed, step S650 may include: updating the first data length to the difference between the data length of the first portion of data and the data length of the second portion of data. When step S630 is performed, if the data length of the first portion of data is greater than the maximum user data amount of the j-th application mark area, the second portion of data in the first portion of data is written. In this case, the first data length may be updated to the difference between the data length of the first portion of data and the data length of the second portion of data.

[0116] In some embodiments, when step S640 is performed, step S650 may include: updating the first data length to 0. When step S640 is performed, if the data length of the first portion of data is less than or equal to the maximum user data amount of the j-th application mark area, the first portion of data is all written. In this case, the first data length may be updated to 0. This means that all user data has been written into the first video code stream.

[0117] In some embodiments, for the ith first video frame, the writing of user data can be specifically implemented as follows: obtaining the ith first video frame from the first video code stream and storing it in the first storage area; parsing the ith first video frame in the first storage area; copying the marker SOI in the ith first video frame to the third storage area; copying the marker APP0 and the corresponding field in the ith video frame to the third storage area; for at least one application marker area in the ith first video frame, executing the writing of user data, and copying the application marker area where the user data is written to the third storage area; copying other markers and corresponding fields in the ith first video frame to the third storage area. In this way, a second video frame corresponding to the ith first video frame and having the user data written therein can be obtained in the third storage area.

[0118] In step S520, it is determined whether the user data is completely written into the first video frame.

[0119] After the writing of the application mark area in the i-th first video frame is completed, it may be determined whether the user data is completely written into the first video frame.

[0120] In some embodiments, when user data is written to all application mark areas in the i-th first video frame and there is unwritten user data, it can be determined that the user data is not all written to the first video frame. In some embodiments, when user data is written to all application mark areas in the i-th first video frame and there is no unwritten user data, it can be determined that the user data is all written to the first video frame. In some embodiments, when user data is written to part of the application mark areas in the i-th first video frame, it can be determined that the user data is all written to the first video frame.

[0121] In one example, in step S520, it can be determined whether the user data is completely written into the first video frame according to the first data length. For example, when writing of all application mark areas of the first video frame is completed, if the first data length is greater than 0, it is determined that the user data is not completely written into the first video frame. For example, when writing of part or all application mark areas of the first video frame is completed, if the first data length is equal to 0, it is determined that the user data is completely written into the first video frame.

[0122] In step S530 , if the user data has not been completely written, the value of i is set to i+1, and the process returns to step S510 .

[0123] If it is determined that the user data has not been fully written, the value of i may be updated. In one example, the value of i is set to i+1, thereby achieving accumulation of i. After accumulating the value of i, the process may return to step S510. In this way, the user data may be written to the first video frame of the updated i index.

[0124] In step S540, when all the user data is written, a second video frame is obtained.

[0125] When it is determined that all the user data has been written, the second video frame may be obtained based on the first video frame in which the user data has been written.

[0126] In some embodiments, the number of the second video frames may be one or more. Specifically, the number of the second video frames may be equal to the number of the first video frames.

[0127] In step S430, a second video code stream is generated based on the second video frame.

[0128] Here, after obtaining one or more second video frames containing user data, a second video code stream may be generated based on the second video frames.

[0129] In some embodiments, the first device 201 may read the second video frame from the third storage area and generate a second video code stream.

[0130] In some embodiments, the generation of the second video code stream can be continuous. In the case where a portion of the user data is written into the first video frame to obtain the second video frame, the first device 201 can generate the second video code stream based on the second video frame already existing in the third storage area. In the case where the user data is continuously written and a subsequent second video frame is obtained, the newly obtained second video frame can be added to the second video code stream. In this way, the first device 201 can continuously perform the following process: obtain the first video frame in the first video code stream; write the user data to be transmitted in the first video frame to obtain the second video frame; generate the second video code stream based on the second video frame that has been obtained.

[0131] In step S440, the second video code stream is sent.

[0132] After generating the second video code stream, the first device 201 may send the second video code stream to the third device 203 .

[0133] In some embodiments, the second video code stream may be obtained through a second link. The second link may be, for example, a dedicated link for transmitting the second video code stream.

[0134] In some embodiments, the second link may have the same function as the first link. For example, since the first video code stream and the second video code stream use the same encoding technology, the first link and the second link may both be links for transmitting the video code stream using the encoding technology. In one example, the first video code stream may be an MJPEG video code stream, and the second video code stream is also an MJPEG video code stream. In this case, the first link is used to transmit the MJPEG video code stream, and the second link is also used to transmit the MJPEG video code stream.

[0135] Through the above steps S410 to S440, after obtaining the first video code stream, the first device can encapsulate the user data in the application identification area of ​​the first video frame of the first video code stream to obtain the second video code stream. After that, the first device can send the second video code stream. In the case where the video code stream and the user data need to be transmitted simultaneously, the user data is written into the application mark area of ​​the video frame of the video code stream to generate a new video code stream. The new video code stream uses the video coding of the original video code stream, and the new video code stream can be transmitted through the transmission link corresponding to the video code stream. In this way, the transmission of the video code stream and the user data can be realized simultaneously only through the link used for the video code stream, without the need to establish a new transmission link for the user data. The hardware circuits of the receiving end and the transmitting end in the electronic device only need to support the link of the video code stream, thereby reducing the design and manufacturing cost of the hardware circuit and reducing the complexity of the hardware circuit.

[0136] Figure 7 FIG. 1 is a flow chart of an exemplary embodiment of a data transmission method provided by an embodiment of the present disclosure. Figure 7 As shown, the data transmission method in the embodiment of the present disclosure may include steps S701 to S718.

[0137] In this embodiment, the first video code stream may be an MJPEG video code stream, and the first video frame may be an MJPEG video frame. In addition, the first storage space may be a data area A, the second storage space may be a data area B, the third storage space may be a data area C, and the fourth storage space may be a data area T.

[0138] In step S701, a video frame is obtained from the MJPEG video code stream and stored in data area A.

[0139] In step S702, the user data to be transmitted is obtained and stored in data area B.

[0140] In step S703, the marker SOI at the first two bytes in the video frame is parsed and copied to the data area C. After that, the marker APP0 at the two bytes after SOI is parsed to obtain the data length of the application marker area corresponding to APP0, and the data of the application marker area corresponding to the marker APP0 is copied to the data area C according to the obtained data length.

[0141] In step S704, a variable j is set to 1. The variable j is the index of the application mark area where data is written in the video frame.

[0142] In step S705, the marker of the j-th application marker area, ie, 0xFFEj, is written into the allocated data area T.

[0143] In step S706, the data length of data area B is obtained. The data length is the data length of the unwritten user data. If the data length of data area B is greater than the data length that can be accommodated by the j-th application mark area (for example, 65523 bytes), continue to execute step S707; otherwise, continue to execute step S713.

[0144] In step S707, 0xFFFF, user data type, video frame rate, and reserved information are written to the data area T in sequence. 0xFFFF indicates that the data length of the j-th application mark area is 65535 bytes. For the user data type, 0x01 can be written to indicate that the data type of the user data is an H.264-encoded video stream. For the video frame rate, 25 can be written to indicate that the number of frames per second is 25. The reserved information can all be 0.

[0145] In step S708, 0xFFF5 is written into the data area T. 0xFFF5 indicates that the data length of the user data written in the jth application tag area is 65523 bytes. It can be understood that, for the user data length, writing 0xFFF5 indicates that the data length of the user data is 65523 bytes, because 0xFFF5 occupies 2 bytes.

[0146] In step S709, 65523 bytes of user data are read from data area B and written into data area T. The user data is located after 0xFFF5.

[0147] In step S710, all data in the data area T are copied to the data area C. By executing steps S705 to S709, the writing of user data to the j-th application mark area can be completed. All data of the application mark area containing user data is stored in area T. In this step, all data of the j-th application mark area are copied from the data area T to the data area C. It can be understood that in the data area C, the data of the j-th application mark area can be located after the data of the j-1-th application mark area.

[0148] In step S711, it is determined whether j is equal to n. By comparing the magnitude relationship between j and n, it can be determined whether the writing of user data to all application mark areas in the video frame is completed. If j is less than n, continue to step S712; otherwise, continue to step S716.

[0149] In step S712, j is set to j+1, and the data length of the unwritten user data in data area B is updated. Thereafter, the process jumps to step S705, so as to write user data to the next application mark area.

[0150] In step S713, the data length, user data type, video frame rate, and reserved information are written to the data area T in sequence. For the data length, the written value can be equal to or less than 0xFFFF, that is, the data length of the j-th application mark area is less than or equal to 65535 bytes. For the user data type, 0x01 can be written to indicate that the data type of the user data is an H.264-encoded video stream. For the video frame rate, 25 can be written, indicating that the number of frames per second is 25. The reserved information can all be 0. It can be understood that the written data length is equal to the data length of the user data not written in the data area B plus 12 bytes.

[0151] In step S714, the user data length is written into the data area T. For the user data length, the written value may be equal to or less than 0xFFF5, that is, the data length of the user data written in the jth application mark area is less than or equal to 65523 bytes. It can be understood that the user data length is equal to the data length of the user data not written in the data area B plus 2 bytes.

[0152] In step S715, all data in the data area T are copied to the data area C. By executing steps S713 to S715, the writing of user data to the j-th application mark area can be completed. All data of the application mark area containing user data is stored in area T. In this step, all data of the j-th application mark area are copied from the data area T to the data area C. It can be understood that in the data area C, the data of the j-th application mark area can be located after the data of the j-1-th application mark area.

[0153] In step S716, the data length of the unwritten user data in data area B is updated. If the jump is from step S711 to step S716, the updated data length of the unwritten user data is greater than 0. If the jump is from step S715 to step S716, the updated data length of the unwritten user data is equal to 0.

[0154] In step S717, it is determined whether the data length of the updated unwritten user data is 0. According to the size of the data length of the updated unwritten user data, it can be determined whether to continue writing the user data. If the data length of the updated unwritten user data is greater than 0, continue to step S718; otherwise, end the process.

[0155] In step S718, it is determined whether the MJPEG video code stream has any more video frames. If the MJPEG video code stream has any video frames, the process goes to step S701 to obtain the next video frame; otherwise, the process ends.

[0156] It should be noted that the data transmission method in the embodiment of the present disclosure can be implemented by hardware, software, or a combination of hardware and software. For example, each step in the data transmission method can be implemented by a hardware circuit. For example, each step in the data transmission method can be implemented by a processor executing a code. For example, a part of the steps in the data transmission method can be implemented by a hardware circuit, and another part of the steps can be implemented by a processor executing a code.

[0157] Based on the same inventive concept, the present disclosure provides a data transmission device. Figure 8 Schematic diagram of the structure of the data transmission device provided by the embodiment of the present disclosure. Figure 8As shown, the data transmission device 800 includes: a code stream acquisition module 801, an encapsulation module 802, a code stream generation module 803, and a sending module 804. The code stream acquisition module 801 is configured to acquire a first video code stream, wherein the first video code stream includes at least one first video frame. The code stream encapsulation module 802 is configured to sequentially write user data to be transmitted into the application mark area of ​​at least one first video frame to obtain a second video frame. The code stream generation module 803 is configured to generate a second video code stream based on the second video frame. The sending module 804 is configured to send the second video code stream.

[0158] In some embodiments, the encapsulation module 802 can be configured to: write user data to the application mark area of ​​the i-th first video frame; determine whether the user data is completely written into at least one first video frame; if the user data is not completely written, set the value of i to i+1, and return to the step of writing user data to the application mark area of ​​the i-th first video frame; if the user data is completely written, obtain a second video frame; wherein i is a positive integer.

[0159] In some embodiments, the i-th first video frame includes n application mark areas; wherein the encapsulation module 802 can be configured to: for the j-th application mark area in the i-th first video frame, obtain the data length of the first part of the data that has not been written in the user data; determine whether the data length of the first part of the data is greater than the maximum user data amount of the j-th application mark area; when the data length of the first part of the data is greater than the maximum user data amount of the j-th application mark area, perform the following steps in sequence: write the second part of the data in the first part of the data to the j-th application mark area, wherein the data length of the second part of the data is the maximum user data amount of the j-th application mark area; set the value of j to j+1, and return to the step of obtaining the data length of the first part of the data that has not been written in the user data; when the data length of the first part of the data is equal to or less than the maximum user data amount of the j-th application mark area, write the first part of the data to the j-th application mark area; wherein j and n are both positive integers, 1<j≤n.

[0160] In some embodiments, the encapsulation module 802 can be configured to: write relevant information of the jth application marking area into the jth application marking area; wherein the relevant information includes at least one of the following: marking code, data length, user data type, video frame rate, reservation information, user data length.

[0161] In some embodiments, the data length of the first part of data can be represented by the first data length; wherein the encapsulation module 802 can be configured to: when the second part of data is written to the j-th application marker area, update the first data length to the difference between the data length of the first part of data and the data length of the second part of data; when the first part of data is written to the j-th application marker area, update the first data length to 0.

[0162] In some implementations, the encapsulation module 802 may be configured to: when the first data length is equal to 0, determine that the user data is completely written; and when the first data length is greater than 0, determine that the user data is not completely written.

[0163] In some implementations, the user data includes at least one of the following: a timestamp, subtitle data, audio data, and a third video code stream.

[0164] Based on the same inventive concept, an embodiment of the present disclosure provides an electronic device. Fig. 9 Schematic diagram of the structure of the electronic device provided by the embodiment of the present disclosure. Fig. 9 As shown, the electronic device 900 includes a processor 901 and a memory 902. The memory 902 is connected to the processor 901 and is used to store executable instructions. The processor 901 is configured to execute the executable instructions in the memory 902 to implement the data transmission method in each embodiment of the present disclosure.

[0165] In some embodiments, the electronic device 900 may further include an image acquisition module, such as a camera, which acquires real-time images and generates multimedia content such as MJPEG video streams, H.264 video streams, and audio.

[0166] In some embodiments, electronic device 900 may also include one or more output devices that enable presentation of multimedia content, including one or more speakers and / or one or more visual display screens.

[0167] In some embodiments, the electronic device 900 may further include a network communication module for reaching other devices via one or more (wired or wireless) network interfaces. In one embodiment, the network interface may include: Bluetooth, Wireless Fidelity (Wi-Fi), Universal Serial Bus (USB), etc.

[0168] Of course, the above is only an example of the electronic device 900, and the embodiment of the present disclosure does not limit the specific structure of the electronic device 900.

[0169] Based on the same inventive concept, an embodiment of the present disclosure provides a data transmission system. Fig.10This is a schematic diagram of the structure of the data transmission system provided by the embodiment of the present disclosure. Fig.10 As shown, the data transmission system 1000 includes a first device 201, a second device 202, and a third device 203. The first device 201, the second device 202, and the third device 203 can implement the data transmission method in each embodiment of the present disclosure.

[0170] In some embodiments, the first device 201 may include a processor 901a, a memory 902a, and a network interface 1001. The network interface 1001 is used to communicate with the second device 202 and the third device 203. In one example, the network interface 1001 may be a USB.

[0171] In some embodiments, the second device 202 may include a processor 901b, a memory 902b, a network interface 1002, and an image acquisition module 1003. The network interface 1002 is used to communicate with the network interface 1001 of the first device 202. In one example, the network interface 1002 may be a USB. In one example, the image acquisition module 1003 may be a camera.

[0172] In some embodiments, the third device 203 may include a processor 901c, a memory 902c, a network interface 1004, a network interface 1005, and an output device 1006. The network interface 1004 is used to communicate with the network interface 1001 of the first device 202. The network interface 1005 is used to communicate with a cloud server. In one example, the network interface 1004 may be a USB. In one example, the network interface 1005 may be a Wi-Fi. In one example, the output device 1006 may be a visual display screen.

[0173] Of course, the above is only an example of the data transmission system 1000, and the embodiment of the present disclosure does not limit the specific structure of the data transmission system 1000.

[0174] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0175] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data transmission method, characterized in that: The method comprises: Acquire a first video code stream, wherein the first video code stream includes at least one first video frame; Writing the user data to be transmitted into the application mark area of ​​the at least one first video frame in sequence to obtain a second video frame; Based on the second video frame, generate a second video stream; Send the second video code stream.

2. The method according to claim 1, characterized in that The step of sequentially writing the user data to be transmitted into the application mark area of ​​the at least one first video frame to obtain a second video frame includes: Writing the user data into the application mark area of ​​the i-th first video frame; determining whether the user data is completely written into the at least one first video frame; In the case that the user data has not been completely written, the value of i is set to i+1, and the process returns to the step of writing the user data into the application mark area of ​​the i-th first video frame; When all the user data are written, obtaining the second video frame; Wherein, i is a positive integer.

3. The method according to claim 2, characterized in that The i-th first video frame includes n application mark areas; The step of writing the user data into the application mark area of ​​the i-th first video frame comprises: for the j-th application mark area in the i-th first video frame, Obtaining the data length of the first part of the user data that has not been written; Determining whether the data length of the first part of data is greater than the maximum user data amount of the jth application mark area; When the data length of the first part of data is greater than the maximum user data amount of the j-th application mark area, the following steps are performed in sequence: Writing the second part of the data in the first part of the data into the j-th application mark area, wherein the data length of the second part of the data is the maximum amount of user data in the j-th application mark area; The value of j is set to j+1, and the process returns to the step of obtaining the data length of the first part of the user data that has not been written; When the data length of the first part of data is equal to or less than the maximum user data amount of the j-th application mark area, writing the first part of data into the j-th application mark area; Wherein, j and n are both positive integers, 1<j≤n.

4. The method according to claim 3, characterized in that The step of writing the second part of data into the j-th application mark area comprises: Write relevant information of the j-th application marking area into the j-th application marking area; The relevant information includes at least one of the following: a marking code, a region data length, a user data type, a video frame rate, reserved information, and a user data length.

5. The method according to claim 3, characterized in that: The data length of the first part of data is represented by a first data length; The step of writing the user data into the application mark area of ​​the i-th first video frame further includes: When writing the second portion of data to the j-th application marker area, updating the first data length to the difference between the data length of the first portion of data and the data length of the second portion of data; When the first portion of data is written to the j-th application marker area, the first data length is updated to 0.

6. The method according to claim 5, characterized in that The determining whether the user data is completely written into the at least one first video frame comprises: When the first data length is equal to 0, determining that all the user data are written; When the first data length is greater than 0, it is determined that the user data has not been completely written.

7. The method according to any one of claims 1 to 6, characterized in that The user data includes at least one of the following: a timestamp, subtitle data, audio data, and a third video code stream.

8. A data transmission device, characterized in that: The device comprises: A code stream acquisition module, configured to acquire a first video code stream, wherein the first video code stream includes at least one first video frame; An encapsulation module, configured to sequentially write the user data to be transmitted into the application mark area of ​​the at least one first video frame to obtain a second video frame; A code stream generating module, configured to generate a second video code stream based on the second video frame; The sending module is configured to send the second video code stream.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory connected to the processor and configured to store executable instructions; The processor is configured to execute the executable instructions in the memory to implement the data transmission method according to any one of claims 1 to 7.

10. A storage medium having executable instructions stored thereon, characterized in that: When the executable instructions are executed by a processor, the processor is caused to implement the data transmission method according to any one of claims 1 to 7.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the data transmission method according to any one of claims 1 to 7 is implemented.

Images