Streaming method, device, terminal and storage medium
By setting the video encoding type identifier in the RTMP push streaming protocol and encapsulating H265 or H264 data, the problem that RTMP push streaming cannot achieve video above 4K is solved, and the high-definition video push streaming is realized to meet higher resolution needs.
Patent Information
- Application Number
- CN202210015842.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-07
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-01-07
AI Technical Summary
The existing RTMP streaming technology cannot achieve high-definition and high-quality video streaming of 4K or above, and is limited by the video encoding type.
Using the method based on the RTMP-H265 push streaming protocol, videocodecid is identified by setting the video encoding type, and using this identification is used to form FLV FILE BODY and FLV FILE HEADER, and video data of H265 or H264 encoding type is encapsulated and pushed.
It realizes the streaming of high-definition and high-quality video data above 4K, meets the needs of higher resolution video data, and expands the applicable prospects of streaming.
Smart Images

Figure CN114363662B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of streaming, and in particular to a streaming method, device, terminal and storage medium. Background Art
[0002] In recent years, due to the popularity of high-definition and high-quality videos, higher requirements have been put forward for streaming technology. In the existing technology, RTMP streaming is limited by the video encoding type of the streaming, and it is impossible to achieve streaming above 4K, such as 6K, 8K or even 12K videos.
[0003] Therefore, it is necessary to provide a streaming method based on the RTMP-H265 streaming protocol to solve the above technical problems. Summary of the invention
[0004] The main purpose of the present invention is to provide a streaming method, device, terminal, and computer-readable storage medium, aiming to solve the problem that high-definition and high-quality video data streaming cannot be achieved in the prior art.
[0005] To achieve the above object, the present invention provides a streaming method based on the RTMP-H265 streaming protocol, the streaming method is applied to the RTMP streaming end, and the streaming method includes:
[0006] Setting a video coding type identifier videocodecid based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type;
[0007] Use the set videocodecid and the structure of PreviousTagSize, NextTagHeader, and NextTagData to form FLVFILEBODY in a circular arrangement;
[0008] Use Signature, Version, Flags and DataOffset to form FLVFILEHEADER;
[0009] Complete data encapsulation of the first video data of H265 encoding type or H264 encoding type with FLVFILEBODY and the corresponding FLVFILEHEADER;
[0010] The encapsulated second video data is pushed.
[0011] Optionally, setting a video coding type identifier videocodecid based on the video coding type includes:
[0012] The preset position corresponding to the videocodecid or the random position in the preset area corresponding to the videocodecid is set to the video coding identification character 7 or 12.
[0013] Optionally, setting a video coding type identifier videocodecid based on the video coding type includes:
[0014] Determine the setting location of the videocodecid based on the preset storage path information; wherein the preset storage path information includes the storage level information and / or the bit sequence information of the videocodecid; the setting location includes one or more;
[0015] The videocodecid is set based on the setting position and the video encoding type.
[0016] Optionally, the preset storage path information is storage level information of the videocodecid; the storage level information includes at least a first preset storage level and a first preset sub-storage level attached to the first preset storage level; and determining the setting location of the videocodecid based on the preset storage path information includes:
[0017] Finding a first-level location corresponding to the first preset storage level;
[0018] Searching for a first sub-level position corresponding to the first preset sub-storage level under the first level position;
[0019] The setting position of the video coding type identifier videocodecid is determined based on the first sub-level position.
[0020] Optionally, the preset storage path information is the storage level information of the videocodecid; the storage level information at least includes the second preset storage level to the nth preset storage level arranged in sequence, wherein n>2; the determining the setting position of the videocodecid based on the preset storage path information includes:
[0021] Starting from the second preset storage level, searching and traversing each preset storage level position in sequence until the nth level position corresponding to the nth preset storage level is determined;
[0022] The setting position of the video coding type identifier videocodecid is determined based on the n-th level position.
[0023] Optionally, the streaming method is applied to a streaming system, the streaming system at least comprising an RTMP streaming end, an RTMP streaming server and an RTMP player; the streaming method comprises:
[0024] The RTMP streaming end executes the steps of the streaming method as described above, wherein when the RTMP streaming end executes the step of streaming the encapsulated second video data, the RTMP streaming end pushes the second video data to the RTMP streaming server;
[0025] The RTMP streaming server receives the second video data pushed by the RTMP streaming end, and extracts the videocodecid character in the second video data;
[0026] If the videocodecid character in the second video data is 12, the RTMP streaming server determines that the second video data is of H265 encoding type;
[0027] If the videocodecid character in the second video data is 7, the RTMP streaming server determines that the second video data is of H264 encoding type;
[0028] The RTMP streaming server sends the second video data to the RTMP player;
[0029] The RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodeci d character in the second video data;
[0030] If the videocodecid character in the second video data is 12, the RTMP player decodes and plays the second video data using an H265 decoder;
[0031] If the videocodecid character in the second video data is 7, the RTMP player uses the H264 decoder to decode and play the second video data.
[0032] Optionally, the RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid character in the second video data, including:
[0033] The RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid characters in the second video data from a plurality of videocodecid setting positions;
[0034] The RTMP player receives the second video data sent by the RTMP streaming server, and after the step of extracting the videocodeci d character in the second video data, comprises:
[0035] Compare and verify the videocodecid characters in multiple preset positions;
[0036] If the comparison verification results are inconsistent, verification result information indicating a verification error is returned to the RTMP stream server.
[0037] To achieve the above object, the present invention further provides a flow-pushing device, the flow-pushing device comprising:
[0038] A videocodecid setting module, used to set a video coding type identifier videocodecid based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type;
[0039] The first forming module is used to form FLV FILE BODY by using the set videocodecid and the structure in which PreviousTagSize, NextTagHeader and NextTagData are arranged in a circular sequence;
[0040] The second forming module is used to form FLVFILE HEADER using Signature, Version, Flags and DataOffset;
[0041] A data encapsulation module, used for completing data encapsulation of the first video data of the H265 encoding type or the H264 encoding type with the FLV FILE BODY and the corresponding FLV FILE HE ADER;
[0042] The streaming module is used to perform streaming operation on the encapsulated second video data.
[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides a streaming terminal, which includes a processor and a memory, and a streaming program is stored in the memory. When the streaming program is executed by the processor, the steps of the streaming method described above are implemented.
[0044] In addition, to achieve the above-mentioned purpose, the present invention further provides a computer-readable storage medium on which a streaming program is stored, and when the streaming program is executed by a processor, the steps of the streaming method described above are implemented.
[0045] Compared with the prior art, the streaming method, device, terminal and computer-readable storage medium provided by the present invention have the following beneficial effects: by setting a video coding type identifier videocodecid based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type, and then using the set videocodecid and a structure in which PreviousTagSize, NextTagHeader and NextTagData are arranged in a circular sequence to form an FLV FILEBODY, and then using Signature, Version, Flags and DataOffset to form an FLV FILE HEADER, the FLV FILEBODY and the corresponding FLVFILEHEADER complete the data encapsulation of the first video data of the H265 coding type or the H264 coding type, and finally the encapsulated second video data is streamed, thereby realizing the streaming of video data of the H265 video coding type using RTMP, and then realizing the streaming of high-definition and high-quality video data above 4k, meeting the needs of streaming of higher-definition video data, and having a broader application prospect. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a schematic diagram of the system structure of the hardware operating environment involved in the embodiment of the present invention;
[0047] Figure 2 This is a flow chart of an embodiment of a streaming method of the present invention;
[0048] Figure 3 A schematic diagram of the general structure of FLV encapsulation of video data in the present invention;
[0049] Figure 4 This is a diagram showing specific data examples of FLV FILE BODY in one embodiment of the present invention;
[0050] Figure 5 This is a diagram showing specific data examples of FLV FILE HEADER in one embodiment of the present invention;
[0051] Figure 6 It is a schematic diagram of the flow chart of another embodiment of the streaming method of the present invention;
[0052] Figure 7 It is a schematic diagram of the flow chart of another embodiment of the streaming method of the present invention;
[0053] Figure 8 FIG. 4 is a flow chart of another embodiment of the streaming method of the present invention. DETAILED DESCRIPTION
[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0056] In the prior art, RTMP streaming is limited by the video encoding type of the streaming, and cannot achieve streaming above 4K, such as 6K, 8K or even 12K video.
[0057] In order to solve the above technical problems, the present invention provides a streaming method. In the method, a video coding type identifier videocodecid is set based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type, and then an FLV FILE BODY is formed by using the set videocodecid and a structure in which PreviousTagSize, NextTagHeader, and NextTagData are arranged in a circular sequence, and then an FLV FILE HEADER is formed by using Signature, Version, Flags, and DataOffset, and the FLV FILE BODY and the corresponding FLVFILE HEADER are used to complete data encapsulation of first video data of the H265 coding type or the H264 coding type, and finally the encapsulated second video data is pushed, thereby realizing the use of RTMP to push the video data of the H265 video coding type, and then realizing the push of high-definition and high-quality video data above 4k.
[0058] like Figure 1 As shown, Figure 1 It is a schematic diagram of the system structure of the hardware operating environment involved in the embodiment of the present invention.
[0059] The terminal of the embodiment of the present invention can be a terminal device with computing capability, or a PC, or a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group AudioLayer IV) player, a portable computer, or other mobile terminal devices with display capabilities.
[0060] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0061] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the mobile terminal may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.
[0062] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0063] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a streaming program.
[0064] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the streaming program stored in the memory 1005 and perform the following operations:
[0065] Setting a video coding type identifier videocodecid based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type;
[0066] The FLV FILE BODY is formed by using the set videocodecid and the structure of PreviousTagSize, NextTagHeader, and NextTagData arranged in a circular sequence;
[0067] Use Signature, Version, Flags and DataOffset to form FLV FILE HEADER;
[0068] Complete data encapsulation of the first video data of the H265 encoding type or the H264 encoding type with the FLV FILE BODY and the corresponding FLV FILE HEADER;
[0069] The encapsulated second video data is pushed.
[0070] Furthermore, the processor 1001 may call the streaming program stored in the memory 1005, and further perform the following operations:
[0071] The preset position corresponding to the videocodecid or the random position in the preset area corresponding to the videocodecid is set to the video coding identification character 7 or 12.
[0072] Furthermore, the processor 1001 may call the streaming program stored in the memory 1005, and further perform the following operations:
[0073] Determine the setting location of the videocodecid based on the preset storage path information; wherein the preset storage path information includes the storage level information and / or the bit sequence information of the videocodecid; the setting location includes one or more;
[0074] The videocodecid is set based on the setting position and the video encoding type.
[0075] Furthermore, the processor 1001 may call the streaming program stored in the memory 1005, and further perform the following operations:
[0076] Finding a first-level location corresponding to the first preset storage level;
[0077] Searching for a first sub-level position corresponding to the first preset sub-storage level under the first level position;
[0078] The setting position of the video coding type identifier videocodecid is determined based on the first sub-level position.
[0079] Furthermore, the processor 1001 may call the streaming program stored in the memory 1005, and further perform the following operations:
[0080] Starting from the second preset storage level, searching and traversing each preset storage level position in sequence until the nth level position corresponding to the nth preset storage level is determined;
[0081] The setting position of the video coding type identifier videocodecid is determined based on the n-th level position.
[0082] Furthermore, the processor 1001 may call the streaming program stored in the memory 1005, and further perform the following operations:
[0083] The RTMP streaming end executes the steps of the streaming method as described above, wherein when the RTMP streaming end executes the step of streaming the encapsulated second video data, the RTMP streaming end pushes the second video data to the RTMP streaming server;
[0084] The RTMP streaming server receives the second video data pushed by the RTMP streaming end, and extracts the videocodecid character in the second video data;
[0085] If the videocodecid character in the second video data is 12, the RTMP streaming server determines that the second video data is of H265 encoding type;
[0086] If the videocodecid character in the second video data is 7, the RTMP streaming server determines that the second video data is of H264 encoding type;
[0087] The RTMP streaming server sends the second video data to the RTMP player;
[0088] The RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid character in the second video data;
[0089] If the videocodecid character in the second video data is 12, the RTMP player decodes and plays the second video data using an H265 decoder;
[0090] If the videocodecid character in the second video data is 7, the RTMP player uses the H264 decoder to decode and play the second video data.
[0091] Furthermore, the processor 1001 may call the streaming program stored in the memory 1005, and further perform the following operations:
[0092] The RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid characters in the second video data from a plurality of videocodecid setting positions;
[0093] The RTMP player receives the second video data sent by the RTMP streaming server, and after the step of extracting the videocodecid character in the second video data, comprises:
[0094] Compare and verify the videocodecid characters in multiple preset positions;
[0095] If the comparison verification results are inconsistent, verification result information indicating a verification error is returned to the RTMP stream server.
[0096] Reference Figure 2 , Figure 2 This is a flow chart of an embodiment of the streaming method of the present invention.
[0097] The streaming method of the present invention is based on the RTMP-H265 streaming protocol. In the general prior art, the video encoding format of the general RTMP streaming is H264. The present invention proposes an RTMP-H265 streaming method, the difference of which is that the H265 video format can be used for streaming. The RTMP-H265 streaming protocol is developed based on the traditional RTMP streaming protocol. The RTMP protocol is based on the TCP protocol, and the RTMP protocol uses FLV format-based encapsulation of video data in streaming. The streaming method of this embodiment can be applied to the RTMP streaming end. In some embodiments, the streaming method of the present invention includes the following steps:
[0098] Step S10, setting a video coding type identifier videocodecid based on the video coding type, wherein the video coding type at least includes an H265 coding type or an H264 coding type.
[0099] The video coding type identifier videocodecid in the embodiment of the present invention is used to identify the video coding type, and the video coding type includes but is not limited to the H265 coding type and the H264 coding type. Among them, the H265 coding type corresponds to the video coding type identification character 12, and the H264 coding type corresponds to the video coding type character 7. Setting the video coding type identifier videocodecid means setting the value of the storage location or setting location corresponding to the videocodecid to character 12 or character 7 according to the actual coding type of the video data to be pushed. Videocodecid is generally set in the FLV meta information metadata, and metadata is some information that describes various attributes of the FLV file. This information is saved in the AMF format at the beginning of the file. FLV meta information can include: audiochannels, audiocodecid, audiodatarate, audiodevice, audioinputvolume, audiosamplerate, creationdate, duration (mediafiles only), fmleversion (Flash Media Live Encoder version) (media files only), framerate, height, lastkeyframetimestamp (media files only), lasttimestamp (mediafiles only) only), presetname, videocodecid, videodatarate, videodevice, videokeyframe_frequency, width.
[0100] Step S20, forming FLV FILE BODY by using the set videocodecid and the structure in which PreviousTagSize, NextTagHeader and NextTagData are arranged in a circular sequence;
[0101] The videocodecid of the present invention is set in a specific position in the FLV FILE BODY or a random position in a specific area. The videocodecid that has been set means that it has been set in the FLV FILE BODY.
[0102] like Figure 3 As shown, Figure 3The general structure diagram of FLV encapsulated video data. The video data encapsulated by FLV consists of two parts: FLV FILE BODY and FLV FILEHEADER. Among them, FLV FILE BODY consists of Previous Tag size and Tag loop, and Tag can be divided into TagHeader and TagData. Therefore, FLV FILE BODY is formed by the following loop structure: prev_tag_size+NextTagHeader+NextTagData|prev_tag_size+NextTagHeader+NextTagData|prev_tag_size+NextTagHeader+NextTagData|……. Among them, the size of prev_tag_size is 4 bytes, marking the size of the previous TagHeader + TagData. Because there is no corresponding data before the first Tag, the four bytes are 0, and the four bytes here are big endian; the size of NextTagHeader is generally 11 bytes, which contains tag type, timestamp, data size, etc.; NextTagData is the real data, that is, the raw data of NALU of H264 or H265 (that is, remove the start code).
[0103] like Figure 4 As shown, Figure 4 FIG. 4 is a diagram showing specific data of FLV FILE BODY in an embodiment of the present invention. In some embodiments, FLV FILE BODY is composed of a series of back-pointers (such as Figure 4 The back-pointers are the prev_tag_size as mentioned above, which is 4 bytes and is used to indicate the size of the previous tag. The tag consists of the following parts: tag type (such as Figure 4 The part shown in block 5), tag data size (such as Figure 4 The part shown in block 6), TimestampsExtended (such as Figure 4 The part shown in block 1), stream id (such as Figure 4 The part shown in block 2) and tagdata (such as Figure 4The part shown in block 7). Among them, the size of tag type is 1 byte, which is used to identify the tag type. The value is set to character 8 for Audio, the value is set to character 9 for Video, and the value is set to character 18 for scripts. The size of tag data size is 3 bytes, which is used to indicate the length of tag data. Timestamps is timestamp, the size is 3 bytes, TimestampsExtended timestamp extension field, the extension part is 1 byte, plus the original 3 bytes, a total of 4 bytes. Stream id is 3 bytes in total and is always set to 0. In addition, Figure 4 The portion shown in block 3 in the figure represents the front header portion.
[0104] Step S30, forming an FLV FILE HEADER using Signature, Version, Flags and DataOffset;
[0105] like Figure 5 As shown, Figure 5 FIG. 4 is a diagram showing specific data of FLV FILE HEADER in an embodiment of the present invention. In some embodiments, FLV FILE HEADER is composed of Signature (such as Figure 5 The part shown in block 1), Version (such as Figure 5 The part shown in block 2), Flags (such as Figure 5 The part shown in block 3) and DataOffset (as shown in Figure 5 The size of Signature is 3 bytes, and fixed characters are set to mark FLV. When the first three characters of FLV FILEHEADER are identified as FLV, the video data is determined to be an FLV file. The size of Version is 1 byte, and is used to mark the version number of FLV. The size of Flags is 1 byte, and is used for content marking. The 0th and second bits respectively indicate the existence of video and audio (1 indicates existence, 0 indicates non-existence). Figure 5 The corresponding value is 0x05, which is 00000101, indicating that there is both video and audio. The size of DataOffset is 4 bytes, indicating the length of FLV FILE HEADER, which is usually fixed to 9.
[0106] Step S40 , completing data encapsulation of the first video data of the H265 encoding type or the H264 encoding type with the FLV FILE BODY and the corresponding FLV FILE HEADER.
[0107] Based on the above steps, after the FLV FILE HEADER and the corresponding FLV FILE BODY are formed, the first video data to be pushed can be encapsulated using the obtained FLV FILE HEADER and the corresponding FLV FILE BODY to obtain the second video data after encapsulation. The first video data to be pushed can be H265 encoded video data or H264 encoded video data.
[0108] Step S50: Perform a streaming operation on the encapsulated second video data.
[0109] After obtaining the encapsulated second video data, a streaming operation is performed on the second video data. Specifically, the RTMP streaming end can push the encapsulated second video data to the RTMP streaming server, and then the RTMP streaming server pushes the second video data to the playback end for playback.
[0110] The streaming method of the present invention is applied to the RTMP streaming end, and sets the video coding type identifier videocodecid based on the video coding type, wherein the video coding type includes at least H265 coding type or H264 coding type; forms FLV FILE BODY using the set videocodecid and the structure in which PreviousTagSize, NextTagHeader, and NextTagData are arranged in a circular sequence; forms FLV FILEHEADER using Signature, Version, Flags, and DataOffset; completes data encapsulation of the first video data of H265 coding type or H264 coding type with FLV FILE BODY and the corresponding FLV FILE HEADER; and performs streaming operation on the encapsulated second video data. Through the above method, compared with the prior art, the beneficial effect achieved is that it can realize the streaming of video data of the video coding type of H265 using RTMP, and then realize the streaming of high-definition and high-quality video data above 4k, which meets the needs of streaming of higher-definition video data and has a broader application prospect.
[0111] In some embodiments, step S10 includes:
[0112] Step S11, setting the preset position corresponding to the videocodecid or the random position in the preset area corresponding to the videocodecid to the video coding identification character 7 or 12.
[0113] like Figure 6 As shown, Figure 6FIG. 1 is a flow chart of another embodiment of the streaming method of the present invention. In some embodiments, the setting position of the videocodecid can be preset, that is, the preset position, specifically, Figure 4 As shown, the position shown in block 9 can be set as a fixed position for setting videocodecid as a preset position. When the step of setting videocodecid is performed before streaming, the value of the preset position is set to 7 (the video coding type is H264 coding type) or 12 (the video coding type is H265 coding type) based on the video coding type of the video data to be streamed. Then after the subsequent streaming, the receiving end reads the videocodecid character from the preset position again, and determines whether the received video data is H265 video coding data or H264 video coding data based on the videocodecid character read from the preset.
[0114] In some embodiments, a setting area for videocodecid, i.e., a preset area, can be pre-set, wherein the preset area contains multiple positions that can be used to set videocodecid. When performing the step of setting videocodecid before streaming, a random position in the preset area can be determined based on a random algorithm to set the videocodecid, and then a position identifier of the videocodecid is added. Then, after the subsequent streaming, the receiving end reads the videocodecid character from the corresponding position based on the position identifier. Specifically, Figure 4 As shown, the area shown in block 8 can be set as an area for setting videocodecid as a preset area. When setting videocodecid, one or more random positions can be randomly determined from the area shown in block 8 based on a random algorithm to set videocodecid.
[0115] It should be noted that Figure 4 The blocks 8 or 9 shown are only used to illustrate the preset position or preset area, and only serve to explain the preset position or preset area, but not to limit the location of the preset position or preset area, and do not necessarily correspond to the actual videocodecid setting position or area.
[0116] In the above streaming method, the preset position corresponding to the videocodecid or the random position in the preset area corresponding to the videocodecid is set to the video coding identification character 7 or 12, which can avoid the videocodecid from being modified after subsequent streaming, improve the stability of the data, and facilitate streaming identification and reading.
[0117] like Figure 7 As shown, Figure 7This is a flow chart of another embodiment of the streaming method of the present invention. In some embodiments, step S10 includes:
[0118] Step S12, determining the setting location of the videocodecid based on the preset storage path information; wherein the preset storage path information includes the storage level information and / or sequence information of the videocodecid; the setting location includes one or more.
[0119] The preset storage path information refers to the preset path information about the storage location of the videocodecid characters. The preset storage path information includes the storage level information of the videocodecid. The sequence information refers to information such as a specific location or a location number. Specifically, in some embodiments, the preset storage path information is the storage level information of the videocodecid; the storage level information includes at least a first preset storage level and a first preset sub-storage level attached to the first preset storage level. All data storage locations can be divided into multiple parallel first-layer storage levels in advance, and the first preset storage level refers to which storage level among these parallel first-layer storage levels the videocodecid is stored in. Under each first-layer storage level, a second-layer storage level corresponding to the storage level attached to the first layer can be set, that is, a sub-storage level of the first-layer storage level. The first preset storage level refers to which second-layer storage level the videocodecid is stored in. In some embodiments, the first preset sub-storage level may correspond to a specific storage location, and the preset storage path information may be accurate to the level; in some embodiments, the second-level storage level may be further configured with a subordinate subordinate level, and the preset storage path information may also include specific digital information. Figure 4 , block 8 shows a first preset storage level, and block 9 shows a first preset sub-storage level. In some embodiments, step S12 may include:
[0120] Step S121, searching for a first-level position corresponding to the first preset storage level.
[0121] As mentioned above, when setting videocodecid, first find the first level position corresponding to the first preset storage level, refer to Figure 4 The position shown in block 8 is shown.
[0122] Step S122: searching the first sub-level position corresponding to the first preset sub-storage level under the first level position.
[0123] After determining the first level position, that is, the position shown in block 8, the first sub-level position corresponding to the first preset sub-level pair is searched in the first level position, that is, Figure 4 The position shown in block 9 is shown.
[0124] Step S123: determine a setting position of a video coding type identifier videocodecid based on the first sub-level position.
[0125] Based on the above steps, if the first sub-level position can only accommodate one videocodecid setting position, such as Figure 4 Block 9 shown may directly store the videocodecid in the first level location.
[0126] Of course, in some embodiments, the first sub-level position may include space that can accommodate multiple videocodecid setting positions. In this case, one or more setting positions can be determined from the first level position based on a random algorithm to set the videocodecid.
[0127] In the above streaming method, the first level position corresponding to the first preset storage level is searched; the first sub-level position corresponding to the first preset sub-storage level is searched under the first level position; and the setting position of the video coding type identifier videocodecid is determined based on the first sub-level position. In the above manner, the setting position of videocodecid is determined based on at least two storage levels, which can improve the stability of videocodecid and facilitate setting and searching.
[0128] Furthermore, in some embodiments, the storage levels may be divided based on the actual data type of the FLV-encapsulated video data, for example, HEADER and BODY are the first level, and the respective subordinate data types correspond to the second level, and so on.
[0129] Step S13, setting the videocodecid based on the setting position and the video encoding type.
[0130] After determining the setting position based on the above steps, the value of the setting position is set to an identification character corresponding to the video coding type according to the actual video coding type of the video data, thereby completing the setting of videocodecid.
[0131] The above streaming method determines the setting position of the videocodecid based on the preset storage path information; wherein the preset storage path information includes the storage level information and / or the bit sequence information of the videocodecid; the setting position includes one or more; the videocodecid is set based on the setting position and the video encoding type. Through the above method, the storage path information can be set in advance to determine the setting position of the videocodecid, which can improve the stability of the videocodecid and facilitate the setting and searching of the videocodecid.
[0132] In addition, in some embodiments, the preset storage path information is the storage level information of the videocodecid; the storage level information at least includes the second preset storage level to the nth preset storage level arranged in sequence, where n>2.
[0133] Specifically, in some embodiments, the reference start data to the data storing videocodecid can be divided only to the first storage level, and after the first storage level divides the second preset storage level to the nth preset storage level, no subordinate lower storage level is divided or set. The second preset storage level here only serves to distinguish the first preset storage level, and has no order or size effect with the first preset storage level.
[0134] The parallel second preset storage level to the nth preset storage level can be divided based on the data type actually stored, and each data type occupies one preset storage level. The second preset storage level is the starting reference level, and the videocodecid is stored in the nth preset storage level. Of course, the preset storage levels can also be divided in an equal digit manner, so that the second preset storage level to the nth preset storage level are partitions of the same size.
[0135] Step S12 includes:
[0136] Step S124, starting from the second preset storage level, searching and traversing each preset storage level position in sequence until the nth level position corresponding to the nth preset storage level is determined;
[0137] When determining the setting position of videocodecid, first identify the second preset storage level of the starting reference layer, then start from the second preset storage level, search and traverse each preset storage level in turn, count the searched and traversed levels until reaching the nth preset storage level where videocodecid is located, and determine the nth level position corresponding to the nth preset storage level.
[0138] Step S125, determining a setting position of the video coding type identifier videocodecid based on the n-th level position.
[0139] In some embodiments, the size of the n-th level position can be divided to be consistent with the actual size of the setting position of videocodecid. In this case, the n-th level position is directly used as the setting position of videocodecid. In some embodiments, the n-th level position can be divided into a storage area that can accommodate the setting positions of multiple videocodecid. In this case, one or more setting positions for setting videocodecid can be determined based on a random algorithm.
[0140] In the above streaming method, the preset storage path information is the storage level information of the videocodecid; the storage level information at least includes the second preset storage level to the nth preset storage level arranged in sequence, wherein n>2; starting from the second preset storage level, search and traverse the positions of each preset storage level in sequence until the nth level position corresponding to the nth preset storage level is determined; the setting position of the video coding type identifier videocodecid is determined based on the nth level position. In the above manner, only one preset storage level is set, and the setting position of videocodecid is determined according to one preset storage level, which simplifies the data structure and facilitates setting and searching of videocodecid.
[0141] like Figure 8 As shown, Figure 8 FIG. 4 is a flow chart of another embodiment of the streaming method of the present invention.
[0142] In some embodiments, the streaming method of the present invention can be applied to a streaming system, which at least includes an RTMP streaming end, an RTMP streaming server, and an RTMP player. Specifically, the streaming method includes the following steps.
[0143] First, the RTMP streaming end executes step S50, which is specifically performed as follows:
[0144] Step S51: the RTMP streaming end pushes the second video data to the RTMP streaming server.
[0145] Based on the above streaming method, after the RTMP streaming end encapsulates the FLV video data, it obtains the encapsulated video data, i.e., the second video data. The RTMP streaming end then pushes the second video data to the RTMP streaming server based on a communication network, such as a wired communication network or a wireless communication network (such as WiFi or a cellular data network, etc.).
[0146] Step S60, the RTMP streaming server receives the second video data pushed by the RTMP streaming end, and extracts the videocodecid character in the second video data;
[0147] Based on the above streaming method, when the RTMP streaming server receives the second video data pushed by the RTMP streaming end, the videocodecid character is read from the above setting position. If the setting position is determined randomly, the videocodecid character is read after the setting position is determined based on the identification information of the setting position. If there are multiple setting positions, the videocodecid character can be read from one setting position according to the pre-setting or the videocodecid characters of multiple or even all setting positions can be read.
[0148] Step S70: If the videocodecid character in the second video data is 12, the RTMP streaming server determines that the second video data is of H265 encoding type.
[0149] Step S80: If the videocodecid character in the second video data is 7, the RTMP streaming server determines that the second video data is of H264 encoding type.
[0150] After the RTMP stream server extracts the videocodecid character, if the extracted character is 12, it is determined that the second video data is video data of the H265 encoding type; if the extracted character is 7, it is determined that the second video data is video data of the H264 encoding type.
[0151] Step S90: the RTMP streaming server sends the second video data to the RTMP player.
[0152] The RTMP streaming server then sends the second video data to the RTMP player based on the wired communication network or the wireless communication network.
[0153] Step S100: the RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid characters in the second video data.
[0154] When the RTMP player receives the second video data pushed by the RTMP streaming server, the videocodecid character is read from the above setting position. If the setting position is determined randomly, the videocodecid character is read after the setting position is determined based on the identification information of the setting position. If there are multiple setting positions, the videocodecid character can be read from one setting position according to the pre-setting or the videocodecid characters of multiple or even all setting positions can be read.
[0155] Step S110: If the videocodecid character in the second video data is 12, the RTMP player uses the H265 decoder to decode and play the second video data.
[0156] Step S120: If the videocodecid character in the second video data is 7, the RTMP player uses the H264 decoder to decode and play the second video data.
[0157] The RTMP player includes at least two types of decoders, namely, an H265 decoder and an H264 decoder, to respectively decode and play the second video data of the H265 encoding type or the H264 encoding type. Based on the above steps, if the videocodecid character in the second video data read by RTMP is 12, it means that the second video data is video data of the H265 encoding type, and the H265 decoder can be used to decode and play the second video data. If the videocodecid character in the second video data read by RTMP is 7, it means that the second video data is video data of the H264 encoding type, and the H264 decoder can be used to decode and play the second video data.
[0158] The above streaming method is applied in a streaming system including an RTMP streaming end, an RTMP streaming server, and an RTMP player. After streaming, the RTMP player determines the video encoding type based on the videocodecid character and uses the corresponding decoder to decode and play, thereby achieving that video data of different encoding types can be streamed and played in the RTMP streaming system, expanding its applicable scenarios and meeting the needs of high-definition video data streaming.
[0159] In some embodiments, step S100 may include:
[0160] Step S101: the RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid characters in the second video data from a plurality of videocodecid setting positions.
[0161] Based on the above streaming method, in some embodiments, there may be multiple videocodecid setting positions, and videocodecid may be set in multiple setting positions. The videocodecid characters in multiple setting positions of the same FLV video data are the same in principle. When the RTMP player receives the second video data, it may read the corresponding videocodecid characters from the multiple setting positions respectively.
[0162] Step S100 may include:
[0163] Step S130, comparing and verifying the videocodecid characters in a plurality of preset positions.
[0164] Then, the videocodecid characters read from the multiple setting positions are compared and checked to determine whether the multiple videocodecid characters are consistent.
[0165] Step S140: If the comparison verification results are inconsistent, verification result information indicating a verification error is returned to the RTMP streaming server.
[0166] If multiple videocodecid characters are consistent, the corresponding decoder is used for decoding and playback. If the comparison verification results are inconsistent, it means that data changes have occurred during the push process. In this case, in order to avoid decoding failure and waste of decoding time, the verification result information of the verification error can be returned to the RTMP stream server, so that the RTMP stream server or the RTMP push end can re-push the corresponding video data.
[0167] In the above streaming method, the RTMP player receives the second video data sent by the RTMP streaming server, extracts the videocodecid characters in the second video data from multiple videocodecid setting positions; compares and verifies the videocodecid characters in multiple preset positions; if the comparison and verification results are inconsistent, returns the verification result information of the verification error to the RTMP streaming server. Through the above method, the changed data in the streaming process can be identified in time, and after verification, it can be ensured that the wrong decoder will not be used for decoding, avoiding decoding failure.
[0168] In addition, the present invention also provides a flow pushing device.
[0169] The flow-pushing device of the present invention comprises:
[0170] A videocodecid setting module, used to set a video coding type identifier videocodecid based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type;
[0171] The first forming module is used to form FLV FILE BODY by using the set videocodecid and the structure in which PreviousTagSize, NextTagHeader and NextTagData are arranged in a circular sequence;
[0172] The second forming module is used to form FLVFILEHEADER using Signature, Version, Flags and DataOffset;
[0173] A data encapsulation module is used to encapsulate the first video data of the H265 encoding type or the H264 encoding type with the FLVFILEBODY and the corresponding FLV FILE HEADER;
[0174] The streaming module is used to perform streaming operation on the encapsulated second video data.
[0175] The specific implementation of the flow-pushing device of the present invention can refer to the various embodiments of the flow-pushing method of the present invention, and will not be described in detail here.
[0176] In addition, the present invention also provides a streaming terminal.
[0177] The streaming terminal includes a processor and a memory, wherein a streaming program is stored in the memory, and when the streaming program is executed by the processor, the steps of the streaming method described above are implemented.
[0178] Among them, the method implemented when the streaming program running on the processor is executed can refer to the various embodiments of the streaming method of the present invention, and will not be repeated here.
[0179] In addition, an embodiment of the present invention further provides a computer-readable storage medium.
[0180] The computer-readable storage medium of the present invention stores a streaming program, and when the streaming program is executed by a processor, the steps of the streaming method described above are implemented.
[0181] Among them, the method implemented when the streaming program running on the processor is executed can refer to the various embodiments of the streaming method of the present invention, and will not be repeated here.
[0182] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0183] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0185] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A streaming method, characterized in that: The streaming method is applied to the RTMP streaming end, and the streaming method includes: Setting a video coding type identifier videocodecid based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type; The FLV FILE BODY is formed by using the set videocodecid and the structure of PreviousTagSize, NextTagHeader, and NextTagData arranged in a circular sequence; Use Signature, Version, Flags and DataOffset to form FLV FILE HEAD ER; Complete data encapsulation of the first video data of the H265 encoding type or the H264 encoding type with the FLV FILE BODY and the corresponding FLV FILE HEADER; Performing a streaming operation on the encapsulated second video data; The setting of the video coding type identifier videocodecid based on the video coding type includes: Determine the setting position of the videocodecid based on the preset storage path information; wherein the preset storage path information includes the storage level information and / or the bit sequence information of the videocodecid; the setting position includes multiple, so that the videocodecid characters read from the multiple setting positions are subsequently compared and verified; Setting the videocodecid based on the setting position and the video encoding type; The preset storage path information is the storage level information of the videocodecid; the storage level information at least includes the second preset storage level to the nth preset storage level arranged in sequence, wherein n>2; the determining the setting position of the videocodecid based on the preset storage path information includes: Starting from the second preset storage level, searching and traversing each preset storage level position in sequence until the nth level position corresponding to the nth preset storage level is determined; The setting position of the video coding type identifier videocodecid is determined based on the n-th level position.
2. The streaming method according to claim 1, characterized in that: The setting of the video coding type identifier videocodecid based on the video coding type includes: The preset position corresponding to the videocodecid or the random position in the preset area corresponding to the videocodecid is set to the video coding identification character 7 or 12.
3. The streaming method according to claim 1, characterized in that: The preset storage path information is the storage level information of the videocodecid; The storage level information at least includes a first preset storage level and a first preset sub-storage level attached to the first preset storage level; The step of determining the setting location of the videocodecid based on the preset storage path information includes: Finding a first-level location corresponding to the first preset storage level; Searching for a first sub-level position corresponding to the first preset sub-storage level under the first level position; The setting position of the video coding type identifier videocodecid is determined based on the first sub-level position.
4. A streaming method, characterized in that: The streaming method is applied to a streaming system, which at least includes an RTMP streaming end, an RTMP streaming server, and an RTMP player; the streaming method includes: The RTMP streaming end executes the steps of the streaming method according to any one of claims 1 to 3, wherein when the RTMP streaming end executes the step of streaming the encapsulated second video data, the RTMP streaming end pushes the second video data to the RTMP streaming server; The RTMP streaming server receives the second video data pushed by the RTMP streaming end, and extracts the videocodecid character in the second video data; If the videocodecid character in the second video data is 12, the RTMP streaming server determines that the second video data is of H265 encoding type; If the videocodecid character in the second video data is 7, the RTMP streaming server determines that the second video data is of H264 encoding type; The RTMP streaming server sends the second video data to the RTMP player; The RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid character in the second video data; If the videocodecid character in the second video data is 12, the RTMP player decodes and plays the second video data using an H265 decoder; If the videocodecid character in the second video data is 7, the RTMP player uses the H264 decoder to decode and play the second video data.
5. The streaming method according to claim 4, characterized in that: The RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid character in the second video data, including: The RTMP player receives the second video data sent by the RTMP streaming server, and extracts the videocodecid characters in the second video data from a plurality of videocodecid setting positions; The RTMP player receives the second video data sent by the RTMP streaming server, and after the step of extracting the videocodecid character in the second video data, comprises: Compare and verify the videocodecid characters in multiple preset positions; If the comparison verification results are inconsistent, verification result information indicating a verification error is returned to the RTMP stream server.
6. A flow-pushing device, characterized in that: The flow-pushing device comprises: A videocodecid setting module, used to set a video coding type identifier videocodecid based on a video coding type, wherein the video coding type includes at least an H265 coding type or an H264 coding type; The first forming module is used to form FLV FILE BODY by using the set videocodecid and the structure in which PreviousTagSize, NextTagHeader and NextTagData are arranged in a circular sequence; The second forming module is used to form FLV FILEHEADER using Signature, Version, Flags and DataOffset; A data encapsulation module, used for completing data encapsulation of the first video data of the H265 encoding type or the H264 encoding type with the FLV FILE BODY and the corresponding FLV FILE HEADER; A streaming module, used for streaming the encapsulated second video data; The setting of the video coding type identifier videocodecid based on the video coding type includes: Determine the setting position of the videocodecid based on the preset storage path information; wherein the preset storage path information includes the storage level information and / or the bit sequence information of the videocodecid; the setting position includes multiple, so that the videocodecid characters read from the multiple setting positions are subsequently compared and verified; Setting the videocodecid based on the setting position and the video encoding type; The preset storage path information is the storage level information of the videocodecid; the storage level information at least includes the second preset storage level to the nth preset storage level arranged in sequence, wherein n>2; the determining the setting position of the videocodecid based on the preset storage path information includes: Starting from the second preset storage level, searching and traversing each preset storage level position in sequence until the nth level position corresponding to the nth preset storage level is determined; The setting position of the video coding type identifier videocodecid is determined based on the n-th level position.
7. A streaming terminal, characterized in that: The streaming terminal comprises a processor and a memory, wherein a streaming program is stored in the memory, and when the streaming program is executed by the processor, the steps of the streaming method according to any one of claims 1 to 5 are implemented.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a streaming program, and when the streaming program is executed by a processor, the steps of the streaming method according to any one of claims 1 to 5 are implemented.
