Data stream transmission method, pushing method, processing method, device and equipment

Abstract

The invention relates to a data stream transmission method, a data stream pushing method, a data stream processing method, a data stream pushing device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: receiving a data stream sent by a stream pushing end, and extracting file header information of the data stream; identifying the data type of the data stream based on pre-stored protocol information and the file header information; the protocol information is obtained through pre-negotiation and registration of a flow pushing end and a flow pulling end, and is used for indicating a data type; determining a target transmission interface in a transmission channel according to the identified data type, and pushing the data stream to a stream pulling end through the target transmission interface; and the transmission channel comprises at least two transmission interfaces. According to the method, the data streams of different data types are pushed through the same transmission channel, pushing of the data streams of multiple data types between the stream pushing end and the stream pulling end is achieved on the premise that multiple transmission channels do not need to be maintained, the resource utilization rate is greatly increased, and resource waste is avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data stream transmission method, a push method, a processing method, an apparatus, a computer device, a storage medium and a computer program product. Background Art

[0002] With the popularization of the Internet and the increase in bandwidth, streaming media technology has been widely used in many fields. For example, live broadcasting, video conferencing and other fields use streaming media technology to provide real-time video services. In streaming media transmission, video and audio data are transmitted from the push end to the server through a specific protocol (such as the RTMP protocol (Real Time Messaging Protocol)), and then transmitted to the pull end through the server. The RTMP protocol allows data to be transmitted in the form of a stream, so the pull end can start playing the data immediately without having to wait for the entire file to be downloaded.

[0003] During the entire streaming media transmission process, except for transmitting video stream data, the push end and the pull end cannot perform basic information communication, and can only open another information channel to transmit basic information (for example, barrage and other information). Since another data channel is opened, the server, push end and pull end need to maintain two information channels at the same time, which will cause a waste of resources. Summary of the invention

[0004] Based on this, it is necessary to provide a data stream transmission method, push method, processing method, device, computer equipment, storage medium and computer program product that can avoid waste of resources in response to the above technical problems.

[0005] In one aspect, the present application provides a data stream transmission method. The method comprises:

[0006] Receive the data stream sent by the streaming end, and extract the file header information of the data stream;

[0007] Based on the pre-stored protocol information and the file header information, the data type of the data stream is identified; the protocol information is obtained by pre-negotiation and registration between the stream push end and the stream pull end, and is used to indicate the data type;

[0008] The target transmission interface in the transmission channel is determined according to the identified data type, and the data stream is pushed to the stream pulling end through the target transmission interface; the transmission channel includes at least two transmission interfaces.

[0009] In one embodiment, the protocol information includes a plurality of intent identifiers, the intent identifier is used to identify an intended operation, and the identifying the data type of the data stream based on the pre-stored protocol information and the file header information includes:

[0010] Determine the field content of the specified field in the file header information;

[0011] Searching for the field content in multiple intent identifiers of the protocol information;

[0012] If an intent identifier matching the field content is found among the multiple intent identifiers, it is determined that the data type of the data stream is an information type.

[0013] In one embodiment, the method further comprises:

[0014] If an intent identifier that matches the field content is not found among the multiple intent identifiers, it is determined that the data type of the data stream is an audio or video type.

[0015] In one embodiment, the method further comprises:

[0016] If the intent identifier matching the field content indicates that the data stream is an empty data frame, the empty data frame is used as a heartbeat packet sent by the streaming end, and the connection with the streaming end is maintained according to the heartbeat packet.

[0017] In one of the embodiments, determining a target transmission interface in a transmission channel according to the identified data type, and pushing the data stream through the target transmission interface includes:

[0018] If the data type is information type, the first transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the first transmission interface, and the stream pulling end is instructed to perform a corresponding intent operation according to the intent identifier matching the field content;

[0019] If the data type is audio or video, the second transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the second transmission interface, and the stream pulling end is instructed to display the data stream.

[0020] In one embodiment, the method further comprises:

[0021] Generate protocol information according to a plurality of intent identifiers pre-negotiated by the stream push end and the stream pull end;

[0022] The protocol information is stored; wherein the intention identifier is used to identify an intention operation pre-negotiated by the stream push end and the stream pull end.

[0023] On the other hand, the present application provides a data stream pushing method, the method comprising:

[0024] Collect data to be transmitted;

[0025] If the data type of the data to be transmitted is information type, encapsulating the data to be transmitted into a data frame according to a preset network protocol, and in the process of encapsulation, adding at least one intent identifier in a specified field of the file header information of the data frame;

[0026] If the data type of the data to be transmitted is audio or video, directly encapsulating the data to be transmitted into a data frame according to the preset network protocol;

[0027] The encapsulated data frames are pushed to the server as data streams.

[0028] In one embodiment, the method further comprises:

[0029] According to the sending cycle of the heartbeat packet, empty data is generated, and according to the preset network protocol, the empty data is encapsulated into an empty data frame, and in the process of encapsulation, a matching intent identifier is added to a specified field of the file header information of the empty data frame; the matching intent identifier is used to instruct the server to use the empty data frame as a heartbeat packet, and maintain the connection with the streaming end according to the heartbeat packet.

[0030] On the other hand, the present application provides a data stream processing method, the method comprising:

[0031] Receive the data stream pushed by the server through the target transmission interface in the transmission channel;

[0032] Determining the data type of the data stream according to the type of the target transmission interface;

[0033] The data stream is processed according to a processing logic corresponding to the determined data type.

[0034] In one embodiment, determining the data type of the data stream according to the type of the target transmission interface includes:

[0035] If the target transmission interface is the first transmission interface, determining that the data type of the data stream is an information type;

[0036] If the target transmission interface is the second transmission interface, it is determined that the data type of the data stream is an audio or video type.

[0037] In one embodiment, the processing the data stream according to the processing logic corresponding to the determined data type includes:

[0038] If the data type is information type, extracting the intent identifier in the specified field of the file header information of the data stream, and performing the intent operation corresponding to the intent identifier according to the extracted intent identifier;

[0039] If the data type is audio or video, the data stream is displayed.

[0040] On the other hand, the present application provides a data stream transmission device, the device comprising:

[0041] A receiving module, used to receive the data stream sent by the streaming end and extract the file header information of the data stream;

[0042] An identification module, used to identify the data type of the data stream based on the pre-stored protocol information and the file header information; the protocol information is obtained by pre-negotiation and registration between the stream push end and the stream pull end, and is used to indicate the data type;

[0043] The sending module is used to determine the target transmission interface in the transmission channel according to the identified data type, and push the data stream to the stream pulling end through the target transmission interface; the transmission channel includes at least two transmission interfaces.

[0044] On the other hand, the present application provides a data stream pushing device, the device comprising:

[0045] A collection module, used for collecting data to be transmitted;

[0046] A first encapsulation module is used for encapsulating the data to be transmitted into a data frame according to a preset network protocol if the data type of the data to be transmitted is information type, and in the process of encapsulation, adding at least one intent identifier in a specified field of the file header information of the data frame;

[0047] A second encapsulation module is used to directly encapsulate the data to be transmitted into a data frame according to the preset network protocol if the data type of the data to be transmitted is audio or video;

[0048] The streaming module is used to push the encapsulated data frames as data streams to the server.

[0049] On the other hand, the present application provides a data stream processing device, the device comprising:

[0050] The stream pulling module is used to receive the data stream pushed by the server through the target transmission interface in the transmission channel;

[0051] A classification module, used to determine the data type of the data stream according to the type of the target transmission interface;

[0052] The execution module is used to process the data stream according to the processing logic corresponding to the determined data type.

[0053] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned data stream transmission method when executing the computer program.

[0054] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned data stream transmission method are implemented.

[0055] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned data stream transmission method are implemented.

[0056] The above-mentioned data stream transmission method, push method, processing method, device, computer equipment, storage medium and computer program product receive the data stream sent by the push end, and extract the file header information of the data stream; based on the pre-stored protocol information and file header information, identify the data type of the data stream. Among them, the protocol information is obtained by pre-negotiation and registration between the push end and the pull end, and is used to indicate the data type. Then, the target transmission interface can be determined from the multiple transmission interfaces in the transmission channel according to the identified data type, and the data stream is pushed to the pull end through the target transmission interface. In the above process, the data stream pushed by the push end has different data types. Based on the pre-stored protocol information and file header information, the data type of the data stream can be identified. After that, at least two transmission interfaces are encapsulated into a transmission channel, and data streams of different data types are transmitted through different transmission interfaces in the same transmission channel. Data streams of different data types can be pushed through the same transmission channel. Without maintaining multiple transmission channels, the push of data streams of multiple data types between the push end and the pull end is realized, which greatly improves resource utilization and avoids resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 An application environment diagram of a data stream transmission method in an embodiment;

[0058] Figure 2 A schematic diagram of a flow chart of a data stream transmission method in an embodiment;

[0059] Figure 3 A schematic diagram of data stream transmission in a conventional scheme in an embodiment;

[0060] Figure 4 A schematic diagram of data stream transmission in an embodiment;

[0061] Figure 5 is a timing diagram of data stream transmission in one embodiment;

[0062] Figure 6 A schematic diagram of a flow chart of a data stream pushing method in one embodiment;

[0063] Figure 7 A schematic diagram of a flow chart of a data stream processing method in one embodiment;

[0064] Figure 8 is a structural block diagram of a data stream transmission device in one embodiment;

[0065] Fig. 9 It is a structural block diagram of a data stream pushing device in one embodiment;

[0066] Fig.10 is a structural block diagram of a data stream processing device in one embodiment;

[0067] Fig.11 is an internal structure diagram of a computer device in one embodiment;

[0068] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] The data stream transmission method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the push stream end 102 communicates with the server 104 through the network, and the push stream end 102 transmits the collected data stream to the pull stream end 106 through the server 104. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The server 104 receives the data stream sent by the push stream end 102 and extracts the file header information of the data stream; the server 104 identifies the data type of the data stream based on the pre-stored protocol information and file header information; the protocol information is pre-negotiated and registered by the push stream end and the pull stream end, and is used to indicate the data type; the server 104 determines the target transmission interface in the transmission channel according to the identified data type, and pushes the data stream to the pull stream end 106 through the target transmission interface; the transmission channel includes at least two transmission interfaces.

[0071] The streaming end 102 and the streaming end 106 may be, but are not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart TVs, smart car devices, etc. Portable wearable devices may be smart watches, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0072] Before describing in detail the data stream transmission method, data stream push method, and data stream processing method provided by the present application, an example is first given to illustrate the application scenario of the present application:

[0073] In some embodiments, when streaming media technology is applied to a live broadcast platform, the anchor transmits the audio and video stream to the server through the push end, and the audience obtains the audio and video stream from the server through the pull end for viewing. The push end usually needs to use professional live broadcast equipment, such as high-definition cameras, microphones, encoders, etc., to ensure the quality and stability of the live broadcast. The pull end needs to have a stable network connection and sufficient bandwidth to ensure the audience's viewing experience.

[0074] In some embodiments, when streaming media technology is used in video conferencing, the participants transmit the audio and video streams to the server through the push end, and other participants obtain the audio and video streams from the server through the pull end for viewing and communication. Both the push end and the pull end need to have high-definition cameras, microphones, codecs and other equipment to ensure the quality and stability of the conference.

[0075] In some embodiments, when streaming media technology is applied in distance education, teachers transmit teaching videos to the server through the push end, and students obtain teaching videos from the server through the pull end for viewing and learning. The push end needs to have high-definition cameras, microphones, codecs and other equipment to ensure teaching quality and stability. The pull end needs to have a stable network connection and sufficient bandwidth to ensure students' viewing experience.

[0076] In some embodiments, when streaming media technology is used in real-time monitoring, the monitoring device transmits the monitoring video to the server through the push end, and the monitoring center obtains the monitoring video from the server through the pull end for viewing and analysis. The push end needs to have high-definition cameras, codecs and other equipment to ensure monitoring quality and stability. The pull end needs to have a stable network connection and sufficient bandwidth to ensure the viewing experience of the monitoring center.

[0077] In some embodiments, when streaming media technology is applied in a vehicle, the driving recorder can transmit the video stream of the vehicle during driving to the server or cloud storage through the push stream end, thereby realizing remote monitoring and storage. At the same time, the owner or manager can obtain the video stream from the server through the pull stream end for viewing and analysis to understand the driving condition and safety status of the vehicle. The in-vehicle navigation system can transmit navigation information to the in-vehicle display or voice system through the push stream end, thereby realizing in-vehicle navigation services. At the same time, passengers can obtain navigation information from the server through the pull stream end for viewing and listening, so as to more accurately understand the vehicle's location and driving route.

[0078] It should be noted that the above-mentioned application scenarios are only used for illustrative purposes and are not used to limit the application scenarios of the present application. Based on actual business needs, the present application can also be applied in other application scenarios, and the present application document does not limit this.

[0079] In one embodiment, Figure 2 As shown, a data stream transmission method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0080] Step 202: Receive the data stream sent by the streaming end and extract the file header information of the data stream.

[0081] The push end is the data stream acquisition terminal, and the pull end is the data stream receiving terminal. The data stream includes continuous data frames, each of which includes a file header and a file body. The file header carries information such as the file type, data stream format, and timestamp, and the file body carries the actual file content or data, such as text, images, audio or video data, etc. The file header information of the data stream refers to the information carried in the file header of each data frame.

[0082] Specifically, the server establishes a connection with the streaming end. After the connection is established, the server receives the data stream sent by the streaming end through the connection, parses the data stream, and extracts the file header information of each data frame in the data stream.

[0083] Step 204, identifying the data type of the data stream based on the pre-stored protocol information and file header information; the protocol information is obtained through pre-negotiation and registration between the stream push end and the stream pull end, and is used to indicate the data type.

[0084] The pre-stored protocol information refers to the information that the push end and the pull end negotiate in advance and register with the server, which is used to indicate the data type of the data stream. The push end and the pull end negotiate and register the negotiated protocol information with the server, and send the push end other forms of intentions by negotiating the protocol information.

[0085] The data types of data streams include audio and video and information. The audio and video data streams are used to transmit audio and video data, including video frames and audio frames in various compression encoding formats; the information data streams are used to transmit control information and data of various network protocols, such as TCP / IP (Transmission Control Protocol / Internet Protocol) data packets, HTTP (Hypertext Transfer Protocol) request and response messages, etc.

[0086] Specifically, the server matches the extracted file header information with the pre-stored protocol information. If the file header information matches the protocol information successfully, the data type of the data stream is determined to be information type; if the file header information matches the protocol information unsuccessfully, the data type of the data stream is determined to be audio and video type.

[0087] Step 206, determine the target transmission interface in the transmission channel according to the identified data type, and push the data stream to the stream pulling end through the target transmission interface; the transmission channel includes at least two transmission interfaces.

[0088] The transmission channel includes at least two transmission interfaces. In other words, at least two transmission interfaces are encapsulated into one transmission channel, and data streams of different data types are transmitted through different transmission interfaces.

[0089] Figure 3 This is a diagram of data stream transmission in the traditional solution. In the traditional solution, audio and video data streams are pushed through the RTMP protocol, but the sending and receiving of information data streams (bullet comments and other information) are implemented using the WebSocket protocol (a network protocol for two-way communication between a web browser and a server), adding another protocol channel. In the scenario of small data transmission, there are not so many information data streams to be transmitted except for audio and video data streams. Therefore, maintaining another information channel will cause a waste of resources, and both the server and the stream puller maintain two information channels at the same time, which will increase the overall complexity.

[0090] Figure 4 This is a schematic diagram of data stream transmission in an embodiment of the present application. In this embodiment, data streams of different data types are transmitted through different transmission interfaces in the same transmission channel. On the basis of realizing the push of basic information between the push end and the pull end, there is no need to maintain multiple information channels.

[0091] Specifically, the server determines the target transmission interface of the data stream in the transmission channel according to the identified data type, and pushes the data stream to the stream pulling end through the target transmission interface.

[0092] In the above data stream transmission method, the data stream sent by the push stream end is received, and the file header information of the data stream is extracted; based on the pre-stored protocol information and file header information, the data type of the data stream is identified. Among them, the protocol information is obtained by pre-negotiation and registration between the push stream end and the pull stream end, and is used to indicate the data type. Then, the target transmission interface can be determined from the multiple transmission interfaces in the transmission channel according to the identified data type, and the data stream is pushed to the pull stream end through the target transmission interface. In the above process, the data stream pushed by the push stream end has different data types. Based on the pre-stored protocol information and file header information, the data type of the data stream can be identified. After that, at least two transmission interfaces are encapsulated into a transmission channel, and data streams of different data types are transmitted through different transmission interfaces in the same transmission channel. Data streams of different data types can be pushed through the same transmission channel. Without maintaining multiple transmission channels, the push of data streams of multiple data types between the push stream end and the pull stream end is realized, which greatly improves resource utilization and avoids resource waste.

[0093] In one embodiment, identifying the data type of the data stream based on pre-stored protocol information and file header information includes:

[0094] Determine the field content of the specified field in the file header information; search for the field content in multiple intent identifiers of the protocol information; if an intent identifier matching the field content is found in the multiple intent identifiers, determine that the data type of the data stream is an information class.

[0095] In some embodiments, if an intent identifier that matches the field content is not found among multiple intent identifiers, the data type of the data stream is determined to be an audio or video type.

[0096] The file header information includes multiple fields, and different information is added to different fields. For example, the file header information includes field A, field B, and field C, wherein field A adds an intent identifier, field B adds a file type, and field C adds a timestamp. In this embodiment, the specified field is configured to contain field content for matching with the protocol information, that is, the field content in the specified field is matched with the pre-stored protocol information.

[0097] The field contents of the specified fields in the file header information of data streams of different data types are different. For example, the specified fields in the file header information of the audio and video data stream have no field contents, or have field contents that identify the data stream as an audio and video data stream, and the specified fields in the file header information of the information data stream have field contents that match the protocol information.

[0098] The protocol information includes multiple intent identifiers, which are used to identify an intended operation. The intent identifier in the protocol information is the coded data that is pre-negotiated between the push end and the pull end and is used to identify an intended operation. For example, the push end and the pull end agree to use intent identifier 000 to identify the intended operation of adjusting the resolution, and use intent identifier 101 to identify the intended operation of hiding data.

[0099] Intent operations refer to the control and operation of streaming media playback, transmission, source selection and other functions. For example, intent operations include operations such as play, pause, stop, jump, etc., thereby realizing the control of streaming media playback. For example, intent operations include operations such as selecting different video sources, audio sources or streaming media services, thereby realizing the selection of streaming media sources. For example, intent operations include operations such as adjusting transmission speed, transmission quality, and transmission protocol, thereby realizing the control of streaming media transmission. For example, through intent operations such as sharing streaming media content with social media applications and linking with map applications to display geographic locations, the interaction of streaming media with other applications can be realized.

[0100] Specifically, the server extracts the field content of the specified field in the file header information of the data stream, searches for the field content in multiple intent identifiers of the protocol information, and if an intent identifier that matches the field content is found among the multiple intent identifiers, the data type of the data stream is determined to be an information type; if an intent identifier that matches the field content is not found among the multiple intent identifiers, the data type of the data stream is determined to be an audio or video type.

[0101] For example, an intent identifier in the protocol information is "101,101,101", file header information 1 of data frame 1 is private byte[] spspps = {0, 0, 0, 1,}, and file header information 2 of data frame 2 is public staticbyte[] temporaryIFrame = new byte[]{101, 101, 101, 0, 0, 0, 1}. It can be seen that there is no field content in the header field of file header information 1, and there is field content in the header field of file header information 2 that matches the intent identifier "101,101,101". Therefore, it is determined that the data type of data frame 1 is audio and video type, and the data type of data frame 2 is information type.

[0102] In this embodiment, the field content of the specified field in the file header information is determined, and the field content is searched in multiple intent identifiers of the protocol information. If an intent identifier matching the field content is found in the multiple intent identifiers, the data type of the data stream is determined to be information type. In the above process, the field content corresponding to an intent identifier in the protocol information is added to the specified field of the file header information of the data stream, so that the data stream can be identified as an information type data stream, which is convenient for quickly determining the data type of the data stream according to the field content of the specified field of the file header information, and the type of intent identifier in the protocol information can be pre-negotiated, and multiple types of information type data streams can be transmitted without adding any hardware, which expands the application scenario.

[0103] In one embodiment, in the traditional solution, when the streaming end pauses streaming, the server will mistakenly think that streaming has stopped and actively disconnect the connection between the streaming end and the server, causing the streaming end to be unable to continue streaming. In order to solve the above problem, the corresponding solution in the traditional solution is that the server sets a connection duration parameter to extend the survival time of the connection. However, the connection duration parameter set in this way may not be effective in some cases, and there is instability and the problem of being unable to guarantee the long-term survival of the connection. Therefore, in order to ensure the long-term survival of the connection between the streaming end and the server, in this embodiment, the data stream transmission method also includes:

[0104] If the intent identifier that matches the field content indicates that the data stream is an empty data frame, the empty data frame is used as a heartbeat packet sent by the streaming end, and the connection with the streaming end is maintained based on the heartbeat packet.

[0105] Among them, an empty data frame refers to a data frame that does not contain a valid load. The empty data frame is an empty data generated by the push end according to the sending cycle of the heartbeat packet, and the data frame is encapsulated by a preset network protocol. In this embodiment, the empty data frame is identified by the intention identifier pre-negotiated by the push end and the pull end. The intention identifier corresponding to the empty data frame is used to instruct the server to use the empty data frame as a heartbeat packet and maintain the connection with the push end according to the heartbeat packet.

[0106] Specifically, the server extracts the field content of the specified field in the file header information of the data frame. If the field content matches an intent identifier of the protocol information, and the intent identifier is used to indicate that the data stream is an empty data frame, the server uses the empty data frame as a heartbeat packet sent by the streaming end, and maintains the connection with the streaming end based on the heartbeat packet.

[0107] In this embodiment, an empty data frame replaces the heartbeat packet. Therefore, the streaming end does not need to send additional heartbeat packets or set connection duration parameters, thereby ensuring the long-term survival of the connection between the streaming end and the server.

[0108] In one embodiment, when the push end pushes the audio and video data stream, each audio and video frame carries a timestamp, and the pull end displays the audio and video according to the timestamp of the audio and video frame. Based on this scenario, if the information data stream and the audio and video data stream are transmitted to the pull end through the same transmission channel, then the timestamp corresponding to the information data stream will disturb the timestamp of the audio and video frame, resulting in a distorted screen on the audio and video displayed by the pull end. For example, the push end continuously sends the audio and video data stream and the information data stream, and the pull end obtains the audio and video data stream and displays the audio and video normally. After that, the pull end obtains the information data stream and needs to parse the information data stream and perform the corresponding intention operation. In this process, the pull end pauses the display of audio and video, receives the next audio and video data stream, and then displays the audio and video normally. During the period when the pull end pauses the display of audio and video, the display screen of the pull end will have a distorted screen. Therefore, in order to solve the above problem, the embodiment of the present application encapsulates at least two transmission interfaces into a transmission channel, and transmits data streams of different data types through different transmission interfaces, thereby solving the problem that the same transmission channel transmits data streams of different data types, resulting in a distorted screen on the stream pulling end. Specifically, the target transmission interface in the transmission channel is determined according to the identified data type, and the data stream is pushed through the target transmission interface, including:

[0109] If the data type is information type, the first transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the first transmission interface, and the stream pulling end is instructed to perform the corresponding intention operation according to the intention identifier that matches the field content; if the data type is audio and video type, the second transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the second transmission interface, and the stream pulling end is instructed to display the data stream.

[0110] The first transmission interface and the second transmission interface are encapsulated into a transmission channel, and the information data stream is transmitted through the first transmission interface, and the audio and video data stream is transmitted through the second transmission interface. Since the data streams transmitted by the second transmission interface are all audio and video data streams, and are not mixed with information data streams, the stream pulling end can display continuous audio and video without screen distortion.

[0111] Specifically, after the server identifies the data type of the data stream, if the identified data type is information type, the first transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the first transmission interface, and the stream pulling end is instructed to perform the corresponding intention operation according to the intention identifier that matches the field content; if the identified data type is audio and video type, the second transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the second transmission interface, and the stream pulling end is instructed to display the data stream.

[0112] In this embodiment, at least two transmission interfaces are encapsulated into a transmission channel, and data streams of different data types are transmitted through different transmission interfaces, ensuring that the streaming end receives a continuous data stream without mixed information types, so that the streaming end can display continuous audio and video, thereby solving the problem of screen distortion at the streaming end caused by transmitting data streams of different data types through the same transmission channel.

[0113] In one embodiment, the data stream transmission method further includes:

[0114] Generate protocol information according to multiple intent identifiers pre-negotiated by the streaming end and the streaming end; store the protocol information; wherein the intent identifier is used to identify an intent operation pre-negotiated by the streaming end and the streaming end.

[0115] The push end and the pull end can define the intent identifier corresponding to the intent operation according to the intent operation required by the current scenario, and generate protocol information according to the defined intent identifier and its corresponding intent operation. The push end, the pull end and the server all store the protocol information.

[0116] Figure 5 is a timing diagram of data stream transmission in one embodiment, such as Figure 5 As shown, when the push end sends an information-type data stream, it first finds the intent identifier corresponding to the required intent operation according to the protocol information, adds the intent identifier to the specified field in the file header information of the information-type data stream, and pushes it to the server. If the server recognizes that the data type of the data stream is information-type, it transmits it to the pull end through the first transmission interface of the transmission channel. The pull end determines the intent identifier corresponding to the field content of the specified field in the data stream and the intent operation corresponding to the intent identifier according to the protocol information, and executes the intent operation.

[0117] Specifically, the stream pulling end and the stream pushing end define the intent identifiers corresponding to different intent operations. The server generates protocol information based on multiple pre-negotiated intent identifiers, and stores the protocol information in the module in the server that processes the requests from the stream pulling end and the stream pushing end.

[0118] In this embodiment, the server identifies and forwards the data stream according to a predefined protocol file. During this process, the push end and the pull end can flexibly define the protocol file by pre-negotiating the intent identifier, so that the server can process the data stream or transmit other data information. Without adding any hardware, data streams of various information types can be transmitted, expanding the application scenarios.

[0119] In one embodiment, Figure 6 As shown, a data stream push method is provided, which is applied to Figure 1The streaming end in the example specifically includes the following steps:

[0120] Step 602: Collect data to be transmitted.

[0121] The data to be transmitted includes audio and video data, information data, etc. Audio and video data are collected through cameras and microphones. Information data includes control instructions and status data. Control instructions can be collected through input devices (keyboards, buttons, switches, etc.) and network transmission, and status data can be collected through sensors.

[0122] Step 604: If the data type of the data to be transmitted is information type, encapsulate the data to be transmitted into a data frame according to the preset network protocol, and during the encapsulation process, add at least one intent identifier in a specified field of the file header information of the data frame.

[0123] The preset network protocol refers to the protocol used in streaming media technology, such as the RTMP protocol. The file header information refers to the information carried in the file header of each data frame. The data to be transmitted of the information type is used to transmit control information and data of various network protocols, such as data packets of the TCP / IP protocol, request and response messages of the HTTP protocol, etc.

[0124] In some embodiments, if the data to be transmitted is a control instruction, an intent identifier corresponding to the control instruction is added to a specified field of the file header information of the data to be transmitted; if the data to be transmitted is status data, an intent identifier is added to a specified field of the file header information of the data to be transmitted, and the intent identifier is used to mark the type of the status data and the specific content of the status data.

[0125] Specifically, if the data type of the data to be transmitted is information type, the streaming end determines the intent identifier corresponding to the data to be transmitted, encapsulates the data to be transmitted into a data frame according to the preset network protocol, and during the encapsulation process, adds the corresponding intent identifier in the specified field of the file header information of the data frame.

[0126] Step 606: If the data type of the data to be transmitted is audio or video, the data to be transmitted is directly encapsulated into a data frame according to a preset network protocol.

[0127] Among them, the audio and video data stream is used to transmit audio and video data, including video frames and audio frames in various compression encoding formats.

[0128] Step 608: Push the encapsulated data frames as data streams to the server.

[0129] In this embodiment, if the data type of the data to be transmitted is information type, the data to be transmitted is encapsulated into data frames according to the preset network protocol, and during the encapsulation process, at least one intent identifier is added to the designated field of the file header information of the data to be transmitted; if the data type of the data to be transmitted is audio and video type, the data to be transmitted is directly encapsulated into data frames according to the preset network protocol. Through the above encapsulation method, the file header information of the information type data stream is different from the file header information of the audio and video type data stream, and the audio and video type data stream and the information type data stream are transmitted to the server using the same transmission channel, so that the server can quickly identify the data type of the data stream according to the file header information.

[0130] In one embodiment, in a traditional solution, because the streaming end suspends streaming, the server mistakenly believes that streaming has stopped and actively disconnects the connection between the streaming end and the server, and the streaming end cannot continue streaming. In order to solve the above problem, the corresponding solution in the traditional solution is that the server sets a connection duration parameter to extend the survival time of the connection. However, the connection duration parameter set in this way may not be effective in some cases, and there is instability and the problem of being unable to guarantee the long-term survival of the connection. Therefore, in order to ensure the long-term survival of the connection between the streaming end and the server, in this embodiment, the data stream transmission method also includes:

[0131] According to the sending cycle of the heartbeat packet, empty data is generated, and according to the preset network protocol, the empty data is encapsulated into an empty data frame, and in the process of encapsulation, a matching intent identifier is added to the specified field of the file header information of the empty data frame; the matching intent identifier is used to instruct the server to use the empty data frame as a heartbeat packet, and maintain the connection with the streaming end based on the heartbeat packet.

[0132] Here, empty data refers to data containing any valid information.

[0133] Specifically, the streaming end pre-sets an empty data frame obtained after encapsulating empty data to replace the heartbeat packet, and instructs the server to use the empty data frame as the heartbeat packet sent by the streaming end, maintains the connection with the streaming end according to the heartbeat packet, and the streaming end presets the same intent identifier for the empty data according to the function of the empty data. The streaming end generates empty data according to the sending cycle of the heartbeat packet, and encapsulates the empty data into an empty data frame according to the preset network protocol, and in the process of encapsulation, adds the corresponding intent identifier in the designated field of the file header information of the empty data.

[0134] In this embodiment, an empty data frame replaces the heartbeat packet, and the empty data frame is generated by the streaming end according to the sending cycle of the heartbeat packet. Therefore, the streaming end does not need additional heartbeat packets or setting connection duration parameters to ensure the long-term survival of the connection between the streaming end and the server.

[0135] In one embodiment, Figure 7 As shown, a data stream transmission method is provided, which is applied to Figure 1 The pull end in the pipeline includes:

[0136] Step 702: receiving a data stream pushed by a server through a target transmission interface in a transmission channel.

[0137] The transmission channel includes at least two transmission interfaces, and different transmission interfaces are used to transmit data streams of different data types.

[0138] Step 704: determine the data type of the data stream according to the type of the target transmission interface.

[0139] Step 706: Process the data stream according to the processing logic corresponding to the determined data type.

[0140] Among them, the stream pulling end has different processing logic corresponding to data streams of different data types. For example, if the data type of the data stream is audio and video, the audio and video data carried by the data stream is directly displayed; if the data type of the data stream is information, the same intention operation is performed according to the intention identifier carried by the information data stream.

[0141] In this embodiment, data streams of different data types are pushed through the same transmission channel. Without maintaining multiple transmission channels, the push of data streams of multiple data types between the push end and the pull end is achieved, which greatly improves resource utilization, avoids resource waste, and ensures that the pull end receives a continuous and unmixed information data stream, so that the pull end can display continuous audio and video, thereby solving the problem of screen distortion at the pull end caused by the transmission of data streams of different data types through the same transmission channel.

[0142] In one embodiment, determining the data type of the data stream according to the type of the target transmission interface includes:

[0143] If the target transmission interface is the first transmission interface, the data type of the data stream is determined to be an information type; if the target transmission interface is the second transmission interface, the data type of the data stream is determined to be an audio and video type.

[0144] In one embodiment, processing the data stream according to the processing logic corresponding to the determined data type includes:

[0145] If the data type is information type, the intent identifier in the specified field in the file header information of the data stream is extracted, and the intent operation corresponding to the intent identifier is performed according to the extracted intent identifier; if the data type is audio and video type, the data stream is displayed.

[0146] In this embodiment, the server identifies the data type of the data stream based on pre-stored protocol information and file header information, and after transmitting the data stream through a target transmission interface that matches the data type, the stream pulling end can know the data type of the data stream based on the type of the target transmission interface. The stream pulling end does not need to make a judgment again, thereby improving the data processing efficiency of the stream pulling end.

[0147] In a detailed embodiment, the data stream transmission method includes the following steps:

[0148] 1. The streaming end collects the data to be transmitted.

[0149] 2. If the data type of the data to be transmitted is information type, the streaming end executes step 3; if the data type of the data to be transmitted is audio or video type, the streaming end executes step 4.

[0150] 3. The streaming end encapsulates the data to be transmitted into a data frame according to the preset network protocol, and during the encapsulation process, adds at least one intent identifier to a specified field of the file header information of the data frame, and executes step 5.

[0151] 4. The streaming end directly encapsulates the data to be transmitted into data frames according to the preset network protocol and executes step 5.

[0152] 5. The streaming end generates empty data according to the sending cycle of the heartbeat packet, and encapsulates the empty data into an empty data frame according to the preset network protocol. During the encapsulation process, a matching intent identifier is added to the specified field of the file header information of the empty data frame; the matching intent identifier is used to instruct the server to use the empty data frame as a heartbeat packet and maintain the connection with the streaming end based on the heartbeat packet.

[0153] 6. The streaming end pushes the encapsulated data frames to the server as a data stream.

[0154] 7. The server receives the data stream sent by the streaming end and extracts the file header information of the data stream.

[0155] 8. The server determines the field content of the specified field in the file header information.

[0156] 9. The server searches for the field content in multiple intent identifiers of the protocol information; if an intent identifier matching the field content is found in the multiple intent identifiers, the data type of the data stream is determined to be an information type, and step ten is executed; if an intent identifier matching the field content is not found in the multiple intent identifiers, the data type of the data stream is determined to be an audio and video type, and step thirteen is executed;

[0157] The protocol information is obtained through pre-negotiation and registration between the stream push end and the stream pull end, and is used to indicate the data type; the protocol information includes multiple intent identifiers, and the intent identifier is used to identify an intended operation.

[0158] 10. If the intent identifier that matches the field content indicates that the data stream is an empty data frame, execute step 11; if the intent identifier that matches the field content does not indicate that the data stream is an empty data frame, execute step 12.

[0159] 11. The server uses the empty data frame as the heartbeat packet sent by the streaming end, and maintains the connection with the streaming end based on the heartbeat packet.

[0160] 12. The server uses the first transmission interface in the transmission channel as the target transmission interface, pushes the data stream to the stream pulling end through the first transmission interface, and instructs the stream pulling end to perform the corresponding intention operation according to the intention identifier matching the field content, and executes step 14.

[0161] 13. The server uses the second transmission interface in the transmission channel as the target transmission interface, pushes the data stream to the stream pulling end through the second transmission interface, and instructs the stream pulling end to display the data stream, and executes step 14.

[0162] 14. The stream pulling end receives the data stream pushed by the server through the target transmission interface in the transmission channel.

[0163] 15. If the target transmission interface is the first transmission interface, the stream pulling end determines that the data type of the data stream is information type, extracts the field content of the specified field in the file header information of the data stream, and performs corresponding intention operations according to the field content.

[0164] 16. If the target transmission interface is the second transmission interface, the stream pulling end determines that the data type of the data stream is an audio or video type, and displays the data stream.

[0165] In this embodiment, by adding field content corresponding to an intent identifier in the protocol information to the specified field of the file header information of the data stream, the data stream can be identified as an information-type data stream, which is convenient for quickly determining the data type of the data stream according to the field content of the specified field of the file header information, and the type of intent identifier in the protocol information can be negotiated in advance. Without adding any hardware, various information-type data streams can be transmitted, thereby expanding the application scenarios; at least two transmission interfaces are encapsulated into a transmission channel, and data streams of different data types are transmitted through different transmission interfaces in the same transmission channel, so that data streams of different data types can be pushed through the same transmission channel. Without maintaining multiple transmission channels, the push of data streams of multiple data types between the push end and the pull end is realized, which greatly improves resource utilization and avoids resource waste.

[0166] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0167] Based on the same inventive concept, the embodiment of the present application also provides a data stream transmission device for implementing the data stream transmission method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more data stream transmission device embodiments provided below can refer to the limitations on the data stream transmission method above, and will not be repeated here.

[0168] In one embodiment, Figure 8 As shown, a data stream transmission device is provided, comprising:

[0169] The receiving module 801 is used to receive the data stream sent by the streaming end and extract the file header information of the data stream.

[0170] The identification module 802 is used to identify the data type of the data stream based on the pre-stored protocol information and file header information; the protocol information is obtained through pre-negotiation and registration between the stream push end and the stream pull end, and is used to indicate the data type.

[0171] The sending module 803 is used to determine the target transmission interface in the transmission channel according to the identified data type, and push the data stream to the stream pulling end through the target transmission interface; the transmission channel includes at least two transmission interfaces.

[0172] In one embodiment, the protocol information includes multiple intent identifiers, and the intent identifier is used to identify an intended operation. The identification module 802 is also used to determine the field content of a specified field in the file header information; search for the field content in the multiple intent identifiers of the protocol information; if an intent identifier that matches the field content is found in the multiple intent identifiers, the data type of the data stream is determined to be an information class.

[0173] In one embodiment, the identification module 802 is further configured to determine that the data type of the data stream is an audio or video type if no intent identifier matching the field content is found among the multiple intent identifiers.

[0174] In one embodiment, the identification module 802 is further used to use the empty data frame as a heartbeat packet sent by the streaming end if the intent identifier matching the field content indicates that the data stream is an empty data frame, and maintain the connection with the streaming end based on the heartbeat packet.

[0175] In one embodiment, the sending module 803 is also used to, if the data type is information type, use the first transmission interface in the transmission channel as the target transmission interface, push the data stream to the stream pulling end through the first transmission interface, and instruct the stream pulling end to perform the corresponding intention operation according to the intention identifier that matches the field content; if the data type is audio and video type, use the second transmission interface in the transmission channel as the target transmission interface, push the data stream to the stream pulling end through the second transmission interface, and instruct the stream pulling end to display the data stream.

[0176] In one embodiment, the device also includes a protocol module, which is used to generate protocol information based on multiple intention identifiers pre-negotiated by the push end and the pull end; store the protocol information; wherein the intention identifier is used to identify an intention operation pre-negotiated by the push end and the pull end.

[0177] In one embodiment, Fig. 9 As shown, a data stream pushing device is provided, comprising:

[0178] The acquisition module 901 is used to acquire data to be transmitted.

[0179] The first encapsulation module 902 is used to encapsulate the data to be transmitted into a data frame according to a preset network protocol if the data type of the data to be transmitted is information type, and in the process of encapsulation, add at least one intent identifier in a specified field of the file header information of the data frame.

[0180] The second encapsulation module 903 is used to directly encapsulate the data to be transmitted into a data frame according to a preset network protocol if the data type of the data to be transmitted is audio or video.

[0181] The streaming module 904 is used to push the encapsulated data frames as data streams to the server.

[0182] In one embodiment, the first encapsulation module 902 is used to generate empty data according to the sending cycle of the heartbeat packet, and encapsulate the empty data into an empty data frame according to a preset network protocol, and in the process of encapsulation, add a matching intent identifier in a designated field of the file header information of the empty data frame; the matching intent identifier is used to instruct the server to use the empty data frame as a heartbeat packet and maintain the connection with the streaming end based on the heartbeat packet.

[0183] In one embodiment, Fig.10 As shown, a data stream processing device is provided, comprising:

[0184] The stream pulling module 1001 is used to receive the data stream pushed by the server through the target transmission interface in the transmission channel.

[0185] The classification module 1002 is used to determine the data type of the data flow according to the type of the target transmission interface.

[0186] The execution module 1003 is used to process the data stream according to the processing logic corresponding to the determined data type.

[0187] In one embodiment, the classification module 1002 is further used to determine the data type of the data stream as information if the target transmission interface is the first transmission interface; and to determine the data type of the data stream as audio and video if the target transmission interface is the second transmission interface.

[0188] In one embodiment, the execution module 1003 is further used to extract the field content of the specified field in the file header information of the data stream if the data type is information type, and perform a corresponding intention operation according to the field content;

[0189] If the data type is audio or video, the data stream is displayed.

[0190] Each module in the above data stream transmission device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0191] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the protocol information pre-negotiated and registered by the push stream end and the pull stream end. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data stream transmission method is implemented.

[0192] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data stream transmission method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0193] Those skilled in the art will understand that Fig.11 and Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0194] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0195] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0196] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0197] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0198] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0199] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data stream transmission method, characterized in that: The method comprises: Receive the data stream sent by the streaming end, and extract the file header information of the data stream; Based on the pre-stored protocol information and the file header information, the data type of the data stream is identified; the protocol information is obtained by pre-negotiation and registration between the stream push end and the stream pull end, and is used to indicate the data type; The target transmission interface in the transmission channel is determined according to the identified data type, and the data stream is pushed to the stream pulling end through the target transmission interface; the transmission channel includes at least two transmission interfaces.

2. The method according to claim 1, characterized in that The protocol information includes a plurality of intent identifiers, and the intent identifier is used to identify an intended operation. The identifying the data type of the data stream based on the pre-stored protocol information and the file header information includes: Determine the field content of the specified field in the file header information; Searching for the field content in multiple intent identifiers of the protocol information; If an intent identifier matching the field content is found among the multiple intent identifiers, it is determined that the data type of the data stream is an information type.

3. The method according to claim 2, characterized in that The method further comprises: If an intent identifier that matches the field content is not found among the multiple intent identifiers, it is determined that the data type of the data stream is an audio or video type.

4. The method according to claim 2, characterized in that: The method further comprises: If the intent identifier matching the field content indicates that the data stream is an empty data frame, the empty data frame is used as a heartbeat packet sent by the streaming end, and the connection with the streaming end is maintained according to the heartbeat packet.

5. The method according to claim 2, characterized in that: The step of determining a target transmission interface in a transmission channel according to the identified data type, and pushing the data stream through the target transmission interface, comprises: If the data type is information type, the first transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the first transmission interface, and the stream pulling end is instructed to perform a corresponding intention operation according to the intention identifier matching the field content; If the data type is audio or video, the second transmission interface in the transmission channel is used as the target transmission interface, the data stream is pushed to the stream pulling end through the second transmission interface, and the stream pulling end is instructed to display the data stream.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Generate protocol information according to multiple intention identifiers pre-negotiated by the stream push end and the stream pull end; The protocol information is stored; wherein the intention identifier is used to identify an intention operation pre-negotiated by the stream push end and the stream pull end.

7. A data stream pushing method, characterized in that: The method comprises: Collect data to be transmitted; If the data type of the data to be transmitted is information type, encapsulating the data to be transmitted into a data frame according to a preset network protocol, and in the process of encapsulation, adding at least one intent identifier in a specified field of the file header information of the data frame; If the data type of the data to be transmitted is audio or video, directly encapsulating the data to be transmitted into a data frame according to the preset network protocol; The encapsulated data frames are pushed to the server as data streams.

8. The method according to claim 7, characterized in that The method further comprises: According to the sending cycle of the heartbeat packet, empty data is generated, and according to the preset network protocol, the empty data is encapsulated into an empty data frame, and in the process of encapsulation, a matching intent identifier is added to a specified field of the file header information of the empty data frame; the matching intent identifier is used to instruct the server to use the empty data frame as a heartbeat packet, and maintain the connection with the streaming end according to the heartbeat packet.

9. A data stream processing method, characterized in that: The method comprises: Receive the data stream pushed by the server through the target transmission interface in the transmission channel; Determining the data type of the data stream according to the type of the target transmission interface; The data stream is processed according to a processing logic corresponding to the determined data type.

10. The method according to claim 9, characterized in that The determining the data type of the data stream according to the type of the target transmission interface includes: If the target transmission interface is the first transmission interface, determining that the data type of the data stream is an information type; If the target transmission interface is the second transmission interface, it is determined that the data type of the data stream is an audio or video type.

11. The method according to claim 10, characterized in that The processing of the data stream according to the processing logic corresponding to the determined data type includes: If the data type is information type, extracting the intent identifier in the specified field of the file header information of the data stream, and performing the intent operation corresponding to the intent identifier according to the extracted intent identifier; If the data type is audio or video, the data stream is displayed.

12. A data stream transmission device, characterized in that: The device comprises: A receiving module, used to receive the data stream sent by the streaming end and extract the file header information of the data stream; An identification module, used to identify the data type of the data stream based on the pre-stored protocol information and the file header information; the protocol information is obtained by pre-negotiation and registration between the stream push end and the stream pull end, and is used to indicate the data type; The sending module is used to determine the target transmission interface in the transmission channel according to the identified data type, and push the data stream to the stream pulling end through the target transmission interface; the transmission channel includes at least two transmission interfaces.

13. A data stream pushing device, characterized in that: The device comprises: A collection module, used for collecting data to be transmitted; A first encapsulation module is used for encapsulating the data to be transmitted into a data frame according to a preset network protocol if the data type of the data to be transmitted is information type, and in the process of encapsulation, adding at least one intent identifier in a specified field of the file header information of the data frame; A second encapsulation module is used to directly encapsulate the data to be transmitted into a data frame according to the preset network protocol if the data type of the data to be transmitted is audio or video; The streaming module is used to push the encapsulated data frames as data streams to the server.

14. A data stream processing device, characterized in that: The device comprises: The stream pulling module is used to receive the data stream pushed by the server through the target transmission interface in the transmission channel; A classification module, used to determine the data type of the data stream according to the type of the target transmission interface; The execution module is used to process the data stream according to the processing logic corresponding to the determined data type.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images