Data transmission method, device, equipment and medium based on visual network
By negotiating and switching packet loss error correction mode in the video network environment, the packet loss problem in audio and video data transmission in the video network is solved, the integrity and transmission efficiency of data packets are improved, and higher quality audio and video services are achieved.
Patent Information
- Application Number
- CN202010839942.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-19
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-08-19
AI Technical Summary
In a video network environment, packet loss often occurs during audio and video data transmission, resulting in stuttering of video images and audio loss, and cannot guarantee high-quality real-time services.
By negotiating the switching of packet loss error correction mode between the sending end and the receiving end, the mode switching command is generated based on the packet loss rate and the preset packet loss rate threshold of the audio and video data, and dynamically adjust the packet loss error correction mode, thereby in different network environments Adapt to different response mechanisms to improve packet integrity and transmission efficiency.
It effectively reduces the packet loss rate of audio and video data, improves the smoothness and service quality of audio and video, avoids network congestion caused by frequent retransmissions, and improves the flexibility and efficiency of data transmission.
Smart Images

Figure CN112165655B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of network communication technology, and in particular to a data transmission method, device, equipment and medium based on visual networking. Background Art
[0002] The visual network adopts the world's most advanced VisionVera real-time HD video exchange technology, realizing real-time transmission of HD video across the entire network that is currently impossible with the Internet. It integrates dozens of video, voice, image, text, communication, data and other services such as HD video conferencing, video surveillance, remote training, intelligent monitoring and analysis, emergency command, video calls, live broadcasts, TV mails, information releases, etc. into one system platform, and realizes real-time interconnection of HD quality video communications through a variety of terminal devices.
[0003] In the related art, when transmitting data in a visual network, packet loss is inevitable. Especially for the transmission of audio and video data, if packet loss occurs, it will cause the video to freeze and the audio to be lost. Summary of the invention
[0004] In view of the above problems, an embodiment of the present invention provides a data transmission method, device, electronic device and storage medium based on a visual network, aiming to reduce the audio and video data packet loss rate in a visual network environment and improve the fluency of audio and video.
[0005] A first aspect of an embodiment of the present invention discloses a data transmission method based on a visual network, the method being applied to a sending end and comprising:
[0006] Send audio and video data to the receiving end according to the current packet loss error correction mode;
[0007] In response to a mode switching instruction sent by the receiving end, switching the current packet loss error correction mode, wherein the mode switching instruction is generated by the receiving end according to a size relationship between a packet loss rate when receiving audio and video data and a preset packet loss rate threshold;
[0008] Sending audio and video data to the receiving end according to the switched packet loss error correction mode;
[0009] Among them, the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs.
[0010] Optionally, sending the audio and video data to the receiving end according to the switched packet loss error correction mode includes:
[0011] According to the visual networking protocol, the audio and video data to be sent are encapsulated into multiple audio and video data packets;
[0012] Adding an error correction protocol frame to the frame header data of each of the plurality of audio and video data packets to obtain a plurality of first audio and video data packets; the error correction protocol frame includes an error correction type frame header and a data transmission frame header, the error correction type frame header is used to characterize the currently adopted packet loss correction mode, and the data transmission frame header is used to characterize the attributes of the currently transmitted audio and video data packets;
[0013] According to the switched packet loss error correction mode, the error correction type frame header and the data transmission frame header included in each of the plurality of first audio and video data packets are set to obtain a plurality of second audio and video data packets;
[0014] The multiple second audio and video data packets are sent to a receiving end.
[0015] Optionally, according to the switched packet loss error correction mode, the error correction type frame header and the data transmission frame header included in each of the plurality of first audio and video data packets are set to obtain a plurality of second audio and video data packets, including:
[0016] When the packet loss correction mode after the switch is the first packet loss correction mode, determining the total length, the total number and the length of each audio and video data packet of the plurality of first audio and video data packets;
[0017] The first packet loss correction mode is written into the error correction type frame header of each of the multiple first audio and video data packets, and the total length, the total number and the length of each audio and video data packet are written into the data transmission frame header of each of the multiple first audio and video data packets to obtain the multiple second audio and video data packets.
[0018] Optionally, according to the switched packet loss error correction mode, the error correction type frame header and the data transmission frame header included in each of the plurality of first audio and video data packets are set to obtain a plurality of second audio and video data packets, including:
[0019] When the packet loss correction mode after switching is the second packet loss correction mode, the error correction type frame header and the data transmission frame header of each of the multiple first audio and video data packets are set according to the default protocol header corresponding to the second packet loss correction mode to obtain the multiple second audio and video data packets.
[0020] Optionally, the error correction type frame header includes a coding type, and the method further includes:
[0021] When the packet loss correction mode after the switch is the second packet loss correction mode, encoding the audio and video data in the multiple first audio and video data packets according to the currently enabled encoding protocol to obtain multiple first video data packets to be encapsulated;
[0022] According to the default protocol header corresponding to the second packet loss error correction mode, setting the error correction type frame header and the data transmission frame header of each of the plurality of first audio and video data packets includes:
[0023] According to the default protocol header corresponding to the second packet loss correction mode, the error correction type frame header and the data transmission frame header of each of the multiple first audio and video data packets to be encapsulated are set, and the type of the encoding protocol is written into the encoding type of each of the multiple first audio and video data packets to be encapsulated.
[0024] A second aspect of an embodiment of the present invention provides a data transmission method based on a visual network, the method being applied to a receiving end, comprising:
[0025] Receive audio and video data sent by the transmitter, and parse the audio and video data to determine the current packet loss correction mode; wherein the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first protocol is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs;
[0026] Determine the packet loss rate of the audio and video data, and determine whether to switch the current packet loss error correction mode according to the size relationship between the packet loss rate and a preset packet loss rate threshold;
[0027] In the case of determining to switch the current packet loss error correction mode, sending a mode switching instruction to the sending end so that the sending end switches the current packet loss error correction mode;
[0028] Receive the audio and video data sent by the sending end according to the switched packet loss error correction mode.
[0029] Optionally, determining whether to switch the current packet loss error correction mode according to a magnitude relationship between the packet loss rate and a preset packet loss rate threshold includes:
[0030] In a case where the current packet loss error correction mode is the first packet loss error correction mode, if the packet loss rate is less than the preset packet loss rate threshold, determining to switch the current packet loss error correction mode;
[0031] In a case where the current packet loss correction mode is the second packet loss correction mode, if the packet loss rate is greater than the preset packet loss rate threshold, it is determined to switch the current packet loss correction mode.
[0032] Optionally, the packet header of the audio and video data includes an error correction type frame header and a data transmission frame header, and parsing the audio and video data to determine the current packet loss error correction mode includes:
[0033] Parsing the error correction type frame header of the audio and video data to determine the current packet loss error correction mode;
[0034] Determining the packet loss rate of the audio and video data includes:
[0035] When the current packet loss error correction mode is the first packet loss error correction mode, parsing the data transmission frame header of the audio and video data to obtain an expected total length of the data, where the expected total length of the data is used to represent the total length of the audio and video data sent by the sending end;
[0036] Determine the packet loss rate of the audio and video data according to the total length of the audio and video data actually received and the expected total length of the preset data;
[0037] The method further comprises:
[0038] When the current packet loss correction mode is the first packet loss correction mode, a request is made to the sending end to retransmit the lost audio and video data.
[0039] Optionally, the error correction type frame header further includes a coding type, and the method further includes:
[0040] According to the currently enabled encoding protocol, the audio and video data in the plurality of first audio and video data packets are encoded to obtain a plurality of first video data packets to be encapsulated;
[0041] The error correction type frame header includes a coding type, and according to the second packet loss error correction mode, the error correction type frame header and the data transmission frame header of each of the plurality of first audio and video data packets are set, and the method further includes:
[0042] The type of the encoding protocol is written into the encoding type of each of the multiple first audio and video data packets to be encapsulated.
[0043] According to a third aspect of an embodiment of the present invention, a data transmission device based on a visual network is provided, the device being located at a transmitting end and comprising:
[0044] A first data sending module, used for sending audio and video data to a receiving end according to a current packet loss error correction mode;
[0045] A switching module, configured to switch the current packet loss error correction mode in response to a mode switching instruction sent by the receiving end, wherein the mode switching instruction is generated by the receiving end according to a magnitude relationship between a packet loss rate when receiving audio and video data and a preset packet loss rate threshold;
[0046] A second data sending module, used for sending audio and video data to the receiving end according to the switched packet loss error correction mode;
[0047] Among them, the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol that does not require retransmission when packet loss occurs.
[0048] According to a fourth aspect of an embodiment of the present invention, a data transmission device based on a visual network is provided, wherein the device is located at a receiving end and includes:
[0049] A first receiving module is used to receive audio and video data sent by a transmitting end, and parse the audio and video data to determine a current packet loss correction mode; wherein the current packet loss correction mode is a first packet loss correction mode or a second packet loss correction mode, the first protocol is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs;
[0050] A determination module, used to determine the packet loss rate of the audio and video data, and determine whether to switch the current packet loss error correction mode according to the size relationship between the packet loss rate and a preset packet loss rate threshold;
[0051] An instruction sending module, configured to send a mode switching instruction to the sending end when it is determined to switch the current packet loss error correction mode, so that the sending end switches the current packet loss error correction mode;
[0052] The second receiving module is used to receive the audio and video data sent by the sending end according to the switched packet loss error correction mode.
[0053] According to a fifth aspect of the embodiments of the present invention, an electronic device is further disclosed, including:
[0054] one or more processors; and
[0055] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the device to execute the data transmission method based on visual networking as described in the first aspect of the embodiment of the present invention.
[0056] The sixth aspect of the embodiments of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the data transmission method based on visual networking as described in the first aspect of the embodiments of the present invention.
[0057] In this embodiment, the transmitting end can send audio and video data to the receiving end according to the current packet loss correction mode; and switch the current packet loss correction mode in response to the mode switching instruction sent by the receiving end, wherein the mode switching instruction is generated by the receiving end according to the size relationship between the packet loss rate when receiving the audio and video data and the preset packet loss rate threshold; then, send the audio and video data to the receiving end according to the switched packet loss correction mode. Among them, the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs.
[0058] Compared with the prior art, the embodiments of the present invention have at least the following advantages:
[0059] On the one hand, since the sending end can generate a mode switching instruction according to the size relationship between the packet loss rate of the audio and video data and the preset packet loss rate threshold when the sending end is sending audio and video data to the receiving end, that is, the receiving end can decide whether to trigger the sending end to switch the current packet loss correction mode according to the packet loss rate of the audio and video data. In this way, the receiving end can switch between the first packet loss correction mode and the second packet loss correction mode according to the mode switching instruction, so that the sending end can dynamically change the packet loss correction mode in the process of sending audio and video data, thereby improving the flexibility of responding to data packet loss during the transmission of audio and video data.
[0060] On the other hand, different response mechanisms to packet loss between the sender and the receiver can be enabled in different packet loss correction modes. For example, in the first packet loss correction mode, lost data packets are retransmitted, which can improve the integrity of the data packets received by the receiver, thereby avoiding audio and video freezes. For another example, in the second packet loss correction mode, lost data packets may not be retransmitted, thus avoiding network congestion caused by frequent retransmissions, thereby improving the data transmission efficiency between the receiver and the sender and improving the fluency of audio and video. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0062] Figure 1 is a communication environment diagram of an embodiment of the present invention;
[0063] Figure 2 It is a flowchart of the steps of a data transmission method based on visual networking according to an embodiment of the present invention;
[0064] Figure 3 is a flowchart of the steps of sending audio and video data to the receiving end according to the switched packet loss error correction mode according to an embodiment of the present invention;
[0065] Figure 4 is a schematic diagram of data content included in an audio and video data packet according to an embodiment of the present invention;
[0066] Figure 5 is a flowchart of another method for data transmission based on visual networking according to an embodiment of the present invention;
[0067] Figure 6 A schematic diagram of the structure of a data transmission device based on a visual network according to an embodiment of the present invention;
[0068] Figure 7 A schematic diagram of the structure of another data transmission device based on visual networking according to an embodiment of the present invention. DETAILED DESCRIPTION
[0069] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0070] When transmitting audio and video data in a visual network, the data is generally transmitted in accordance with the visual network transmission protocol. The visual network transmission protocol generally adds a V2V protocol header unique to the visual network on the basis of the traditional Ethernet protocol header of the data to ensure that the devices in the visual network can parse the V2V protocol header and thus use the data normally.
[0071] Although the visual network is a real-time high-definition transmission network, the network bandwidth and network resources in the visual network are controllable as a whole and divided and reserved in advance. That is, when a device in the visual network needs to use the visual network resources, it needs to apply for a reservation in advance to obtain the bandwidth resources allocated by the visual network, so as to ensure that there are bandwidth resources when using it.
[0072] However, with the surge in business volume in the visual network and its interconnection with the Internet, even if the visual network adopts the above-mentioned bandwidth resource allocation method, it still cannot avoid the network bandwidth pressure caused by the surge in devices, which may cause the network environment of the visual network to become unstable. Therefore, when transmitting audio and video data in the visual network, there will be the problem of audio and video data packet loss caused by the unstable network environment.
[0073] Once the audio and video data packets are lost, it may cause the receiving device at the other end to experience video freezes and audio loss, making it impossible to guarantee that the real-time audio and video services in the visual network have a high service quality, such as video conferencing.
[0074] In view of this, the applicant has proposed the following core technical concept in view of the characteristics of data transmission in the visual network: starting from the visual network transmission protocol framework, a packet loss correction mechanism based on the L2 layer network is designed, so that in the process of transmitting audio and video data between the sender and the receiver, according to the packet loss rate of the audio and video data, the packet loss correction mechanism can be negotiated to change, so as to improve the smoothness of audio and video data transmission in the case of poor visual network environment, and reduce the packet loss rate in the case of better network environment, so as to reduce the jamming of audio and video, thereby improving the quality of real-time services in the visual network. Among them, the L2 layer network refers to the second layer in the OSI seven-layer model, that is, the data link layer.
[0075] Reference Figure 1 As shown, a communication environment diagram of an embodiment of the present application is shown, the communication environment is located in a visual network, including a receiving end and a sending end, wherein the receiving end and the sending end communicate with each other through a visual network protocol. The receiving end and the sending end can both be smart devices.
[0076] Combination Figure 1 The implementation environment shown in the figure introduces the overall concept of the data transmission method based on the visual network in the embodiment of the present application. Figure 1 As shown, the receiving end can decode the error correction type of the received audio and video data during the process of receiving the audio and video data to determine the current packet loss correction mode, and determine whether to respond to the current packet loss correction mode according to the packet loss rate and the degree of network congestion. Specifically, the response includes a response to packet loss processing according to the current packet loss correction mode when the current packet loss correction mode is not switched; and includes a response to sending a mode switching instruction to the sending end when the current packet loss correction mode is switched.
[0077] In the process of sending audio and video data, the sending end can switch between the first packet loss correction mode and the second packet loss correction mode in response to the mode switching instruction sent by the receiving end. For example, the first packet loss correction mode is switched to the second packet loss correction mode, or the second packet loss correction mode is switched to the first packet loss correction mode, and then the audio and video data are sent according to the switched packet loss correction mode.
[0078] Combine the following Figure 1 The communication environment shown provides a detailed introduction to the data transmission method based on visual networking of the present application from both the sending and receiving ends.
[0079] Reference Figure 2 As shown, a flowchart of a method for data transmission based on visual networking according to an embodiment of the present invention is shown. The method is applied to a sending end and may specifically include the following steps:
[0080] Step S201: sending audio and video data to a receiving end according to the current packet loss error correction mode.
[0081] Among them, the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs.
[0082] In this embodiment, the packet loss correction mode can be understood as a processing mechanism for the problem of data packet loss when transmitting data between the sender and the receiver in the visual network. Among them, the first packet loss correction mode can be considered as: when data packet is lost, the receiving end requests the sending end to process the lost data. And the second packet loss correction mode can be considered as: when data packet is lost, the receiving end does not need to request the sending end to retransmit the lost data.
[0083] The audio and video data sent by the sending end may be audio and video data collected in real time, and when the sending end sends the audio and video data, the audio and video data may be sent according to the currently enabled packet loss correction mode. In specific implementation, a data transmission protocol may be set in the data packet header of the audio and video data, and the data transmission protocol may be used to indicate the packet loss transmission mechanism between the sending end and the receiving end. Therefore, the sending end may set the data packet header of the audio and video data according to the currently enabled packet loss correction mode, so that the receiving end may parse the data packet header to clarify the packet loss correction mode used by the sending end.
[0084] Step S202: In response to a mode switching instruction sent by the receiving end, the current packet loss error correction mode is switched, wherein the mode switching instruction is generated by the receiving end according to the relationship between the packet loss rate when receiving audio and video data and a preset packet loss rate threshold.
[0085] In this embodiment, the preset packet loss rate threshold can be understood as a threshold node for determining whether to switch the packet loss correction mode. The receiving end can determine whether to switch the current packet loss correction mode based on the packet loss rate of the audio and video data, the preset packet loss rate threshold and the currently used packet loss correction mode.
[0086] Among them, the preset packet loss rate threshold can be set to 10%. When the packet loss rate of audio and video data is greater than or equal to 10%, it means that there is a lot of packet loss, and the audio and video data may not be decoded normally at the receiving end, so as to play normal audio and video. In this case, the first packet loss correction mode can be turned on, that is, if the current packet loss correction mode is not the first packet loss correction mode, the receiving end generates a mode switching instruction to enable the sending end to switch the current packet loss correction mode to the first packet loss correction mode, and the sending end then retransmits the lost data to the receiving end, so that the receiving end can decode the complete audio and video data.
[0087] When the packet loss rate of audio and video data is less than 10%, it means that the packet loss is small, and generally does not affect the normal decoding of audio and video data at the receiving end. In this case, in order to improve the transmission efficiency and real-time performance of audio and video data, the second packet loss correction mode can be turned on. That is, if the current packet loss correction mode is not the second packet loss correction mode, the receiving end generates a mode switching instruction to enable the sending end to switch the current packet loss correction mode to the second packet loss correction mode. At this time, the sending end does not need to retransmit the lost data to the receiving end, so as to improve the transmission efficiency of audio and video data between the receiving end and the receiving end.
[0088] When the receiving end determines that the current packet loss correction mode needs to be switched, a mode switching instruction can be generated and sent to the sending end, so that the sending end switches the current packet loss correction mode in response to the mode switching instruction. Specifically, when the current packet loss correction mode is the first packet loss correction mode, the first packet loss correction mode can be switched to the second packet loss correction mode according to the mode switching instruction. When the current packet loss correction mode is the second packet loss correction mode, the second packet loss correction mode can be switched to the first packet loss correction mode according to the mode switching instruction.
[0089] Step S203: sending audio and video data to the receiving end according to the switched packet loss error correction mode.
[0090] In this embodiment, the sending end can set the data transmission protocol in the data packet header of the audio and video data to be sent according to the switched packet loss correction mode, that is, use the data transmission protocol corresponding to the switched packet loss correction mode to encapsulate the audio and video data, and then send the encapsulated audio and video data to the receiving end in the form of a data packet.
[0091] By adopting the technical solution of the embodiment of the present invention, since the sending end can determine whether to switch the current packet loss correction mode according to the relationship between the packet loss rate of the audio and video data and the preset packet loss rate threshold when the sending end sends audio and video data to the receiving end, when switching is required, a mode switching instruction is generated so that the sending end can switch the current packet loss correction mode to the first packet loss correction mode or the second packet loss correction mode according to the mode switching instruction, so that the sending end can dynamically change the packet loss correction mode when sending audio and video data, thereby improving the flexibility of responding to data packet loss during audio and video data transmission.
[0092] On the other hand, since the response mechanism to data packet loss is different under different packet loss correction modes, it is possible to dynamically adapt to the current visual network environment to adapt to the packet loss response mechanism of the current visual network environment, reduce the packet loss rate of audio and video data to improve the fluency of audio and video data. For example, in the first packet loss correction mode, the lost data packets are retransmitted, which can improve the integrity of the data packets received by the receiving end, reduce the packet loss rate, and thus avoid audio and video jams. For another example, in the second packet loss correction mode, that is, when the number of lost data packets is small, the lost data packets may not be retransmitted, thus avoiding network congestion caused by frequent retransmissions, thereby improving the data transmission efficiency between the receiving end and the sending end, and improving the fluency of audio and video.
[0093] Below, the specific implementation method of how the present application sends audio data to the receiving end according to the switched packet loss error correction mode is described as follows:
[0094] In one implementation, generally speaking, the audio and video data can be first encapsulated according to the visual networking protocol to obtain multiple audio and video data packets that can be used for transmission in the visual networking, and then based on the packet loss correction mechanism of the L2 layer network, a protocol header corresponding to the switched packet loss correction mode is set in the packet header of each audio and video data packet, thereby completing the re-encapsulation of the audio and video data packets. Figure 3 As shown, a flowchart of the steps of sending audio and video data to the receiving end according to the switched packet loss correction mode in this embodiment is shown, as Figure 3 As shown, the following steps may be specifically included:
[0095] Step S2021: According to the visual networking protocol, the audio and video data to be sent are encapsulated into multiple audio and video data packets.
[0096] In this embodiment, a data packet of audio and video data includes a frame header part and a protocol payload part, wherein the protocol payload includes the original audio and video data. The frame header part includes an Ethernet header, a visual network exchange protocol header, and an error correction protocol frame. When sending audio and video data, the sender can encapsulate the audio and video data collected in real time into multiple audio and video data packets, and the frame header of each audio and video data packet includes an Ethernet header and a visual network exchange protocol header.
[0097] Step S2022: adding an error correction protocol frame to the frame header data of each of the multiple audio and video data packets to obtain multiple first audio and video data packets; the error correction protocol frame includes an error correction type frame header and a data transmission frame header, the error correction type frame header is used to characterize the currently adopted packet loss correction mode, and the data transmission frame header is used to characterize the properties of the currently transmitted audio and video data packet.
[0098] In this embodiment, after obtaining multiple audio and video data, an error correction protocol frame can be added to the frame header of each audio and video data packet. The error correction protocol frame header can be located in a preset field in the header of the audio and video data packet. That is, when the audio and video data are encapsulated using the visual networking protocol to obtain multiple audio and video data packets, the preset field of the audio and video data packet can be left empty first, and then the error correction protocol frame can be added to the preset field to obtain the first audio and video data packet.
[0099] Step S2023: according to the switched packet loss correction mode, the error correction type frame header and the data transmission frame header included in each of the plurality of first audio and video data packets are set to obtain a plurality of second audio and video data packets.
[0100] In this embodiment, after obtaining a plurality of first audio and video data packets, the error correction type frame header and the data transmission frame header of each of the plurality of first audio and video data packets may be respectively set.
[0101] Among them, the error correction type frame header can use different codes to represent the packet loss correction mode after switching, for example, "0" represents the second packet loss correction mode, and "1" represents the first packet loss correction mode. Different fields in the data transmission frame header can represent different properties of the data packet. For example, the value of the first field in the data transmission frame header can represent the total length of the data packet to be sent, and the value of the second field can represent the sequence number of the data packet. After the setting is completed, the second audio and video data packet is obtained, and then the second audio and video data packet is sent to the receiving end.
[0102] Among them, refer to Figure 4 As shown, a schematic diagram showing the data content included in an audio and video data packet in one embodiment is shown, such as Figure 4 As shown in the figure, an audio and video data packet includes a frame header and a payload, wherein the frame header includes an Ethernet header, a visual network exchange protocol header, an error correction type header, and a V2V protocol payload. The frame header includes a message type header, an error correction type header, and a data transmission header, and the V2V protocol payload carries the original audio and video data.
[0103] Among them, the message type header is used to characterize the response mechanism of the currently adopted packet loss correction mode, the error correction type header is used to characterize the currently adopted packet loss correction mode, and the data transmission header is used to characterize the properties of the currently transmitted audio and video data packets. Among them, the data transmission header is mainly used to confirm the retransmitted data packets for the receiving end when the lost data packets need to be retransmitted.
[0104] Combination Figure 4 As shown, in some specific examples, different protocol headers can be set for audio and video data packets according to different packet loss error correction modes, specifically:
[0105] When the packet loss correction mode after switching is the first packet loss correction mode, the total length, the total number and the length of each audio and video data packet of the multiple first audio and video data packets can be determined; the first packet loss correction mode is written into the error correction type frame header of each of the multiple first audio and video data packets, and the total length, the total number and the length of each audio and video data packet are written into the data transmission frame header of each of the multiple first audio and video data packets to obtain the multiple second audio and video data packets.
[0106] In this embodiment, the first packet loss correction mode is a mode in which the lost data packet needs to be retransmitted when the packet is lost. Therefore, the encoding type corresponding to the first packet loss correction mode can be set in the corresponding field of the error correction type header, so that the receiving end starts the retransmission mechanism when the packet is lost according to the first packet loss correction mode. In addition, since the first packet loss correction mode is a mode in which the lost data packet needs to be retransmitted when the packet is lost, the total length of multiple audio and video data packets to be sent, the packet sequence number of the current audio and video data packet, and the length of the current audio and video data packet can be set in the corresponding field in the data transmission frame header, so that the receiving end can determine the packet loss rate according to the data transmission frame header, and determine the packet sequence number of the lost data packet, so as to retransmit the lost data packet.
[0107] When the packet loss correction mode after switching is the second packet loss correction mode, the error correction type frame header and the data transmission frame header of each of the multiple first audio and video data packets can be set according to the second packet loss correction mode to obtain the multiple second audio and video data packets.
[0108] In this embodiment, the second packet loss correction mode is a mechanism that does not require retransmission when packet loss occurs. Therefore, the encoding type corresponding to the second packet loss correction mode can be set in the corresponding field of the error correction type header, so that the receiving end receives the audio and video data according to the second packet loss correction mode, and does not need to start the retransmission mechanism when packet loss is determined. And because the second packet loss correction mode is a mechanism that does not require retransmission when packet loss occurs, the corresponding fields of the data transmission frame header of the audio and video data packets can be set to default values, for example, all set to 0. In this case, the receiving end does not need to determine the packet sequence number of the lost packet.
[0109] It should be noted that the second packet loss correction mode of the embodiment of the present application can be forward error correction (FEC for short). Forward error correction is an error control method, which means that the signal is encoded according to a certain algorithm before being sent to the transmission channel, and a redundant code with the characteristics of the signal itself is added. The received signal is decoded according to the corresponding algorithm at the receiving end, so as to find the error code generated during the transmission process and correct it. Therefore, when forward error correction is used as the second packet loss correction mode, the receiving end can automatically decode the received audio and video data and correct the error. When the packet loss rate is small, the receiving end does not need to request the sending end to retransmit the lost data.
[0110] like Figure 4 As shown, in a specific example, the error correction type frame header may also include a coding type, and the error correction coding type may be used to characterize the coding protocol currently encoding the audio and video data.
[0111] When a plurality of first audio and video data packets are obtained, the audio and video data in the plurality of first audio and video data packets can be encoded according to the currently enabled encoding protocol to obtain a plurality of first video data packets to be encapsulated. Furthermore, while setting the error correction type frame header and data transmission frame header of each of the plurality of first audio and video data packets, the type of the encoding protocol can also be written into the encoding type of each of the plurality of first audio and video data packets to be encapsulated, thereby obtaining a second audio and video data packet.
[0112] In this embodiment, the value or symbol of the corresponding field in the error correction type frame header represents the encoding protocol of the audio and video data, where different encoding protocols may correspond to different values or different symbols. Figure 4 As shown, “0” indicates Turbo coding, “1” indicates LDPC coding, and “2” indicates RA coding.
[0113] Step S2024: Send the multiple second audio and video data packets to the receiving end.
[0114] In this embodiment, after obtaining multiple second audio and video data packets corresponding to the switched packet loss correction mode through the above example, the transmitting end can send the multiple second audio and video data packets to the receiving end, so that the receiving end determines the packet loss rate and performs different processing on the packet loss according to the switched packet loss correction mode. When receiving the audio and video data packets, the receiving end can decode the audio and video data according to the encoding type in the error correction type frame header and adopt the corresponding encoding and decoding protocol to play normally.
[0115] Of course, in practice, when the sending end sends audio and video data packets to the receiving end according to the first packet loss correction mode, the sending end can also respond to the retransmission request sent by the receiving end, wherein the retransmission request can carry the packet sequence number of the lost data packet. The sending end can then retransmit the corresponding lost audio and video data packet to the receiving end based on the packet sequence number of the data packet in the retransmission request.
[0116] Based on the same inventive concept, Figure 5 As shown, a flowchart of the steps of a data transmission method based on a visual network in another embodiment is shown, which is applied to a receiving end, such as Figure 5 As shown, the following steps may be specifically included:
[0117] Step S501: receiving audio and video data sent by a transmitter, and analyzing the audio and video data to determine a current packet loss error correction mode.
[0118] Among them, the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first protocol is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs.
[0119] In this embodiment, the transmitting end may send the audio and video data collected in real time to the receiving end in the form of data packets, and then the receiving end may parse the frame header of the received audio and video data packets to determine the current packet loss error correction mode.
[0120] In an example, the packet header of the audio and video data includes an error correction type frame header and a data transmission frame header, and the error correction type frame header of the audio and video data can be parsed to determine the current packet loss error correction mode.
[0121] like Figure 4 As shown, each audio and video data packet may include an error correction type frame header and a data transmission frame header, wherein the error correction type frame header is used to identify the current packet loss correction mode, and the data transmission frame header is used to identify the properties of the audio and video data packet. The receiving end may parse the value of the corresponding field in the error correction type frame header of the audio and video data packet to determine the current packet loss correction mode. For example, Figure 4 As shown, the receiving end parses the value of the first field in the error correction type header and determines that the value is "0", which indicates that the current packet loss error correction mode is the second packet loss error correction mode.
[0122] Step S502: Determine the packet loss rate of the audio and video data, and determine whether to switch the current packet loss error correction mode based on the size relationship between the packet loss rate and a preset packet loss rate threshold.
[0123] In this embodiment, the receiving end can determine the packet loss rate of the audio and video data according to the current packet loss correction mode. When the current packet loss correction mode is the second packet loss correction mode, that is, when the forward error correction is used, the packet loss rate of the audio and video data is determined according to the packet loss confirmation means of the forward error correction. When the current packet loss correction mode is the first packet loss correction mode, the packet loss rate of the audio and video data can be determined according to the attributes of the data packet represented by the data transmission frame header.
[0124] Specifically, when the current packet loss correction mode is the first packet loss correction mode, the data transmission frame header of the audio and video data can be parsed to obtain the expected total length of the data, and the expected total length of the data is used to characterize the total length of the audio and video data sent by the sender; and the packet loss rate of the audio and video data is determined based on the total length of the audio and video data actually received and the preset expected total length of the data.
[0125] Among them, the total length of audio and video data refers to the sum of the lengths of multiple audio and video data packets to be sent by the sender. For example, the sender wants to send 100 audio and video data packets with a total length of 1MB, then the expected total length of the preset data is 1MB, and the receiving end actually receives 75 audio and video data packets with a total length of 785kb. The packet loss rate is: the ratio of the difference between the total length of the actual received data packets and the expected total length of the preset data to the expected total length of the preset data, which is 22%.
[0126] Of course, in practice, the packet loss rate can also be the ratio of the difference between the number of audio and video data packets to be sent and the number of data packets actually received, to the total number of audio and video data packets to be sent. For example, if 75 data packets are received and 10 data packets are actually to be sent, the packet loss rate is 25.
[0127] The preset packet loss rate threshold can be preset according to experience or actual needs, for example, it can be set to 10%. Furthermore, the receiving end can determine whether to switch the current packet loss correction mode according to whether the size relationship between the packet loss rate of the audio and video data and the preset packet loss rate threshold matches the current packet loss correction mode.
[0128] Among them, the relationship between the packet loss rate of audio and video data and the preset packet loss rate threshold can reflect the quality of the current visual network environment. In practice, the receiving end can dynamically change the packet loss correction mode that is compatible with the current visual network environment based on the quality of the current visual network environment.
[0129] In specific implementation, when the current packet loss correction mode is the first packet loss correction mode, if the packet loss rate is less than the preset packet loss rate threshold, it is determined to switch the current packet loss correction mode; when the current packet loss correction mode is the second packet loss correction mode, if the packet loss rate is greater than the preset packet loss rate threshold, it is determined to switch the current packet loss correction mode.
[0130] Among them, when the packet loss rate of audio and video data is greater than or equal to the preset packet loss rate threshold, it means that there is a lot of packet loss, and the audio and video data may not be decoded normally at the receiving end, so as to play normal audio and video. In this case, the first packet loss correction mode can be turned on, that is, if the current packet loss correction mode is not the first packet loss correction mode (in practice, it works in the second packet loss correction mode), the receiving end generates a mode switching instruction to enable the sending end to switch the current packet loss correction mode to the first packet loss correction mode, and the sending end then retransmits the lost data to the receiving end, so that the receiving end can decode the complete audio and video data.
[0131] Among them, when the receiving end requests the sending end to retransmit the lost audio and video data in the first packet loss correction mode, it can determine the packet sequence number of the lost data packet based on the packet sequence number of the received data packet, and then send a retransmission request to the sending end. The retransmission request can carry the packet sequence number of the data packet that needs to be retransmitted, so that the sending end retransmits these lost data packets.
[0132] Among them, when the packet loss rate of audio and video data is less than the preset packet loss rate threshold, it means that the packet loss is small, and generally does not affect the normal decoding of audio and video data at the receiving end. In this case, in order to improve the transmission efficiency and real-time performance of audio and video data, the second packet loss correction mode can be turned on. That is, if the current packet loss correction mode is not the second packet loss correction mode, the receiving end generates a mode switching instruction to enable the sending end to switch the current packet loss correction mode to the second packet loss correction mode. At this time, the sending end does not need to retransmit the lost data to the receiving end, so as to improve the audio and video data transmission efficiency between the receiving end and the receiving end.
[0133] Of course, when the current packet loss error correction mode is the first packet loss error correction mode, the receiving end may also request the sending end to retransmit the lost audio and video data.
[0134] In this case, since the sender can number the audio and video data packets to be sent in sequence, the receiver can determine the packet numbers of the unreceived audio and video data packets based on the packet numbers of the audio and video data packets actually received, and thus request the lost data packets from the sender.
[0135] In an optional example, the receiving end may determine whether to switch the current packet loss correction mode based on the relationship between the packet loss rate and a preset packet loss rate threshold and the current network congestion status of the visual network.
[0136] In specific implementation, if the packet loss rate is greater than or equal to the preset packet loss rate threshold and the network congestion level is high, it means that there are many packet losses and the network environment is not good. At this time, if the current packet loss correction mode is the first packet loss correction mode, it means that the network congestion cannot guarantee the efficiency of retransmitting data packets. At this time, it can be switched to the second packet loss correction mode for audio and video data transmission. Of course, if the packet loss rate is greater than or equal to the preset packet loss rate threshold and the network congestion level is low, if the current packet loss correction mode is the first packet loss correction mode, there is no need to switch the mode. If the current packet loss correction mode is the second packet loss correction mode, it is determined to switch to the first packet loss correction mode.
[0137] Specifically, if the packet loss rate is less than the preset packet loss rate threshold, then regardless of whether the network congestion level is high or low, in practice, the second packet loss correction mode can be used to transmit audio and video data. That is, in this case, when the current packet loss correction mode is the first packet loss correction mode, it is determined to switch the first packet loss correction mode to the second packet loss correction mode.
[0138] Step S503: When it is determined to switch the current packet loss error correction mode, a mode switching instruction is sent to the sending end, so that the sending end switches the current packet loss error correction mode.
[0139] In this embodiment, when the receiving end determines to switch the current packet loss error correction mode, the receiving end may send a mode switching instruction to the sending end, so that the sending end switches the current packet loss error correction mode.
[0140] For example, when the packet loss rate of audio and video data is less than a preset packet loss rate threshold, and the current packet loss correction mode is the first packet loss correction mode, a mode switching instruction can be sent to the sending end so that the sending end switches the first packet loss correction mode to the second packet loss correction mode to improve transmission efficiency. When the packet loss rate of audio and video data is greater than or equal to the preset packet loss rate threshold, and the current packet loss correction mode is the second packet loss correction mode, a mode switching instruction can be sent to the sending end so that the sending end switches the second packet loss correction mode to the first packet loss correction mode to reduce the packet loss rate.
[0141] Step S504: receiving the audio and video data sent by the transmitting end according to the switched packet loss error correction mode.
[0142] In this embodiment, the process of the transmitter sending audio and video data according to the switched packet loss error correction mode can refer to the above-mentioned process from step S2021 to step S2023.
[0143] In this embodiment, the error correction type frame header also includes a coding type. When the current packet loss correction mode is the second packet loss correction mode, or when the packet loss correction mode after switching is the second packet loss correction mode, the coding type in the error correction type frame header of the audio and video data can be parsed to determine the coding protocol corresponding to the audio and video data; and the audio and video data can be decoded according to the coding protocol to play the audio and video data.
[0144] The error correction type frame header of each audio and video data packet may also include a coding type. Generally speaking, in the second packet loss error correction mode, that is, the forward error correction mode, the corresponding coding protocol will be adopted to encode the audio and video data so that the receiving end can correct the error code during the transmission process. Therefore, in this mode, the receiving end will parse the coding type of the audio and video data to obtain the coding protocol used to encode the audio and video data, and then use the corresponding decoding protocol to decode it to achieve the playback of the audio and video data in the second packet loss error correction mode.
[0145] When the transmitting end sends the audio and video data through the first packet loss error correction mode, the receiving end needs to request the transmitting end to retransmit the lost data, so any encoding protocol can be used to encode the audio and video data. In this case, the receiving end can reversely decode the audio and video data. Of course, in practice, the receiving end can also parse the encoding type of the audio and video data in the first packet loss error correction mode to decode the audio and video data using the corresponding decoding protocol.
[0146] By adopting the technical solution of the embodiment of the present invention, the receiving end can parse the audio and video data sent by the sending end to determine the current packet loss correction mode, and determine whether to send a mode switching instruction to the sending end based on the relationship between the packet loss rate of the audio and video data and the preset packet loss rate threshold. If so, the sending end switches the current packet loss correction mode according to the mode switching instruction, thereby sending the audio and video data according to the switched packet loss correction mode.
[0147] Compared with the prior art, the technical solution of the embodiment of the present invention has at least the following advantages:
[0148] First, since the receiving end can determine the packet loss rate of the audio and video data during the process of receiving the audio and video data, and decide whether to switch the current packet loss correction mode according to the relationship between the packet loss rate and the preset packet loss rate threshold, when switching is required, a mode switching instruction is sent to the sending end, so that the sending end can switch the current packet loss correction mode to the first packet loss correction mode or the second packet loss correction mode according to the mode switching instruction, so that in the process of receiving the audio and video data, the packet loss correction mode can be dynamically changed, thereby improving the flexibility of responding to data packet loss during the audio and video data transmission process.
[0149] Second, because the response mechanism to data packet loss is different under different packet loss correction modes, it is possible to dynamically adapt to the current visual network environment to adapt to the packet loss response mechanism of the current visual network environment, reduce the packet loss rate of audio and video data to improve the fluency of audio and video data. For example, in the first packet loss correction mode, the lost data packets are retransmitted, which can improve the integrity of the data packets received by the receiving end, reduce the packet loss rate, and thus avoid audio and video jams. For another example, in the second packet loss correction mode, that is, when the number of lost data packets is small, the lost data packets may not be retransmitted, thus avoiding network congestion caused by frequent retransmissions, thereby improving the data transmission efficiency between the receiving end and the sending end, and improving the fluency of audio and video.
[0150] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the described action sequence because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0151] refer to Figure 6 , shows a data transmission device based on visual networking, which is applied to the sending end and may specifically include the following modules:
[0152] A first data sending module 601, configured to send audio and video data to a receiving end according to a current packet loss error correction mode;
[0153] A switching module 602, configured to switch the current packet loss error correction mode in response to a mode switching instruction sent by the receiving end, wherein the mode switching instruction is generated by the receiving end according to a magnitude relationship between a packet loss rate when receiving audio and video data and a preset packet loss rate threshold;
[0154] A second data sending module 603, configured to send audio and video data to the receiving end according to the switched packet loss error correction mode;
[0155] Among them, the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol that does not require retransmission when packet loss occurs.
[0156] Optionally, the switching module 602 may specifically include the following units:
[0157] An encapsulation unit, used to encapsulate the audio and video data to be sent into multiple audio and video data packets according to the visual networking protocol;
[0158] A protocol adding unit, configured to add an error correction protocol frame to the frame header data of each of the plurality of audio and video data packets to obtain a plurality of first audio and video data packets; the error correction protocol frame comprises an error correction type frame header and a data transmission frame header, the error correction type frame header is used to characterize the currently adopted packet loss correction mode, and the data transmission frame header is used to characterize the properties of the currently transmitted audio and video data packets;
[0159] A setting unit, configured to set the error correction type frame header and the data transmission frame header respectively included in the plurality of first audio and video data packets according to the switched packet loss correction mode, so as to obtain a plurality of second audio and video data packets;
[0160] The sending unit is used to send the multiple second audio and video data packets to the receiving end.
[0161] Optionally, the setting unit may specifically include the following subunits:
[0162] a determination subunit, configured to determine the total length, the total number, and the length of each audio and video data packet of the plurality of first audio and video data packets when the packet loss correction mode after the switch is the first packet loss correction mode;
[0163] A writing unit is used to write the first packet loss correction mode into the error correction type frame header of each of the multiple first audio and video data packets, and write the total length, the total number and the length of each audio and video data packet into the data transmission frame header of each of the multiple first audio and video data packets to obtain the multiple second audio and video data packets.
[0164] Optionally, the setting unit may specifically include the following subunits:
[0165] A subunit is set for setting the error correction type frame header and the data transmission frame header of each of the multiple first audio and video data packets according to the default protocol header corresponding to the second packet loss correction mode when the packet loss correction mode after switching is the second packet loss correction mode, so as to obtain the multiple second audio and video data packets.
[0166] Optionally, the error correction type frame header includes a coding type, and the device may further include a module:
[0167] An encoding module, used for encoding the audio and video data in the plurality of first audio and video data packets according to a currently enabled encoding protocol to obtain a plurality of first video data packets to be encapsulated;
[0168] The setting subunit may be specifically configured to write the type of the encoding protocol into the encoding type of each of the plurality of first audio and video data packets to be encapsulated.
[0169] Reference Figure 7 As shown, a structural block diagram of a data transmission device based on visual networking in another embodiment is shown, and the device is located at the receiving end and may specifically include the following modules:
[0170] The first receiving module 701 is used to receive the audio and video data sent by the transmitting end, and parse the audio and video data to determine the current packet loss correction mode; wherein the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first protocol is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs;
[0171] A determination module 702 is used to determine the packet loss rate of the audio and video data, and determine whether to switch the current packet loss error correction mode according to the size relationship between the packet loss rate and a preset packet loss rate threshold;
[0172] An instruction sending module 703 is used to send a mode switching instruction to the sending end when it is determined to switch the current packet loss error correction mode, so that the sending end switches the current packet loss error correction mode;
[0173] The second receiving module 704 is used to receive the audio and video data sent by the sending end according to the switched packet loss error correction mode.
[0174] Optionally, the first receiving module 701 may specifically include the following units:
[0175] The first unit is configured to determine, when the current packet loss correction mode is the first packet loss correction mode, if the packet loss rate is less than the preset packet loss rate threshold, to switch the current packet loss correction mode;
[0176] The second unit is used to determine to switch the current packet loss correction mode when the current packet loss correction mode is the second packet loss correction mode and if the packet loss rate is greater than the preset packet loss rate threshold.
[0177] Optionally, the packet header of the audio and video data includes an error correction type frame header and a data transmission frame header, and the determination module 701 can be specifically used to parse the error correction type frame header of the audio and video data to determine the current packet loss error correction mode;
[0178] The determination module 702 may specifically include the following units:
[0179] a length determination unit, configured to parse the data transmission frame header of the audio and video data to obtain an expected total length of the data when the current packet loss error correction mode is the first packet loss error correction mode, wherein the expected total length of the data is used to represent the total length of the audio and video data sent by the sending end;
[0180] A determination unit, configured to determine a packet loss rate of the audio and video data according to a total length of the currently actually received audio and video data and an expected total length of the preset data;
[0181] The device may also include the following modules:
[0182] The retransmission module is used to request the sending end to retransmit the lost audio and video data when the current packet loss error correction mode is the first packet loss error correction mode.
[0183] Optionally, the error correction type frame header further includes a coding type, and the device may further include the following modules:
[0184] A parsing module, configured to parse the coding type in the error correction type frame header of the audio and video data to determine the coding protocol corresponding to the audio and video data when the current packet loss error correction mode is the second packet loss error correction mode;
[0185] A decoding module is used to decode the audio and video data according to the encoding protocol to play the audio and video data.
[0186] It should be noted that the device embodiment is similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the method embodiment.
[0187] An embodiment of the present invention further provides an electronic device, including:
[0188] one or more processors; and
[0189] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the device to execute the data transmission method based on visual networking as described in any of the above embodiments of the present invention.
[0190] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program that enables a processor to execute a data transmission method based on visual networking as described in any of the above embodiments of the present invention.
[0191] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0192] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0193] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0194] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0196] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0197] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0198] The above is a detailed introduction to a data transmission method, device, equipment and storage medium based on visual networking provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A data transmission method based on visual networking, It is characterized in that The method is applied to a sending end, comprising: According to the current packet loss correction mode, audio and video data are sent to the receiving end; wherein the data packet header of the audio and video data includes an error correction type frame header and a data transmission frame header; wherein the error correction type frame header is used to characterize the currently adopted packet loss correction mode, and the data transmission frame header is used to characterize the properties of the currently transmitted audio and video data packet; wherein the current packet loss correction mode is the first packet loss correction mode or the second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs; wherein, when the current packet loss correction mode is the first packet loss correction mode, the total length, total number and length of each audio and video data packet of the audio and video data packet are written into the data transmission frame header, so that the receiving end can determine the lost data packet; when the current packet loss correction mode is the second packet loss correction mode, the value in the data transmission frame header is the default value; In response to a mode switching instruction sent by the receiving end, the current packet loss error correction mode is switched, the mode switching instruction is generated by the receiving end according to the size relationship between the packet loss rate when receiving audio and video data and a preset packet loss rate threshold; wherein, switching the current packet loss error correction mode includes: when the packet loss rate is greater than or equal to the preset packet loss rate threshold, if the current network congestion level is high, switching the first packet loss error correction mode to the second packet loss error correction mode; if the current network congestion level is low and the current packet loss error correction mode is the second packet loss error correction mode, switching the second packet loss error correction mode to the first packet loss error correction mode; According to the switched packet loss error correction mode, the audio and video data are sent to the receiving end.
2. The method according to claim 1, It is characterized in that Sending audio and video data to the receiving end according to the switched packet loss error correction mode includes: According to the visual networking protocol, the audio and video data to be sent are encapsulated into multiple audio and video data packets; Adding an error correction protocol frame to the frame header data of each of the plurality of audio and video data packets to obtain a plurality of first audio and video data packets; the error correction protocol frame includes the error correction type frame header and the data transmission frame header; According to the switched packet loss error correction mode, the error correction type frame header and the data transmission frame header included in each of the plurality of first audio and video data packets are set to obtain a plurality of second audio and video data packets; The multiple second audio and video data packets are sent to a receiving end.
3. The method according to claim 2, It is characterized in that According to the switched packet loss correction mode, the error correction type frame header and the data transmission frame header included in each of the plurality of first audio and video data packets are set to obtain a plurality of second audio and video data packets, including: When the packet loss correction mode after the switch is the first packet loss correction mode, determining the total length, the total number and the length of each audio and video data packet of the plurality of first audio and video data packets; The first packet loss correction mode is written into the error correction type frame header of each of the multiple first audio and video data packets, and the total length, the total number and the length of each audio and video data packet are written into the data transmission frame header of each of the multiple first audio and video data packets to obtain the multiple second audio and video data packets.
4. The method according to claim 2, It is characterized in that According to the switched packet loss correction mode, the error correction type frame header and the data transmission frame header included in each of the plurality of first audio and video data packets are set to obtain a plurality of second audio and video data packets, including: When the packet loss correction mode after switching is the second packet loss correction mode, the error correction type frame header and the data transmission frame header of each of the multiple first audio and video data packets are set according to the second packet loss correction mode to obtain the multiple second audio and video data packets.
5. The method according to claim 4, It is characterized in that The method further comprises: According to the currently enabled encoding protocol, the audio and video data in the plurality of first audio and video data packets are encoded to obtain a plurality of first video data packets to be encapsulated; The error correction type frame header includes a coding type, and according to the second packet loss error correction mode, the error correction type frame header and the data transmission frame header of each of the plurality of first audio and video data packets are set, and the following also includes: The type of the encoding protocol is written into the encoding type of each of the multiple first audio and video data packets to be encapsulated.
6. A data transmission method based on visual networking, It is characterized in that The method is applied to a receiving end, comprising: Receive audio and video data sent by a transmitter, and parse the audio and video data to determine a current packet loss correction mode; wherein the current packet loss correction mode is a first packet loss correction mode or a second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs; the data packet header of the audio and video data includes an error correction type frame header and a data transmission frame header; wherein the error correction type frame header is used to characterize the currently adopted packet loss correction mode, and the data transmission frame header is used to characterize the properties of the currently transmitted audio and video data packet; Determine the packet loss rate of the audio and video data, and determine whether to switch the current packet loss correction mode according to the size relationship between the packet loss rate and a preset packet loss rate threshold; wherein, the determining whether to switch the current packet loss correction mode includes: when the packet loss rate is greater than or equal to the preset packet loss rate threshold, if the current network congestion level is high, then switch the first packet loss correction mode to the second packet loss correction mode; if the current network congestion level is low and the current packet loss correction mode is the second packet loss correction mode, then switch the second packet loss correction mode to the first packet loss correction mode; In the case of determining to switch the current packet loss error correction mode, sending a mode switching instruction to the sending end so that the sending end switches the current packet loss error correction mode; Receiving audio and video data sent by the transmitting end according to the switched packet loss error correction mode; Among them, when the current packet loss correction mode is the first packet loss correction mode, the total length, total number and length of each audio and video data packet of the audio and video data packet are written into the data transmission frame header of the audio and video data, so that the receiving end can determine the lost data packet; when the current packet loss correction mode is the second packet loss correction mode, the value in the data transmission frame header is the default value.
7. The method according to claim 6, It is characterized in that Parsing the audio and video data to determine the current packet loss error correction mode includes: Parsing the error correction type frame header of the audio and video data to determine the current packet loss error correction mode; Determining the packet loss rate of the audio and video data includes: When the current packet loss error correction mode is the first packet loss error correction mode, parsing the data transmission frame header of the audio and video data to obtain an expected total length of the data, where the expected total length of the data is used to represent the total length of the audio and video data sent by the sending end; Determine the packet loss rate of the audio and video data according to the total length of the audio and video data actually received and the expected total length of the data; The method further comprises: When the current packet loss error correction mode is the first packet loss error correction mode, a request is made to the sending end to retransmit the lost audio and video data.
8. The method according to claim 7, It is characterized in that The error correction type frame header also includes a coding type, and the method further includes: When the current packet loss error correction mode is the second packet loss error correction mode, parsing the coding type in the error correction type frame header of the audio and video data to determine the coding protocol corresponding to the audio and video data; The audio and video data are decoded according to the encoding protocol to play the audio and video data.
9. A data transmission device based on visual networking, It is characterized in that The device is located at the transmitting end and includes: A first data sending module is used to send audio and video data to a receiving end according to a current packet loss correction mode; wherein the data packet header of the audio and video data includes an error correction type frame header and a data transmission frame header; wherein the error correction type frame header is used to characterize the currently adopted packet loss correction mode, and the data transmission frame header is used to characterize the properties of the currently transmitted audio and video data packet; wherein the current packet loss correction mode is a first packet loss correction mode or a second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs; wherein, when the current packet loss correction mode is the first packet loss correction mode, the total length, total number and length of each audio and video data packet of the audio and video data packet are written into the data transmission frame header, so that the receiving end can determine the lost data packet; when the current packet loss correction mode is the second packet loss correction mode, the value in the data transmission frame header is a default value; A switching module, configured to switch the current packet loss correction mode in response to a mode switching instruction sent by the receiving end, wherein the mode switching instruction is generated by the receiving end according to the size relationship between the packet loss rate when receiving audio and video data and a preset packet loss rate threshold; wherein switching the current packet loss correction mode comprises: when the packet loss rate is greater than or equal to the preset packet loss rate threshold, if the current network congestion level is high, switching the first packet loss correction mode to the second packet loss correction mode; if the current network congestion level is low and the current packet loss correction mode is the second packet loss correction mode, switching the second packet loss correction mode to the first packet loss correction mode; The second data sending module is used to send audio and video data to the receiving end according to the switched packet loss error correction mode.
10. A data transmission device based on visual networking, It is characterized in that The device is located at the receiving end and includes: A first receiving module is used to receive audio and video data sent by a transmitting end, and parse the audio and video data to determine a current packet loss correction mode; wherein the current packet loss correction mode is a first packet loss correction mode or a second packet loss correction mode, the first packet loss correction mode is a protocol mode that requires retransmission when packet loss occurs, and the second packet loss correction mode is a protocol mode that does not require retransmission when packet loss occurs; the data packet header of the audio and video data includes an error correction type frame header and a data transmission frame header; wherein the error correction type frame header is used to characterize the currently adopted packet loss correction mode, and the data transmission frame header is used to characterize the properties of the currently transmitted audio and video data packet; A determination module, used to determine the packet loss rate of the audio and video data, and determine whether to switch the current packet loss error correction mode according to the size relationship between the packet loss rate and a preset packet loss rate threshold; wherein determining whether to switch the current packet loss error correction mode includes: when the packet loss rate is greater than or equal to the preset packet loss rate threshold, if the current network congestion level is high, switching the first packet loss error correction mode to the second packet loss error correction mode; if the current network congestion level is low and the current packet loss error correction mode is the second packet loss error correction mode, switching the second packet loss error correction mode to the first packet loss error correction mode; An instruction sending module, configured to send a mode switching instruction to the sending end when it is determined to switch the current packet loss error correction mode, so that the sending end switches the current packet loss error correction mode; A second receiving module, used to receive the audio and video data sent by the sending end according to the switched packet loss error correction mode; Among them, when the current packet loss correction mode is the first packet loss correction mode, the total length, total number and length of each audio and video data packet of the audio and video data packet are written into the data transmission frame header of the audio and video data, so that the receiving end can determine the lost data packet; when the current packet loss correction mode is the second packet loss correction mode, the value in the data transmission frame header is the default value.
11. An electronic device, It is characterized in that It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, it implements the data transmission method based on visual networking as described in any one of claims 1-5 or any one of claims 6-8.
12. A computer-readable storage medium, It is characterized in that The computer program stored therein enables the processor to execute the data transmission method based on visual networking as described in any one of claims 1-5 or any one of claims 6-8.
