Data interaction method and system based on private protocol

Abstract

The invention relates to the technical field of communication. The invention provides a data interaction method and system based on a private protocol. The method comprises the following steps: sending a login request subjected to private protocol encapsulation processing to a server; after the authentication is passed, establishing long connection between the sending end and the server end, so as to add a downloading processing program for appointed processing of an app request when a connection channel is established at a corresponding port; based on a private protocol, performing data enhancement on to-be-sent data through a self-defined mixed negative feedback request and specified forward error correction, and then sending the to-be-sent data to the server side, specifically comprising the following steps: classifying a serial number for each data packet to detect whether the packet is lost or not; when packet loss of the to-be-sent data is determined, delay calculation of data recovery is carried out; and after the packet loss recovery processing is executed, receiving response information of a stream pushing result or a stream pulling result from the server. According to the invention, a more optimized data interaction method is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of communications and provides a data interaction method and system based on a private protocol. Background Art

[0002] The application of network cameras is becoming more and more extensive. How to make full use of resources to manage more cameras, and at the same time adapt to various new business scenarios quickly and conveniently is a problem that service providers are concerned about. In the prior art, camera devices use the streaming function of RTMP (RealTime Messaging Protocol). When the camera communicates with the server and the app, there are two-way communication and calling scenarios. If other signaling interactions and information transmission are involved in streaming, the real-time messaging protocol is not applicable, and other protocols need to be used for interaction. Since the RTMP protocol is a standardized protocol, camera and app login requests or app streaming requests cannot be used directly. In addition, due to the standardization of the RTMP protocol, there is a risk of being easily attacked and analyzed. In addition, different protocols require different parsers, and parsing adaptation must be done, generating a large amount of redundant code.

[0003] In addition, there is still much room for improvement in many aspects, such as how to reduce latency, improve transmission efficiency, and how to increase the number of camera connections under the same CPU load.

[0004] Therefore, it is necessary to provide a data interaction method based on a private protocol to solve the above problems. Summary of the invention

[0005] The present invention provides a data interaction method and system based on a private protocol to solve the technical problems in the prior art that when a camera communicates with a server and an app, there are two-way communication and calling scenarios, and when streaming involves other signaling interactions and information transmission, the camera and app login requests or the app streaming requests cannot directly use the RTMP protocol. When using the RTMP protocol, there is a risk of being easily attacked and analyzed due to its standardization. Different protocols require different parsers for parsing and adaptation, resulting in the generation of a large amount of redundant code, etc. The technical problem to be solved by the present invention is achieved through the following technical solutions.

[0006] The first aspect of the present invention proposes a data interaction method based on a private protocol, the method comprising: sending a login request encapsulated by a private protocol to a server; after authentication, establishing a long connection between the sending end and the server, so as to add a download processing program for processing app requests when establishing a connection channel at the corresponding port; based on the private protocol, data enhancement is performed on the data to be sent through a customized hybrid negative feedback request and a specified forward error correction, and then the data is sent to the server, specifically comprising: classifying a sequence number for each data packet to detect whether a packet is lost; when it is determined that the data to be sent is lost, performing a delay calculation for data recovery; after performing packet loss recovery processing, receiving response information of the push stream result or the pull stream result from the server.

[0007] The second aspect of the present invention proposes a data interaction system based on a private protocol, which executes the data interaction method based on a private protocol described in the first aspect of the present invention. The data interaction system includes: a first sending module, which sends the login request encapsulated by the private protocol to the server; a connection processing module, after authentication, enables the sending end to establish a long connection with the server, so as to add a download processing program for processing app requests when establishing a connection channel on the corresponding port; a second sending module, based on the private protocol, performs data enhancement on the data to be sent through a customized hybrid negative feedback request and specified forward error correction, and then sends it to the server, specifically including: classifying a sequence number for each data packet to detect whether it is lost; when it is determined that the data to be sent is lost, performing a delay calculation for data recovery; a receiving processing module, after performing packet loss recovery processing, receives response information of the push stream result or the pull stream result from the server.

[0008] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the data interaction method based on the private protocol described in the first aspect of the present invention.

[0009] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data interaction method based on a private protocol as described in the first aspect of the present invention.

[0010] The embodiments of the present invention include the following advantages:

[0011] Compared with the prior art, the present invention uses a private protocol to achieve two-way communication, freely designs the data transmission format, makes the protocol more flexible, and serializes the data in binary format to make the data packets to be sent smaller, improve the transmission efficiency, reduce the bandwidth usage and transmission delay; the private protocol is more secure.

[0012] In addition, the server monitors different ports for camera push streaming and app pull streaming, and uses ports to map with upstream business scenarios, so that the business logic is differentiated on the port, simplifying the system design. The camera push stream is continuous media packet data, while the App traffic is sometimes intermittent requests. By using different ports, the server can be optimized according to their respective traffic patterns to improve overall performance.

[0013] In addition, using non-blocking IO, it can handle a large number of concurrent connections without performance degradation due to blocking. Limited thread resources can handle a large number of IO operations, and the request processing delay is low. By establishing a connection channel and inserting different handlers, it can flexibly handle custom protocols for encoding and decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the steps of the data interaction method based on the private protocol of the present invention;

[0015] Figure 2 It is a schematic diagram of a framework of an application example of the data interaction method based on a private protocol of the present invention;

[0016] Figure 3 It is a partial detail schematic diagram of an application example of the data interaction method based on a private protocol of the present invention;

[0017] Figure 4 It is a flowchart of a specific implementation method of the data interaction method based on the private protocol of the present invention;

[0018] Figure 5 is a schematic diagram of a specific example of a private protocol in the data interaction method based on a private protocol of the present invention;

[0019] Figure 6 It is a structural block diagram of the data interaction system based on the private protocol of the present invention;

[0020] Figure 7 is a schematic structural diagram of an electronic device embodiment according to the present invention;

[0021] Figure 8 It is a schematic diagram of the structure of a computer-readable medium embodiment according to the present invention. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In view of the above problems, the present invention proposes a data interaction method based on a private protocol. The method sends a login request encapsulated by a private protocol to a server; after authentication, the sending end establishes a long connection with the server, so as to add a download processing program for processing app requests when establishing a connection channel on the corresponding port, and based on the private protocol, the data to be sent is enhanced by a customized hybrid negative feedback request and a specified forward error correction, and then sent to the server, wherein each data packet is classified into a sequence number to detect whether a packet is lost. When it is determined that the data to be sent is lost, a delay calculation for data recovery is performed, which can effectively handle the packet loss problem and reduce the delay time of data recovery. After performing the packet loss recovery process, the response information of the push stream result or the pull stream result from the server is received, which can more effectively recover the lost data.

[0024] Example 1

[0025] Refer to the following Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 , the contents of the present invention will be described in detail.

[0026] Figure 1 It is a step flow chart of the data interaction method based on the private protocol of the present invention. Figure 2 It is a schematic diagram of a framework of an application example of the data interaction method based on a private protocol of the present invention.

[0027] like Figure 1 As shown, in step S101, the login request encapsulated by the private protocol is sent to the server.

[0028] exist Figure 2 In the application example, the server imports Netty-related dependencies and declares the server class. There are multiple ServerBootstraps under the same server class, which listen to different ports to deal with different business scenarios. Among them, ServerBootstrap is a tool used to create a connection channel for the server. For example, the business scenarios include scenarios for interacting with cameras and scenarios for interacting with apps. When watching with a mobile phone, pull the content of the camera and pull the content of the app. Access rights can be set for different ports separately. For example, the internal control port can limit access to specific IPs. Specifically, port 1 is used internally, port 2 is used to interact with apps, and port 3 is used to interact with cameras. The ports are mapped to the upstream business scenarios, so that the business logic is distinguished on the ports, which can effectively distinguish the business and facilitate troubleshooting. For example, the internal access status can be obtained by detecting the access status of port 1.

[0029] Specifically, it includes a server, a camera that can communicate with the server, and a client. The server includes one or more servers. The server acts as a relay for data communication between the camera and the client (i.e., camera <----> server <----> app), and provides different ports for the camera and the client (specifically, Figure 2 For example, requests initiated by the camera, such as pushing streams, pushing pictures, and pushing events, use port A, while the app pulls streams and sends signals using port C.

[0030] Furthermore, the client and the server can pre-set the protocol content, such as the media packet field, signaling field, and the corresponding meanings of the various field contents. For example, MessageType = 1 (message type) is a login request, 2 is a server (server) response, 3 and 4 are ping / pong (heartbeat), 5 is a media packet..., etc. For details, please refer to Figure 3 .

[0031] For example, when the server receives a private protocol packet, it first decodes the message type = 1, that is, it knows that it is a login request, obtains the content in the request to perform login logic processing, and then encapsulates the result into a response packet with message type = 2, writes the response content in the response, and returns it to the client.

[0032] Optionally, the server also includes a port B that matches the internal interface. The port B can be used as a communication interface between the internal interface and the camera. The server can also provide, for example, streaming media services, protocol conversion services, playback services or other services.

[0033] In addition, the server includes various handlers corresponding to various business processes. The server is configured with a server boot program to listen to different ports. For example, various configurations of the server are loaded when Springboot is booted. After the Ioc container is initialized, the server is obtained and started. Then, the server reads the corresponding configuration, and each server boot program listens to its own port, ready to manage and maintain the connection between the camera and the server, and the connection between the client and the server.

[0034] It should be noted that the client generally refers to a device capable of live broadcast or watching live broadcast (such as including a live broadcast app), which can be a smart phone or tablet computer, but is not limited to this.

[0035] In a specific embodiment, Figure 4 As shown, the login request after the private protocol encapsulation is sent to the server.

[0036] Specifically, a login request is sent from a sending end to a server end, where the sending end is a camera or a client end.

[0037] For example, after the camera device is serialized based on protobuf (for example, in binary format), it sends a signal to the corresponding port (specifically Figure 2 Port A) sends a login request with MessageType = 1. Through binary format serialization, the data packet to be sent is smaller, the transmission efficiency is higher, and the bandwidth usage and transmission delay time are reduced.

[0038] Specifically, the login request is a login request encapsulated and processed using a private protocol. The encapsulation format of the private protocol for various requests and data to be sent is protocol content length and protocol content, and the protocol content includes file package type, data packet type, version identifier, sequence number, timestamp, effective load length, effective load, redundant data, and packet tail information. More specifically, the format of length+Protobuf_data is adopted, and length is used to represent the network byte order. Protobuf_data is used to represent the private protocol content.

[0039] Optionally, the data processed using the private protocol includes all data including signaling and media packets.

[0040] The processed login request is encapsulated using a private protocol. After successful login, a long connection is established between the camera and the server to further enhance the streaming data waiting to be sent. Specifically, the private protocol is used to enhance the data through a customized hybrid negative feedback request (NACK) and a specified forward error correction (FEC).

[0041] In addition, the server also listens to the server bootstrap (ServerBootstrap) of port A, which is responsible for processing the camera's login request and adding an upload handler (UploadHandler) that specifically processes the camera's login request on the established connection channel. The server determines whether to provide services based on the load capacity and the number of connections, and performs authentication. The connection data specifically refers to the number of connected cameras. The number of connections is determined in order to maintain the number of connections in the memory (in order to ensure the load capacity). When a camera is connected, the number of connections increases by one. When a camera is offline, the number of connections decreases by one.

[0042] In addition, for port A, all interactions between the camera and the server are implemented using the upload handler. After logging in, the camera and the server establish a long connection. The camera reports streaming data (media packets), the camera reports heartbeats, and the server sends signals (such as making the camera sound or rotating) through this long connection.

[0043] It should be noted that, in fact, the upload processing program is responsible for a lot of work, which will not be described in detail here. The above is only explained as an optional example and should not be understood as a limitation of the present invention.

[0044] Next, in step S102, after authentication, the sending end establishes a long connection with the server, so as to add a download processing program for processing the app request when establishing a connection channel on the corresponding port.

[0045] Specifically, the server authenticates the client ID, user device ID, user information, etc. in the login request based on the received login request. If the authentication is successful, a response message indicating that the authentication is successful is returned to the sender. If the authentication is not successful, a response message indicating that the authentication is not successful is returned to the sender and the session is terminated. For details, see Figure 4 .

[0046] In an optional implementation, when the sending end is a camera, after authentication, a long connection is established between the camera and the server, so that a download processing program for processing app requests is added when a connection channel is established on the corresponding port. The camera sends a heartbeat packet (such as ping) based on a private protocol at a specified interval. When the server receives the heartbeat packet, it returns a response message (such as pong) to the camera. When the server does not receive the heartbeat packet, the corresponding memory is released.

[0047] In another optional implementation, when the sending end is a client, after authentication, the client establishes a long connection with the server, so that a download processing program for processing app requests is added when a connection channel is established on the corresponding port. Further, when the sending end is a client and the client is connected to the server, a port corresponding to the client is requested, and when a communication channel is established, a download processing program for processing app requests is added, so that data communication is realized by transparently transmitting signaling to the camera through a negotiated protocol.

[0048] It should be noted that the above is only described as an optional example and should not be understood as a limitation to the present invention.

[0049] Next, in step S103, based on a private protocol, the data to be sent is enhanced through a customized hybrid negative feedback request and specified forward error correction and then sent to the server, specifically including: classifying the sequence number for each data packet to detect whether it is lost; when it is determined that the data to be sent is lost, performing delay calculation for data recovery.

[0050] like Figure 4As shown, the app sends a signal to the current camera through the server, and the server parses the signal, adds processing signal, and sends the processing information to the current camera, that is, the signal is sent successfully.

[0051] When the signal is successfully sent, the camera, based on a private protocol, uses a custom hybrid negative feedback request and specified forward error correction to enhance the data to be sent, and then sends it to the server, which then sends it to the downstream client, indicating that the streaming is successful.

[0052] Specifically, a sequence number is classified for each data packet to detect whether a packet is lost. When it is determined that the data to be sent is lost, a delay calculation for data recovery is performed.

[0053] When the receiving end detects packet loss, it dynamically controls the use of a customized hybrid negative feedback request or specifies forward error correction based on the network conditions.

[0054] When the packet loss rate is greater than the preset value, the specified forward error correction is used first.

[0055] Specifically, the server calculates the packet loss rate through the packet loss rate of the heartbeat packet. The heartbeat packet has a sequence number, and the packet loss rate = (the number of Ping packets that have not received a response / the total number of Ping packets sent) × 100%.

[0056] In addition, the private protocol also encapsulates the timestamp, and the server can obtain the actual delay through the difference between the local timestamp and the protocol packet timestamp.

[0057] When the calculated delay is RTT and the packet loss rate is p, the following expression exists:

[0058] t 重传 =RTT*1 / (1-p)

[0059] Among them, t 重传 It represents the time required for retransmission calculated by the server; RTT represents the calculated delay as RTT; p represents the packet loss rate detected by the receiving end.

[0060] t 前向纠错 =C 编码 *n+C 解码 *(n+r)

[0061] Among them, t 前向纠错 Indicates the time required to use the specified forward error correction calculated by the server; C 编码 represents the encoding complexity, n represents the original packet size, C 解码 represents the decoding complexity, and r represents the redundant packet size.

[0062] Specifically, the server calculates t 重传and t 前向纠错 , compare t 重传 and t 前向纠错 The size of is used to determine whether to use retransmission or forward error correction, that is, dynamically control the use of a customized hybrid negative feedback request (corresponding to retransmission) or the use of specified forward error correction.

[0063] In another embodiment, the packet loss threshold and the maximum threshold may be written in the configuration file, and the actual packet loss rate is compared with the packet loss threshold in the configuration. When the actual packet loss rate is greater than the packet loss threshold, the specified forward error correction is used. When the actual packet loss rate is greater than the packet loss threshold and greater than the maximum threshold, the specified forward error correction is used. When the actual packet loss rate is less than or equal to the packet loss threshold, resending is used.

[0064] When the delay time from the sending end to the receiving end is less than the preset time, the customized mixed negative feedback request is used preferentially.

[0065] In an optional implementation, when the receiving end detects packet loss and determines to use a custom hybrid negative feedback request, the receiving end immediately sends the custom hybrid negative feedback request to the sending end. When the custom hybrid negative feedback request is used, the following expression is used to calculate the total retransmission delay RTD:

[0066]

[0067] Wherein, RTD represents the total retransmission delay RTD calculated when the customized hybrid negative feedback request is used; p is the packet loss rate; EED is the end-to-end total delay, that is, from the sending end to the receiving end, which is specifically calculated using the following expression:

[0068]

[0069] Among them, EED is the total delay from end to end, that is, from the sender to the receiver; is the network transmission delay, also represented by PD, d is the distance between the sender and the receiver, and v is the transmission speed from the sender to the receiver; It represents the transmission delay from the sender to the receiver, also represented by TD. represents the queuing delay, L represents the size of a data packet in the waiting-to-be-sent data, R represents the bandwidth, and N is the number of data packets in the waiting-to-be-sent queue; t p1 Indicates program processing delay, the sending end includes a camera and a client, and the receiving end includes a server.

[0070] In another optional implementation, when the receiving end detects packet loss and determines to use the specified forward error correction, the following expression is used to calculate the delay time:

[0071] FECD=EED+t p2 ,

[0072] Where FECD represents the delay time calculated when using the specified forward error correction; t p2 Indicates the calculated delay of packet loss recovery; EED is the end-to-end total delay, that is, from the sender to the receiver, and is calculated using the following expression:

[0073]

[0074] Among them, EED is the total delay from end to end, that is, from the sender to the receiver; is the network transmission delay, also represented by PD, d is the distance between the sender and the receiver, and v is the transmission speed from the sender to the receiver; It represents the transmission delay from the sender to the receiver, also represented by TD. represents the queuing delay, L represents the size of a data packet in the waiting-to-be-sent data, R represents the bandwidth, and N is the number of data packets in the waiting-to-be-sent queue; t p1 Indicates a delay in program processing.

[0075] It should be noted that the specified forward error correction (FEC) specifically refers to adding redundant packets when sending the data to be sent, so that even if the receiving end (specifically the server) loses some data packets, it can recover the lost data through the redundant information in the redundant packets. In this case, the server recovers the lost packets through the received original data packets and the redundant packets without requesting retransmission.

[0076] The customized hybrid negative feedback request means that when the sender (such as a camera) sends data to be sent, it assigns a sequence number to each packet (i.e., multiple data packets formed by the data to be sent) and caches the packets sent within a certain period of time (such as 5s to 20s) for subsequent retransmission.

[0077] Specifically, when the receiving end detects packet loss (for example, the received sequence number is discontinuous), it immediately sends a custom hybrid negative feedback request (i.e., NACK request). After the sending end (such as a camera) receives the NACK request, it finds the corresponding lost packets according to the sequence number in the NACK message and resends these lost packets.

[0078] The data to be sent is encapsulated according to a private protocol before being sent. The data encapsulation format of the private protocol is the protocol content length and the protocol content. The protocol content includes the file package type, data packet type, version identifier, serial number, timestamp, effective load length, effective load, redundant data, and packet tail information.

[0079] For example, if the data to be sent is a media packet (for example, represented by MediaPackage), in addition to streaming media data, it can also include packet type, message type, timestamp, etc. For details, see Figure 5 After the media package is uploaded to the server, it can be transparently transmitted to the app or object storage services. When the app connects to the server, it requests the response port and adds a download handler (i.e. DownloadHandler) dedicated to handling app requests when establishing the connection channel. After the app establishes a communication connection with the server, it can transparently transmit the processing signaling to the camera through the negotiated protocol to achieve two-way communication.

[0080] Through customized hybrid negative feedback request and specified forward error correction, the data to be sent is enhanced before sending. The customized hybrid negative feedback request and specified forward error correction are controlled and adjusted according to the network conditions. Specifically, the specified forward error correction (expressed by FEC) is used when there is high packet loss, and the use of NACK is reduced when there is high latency, that is, the specified forward error correction is used. The business is used to make a distinction directly: only FEC is used for live streaming, and only NACK is used for video playback. During live streaming, FEC can immediately recover lost data through redundant packets, without waiting for retransmission and introducing no additional delay. The playback recovers data through NACK, without the need for additional redundant packets, which can save bandwidth.

[0081] It should be noted that the above is only described as an optional example and should not be understood as a limitation to the present invention.

[0082] Next, in step S104, after performing the packet loss recovery process, response information of the streaming result or the streaming result is received from the server.

[0083] Specifically, when the camera is live streaming, only the specified forward error correction is used, and the lost data can be immediately recovered through the redundant packets without waiting for retransmission. When the lost data is recovered through the redundant packets, the successful streaming result is sent to the sender. If the lost data is not recovered, the packet loss recovery process is repeated until the lost data is recovered through the redundant packets.

[0084] For video playback, only a customized hybrid negative feedback request is used to recover lost data without redundant packets.

[0085] After the lost data is recovered through the customized hybrid negative feedback request and the packet loss recovery process is performed, a streaming result indicating successful streaming is sent to the sending end.

[0086] In another specific implementation, when a mobile phone APP wants to play the video of the current camera or the historical video, the mobile phone pulls the video content, i.e., pulls the stream, from the corresponding port of the server.

[0087] It should be noted that the above is only described as an optional example and should not be understood as a limitation to the present invention.

[0088] Compared with the prior art, the present invention sends the login request after private protocol encapsulation to the server; after authentication, the sending end establishes a long connection with the server, so as to add a download processing program for processing the app request when the connection channel is established on the corresponding port, and based on the private protocol, the data to be sent is enhanced by a customized hybrid negative feedback request and a specified forward error correction, and then sent to the server, wherein each data packet is classified into a sequence number to detect whether the packet is lost, and when it is determined that the data to be sent is lost, a delay calculation for data recovery is performed, which can effectively handle the packet loss problem and reduce the delay time of data recovery. After performing the packet loss recovery process, the response information of the push stream result or the pull stream result from the server is received, which can more effectively recover the lost data.

[0089] Example 2

[0090] The following is a system embodiment of the present invention, which can be used to implement the method embodiment of the present invention. For details not disclosed in the system embodiment of the present invention, please refer to the method embodiment of the present invention.

[0091] Figure 6 1 is a schematic diagram of an example of a data interaction system based on a private protocol according to the present invention. Figure 6 , a data interaction system based on a private protocol is described. The data interaction system is used to execute the data interaction method described in the first aspect of the present invention.

[0092] like Figure 6 As shown, the data interaction system 600 includes a first sending module 610 , a connection processing module 620 , a second sending module 630 and a receiving processing module 640 .

[0093] In a specific implementation, the first sending module 610 sends the login request after private protocol encapsulation to the server. The connection processing module 620, after authentication, enables the sending end to establish a long connection with the server, so as to add a download handler for processing app requests when establishing a connection channel on the corresponding port. The second sending module 630, based on a private protocol, performs data enhancement on the data to be sent through a customized hybrid negative feedback request and specified forward error correction, and then sends it to the server, specifically including: classifying a sequence number for each data packet to detect whether there is packet loss; when it is determined that the data to be sent has packet loss, a delay calculation for data recovery is performed. After performing packet loss recovery processing, the receiving processing module 640 receives response information on the push stream result or the pull stream result from the server.

[0094] According to an optional implementation, when the receiving end detects packet loss, it dynamically controls the use of a custom hybrid negative feedback request or a specified forward error correction based on the network conditions; when the packet loss rate is greater than a preset value, the specified forward error correction is used preferentially; when the delay time from the sending end to the receiving end is less than a predetermined time, the custom hybrid negative feedback request is used preferentially.

[0095] According to an optional implementation manner, when the receiving end detects packet loss and determines to use a custom hybrid negative feedback request, the custom hybrid negative feedback request is immediately sent to the sending end. When the custom hybrid negative feedback request is used, the following expression is used to calculate the total retransmission delay RTD:

[0096]

[0097] Wherein, RTD represents the total retransmission delay RTD calculated when the customized hybrid negative feedback request is used; p is the packet loss rate; EED is the end-to-end total delay, that is, from the sending end to the receiving end, which is specifically calculated using the following expression:

[0098]

[0099] Among them, EED is the total delay from end to end, that is, from the sender to the receiver; is the network transmission delay, also represented by PD, d is the distance between the sender and the receiver, and v is the transmission speed from the sender to the receiver; It represents the transmission delay from the sender to the receiver, also represented by TD. represents the queuing delay, L represents the size of a data packet in the waiting-to-be-sent data, R represents the bandwidth, and N is the number of data packets in the waiting-to-be-sent queue; t p1 Indicates program processing delay, the sending end includes a camera and a client, and the receiving end includes a server.

[0100] According to an optional implementation manner, when the receiving end detects packet loss and determines to use the specified forward error correction, the following expression is used to calculate the delay time:

[0101] FECD=EED+t p2 ,

[0102] Where FECD represents the delay time calculated when using the specified forward error correction; t p2 Indicates the calculation delay of data recovery for packet loss data; EED is the total end-to-end delay, that is, from the sender to the receiver, and is calculated using the following expression:

[0103]

[0104] Among them, EED is the total delay from end to end, that is, from the sender to the receiver; is the network transmission delay, also represented by PD, d is the distance between the sender and the receiver, and v is the transmission speed from the sender to the receiver; It represents the transmission delay from the sender to the receiver, also represented by TD. represents the queuing delay, L represents the size of a data packet in the waiting-to-be-sent data, R represents the bandwidth, and N is the number of data packets in the waiting-to-be-sent queue; t p1 Indicates program processing delay, the sending end includes a camera and a client, and the receiving end includes a server.

[0105] According to an optional implementation manner, the data to be sent is encapsulated according to a private protocol before being sent, and the data encapsulation format of the private protocol is the protocol content length and the protocol content. The protocol content includes the file package type, the data packet type, the version identifier, the serial number, the timestamp, the effective load length, the effective load, the redundant data, and the packet tail information.

[0106] According to an optional implementation manner, the server can serve as a relay for data communication between the camera and the client, and provide different ports for the camera and the client respectively. The server is configured with a server boot program for monitoring different ports.

[0107] When the sending end is a camera, the camera sends a heartbeat packet based on a private protocol at specified intervals. When the server receives the heartbeat packet, it returns a response message to the camera. When the server does not receive the heartbeat packet, it releases the corresponding memory.

[0108] When the sender is a client, when the client connects to the server, it requests the port corresponding to the client, and when establishing the communication channel, adds a download handler that specifies the processing of app requests, so that the signaling is transparently transmitted to the camera through the negotiated protocol to achieve data communication.

[0109] According to an optional implementation manner, when the receiving end detects packet loss, dynamically controlling the use of a customized hybrid negative feedback request or specifying forward error correction according to network conditions also includes:

[0110] When the calculated delay is RTT and the packet loss rate is p, the following expression exists:

[0111] t 重传 =RTT*1 / (1-p)

[0112] Among them, t 重传 It represents the time required for retransmission calculated by the server; RTT represents the calculated delay as RTT; p represents the packet loss rate detected by the receiving end.

[0113] t 前向纠错=C 编码 *n+C 解码 *(n+r)

[0114] Among them, t 前向纠错 Indicates the time required to use the specified forward error correction calculated by the server; C 编码 represents the encoding complexity, n represents the original packet size, C 解码 represents the decoding complexity, and r represents the redundant packet size.

[0115] Specifically, the server calculates t 重传 and t 前向纠错 , and compare t 重传 and t 前向纠错 The size of the bit is used to determine whether to use retransmission or forward error correction, that is, dynamically control the use of a customized hybrid negative feedback request or the use of specified forward error correction.

[0116] It should be noted that due to Figure 6 Data interaction method and data interaction system executed by Figure 1 The data interaction methods in the examples are roughly the same, so the description of the same parts is omitted.

[0117] Compared with the prior art, the present invention sends the login request after private protocol encapsulation to the server; after authentication, the sending end establishes a long connection with the server, so as to add a download processing program for processing the app request when the connection channel is established on the corresponding port, and based on the private protocol, the data to be sent is enhanced by a customized hybrid negative feedback request and a specified forward error correction, and then sent to the server, wherein each data packet is classified into a sequence number to detect whether the packet is lost, and when it is determined that the data to be sent is lost, a delay calculation for data recovery is performed, which can effectively handle the packet loss problem and reduce the delay time of data recovery. After performing the packet loss recovery process, the response information of the push stream result or the pull stream result from the server is received, which can more effectively recover the lost data.

[0118] Figure 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

[0119] like Figure 7 As shown, the electronic device is presented in the form of a general computing device. The processor may be one or more and work in coordination. The present invention does not exclude distributed processing, that is, the processor may be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0120] The memory stores a computer executable program, which is usually a machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least part of the steps in the method.

[0121] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0122] Optionally, in this embodiment, the electronic device further includes an I / O interface, which is used for the electronic device to exchange data with an external device. The I / O interface can represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0123] It should be understood that Figure 7 The electronic device shown is only an example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements such as buttons, keyboards, etc. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least part of the steps of the method, it can be considered as an electronic device covered by the present invention.

[0124] Through the above description of the implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by combining software with necessary hardware. Figure 8 As shown, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of commands to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiment of the present invention.

[0125] The software product may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0126] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by a command execution system, an apparatus, or a device or for use in combination with the program. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0127] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0128] The computer-readable medium carries one or more programs. When the one or more programs are executed by a device, the computer-readable medium implements the data interaction method of the present disclosure.

[0129] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0130] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several commands to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.

[0131] It should be noted that the above detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present application belongs.

[0132] In the above detailed description, reference is made to the accompanying drawings, which form a part of this document. In the accompanying drawings, similar symbols typically identify similar components unless the context indicates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be used, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data interaction method based on a private protocol, characterized in that: The data interaction method comprises: Send the login request after private protocol encapsulation to the server; After authentication, the sender establishes a persistent connection with the server, and adds a download handler for app requests when establishing a connection channel on the corresponding port; Based on a private protocol, the data to be sent is enhanced by a customized hybrid negative feedback request and a specified forward error correction, and then sent to the server, specifically including: classifying the sequence number of each data packet to detect whether it is lost; when it is determined that the data to be sent is lost, delay calculation of data recovery is performed; After performing the packet loss recovery process, a response message of the streaming result or the streaming result is received from the server.

2. The data interaction method based on a private protocol according to claim 1, characterized in that: When it is determined that packet loss occurs in the data to be sent, delay calculation for data recovery is performed, including: When the receiving end detects packet loss, it dynamically controls the use of a custom hybrid negative feedback request or specifies forward error correction based on the network conditions; When the packet loss rate is greater than the preset value, the specified forward error correction is used first; When the delay time from the sending end to the receiving end is less than the preset time, the customized mixed negative feedback request is used preferentially.

3. The data interaction method based on a private protocol according to claim 2, characterized in that: Further including: When the receiving end detects packet loss and determines to use a custom hybrid negative feedback request, the receiving end immediately sends the custom hybrid negative feedback request to the sending end. When the custom hybrid negative feedback request is used, the following expression is used to calculate the total retransmission delay RTD: Wherein, RTD represents the total retransmission delay RTD calculated when the customized hybrid negative feedback request is used; p is the packet loss rate; EED is the end-to-end total delay, that is, from the sending end to the receiving end, which is specifically calculated using the following expression: Among them, EED is the total delay from end to end, that is, from the sender to the receiver; is the network transmission delay, also represented by PD, d is the distance between the sender and the receiver, and v is the transmission speed from the sender to the receiver; It represents the transmission delay from the sender to the receiver, also represented by TD. represents the queuing delay, L represents the size of a data packet in the waiting-to-be-sent data, R represents the bandwidth, and N is the number of data packets in the waiting-to-be-sent queue; t p1 Indicates program processing delay, the sending end includes a camera and a client, and the receiving end includes a server.

4. The data interaction method based on a private protocol according to claim 2, characterized in that: Further including: When the receiving end detects packet loss and determines to use the specified forward error correction, the following expression is used to calculate the delay time: FECD=EED+t p2 , Where FECD represents the delay time calculated when using the specified forward error correction; t p2 Indicates the calculation delay of data recovery for packet loss data; EED is the total end-to-end delay, that is, from the sender to the receiver, and is calculated using the following expression: Among them, EED is the total delay from end to end, that is, from the sender to the receiver; is the network transmission delay, also represented by PD, d is the distance between the sender and the receiver, and v is the transmission speed from the sender to the receiver; It represents the transmission delay from the sender to the receiver, also represented by TD. represents the queuing delay, L represents the size of a data packet in the waiting-to-be-sent data, R represents the bandwidth, and N is the number of data packets in the waiting-to-be-sent queue; t p1 Indicates program processing delay, the sending end includes a camera and a client, and the receiving end includes a server.

5. The data interaction method based on a private protocol according to claim 1, characterized in that: include: The data to be sent is encapsulated according to a private protocol before being sent. The data encapsulation format of the private protocol is the protocol content length and the protocol content. The protocol content includes the file package type, data packet type, version identifier, serial number, timestamp, effective load length, effective load, redundant data, and packet tail information.

6. The data interaction method based on a private protocol according to claim 1, characterized in that: include: The server can be used as a relay for data communication between the camera and the client, and provide different ports for the camera and the client respectively; The server is configured with a server boot program for monitoring different ports.

7. The data interaction method based on a private protocol according to claim 1, characterized in that: include: When the sending end is a camera, the camera sends a heartbeat packet based on a private protocol at a specified interval. When the server receives the heartbeat packet, it returns a response message to the camera. When the server does not receive the heartbeat packet, it releases the corresponding memory. When the sender is a client, when the client connects to the server, it requests the port corresponding to the client, and when establishing the communication channel, adds a download handler that specifies the processing of app requests, so that the signaling is transparently transmitted to the camera through the negotiated protocol to achieve data communication.

8. The data interaction method based on a private protocol according to claim 2, characterized in that: When the receiving end detects packet loss, dynamically controlling the use of a customized hybrid negative feedback request or specifying forward error correction according to network conditions, further comprising: When the calculated delay is RTT and the packet loss rate is p, the following expression exists: t 重传 =RTT*1 / (1-p) Among them, t 重传 represents the time required for retransmission calculated by the server; RTT represents the calculated delay as RTT; p represents the packet loss rate detected by the receiving end; t 前向纠错 =C 编码 *n+C 解码 *(n+r) Among them, t 前向纠错 Indicates the time required to use the specified forward error correction calculated by the server; C 编码 represents the encoding complexity, n represents the original packet size, C 解码 represents the decoding complexity, r represents the redundant packet size; Specifically, the server calculates t 重传 and t 前向纠错 , and compare t 重传 and t 前向纠错 The size of the bit is used to determine whether to use retransmission or forward error correction, that is, dynamically control the use of a customized hybrid negative feedback request or the use of specified forward error correction.

9. A data interaction system based on a private protocol, characterized in that: The data interaction method according to any one of claims 1 to 8 is executed, and the data interaction system comprises: The first sending module sends the login request encapsulated by the private protocol to the server; The connection processing module, after authentication, enables the sender to establish a long connection with the server, so as to add a download processing program for processing the app request when establishing a connection channel on the corresponding port; The second sending module, based on a private protocol, performs data enhancement on the data to be sent through a customized hybrid negative feedback request and a specified forward error correction, and then sends it to the server, specifically including: classifying the sequence number of each data packet to detect whether the packet is lost; when it is determined that the data to be sent is lost, performing delay calculation for data recovery; The receiving and processing module receives response information of the streaming result or the streaming result from the server after executing the packet loss recovery process.

10. The data interaction system based on a private protocol according to claim 9, characterized in that: include: The encapsulation and sending module is used to send the login request after the private protocol encapsulation processing to the server; The connection module, after passing the authentication, enables the sender to establish a long connection with the server, so as to add a download processing program for processing the app request when establishing the connection channel on the corresponding port; The data enhancement module, based on a private protocol, performs data enhancement on the data to be sent through a customized hybrid negative feedback request and a specified forward error correction, and then sends it to the server. Specifically, it includes: classifying the sequence number of each data packet to detect whether it is lost; when it is determined that the data to be sent is lost, performing delay calculation for data recovery; After performing the packet loss recovery process, a response message of the streaming result or the streaming result is received from the server.

Images