Real-time communication processing method, terminal device, storage medium and program product

Abstract

The invention discloses a real-time communication processing method, terminal equipment, a storage medium and a program product. The sending end extracts data frames from the collected streaming media data in real time; calculating a hash value corresponding to the data frame, and calculating a corresponding signature by using a private key of the sending end when the data frame is a key frame; the original data used for calculation comprises a data frame and a hash value corresponding to the previous data frame; and then sending the data frame and the corresponding hash value and signature to a receiving end. And after receiving the data frames, the corresponding hash values and the signatures, a receiving end respectively verifies the data frames, the corresponding hash values and the signatures, and renders and plays the corresponding data frames after the verification is passed. By adopting the scheme, the data frame in the data stream sent by the sending end during real-time communication has relatively strong tamper-proof capability and relatively good verifiability, and the receiving end can effectively verify the integrity of the content and the authenticity of the source, so that real and reliable information is displayed for a user at the receiving end, and the security guarantee for the user during real-time communication is improved.

Description

Technical Field

[0001] The present application relates to the field of real-time communication technology, and in particular to a real-time communication processing method applied to a transmitter and a corresponding real-time communication processing method applied to a receiver. In addition, the present application also relates to a corresponding terminal device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Real-time communication generally refers to the process of instantly exchanging information between two or more participants without significant delay. During real-time communication (such as video calls, voice calls, and online live broadcasts), due to the openness of network transmission, real-time communication data is at risk of being stolen, tampered with, or forged.

[0003] Existing technologies mainly rely on the security mechanism of the transmission protocol (such as TLS / DTLS) to protect the security of the communication channel, which to a certain extent guarantees the security of the data transmitted by real-time communication. However, once the security mechanism is hacked by an attacker, the attacker can still tamper with and forge the data in the real-time communication process, causing the receiving end user to be misled or even deceived by the received non-authentic information. Summary of the invention

[0004] One purpose of the present application is to provide a real-time communication processing method, in which the sending end extracts data frames from the streaming media data in real time and processes them, so that the data frames have strong anti-tampering capabilities and good verifiability. In this way, the receiving end can effectively verify the integrity and source authenticity of the streaming media data content itself, thereby displaying real and reliable information to the receiving end user.

[0005] The first aspect of the present application provides a real-time communication processing method, which is applied to a sending end, and the method includes: extracting a current data frame from collected streaming media data in real time; calculating a hash value corresponding to the current data frame; wherein the original data used to calculate the hash value includes: the current data frame, and the hash value corresponding to the previous data frame of the current data frame; when the current data frame is a key frame, using the private key of the sending end to calculate the signature corresponding to the key frame, wherein the original data used to calculate the signature includes: the key frame, and the hash value corresponding to the previous data frame of the key frame; the private key of the sending end is used to identify the identity of the sending end or to identify the identity of a user using the sending end; sending the current data frame and the hash value corresponding to the current data frame to a receiving end; when the current data frame is a key frame, also sending the signature corresponding to the key frame to the receiving end.

[0006] In combination with the first aspect, in a possible implementation method, sending the current data frame and the hash value corresponding to the current data frame to a receiving end includes: embedding the hash value corresponding to the current data frame into the metadata of the current data frame; sending the current data frame and the metadata of the current data frame to the receiving end.

[0007] In combination with the first aspect, in a possible implementation method, when the current data frame is a key frame, sending the signature corresponding to the key frame to the receiving end, including: when the current data frame is a key frame, embedding the signature corresponding to the key frame into the metadata of the current data frame; sending the current data frame and the metadata of the current data frame to the receiving end.

[0008] In combination with the first aspect, in a possible implementation method, when the current data frame is a key frame, sending the signature corresponding to the key frame to the receiving end includes: when the current data frame is a key frame, sending the signature corresponding to the key frame to the receiving end through an independent channel.

[0009] In combination with the first aspect, in a possible implementation, when the current data frame is the first data frame extracted from streaming media data collected during real-time communication, the original data used to calculate the hash value includes: the first data frame and a preset initial value.

[0010] The second aspect of the present application provides a real-time communication processing method, which is applied to a receiving end, and the method includes: receiving a data frame to be verified and a corresponding hash value to be verified; using the data frame to be verified and the hash value corresponding to the previous data frame of the data frame to be verified to verify the hash value to be verified; when the data frame to be verified is a key frame, using the public key of the sender, the data frame to be verified, and the hash value corresponding to the previous data frame of the data frame to be verified to verify the received signature to be verified; if the verification of the hash value to be verified is successful, and when the data frame to be verified is a key frame, the verification of the signature to be verified is successful, then rendering and playing the data frame to be verified.

[0011] In combination with the second aspect, in a possible implementation, the method further includes: when any one of all the hash values ​​to be verified and the signatures to be verified fails to pass the verification, generating a prompt message to prompt that an abnormality exists in the current real-time communication.

[0012] In conjunction with the second aspect, in a possible implementation, the method further includes: generating a prompt message to prompt that an abnormality exists in the current real-time communication when no signature to be verified passes the verification within a preset time length.

[0013] The third aspect of the present application provides a terminal device, comprising: a memory configured to store predetermined computer instructions; and a processor configured to execute the predetermined computer instructions to implement any possible method of the first aspect.

[0014] The fourth aspect of the present application provides a terminal device, comprising: a memory configured to store predetermined computer instructions; and a processor configured to execute the predetermined computer instructions to implement any possible method of the second aspect.

[0015] The fifth aspect of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes any possible method of the first aspect and / or executes any possible method of the second aspect.

[0016] The sixth aspect of the present application provides a computer program product, including a computer program, which, when executed, enables a computer to execute any possible method of the first aspect and / or any possible method of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The following is a schematic diagram of the system architecture of an exemplary application scenario of the present application.

[0018] Figure 2 The present invention is a flowchart of an exemplary real-time communication processing method according to an embodiment of the present application.

[0019] Figure 3 The figure is an exemplary schematic diagram of the processing of streaming media data by the sending end in the embodiment of the present application.

[0020] Figure 4 This is an exemplary schematic diagram of the receiving end processing the received streaming media data in the embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to clearly and completely describe the technical solution of the present application, further explanation will be given below in conjunction with embodiments and drawings.

[0022] Generally, a signature method is used for the transmitted data, that is, the sender signs the transmitted data, and the receiver verifies whether the signature is passed on the received data, which can ensure the authenticity and integrity of the data and prevent the data from being tampered with or forged during the transmission process. However, the data processed and transmitted by this method is usually file-level data, which is used in application scenarios with low time requirements (i.e. real-time), but it is difficult to meet the requirements for application scenarios that require real-time communication. For example, in application scenarios such as video conferencing / calls and online live broadcasts, the sender needs to collect real-time video streams and send them continuously, while the receiver continues to receive the video streams and play them, thereby minimizing the delay between the sender and the receiver to achieve the purpose of real-time communication. Because of this, in real-time communication, the sender generally does not wait until the entire video is collected and signed before sending it to the receiver.

[0023] For real-time communication, even if the sender signs every data frame in the data stream, there are still some defects. For example, when some video frames in the video stream sent by the sender are deleted, the information expressed by the video stream may have changed. However, the receiver verifies the signature of each video frame retained in the video stream, and it can still be verified. That is, this method cannot detect whether some data frames in the data stream of real-time communication have been deleted, and the receiving end user may still be misled by untrue and unreliable information.

[0024] To this end, an embodiment of the present application proposes a solution, whereby the sending end performs certain processing on the data frames in the data stream of real-time communication so that it has strong anti-tampering capabilities and good verifiability; the receiving end performs corresponding verification after receiving it, and only renders and plays it after verification, thereby quickly detecting whether the video, voice and other content data transmitted by the real-time communication has been tampered with or forged, whether it comes from the real sending end, and effectively verifying the integrity of the content itself and the authenticity of the source, so as to display real and reliable information to the receiving end users, and improve the security protection of users during real-time communication.

[0025] The following will first introduce the system architecture of an exemplary real-time communication scenario to which the present solution can be applied, and then explain the processing procedures of the sending end and the receiving end respectively.

[0026] See also Figure 1 ,The system may include a sending end and one or more receiving ends, and in some cases may also include other physical and / or virtual devices, such as Figure 1 The server side shown, etc.

[0027] The sending end may be a terminal device with acquisition, processing, and communication capabilities, such as a mobile phone, tablet computer, personal computer, wearable device, smart camera, smart doorbell, online conference terminal, augmented reality (AR) / virtual reality (VR) device, etc. The sending end may also be developed as a client that can be installed and deployed on the aforementioned terminal device or other possible electronic devices, and has the ability to call the terminal device and / or cooperate with the terminal device for acquisition, processing, and communication. This application does not limit the specific product form / type of the sending end.

[0028] Streaming data mainly refers to data streams transmitted in real time over the network. These data are usually generated, transmitted and processed continuously, and do not need to be downloaded or stored before transmission. One of the characteristics of streaming data is that the data can be processed or played while being received at the receiving end, rather than waiting for all data to be transmitted.

[0029] The transmitter can collect video, voice, device interface change process, etc. from the real environment to form streaming media data for real-time communication. It is understandable that the transmitter can collect multiple different types of streaming media data at the same time, or can collect a single type of streaming media data, and this application does not limit this.

[0030] The sender has a pair of asymmetric keys, which are used to identify and prove the identity of the sender or the identity of the user using the sender. The key pair includes a corresponding private key SK and a public key PK. The private key SK can be held only by the sender, or can only be used by the sender or the user using the sender, while the public key PK can be sent to one or more receiving ends.

[0031] In some implementations, in order to identify and verify the identity of each sender or each user, a pair of keys may be generated for each sender or each user. A trusted certificate authority (CA) then issues a digital certificate for the sender or user based on the public key PK. The certificate can be used to uniquely identify the identity of the client or user and contains information about the public key PK, which can be distributed to other people or devices such as the receiver when needed.

[0032] The server is deployed in the server, and is connected to the sender and the receiver respectively, to provide background services for the sender and the receiver. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or cloud computing service center that provides cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communications, middleware services, security services, content distribution services, big data and artificial intelligence platforms and other cloud computing services. It can also be a combination of physical servers and cloud servers, etc. This application does not limit the specific product forms of the server and the server.

[0033] After the sending end processes the data frames in the streaming media data, it can be transmitted to the server through the network, and then transmitted to one or more receiving ends.

[0034] The receiving end may be a terminal device with processing, communication, and playback capabilities, such as a mobile phone, tablet computer, personal computer, wearable device, online conference terminal, smart TV, augmented reality (AR) / virtual reality (VR) device, etc. The receiving end may also be developed as a client that can be installed and deployed on the aforementioned terminal device or other possible electronic devices, and has the ability to call the terminal device and / or cooperate with the terminal device for processing, communication, and playback. This application does not limit the specific product form / type of the receiving end.

[0035] It can be understood that the distinction between the sender and the receiver in the embodiments of the present application is mainly to facilitate the description of the processing process of real-time communication. In actual application scenarios, the sender can also have the capabilities of the aforementioned receiver at the same time, and the receiver can also have the capabilities of the aforementioned sender at the same time, that is, the functions of the two can be interchanged, or they can have multiple functions at the same time, so as to realize one-way or two-way real-time communication.

[0036] The following will take the data processing and transfer process of a one-way streaming media data in a real-time communication process as an example to further explain the respective processing processes of the sending end and the receiving end and the functions that can be achieved.

[0037] See also Figure 2 , Figure 2 The flowchart of an exemplary real-time communication processing method of the embodiment of the present application is as follows: S101, the sending end extracts the current data frame from the collected streaming media data in real time.

[0038] The data frame is the basic unit for performing hash operations and signatures in the embodiments of the present application. For real-time communication application scenarios, the streaming media data may generally include multiple sequential data frames, such as video frames or audio frames.

[0039] A video frame is used to represent a single frame of a video image sequence. Each video frame represents an image at a certain moment. When these video frames with a sequence are played continuously at a certain speed, users will feel a smooth dynamic effect.

[0040] An audio frame is a basic unit of audio data at a certain moment, which usually represents a group of audio sample data. These samples are collected and organized together to form a part of the audio signal. Similar to the video frame in the video data stream, the audio frame is a basic component of the audio data stream.

[0041] In real-time communication application scenarios, the sender or other acquisition devices continuously collect and transmit data to form a data stream, namely, streaming media data.

[0042] The transmitting end extracts and processes the streaming media data frame by frame in real time and in order. For ease of description, the data frame extracted at a certain moment is referred to as the current data frame in the embodiment of the present application. It is understandable that the current data frame extracted at different moments may be different, and the processing of different current data frames may repeat steps S101 to S103.

[0043] The method of extracting the current data frame may adopt different methods according to different types and formats of the streaming media data.

[0044] In some implementations, when the streaming media data includes a video stream, video frames may be extracted from the video according to a first preset rule. Existing tools or libraries, such as FFmpeg, OpenCV, video encoders, etc., may be used to extract the video frames.

[0045] Optionally, in some cases the resolution or frame rate of the captured video stream may be high. In this case, the sender can appropriately reduce the extracted and processed data frames. This can speed up the processing speed of the sender and receiver without affecting the expression of the video stream content and the viewing effect of the receiver, thereby reducing delays, freezes and other problems in real-time communication.

[0046] Optionally, a GPU or a dedicated hardware encoder may be used to accelerate the extraction of data frames, calculation of hash values ​​and / or signatures, etc., so as to better meet the low latency requirements of real-time communication.

[0047] In some implementations, when the streaming media data includes an audio stream, audio frames can be extracted from the audio stream according to a third preset rule. Audio frames usually divide the audio signal into time windows of fixed size. Common audio frame sizes depend on application requirements, and a time window of 20-40 milliseconds is usually used, and the number of samples corresponding to each window is determined by the sampling rate of the audio. Exemplarily, the audio frame can be extracted using a sliding window method to divide the audio signal into frames by a fixed time length or number of samples.

[0048] It is understandable that during extraction, information such as a number or a timestamp may be recorded for each extracted data frame to record the sequence of different data frames, and to facilitate identification of adjacent data frames before and after different data frames when needed.

[0049] It is also understandable that when the streaming media data contains multiple different types of data such as audio, video, etc., they can be processed separately, that is, the audio frames and video frames are extracted separately to calculate the hash values, and their key frames are signed separately.

[0050] S102, the sending end calculates the hash value corresponding to the current data frame.

[0051] The original data used to calculate the hash value includes: the current data frame, and the hash value corresponding to the previous data frame of the current data frame.

[0052] For example, suppose a video stream data such as Figure 3 As shown, the sender extracts the video frames in the video stream in real time. For example, the current data frame extracted at a certain moment is D(i), and then calculates the hash value H(i) corresponding to D(i). The hash value H(i-1) corresponding to the previous video frame D(i-1) of D(i) and D(i) are used as the original data, and a hash operation is performed to obtain the hash value H(i). That is, H(i) = Hash(D(i)||H(i-1)).

[0053] The subsequent sending end can also extract the next video frame D(i+1) as the current data frame, and calculate the hash value H(i+1)=Hash(D(i+1)||H(i)) corresponding to D(i+1). Similarly, the hash value corresponding to each data frame can be calculated.

[0054] It should be noted that, when the current data frame is the first data frame D(0) extracted from the streaming media data collected during real-time communication, since there is no previous data frame, a preset data can be used instead. That is, in this case, the original data used to calculate the hash value H(0) corresponding to the data frame includes: the first data frame D(0), and the preset initial value Initial_Value, H(0) = Hash(D(0)||Initial_Value).

[0055] Exemplarily, the preset initial value may be a fixed value, such as 0 or other fixed values, or may be assigned a variable value according to certain rules. The present application does not limit the specific value-taking method and value of the preset initial value.

[0056] S103: When the current data frame is a key frame, the sender uses the sender's private key to calculate a signature corresponding to the key frame.

[0057] It can be understood that the key frame itself is also a data frame, which is determined from the data frame according to certain rules. If the current data frame meets certain preset conditions, it can be determined as a key frame.

[0058] In some implementations, when the streaming media data includes a video stream, a key frame may be determined from the video frame according to a second preset rule. For example, an existing key frame determination method may be used, such as taking an I frame (Intra Frame) of a video as a key frame, and taking a P frame and a B frame as non-key frames.

[0059] In some implementations, when the streaming media data includes an audio stream, a key frame may be determined from the audio frame according to a fourth preset rule. For example, one may be selected as a key frame every certain number (eg, 5) of audio frames.

[0060] For a key frame, in addition to calculating its corresponding hash value, the private key held by the sender is also used to calculate its corresponding signature. The original data used for the signature includes the key frame and the hash value corresponding to the previous data frame. For example, Figure 3 As shown, assuming that it can be determined that D(i) is a key frame, the private key SK is used to sign D(i) and H(i-1) to obtain the signature S(i). That is, S(i) = Sign_SK(D(i)||H(i-1)).

[0061] It is understandable that if the current data frame extracted at a certain moment is not a key frame, step S103 may not be performed. However, since streaming media data is continuously collected and processed in real-time communication, there will always be key frames in some data frames before or after the current data frame. For those key frames, the step of calculating their signatures is similar to step S103 here.

[0062] S104, the sending end sends the current data frame and the hash value corresponding to the current data frame to the receiving end; when the current data frame is a key frame, the sending end also sends the signature corresponding to the key frame to the receiving end.

[0063] There are many ways to send the hash value corresponding to the current data frame and the signature corresponding to the key frame to adapt to different possible needs such as different networks or storage. This application does not limit the specific method of sending.

[0064] In some implementations, step S104 may include: embedding a hash value corresponding to the current data frame into metadata of the current data frame; and sending the current data frame and the metadata of the current data frame to a receiving end.

[0065] Metadata is data that describes the content, structure, attributes and other information of data, but does not include the actual content of the data. In other words, metadata is "data" about data. Exemplarily, the metadata of a video frame may include the resolution, frame rate, bit rate, encoding format, color format, time interval between key frames (I frames), the protocol used by the video stream, the type of video frame and its compression level, stream identifier, etc. Exemplarily, the metadata of an audio stream may include the audio sampling rate, bit depth, number of channels, bit rate, encoding format, protocol used by the audio stream, channel order, noise suppression and echo cancellation strategy, stream identifier, etc. The hash value corresponding to the current data frame is embedded in the metadata, and its hash value can be sent to the receiving end together with the metadata and the data frame.

[0066] In other implementations, the hash value corresponding to the current data frame may also be put into an independent file (referred to as the first file for easy distinction) and sent out. In this case, the independent file also needs to record the correspondence between the hash value and the corresponding data frame.

[0067] In some implementations, if the current data frame is a key frame, the signature corresponding to the key frame may be embedded in the corresponding metadata and sent to the receiving end.

[0068] In other implementations, the signature corresponding to the key frame may also be put into an independent file (referred to as the second file for easy distinction) and sent out. In this case, the second file also needs to record the correspondence between the signature and the corresponding data frame.

[0069] Optionally, the first file and / or the second file may be sent to the receiving end through an independent channel.

[0070] It is understandable that when the hash value and the signature are stored in independent files and sent, the aforementioned first file and second file may be different files or the same file, and this application does not limit this.

[0071] It can be understood that the above steps S101 to S104 describe the processing process of the current data frame at a certain moment, and the processing process of other data frames as the current data frame is similar.

[0072] S201, a receiving end receives a data frame to be verified and a corresponding hash value to be verified.

[0073] The receiving end can receive streaming media data from the server through the network, which includes at least one data frame to be verified and its corresponding hash value. When the data frame is a key frame, the receiving end will also normally receive the signature corresponding to the key frame. In the embodiment of the present application, the verification process of a certain data frame is taken as an example to introduce the processing process of the receiving end. For ease of description, the data frame is referred to as a data frame to be verified, and its corresponding hash value is referred to as a hash value to be verified. Accordingly, when the data frame is a key frame, the signature corresponding to the corresponding key frame is referred to as a signature to be verified.

[0074] For example, see Figure 4 , assuming that the data frame that needs to be verified currently, that is, the data frame to be verified is D'(i), and it is a key frame, then its corresponding hash value to be verified is H(i), and the corresponding signature is S(i).

[0075] S202, the receiving end verifies the hash value to be verified by using the data frame to be verified and the hash value corresponding to the previous data frame of the data frame to be verified.

[0076] continue Figure 4 In the example shown, the receiving end performs a hash operation on D'(i) and H(i-1) as the original data to obtain H'(i). If H'(i) is consistent with the hash value H(i) to be verified, the hash value corresponding to the data frame D'(i) to be verified is verified successfully.

[0077] It is understandable that when the previous data frame D'(i-1) and the next data frame D'(i+1) of D'(i) are used as data frames to be verified, the corresponding hash value verification methods are similar, and so on.

[0078] Since streaming media data is processed in a certain order during real-time communication, the receiving end executes verification D'(i), which means that D'(i-1) has been verified, and the corresponding H'(i-1)=H(i-1), H(i-1) is authentic and reliable. Therefore, the hash value verification corresponding to the data frame D'(i) to be verified shows that D'(i) itself has not been tampered with, and its previous data frame is indeed D'(i-1). The two are adjacent data frames when processed and sent by the sending end, and there is no deleted data frame between the two.

[0079] S203, when the data frame to be verified is a key frame, the receiving end verifies the received signature to be verified by using the public key of the sending end, the data frame to be verified, and the hash value corresponding to the previous data frame of the data frame to be verified.

[0080] The receiving end uses the public key to verify the signature corresponding to a key frame, which corresponds to the signing process. Figure 4 , take the verification of signature S(i) as an example. When signing, hash operation is performed with D(i) and H(i-1) as the original data, and encrypted with private key SK to obtain S(i). Accordingly, when verifying the signature, private key PK can be used to decrypt S(i) to obtain decrypted data; hash operation is performed with D'(i) and H(i-1) as the original data to obtain operation data; decrypted data and operation data are compared. If the two are consistent, signature S(i) is verified. If they are inconsistent, signature S(i) fails to be verified. That is, Verify_PK(S(i),D'(i)||H(i-1)). S(i) is verified, indicating that the key frame D'(i) is consistent with the corresponding key unit D(i) during the original processing at the sender, that is, D'(i) is indeed from the sender and has not been modified, and is authentic.

[0081] It should be noted that, when the data frame to be verified is a key frame, the receiving end may first verify the corresponding hash value to be verified, or may first verify the signature to be verified, or may perform both at the same time, which is not limited in this application.

[0082] S204: If the verification of the hash value to be verified is successful, and if the data frame to be verified is a key frame, the verification of the signature to be verified is successful, then the data frame to be verified is rendered and played.

[0083] S205: When any one of all the hash values ​​to be verified and the signatures to be verified fails to pass the verification, a prompt message is generated to prompt that an abnormality exists in the current real-time communication.

[0084] In this way, the source of each key frame and its previous data frame in the video stream is guaranteed to be credible through private key signature, and the hash values ​​of the key frame and the previous data frame are prevented from being forged or illegally tampered with. At the same time, by calculating the hash value corresponding to each data frame, an inter-frame hash chain is established between adjacent data frames in the streaming media data of real-time communication. Any tampering with a single data frame will destroy the chain structure of the inter-frame hash chain, resulting in failure to pass verification. In this way, it is possible to verify whether the source of the streaming media data during real-time communication is authentic, whether each frame has been tampered with or forged, and whether a frame or fragment has been deleted, thereby better ensuring the authenticity and reliability of real-time communication.

[0085] In some application scenarios, the receiving end may not communicate with the sending end in real time from the beginning, but may start real-time communication in the middle. For example, in some online live broadcast scenarios, the sending end used by the anchor sends streaming media data to all receiving ends entering the live broadcast room through the network from the beginning, while some users enter the live broadcast room in the middle, and the first data frame they receive may not be a key frame. If the receiving end does not receive and verify the signature corresponding to the key frame for a period of time, it cannot be fully guaranteed that the data frames played from the first data frame are necessarily authentic and reliable. For example, for AI-generated streaming media data, if the entire video together with the hash values ​​corresponding to all data frames are forged, this situation cannot be detected.

[0086] To solve this problem, in some implementations, the above method may include:

[0087] S206: If no signature to be verified passes verification within a preset time period, a prompt message is generated to prompt that an abnormality exists in the current real-time communication.

[0088] A first terminal device is provided in an embodiment of the present application, comprising: a first memory configured to store predetermined computer instructions; and a first processor configured to execute the predetermined computer instructions to implement part or all of the steps of the method executed by the aforementioned sending end.

[0089] A second terminal device is provided in an embodiment of the present application, comprising: a second memory configured to store predetermined computer instructions; and a second processor configured to execute the predetermined computer instructions to implement part or all of the steps of the method executed by the aforementioned receiving end.

[0090] It can be understood that the first terminal device can also implement part or all of the functions that the receiving end can implement; the second terminal device can also implement part or all of the functions that the sending end can implement.

[0091] The embodiment of the present application also provides a system including a first terminal device and a second terminal device. Optionally, the system may also include a server. The system may be applied in a variety of different scenarios.

[0092] For example, this solution is used in the application scenario of video conferencing. For each terminal / user participating in the video conference, the source and content of the streaming media data received are authentic and reliable. When each terminal participating in the video conference acts as a sender and a receiver at the same time, the authenticity and reliability of each other can be ensured. At the same time, the solution can also ensure that the content of the video conference is not stolen or tampered with, and the privacy of the conference is guaranteed.

[0093] As another example, by adopting this solution in the application scenario of online live broadcast, it can ensure that ordinary users entering the live broadcast room watch the real live broadcast content from the anchor that has not been illegally tampered with or forged, prevent the live broadcast content from being illegally tampered with or forged, and improve the credibility of the live broadcast platform.

[0094] In addition, the system can also be used in application scenarios such as voice calls and live game broadcasts.

[0095] Those skilled in the art should understand that, in addition to the methods and terminal devices described above, the embodiments of the present application may also be provided as a client, a computer-readable storage medium or a computer program product.

[0096] The method performed by the transmitting end and the receiving end provided in the embodiments of the present application can be provided as a computer-readable storage medium. That is, a computer-readable storage medium is also provided in the embodiments of the present application, storing a computer program, and when the computer program is executed by a processor, the processor executes part or all of the steps in any implementation of the aforementioned processing method or verification method.

[0097] The present application may also take the form of a computer program product embodied on one or more computer readable storage media having computer usable program code embodied therein.

[0098] Those of ordinary skill in the art will appreciate that the units and steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0099] It should be understood that in various embodiments of the present application, the execution order of each step should be determined by its function and internal logic, and the size of the step number does not mean the order of execution and does not constitute any limitation on the implementation process of the embodiment.

[0100] It should also be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, the words "first", "second", etc. are used in the embodiments of the present application to distinguish between the same items or similar items with substantially the same functions and effects, or concepts with certain differences. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the difference.

[0101] The same or similar parts between the various embodiments in this specification can be referred to each other. As long as there is no logical contradiction, different embodiments can be combined with each other.

Claims

1. A real-time communication processing method, characterized in that: Applied to the sending end, the method includes: Extract the current data frame from the collected streaming media data in real time; Calculating a hash value corresponding to the current data frame; wherein the original data used to calculate the hash value includes: the current data frame, and a hash value corresponding to a previous data frame of the current data frame; In the case where the current data frame is a key frame, a signature corresponding to the key frame is calculated using the private key of the sender, wherein the original data used to calculate the signature includes: the key frame, and a hash value corresponding to a previous data frame of the key frame; the private key of the sender is used to identify the identity of the sender or to identify the identity of a user using the sender; The current data frame and the hash value corresponding to the current data frame are sent to the receiving end; when the current data frame is a key frame, the signature corresponding to the key frame is also sent to the receiving end.

2. The method according to claim 1, characterized in that: Sending the current data frame and the hash value corresponding to the current data frame to the receiving end includes: Embedding a hash value corresponding to the current data frame into metadata of the current data frame; The current data frame and metadata of the current data frame are sent to a receiving end.

3. The method according to claim 2, characterized in that In the case where the current data frame is a key frame, sending a signature corresponding to the key frame to the receiving end includes: In the case where the current data frame is a key frame, embedding the signature corresponding to the key frame into the metadata of the current data frame; or, In the case where the current data frame is a key frame, a signature corresponding to the key frame is sent to the receiving end through an independent channel.

4. The method according to any one of claims 1 to 3, characterized in that: In the case where the current data frame is the first data frame extracted from streaming media data collected during real-time communication, the original data used to calculate the hash value includes: the first data frame and a preset initial value.

5. A real-time communication processing method, characterized in that: Applied to the receiving end, the method includes: Receive a data frame to be verified and a corresponding hash value to be verified; Verify the hash value to be verified by using the data frame to be verified and the hash value corresponding to the previous data frame of the data frame to be verified; In the case where the data frame to be verified is a key frame, the received signature to be verified is verified using the public key of the sender, the data frame to be verified, and the hash value corresponding to the previous data frame of the data frame to be verified; If the verification of the hash value to be verified is passed, and if the data frame to be verified is a key frame, the verification of the signature to be verified is passed, then the data frame to be verified is rendered and played.

6. The method according to claim 5, characterized in that Also includes: In the case that any one of all the hash values ​​to be verified and the signatures to be verified fails to pass the verification, a prompt message is generated to prompt that an abnormality exists in the current real-time communication.

7. The method according to any one of claims 5 to 6, characterized in that: Also includes: If no signature to be verified passes verification within a preset time length, a prompt message is generated to prompt that an abnormality exists in the current real-time communication.

8. A terminal device, characterized in that: include: a memory configured to store predetermined computer instructions; A processor configured to execute the predetermined computer instructions to implement the method described in any one of claims 1 to 4; or to implement the method described in any one of claims 5 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 4, or the method according to any one of claims 5 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images