Real-time communication processing method, terminal device, storage medium and program product

Abstract

The application discloses a processing method of real-time communication, a terminal device, a storage medium and a program product. A sending end extracts a data frame from collected stream media data in real time; a hash value corresponding to the data frame is calculated, and a signature corresponding to the data frame is calculated using a private key of the sending end in the case that the data frame is a key frame; original data used for calculation includes hash values corresponding to the data frame and a previous data frame of the data frame; and then the data frame and the corresponding hash value and signature are sent to a receiving end. The receiving end verifies the data frame and the corresponding hash value and signature respectively after receiving the data frame and the corresponding hash value and signature, and only when the verification is passed, the corresponding data frame is rendered and played. According to the scheme, the data frame in the data stream sent by the sending end in real-time communication has strong tamper resistance and good verifiability, the receiving end can effectively verify the integrity and authenticity of the content itself, thereby real and reliable information is displayed for the receiving end user, and the security guarantee for the user in real-time communication is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of real-time communication, in particular to a real-time communication processing method applied to a sending terminal and a corresponding real-time communication processing method applied to a receiving terminal. In addition, the present application also relates to a corresponding terminal device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] Real-time communication generally refers to the process of exchanging information between two or more participants in real time without significant delay. During real-time communication (such as video call, voice call, online live broadcast), due to the openness of network transmission, real-time communication data faces the risk of being stolen, tampered with, and forged.

[0003] The prior art mainly relies on the security mechanism of the transmission protocol (such as TLS / DTLS) to protect the security of the communication channel, which to some extent guarantees the security of the data transmitted by the real-time communication. However, once the security mechanism is invaded by an attacker, the attacker can still tamper with and forge the data in the real-time communication process, causing the receiving end user to be misled or even cheated by the non-true information received. SUMMARY

[0004] One purpose of the present application is to provide a real-time communication processing method, in which the sending terminal processes the data frame extracted in real time from the stream media data, so that the data frame has strong tamper-proofing ability and good verifiability. In this way, the receiving end can effectively verify the integrity and authenticity of the stream media data content itself, thereby showing the receiving end user with true and reliable information.

[0005] The first aspect of the present application provides a real-time communication processing method applied to a sending terminal, the method comprising: extracting a current data frame in real time from collected stream media data; calculating a hash value corresponding to the current data frame; wherein the original data used for calculating the hash value includes: the current data frame, and a hash value corresponding to a previous data frame of the current data frame; in the case that the current data frame is a key frame, using a private key of the sending terminal to calculate a signature corresponding to the key frame, wherein the original data used for calculating the signature includes: the key frame, and a hash value corresponding to a previous data frame of the key frame; the private key of the sending terminal is used to identify the identity of the sending terminal or to identify the identity of a user using the sending terminal; sending the current data frame and the hash value corresponding to the current data frame to a receiving terminal; in the case that the current data frame is a key frame, also sending the signature corresponding to the key frame to the receiving terminal.

[0006] With reference to the first aspect, in a possible implementation manner, the sending, to the receiving end, of the current data frame and the hash value corresponding to the current data frame comprises: embedding the hash value corresponding to the current data frame into metadata of the current data frame; and sending, to the receiving end, the current data frame and the metadata of the current data frame.

[0007] With reference to the first aspect, in a possible implementation manner, in a case where the current data frame is a key frame, the sending, to the receiving end, of the signature corresponding to the key frame comprises: in a case where the current data frame is a key frame, embedding the signature corresponding to the key frame into metadata of the current data frame; and sending, to the receiving end, the current data frame and the metadata of the current data frame.

[0008] With reference to the first aspect, in a possible implementation manner, in a case where the current data frame is a first data frame extracted from stream media data collected during real-time communication, the original data used for calculating the hash value comprises: the first data frame and a preset initial value.

[0009] With reference to the first aspect, in a possible implementation manner, in a case where the current data frame is a first data frame extracted from stream media data collected during real-time communication, the original data used for calculating the hash value comprises: the first data frame and a preset initial value.

[0010] The second aspect of the present application provides a real-time communication processing method applied to a receiving end, the method comprising: receiving a data frame to be verified and a corresponding hash value to be verified; verifying the hash value to be verified by using the data frame to be verified and a hash value corresponding to a previous data frame of the data frame to be verified; in a case where the data frame to be verified is a key frame, verifying a received signature to be verified by using a public key of a sending end, the data frame to be verified and the hash value corresponding to the previous data frame of the data frame to be verified; and rendering and playing the data frame to be verified if the verification of the hash value to be verified passes and the verification of the signature to be verified passes in the case where the data frame to be verified is a key frame.

[0011] With reference to the second aspect, in a possible implementation manner, the method further comprises: in a case where any one of all the hash values to be verified and the signature to be verified fails to pass verification, generating prompt information to prompt that there is an exception in current real-time communication.

[0012] With reference to the second aspect, in a possible implementation manner, the method further comprises: in a case where no signature to be verified passes verification within a preset time length, generating prompt information to prompt that there is an exception in current real-time communication.

[0013] The third aspect of the present application provides a terminal device, comprising: a memory configured to store predetermined computer instructions; and a processor configured to execute the predetermined computer instructions to implement any possible method of the first aspect.

[0014] The fourth aspect of the present application provides a terminal device, comprising: a memory configured to store predetermined computer instructions; and a processor configured to execute the predetermined computer instructions to implement any possible method of the second aspect.

[0015] The fifth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, causes the processor to execute any possible method of the first aspect and / or any possible method of the second aspect.

[0016] The sixth aspect of the present application provides a computer program product, which comprises a computer program, and the computer program, when executed, causes a computer to execute any possible method of the first aspect and / or any possible method of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A system architecture diagram for an exemplary application scenario of the present application.

[0018] Figure 2 A flowchart of an exemplary real-time communication processing method for an embodiment of the present application.

[0019] Figure 3 An exemplary diagram for processing a stream media data by a sending end in an embodiment of the present application.

[0020] Figure 4 An exemplary diagram for processing a received stream media data by a receiving end in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To clearly and completely describe the technical solutions of the present application, further description will be made in combination with embodiments and drawings.

[0022] Generally, the data transmitted is signed, that is, the sending end signs the data to be transmitted, and the receiving end verifies whether the signature of the received data is passed, which can guarantee the authenticity and integrity of the data and prevent the data from being tampered with or forged during transmission. However, the data processed and transmitted by this method is usually file-level data, which is applied to application scenarios with low time requirements (i.e., real-time). For application scenarios requiring real-time communication, it is difficult to meet the needs. For example, in the application scenarios of video conference / call, online live broadcast, etc., the sending end needs to continuously send out the real-time collected video stream, and the receiving end continuously receives the video stream and plays it, so as to reduce the time delay between the sending end and the receiving end as much as possible, so as to achieve the purpose of real-time communication. Therefore, in real-time communication, the sending end generally does not wait until it collects a whole video and signs it before sending it to the receiving end.

[0023] For real-time communication, even if the sending end signs each data frame in the data stream, there are still some defects. For example, when some video frames in the video stream sent by the sending end are deleted, the information expressed by the video stream may have changed. The receiving end verifies the signature of each video frame remaining in the video stream, which can still be verified. That is, this method cannot detect whether some data frames in the data stream are deleted in real-time communication, and the receiving end user may still be misled by non-real and unreliable information.

[0024] Therefore, an embodiment of the present application proposes a solution. The sending end processes the data frames in the data stream of real-time communication, so that they have strong tamper resistance and good verifiability. The receiving end performs corresponding verification after receiving, and renders and plays only when the verification is passed, so as to quickly detect whether the video, voice and other content data transmitted in real-time communication are tampered with or forged, whether they come from the real sending end, and effectively verify the integrity and authenticity of the content itself, so as to show the real and reliable information to the receiving end user and improve the security protection of the user in real-time communication.

[0025] First, a system architecture of an exemplary real-time communication scenario to which the present solution can be applied will be introduced, and then the processing procedures of the sending end and the receiving end will be described respectively.

[0026] Referring to Figure 1 , the system can include a sending end and one or more receiving ends, and in some cases, it can also include other physical and / or virtual devices, such as Figure 1 , a server, etc.

[0027] The sending terminal can be a terminal device with the capability of collecting, processing and communicating, such as a mobile phone, a tablet computer, a personal computer, a wearable device, a smart camera, a smart doorbell, an online conference terminal, an augmented reality (AR) / virtual reality (VR) device, etc. The sending terminal can also be developed as a client capable of being installed and deployed on the terminal device or other possible electronic device, with the capability of calling and / or cooperating with the terminal device to collect, process and communicate. The specific product form / type of the sending terminal is not limited in the present application.

[0028] Streaming data mainly refers to a data stream transmitted in real time through a network. The data is usually continuously generated, transmitted and processed, and does not need to be downloaded or stored in whole before transmission. One of the features of streaming data is that the data can be processed or played at the receiving end while being received, instead of waiting for all the data to be transmitted.

[0029] The sending terminal can collect video, voice, device interface change process, etc. from the real environment to form streaming data for real-time communication. It can be understood that the sending terminal can synchronously collect multiple types of streaming data at the same time, or collect a single type of streaming data, which is not limited in the present application.

[0030] The sending terminal has a pair of asymmetric keys for identifying and proving the identity of the sending terminal or the identity of the user using the sending terminal. The pair of keys includes a corresponding private key SK and a public key PK. The private key SK can be held only by the sending terminal, or only the sending terminal or the user using the sending terminal can use it, and the public key PK can be sent to one or more receiving terminals.

[0031] In some implementations, in order to identify and verify the identity of each sending terminal or each user, a pair of keys can be generated for each sending terminal or each user. Then a trusted certificate authority (CA) can issue a digital certificate for the sending terminal or user based on the public key PK. The certificate can be used to uniquely identify the identity of the client or user, contains the information of the public key PK, and can be distributed to the receiving terminal or other people or devices when needed.

[0032] The server is deployed on a physical server and communicates with both the sending and receiving ends to provide backend services. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, a cloud server or cloud service center providing cloud services such as cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, security services, content distribution services, big data and artificial intelligence platforms, or a combination of physical servers and cloud servers. This application does not limit the specific product form of the server or the physical server.

[0033] After the sending end processes the data frames in the streaming media data, they can be transmitted to the server via the network, and then to one or more receiving ends.

[0034] The receiving end can be a terminal device with processing, communication, and playback capabilities, such as a mobile phone, tablet computer, personal computer, wearable device, online conferencing terminal, smart TV, augmented reality (AR) / virtual reality (VR) device, etc. The receiving end can also be developed as a client that can be installed and deployed on the aforementioned terminal devices or other possible electronic devices, possessing the ability to invoke the terminal device and / or cooperate with the terminal device for processing, communication, and playback. This application does not limit the specific product form / type of the receiving end.

[0035] It is understood that the distinction between the sending end and the receiving end in this application embodiment is mainly for the purpose of clearly describing the real-time communication process. In actual application scenarios, the sending end may also have the aforementioned receiving end capabilities, and the receiving end may also have the aforementioned sending end capabilities. That is, the functions of the two can be interchanged, or they may have multiple functions at the same time, so as to realize one-way or two-way real-time communication.

[0036] The following section will take the data processing and flow of a one-way streaming media in a real-time communication process as an example to further explain the processing procedures of the sending and receiving ends and the functions they can achieve.

[0037] See Figure 2 , Figure 2 This is a flowchart illustrating an exemplary real-time communication processing method according to an embodiment of this application. S101, the sending end extracts the current data frame from the collected streaming media data in real time.

[0038] Data frames are the basic units for hashing and signing in this application embodiment. For real-time communication applications, streaming media data typically includes multiple sequential data frames, such as video frames or audio frames.

[0039] A video frame refers to a single frame of video in a sequence of video frames. Each video frame represents an image at a certain time. When the video frames with the sequence are played continuously at a certain speed, the user will experience a smooth dynamic effect.

[0040] An audio frame refers to a basic unit of audio data at a certain time. It usually represents a group of audio sample data collected and organized together to form part of an audio signal. Similar to the video frame in the video data stream, the audio frame is a basic component of the audio data stream.

[0041] In the application scenario of real-time communication, the sending end or other collection devices continuously collect and transmit data streams, i.e., streaming media data.

[0042] The sending end extracts, processes, and transmits the data frames in sequence in real time. For ease of description, the data frame extracted at a certain time is referred to as the current data frame. It can be understood that the current data frame extracted at different times can be different, and the processing of different current data frames can repeat steps S101-S103.

[0043] The way of extracting the current data frame can be different according to the type and format of the streaming media data.

[0044] In some implementations, when the streaming media data includes a video stream, the video frame can be extracted from the video according to a first preset rule. The extraction of the video frame can use existing tools or libraries, such as FFmpeg, OpenCV, video encoder, etc.

[0045] Optionally, in some cases, the resolution or frame rate of the collected video stream can be high. In this case, the sending end can appropriately reduce the extracted and processed data frames, which can speed up the processing speed of the sending end and the receiving end, reduce the latency, stuttering, and other problems of real-time communication, while not affecting the expression of the video stream content and the viewing effect of the receiving end.

[0046] Optionally, GPU or special hardware encoder, etc. can be used to accelerate the extraction of the data frame, the calculation of the hash value and / or the signature, so as to better meet the low-latency requirement of real-time communication.

[0047] In some implementations, when streaming media data includes audio streams, audio frames can be extracted from the audio stream according to a third preset rule. Audio frames are typically created by dividing the audio signal into fixed-size time windows. Common audio frame sizes depend on application requirements, typically using time windows of 20-40 milliseconds, with the number of samples per window determined by the audio sampling rate. For example, audio frames can be extracted using a sliding window approach, dividing the audio signal into frames based on a fixed time length or number of samples.

[0048] Understandably, during extraction, information such as number or timestamp can be recorded for each extracted data frame to record the order of different data frames, and to facilitate the identification of adjacent data frames when needed.

[0049] It is also understandable that when streaming media data contains multiple different types of data such as audio and video, they can be processed separately, that is, audio frames and video frames can be extracted separately to calculate hash values, and their keyframes can be signed separately.

[0050] S102, the sending end calculates the hash value corresponding to the current data frame.

[0051] The raw data used to calculate the hash value includes: the current data frame and the hash value of the previous data frame.

[0052] For example, suppose a video stream data is as follows Figure 3 As shown, the sending end extracts video frames from the video stream in real time. For example, the current data frame extracted at a certain moment is D(i), and then calculates the hash value H(i) corresponding to D(i). Using the hash value H(i-1) corresponding to the previous video frame D(i-1) of D(i) and D(i) as the original data, a hash operation is performed to obtain the hash value H(i). That is, H(i) = Hash(D(i)||H(i-1)).

[0053] The sending end can then extract the next video frame D(i+1) as the current data frame and calculate the hash value H(i+1) = Hash(D(i+1)||H(i)) corresponding to D(i+1). This process can be repeated to calculate the hash value for each data frame.

[0054] It should be noted that in the case of the first data frame D(0) extracted from the streaming media data collected in the current data frame for real-time communication, a preset data can be used instead since there is no previous data frame. That is, in this case, the original data used for calculating the hash value H(0) corresponding to the data frame includes the first data frame D(0) and a preset initial value Initial_Value, H(0) = Hash(D(0) || Initial_Value).

[0055] Exemplarily, the preset initial value can be a fixed value such as 0 or other fixed values, or can be assigned a variable value according to certain rules. The present application does not limit the specific value and value of the preset initial value.

[0056] S103, in the case of the current data frame being a key frame, the sender uses the private key of the sender to calculate the signature corresponding to the key frame.

[0057] It can be understood that the key frame itself is also a data frame, which is determined from the data frame according to certain rules. If the current data frame meets certain preset conditions, it can be determined as a key frame.

[0058] In some implementations, in the case where the streaming media data includes a video stream, the key frame can be determined from the video frame according to a second preset rule. Exemplarily, an existing key frame determination method can be used, for example, the I frame (Intra Frame) of the video is used as the key frame, and the P frame and B frame are used as non-key frames.

[0059] In some implementations, in the case where the streaming media data includes an audio stream, the key frame can be determined from the audio frame according to a fourth preset rule. Exemplarily, every certain number of audio frames (for example, 5) can be selected as a key frame.

[0060] For the key frame, in addition to calculating the corresponding hash value, the private key held by the sender is also used to calculate the corresponding signature, and the original data used for the signature includes the key frame and the hash value corresponding to the previous data frame. Exemplarily, as shown in Figure 3 , assuming that D(i) is determined to be a key frame, the private key SK is used to sign D(i) and H(i-1) to obtain the signature S(i). That is, S(i) = Sign_SK(D(i) || H(i-1)).

[0061] It can be understood that if the current data frame extracted at a certain moment is not a key frame, the step of S103 can not be performed. However, due to the continuous acquisition and processing of the stream media data in real-time communication, there will always be a key frame in some data frames before or after the current data frame, and the step of calculating the signature of the key frame is similar to the step of S103.

[0062] S104, the sending end sends the current data frame and the hash value corresponding to the current data frame to the receiving end; and in the case that the current data frame is a key frame, the signature corresponding to the key frame is also sent to the receiving end.

[0063] The sending of the hash value corresponding to the current data frame and the signature corresponding to the key frame can have various implementation manners to adapt to different network or storage requirements, and the specific manner of sending is not limited in the present application.

[0064] In some implementation manners, the step of S104 can include embedding the hash value corresponding to the current data frame into the metadata of the current data frame, and sending the current data frame and the metadata of the current data frame to the receiving end.

[0065] Metadata is data that describes the content, structure, attributes and other information of data, but does not include the actual content of the data. In other words, metadata is "data" about data. Exemplarily, the metadata of a video frame can include the resolution, frame rate, bit rate, encoding format, color format, time interval between key frames (I frames), protocol used by the video stream, type of video frame and its compression level, stream identifier, etc. Exemplarily, the metadata of an audio stream can include the sampling rate, bit depth, number of channels, bit rate, encoding format, protocol used by the audio stream, sound channel order, noise suppression and echo cancellation strategy, stream identifier, etc. Embedding the hash value corresponding to the current data frame into the metadata, the hash value can be sent to the receiving end together with the metadata and the data frame.

[0066] In other implementation manners, the hash value corresponding to the current data frame can also be put into an independent file (referred to as a first file for easy distinction) and sent out, and at this time the independent file also needs to record the correspondence between the hash value and the corresponding data frame.

[0067] In some implementation manners, if the current data frame is a key frame, the signature corresponding to the key frame can be embedded into the corresponding metadata and sent to the receiving end.

[0068] In other implementation manners, the signature corresponding to the key frame can also be put into an independent file (referred to as a second file for easy distinction) and sent out, and at this time the second file also needs to record the correspondence between the signature and the corresponding data frame.

[0069] Optionally, the first file and / or the second file can be sent to the receiving end through independent channels.

[0070] It can be understood that when the hash value and the signature are both stored in independent files for sending, the aforementioned first file and the second file can be different files or the same file, which is not limited in the present application.

[0071] It can be understood that the steps of S101 to S104 described above describe the processing procedure of the current data frame at a certain time, and the processing procedure of other data frames as the current data frame is similar.

[0072] S201, the receiving end receives a data frame to be verified and a corresponding hash value to be verified.

[0073] The receiving end can receive streaming media data from the service end through a network, which contains at least one data frame to be verified and a corresponding hash value thereof. In the case of a key frame, the receiving end will normally also receive the signature corresponding to the key frame. In the embodiment of the present application, the verification procedure of a certain data frame is taken as an example to introduce the processing procedure of the receiving end. For ease of description, the data frame is referred to as a data frame to be verified, and the corresponding hash value is referred to as a hash value to be verified. Correspondingly, in the case of a key frame, the corresponding signature of the key frame is referred to as a signature to be verified.

[0074] Exemplarily, referring to Figure 4 , it is assumed that the data frame to be verified, i.e., the data frame to be verified, is D'(i), and it is a key frame, and the corresponding hash value to be verified H(i) and the corresponding signature S(i).

[0075] S202, the receiving end verifies the hash value to be verified by using the data frame to be verified and the hash value corresponding to the previous data frame of the data frame to be verified.

[0076] Continuing Figure 4 the example shown in the figure, the receiving end performs hash operation on D'(i) and H(i-1) as original data to obtain H'(i), and if H'(i) is consistent with the hash value to be verified H(i), the hash value corresponding to the data frame to be verified D'(i) is verified.

[0077] It can be understood that when the previous data frame D'(i-1) and the next data frame D'(i+1) of D'(i) are taken as data frames to be verified, the corresponding hash value verification mode is similar, and the same can be applied by analogy.

[0078] Since the streaming data is processed in a certain order in real-time communication, the receiving end performs verification on D'(i) to indicate that D'(i-1) has been verified, and the corresponding H'(i-1) = H(i-1), which is authentic and reliable. Therefore, the verification of the hash value corresponding to the to-be-verified data frame D'(i) passing indicates that D'(i) itself is not tampered with, and the previous data frame is indeed D'(i-1). When the data frames are processed and sent at the sending end, they are adjacent data frames, and there is no deleted data frame between them.

[0079] S203, in the case where the to-be-verified data frame is a key frame, the receiving end verifies the received to-be-verified signature by using the public key of the sending end, the to-be-verified data frame, and the hash value corresponding to the previous data frame of the to-be-verified data frame.

[0080] The receiving end verifies the signature corresponding to a certain key frame by using the public key, which is corresponding to the signature process. See Figure 4 Taking the verification of the signature S(i) as an example, in the signature, the hash operation is performed on D(i) and H(i-1) as original data, and the private key SK is used for encryption to obtain S(i). Correspondingly, in the verification of the signature, the private key PK can be used to decrypt S(i) to obtain decrypted data, and the hash operation is performed on D'(i) and H(i-1) as original data to obtain operation data. The decrypted data and the operation data are compared, if they are consistent, the signature S(i) is verified, if they are inconsistent, the signature S(i) is not verified. That is, Verify_PK(S(i), D'(i) || H(i-1)). S(i) is verified, which indicates that the key frame D'(i) is consistent with the corresponding key unit D(i) when the sending end is originally processed, that is, D'(i) is indeed from the sending end and has not been modified, and is authentic and reliable.

[0081] It should be noted that in the case where the to-be-verified data frame is a key frame, the receiving end can first verify the corresponding to-be-verified hash value, or first verify the to-be-verified signature, or simultaneously verify, which is not limited in the present application.

[0082] S204, if the verification of the to-be-verified hash value passes, and in the case where the to-be-verified data frame is a key frame, the verification of the to-be-verified signature passes, the to-be-verified data frame is rendered and played.

[0083] S205, in the case where any one of the to-be-verified hash value and the to-be-verified signature does not pass the verification, a prompt information is generated to prompt that there is an exception in the current real-time communication.

[0084] In this way, the source of each key frame and the previous data frame in the video stream is ensured to be credible by private key signature, and the hash values of the key frame and the previous data frame are prevented from being forged or illegally tampered with. At the same time, by calculating the hash value corresponding to each data frame, an inter-frame hash chain is established between adjacent data frames in the streaming media data of real-time communication. Any tampering with a single data frame will break the chain structure of the inter-frame hash chain, resulting in failure to pass the verification. In this way, it can be verified whether the source of the streaming media data in real-time communication is real, whether each frame is tampered with or forged, and whether there is a case of deleting a certain frame or segment, thereby better guaranteeing the authenticity and reliability of real-time communication.

[0085] In some application scenarios, the receiving end may not start real-time communication with the sending end from the beginning, but start real-time communication in the middle. For example, in some online live streaming scenarios, the sending end used by the host starts to send streaming media data to all receiving ends entering the live room through the network from the beginning, but some users enter the live room in the middle, and the first data frame received by the receiving end may not be a key frame. If the receiving end does not receive and verify the signature corresponding to the key frame for a period of time, it cannot be completely guaranteed that the data frames played from the first data frame are necessarily real and reliable. For example, for AI-generated streaming media data, if the entire video is forged together with all the hash values corresponding to the data frames, this situation cannot be detected.

[0086] To solve this problem, in some implementation manners, the above method can include:

[0087] S206, if no verified signature passes the verification within a preset time length, generating a prompt information to prompt that there is an abnormality in the current real-time communication.

[0088] In the embodiments of the present application, a first terminal device is provided, which includes: a first memory configured to store predetermined computer instructions; and a first processor configured to run the predetermined computer instructions to implement part or all of the steps of the method performed by the sending end.

[0089] In the embodiments of the present application, a second terminal device is provided, which includes: a second memory configured to store predetermined computer instructions; and a second processor configured to run the predetermined computer instructions to implement part or all of the steps of the method performed by the receiving end.

[0090] It can be understood that the first terminal device can also implement part or all of the functions that the receiving end can implement; and the second terminal device can also implement part or all of the functions that the sending end can implement.

[0091] The embodiments of the present application also provide a system comprising the first terminal device and the second terminal device. Optionally, the system can further comprise a server. The system can be applied in various scenarios.

[0092] Exemplarily, the scheme is applied in the application scenario of video conference, and for each terminal / user participating in the video conference, the source and content of the received stream media data are real and reliable. When each terminal participating in the video conference simultaneously acts as a sending terminal and a receiving terminal, the real and reliable of each other can be ensured. Meanwhile, the scheme can also guarantee that the video conference content is not stolen or tampered with, and the conference privacy is guaranteed.

[0093] Exemplarily, the scheme is applied in the application scenario of online live broadcast, and the ordinary user entering the live room can watch the real, non-illegally tampered or forged live broadcast content from the anchor, preventing the live broadcast content from being illegally tampered or forged, and improving the trustworthiness of the live broadcast platform.

[0094] In addition, the system can also be applied in the application scenarios of voice call, game live broadcast, etc.

[0095] Those skilled in the art should understand that, in addition to the foregoing described methods and terminal devices, the embodiments of the present application can also be provided as a client, a computer readable storage medium or a computer program product.

[0096] The method performed by the sending terminal and the receiving terminal provided by the embodiments of the present application can be provided as a computer readable storage medium. That is, the embodiments of the present application also provide a computer readable storage medium storing a computer program, and the computer program is executed by a processor to make the processor execute part or all steps in any implementation manner of the foregoing processing method or verification method.

[0097] The present application can also be in the form of a computer program product implemented on one or more computer readable storage media containing computer usable program code.

[0098] Those skilled in the art can appreciate that the units and steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software, or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0099] It should be understood that, in various embodiments of the present application, the execution order of each step should be determined according to its function and inherent logic, and the size of the step number does not mean the execution order, and does not constitute any limitation on the implementation process of the embodiments.

[0100] It should also be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, the same items or similar items with basically the same function and role, or the concepts with certain differences are distinguished by using "first", "second" and the like in the embodiments of the present application. Those skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and "first", "second" and the like also do not limit certain differences.

[0101] The same and similar parts among various embodiments in the specification can be referred to each other. As long as there is no logical contradiction, different embodiments can be combined with each other.

Claims

1. A real-time communication processing method, characterized in that, Applied to the sending end, the method includes: Extract the current data frame in real time from the collected streaming media data; A hash operation is performed on the original data used to calculate the hash value to obtain the hash value corresponding to the current data frame; wherein, the original data used to calculate the hash value includes: the current data frame, and the hash value corresponding to the previous data frame extracted before the current data frame; in the case that the current data frame is the first data frame extracted from the streaming media data collected during real-time communication, a preset initial value is used to replace the hash value corresponding to the previous data frame; When the current data frame is a keyframe, the private key of the sending end is used to perform a signature operation on the original data used to calculate the signature, thereby calculating the signature corresponding to the keyframe. The original data used to calculate the signature includes: the keyframe, and the hash value corresponding to the previous data frame extracted before the keyframe. The private key of the sending end is used to identify the identity of the sending end or to identify the identity of the user using the sending end. The current data frame and its corresponding hash value are sent to the receiving end; if the current data frame is a key frame, the signature corresponding to the key frame is also sent to the receiving end.

2. The method according to claim 1, characterized in that, Sending the current data frame and the hash value corresponding to the current data frame to the receiving end includes: Embed the hash value corresponding to the current data frame into the metadata of the current data frame; Send the current data frame and its metadata to the receiving end.

3. The method according to claim 2, characterized in that, If the current data frame is a keyframe, send the signature corresponding to the keyframe to the receiving end, including: If the current data frame is a keyframe, the signature corresponding to the keyframe is embedded in the metadata of the current data frame; or, If the current data frame is a key frame, the signature corresponding to the key frame is sent to the receiving end through an independent channel.

4. A real-time communication processing method, characterized in that, Applied to the receiving end, the method includes: Receive the data frame to be verified and the corresponding hash value to be verified; A hash operation is performed on the data frame to be verified and the hash value corresponding to the previous data frame of the received data frame to be verified, so as to verify the hash value to be verified; wherein, if the data frame to be verified is the first data frame received, a preset initial value is used to replace the hash value corresponding to the previous data frame. If the data frame to be verified is a key frame, a signature verification operation is performed using the public key of the sender, the data frame to be verified, the hash value corresponding to the previous data frame of the data frame to be verified, and the received signature to be verified, in order to verify the signature to be verified. If the verification of the hash value to be verified passes, and the verification of the signature to be verified passes if the data frame to be verified is a keyframe, then the data frame to be verified is rendered and played.

5. The method according to claim 4, characterized in that, Also includes: If any of the unverified hash values ​​and unverified signatures fails verification, a prompt message is generated to indicate that there is an anomaly in the current real-time communication.

6. The method according to any one of claims 4 to 5, characterized in that, Also includes: If no signature to be verified passes verification within a preset time period, a prompt message is generated to indicate that there is an anomaly in the current real-time communication.

7. A terminal device, characterized in that, include: Memory, configured to store predetermined computer instructions; A processor configured to execute the predetermined computer instructions to implement the method of any one of claims 1 to 3; or to implement the method of any one of claims 4 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the processor performs the method as described in any one of claims 1 to 3, or performs the method as described in any one of claims 4 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is run, it causes the computer to perform the method according to any one of claims 1 to 6.

Images