Real-time media data processing method and device and electronic equipment

Abstract

The embodiment of the invention provides a real-time media data processing method and device and electronic equipment, and the method comprises the steps: building a session with a transmitting end, and obtaining the real-time media data transmitted by the transmitting end; identifying whether sensitive information exists in the real-time media data, and generating an identification result; and managing the session according to the identification result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of information technology, and in particular to a real-time media data processing method, device and electronic device. Background Art

[0002] With the development of real-time communications (RTC) technology, applications such as real-time audio and video calls, video conferencing, online education, and telemedicine have grown massively.

[0003] WebRTC is an open Internet standard for embedding real-time multimedia communication capabilities into web pages. WebRTC can use application programming interfaces (APIs) to equip devices with RTC capabilities without requiring users to download plug-ins.

[0004] RTC can bypass security protection services such as Web Application Firewall (WAF), Intrusion Detection System (IDS), Intrusion Prevention System (IPS), etc. Since RTC is a long-term communication and is encrypted after establishment, current security protection services cannot identify sensitive data in RTC communication. Summary of the invention

[0005] Due to restrictions in some regions or hospital policies, patient information can only be stored in the hospital intranet and cannot be exported to the external network or shared with other organizations. Therefore, in RTC-based remote consultation, remote consultation, video conferencing and other application scenarios, how to avoid the leakage of sensitive data has become an urgent problem to be solved.

[0006] In response to at least one of the above problems, embodiments of the present application provide a real-time media data processing method, device, and electronic device.

[0007] According to one aspect of an embodiment of the present application, a real-time media data processing method is provided, including:

[0008] Establishing a session with a sending end and obtaining real-time media data sent by the sending end;

[0009] Identify whether there is sensitive information in the real-time media data, and generate an identification result;

[0010] The session is managed according to the recognition result.

[0011] According to one aspect of an embodiment of the present application, a real-time media data processing device is provided, including:

[0012] An acquisition unit, which establishes a session with a sending end and acquires real-time media data sent by the sending end;

[0013] an identification unit, which identifies whether sensitive information exists in the real-time media data and generates an identification result;

[0014] A session management unit manages the session according to the recognition result.

[0015] According to one aspect of an embodiment of the present application, a storage medium storing a computer-readable program is provided, wherein the computer-readable program enables a computer to execute the real-time media data processing method described in the aforementioned aspect.

[0016] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising the real-time media data processing apparatus described in the aforementioned aspect.

[0017] With reference to the following description and accompanying drawings, the specific embodiments of the present application are disclosed in detail, indicating the way in which the principles of the present application can be adopted. It should be understood that the embodiments of the present application are not limited in scope. Within the scope of the terms of the appended claims, the embodiments of the present application include many changes, modifications and equivalents.

[0018] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation methods of the present application, and together with the text description, explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without creative work. In the drawings:

[0020] Figures 1 to 3 This is a schematic diagram of three typical network topologies of WebRTC;

[0021] Figure 4 is a schematic diagram of a real-time media data processing method according to an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of the method of operation of the embodiment of the present application;

[0023] Figure 6 is a schematic diagram of a process of extracting text information using an OCR engine according to an embodiment of the present application;

[0024] Figure 7is a schematic diagram of a real-time media data processing method according to an embodiment of the present application;

[0025] Figure 8 is a schematic diagram of a real-time media data processing device according to an embodiment of the present application;

[0026] Fig. 9 It is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] With reference to the accompanying drawings, the foregoing and other features of the embodiments of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the embodiments of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the embodiments of the present application include all modifications, variations and equivalents that fall within the scope of the attached claims.

[0028] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements from the title, but do not represent the spatial arrangement or time sequence of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the terms listed in association. The terms "comprise", "include", "have", etc. refer to the existence of the stated features, elements, components or components, but do not exclude the existence or addition of one or more other features, elements, components or components. In the present application, the words "connect", "connected", "coupled", etc. and the like involved in the embodiments are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly.

[0029] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

[0030] Features described and / or shown for one embodiment may be used in one or more other embodiments in the same or similar manner, combined with features in other embodiments, or replace features in other embodiments. The term "includes / comprising" when used herein refers to the presence of a feature, an entirety, a step, or a component, but does not exclude the presence or addition of one or more other features, entireties, steps, or components.

[0031] To facilitate understanding, the following describes the technologies involved in the embodiments of the present application.

[0032] Session Traversal Utilities for NAT (STUN) and Traversal Using Relays around NAT (TURN) are protocols used to solve the Network Address Translation (NAT) problem. In WebRTC applications, STUN and TURN servers are used to help participants establish peer-to-peer (P2P) connections behind NAT. STUN servers are used to detect NAT types and IP addresses. When participants are in different network environments, STUN servers can enable them to exchange IP address information and establish P2P connections, but sometimes NAT blocks the transmission of P2P messages. TURN servers can act as relay nodes between two participants in an end-to-end connection to transmit messages. STUN servers and TURN servers provide the necessary technical support for WebRTC scenarios.

[0033] WebRTC includes three typical network topologies: mesh topology, multipoint control unit (MCU) topology and selective forwarding unit (SFU) topology.

[0034] Mesh is the simplest network structure for WebRTC multi-party sessions. In this structure, each participant sends media streams to all other participants and receives media streams sent by all other participants at the same time, without the participation of a media server. Figure 1 This is a diagram of the Mesh network topology. In the Mesh network structure, each participant is connected to each other in a P2P manner, and data exchange basically does not go through a central server (in some scenarios where P2P cannot be used, it will go through a TURN server). Since each participant needs to provide independent media streams for other participants, N-1 uplinks and N-1 downlinks are required (the links are as follows Figure 1 As shown by the line segment with an arrow in the middle, it should be noted that since the participants are interconnected, for one participant, the link that sends the media stream can be regarded as an uplink, and the link that receives the media stream can be regarded as a downlink, and vice versa).

[0035] The multipoint control unit MCU (Multipoint Control Unit) is a traditional centralized network structure. Figure 2This is a schematic diagram of the MCU network topology. Each participant is only connected to the central server. The MCU central server merges the media streams of all participants to generate a media stream containing the screens of all participants. Participants only need to pull the merged screen. In this scenario, each participant only needs one uplink and one downlink (where the link where the participant sends the media stream to the central server is the uplink, and the link where the central server sends the media stream to the participant is the downlink).

[0036] The Selective Forwarding Unit (SFU) is also a centralized network structure. Figure 3 This is a schematic diagram of the SFU network topology. In the SFU network structure, there is still a central server, but the central server is only responsible for forwarding and does not perform media processing tasks that require large resource consumption, such as confluence and transcoding. Each participant sends media data to the central server and directly receives media data from each other participant from the central server. Each participant requires 1 uplink and N-1 downlinks (where the link where the participant sends the media stream to the central server is the uplink, and the link where the central server sends the media stream to the participant is the downlink).

[0037] The SFU server is considered as a WebRTC participant. It establishes a one-to-one connection with all other participants and acts as a bridge, forwarding the media data of each participant. The SFU server has the ability to replicate media data and can forward the data of one participant to multiple participants. Unlike the TURN server, the TURN server is only an auxiliary data forwarding channel provided for the WebRTC client. It performs transparent data forwarding when P2P cannot be used. The TURN server does not have the ability to replicate and forward media data.

[0038] The embodiment of the present application provides a real-time media data processing method.

[0039] Figure 4 is a schematic diagram of a real-time media data processing method according to an embodiment of the present application. Figure 4 As shown, the method includes:

[0040] 401, establishing a session with a sending end, and obtaining real-time media data sent by the sending end;

[0041] 402, identifying whether sensitive information exists in the real-time media data, and generating an identification result;

[0042] 403: Manage the session according to the recognition result.

[0043] In some embodiments, the real-time media data communication in the present application adopts a centralized network topology, such as the aforementioned MCU or SFU, that is, each media sending end is only connected to the central server, and the functions performed by the aforementioned 401-403 can be integrated into the processor of the central server, and 401-403 are executed by the central server, or integrated into a chip connected to the processor of the central server, and 401-403 are executed under the control of the processor of the central server.

[0044] In some embodiments, the above-mentioned sending end can be one or more user devices (browsers, applications, etc.) participating in real-time communication (such as video conferencing, telemedicine, etc.). The user device can be a smart phone, a laptop computer, a tablet computer, a TV, etc. The central server and the sending end use the Session Description Protocol (SDP) to establish a session (including the caller or the called party). The steps of establishing a session include: the caller configures the session parameters (for example, whether the session includes media streams, media stream types, transmission protocols, media formats and other parameters), creates an initiation (Offer) signaling including the SDP description of the caller, forwards the Offer signaling to the called party via the server, the called party extracts the SDP description in the Offer signaling, and creates an answer (Answer) signaling, returns the answer signaling to the caller, and then the caller and the called party send their respective address + port number information to each other, and a session connection can be established. For the operation of establishing a session, please refer to the relevant technology, which will not be repeated here.

[0045] In some embodiments, after a session is established, real-time media data can be obtained through a media-specific interface (e.g., MediaStream API). The media-specific interface can access a sending end input device (e.g., a camera and / or microphone) to obtain real-time media data. The real-time media data is a real-time media stream, including at least one of an audio stream and a video stream. The format of the real-time media data includes but is not limited to MP3, MP4, AVI, MOV, M4V, etc., which are not given examples one by one here.

[0046] In some embodiments, after the acquired real-time media data passes through the gateway, the real-time media data is stored (cached) (incremental storage) to identify whether sensitive information exists in the real-time media data. For example, the real-time media data can be stored (cached) by the STUN server or TURN server of the aforementioned central server, but the embodiments of the present application are not limited to this.

[0047] Figure 5 402 is a schematic diagram of the implementation method in the embodiment of the present application. Figure 5 As shown, operation 402 includes:

[0048] 501, extracting text information from the real-time media data;

[0049] 502, determining whether the text information includes sensitive information. When the text information includes sensitive information, the recognition result is that sensitive information exists; when the text information does not include sensitive information, the recognition result is that sensitive information does not exist.

[0050] In some embodiments, in 501, an optical character recognition (OCR) engine can be used to extract text information from the real-time media data, and the text information can be information in TXT format, that is, the optical character recognition engine is used to extract text information from the real-time video stream, for example, the optical character recognition engine is used to read the real-time video stream from the aforementioned STUN server or TURN server, and recognize the text information.

[0051] Figure 6 : is a schematic diagram of the process of extracting text information by the optical character recognition engine in the embodiment of the present application, such as Figure 6 As shown, for a real-time video stream, including multiple frames of images, the video stream will contain low-quality image frames due to rapid changes in lighting conditions, camera lenses and motion blur. In order to improve the accuracy of text information recognition and reduce computing costs, blur detection can be performed on the real-time video stream first, and blurry / low-quality image frames can be discarded / filtered. For example, a blur detection algorithm based on fast Fourier transform FFT is used to detect blurry / low-quality image frames and discard the above image frames. The blur detection algorithm can refer to the relevant technology and will not be described here one by one. After filtering out low-quality image frames, the remaining image frames are preprocessed frame by frame, and the preprocessing includes at least one of the following processes: contour detection, constructing a transformation matrix, performing perspective transformation, correcting image viewing angle, and binarization. The implementation methods of the above various preprocessing can refer to the relevant technology and will not be described one by one here; the preprocessed image frames are obtained, and OCR recognition is performed on each processed image frame. The OCR recognition can use the OCR engine of Tesseract or Pytesseract, first calculate and obtain an adaptive threshold to obtain a binary image, then search for text lines and words, recognize character contours, and finally generate words according to character contours, and output the recognized text information. For details, please refer to the relevant technology, and the embodiments of the present application are not limited to this.

[0052] The above example illustrates how to extract text information when the media data is a video stream. When the media data is an audio stream, a speech recognition algorithm can be used to convert the audio stream into text information. The speech recognition algorithm includes but is not limited to a hidden Markov model HMM, a Gaussian mixture model GMM, a deep neural network DNN, a recurrent neural network RNN, a long short-term memory network LSTM, etc. For details, please refer to the relevant technologies, which will not be described here one by one.

[0053] In some embodiments, when the media data includes both an audio stream and a video stream, the methods in the above embodiments can be combined to respectively identify text information in the audio stream and the video stream. Optionally, the text information extracted from the audio stream and the video stream can be time-synchronized to obtain audio text information and video text information corresponding to the same moment.

[0054] In some embodiments, the sensitive information includes: at least one of personal health information and personal identification information; for example, personal health information includes disease diagnosis information, hospitalization information, medical history information, fertility information, etc., and personal identification information includes name, ID information, etc., which are not listed one by one here.

[0055] In some embodiments, in 502, a database or lookup table (LUT) of different types of sensitive keywords can be pre-set, such as different disease diagnosis information keywords, medical history information keywords, fertility information keywords, etc., and the extracted text information is compared with the pre-set sensitive keywords (character matching). When the text information contains the pre-set sensitive keywords, the recognition result is that sensitive information exists; when the text information does not contain the pre-set sensitive keywords, the recognition result is that sensitive information does not exist; or, considering the ambiguity or polysemy of some text information, when character matching is performed, the confidence of the matching result is generated. When the confidence of matching the pre-set sensitive keywords in the text information is greater than or equal to a first threshold, the recognition result is that sensitive information exists; when the confidence of matching the pre-set sensitive keywords in the text information is less than or equal to a second threshold, the recognition result is that sensitive information does not exist; when the confidence of matching the pre-set sensitive keywords in the text information is greater than the second threshold but less than the first threshold, the recognition result is that sensitive information may exist. The first threshold and the second threshold can be determined as needed, and the embodiments of the present application are not limited to this.

[0056] In some embodiments, in 502, a relevant semantic recognition algorithm can be used to identify the extracted text information, perform text classification, and determine whether the text information includes sensitive information. For example, when it is impossible to pre-set keywords such as name or ID information for personal identification information, the name category and ID number category can be pre-set, and a semantic recognition algorithm (a neural network model, which can be pre-trained with the above-mentioned categories of text) can be used to semantically divide the text information and perform text classification. When the classification result is that the text information includes text of the above-mentioned categories, the recognition result is that sensitive information exists. When the classification result is that the text information does not include text of the above-mentioned categories, the recognition result is that sensitive information does not exist. The above keywords and categories are only examples, and the embodiments of the present application are not limited to this; or, considering certain text There is an ambiguous or polysemy problem in the information. When performing text classification, the confidence of the classification result is generated. When the classification result is that the confidence of the text information including the above-mentioned category of text is greater than or equal to the first threshold, the recognition result is that sensitive information exists. When the classification result is that the confidence of the text information including the above-mentioned category of text is less than or equal to the second threshold, the recognition result is that sensitive information does not exist. When the classification result is that the confidence of the text information including the above-mentioned category of text is greater than the second threshold but less than the first threshold, the recognition result is that sensitive information may exist. The first threshold and the second threshold can be determined as needed, and the embodiments of the present application are not limited to this.

[0057] In some embodiments, for real-time video streams, the recognition results of text information extracted from each frame of image (the remaining frames after filtering out low-quality frames) can be determined in units of one image frame, but the embodiments of the present application are not limited to this, and can also be in units of N image frames (N is an integer greater than 1), and the text information extracted from N image frames can be merged to identify sensitive information and generate recognition results. For real-time audio streams, the recognition results of text information extracted within a predetermined time period can be determined in units of audio within the predetermined time period, and the predetermined time period can be the same time period as the time of M frames (M is an integer greater than or equal to 1) of images, or the predetermined time period may not correspond to an image frame, and the embodiments of the present application are not limited to this.

[0058] In the above example, extracting text information from the real-time media data and determining whether the text information includes sensitive information is taken as an example, but the present application is not limited to this. For example, the sensitive information can also be represented as an image, such as a lesion image or a medical scan image or an ID image or a personal photo, etc. A pre-trained neural network model can be used to identify whether each frame image of the real-time video stream contains images of the above-mentioned lesion image or medical scan image or ID image or personal photo category. When it is contained, the recognition result is determined to be the presence of sensitive information; when it is not contained, the recognition result is determined to be the absence of sensitive information; or, when performing image classification, a confidence level of the classification result is generated. When the confidence level of the image containing the above-mentioned category is greater than or equal to a first threshold, the recognition result is the presence of sensitive information; when the confidence level of the image containing the above-mentioned category is less than or equal to a second threshold, the recognition result is the absence of sensitive information; when the confidence level of the image containing the above-mentioned category is greater than the second threshold but less than the first threshold, the recognition result is that sensitive information may exist. The first threshold and the second threshold can be determined as needed, and the embodiments of the present application are not limited to this.

[0059] The pre-set database or lookup table (LUT) of sensitive keywords in the above embodiments, or the pre-trained neural network model can be stored in a memory in the central server, or in an independent memory connected to the central server, but the embodiments of the present application are not limited to this.

[0060] In some embodiments, in 403, the session is managed according to the identification result. When the identification result is that sensitive information exists or may exist, the session is terminated, wherein terminating the session includes closing the corresponding session connection and releasing resources, stopping the transmission of real-time media streams, and avoiding leakage of sensitive information; when the identification result is that the sensitive information does not exist, the session is maintained, and the subsequent media streams are continuously monitored for sensitive information. Alternatively, when the identification result is that sensitive information exists, the session is terminated, wherein terminating the session includes closing the corresponding session connection and releasing resources, stopping the transmission of real-time media streams, and avoiding leakage of sensitive information; when the identification result is that the sensitive information does not exist, the session is maintained, and the subsequent media streams are continuously monitored for sensitive information; when the identification result is that the sensitive information does not exist, the session is maintained, and the subsequent media streams are continuously monitored for sensitive information; when the identification result is that sensitive information may exist, an alarm is issued, and the subsequent media streams are continuously monitored for sensitive information.

[0061] In some embodiments, an alarm may be issued on at least one of the central server and the sending end. The manner of issuing the alarm includes but is not limited to a sound alarm (such as a beep or voice broadcast), or a pop-up alarm superimposed on a real-time video stream, or a graphic alarm displayed on a graphical user interface of a browser or application on the central server or the sending end, etc. The content of the alarm includes the recognition result and may optionally include the type of sensitive information. Participants may also choose to ignore the alarm or manually choose to terminate the session.

[0062] For example, when the real-time media data is a real-time video stream or an audio stream, the session is managed according to the recognition result of the real-time video stream or the audio stream. When the recognition result of the real-time video stream or the audio stream is that sensitive information exists or may exist, the session is terminated; or, when the recognition result of the real-time video stream or the audio stream is that sensitive information may exist, an alarm is issued, and the subsequent video stream or the audio stream is continuously monitored for sensitive information. When the recognition result of the real-time video stream or the audio stream is that the sensitive information does not exist, the session is maintained, and the subsequent video stream or the audio stream is continuously monitored for sensitive information.

[0063] For example, when the real-time media data is a real-time video stream and a real-time audio stream, the session is managed according to the recognition results of the real-time video stream and the audio stream. When the recognition results of the real-time video stream and the real-time audio stream are the same, the session management is performed according to the above example. When the recognition results of the real-time video stream and the real-time audio stream are different, and when there is no sensitive information and there may be sensitive information, respectively, an alarm is issued, and the subsequent video stream and audio stream are continuously monitored for sensitive information. Or when there is sensitive information and there is no sensitive information, respectively, the session is terminated, or when there is sensitive information and there may be sensitive information, respectively, the session is terminated. The above is only an example description, and the embodiments of the present application are not limited to this.

[0064] Figure 7 : is a schematic diagram of a real-time media data processing method according to an embodiment of the present application. Taking a real-time video stream as an example, the method includes:

[0065] 701, establishing a session with a sending end, acquiring and storing a video stream sent by the sending end in real time;

[0066] 702, using an FFT blur detection algorithm to detect blurry / low-quality image frames, and discarding the blurry / low-quality image frames;

[0067] 703, for the remaining image frames, extract text information from them frame by frame using the OCR engine, and set P=1;

[0068] 704, determine whether the text information corresponding to the Pth frame of the remaining image frames includes sensitive information. When the recognition result is that sensitive information exists, execute 705: terminate the session, including the corresponding session connection and release of resources, and stop the transmission of the real-time video stream. When the recognition result is that sensitive information may exist, execute 706: issue an alarm, set P=P+1, and return to 704; otherwise, execute 707: maintain the session, set P=P+1, and return to 704.

[0069] The implementation of 701 to 707 may refer to the aforementioned 401 to 403, and the repeated parts will not be repeated.

[0070] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0071] The embodiment of the present application also provides a real-time media data processing device, which corresponds to the real-time media data processing method in the above-mentioned embodiment.

[0072] Figure 8 is a schematic diagram of a real-time media data processing device according to an embodiment of the present application. Figure 8 As shown, the real-time media data processing device 800 includes:

[0073] An acquisition unit 801 is configured to establish a session with a sending end and acquire real-time media data sent by the sending end;

[0074] An identification unit 802, which identifies whether sensitive information exists in the real-time media data and generates an identification result;

[0075] The session management unit 803 manages the session according to the recognition result.

[0076] The implementation of the acquisition unit 801 , the identification unit 802 , and the session management unit 803 may refer to 401 to 403 , which will not be repeated here.

[0077] In some embodiments, optionally, the device further comprises:

[0078] A relay unit 804, which is used to store the acquired real-time media data;

[0079] Furthermore, the identification unit 802 reads the real-time media data from the relay unit, identifies whether sensitive information exists in the real-time media data, and generates an identification result. The relay unit 804 may be the aforementioned STUN or TURN server, but the embodiment of the present application is not limited thereto.

[0080] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0081] The present application also provides an electronic device, Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application. Fig. 9 As shown, the electronic device 900 may include: at least one interface (not shown in the figure), a processor (eg, a central processing unit (CPU)) 901 , and a memory 902 ; the memory 902 is coupled to the processor 901 .

[0082] In some embodiments, the functions of the real-time media data processing device 800 described in the foregoing embodiments may be integrated into the processor 901. For example, the processor 901 may be configured to: establish a session with a sender, obtain real-time media data sent by the sender; identify whether sensitive information exists in the real-time media data, and generate an identification result; and manage the session according to the identification result.

[0083] Alternatively, the real-time media data processing device 800 may be configured as a chip connected to the processor 901 , and the corresponding functions may be implemented under the control of the processor 901 .

[0084] It is worth noting that the electronic device 900 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0085] In the embodiment of the present application, the processor 901 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices.

[0086] In some embodiments, the memory 902 can store various data; in addition, it also stores a real-time media data processing program 903 and real-time media data and a database or lookup table (LUT) of pre-set sensitive keywords, or a pre-trained neural network model, and executes the program 903 under the control of the processor 901.

[0087] In some embodiments, the electronic device may be, for example, a personal computer, a server, a cloud computer, a workstation, a laptop computer, a mobile terminal, etc.; however, the embodiments of the present application are not limited thereto.

[0088] The embodiment of the present application also provides a computer-readable program, wherein when the program is executed in an electronic device, the program enables the electronic device to execute the real-time media data processing method of the embodiment.

[0089] The embodiment of the present application also provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the real-time media data processing method of the embodiment.

[0090] An embodiment of the present application also provides a medical device (not shown). The medical device described herein includes but is not limited to a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, a C-arm imaging device, a positron emission tomography (PET) device, a single photon emission computed tomography (SPECT) device, an ultrasound device, an X-ray imaging device, or any other suitable medical device.

[0091] For example, the medical device may include the aforementioned real-time media data processing device 800 or electronic device 900. Alternatively, the medical device may be connected to the aforementioned real-time media data processing device 800 or electronic device 900, and the real-time media data processing method may also be independently or jointly implemented by the aforementioned medical device, a computer device connected to the medical device, and a computer device connected to the Internet cloud. The embodiments of the present application are not limited thereto.

[0092] One of the beneficial effects of the embodiments of the present application is that by identifying whether sensitive information exists in real-time media data and managing the session based on the identification result, when sensitive information is identified, an alarm can be issued or the session can be closed in a timely manner to prevent the leakage of sensitive information in the real-time media stream and protect the privacy of patients.

[0093] In addition, an optical character recognition engine is used to extract text information from the real-time video stream, and to identify whether sensitive information exists based on the text information. This makes it possible to efficiently and accurately identify sensitive data in the real-time video stream.

[0094] The above devices and methods of the present application can be implemented by hardware, or by hardware combined with software. The present application relates to such a computer-readable program, which, when executed by a logic component, enables the logic component to implement the above-mentioned devices or components, or enables the logic component to implement the various methods or steps described above. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0095] The method / device described in conjunction with the embodiments of the present application may be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams shown in the figure and / or one or more combinations of the functional block diagrams may correspond to various software modules of the computer program flow or to various hardware modules. These software modules may correspond to the various steps shown in the figure, respectively. These hardware modules may be implemented by solidifying these software modules, for example, using a field programmable gate array (FPGA).

[0096] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in a memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if a device (such as a mobile terminal) uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0097] For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in the present application. For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0098] The present application is described above in conjunction with specific implementation methods, but it should be clear to those skilled in the art that these descriptions are exemplary and are not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles of the present application, and these modifications and variations are also within the scope of the present application.

Claims

1. A real-time media data processing method, characterized in that: The method comprises: Establishing a session with a sending end and obtaining real-time media data sent by the sending end; Identify whether there is sensitive information in the real-time media data, and generate an identification result; The session is managed according to the recognition result.

2. The method according to claim 1, characterized in that The real-time media data includes at least one of an audio stream and a video stream.

3. The method according to claim 1, characterized in that The sensitive information includes at least one of personal health information and personal identification information.

4. The method according to claim 1, characterized in that Identifying whether sensitive information exists in the real-time media data and generating an identification result includes: extracting text information from the real-time media data; Determine whether the text information includes sensitive information. When the text information includes sensitive information, the recognition result is that sensitive information exists; when the text information does not include sensitive information, the recognition result is that sensitive information does not exist.

5. The method according to claim 4, characterized in that An optical character recognition engine is used to extract text information from the real-time media data.

6. The method according to claim 1, characterized in that Managing the session according to the recognition result includes: When the identification result shows that sensitive information exists, terminating the session; When the identification result is that the sensitive information does not exist, the session is maintained.

7. The method according to claim 6, characterized in that Managing the session according to the recognition result also includes: When the identification result indicates that sensitive information may exist, an alarm is issued.

8. A real-time media data processing device, characterized in that: The device comprises: An acquisition unit, which establishes a session with a sending end and acquires real-time media data sent by the sending end; an identification unit, which identifies whether sensitive information exists in the real-time media data and generates an identification result; A session management unit manages the session according to the recognition result.

9. The device according to claim 8, characterized in that The device also includes: A relay unit, which is used to store the acquired real-time media data; Furthermore, the identification unit reads the real-time media data from the relay unit, identifies whether sensitive information exists in the real-time media data, and generates an identification result.

10. A storage medium storing a computer-readable program, characterized in that: The computer-readable program enables a computer to execute the real-time media data processing method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: The electronic device comprises the real-time media data processing device according to claim 7.

Images