Media stream processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention discloses a media stream processing method and device, equipment, a storage medium and a program product, and the method comprises the steps: receiving media stream copy information transmitted by a new wireless voice media surface in a call process between a first terminal and a second terminal, the media stream copy information comprises a copy media stream obtained by copying a first media stream sent by the first terminal; performing fraud behavior detection on the copied media stream through a media stream detection model to obtain a target detection result; and under the condition that the target detection result is that a fraud behavior exists, generating subtitles representing anti-fraud prompt information, sending the subtitles to the new wireless voice media surface, and enabling the new wireless voice media surface to superpose the subtitles on the first media stream, and sending the first media stream superposed with the subtitles to the second terminal.

Description

Technical Field

[0001] This application belongs to the technical field of voice call services, and specifically relates to a media stream processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Some call services, such as the new 5G call service, belong to the Voice over Long-Term Evolution (VoLTE) audio and video services. In related technologies, the anti-fraud scheme for the new 5G call service is the same as the anti-fraud scheme for the IP Multimedia Subsystem (IMS) audio and video call service. For IMS audio and video call service, the S-CSCF network element of the IMS core network triggers the report to the IMS fraud call prevention system through the Initial Filter Criteria (iFC) during call signaling negotiation. The call is then identified and allowed according to the policy control of the management system (whether it is a fraudulent number, whether there is malicious behavior).

[0003] In new call service scenarios, the technical solutions for artificial intelligence (AI) fraud (such as voice spoofing, face swapping, and facial expression attribute spoofing) cannot reliably solve the problems of potential fraud risks, that is, the accuracy of fraud behavior identification is low. Summary of the Invention

[0004] To address the issue of low accuracy in fraud detection in related technologies, this application proposes a media stream processing method, apparatus, device, storage medium, and program product.

[0005] This application provides a media stream processing method, applied in a first platform, the method comprising:

[0006] During a call between the first terminal and the second terminal, media stream copying information sent by the Voice Over New Radio (VoNR+) media plane is received. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal.

[0007] Fraud detection is performed on the copied media stream using a media stream detection model to obtain the target detection result;

[0008] If the target detection result indicates fraudulent behavior, a subtitle indicating anti-fraud prompt information is generated, the subtitle is sent to the new wireless voice media surface, the new wireless voice media surface overlays the subtitle onto the first media stream, and the first media stream with the overlaid subtitle is sent to the second terminal.

[0009] In some embodiments, the media stream detection model includes a video detection model and an audio detection model; the step of detecting fraudulent behavior in the copied media stream using the media stream detection model to obtain a detection result includes: separating the video stream and the audio stream in the copied media stream; detecting fraudulent behavior in the video stream using the video detection model to obtain a video detection result; detecting fraudulent behavior in the audio stream using the audio detection model to obtain an audio detection result; and obtaining the target detection result based on the video detection result and the audio detection result.

[0010] As can be seen, the embodiments of this application can separate the video stream and the audio stream from the copied media stream, and then perform fraud detection on the video stream and the audio stream respectively to obtain the video detection result and the audio detection result. Then, based on the comprehensive consideration of the video detection result and the audio detection result, the target detection result can be determined more accurately, which is conducive to improving the accuracy of fraud detection.

[0011] In some embodiments, the media stream detection module further includes an image detection model; before receiving media stream copy information sent by a new wireless voice media plane, the method further includes: receiving a captured image from the camera of the first terminal; obtaining the target detection result based on the video detection result and the sound detection result includes: performing fraud detection on the captured image using the image detection model to obtain the image detection result; and obtaining the target detection result based on the image detection result, the video detection result, and the sound detection result.

[0012] As can be seen, the embodiments of this application can separate the video stream and the audio stream from the copied media stream. After receiving the captured image sent by the first terminal, fraud detection is performed on the captured image, video stream and audio stream respectively to obtain image detection results, video detection results and audio detection results. Then, based on the comprehensive consideration of the image detection results, video detection results and audio detection results, the target detection result can be determined more accurately, that is, it is beneficial to improve the accuracy of fraud detection.

[0013] In some embodiments, the media stream copying information further includes location information, which indicates the position of the first media stream on the new wireless voice media surface; sending the subtitles to the new wireless voice media surface, so that the new wireless voice media surface overlays the subtitles onto the first media stream, includes: sending the subtitles and the location information to the new wireless voice media surface, so that the new wireless voice media surface overlays the subtitles onto the first media stream according to the location information. It can be seen that when the first platform sends the subtitles and location information to the new wireless voice media surface, the new wireless voice media surface can accurately associate the first media stream and the subtitles according to the location information, thereby accurately adding the subtitles.

[0014] In some embodiments, the media stream copying information further includes the receiving address of the first media stream; the step of sending the subtitle to the new wireless voice media surface, causing the new wireless voice media surface to overlay the subtitle on the first media stream, and sending the first media stream with the overlaid subtitle to the second terminal includes: sending the subtitle and the receiving address of the first media stream to the new wireless voice media surface, causing the new wireless voice media surface to overlay the subtitle, and sending the first media stream with the overlaid subtitle to the second terminal according to the receiving address of the first media stream.

[0015] It can be seen that the new wireless voice media interface can accurately send the first media stream with superimposed subtitles to the second terminal based on the receiving address of the first media stream, so that the second terminal can be aware of the fraudulent behavior in the media stream in a timely manner.

[0016] In some embodiments, generating a subtitle indicating anti-fraud warning information when the target detection result indicates fraudulent behavior includes: generating the subtitle indicating anti-fraud warning information in response to receiving a subtitle addition request sent by a first application server and the target detection result indicating fraudulent behavior.

[0017] This application embodiment also provides another media stream processing method, applied in a first server, the method comprising:

[0018] During a call between the first terminal and the second terminal, a media stream copying request is sent to the new wireless voice media plane through the new wireless voice capability network element (VoNR+ capability network element), so that the new wireless voice media plane sends media stream copying information to the first platform. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal.

[0019] The first platform is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain a target detection result; if the target detection result indicates the presence of fraudulent behavior, it generates a subtitle indicating anti-fraud prompt information and sends the subtitle to the new wireless voice media surface; the new wireless voice media surface is used to overlay the subtitle onto the first media stream and send the first media stream with the overlaid subtitle to the second terminal.

[0020] In some embodiments, after sending a media stream copy request to a new wireless voice media plane via a new wireless voice capability network element, the method further includes: obtaining a target detection result indicating whether fraudulent behavior exists in the copied media stream; if the target detection result indicates fraudulent behavior exists, generating a subtitle addition request and sending the subtitle addition request to the first platform. Thus, by sending the subtitle addition request to the first platform, the first server can trigger the first platform to generate subtitles indicating anti-fraud prompts, thereby enabling anti-fraud prompts.

[0021] In some embodiments, before the first terminal and the second terminal communicate, the method further includes: receiving at least the subscription information for anti-fraud services from the second terminal. Thus, by storing the subscription information for the second terminal's anti-fraud services on the first server, the first server can provide anti-fraud value-added services to the second terminal.

[0022] This application embodiment also provides a media stream processing apparatus, applied in a first platform, the apparatus comprising:

[0023] The receiving module is used to receive media stream copying information sent by a new wireless voice media plane during a call between the first terminal and the second terminal. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal.

[0024] The detection module is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain the target detection result;

[0025] The first processing module is configured to generate a subtitle indicating anti-fraud prompts when the target detection result indicates fraudulent behavior, send the subtitles to the new wireless voice media surface, overlay the subtitles onto the first media stream on the new wireless voice media surface, and send the first media stream with the overlaid subtitles to the second terminal.

[0026] This application embodiment also provides another media stream processing apparatus, applied in a first server, the apparatus comprising:

[0027] The generation module is used to generate a media stream copy request during a call between the first terminal and the second terminal. The media stream copy information includes copying the first media stream sent by the first terminal to obtain a copied media stream.

[0028] The second processing module is used to send a media stream copying request to the new wireless voice media plane through the new wireless voice capability network element, so that the new wireless voice media plane sends the media stream copying information to the first platform.

[0029] The first platform is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain a target detection result; if the target detection result indicates the presence of fraudulent behavior, it generates a subtitle indicating anti-fraud prompt information and sends the subtitle to the new wireless voice media surface; the new wireless voice media surface is used to overlay the subtitle onto the first media stream and send the first media stream with the overlaid subtitle to the second terminal.

[0030] This application also provides an electronic device, which includes a processor and a memory for storing a computer program that can run on the processor; wherein the processor is used to run the computer program to execute any of the media stream processing methods applied to a first platform or any of the media stream processing methods applied to a first server.

[0031] This application embodiment also provides a computer storage medium storing a computer program, which, when executed by a processor, implements any of the media stream processing methods applied to the first platform or any of the media stream processing methods applied to the first server.

[0032] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the media stream processing methods applied to the first platform or any of the media stream processing methods applied to the first server.

[0033] As can be seen, the embodiments of this application can detect fraudulent behavior in copied media streams through a media stream detection model, which is beneficial for AI fraud such as voice spoofing, face swapping, and facial expression attribute spoofing, thereby improving the accuracy of fraud identification. Furthermore, after generating subtitles that represent anti-fraud prompts, the second terminal can be informed of the fraudulent behavior in the media stream in a timely manner by sending the first media stream with superimposed subtitles to the second terminal. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the architecture of the new call system provided in the embodiments of this application;

[0035] Figure 2 This is a flowchart illustrating a media stream processing method applied to a first platform according to an embodiment of this application;

[0036] Figure 3 This is a flowchart of a media stream processing method applied to a first server according to an embodiment of this application;

[0037] Figure 4 An interactive flowchart of the media stream processing method provided in the embodiments of this application;

[0038] Figure 5 This is a schematic diagram of the structure of a media stream processing device applied to a first platform according to an embodiment of this application;

[0039] Figure 6 This is a schematic diagram of the structure of a media stream processing device applied to a first server according to an embodiment of this application;

[0040] Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] With the increasing intelligence and larger screens of mobile terminals, users' demands for real-time communication are no longer limited to voice or image exchange between callers. More complex interactive needs are gradually emerging, giving new life and vitality to call services, leading to the birth of 5G new calling. In the future, 5G new calling will provide users with a brand-new multimedia, visual, and fully interactive experience, injecting new vitality into the development of 5G and revolutionizing traditional calling methods.

[0042] However, with the widespread application of 5G new calling technologies, their security issues are becoming increasingly prominent. Currently, new communication technologies such as deep learning video spoofing, synthetic voice, and social robots are becoming tools for telecommunications network fraud, possessing extremely high deceptiveness. Fraudsters may use deep learning algorithms to forge videos, use AI algorithms to mimic real voices, generate realistic speech, and synthesize a person's face and voice into the video to make it look like a specific person, before making 5G calls to further commit fraud.

[0043] To address the aforementioned security issues in new 5G call service scenarios, this application proposes a technical solution. By employing the technical solution of this application, fraudulent activity can be effectively identified during real-time calls between two terminals, and a warning can be issued, thereby minimizing the risk of potential telecom fraud.

[0044] The technical solution of this application embodiment can be implemented based on the 5G new voice system. Figure 1 This is a schematic diagram of the architecture of the 5G new calling system provided in the embodiments of this application, such as... Figure 1As shown, in the 5G new calling system, the VoNR+ capability network element 101, the Session Border Controller-User plane (SBC-U) 102, and the VoNR+ media plane 103 are existing network elements in the new calling system. The SBC-U 102 can form a communication connection with the VoNR terminal 104 and the VoNR+ media plane 103, and the VoNR+ media plane 103 can communicate and interact with the VoNR+ capability network element 101. This embodiment introduces a first server 105 and a first platform 106 based on the existing network elements to realize the anti-fraud function of the 5G new calling system. For example, the first server 105 can be an anti-fraud service application server (AS) or other application servers, and the first platform 106 can be an anti-fraud capability platform or other platforms.

[0045] The first server 105 can provide anti-fraud value-added services and is responsible for controlling the anti-fraud services, controlling the VoNR+ media plane 103 to perform media stream copying and subtitle synthesis. The first platform 106 can internally process the media stream pushed by the VoNR+ media plane 103, detect whether the media stream has fraudulent behavior, and obtain the target detection result; the first platform 106 can also select the subtitles to be synthesized for the media stream based on the target detection result and return the subtitles to the VoNR+ capability network element 101.

[0046] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application. Furthermore, the embodiments provided below are some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination.

[0047] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element.

[0048] This application provides a media stream processing method and apparatus. The media stream processing method provided in this application includes a series of steps, but the media stream processing method provided in this application is not limited to the steps described. Similarly, the media stream processing apparatus provided in this application includes a series of modules, but the apparatus provided in this application is not limited to the modules explicitly described, but may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0049] Figure 2 This is a flowchart of a media stream processing method applied to a first platform according to an embodiment of this application, such as... Figure 2 As shown, the process includes:

[0050] Step 201: During the call between the first terminal and the second terminal, receive media stream copying information sent by the new wireless voice media plane. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal.

[0051] In this embodiment, the first terminal and the second terminal can be VoNR terminals or other terminals that can provide audio and video call services, and the new wireless voice media plane is the VoNR+ media plane 103 described above.

[0052] In some embodiments, an anti-fraud service agreement may be signed before performing step 201. The anti-fraud service agreement signing process may include the following steps:

[0053] Step S1: The Business & Operation Support System (BOSS) of the call terminal signs up for terminal-level anti-fraud services with the new call operation management platform.

[0054] The calling terminal can be either the first terminal or the second terminal mentioned above. For example, the BOSS (Best Boss) of the calling terminal sends a subscription request for the anti-fraud service to the new calling operation management platform. The new calling operation management platform saves the subscription relationship of the user number range of the calling terminal for the service, and then returns a successful subscription response to the BOSS.

[0055] Step S2: The new call operation management platform signs a terminal-level anti-fraud service agreement with the first server 105 and the VoNR+ capability network element 101.

[0056] For example, the new call operation management platform sends a request for new user subscription information to the first server 105 and the VoNR+ capability network element 101 respectively. After the first server 105 and the VoNR+ capability network element 101 save the relevant subscription information, they return the subscription result.

[0057] Step S3: VoNR+ capability network element 101 signs up for terminal-level anti-fraud service with VoLTE network element. VoLTE network element saves the anti-fraud service signing information to Unified Data Management (UDM) network element through transparent data transmission.

[0058] During a call between the first and second terminals, the audio and video media streams can be anchored to the VoNR+ media plane 103, and then the transmission path of the audio and video media streams on the user plane can be determined. In one example, the first terminal can be denoted as UE-A, the second terminal can be denoted as UE-B, and the transmission path of the audio and video media streams on the user plane is UE-A<-->SBC-U<-->VoNR+ media plane 103<-->SBC-U<-->UE-B.

[0059] After the audio and video media streams are anchored to the VoNR+ media plane 103, the first platform 106 can receive the media stream copying information sent by the media plane.

[0060] In some embodiments, the first media stream sent by the first terminal is the aforementioned audio and video media stream. Based on the transmission path of the aforementioned audio and video media stream, when the VoNR+ media plane 103 receives the first media stream, it can copy the first media stream to obtain a copied media stream, and send the copied media stream to the first platform 106.

[0061] Step 202: Detect fraudulent behavior in the copied media stream using a media stream detection model to obtain the target detection result.

[0062] In this embodiment of the application, there may be one or more media stream detection modules. After the media stream detection model is pre-trained, the media stream detection model can be deployed to the first platform 106.

[0063] The target detection result may indicate whether fraudulent behavior exists or not. In some embodiments, the target detection result can be represented by an anti-fraud judgment result field. When the target detection result indicates that fraudulent behavior exists, the anti-fraud judgment result field is set to 1; when the target detection result indicates that fraudulent behavior does not exist, the anti-fraud judgment result field is set to 0.

[0064] Step 203: If the target detection result indicates the presence of fraudulent behavior, generate a subtitle indicating anti-fraud prompt information, send the subtitle to the new wireless voice media surface, overlay the subtitle onto the first media stream on the new wireless voice media surface, and send the first media stream with the overlaid subtitle to the second terminal.

[0065] In some embodiments, this step is implemented as follows: if the target detection result indicates the presence of fraudulent behavior, and a subtitle addition request is received from the first application server, a subtitle representing anti-fraud warning information is generated.

[0066] In some embodiments, if the target detection result indicates that there is no fraudulent activity, the first platform 106 does not need to generate subtitles indicating anti-fraud prompts, and the new wireless voice media surface can directly send the first media stream to the second terminal.

[0067] In practical applications, steps 201 to 203 can be implemented based on a processor, which can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor.

[0068] As can be seen, the embodiments of this application can detect fraudulent behavior in copied media streams through a media stream detection model, which is beneficial for AI fraud such as voice spoofing, face swapping, and facial expression attribute spoofing, thereby improving the accuracy of fraud identification. Furthermore, after generating subtitles that represent anti-fraud prompts, the second terminal can be informed of the fraudulent behavior in the media stream in a timely manner by sending the first media stream with superimposed subtitles to the second terminal.

[0069] In some embodiments of this application, the media stream detection model includes a video detection model and an audio detection model. The process of detecting fraudulent activity in the copied media stream using the media stream detection model to obtain detection results may include: separating the video stream and audio stream from the copied media stream; performing fraudulent activity detection on the video stream using the video detection model to obtain a video detection result; performing fraudulent activity detection on the audio stream using the audio detection model to obtain an audio detection result; and obtaining a target detection result based on the video detection result and the audio detection result.

[0070] Here, after separating the video and audio streams from the copied media stream, the video stream can be input into a video detection model, and the model can then output the video detection result; similarly, the audio stream can be input into an audio detection model, and the model can then output the audio detection result. The video detection result indicates whether the video stream contains fraudulent activity or not, while the audio detection result indicates whether the audio stream contains fraudulent activity or not.

[0071] For example, if the video detection result indicates that the video stream contains fraudulent behavior, or the audio detection result indicates that the audio stream contains fraudulent behavior, it can be determined that the target detection result indicates that the first media stream contains fraudulent behavior. In this case, the anti-fraud judgment result field can be set to 1. If the video detection result indicates that the video stream does not contain fraudulent behavior, and the audio detection result indicates that the audio stream does not contain fraudulent behavior, it can be determined that the target detection result indicates that the first media stream does not contain fraudulent behavior. In this case, the anti-fraud judgment result field can be set to 0.

[0072] As can be seen, the embodiments of this application can separate the video stream and the audio stream from the copied media stream, and then perform fraud detection on the video stream and the audio stream respectively to obtain the video detection result and the audio detection result. Then, based on the comprehensive consideration of the video detection result and the audio detection result, the target detection result can be determined more accurately, which is conducive to improving the accuracy of fraud detection.

[0073] In some embodiments of this application, the media stream detection model further includes an image detection model; before receiving media stream copy information sent by the new wireless voice media plane, the first platform 106 may also receive a captured image from the camera of the first terminal.

[0074] For example, the first platform 106 may include a storage module, a detection module, a separation module, an output module, and a subtitle generation module; wherein, the storage module is used to store the captured image sent by the terminal and the received media stream copy information. The separation module is used to separate the video stream and audio stream of the copied media stream. The detection module is used to perform fraud detection on the captured image, video stream, and audio stream respectively, thereby obtaining the target detection result. The output module is used to output the target detection result. The subtitle generation module is used to generate subtitles indicating anti-fraud prompts when the target detection result indicates the presence of fraudulent behavior.

[0075] The captured image from the first terminal's camera can be a face image or other images. For example, the first terminal's camera can be a single camera, such as a front-facing camera; the first terminal's camera can also be a dual-camera setup, which can be a combination of two color cameras using Red Green Blue (RGB) mode, a combination of an RGB mode color camera and a Mono camera, a combination of a wide-angle lens camera and a telephoto lens camera, or a combination of an RGB mode color camera and a depth camera. The first terminal may also include a camera with focus adjustment functionality.

[0076] Accordingly, the process of obtaining the target detection result based on the video detection result and the sound detection result includes: performing fraud detection on the captured image using an image detection model to obtain the image detection result; and obtaining the target detection result based on the image detection result, the video detection result, and the sound detection result.

[0077] For example, if the image detection result indicates fraud in the captured image, or the video detection result indicates fraud in the video stream, or the audio detection result indicates fraud in the audio stream, it can be determined that the target detection result indicates fraud in the first media stream. In this case, the anti-fraud judgment result field can be set to 1. If the image detection result indicates no fraud in the captured image, and the video detection result indicates no fraud in the video stream, and the audio detection result indicates no fraud in the audio stream, it can be determined that the target detection result indicates no fraud in the first media stream. In this case, the anti-fraud judgment result field can be set to 0.

[0078] As can be seen, the embodiments of this application can separate the video stream and the audio stream from the copied media stream. After receiving the captured image sent by the first terminal, fraud detection is performed on the captured image, video stream and audio stream respectively to obtain image detection results, video detection results and audio detection results. Then, based on the comprehensive consideration of the image detection results, video detection results and audio detection results, the target detection result can be determined more accurately, that is, it is beneficial to improve the accuracy of fraud detection.

[0079] In some embodiments of this application, the media stream replication information also includes location information, which indicates the location of the first media stream on the new wireless voice media plane; the location information can be presented through a location field.

[0080] The process of sending the subtitles to a new wireless voice media surface, and then overlaying the subtitles onto the first media stream on the new wireless voice media surface, includes: sending the subtitles and location information to the new wireless voice media surface, and then having the new wireless voice media surface overlay the subtitles onto the first media stream based on the location information. It can be seen that the first platform then sends the subtitles and location information to the new wireless voice media surface, allowing the new wireless voice media surface to accurately associate the first media stream and the subtitles based on the location information, thereby enabling accurate subtitle addition.

[0081] In some embodiments of this application, the media stream copying information further includes the receiving address of the first media stream; sending subtitles to a new wireless voice media surface, overlaying subtitles onto the first media stream on the new wireless voice media surface, and sending the first media stream with overlaid subtitles to the second terminal, including:

[0082] The subtitles and the receiving address of the first media stream are sent to the new wireless voice media surface, so that the subtitles are superimposed on the new wireless voice media surface, and the first media stream with superimposed subtitles is sent to the second terminal according to the receiving address of the first media stream.

[0083] It can be seen that the new wireless voice media interface can accurately send the first media stream with superimposed subtitles to the second terminal based on the receiving address of the first media stream, so that the second terminal can be aware of the fraudulent behavior in the media stream in a timely manner.

[0084] Figure 3 This is a flowchart of a media stream processing method applied to a first server according to an embodiment of this application, as follows: Figure 3 As shown, the process includes:

[0085] Step 301: During the call between the first terminal and the second terminal, a media stream copying request is sent to the new wireless voice media plane through the new wireless voice capability network element, so that the new wireless voice media plane sends the media stream copying information to the first platform. The media stream copying information includes the copied media stream obtained by copying the first media stream sent by the first terminal.

[0086] Here, the first platform 106 is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain a target detection result; if the target detection result indicates the presence of fraudulent behavior, it generates a subtitle indicating anti-fraud prompt information and sends the subtitle to the new wireless voice media surface; the new wireless voice media surface is used to overlay the subtitle onto the first media stream and send the first media stream with the overlaid subtitle to the second terminal.

[0087] In this embodiment of the application, after receiving a media stream copying request, the new wireless voice media plane can copy the first media stream sent to the first terminal to obtain a copied media stream, and then send the copied media stream to the first platform 106.

[0088] In practical applications, steps 201 to 203 can be implemented based on a processor, which can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0089] As can be seen, the embodiments of this application can detect fraudulent behavior in copied media streams through a media stream detection model, which is beneficial for AI fraud such as voice spoofing, face swapping, and facial expression attribute spoofing, thereby improving the accuracy of fraud identification. Furthermore, after generating subtitles that represent anti-fraud prompts, the second terminal can be informed of the fraudulent behavior in the media stream in a timely manner by sending the first media stream with superimposed subtitles to the second terminal.

[0090] In some embodiments of this application, after the first server 105 sends a media stream copy request to the new wireless voice media plane through the new wireless voice capability network element, it can also obtain a target detection result indicating whether fraudulent behavior exists in the copied media stream. If the target detection result indicates fraudulent behavior, a subtitle addition request is generated and sent to the first platform 106. In this way, by sending the subtitle addition request to the first platform, the first server can trigger the first platform to generate subtitles indicating anti-fraud prompts, thereby enabling anti-fraud prompts.

[0091] In some embodiments of this application, before the first terminal and the second terminal communicate, the first server 105 needs to receive at least the anti-fraud service subscription information of the second terminal. Thus, by receiving the anti-fraud service subscription information of the second terminal on the first server, the first server can provide anti-fraud value-added services to the second terminal.

[0092] Figure 4 The interactive flowchart of the media stream processing method provided in the embodiments of this application is as follows: Figure 4 As shown, the interaction flow includes:

[0093] Step a1: VoNR+ media interface transmits the captured image to the anti-fraud capability platform.

[0094] Here, UE-A can periodically capture several sets of facial images through the camera during a call, and transmit the captured images to the anti-fraud capability platform via the VoNR+ media plane.

[0095] Step a2: After the audio and video media stream is connected, VoNR+ media sends a heartbeat message to the anti-fraud capability platform to indicate that the media stream has been established.

[0096] Step a3: The anti-fraud service AS sends a media stream replication request to the VoNR+ capability network element.

[0097] For example, after the terminal sends the periodically captured images to the anti-fraud capability platform, the anti-fraud capability platform can interact with the anti-fraud service AS to trigger the anti-fraud service AS to execute step a2.

[0098] Step a4: Select the anti-fraud capability platform for VoNR+ capability network elements.

[0099] In this step, the VoNR+ capability network element can select the anti-fraud capability platform based on its own location. The address of the anti-fraud capability platform is URL1.

[0100] Step a5: The VoNR+ capability network element sends a media stream replication request to the VoNR+ media plane.

[0101] The media stream copy request sent to the VoNR+ media face can include the address URL1 of the anti-fraud capability platform.

[0102] Step a6: VoNR+ media pushes media stream copying information to the anti-fraud capability platform.

[0103] The media stream replication information includes Session Description Protocol (SDP) information indicating the send and receive addresses of the first media stream, as well as a location field configured on the VoNR+ media plane.

[0104] Step a7: The anti-fraud platform detects fraudulent behavior in the copied media stream and obtains the target detection results.

[0105] Here, after detecting fraudulent behavior in copied media streams, the anti-fraud capability platform can generate an anti-fraud judgment result field.

[0106] Step a8: The anti-fraud platform sends a 200 OK response to the VoNR+ media page.

[0107] The anti-fraud capability platform sends a 200OK response to VoNR+ media, which includes a field for the anti-fraud judgment result.

[0108] Step a9: The VoNR+ media sends a 200 OK response to the VoNR+ capability network element.

[0109] The anti-fraud judgment result field is included in the 200 OK response sent by VoNR+ media to VoNR+ capability network elements.

[0110] Step a10: The VoNR+ capability network element sends a 200 OK response to the anti-fraud service AS.

[0111] The VoNR+ capability network element carries the anti-fraud judgment result field in the 200 OK response sent to the anti-fraud service AS.

[0112] Step a11: The anti-fraud service AS sends a subtitle stream synthesis request to the VoNR+ capability network element.

[0113] Here, the anti-fraud service AS can determine whether to trigger the sending of a subtitle stream synthesis request to the VoNR+ capability network element based on the anti-fraud judgment result field. That is, when the field is 0, it is determined that there is no fraudulent behavior in the media stream during the communication process, so the process ends; when the field is 1, it is determined that there is fraudulent behavior in the media stream during the communication process, so the anti-fraud service AS is triggered to send a subtitle stream synthesis request to the VoNR+ capability network element.

[0114] Step a12: The VoNR+ capability network element sends a subtitle stream synthesis request to the VoNR+ media plane.

[0115] Step a13: The VoNR+ media sends a 200 OK response to the VoNR+ capability network element.

[0116] Step a14: The VoNR+ capability network element sends a 200 OK response to the anti-fraud service AS.

[0117] Step a15: The anti-fraud service AS sends an application to the anti-fraud capability platform to synthesize anti-fraud caption prompts.

[0118] Step a16: The anti-fraud capability platform sends a 200 OK response to the anti-fraud service AS.

[0119] Step a17: The anti-fraud capability platform generates subtitles that represent anti-fraud prompts.

[0120] Step a18: The anti-fraud platform sends the subtitles to the VoNR+ media page.

[0121] Here, the anti-fraud platform sends the subtitles, the configured location field of the VoNR+ media plane, and the SDP information indicating the first media stream receiving address to the VoNR+ media plane.

[0122] Step a19: Use VoNR+ media to overlay subtitles onto the first media stream to obtain the first media stream with overlaid subtitles.

[0123] The VoNR+ media plane can also send the first media stream with superimposed subtitles to the UE-B based on the first media stream receiving address. The UE-B's display screen can display subtitles indicating anti-fraud warnings, such as "The other end may be engaging in fraudulent activities. Please operate with caution."

[0124] It should be noted that step a1 is an optional step. In some other embodiments, step a2 can be executed directly without performing step a1. In this case, in step a7, it is only necessary to perform fraud detection on the video stream and the audio stream.

[0125] In summary, the technical solution adopted in this application embodiment can effectively identify whether fraudulent behavior exists during audio and video calls between two parties on the terminal, and issue real-time subtitle prompts, thereby minimizing the possibility of subsequent telecommunications fraud. Furthermore, this application embodiment only upgrades the existing new call system and does not require modification of existing business processes. It only requires adding two major components to the 5G new call system: the anti-fraud service AS and the anti-fraud capability platform. The solution is highly feasible and has low implementation costs.

[0126] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0127] Figure 5 This is a schematic diagram of the structure of a media stream processing device applied to a first platform according to an embodiment of this application, as shown below. Figure 5 As shown, the device includes:

[0128] The receiving module 501 is used to receive media stream copying information sent by a new wireless voice media plane during a call between the first terminal and the second terminal. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal.

[0129] Detection module 502 is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain target detection results;

[0130] The first processing module 503 is configured to generate a subtitle indicating anti-fraud prompt information when the target detection result indicates the presence of fraudulent behavior, send the subtitle to the new wireless voice media surface, overlay the subtitle onto the first media stream on the new wireless voice media surface, and send the first media stream with the overlaid subtitle to the second terminal.

[0131] In some embodiments, the media stream detection model includes a video detection model and an audio detection model;

[0132] The detection module 502 is used to detect fraudulent behavior in the copied media stream using a media stream detection model, and obtain detection results, including:

[0133] Separate the video stream and audio stream from the copied media stream;

[0134] Fraud detection is performed on the video stream using the video detection model to obtain video detection results; fraud detection is performed on the audio stream using the audio detection model to obtain audio detection results.

[0135] The target detection result is obtained based on the video detection result and the sound detection result.

[0136] In some embodiments, the media stream detection module further includes an image detection model; the receiving module 501 is further configured to receive a captured image from the camera of the first terminal before receiving media stream copy information sent by the new wireless voice media plane.

[0137] The detection module 502 is used to obtain the target detection result based on the video detection result and the sound detection result, including:

[0138] The image detection model is used to detect fraudulent behavior in the captured image, and the image detection result is obtained.

[0139] The target detection result is obtained based on the image detection result, the video detection result, and the sound detection result.

[0140] In some embodiments, the media stream replication information further includes location information, which indicates the location of the first media stream on the new wireless voice media plane;

[0141] The first processing module 503 is configured to send the subtitles to the new wireless voice media surface, so that the new wireless voice media surface overlays the subtitles onto the first media stream, including:

[0142] The subtitles and the location information are sent to the new wireless voice media surface, so that the new wireless voice media surface overlays the subtitles onto the first media stream according to the location information.

[0143] In some embodiments, the media stream copying information further includes the receiving address of the first media stream;

[0144] The first processing module 503 is configured to send the subtitles to the new wireless voice media surface, causing the new wireless voice media surface to overlay the subtitles onto the first media stream, and to send the first media stream with the overlaid subtitles to the second terminal, comprising:

[0145] The subtitles and the receiving address of the first media stream are sent to the new wireless voice media surface, so that the subtitles are superimposed on the new wireless voice media surface, and the first media stream with the superimposed subtitles is sent to the second terminal according to the receiving address of the first media stream.

[0146] In some embodiments, the first processing module 503 is configured to generate subtitles indicating anti-fraud warning information when the target detection result indicates the presence of fraudulent behavior, including:

[0147] In response to receiving a subtitle addition request from the first application server, and the target detection result indicating fraudulent behavior, the subtitle representing the anti-fraud prompt information is generated.

[0148] In practical applications, the receiving module 501, the detection module 502, and the first processing module 503 can be implemented based on the processor and communication device of the first platform.

[0149] Figure 6 This is a schematic diagram of the structure of a media stream processing device applied to a first server according to an embodiment of this application, as shown below. Figure 6 As shown, the device includes:

[0150] The generation module 601 is used to generate a media stream copying request during a call between the first terminal and the second terminal. The media stream copying information includes copying the first media stream sent by the first terminal to obtain a copied media stream.

[0151] The second processing module 602 is used to send a media stream copying request to the new wireless voice media plane through the new wireless voice capability network element, so that the new wireless voice media plane sends the media stream copying information to the first platform.

[0152] The first platform is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain a target detection result; if the target detection result indicates the presence of fraudulent behavior, it generates a subtitle indicating anti-fraud prompt information and sends the subtitle to the new wireless voice media surface; the new wireless voice media surface is used to overlay the subtitle onto the first media stream and send the first media stream with the overlaid subtitle to the second terminal.

[0153] In some embodiments, the second processing module 602 is further configured to, after sending a media stream copy request to a new wireless voice media plane through a new wireless voice capability network element, obtain a target detection result indicating whether there is fraudulent behavior in the copied media stream, generate a subtitle addition request when the target detection result indicates that there is fraudulent behavior, and send the subtitle addition request to the first platform.

[0154] In some embodiments, the second processing module 602 is further configured to receive at least the anti-fraud service subscription information of the second terminal before the first terminal and the second terminal make a call.

[0155] In practical applications, the generation module 601 and the second processing module 602 can be implemented based on the processor and communication device of the first server.

[0156] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0157] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0158] Correspondingly, this application embodiment further provides a computer program product, the computer program product including computer executable instructions, which are used to implement any of the media stream processing methods provided in this application embodiment.

[0159] Accordingly, this application embodiment further provides a computer storage medium storing computer-executable instructions, which are used to implement any of the media stream processing methods provided in the above embodiments.

[0160] This application also provides an electronic device. Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application, as shown below. Figure 7 As shown, the electronic device 70 may include:

[0161] Memory 701 is used to store executable instructions;

[0162] The processor 702 is used to implement any of the media stream processing methods described above when executing executable instructions stored in the memory 701.

[0163] The processor 702 mentioned above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0164] The aforementioned computer-readable storage medium and memory 701 may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; or it may be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0165] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0166] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0167] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0168] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0169] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0171] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A media stream processing method, characterized in that, When applied to a first platform, the method includes: During a call between the first terminal and the second terminal, media stream copying information sent by a new wireless voice media plane is received. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal. Fraud detection is performed on the copied media stream using a media stream detection model to obtain the target detection result; If the target detection result indicates fraudulent behavior, a subtitle indicating anti-fraud prompt information is generated, the subtitle is sent to the new wireless voice media surface, the new wireless voice media surface overlays the subtitle onto the first media stream, and the first media stream with the overlaid subtitle is sent to the second terminal.

2. The method according to claim 1, characterized in that, The media stream detection model includes a video detection model and an audio detection model; The fraud detection of the copied media stream using a media stream detection model, and the resulting detection results, include: Separate the video stream and audio stream from the copied media stream; Fraud detection is performed on the video stream using the video detection model to obtain video detection results; fraud detection is performed on the audio stream using the audio detection model to obtain audio detection results. The target detection result is obtained based on the video detection result and the sound detection result.

3. The method according to claim 2, characterized in that, The media stream detection module further includes an image detection model; before receiving media stream copy information sent by the new wireless voice media plane, the method further includes: receiving a captured image from the camera of the first terminal; The step of obtaining the target detection result based on the video detection result and the sound detection result includes: The image detection model is used to detect fraudulent behavior in the captured image, and the image detection result is obtained. The target detection result is obtained based on the image detection result, the video detection result, and the sound detection result.

4. The method according to claim 1, characterized in that, The media stream replication information also includes location information, which indicates the position of the first media stream on the new wireless voice media plane; Sending the subtitles to the new wireless voice media surface, so that the new wireless voice media surface overlays the subtitles onto the first media stream, includes: The subtitles and the location information are sent to the new wireless voice media surface, so that the new wireless voice media surface overlays the subtitles onto the first media stream according to the location information.

5. The method according to claim 1, characterized in that, The media stream copying information also includes the receiving address of the first media stream; The step of sending the subtitles to the new wireless voice media surface, causing the new wireless voice media surface to overlay the subtitles onto the first media stream, and sending the first media stream with the overlaid subtitles to the second terminal includes: The subtitles and the receiving address of the first media stream are sent to the new wireless voice media surface, so that the subtitles are superimposed on the new wireless voice media surface, and the first media stream with the superimposed subtitles is sent to the second terminal according to the receiving address of the first media stream.

6. The method according to any one of claims 1 to 5, characterized in that, When the target detection result indicates the presence of fraudulent activity, the process of generating subtitles to display anti-fraud warning information includes: In response to receiving a subtitle addition request from the first application server, and the target detection result indicating fraudulent behavior, the subtitle representing the anti-fraud prompt information is generated.

7. A media stream processing method, characterized in that, Applied to a first server, the method includes: During a call between the first terminal and the second terminal, a media stream copying request is sent to the new wireless voice media plane through the new wireless voice capability network element, so that the new wireless voice media plane sends media stream copying information to the first platform. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal. The first platform is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain a target detection result; if the target detection result indicates the presence of fraudulent behavior, it generates a subtitle indicating anti-fraud prompt information and sends the subtitle to the new wireless voice media surface; the new wireless voice media surface is used to overlay the subtitle onto the first media stream and send the first media stream with the overlaid subtitle to the second terminal.

8. The method according to claim 7, characterized in that, After sending a media stream copy request to the new wireless voice media plane via the new wireless voice capability network element, the method further includes: Obtain target detection results used to indicate whether fraudulent activities exist in copied media streams. When the target detection result indicates the presence of fraudulent behavior, a subtitle addition request is generated and sent to the first platform.

9. The method according to claim 7 or 8, characterized in that, Before the first terminal and the second terminal make a call, the method further includes: receiving at least the anti-fraud service subscription information of the second terminal.

10. A media stream processing device, characterized in that, The device, used in a first platform, includes: The receiving module is used to receive media stream copying information sent by a new wireless voice media plane during a call between the first terminal and the second terminal. The media stream copying information includes a copied media stream obtained by copying the first media stream sent by the first terminal. The detection module is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain the target detection result; The first processing module is configured to generate a subtitle indicating anti-fraud prompts when the target detection result indicates fraudulent behavior, send the subtitles to the new wireless voice media surface, overlay the subtitles onto the first media stream on the new wireless voice media surface, and send the first media stream with the overlaid subtitles to the second terminal.

11. A media stream processing device, characterized in that, The device, used in a first server, includes: The generation module is used to generate a media stream copy request during a call between the first terminal and the second terminal. The media stream copy information includes copying the first media stream sent by the first terminal to obtain a copied media stream. The second processing module is used to send a media stream copying request to the new wireless voice media plane through the new wireless voice capability network element, so that the new wireless voice media plane sends the media stream copying information to the first platform. The first platform is used to detect fraudulent behavior in the copied media stream using a media stream detection model to obtain a target detection result; if the target detection result indicates the presence of fraudulent behavior, it generates a subtitle indicating anti-fraud prompt information and sends the subtitle to the new wireless voice media surface; the new wireless voice media surface is used to overlay the subtitle onto the first media stream and send the first media stream with the overlaid subtitle to the second terminal.

12. An electronic device, characterized in that, The electronic device includes a processor and a memory for storing computer programs capable of running on the processor; wherein, The processor is used to run the computer program to perform the media stream processing method according to any one of claims 1 to 9.

13. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the media stream processing method according to any one of claims 1 to 9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the media stream processing method according to any one of claims 1 to 9.

Images