Methods and devices for detecting audio information, storage media and electronic devices

By processing audio information through voiceprint matching and semantic recognition models, the problem of low accuracy and efficiency in audio information detection in existing technologies is solved, and the effect of quickly identifying invalid accounts and transferring virtual resources is achieved.

CN116996613BActive Publication Date: 2026-11-17TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210447094.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-26
Publication Date
2026-11-17
Estimated Expiration
2042-04-26

AI Technical Summary

Technical Problem

The accuracy and efficiency of existing audio information detection technologies are low, especially the inaccuracy of judging the hang-up reason status code returned by the operator and the excessive consumption of computing resources for recording recognition.

Method used

The target audio information is processed using a voiceprint matching model and a semantic recognition model. The voiceprint matching model is used to match the target voiceprint information, and the semantic recognition model is used to identify semantic information to determine whether the target account is an invalid account, and to transfer virtual resources when the account is identified as invalid.

Benefits of technology

It improves the efficiency and accuracy of audio information detection, reduces the consumption of computing resources, and lowers the detection cost.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116996613B_ABST
    Figure CN116996613B_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, storage medium, and electronic device for detecting audio information. The method includes: acquiring target audio information to be detected; matching target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with labeled voiceprint information; if no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model, performing semantic recognition on the target audio information according to a preset semantic recognition model to obtain a target recognition result; and if the target recognition result indicates that the target account is an invalid account, transferring the virtual resources corresponding to the target account. This application solves the technical problem of low detection efficiency of audio information in related technologies.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a method and apparatus for detecting audio information, a storage medium, and an electronic device. Background Technology

[0002] Currently, the detection of audio information in outbound voice calls is mainly based on the hang-up reason status code returned by the operator, or by recording and recognizing all the ringing audio of the calling user and verifying the accurate hang-up reason through keywords or manual verification.

[0003] The method of judging the hang-up reason status code returned by the operator cannot guarantee the accuracy of the hang-up reason judgment because the hang-up reason status code returned by the operator is customized by different operators in different regions. This leads to the inability to guarantee the accuracy of audio information detection, which in turn affects the efficiency of audio information detection.

[0004] The method of recording and recognizing all ringing audio of calling users requires recording the ringing audio of all users. Therefore, it is necessary to store the complete ringing audio of users and also requires a lot of computing resources for recognition calculation, resulting in low detection efficiency of audio information.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides an audio information detection method and apparatus, storage medium and electronic device, to at least solve the technical problem of low detection efficiency of audio information in related technologies.

[0007] According to one aspect of the embodiments of this application, an audio information detection method is provided, comprising: acquiring target audio information to be detected, wherein the target audio information is audio information generated after a call is initiated to a target account; matching target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with labeled voiceprint information; if no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model, performing semantic recognition on the target audio information according to a preset semantic recognition model to obtain a target recognition result, wherein the semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to indicate whether the target account is the invalid account; if the target recognition result indicates that the target account is the invalid account, transferring virtual resources corresponding to the target account, wherein the virtual resources are configured to allow transfer when the target account is the invalid account.

[0008] According to another aspect of the embodiments of this application, an audio information detection device is also provided, comprising: an acquisition module, configured to acquire target audio information to be detected, wherein the target audio information is audio information generated after a call is initiated to a target account; a matching module, configured to match target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with labeled voiceprint information; an identification module, configured to perform semantic identification on the target audio information according to a preset semantic identification model when no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model, thereby obtaining a target identification result, wherein the semantic identification model is used to identify the semantics of the target audio information, and the target identification result is used to indicate whether the target account is the invalid account; and a transfer module, configured to transfer virtual resources corresponding to the target account when the target identification result indicates that the target account is the invalid account, wherein the virtual resources are configured to be allowed to be transferred when the target account is the invalid account.

[0009] Optionally, the device is configured to acquire target audio information to be detected by: acquiring a set of target accounts, wherein the set of target accounts is a set of accounts provided by a media information publisher; selecting the target account from the set of target accounts and initiating a call request to the target account; and acquiring the target audio information during the process of initiating a call request to the target account.

[0010] Optionally, the device is configured to select the target account from the target account set and initiate a call request to the target account by: obtaining an invalid account identifier provided by the media information publisher, wherein the invalid account identifier is an identifier obtained by the media information publisher to mark invalid accounts in the target account set; selecting the target account from the target account set according to the invalid account identifier and initiating a call request to the target account, wherein the target account is an account in the target account set that has the invalid account identifier.

[0011] Optionally, the device is further configured to: when the target identification result indicates that the target account is not the invalid account, prohibit the transfer of virtual resources corresponding to the target account to the media information publisher.

[0012] Optionally, the device is used to match target voiceprint information in the target audio information using a preset voiceprint matching model in the following manner: parsing the real-time speech stream of the target audio information according to a preset time period to obtain the target voiceprint information; inputting the target voiceprint information into the voiceprint matching model, and matching the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model through the voiceprint matching model.

[0013] Optionally, the device is configured to input the target voiceprint information into the voiceprint matching model in the following manner, and match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model: when the target voiceprint information matches the first voiceprint information, the first labeling result of the first voiceprint information is determined as the target matching result output by the voiceprint matching model, wherein the labeled voiceprint information includes the first voiceprint information, and the first labeling result includes caller abnormality; the device is further configured to: when the target matching result indicates that the call status of the call with the target account is caller abnormality, stop the call with the target account and generate alarm information, wherein the alarm information is used to indicate that the account currently establishing a call with the target account is abnormal.

[0014] Optionally, the device is configured to input the target voiceprint information into the voiceprint matching model in the following manner, and match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model: when the target voiceprint information matches the second voiceprint information, the second labeling result of the second voiceprint information is determined as the target matching result output by the voiceprint matching model, wherein the labeled voiceprint information includes the second voiceprint information, and the second labeling result includes the called party being abnormal; the device is further configured to: when the target matching result indicates that the call status of the call with the target account is the called party being abnormal, stop the call with the target account, and determine the target account as the invalid account.

[0015] Optionally, the device is configured to perform semantic recognition on the target audio information according to a preset semantic recognition model to obtain a target recognition result when no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model: when no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model, the target audio information is saved; the target audio information is input into the semantic recognition model to perform semantic recognition on the target audio information to obtain the target recognition result, wherein the semantic recognition model is used to obtain the text information corresponding to the target audio information and perform semantic recognition on the text information.

[0016] Optionally, the device is further configured to: if the target identification result also indicates that the call status of the call with the target account is abnormal, stop the call with the target account and generate alarm information, wherein the alarm information is used to indicate that the account currently establishing a call with the target account is abnormal; if the target identification result also indicates that the call status of the call with the target account is abnormal, stop the call with the target account and determine that the target account is the invalid account.

[0017] Optionally, the device is further configured to: when the target identification result indicates that the call status of the call with the target account is abnormal, add the third voiceprint information corresponding to the target audio information to the voiceprint matching model, and determine the annotation result corresponding to the third voiceprint information as abnormal caller; when the target identification result indicates that the call status of the call with the target account is abnormal called party, add the fourth voiceprint information corresponding to the target audio information to the voiceprint matching model, and determine the annotation result corresponding to the fourth voiceprint information as abnormal called party.

[0018] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described audio information detection method when running.

[0019] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio information detection method described above.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described audio information detection method through the computer program.

[0021] In this embodiment, the method involves acquiring target audio information to be detected. This target audio information is generated after a call is initiated to a target account. A preset voiceprint matching model is used to match target voiceprint information within the target audio information with labeled voiceprint information. If no labeled voiceprint information corresponding to the target voiceprint information is found using the voiceprint matching model, a preset semantic recognition model is used to perform semantic recognition on the target audio information to obtain a target recognition result. This semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to represent... The system indicates whether the target account is invalid. If the target account is identified as invalid, virtual resources corresponding to the target account are transferred. These virtual resources are configured to be allowed to be transferred when the target account is invalid. By acquiring audio information generated after a call is initiated to the target account, a voiceprint matching model is used for priority matching. If no voiceprint information is found, a semantic recognition model is then used to identify the target account to determine its validity. This achieves the goal of quickly determining whether virtual resources need to be transferred to an invalid target account, thereby improving the technical efficiency of audio information detection and solving the technical problem of low audio information detection efficiency in related technologies. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1This is a schematic diagram of the application environment of an optional audio information detection method according to an embodiment of this application; Figure 2 This is a flowchart illustrating an optional audio information detection method according to an embodiment of this application; Figure 3 This is a schematic diagram of an optional audio information detection method according to an embodiment of this application; Figure 4 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application; Figure 5 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application; Figure 6 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application; Figure 7 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an optional audio information detection device according to an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an optional audio information detection product according to an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows: ASR: Automatic Speech Recognition, a technology that converts speech into text.

[0026] Ringing audio: In telephone communication, the ringing sound produced before the called party answers is called the ringing audio.

[0027] MRCP: Media Resource Control Protocol, is a protocol used by voice servers to control the media resources used for media streaming (such as ASR and TTS) and how to communicate with these resources.

[0028] Voiceprint recognition is a technology that automatically verifies and matches voice characteristics and spoken content.

[0029] Freeswitch is a free, open-source software-based telephone switching system.

[0030] Intelligent cleaning system: The media information publishing platform uses robots to replace customer service, helping media information publishers to call customers and, based on the users' answers, help media information publishers to complete the initial screening and scoring of customer intentions.

[0031] The present application will be described below with reference to embodiments: According to one aspect of the embodiments of this application, an audio information detection method is provided. Optionally, in this embodiment, the above-mentioned audio information detection method can be applied to, for example... Figure 1 The hardware environment shown consists of server 101 and terminal device 103. For example... Figure 1As shown, server 101 is connected to terminal 103 via a network and can be used to provide services for terminal devices or applications installed on terminal devices. Applications can be video applications, instant messaging applications, browser applications, educational applications, game applications, etc. Database 105 can be set up on the server or independently of the server to provide data storage services for server 101, such as a game data storage server. The network can include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). The wireless network includes Bluetooth, Wi-Fi, and other networks that enable wireless communication. Terminal device 103 can be a terminal configured with an application, and can include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, PDAs, MIDs (Mobile Internet Devices), PADs, desktop computers, smart TVs, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, and other computer devices. The server can be a single server, a server cluster consisting of multiple servers, or a cloud server. The application 107 using the above-described audio information detection method is displayed through terminal device 103 or other connected display devices.

[0032] Combination Figure 1 As shown, the above-mentioned audio information detection method can be implemented in terminal device 103 through the following steps: S1, acquire the target audio information to be detected on the terminal device 103, wherein the target audio information is the audio information generated after a call is initiated to the target account; S2, on the terminal device 103, the target voiceprint information in the target audio information is matched using a preset voiceprint matching model. The voiceprint matching model is used to match the target voiceprint information in the target audio information with the labeled voiceprint information. S3, if the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, the terminal device 103 performs semantic recognition on the target audio information according to the preset semantic recognition model to obtain the target recognition result. The semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to indicate whether the target account is an invalid account. S4, if the target identification result indicates that the target account is an invalid account, the virtual resources corresponding to the target account are transferred on the terminal device 103, wherein the virtual resources are configured to be allowed to be transferred when the target account is an invalid account.

[0033] Optionally, in this embodiment, the above-described audio information detection method can also be implemented via a server, for example, Figure 1 It is implemented in server 101 shown; or it is implemented jointly by the terminal device and the server.

[0034] The above is merely an example, and this embodiment does not impose any specific limitations.

[0035] Alternatively, as an alternative implementation method, such as Figure 2 As shown, the method for detecting the above audio information includes: S202, Obtain the target audio information to be detected, wherein the target audio information is the audio information generated after a call is initiated to the target account; S204, matching the target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with the labeled voiceprint information. S206, if the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, the target audio information is semantically recognized according to the preset semantic recognition model to obtain the target recognition result. The semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to indicate whether the target account is an invalid account. S208, if the target identification result indicates that the target account is an invalid account, transfer the virtual resources corresponding to the target account, wherein the virtual resources are configured to be allowed to be transferred when the target account is an invalid account.

[0036] Optionally, in the embodiments of this application, the above-mentioned audio information detection method may be applied to application scenarios that require determining whether a target account is valid, such as media information release scenarios, customer service promotion scenarios, telephone sales scenarios, and epidemiological investigation information inquiry scenarios.

[0037] For example, Figure 3 This is a schematic diagram of an optional audio information detection method according to an embodiment of this application, such as... Figure 3 As shown, the media information publisher published media information 302 through the media information publishing platform. Media information 302 is a "registration form for the xx exhibition", which provides users with an entry point to enter information such as name and mobile phone number. By adding the name "Zhang San" and mobile phone number, the user obtains media information 304 and provides the relevant information of the target account to the media information publisher or the media information publishing platform by clicking the "submit" button displayed on media information 304.

[0038] Optionally, in this embodiment, the target audio information may include, but is not limited to, the audio information generated after the calling account initiates a call to the target account. Specifically, it may include, but is not limited to, the audio information returned by the operator when the call is not answered by the target account, such as "in a call," "phone off," "phone suspended," "busy," "number not in service," etc., or it may include, but is not limited to, "your number is suspended," "payment is overdue," "service is suspended," etc.

[0039] It should be noted that the aforementioned calling account may include, but is not limited to, the account associated with the entity executing the aforementioned audio information detection method, and the calling account may initiate a call to the target account obtained.

[0040] For example, Figure 4 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application, such as... Figure 4 As shown, the specific steps include, but are not limited to, the following: S1, When a media information publisher publishes media information through a media information publishing platform, the media information publishing platform receives a set of accounts provided by the media information publisher; S2, the media information publishing platform initiates a call sequentially to each account in the account set through the calling account; S3, the media information publishing platform determines the validity of the called target account based on the audio information generated during the call. S4: When the target account is determined to be invalid, transfer the virtual resources associated with the target account to the media information publisher.

[0041] Optionally, in this embodiment, the aforementioned preset voiceprint matching model may include, but is not limited to, a biometric identification model, also known as speaker recognition, which converts sound signals into electrical signals and then uses a computer for identification. A voiceprint is a sound wave spectrum carrying speech information displayed using electroacoustic instruments. Training the voiceprint matching model can generally be achieved through, but is not limited to, the following methods: Template matching method: Utilize Dynamic Time Warping (DTW) to align training and testing feature sequences; Nearest neighbor method: During training, all feature vectors are preserved, and during recognition, the K nearest neighbors among the training vectors are found for each vector, and recognition is performed accordingly; Neural network methods: There are many forms, such as multilayer sensing, radial basis function (RBF), etc. Hidden Markov Model (HMM) methods: typically using single-state HMMs or Gaussian mixture models (GMMs); VQ clustering methods (such as LBG): have low algorithm complexity and can be trained in conjunction with HMM methods to achieve better results; Multinomial classifier method.

[0042] It should be noted that, in the embodiments of this application, the trained voiceprint matching model can match the voiceprint information in the target audio information mentioned above, and match different voiceprint information with parameters representing the current voice call status.

[0043] For example, voiceprint information includes three types: A, B, and C. Among them, A is marked as matching "currently in a call" returned by the operator, B is marked as matching "power off" returned by the operator, and C is marked as matching "your phone is out of credit" returned by the operator.

[0044] By matching the voiceprint information carried in the target audio information with the three types of voiceprint information A, B, and C that have been labeled, the corresponding matching results are matched for the target voiceprint information, so as to directly determine the validity of the target account based on the target voiceprint information.

[0045] Optionally, in this embodiment, the target voiceprint information is voiceprint information extracted in real time from the target audio information using a voiceprint feature extraction model. The labeled voiceprint information indicates that each voiceprint in the preset voiceprint information set is labeled with corresponding voice call status parameters in advance. That is, each voiceprint in the preset voiceprint information set used for matching is labeled with corresponding voice call status parameters. The matching of the target voiceprint information in the target audio information may include, but is not limited to, matching the target voiceprint information with the voiceprint information in the preset voiceprint information set, using the voice call status parameters corresponding to the voiceprint information in the preset voiceprint information set that matches the target voiceprint information as the voice call status parameters of the target voiceprint information, and then determining whether the target account is a valid account based on the matched voice call status parameters, so as to perform subsequent operations.

[0046] Optionally, in this embodiment of the application, the aforementioned voiceprint information that is not matched by the voiceprint matching model and corresponds to the target voiceprint information in the target audio information may include, but is not limited to, the corresponding labeled voiceprint information that is not matched by the voiceprint matching model. That is, the voice call status parameters corresponding to the target voiceprint information cannot be determined.

[0047] Optionally, in this embodiment of the application, semantic recognition of the target audio information according to the preset semantic recognition model may include, but is not limited to, when the voice call status parameters corresponding to the target voiceprint information cannot be determined, recording the target audio information and inputting it into the semantic recognition model for recognition to obtain the target recognition result. The semantic recognition model may include, but is not limited to, recognizing the target audio information as text information and searching for the recognition result corresponding to the text information from the database. The target recognition result may include, but is not limited to, indicating whether the target account is a valid account or whether the target account is an invalid account. Whether the target account is a valid account or an invalid account can be pre-configured by the media information publishing platform, or can be further determined by the recognition result of the target audio information.

[0048] For example, taking the application to media information publishing platforms as an example, after the media information provided by the media information publisher is published on the media information publishing platform, a certain percentage of virtual resources are paid to compensate the media information publisher for the invalid accounts (customer service numbers that are switched off, out of service, or invalid numbers after three consecutive calls) in the accounts collected by the media information publisher. This is to compensate the media information publisher for its losses, and the media information publisher is encouraged to actively mark and return invalid accounts.

[0049] Figure 5 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application, such as... Figure 5 As shown, "Zhang San" and "Li Si" are both invalid accounts marked by the media information publisher. Using the aforementioned audio information detection method, the media information publishing platform initiates calls to "Zhang San" and "Li Si" respectively. Taking "Zhang San" as an example, the process includes, but is not limited to, the following steps: S1, obtain "Zhang San's" account and initiate a call to "Zhang San"; S2, Obtain target audio information; S3, matching the target voiceprint information of the target audio information in the call initiated to "Zhang San" through the voiceprint matching model; S4. If no matching result is obtained, the target audio information is input into the semantic recognition model for recognition. S5: If the matching result of S3 indicates that "Zhang San" is a valid account, or if the identification result of S4 indicates that "Zhang San" is a valid account, then the status of "Zhang San" will be marked as "valid", and no further compensation will be paid.

[0050] Taking "Li Si" as an example, the steps include, but are not limited to, the following: S1, obtain "Li Si's" account and initiate a call to "Li Si"; S2, Obtain target audio information; S3, matching the target voiceprint information of the target audio information in the call initiated to "Li Si" through the voiceprint matching model; S4. If no matching result is obtained, the target audio information is input into the semantic recognition model for recognition. S5, if the matching result of S3 indicates that "Li Si" is an invalid account, or the identification result of S4 indicates that "Li Si" is an invalid account, mark the status of "Li Si" as "invalid", and compensate the media information publisher for the virtual resources associated with "Li Si".

[0051] The above is merely an example, and the embodiments in this application do not impose any specific limitations.

[0052] Optionally, in this embodiment of the application, whether the target account is valid may be determined by the matching result of the voiceprint matching model and the target recognition result. The voiceprint matching model is used for matching first, and the recognition model is used for recognition only if no match is found.

[0053] Optionally, in this embodiment, the validity of the target account may include, but is not limited to, the call status corresponding to the voiceprint or semantic recognition of the target audio information being abnormal, such as being in a call, switched off, out of service, busy, or invalid. For different abnormal caller statuses, the same or different subsequent operations can be performed. For example, virtual resources corresponding to the target account can be transferred. The recipient of the virtual resources may include, but is not limited to, the media information publisher that provided the target account. The virtual resources being configured to be allowed to be transferred when the target account is invalid can be understood as follows: when the target account is invalid, the virtual resources can be directly transferred to the media information publisher; or when the target account is invalid, the audio information detection process of other accounts can be performed, and finally, the virtual resources are uniformly transferred to the media information publisher.

[0054] Optionally, in this embodiment of the application, the aforementioned virtual resources may include, but are not limited to, virtual gold, virtual gifts, virtual services, etc.

[0055] In this embodiment, the method involves acquiring target audio information to be detected. This target audio information is generated after a call is initiated to a target account. A preset voiceprint matching model is used to match target voiceprint information within the target audio information with labeled voiceprint information. If no labeled voiceprint information corresponding to the target voiceprint information is found using the voiceprint matching model, a preset semantic recognition model is used to perform semantic recognition on the target audio information to obtain a target recognition result. This semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to represent... The system indicates whether the target account is invalid. If the target account is identified as invalid, virtual resources corresponding to the target account are transferred. These virtual resources are configured to be allowed to be transferred when the target account is invalid. By acquiring audio information generated after a call is initiated to the target account, a voiceprint matching model is used for priority matching. If no voiceprint information is found, a semantic recognition model is then used to identify the target account to determine its validity. This achieves the goal of quickly determining whether virtual resources need to be transferred to an invalid target account, thereby improving the technical efficiency of audio information detection and solving the technical problem of low audio information detection efficiency in related technologies.

[0056] As an optional approach, the target audio information to be detected is obtained, including: Obtain the target account set, which is the set of accounts provided by the media information publisher; Select the target account from the target account set and initiate a call request to the target account; During the process of initiating a call request to the target account, the target's audio information is obtained.

[0057] Optionally, in this application embodiment, the aforementioned target account set may include, but is not limited to, an account set obtained by the media information publisher based on the provided media information, wherein the account information corresponding to the aforementioned account set is provided by the user or obtained with the user's permission.

[0058] Specifically, the aforementioned target account set may include, but is not limited to, accounts marked as invalid by media information publishers.

[0059] Optionally, in this embodiment of the application, selecting a target account from the target account set may include, but is not limited to, randomly selecting a target account from the target account set and deleting the detected target account from the target account set to avoid duplicate detection.

[0060] Optionally, in this embodiment of the application, the above-mentioned call request to the target account may include, but is not limited to, a call request to the target account initiated by the calling account associated with the media information publishing platform, or a call request to the target account initiated by a third-party platform.

[0061] Optionally, in this embodiment of the application, the process of initiating a call request to the target account may include, but is not limited to, making a call to the target account but not yet being answered by the target account.

[0062] This application's embodiments employ a method of acquiring a target account set, where the target account set is a set of accounts provided by the media information publisher. A target account is selected from the target account set, and a call request is initiated to the target account. During the call request process, target audio information is acquired. By acquiring target accounts pre-labeled by the media information publisher, the enthusiasm of the media information publisher for labeling accounts is increased, encouraging them to actively label and return the validity of accounts. This achieves the goal of reducing the detection cost of audio information, thereby improving the technical effect of increasing the detection efficiency of audio information and solving the technical problem of low detection efficiency of audio information in related technologies.

[0063] As an optional approach, a target account is selected from the target account set, and a call request is initiated to the target account, including: Obtain invalid account identifiers provided by the media information publisher, wherein the invalid account identifiers are identifiers obtained by the media information publisher to mark invalid accounts in the target account set; Select a target account from the target account set based on the invalid account identifier, and initiate a call request to the target account. The target account is an account with an invalid account identifier in the target account set.

[0064] Optionally, in this embodiment of the application, the above-mentioned invalid account identifier is an identifier marked by the media information publisher as an invalid account. The media information publisher can preprocess the account set to reduce the number of accounts that need to be detected, thereby improving the detection efficiency of audio information and solving the technical problem of low detection efficiency of audio information in related technologies.

[0065] This application's embodiments utilize the method of obtaining invalid account identifiers provided by media information publishers. These invalid account identifiers are identifiers obtained by the media information publishers to mark invalid accounts in a target account set. Based on the invalid account identifiers, target accounts are selected from the target account set, and call requests are initiated to these target accounts. The target accounts are accounts in the target account set that have invalid account identifiers. By obtaining invalid target accounts pre-marked by the media information publishers, and only detecting invalid target accounts, the compensation costs of the media information publishing platform are reduced, and the detection accuracy is improved.

[0066] As an alternative approach, the method also includes: If the target identification result indicates that the target account is not an invalid account, the transfer of virtual resources corresponding to the target account to the media information publisher is prohibited.

[0067] Optionally, in this embodiment of the application, the fact that the target account is not an invalid account can be understood as the recognition result of the target audio information indicating that the target account is allowed to be connected. In this case, the target account is a valid account, which is recognized by the media information publisher and avoids additional compensation for the incorrectly marked account.

[0068] By adopting the method of prohibiting the transfer of virtual resources corresponding to the target account to the media information publisher when the target identification result indicates that the target account is not an invalid account, the media information publishing platform is prevented from making compensation to accounts that are valid in themselves but marked as invalid by the media information publisher. This reduces the compensation cost of the media information publishing platform and improves the detection accuracy of the target audio information.

[0069] As an optional approach, a preset voiceprint matching model is used to match target voiceprint information in the target audio information, including: The real-time speech stream of the target audio information is parsed according to a preset time period to obtain the target voiceprint information; The target voiceprint information is input into the voiceprint matching model, and the target voiceprint information is matched with the voiceprint information already labeled in the voiceprint matching model.

[0070] Optionally, in this embodiment of the application, the preset time period may include, but is not limited to, every second, every 5 seconds, etc. Specifically, it can be flexibly adjusted according to actual needs.

[0071] Optionally, in the embodiments of this application, the above-mentioned parsing method may include, but is not limited to, performing synchronous processing of the target audio information outside the system, and matching by converting the real-time speech stream of the target audio information into a voiceprint.

[0072] Through the embodiments of this application, the real-time speech stream of the target audio information is parsed according to a preset time period to obtain the target voiceprint information. The target voiceprint information is then input into the voiceprint matching model, and the target voiceprint information is matched with the voiceprint information already labeled in the voiceprint matching model. This method can quickly complete the detection of the target audio information, improve the timeliness of the detection of the target audio information, reduce the compensation cost of the media information publishing platform, and improve the detection accuracy of the target audio information.

[0073] As an alternative solution, Input the target voiceprint information into the voiceprint matching model, and match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model. This includes: when the target voiceprint information matches the first voiceprint information, determining the first labeling result of the first voiceprint information as the target matching result output by the voiceprint matching model. The labeled voiceprint information includes the first voiceprint information, and the first labeling result includes caller anomalies. The above method also includes: if the target matching result indicates that the call status of the call with the target account is abnormal, stopping the call with the target account and generating alarm information, wherein the alarm information is used to indicate that the account currently establishing a call with the target account is abnormal.

[0074] Optionally, in this embodiment of the application, the aforementioned first voiceprint information is voiceprint information that has been pre-annotated with a first annotation result. For example, the voiceprint corresponding to "Your phone is out of service" is pre-annotated as the first voiceprint information as the first annotation result indicating that the caller is abnormal.

[0075] Optionally, in this embodiment of the application, the above-mentioned caller anomaly may include, but is not limited to, caller anomaly, such as the caller number applied for by the media information publisher or media information publishing platform being in arrears or suspended.

[0076] Optionally, in this embodiment of the application, stopping the call with the target account may include, but is not limited to, hanging up the current call, and generating alarm information may include, but is not limited to, generating alarm prompt information to indicate that the current caller is abnormal.

[0077] In this embodiment, when the target voiceprint information matches the first voiceprint information, the first annotation result of the first voiceprint information is determined as the target matching result output by the voiceprint matching model. The annotated voiceprint information includes the first voiceprint information, and the first annotation result includes caller abnormality. When the target matching result indicates that the call status of the call with the target account is caller abnormality, the call with the target account is stopped, and an alarm message is generated. The alarm message is used to indicate that the account currently establishing a call with the target account is abnormal. This method can actively hang up the phone when caller abnormality occurs and generate alarm message in a timely manner to facilitate subsequent continued detection, thereby improving the timeliness of target audio information detection, reducing the compensation cost of the media information publishing platform, and improving the accuracy of target audio information detection.

[0078] As an alternative solution, Input the target voiceprint information into the voiceprint matching model, and match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model. This includes: when the target voiceprint information matches the second voiceprint information, determine the second labeling result of the second voiceprint information as the target matching result output by the voiceprint matching model. The labeled voiceprint information includes the second voiceprint information, and the second labeling result includes the abnormality of the called party. The method also includes: if the target matching result indicates that the call status of the call with the target account is abnormal, stopping the call with the target account and determining that the target account is an invalid account.

[0079] Optionally, in this embodiment of the application, the aforementioned second voiceprint information is voiceprint information that has been pre-annotated with a second annotation result. For example, the voiceprint corresponding to "The phone number you dialed is switched off" is pre-annotated as the second voiceprint information as the second annotation result indicating that the called party is abnormal.

[0080] Optionally, in the embodiments of this application, the above-mentioned abnormality of the called party may include, but is not limited to, abnormality of the called party, such as being in a call, having the phone turned off, having the phone suspended, being busy, having an invalid number, etc.

[0081] Optionally, in this application embodiment, stopping the call with the target account may include, but is not limited to, hanging up the current call, and determining the target account as an invalid account may include, but is not limited to, marking the target account as an invalid account on a media information publishing platform.

[0082] In this embodiment, when the target voiceprint information matches the second voiceprint information, the second annotation result of the second voiceprint information is determined as the target matching result output by the voiceprint matching model. The annotated voiceprint information includes the second voiceprint information, and the second annotation result includes the called party being abnormal. When the target matching result indicates that the call status of the call with the target account is abnormal, the call with the target account is stopped, and the target account is determined to be an invalid account. This method can actively hang up the phone when the called party is abnormal and generate confirmation information to facilitate subsequent compensation to the media information publisher, thereby improving the detection accuracy of the target audio information.

[0083] As an optional approach, if the voiceprint matching model fails to find a matched, labeled voiceprint information corresponding to the target voiceprint information in the target audio information, semantic recognition is performed on the target audio information according to a pre-defined semantic recognition model to obtain the target recognition result, including: If the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, the target audio information is saved. The target audio information is input into the semantic recognition model, and semantic recognition is performed on the target audio information to obtain the target recognition result. The semantic recognition model is used to obtain the text information corresponding to the target audio information and perform semantic recognition on the text information.

[0084] Optionally, in this embodiment of the application, the above-mentioned saving of target audio information may include, but is not limited to, recording the target audio information simultaneously during the process of voiceprint matching of the target audio information, and finally saving the target audio information.

[0085] Optionally, in the embodiments of this application, the above-mentioned identification method may include, but is not limited to, identifying the semantics of the target audio information based on the keywords of the target audio information to obtain the semantics of the target audio information, and determining the final target identification result based on the semantics.

[0086] As an alternative approach, the method also includes: If the target identification result also indicates that the call status of the caller to the target account is abnormal, the call to the target account is stopped and an alarm message is generated. The alarm message is used to indicate that the account currently establishing a call with the target account is abnormal. If the target identification result indicates that the call status of the target account is abnormal, the call with the target account will be stopped, and the target account will be determined to be an invalid account.

[0087] As an alternative approach, the method also includes: If the target recognition result also indicates that the call status of the call with the target account is abnormal, the third voiceprint information corresponding to the target audio information is added to the voiceprint matching model, and the annotation result corresponding to the third voiceprint information is determined to be abnormal caller. If the target recognition result also indicates that the call status of the call with the target account is abnormal, the fourth voiceprint information corresponding to the target audio information is added to the voiceprint matching model, and the annotation result corresponding to the fourth voiceprint information is determined to be abnormal.

[0088] Optionally, in the embodiments of this application, when the target identification result also indicates that the call status of the call with the target account is abnormal, the target identification result can also be used to re-label the third voiceprint information that has not matched the corresponding labeled voiceprint information. When the target identification result also indicates that the call status of the call with the target account is abnormal, the target identification result can also be used to re-label the fourth voiceprint information that has not matched the corresponding labeled voiceprint information.

[0089] Through the embodiments of this application, for target voiceprint information that does not match the corresponding labeled voiceprint information, it is input into a semantic recognition model to obtain the target recognition result, and the target voiceprint information is re-labeled according to the target recognition result, so that when other target accounts obtain the voiceprint information, the corresponding labeled voiceprint information can be matched.

[0090] The embodiments of this application will be further explained below with specific examples: Under the invalid compensation policy of the media information publishing platform, a certain percentage of virtual funds (corresponding to the aforementioned virtual resources) will be paid out to invalid leads (customer numbers that are switched off, out of service, or invalid after three consecutive calls) from the target customer data collection numbers (corresponding to the aforementioned target account set) collected by the media information publisher after the ad campaign. This will compensate the media information publisher for its losses and encourage the media information publisher to actively annotate and return the leads, so that the media information publisher can use the returned standard information to support the advertising.

[0091] In the process of collecting advertiser leads, the system intelligently verifies and judges invalid phone numbers marked as such by media information publishers, including but not limited to the following: Detection of invalid customer data collection leads within advertising scenarios; Based on MRCP, quickly match the reason for hang-up by ringing audio voiceprint; Automatic and parsed annotation of the voiceprint library was completed using Java engineering and ASR technology; In an intelligent call cleaning system, for abnormal numbers, the system needs to quickly detect and notify the handling of abnormal calling numbers (such as calling numbers applied for by media information publishers that are in arrears, suspended, etc.). For abnormal called numbers (leads submitted by customer service) (invalid numbers, out of service, switched off, busy, etc.), the intelligent cleaning system can expedite the disconnection process, improving call efficiency. This includes, but is not limited to, the following: Real-time alerts for abnormal caller IDs in advertising scenarios; Call acceleration during the intelligent customer data cleaning process based on advertising scenarios; The voiceprint matching part of this application adopts the minimum voiceprint matching mode, that is, the voiceprints per second are analyzed and accumulated for voiceprint matching. This can match the corresponding hang-up reason in the shortest time. On the one hand, this method has extremely high accuracy and fast recognition speed, and reduces the use of ASR resources. On the other hand, for hang-up reasons not found in the new voiceprint library, the unrecognized ringing audio is automatically analyzed based on the Java project, and the audio is then input into ASR for speech recognition to obtain the corresponding recognition result. When the corresponding hang-up reason is matched, the corresponding voiceprint is resynchronized to the model library for optimization and supplementation, thus completing the automatic learning and optimization capability of the clue invalidity detection system.

[0092] In advertising scenarios, if the goal of a media information publisher is customer lead collection, they can collect information from forms filled out by users. After users submit leads to the media information publisher, the leads will be automatically collected and synchronized to the lead platform, where they can be viewed, exported, and used for calls.

[0093] In the process of invalid compensation, the validity of data marked as requiring compensation by media information publishers needs to be checked to determine whether invalid leads marked by media information publishers should be compensated. The following points need to be considered when checking the invalidity of leads in order to convince media information publishers and avoid the media information publishing platform from making additional compensation due to incorrectly marked leads.

[0094] 1. Real-time detection of invalid leads (because mobile phone numbers can change at any time every day, so the timeliness of detection is crucial). 2. The accuracy of invalid leads is crucial. The compensation policy only covers invalid numbers, suspended numbers, and numbers that have been switched off for 3 days. Therefore, the accuracy requirement is extremely high. Otherwise, it will result in additional compensation and losses. 3. The accuracy of the ringing audio at the time of the test: Recording and backing up the ringing audio of users dialed during invalid detection can serve as evidence against objections from media information publishers.

[0095] 4. Model self-optimization and improvement capabilities (because mobile phone ringtones are constantly being updated and iterated, the system needs to continuously learn and iterate, recognize more and more diverse ringtone models, and make accurate judgments.)

[0096] Figure 6 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application, such as... Figure 6 As shown, a real-time invalid clue detection system is constructed, including real-time clue detection, real-time recording of ringing audio, automatic ASR recognition of the model, intelligent analysis of ringing audio voiceprints, and intelligent optimization of the voiceprint model library. Specifically, it includes, but is not limited to, the following steps: S1, Obtain clues submitted by media information publishers; S2, determine whether the clues submitted by the media information publisher are marked as invalid clues; S3, if the result of S2 is yes, perform an invalid check on the clue; S4, initiate a voice call to the clue to obtain voiceprint information for voiceprint recognition; S5. Determine whether the voiceprint information is successfully matched. If the match is successful, proceed directly to step S10. If the match fails, proceed to step S7. S6, during the execution of S4, simultaneously records the ringing audio; S7 performs ASR recognition on the recorded ringing audio; S8 performs ASR recognition on the recorded ringing audio while simultaneously performing voiceprint analysis on the ringing audio. S9, optimize the model based on the recognition results and voiceprint analysis results; S10 saves the identification results and sends them back to the system for subsequent compensation.

[0097] The above is merely an example, and the embodiments in this application do not impose any specific limitations.

[0098] This embodiment reduces the labor and evidence costs of verification, improves the accuracy and timeliness of detection, and can cope with the increase in ringing audio, continuously and intelligently optimizing the model. In intelligent cleaning scenarios, it can quickly disconnect abnormal numbers and display the accurate disconnection results to customers, allowing them to intuitively eliminate invalid leads.

[0099] Figure 7 This is a schematic diagram of another optional audio information detection method according to an embodiment of this application, such as... Figure 7As shown, a clue validity detection system is constructed and runs on a FreeSWITCH-based outbound calling platform. By attaching a module and rewriting the method of intercepting voice streams, the ringing audio of the calling user's mobile phone is intercepted for identification, analysis, and recording.

[0100] It's important to note that during a call, the status of the dialed mobile number (normal number, in a call, switched off, out of service, busy, invalid number, etc.) can be quickly determined from the ringing audio of any missed calls, as well as any anomalies in the caller's number (your number is out of service, has outstanding bills, or is suspended). Traditional call systems cannot detect such invalid calls; they will continue calling until the operator actively disconnects. This system utilizes FreeSwitch... The freely embeddable module format performs synchronized processing of ringing audio from the call system outside the system. The detection module transmits the real-time voice stream of the ringing audio to the detection system in real time. The detection system converts the voice stream into a voiceprint in real time for matching to obtain the current calling / called status result. If an abnormal calling is detected, a stop task and alarm are immediately triggered. If an abnormal called is detected, the corresponding result is labeled and the corresponding business logic is executed (invalid compensation, compensation return, intelligent cleaning to complete receipt writing, and hang-up operation). For audio that is not matched, the detection system saves the audio and requests ASR (Audio Recognition) to obtain the corresponding text result. The text result is initially learned (to determine whether it is a specific status such as out of service or invalid number). If it matches, the model library is automatically optimized. If it does not match, it is added to the system's learning library. The model optimization work can be completed through manual labeling.

[0101] 1. Methods for forwarding ringing audio: The system uses Lua scripts built into FreeSwitch to process different business tasks by recording and forwarding ringing audio based on the business type when it receives a scheduling task. 2. Voiceprint self-training verification method: By integrating with the system through the MRCP voiceprint engine, the system can autonomously send training data and analyze voiceprints for model training during self-training. 3. No matching self-learning method: For ringing audio that is not matched in the voiceprint database, the detection system module records this part of the ringing audio separately, and performs ASR speech recognition on this part of the recording using a toolkit. The results of ASR speech recognition are used to perform initial learning with some text keywords. If the learning results are obtained, the model will be trained automatically. In the invalid compensation policy, this system, as a detection tool, not only encourages media information publishers to return leads and helps to support advertising placement, but also increases media information publishers' confidence in placing ads through the invalid compensation policy.

[0102] The intelligent cleaning system has been enhanced with the ability to proactively detect and warn of anomalies, and the system's call efficiency has been greatly improved.

[0103] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0104] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0105] According to another aspect of the embodiments of this application, an audio information detection apparatus for implementing the above-described audio information detection method is also provided. For example... Figure 8 As shown, the device includes: The acquisition module 802 is used to acquire the target audio information to be detected, wherein the target audio information is the audio information generated after a call is initiated to the target account; The matching module 804 is used to match the target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with the labeled voiceprint information. The identification module 806 is used to perform semantic recognition on the target audio information according to a preset semantic recognition model when no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model, and to obtain a target identification result. The semantic recognition model is used to identify the semantics of the target audio information, and the target identification result is used to indicate whether the target account is the invalid account. The transfer module 808 is used to transfer virtual resources corresponding to the target account when the target identification result indicates that the target account is the invalid account, wherein the virtual resources are configured to be allowed to be transferred when the target account is the invalid account.

[0106] As an optional solution, the device is used to acquire the target audio information to be detected in the following manner: Obtain a target account set, wherein the target account set is a set of accounts provided by the media information publisher; Select the target account from the set of target accounts and initiate a call request to the target account; During the process of initiating a call request to the target account, the target audio information is obtained.

[0107] As an optional solution, the device is used to select the target account from the target account set and initiate a call request to the target account in the following manner: Obtain the invalid account identifier provided by the media information publisher, wherein the invalid account identifier is an identifier obtained by the media information publisher for marking invalid accounts in the target account set; The target account is selected from the target account set based on the invalid account identifier, and a call request is initiated to the target account, wherein the target account is an account in the target account set that has the invalid account identifier.

[0108] As an optional solution, the device is also used for: If the target identification result indicates that the target account is not an invalid account, the transfer of virtual resources corresponding to the target account to the media information publisher is prohibited.

[0109] As an optional solution, the device is used to match target voiceprint information in the target audio information using a preset voiceprint matching model in the following manner: The real-time speech stream of the target audio information is parsed according to a preset time period to obtain the target voiceprint information; The target voiceprint information is input into the voiceprint matching model, and the voiceprint matching model is used to match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model.

[0110] As an alternative solution, The device is used to input the target voiceprint information into the voiceprint matching model in the following manner, and to match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model: when the target voiceprint information matches the first voiceprint information, the first labeling result of the first voiceprint information is determined as the target matching result output by the voiceprint matching model, wherein the labeled voiceprint information includes the first voiceprint information, and the first labeling result includes caller anomalies; The device is further configured to: stop the call with the target account and generate an alarm message when the target matching result indicates that the call status of the call with the target account is abnormal, wherein the alarm message is used to indicate that the account currently establishing a call with the target account is abnormal.

[0111] As an alternative solution, The device is used to input the target voiceprint information into the voiceprint matching model in the following manner, and to match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model: when the target voiceprint information matches the second voiceprint information, the second labeling result of the second voiceprint information is determined as the target matching result output by the voiceprint matching model, wherein the labeled voiceprint information includes the second voiceprint information, and the second labeling result includes anomalies of the called party; The device is further configured to: stop the call with the target account and determine the target account as the invalid account when the target matching result indicates that the call status of the call with the target account is abnormal.

[0112] As an optional solution, the device is used to perform semantic recognition on the target audio information according to a preset semantic recognition model when the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, thereby obtaining a target recognition result: If the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, the target audio information is saved. The target audio information is input into the semantic recognition model, and semantic recognition is performed on the target audio information to obtain the target recognition result. The semantic recognition model is used to obtain the text information corresponding to the target audio information and perform semantic recognition on the text information.

[0113] As an optional solution, the device is also used for: If the target identification result also indicates that the call status of the call with the target account is abnormal, the call with the target account is stopped and an alarm message is generated, wherein the alarm message is used to indicate that the account currently establishing a call with the target account is abnormal; If the target identification result also indicates that the call status of the call with the target account is abnormal, the call with the target account is stopped, and the target account is determined to be an invalid account.

[0114] As an optional solution, the device is also used for: If the target recognition result also indicates that the call status of the call with the target account is abnormal, the third voiceprint information corresponding to the target audio information is added to the voiceprint matching model, and the annotation result corresponding to the third voiceprint information is determined to be abnormal caller. If the target recognition result also indicates that the call status of the call with the target account is abnormal, the fourth voiceprint information corresponding to the target audio information is added to the voiceprint matching model, and the annotation result corresponding to the fourth voiceprint information is determined to be abnormal.

[0115] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication section 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit 901, it performs various functions provided in embodiments of this application.

[0116] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] Figure 9 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0118] It should be noted that, Figure 9 The computer system 900 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0119] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which performs various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for system operation. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output interface 905 (I / O interface) is also connected to the bus 904.

[0120] The following components are connected to the input / output interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a local area network card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 190 is also connected to the input / output interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 190 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0121] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit 901, it performs various functions defined in the system of this application.

[0122] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described audio information detection method is also provided. This electronic device may be... Figure 1 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 10 As shown, the electronic device includes a memory 1002 and a processor 1004. The memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps of any of the above method embodiments via the computer program.

[0123] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0124] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: S1, Obtain the target audio information to be detected, wherein the target audio information is the audio information generated after a call is initiated to the target account; S2, matching the target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with the labeled voiceprint information; S3, if the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, the target audio information is semantically recognized according to the preset semantic recognition model to obtain the target recognition result. The semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to indicate whether the target account is an invalid account. S4, if the target identification result indicates that the target account is an invalid account, transfer the virtual resources corresponding to the target account, wherein the virtual resources are configured to be allowed to be transferred when the target account is an invalid account.

[0125] Alternatively, as those skilled in the art will understand, Figure 10 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 10 This does not limit the structure of the aforementioned electronic devices or electronic equipment. For example, electronic devices or electronic equipment may also include components that are more... Figure 10 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 10 The different configurations shown.

[0126] The memory 1002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the audio information detection method and apparatus in this embodiment. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, thereby realizing the aforementioned audio information detection method. The memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1002 may further include memory remotely located relative to the processor 1004, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1002 may be used, but is not limited to, to store information such as target accounts. As an example, such as... Figure 10As shown, the memory 1002 may include, but is not limited to, the acquisition module 802, matching module 804, recognition module 806, and transfer module 808 from the audio information detection device. Furthermore, it may include, but is not limited to, other module units from the audio information detection device, which will not be elaborated upon in this example.

[0127] Optionally, the transmission device 1006 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1006 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0128] In addition, the aforementioned electronic device also includes: a display 1008 for displaying the transferred virtual resources, etc.; and a connection bus 1010 for connecting the various module components in the aforementioned electronic device.

[0129] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0130] According to one aspect of this application, a computer-readable storage medium is provided, from which a processor of a computer device reads computer instructions, and the processor executes the computer instructions, causing the computer device to perform the audio information detection method provided in various alternative implementations of the above-described audio information detection aspect.

[0131] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps: S1, Obtain the target audio information to be detected, wherein the target audio information is the audio information generated after a call is initiated to the target account; S2, matching the target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with the labeled voiceprint information; S3, if the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, the target audio information is semantically recognized according to the preset semantic recognition model to obtain the target recognition result. The semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to indicate whether the target account is an invalid account. S4, if the target identification result indicates that the target account is an invalid account, transfer the virtual resources corresponding to the target account, wherein the virtual resources are configured to be allowed to be transferred when the target account is an invalid account.

[0132] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0133] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0134] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0135] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0139] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting audio information, characterized in that, include: When a media information publisher publishes media information through a media information publishing platform, the target audio information to be detected is obtained, wherein the target audio information is the audio information generated after the media information publishing platform initiates a call to the target account; The target voiceprint information in the target audio information is matched using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with the labeled voiceprint information. If no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model, the target audio information is semantically recognized according to the preset semantic recognition model to obtain the target recognition result. The semantic recognition model is used to recognize the semantics of the target audio information, and the target recognition result is used to indicate whether the target account is an invalid account. If the target identification result indicates that the target account is an invalid account, the virtual resources corresponding to the target account are transferred to the media information publisher, wherein the virtual resources are configured to be allowed to be transferred when the target account is an invalid account; The step of matching the target voiceprint information in the target audio information using a preset voiceprint matching model includes: when the target voiceprint information matches the second voiceprint information in the labeled voiceprint information, determining the second labeling result of the second voiceprint information as the target matching result output by the voiceprint matching model; when the target matching result indicates that the call status of the call with the target account is abnormal, stopping the call with the target account and determining the target account as the invalid account.

2. The method according to claim 1, characterized in that, The acquisition of the target audio information to be detected includes: Obtain a target account set, wherein the target account set is a set of accounts provided by the media information publisher; Select the target account from the set of target accounts and initiate a call request to the target account; During the process of initiating a call request to the target account, the target audio information is obtained.

3. The method according to claim 2, characterized in that, The step of selecting the target account from the target account set and initiating a call request to the target account includes: Obtain the invalid account identifier provided by the media information publisher, wherein the invalid account identifier is an identifier obtained by the media information publisher for marking invalid accounts in the target account set; The target account is selected from the target account set based on the invalid account identifier, and a call request is initiated to the target account, wherein the target account is an account in the target account set that has the invalid account identifier.

4. The method according to claim 2, characterized in that, The method further includes: If the target identification result indicates that the target account is not an invalid account, the transfer of virtual resources corresponding to the target account to the media information publisher is prohibited.

5. The method according to claim 1, characterized in that, The step of matching the target voiceprint information in the target audio information using a preset voiceprint matching model includes: The real-time speech stream of the target audio information is parsed according to a preset time period to obtain the target voiceprint information; The target voiceprint information is input into the voiceprint matching model, and the voiceprint matching model is used to match the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model.

6. The method according to claim 5, characterized in that, The step of inputting the target voiceprint information into the voiceprint matching model and matching the target voiceprint information with the voiceprint information already labeled in the voiceprint matching model includes: when the target voiceprint information matches the first voiceprint information, determining the first labeling result of the first voiceprint information as the target matching result output by the voiceprint matching model, wherein the labeled voiceprint information includes the first voiceprint information, and the first labeling result includes caller anomalies; The method further includes: if the target matching result indicates that the call status of the call with the target account is abnormal, stopping the call with the target account and generating alarm information, wherein the alarm information is used to indicate that the account currently establishing a call with the target account is abnormal.

7. The method according to any one of claims 1 to 6, characterized in that, When the voiceprint matching model fails to find a matched, labeled voiceprint information corresponding to the target voiceprint information in the target audio information, semantic recognition is performed on the target audio information according to a preset semantic recognition model to obtain a target recognition result, including: If the voiceprint matching model fails to match the labeled voiceprint information corresponding to the target voiceprint information in the target audio information, the target audio information is saved. The target audio information is input into the semantic recognition model, and semantic recognition is performed on the target audio information to obtain the target recognition result. The semantic recognition model is used to obtain the text information corresponding to the target audio information and perform semantic recognition on the text information.

8. The method according to claim 7, characterized in that, The method further includes: If the target identification result also indicates that the call status of the call with the target account is abnormal, the call with the target account is stopped and an alarm message is generated, wherein the alarm message is used to indicate that the account currently establishing a call with the target account is abnormal; If the target identification result also indicates that the call status of the call with the target account is abnormal, the call with the target account is stopped, and the target account is determined to be an invalid account.

9. The method according to claim 8, characterized in that, The method further includes: If the target recognition result also indicates that the call status of the call with the target account is abnormal, the third voiceprint information corresponding to the target audio information is added to the voiceprint matching model, and the annotation result corresponding to the third voiceprint information is determined to be abnormal caller. If the target recognition result also indicates that the call status of the call with the target account is abnormal, the fourth voiceprint information corresponding to the target audio information is added to the voiceprint matching model, and the annotation result corresponding to the fourth voiceprint information is determined to be abnormal.

10. An audio information detection device, characterized in that, include: The acquisition module is used to acquire target audio information to be detected when a media information publisher publishes media information through a media information publishing platform. The target audio information is the audio information generated after the media information publishing platform initiates a call to the target account. The matching module is used to match the target voiceprint information in the target audio information using a preset voiceprint matching model, wherein the voiceprint matching model is used to match the target voiceprint information in the target audio information with the labeled voiceprint information. The identification module is used to perform semantic recognition on the target audio information according to a preset semantic recognition model when no labeled voiceprint information corresponding to the target voiceprint information in the target audio information is matched by the voiceprint matching model, and to obtain a target identification result. The semantic recognition model is used to identify the semantics of the target audio information, and the target identification result is used to indicate whether the target account is an invalid account. A transfer module is used to transfer virtual resources corresponding to the target account to the media information publisher when the target identification result indicates that the target account is the invalid account, wherein the virtual resources are configured to be allowed to be transferred when the target account is the invalid account; The device is used to match target voiceprint information in the target audio information using a preset voiceprint matching model in the following manner: when the target voiceprint information matches the second voiceprint information in the labeled voiceprint information, the second labeling result of the second voiceprint information is determined as the target matching result output by the voiceprint matching model; when the target matching result indicates that the call status of the call with the target account is abnormal, the call with the target account is stopped, and the target account is determined to be the invalid account.

11. The apparatus according to claim 10, characterized in that, The device is used to acquire the target audio information to be detected in the following manner: Obtain a target account set, wherein the target account set is a set of accounts provided by the media information publisher; Select the target account from the set of target accounts and initiate a call request to the target account; During the process of initiating a call request to the target account, the target audio information is obtained.

12. The apparatus according to claim 11, characterized in that, The device is configured to select the target account from the target account set and initiate a call request to the target account in the following manner: Obtain the invalid account identifier provided by the media information publisher, wherein the invalid account identifier is an identifier obtained by the media information publisher for marking invalid accounts in the target account set; The target account is selected from the target account set based on the invalid account identifier, and a call request is initiated to the target account, wherein the target account is an account in the target account set that has the invalid account identifier.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or computer at runtime as described in any one of claims 1 to 9.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 9.

15. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 9 through the computer program.

Citation Information

Patent Citations

  • Anti-telephone fraud method and system based on voiceprint recognition and intention analysis

    CN111429918A