A method, device, equipment and readable storage medium for identifying a speaking object

Abstract

The present invention discloses a method, device, equipment and readable storage medium for identifying a speaking object. The method comprises: determining a similarity value of a speaking object in a target audio file; determining a decision value corresponding to a speaking object corresponding to any frame file of the target audio file according to the determined similarity value and a preset dynamic programming model; wherein the preset dynamic programming model is used to determine the decision value of a speaking object corresponding to any frame file according to the speaking objects corresponding to the preceding and following frame files of the any frame file; determining a speaking object corresponding to a maximum value in the decision value as the speaking object of the any frame file, so as to identify the speaking objects corresponding to all frame files in the target audio file.

Description

Technical Field

[0001] The present invention relates to the field of voiceprint recognition technology in financial technology (Fintech), and in particular to a method, device, equipment and readable storage medium for identifying a speaking object. Background Art

[0002] With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming to Fintech, and voice recognition technology is no exception. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on voiceprint recognition technology.

[0003] At present, in various company meetings or seminars, there are usually scenes where multiple people speak. In order to more accurately and efficiently record the content of the speeches and the corresponding speakers in the meetings, related technologies obtain audio files by recording in the meetings and perform speech recognition processing on the audio files to determine the speakers in the meetings and the content of their speeches.

[0004] However, the related art only determines the similarity between the voice information in the audio file and the voices of multiple speakers, and directly determines the speaker corresponding to the voice information based on the similarity. Such a solution may result in poor accuracy in speaker recognition in the audio file due to changes or similarities in the speaker's voice.

[0005] It can be seen that the related art has a technical problem of poor accuracy in speaker recognition. Summary of the invention

[0006] The main purpose of the present invention is to provide a method, device, equipment and readable storage medium for identifying a speaker, aiming to solve the technical problem of poor speaker recognition accuracy in related technologies.

[0007] To achieve the above object, the present invention provides a method for identifying a speaking object, the method comprising:

[0008] Determine a similarity value of a speaker in a target audio file;

[0009] Determine the decision value corresponding to the speaking object corresponding to any frame file of the target audio file according to the determined similarity value and the preset dynamic programming model; wherein the preset dynamic programming model is used to determine the decision value of the speaking object corresponding to any frame file according to the speaking objects corresponding to the preceding and following frame files of the any frame file;

[0010] The speaking object corresponding to the maximum value in the decision value is determined as the speaking object of any frame file, so as to identify the speaking objects corresponding to all the frame files in the target audio file.

[0011] In a possible implementation, determining the decision value corresponding to the speaking object corresponding to any frame file of the target audio file according to the determined similarity value and a preset dynamic programming model includes:

[0012] The similarities of the M speaking objects corresponding to any frame file are processed according to a preset state transition condition to determine the association values ​​of the M speaking objects corresponding to any frame file; wherein the preset state transition condition is used to recursively deduce the association value of any frame file according to the association value of a previous frame file of any frame file, wherein M is a positive integer greater than 1;

[0013] According to the association value and the preset dynamic programming model, the decision values ​​corresponding to the M speaking objects corresponding to any one frame file are determined.

[0014] In a possible implementation manner, determining the decision values ​​corresponding to the M speaking objects corresponding to any one frame file according to the association value and the preset dynamic programming model includes:

[0015] Determine whether any of the frame files is the last frame file of the plurality of frame files, the plurality of frame files being arranged in a division order;

[0016] If any of the frame files is the last frame file, determining a first preset dynamic programming sub-model from the preset dynamic programming model, and determining decision values ​​corresponding to the M speaking objects corresponding to the any frame file according to the association values ​​of the M speaking objects corresponding to the any frame file and the first preset sub-dynamic programming model; and,

[0017] If it is determined that any one of the frame files is not the last frame file, a second preset dynamic programming sub-model is determined from the preset dynamic programming model, and the decision values ​​corresponding to the M speaking objects corresponding to the any one of the frame files are determined according to the association values ​​of the M speaking objects corresponding to the any one of the files and the second preset sub-dynamic programming model.

[0018] In a possible implementation manner, determining the decision values ​​corresponding to the M speaking objects corresponding to any frame file according to the association values ​​of the M speaking objects corresponding to any frame file and the first preset sub-dynamic programming model includes:

[0019] Inputting the association values ​​of the M speaking objects corresponding to the last frame file into the first preset sub-dynamic programming model to obtain the decision values ​​corresponding to the M speaking objects corresponding to the last frame file;

[0020] The first preset sub-dynamic programming model is used to output the association values ​​of the M speaking objects as decision values ​​corresponding to the M speaking objects corresponding to the last frame file.

[0021] In a possible implementation manner, determining the decision values ​​corresponding to the M speaking objects corresponding to any frame file according to the association values ​​of the M speaking objects corresponding to any frame file and the second preset sub-dynamic programming model includes:

[0022] Inputting the association value of any speaking object corresponding to any frame file, the similarity of the speaking object of the frame file after the frame file, and the index value and the preset weight value corresponding to whether the speaking object of the frame file after the frame file is the same as the speaking object into the second preset sub-dynamic programming model, obtaining the decision value corresponding to any speaking object corresponding to any frame file, so as to determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file;

[0023] Among them, the second preset sub-dynamic programming model is used to add the association value of any speaking object and the similarity of the speaking object of the subsequent frame file to obtain a first processing value; to determine the second processing value according to the indicator value and the preset weight value, and to determine the decision value according to the first processing value and the second processing value.

[0024] In a possible implementation manner, before determining the similarity value of the speaking object of the target audio file, the method further includes:

[0025] Determine attribute information of the audio file, wherein the attribute information is used to indicate whether the audio file is recorded;

[0026] If it is determined according to the attribute information that the audio file is a recorded file, dividing the audio file based on the preset frame length to determine a target audio file; and,

[0027] If it is determined that the audio file is a file being recorded, a first file of the file being recorded that has been recorded is divided based on the preset frame length to determine a first audio file; and the first audio file is used as a target audio file to determine the target audio file.

[0028] In a possible implementation, the method further includes:

[0029] If the speaker corresponding to each frame file in the first audio file has been identified, dividing the second file of the recording file that has been recorded based on the preset frame length to determine the second audio file;

[0030] Using the second audio file as a target audio file to determine the target audio file;

[0031] The second file is used to represent an audio file in which the recording of the file being recorded has been completed and the speaker has not been identified.

[0032] In addition, to achieve the above-mentioned purpose, the present invention further provides a device for identifying a speaking object, the device for identifying a speaking object comprising:

[0033] A first determining unit, configured to determine a similarity value of a speaking object in a target audio file;

[0034] A second determination unit is used to determine a decision value corresponding to a speaking object corresponding to any frame file of the target audio file according to the determined similarity value and a preset dynamic programming model; wherein the preset dynamic programming model is used to determine the decision value of the speaking object corresponding to any frame file according to the speaking objects corresponding to the preceding and following frame files of the any frame file;

[0035] The identification unit is used to determine the speaking object corresponding to the maximum value in the decision value as the speaking object of any frame file, so as to identify the speaking objects corresponding to all the frame files in the target audio file.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also provides a device for identifying a speaking object, the device for identifying a speaking object includes a memory, a processor, and a speaking object identification program stored in the memory and executable on the processor, and the program for identifying a speaking object, when executed by the processor, implements the steps of the method for identifying a speaking object.

[0037] In addition, to achieve the above objectives, the present invention further provides a computer-readable storage medium, on which is stored a program for identifying a speaking object, and when the program for identifying a speaking object is executed by a processor, the steps of identifying a speaking object as described above are implemented.

[0038] In the embodiment of the present invention, the similarity value of the speaking object in the target audio file can be determined, and then the decision value corresponding to the speaking object corresponding to any frame file can be determined based on the determined similarity value of the speaking object in the target audio file and the preset dynamic programming model, and the speaking object corresponding to the maximum value in the decision value can be determined as the speaking object of any frame file. That is, the method provided in the embodiment of the present invention not only determines the similarity value of the speaking object corresponding to any frame file, but also combines the preset dynamic programming model, that is, comprehensively determines the decision value corresponding to the speaking object corresponding to any frame file based on the speaking objects of the frame files before and after any frame file, and then selects the speaking object corresponding to the maximum value in the decision value as the speaking object corresponding to any frame file, which can more accurately realize the recognition of the speaking object corresponding to the target audio file. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a method for identifying a speaking object in an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a method for identifying a speaker in a recorded audio file according to an embodiment of the present invention;

[0041] Figure 3 is a flow chart of a method for identifying a speaker in an audio file being recorded in an embodiment of the present invention;

[0042] Figure 4 It is a functional schematic diagram module diagram of a preferred embodiment of the device for identifying a speaking object of the present invention;

[0043] Figure 5 It is a structural diagram of the hardware operating environment involved in the embodiment of the present invention.

[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0046] As mentioned above, there is a problem in the related art that the accuracy of speaker recognition corresponding to voice information in an audio file is low. In view of this, an embodiment of the present invention provides a method for identifying a speaking object, by which the similarity value of a frame file in an audio file can be adjusted to determine the speaking object corresponding to the frame file according to the speaking object corresponding to the adjusted maximum value, thereby improving the accuracy of identifying the speaking object corresponding to the audio file.

[0047] After introducing the design concept of the embodiment of the present invention, the following briefly introduces the application scenarios to which the technical solutions in the embodiments of the present invention are applicable. It should be noted that the application scenarios described in the embodiments of the present invention are for the purpose of more clearly illustrating the technical solutions in the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided in the embodiments of the present invention. Ordinary technicians in this field can know that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present invention are also applicable to similar technical problems.

[0048] The embodiments of the present invention can be applied to any scenario involving multi-person voice interaction, such as company meetings, group meetings, seminars, online meetings, group discussions, etc. In the aforementioned scenarios, the collected voice information can be fed back to the device for identifying the speaking object in a timely manner through the collection device, so that the speaking object corresponding to the voice information can be identified through the device for identifying the speaking object. Specifically, the collection device can be any device that can collect voice, such as a recorder, a voice recorder, etc.

[0049] In a specific implementation process, the data acquisition device and the device for identifying the speaking object can be connected to each other through one or more networks. The network can be a wired network or a wireless network, for example, the wireless network can be a mobile cellular network, or a wireless fidelity (WIREless-Fidelity, WIFI) network, and of course, other possible networks can also be used, which are not limited in the embodiments of the present invention.

[0050] In the embodiment of the present invention, the device for identifying the speaking object may be a server or a terminal, and the terminal may include mobile terminals such as mobile phones, tablet computers, laptop computers, PDAs, and fixed terminals such as digital TVs and desktop computers. The server may include personal computers, large and medium-sized computers, computer clusters, and the like.

[0051] To further illustrate the method for identifying a speaking object provided by an embodiment of the present invention, a detailed description is given below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present invention provides method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present invention. In the actual processing process or when the device is executed, the method can be executed in the order of the method shown in the embodiment or drawings or in parallel (for example, in the application environment of a parallel processor or multi-threaded processing).

[0052] See also Figure 1 , Figure 1 The flowchart of the method for identifying a speaking object provided by an embodiment of the present invention specifically includes:

[0053] Step 101: Determine the target audio file.

[0054] In an embodiment of the present invention, an audio file of a speaker to be identified may be determined, and then a target audio file of the speaker to be identified may be determined. It should be noted that the target audio file in the embodiment of the present invention is used to represent a file that records information of voice interactions of M speakers, where M is a positive integer greater than 1; wherein the target audio file includes a plurality of frame files obtained by dividing the files according to a preset frame length, and the preset frame length is determined according to the acquisition scene corresponding to the target audio file.

[0055] Specifically, the target audio file may be an audio file that has been recorded or an audio file that is being recorded. The following describes in detail the method for determining the target audio file for the audio file that has been recorded and the audio file that is being recorded.

[0056] In the embodiment of the present invention, for the audio file that has been recorded, the method of determining the target audio file is as follows:

[0057] Step a1: Determine attribute information of the audio file, wherein the attribute information is used to indicate whether the audio file is recorded.

[0058] Step a2: If the audio file is determined to be a recorded file according to the attribute information, the audio file is divided based on a preset frame length to determine a target audio file.

[0059] In the embodiment of the present invention, an audio file collected by a collection device may be obtained. When it is determined that the audio file is a recorded audio file according to the attribute information of the audio file, the audio file may be directly obtained.

[0060] In the specific implementation process, the attribute information of the audio file can be added to the audio file in the form of an identification field. For example, the identification "1" indicates that the audio file is an audio file that has been recorded, and the identification "0" identifies the audio file as an audio file that is being recorded. When the identification "1" is parsed out in the audio file, it is determined that the audio file is an audio file that has been recorded, and the audio file is obtained.

[0061] In an embodiment of the present invention, after obtaining an audio file, the audio file can be divided and processed according to a preset frame length, so that a plurality of frame files can be obtained, and the processed audio file can be determined as a target audio file, wherein the preset frame length is determined according to the acquisition scene corresponding to the audio file.

[0062] In the embodiment of the present invention, according to the collection scene corresponding to the audio file, the average speaking speed of the speaker in the scene can be matched accordingly, so that the preset frame length for the audio file can be determined accordingly.

[0063] In an embodiment of the present invention, if the collection scene corresponding to the audio file is a regular meeting, that is, a meeting scene where multiple speakers communicate at a normal speed, the preset frame length can be set to 1 second in consideration of the speaking speed of the speakers in the scene in actual applications. For example, if the frame length of the audio file divided according to the frame length of 1 second is 1 second, and the recording time corresponding to the audio file is 3 seconds, the audio file can be divided into 3 frame files.

[0064] In an embodiment of the present invention, if the collection scene corresponding to the audio file is a multi-person voice interaction scene of brainstorming, that is, a conference scene where multiple speakers communicate at a fast speed, the preset frame length can be set to 0.5 seconds in consideration of the speaking speed of the speakers in the scene in actual applications. For example, if the frame length of the audio file divided according to the frame length of 1 second is 0.5 seconds, and the recording time corresponding to the audio file is 3 seconds, the audio file can be divided into 6 frame files.

[0065] That is, the target audio file in the embodiment of the present invention is a file that has been processed by dividing according to the preset frame length, and because the preset frame length is determined according to the acquisition scene corresponding to the target audio file, the division granularity of the audio file can be more in line with the basic requirements of identifying the speaking object of the current audio file, and can largely achieve that each frame file contains only the voice information of one speaking object, thereby providing a better processing basis for the subsequent recognition of the speaking object corresponding to the target audio file. In addition, by using a finer division granularity to divide the audio file, the voice information corresponding to the obtained frame file is relatively simple, that is, each frame file corresponds to one speaking object.

[0066] In the embodiment of the present invention, for the audio file being recorded, the method of determining the target audio file is specifically as follows:

[0067] Step b1: Determine the attribute information of the audio file, wherein the attribute information is used to indicate whether the audio file is recorded.

[0068] Step b2: If it is determined that the audio file is a recording file, the first file of the recording file that has been recorded is divided based on a preset frame length to determine the first audio file; the first audio file is used as the target audio file to determine the target audio file.

[0069] Step b3: if the speaker corresponding to each frame file in the first audio file has been identified, the second file that has been recorded is divided based on a preset frame length to determine a second audio file;

[0070] Step b4: Using the second audio file as the target audio file to determine the target audio file; wherein the second file is used to represent an audio file that has been recorded and the speaker has not been identified.

[0071] In the embodiment of the present invention, the attribute information of the audio file can be determined. When the audio file is determined to be an audio file being recorded based on the attribute information of the audio file, the file that has been recorded can be processed to identify the speaking object, without waiting until the entire recording of the audio file is completed before identifying the speaking object. That is, the technical solution provided by the embodiment of the present invention can support the identification of the speaking object of the audio file being recorded, thereby expanding the recognition scope of the audio file.

[0072] In an embodiment of the present invention, if it is determined that the audio file currently being acquired is a file being recorded, the first file of the file being recorded that has been recorded can be divided and processed based on a preset frame length, so that the first audio file can be determined, and then the first audio file is used as the target audio file, so that the target audio file can be acquired.

[0073] In an embodiment of the present invention, if the speaker corresponding to the frame file in the first audio file has been identified, the second file of the recording file that has been recorded can be divided based on the preset frame length to determine the second audio file, and then the second audio file is used as the target audio file to obtain the target audio file. The second file is used to represent the audio file of the recording file that has been recorded and the speaker has not been identified. In other words, the technical solution provided by the embodiment of the present invention can perform the identification of the speaker corresponding to the audio file without waiting for the end of the recording, and can determine to perform the identification process of the speaker once for the audio file that has been recorded.

[0074] In an embodiment of the present invention, the recognition process of the speaking object can also be performed once by determining a number of frames of the audio file that has been recorded. For example, if the preset number of frames is 10, the audio file can be divided into frame files with a frame number of 10 based on the preset frame length, and then the recognition process of the speaking object is performed on the frame files corresponding to frames 1-10, and the recognition process of the speaking object is performed on frames 11-20. This method can solve the problem in the prior art that only the entire audio file can be processed, that is, the recognition process of the speaking object can be performed on the audio file that has been recorded while recording, thereby improving the user experience.

[0075] In a specific implementation process, the file being recorded can be divided into a preset number of frames based on a preset frame length to determine the target audio file until the aforementioned file being recorded is completely recorded, that is, the speaking object in the scene corresponding to the file stops voice interaction.

[0076] Step 102: Determine the similarity value of the speaking object in the target audio file.

[0077] In an embodiment of the present invention, after the target audio file is divided into multiple frame files, X-vector can be used to identify M similarity values ​​of M speaking objects corresponding to each frame file, wherein the similarity value is used to characterize the similarity between the speech corresponding to the frame file and the speech of the speaking object.

[0078] For example, it is determined that there are five speakers in the acquisition scene corresponding to the target audio file, namely the first speaker, the second speaker, the third speaker, the fourth speaker and the fifth speaker. Then, X-vector can be used to identify that the similarity value of the first speaker corresponding to the first frame file is 35%, the similarity value of the second speaker is 30%, the similarity value of the third speaker is 40%, the similarity value of the fourth speaker is 60% and the similarity value of the fifth speaker is 82%, so that the similarity values ​​corresponding to the five speakers corresponding to the frame file can be determined.

[0079] Step 103: Determine a decision value corresponding to a speaking object corresponding to any frame file of the target audio file according to the determined similarity value and a preset dynamic programming model; wherein the preset dynamic programming model is used to determine a decision value corresponding to a speaking object corresponding to any frame file according to speaking objects corresponding to previous and next frame files of any frame file.

[0080] In an embodiment of the present invention, after determining the M similarity values ​​of the M speaking objects corresponding to each frame file, the decision values ​​corresponding to the M speaking objects corresponding to any frame file can be determined based on the similarity values ​​of the M speaking objects corresponding to any frame file and a preset dynamic programming model, and the speaking object corresponding to the maximum value in the decision values ​​is determined as the speaking object of any frame file.

[0081] Specifically, the decision value corresponding to the speaking object corresponding to any frame file of the target audio file may be determined by, but not limited to, the following steps:

[0082] Step c1: Processing the similarities of M speaking objects corresponding to any frame file according to a preset state transfer condition to determine the association values ​​of the M speaking objects corresponding to any frame file; wherein the preset state transfer condition is used to recursively infer the association value of any frame file according to the association value of a previous frame file of any frame file, wherein M is a positive integer greater than 1.

[0083] Step c2: Determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file according to the association value and the preset dynamic programming model.

[0084] Specifically, when executing step c2, the following steps may be adopted but are not limited to:

[0085] Step c20: Determine whether any frame file is the last frame file among a plurality of frame files, and the plurality of frame files are arranged in a division order.

[0086] Step c21: If any frame file is the last frame file, determine the first preset dynamic programming sub-model from the preset dynamic programming model, and determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file according to the association values ​​of the M speaking objects corresponding to any frame file and the first preset sub-dynamic programming model.

[0087] In an embodiment of the present invention, the association values ​​of the M speaking objects corresponding to the last frame file are input into the first preset sub-dynamic programming model to obtain the decision values ​​corresponding to the M speaking objects corresponding to the last frame file; wherein the first preset sub-dynamic programming model is used to output the association values ​​of the M speaking objects as the decision values ​​corresponding to the M speaking objects corresponding to the last frame file.

[0088] In the embodiment of the present invention, if any frame file is the last frame file, the speaking object of the last frame file can be determined by combining the similarity between the speaking object corresponding to the previous frame file and the last frame file. In this way, it can be considered that the frame file is determined by using a finer division granularity, and thus it may be a part of a paragraph of a speaking object, that is, it has an association relationship with the previous frame file. Therefore, combined with the speaking object of the previous frame file, there is a certain reference for identifying the speaking object corresponding to the last frame file, thereby improving the accuracy of identifying the speaking object corresponding to the last frame file.

[0089] Step c22: if it is determined that any frame file is not the last frame file, a second preset dynamic programming sub-model is determined from the preset dynamic programming model, and the decision values ​​corresponding to the M speaking objects corresponding to any frame file are determined according to the association values ​​of the M speaking objects corresponding to any file and the second preset sub-dynamic programming model.

[0090] In an embodiment of the present invention, the association value of any speaking object corresponding to any frame file, the similarity of the speaking object of the next frame file of any frame file, and the index value and preset weight value corresponding to whether the speaking object of the next frame file is the same as any speaking object are input into the second preset sub-dynamic programming model to obtain the decision value corresponding to any speaking object corresponding to any frame file, so as to determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file; wherein the second preset sub-dynamic programming model is used to perform addition processing on the association value of any speaking object and the similarity of the speaking object of the next frame file to obtain a first processing value; to determine the second processing value according to the index value and the preset weight value, and to determine the decision value according to the first processing value and the second processing value.

[0091] In the embodiment of the present invention, if any frame file is not the last frame file, the speaking object of any frame file can be determined by combining the similarity between the speaking object corresponding to the frame files before and after the frame files and any frame file. In this way, it can be considered that the frame file is determined by using a finer division granularity, and thus it may be a part of a paragraph of a speaking object, that is, it has an association relationship with the frame file before it. Therefore, combining the speaking objects of the frame files before and after the frame files has a certain auxiliary effect on the determination of the speaking object corresponding to any frame file, thereby improving the accuracy of the recognition of the speaking object corresponding to any frame file.

[0092] In the embodiment of the present invention, if the model representation of the speaking object corresponding to any frame file is determined as:

[0093]

[0094] in, 1 i≠j is an indicator function, which is used to represent that when any speaker i corresponding to any frame file t is different from any speaker j corresponding to the previous frame file of any frame file t, the value is 1, and when they are the same, the value is 0. It is used to characterize the similarity value of any speaker i corresponding to any frame file t; p is a preset parameter, and the value range of p is greater than 0, f(0,j)=0; is an indicator function, which is used to represent that when any speaker i corresponding to any frame file t is different from the speaker in the next frame file t+1 after any frame file t, the value is 1, and when they are the same, the value is 0; A similarity value for representing the speaking object of the frame file t+1 following any frame file t; The corresponding value is used to represent the decision value of any speaker i corresponding to any frame file t; when t = T, argmax i It is used to represent the i corresponding to the maximum value of f(T,i); when t <T,argmaxi To characterize the The corresponding speaker i is the maximum value.

[0095] It should be noted that in the specific implementation process, argmax y f(N, y) is used to represent the function corresponding to y when the function f(N, y) takes the maximum value, where y belongs to any represented value in Y.

[0096] In the embodiment of the present invention, if any frame file is the last frame file among multiple frame files, and when it is determined that the model representation of the speaking object corresponding to any frame file is the model corresponding to the above expression, that is, when t=T, it can be determined that the first preset sub-dynamic programming model can be understood as

[0097] In a specific implementation process, the last frame file can be recorded as the Tth frame, and any speaking object can be recorded as i. The decision value of the speaking object corresponding to the last frame file is expressed as f(T,i), and is determined according to a preset state transfer condition. Specifically, the preset state transfer condition is expressed as Initialize the equation, that is, set f(0,j)=0, then you can calculate f(t,j) corresponding to each frame file starting from the first frame file. Specifically, f(t+1,i) corresponding to the next frame file is determined according to f(t,i) of the previous frame file, that is, the decision value f(T,i) corresponding to the last frame file can be determined according to f(t,i) corresponding to the frame file before the last frame file, and then determine the speaking object corresponding to the maximum value from the M decision values ​​as the speaking object corresponding to the last frame file.

[0098] In the embodiment of the present invention, if any frame file is not the last frame file among the multiple frame files, the second preset sub-dynamic programming model corresponding to any frame file is determined from the aforementioned model for determining the speaking object corresponding to any frame file, and the speaking object of any frame file is determined. In the embodiment of the present invention, when the preset dynamic programming model is the model corresponding to the aforementioned expression, the second preset sub-dynamic programming model can be determined to be

[0099] In the specific implementation process, the preset state transfer condition can be used. Determine f(t,i) or the associated value corresponding to any frame file before the last frame file, and then calculate the decision value corresponding to any speaking object i in each frame file according to the second preset sub-dynamic programming model.

[0100] Step 104: Determine the speaking object corresponding to the maximum value in the decision value as the speaking object of any frame file, so as to identify the speaking objects corresponding to all frame files in the target audio file.

[0101] In the embodiment of the present invention, the similarity value of the speaking object in the target audio file can be determined, and then the decision value corresponding to the speaking object corresponding to any frame file can be determined based on the determined similarity value of the speaking object in the target audio file and the preset dynamic programming model, and the speaking object corresponding to the maximum value in the decision value can be determined as the speaking object of any frame file. That is, the method provided in the embodiment of the present invention not only determines the similarity value of the speaking object corresponding to any frame file, but also combines the preset dynamic programming model, that is, comprehensively determines the decision value corresponding to the speaking object corresponding to any frame file based on the speaking objects of the frame files before and after any frame file, and then selects the speaking object corresponding to the maximum value in the decision value as the speaking object corresponding to any frame file, which can more accurately realize the recognition of the speaking object corresponding to the target audio file.

[0102] In the embodiment of the present invention, in order to better illustrate the solution for determining the speaking object of a frame file in a target audio file provided in the embodiment of the present invention, a specific example is used for illustration below.

[0103] Assume that the target audio file after the division process has a total of 5 frame files, which are frame file 1, frame file 2, frame file 3, frame file 4 and frame file 5 in order, and there are two speaking objects in the corresponding acquisition scene of the target audio file, namely speaking object A and speaking object B. It should be noted that for ease of understanding, speaking object A is written as: speaker A, and speaking object B is written as: speaker B. Then, X-vector can be used to identify the corresponding similarity values ​​of speaker A and speaker B in each of the 5 frame files, as shown in Table 1 below:

[0104]

[0105] Table 1

[0106] Then the calculation can be performed. Specifically, it can be assumed that the preset parameter p = 0.5, and according to Calculation is performed to determine that in the initialization state, f(0,A) = f(0,B) = 0, and then f(t,i) corresponding to frame file 1 can be determined, that is, f(1,A) = 0.8, f(1,B) = 0.2. Further, f(t,i) corresponding to frame file 2 can be determined as:

[0107]

[0108]

[0109] Furthermore, it can be determined that f(t,i) corresponding to frame file 3 is:

[0110]

[0111]

[0112] Furthermore, it can be determined that f(t,i) corresponding to frame file 4 is:

[0113]

[0114]

[0115] Furthermore, it can be determined that f(t,i) corresponding to frame file 5 is:

[0116]

[0117]

[0118] Specifically, the f(t,i) corresponding to each of the above frame files are summarized to obtain the following Table 2:

[0119] f(t,i) Frame file 1 Frame file 2 Frame file 3 Frame file 4 Frame file 5 Speaker A 0.8 1.5 1.7 2.2 2.5 Speaker B 0.2 0.7 1.6 2.0 2.9

[0120] Table 2

[0121] When determining the speaker for the last frame file, i.e., frame file 5, the decision value can be determined according to the first preset dynamic programming model, i.e., calculating π5=argmax i f(5,i), that is, selecting the speaker corresponding to the maximum value in f(5,i) from "2.5" and "2.9", it can be determined that the speaker corresponding to frame file 5 is B.

[0122] Furthermore, the decision value can be determined according to the second preset dynamic programming model, so as to calculate the speaking object corresponding to the first four frame files. Specifically, for frame file 4, that is, when t=4, the decision value corresponding to the speaking object A corresponding to frame file 4 can be determined as: And the decision value corresponding to the speaker B corresponding to frame file 4 is: Obviously, the decision value corresponding to the speaker B is the largest, and it can be determined that the speaker corresponding to frame file 4 is B, that is, π4=B.

[0123] Specifically, for frame file 3, that is, when t=3, the decision value corresponding to the speaker A corresponding to frame file 3 can be determined as: And the decision value corresponding to the speaker B corresponding to frame file 3 is: Obviously, the decision value corresponding to the speaker B is the largest, and it can be determined that the speaker corresponding to the frame file 3 is B, that is, π3=B.

[0124] Specifically, for frame file 2, that is, when t=2, the decision value corresponding to the speaker A corresponding to frame file 2 can be determined as: And the decision value corresponding to the speaker B corresponding to frame file 2 is: Obviously, the decision value corresponding to the speaker A is the largest. It can be determined that the speaker corresponding to the frame file 2 is A, that is, π2=A.

[0125] Specifically, for frame file 1, that is, when t=1, the decision value corresponding to the speaker A corresponding to frame file 1 can be determined as: And the decision value corresponding to the speaker B corresponding to frame file 1 is: Obviously, the decision value corresponding to the speaker A is the largest, and it can be determined that the speaker corresponding to the frame file 1 is A, that is, π1=A.

[0126] According to the above specific examples, it can be clearly known that the solution for determining the speaking object of the frame file in the target audio file provided by the embodiment of the present invention is determined by correspondingly determining the sequence from the last frame file to the first frame file. In this way, the speaking object of each frame file can be determined in combination with the previous and next frame files corresponding to each frame file, thereby improving the accuracy of determining the speaking object.

[0127] The following is an explanation of the process of identifying the speaker of the recorded audio file.

[0128] See also Figure 2 In the embodiment of the present invention, the specific process of identifying the speaking object is as follows:

[0129] Step 201: Acquire an audio file, wherein the audio file is used to represent a file recording information of voice interactions of M speakers, where M is a positive integer greater than 1.

[0130] Step 202: The audio file is divided and processed according to a preset frame length to obtain a plurality of frame files to determine a target audio file, wherein the preset frame length is determined according to a collection scene corresponding to the audio file.

[0131] Step 203: Determine M similarity values ​​of the M speaking objects corresponding to each frame file, wherein the similarity value is used to represent the similarity between the speech corresponding to the frame file and the speech of the speaking object.

[0132] Step 204: Determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file according to the similarity values ​​of the M speaking objects corresponding to any frame file and the preset dynamic programming model, and determine the speaking object corresponding to the maximum value among the decision values ​​as the speaking object of any frame file.

[0133] In the embodiment of the present invention, the specific implementation of step 201 and step 202 can refer to the specific implementation of the above steps a1-a2, and the specific implementation of step 203 and step 204 can refer to the specific implementation of the above steps 103 and step 104, which will not be repeated here.

[0134] In the embodiment of the present invention, for the target audio file that has been recorded, the audio file is divided by a finer division granularity determined by the acquisition scene corresponding to the audio file, so that the frame file contains only the voice information of one speaker. Furthermore, due to the finer division granularity, the voice information of any frame file and the voice information of the previous and next frames may be a segment of the voice of the same person, so the decision value can be determined comprehensively by combining whether the previous and next frames of the frame file are the same speaker and the similarity of the frame file, and the speaker corresponding to the frame file can be determined more accurately according to the decision value, thereby improving the accuracy of identifying the speaker corresponding to the frame file, that is, improving the accuracy of identifying the speaker of the target audio file that has been recorded.

[0135] The following is an explanation of the process of identifying the speaking object of the audio file being recorded.

[0136] See also Figure 3 In the embodiment of the present invention, the specific process of identifying the speaking object is as follows:

[0137] Step 301: Determine a target audio file, wherein the target audio file is determined based on the corresponding file being recorded and the file that has been recorded.

[0138] Step 302: Determine M similarity values ​​of M speaking objects corresponding to each frame file, wherein the similarity value is used to represent the similarity between the speech corresponding to the frame file and the speech of the speaking object, and M is a positive integer greater than 1.

[0139] Step 303: Determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file according to the similarity values ​​of the M speaking objects corresponding to any frame file and the preset dynamic programming model, and determine the speaking object corresponding to the maximum value among the decision values ​​as the speaking object of any frame file.

[0140] Step 304: Determine whether the second file of the recording file that has been recorded is sufficient to be divided based on the preset frame length. If so, execute step 301; if not, execute step 304.

[0141] In the embodiment of the present invention, the specific implementation of step 301 can refer to the implementation content of the previous steps b1-b4, and the implementation content of steps 302 and 303 will not be repeated here. Specifically, after the target audio file of the speaker to be identified is identified for the first time in the audio file being recorded, step 304 can be executed, that is, a new target audio file is determined again, and the speaker identification process is performed on the new target audio file.

[0142] In an embodiment of the present invention, when performing speaker recognition of any file after the first file (for example, the second file), the previous frame file corresponding to the first frame thereof is the last frame file in the first file. In this way, it is possible to avoid the situation where the last frame file of the target audio file intercepted by the first file and the first frame file of the target audio file obtained by the second file correspond to different frames of a sentence of a speaker, resulting in misjudgment, and the speaker corresponding to the entire file being recorded can be recognized more accurately.

[0143] In addition, an embodiment of the present invention further provides a device for identifying a speaking object, referring to Figure 4 , the device for identifying a speaking object comprises:

[0144] A first determining unit 401 is used to determine a similarity value of a speaking object in a target audio file;

[0145] The second determination unit 402 is used to determine the decision value corresponding to the speaking object corresponding to any frame file of the target audio file according to the determined similarity value and the preset dynamic programming model; wherein the preset dynamic programming model is used to determine the decision value of the speaking object corresponding to any frame file according to the speaking objects corresponding to the previous and next frame files of any frame file;

[0146] The identification unit 403 is used to determine the speaking object corresponding to the maximum value in the decision value as the speaking object of any frame file, so as to identify the speaking objects corresponding to all the frame files in the target audio file.

[0147] In a possible implementation manner, the second determining unit 402 is configured to:

[0148] The similarities of the M speaking objects corresponding to any frame file are processed according to a preset state transition condition to determine the association values ​​of the M speaking objects corresponding to any frame file; wherein the preset state transition condition is used to recursively deduce the association value of any frame file according to the association value of a previous frame file of any frame file, wherein M is a positive integer greater than 1;

[0149] According to the association value and the preset dynamic programming model, the decision values ​​corresponding to the M speaking objects corresponding to any one frame file are determined.

[0150] In a possible implementation manner, the second determining unit 402 is configured to:

[0151] Determine whether any of the frame files is the last frame file of the plurality of frame files, the plurality of frame files being arranged in a division order;

[0152] If any of the frame files is the last frame file, determining a first preset dynamic programming sub-model from the preset dynamic programming model, and determining decision values ​​corresponding to the M speaking objects corresponding to the any frame file according to the association values ​​of the M speaking objects corresponding to the any frame file and the first preset sub-dynamic programming model; and,

[0153] If it is determined that any one of the frame files is not the last frame file, a second preset dynamic programming sub-model is determined from the preset dynamic programming model, and the decision values ​​corresponding to the M speaking objects corresponding to the any one of the frame files are determined according to the association values ​​of the M speaking objects corresponding to the any one of the files and the second preset sub-dynamic programming model.

[0154] In a possible implementation manner, the second determining unit 402 is configured to:

[0155] Inputting the association values ​​of the M speaking objects corresponding to the last frame file into the first preset sub-dynamic programming model to obtain the decision values ​​corresponding to the M speaking objects corresponding to the last frame file;

[0156] The first preset sub-dynamic programming model is used to output the association values ​​of the M speaking objects as decision values ​​corresponding to the M speaking objects corresponding to the last frame file.

[0157] In a possible implementation manner, the second determining unit 402 is configured to:

[0158] Inputting the association value of any speaking object corresponding to any frame file, the similarity of the speaking object of the frame file after the frame file, and the index value and the preset weight value corresponding to whether the speaking object of the frame file after the frame file is the same as the speaking object into the second preset sub-dynamic programming model, obtaining the decision value corresponding to any speaking object corresponding to any frame file, so as to determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file;

[0159] Among them, the second preset sub-dynamic programming model is used to add the association value of any speaking object and the similarity of the speaking object of the subsequent frame file to obtain a first processing value; to determine the second processing value according to the indicator value and the preset weight value, and to determine the decision value according to the first processing value and the second processing value.

[0160] In a possible implementation manner, the device further includes a processing unit, configured to:

[0161] Determine attribute information of the audio file, wherein the attribute information is used to indicate whether the audio file is recorded;

[0162] If it is determined according to the attribute information that the audio file is a recorded file, dividing the audio file based on the preset frame length to determine a target audio file; and,

[0163] If it is determined that the audio file is a file being recorded, a first file of the file being recorded that has been recorded is divided based on the preset frame length to determine a first audio file; and the first audio file is used as a target audio file to determine the target audio file.

[0164] In a possible implementation manner, the processing unit is further configured to:

[0165] If the speaker corresponding to each frame file in the first audio file has been identified, dividing the second file of the recording file that has been recorded based on the preset frame length to determine the second audio file;

[0166] Using the second audio file as a target audio file to determine the target audio file;

[0167] The second file is used to represent an audio file in which the recording of the file being recorded has been completed and the speaker has not been identified.

[0168] The specific implementation of the apparatus for identifying a speaking object provided in the embodiment of the present invention is substantially the same as the embodiments of the method for identifying a speaking object described above, and will not be described in detail herein.

[0169] In addition, an embodiment of the present invention also provides a device for identifying a speaking object. Figure 5 As shown, Figure 5 It is a structural diagram of the hardware operating environment involved in the embodiment of the present invention.

[0170] It should be noted that Figure 5 That is, it is a schematic diagram of the structure of the hardware operating environment of the device for identifying the speaking object. The device for identifying the speaking object in the embodiment of the present invention can be the aforementioned server or terminal.

[0171] like Figure 5 As shown, the device for identifying a speaking object may include: a processor 501, such as a CPU, a memory 505, a user interface 503, a network interface 504, and a communication bus 502. Among them, the communication bus 502 is used to realize the connection and communication between these components. The user interface 503 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and optionally, the user interface 503 may also include a standard wired interface and a wireless interface. The network interface 504 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 505 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 505 may also be a storage device independent of the aforementioned processor 501.

[0172] Those skilled in the art will understand that Figure 5 The structure of the device for identifying a speaking object shown in the figure does not constitute a limitation on the device for identifying a speaking object, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0173] like Figure 5 As shown, the memory 505 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a program for identifying a speaking object. The operating system is a program for managing and controlling the hardware and software resources of the device for identifying a speaking object, and supports the operation of the program for identifying a speaking object and other software or programs.

[0174] exist Figure 5 In the device for identifying a speaking object shown, the user interface 503 is mainly used to connect to a collection device and perform data communication with the collection device, and audio files can be collected and / or obtained through the collection device; the network interface 504 is mainly used for a background server and performs data communication with the background server; the processor 501 can be used to call a program for identifying a speaking object stored in a memory 505, and execute the steps of the method for identifying a speaking object as described above.

[0175] The specific implementation of the device for identifying a speaking object of the present invention is substantially the same as the embodiments of the method for identifying a speaking object described above, and will not be described in detail herein.

[0176] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a program for identifying a speaking object is stored. When the program for identifying a speaking object is executed by a processor, the steps of the method for identifying a speaking object as described above are implemented.

[0177] The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the above-mentioned embodiments of the method for identifying a speaking object, and will not be described in detail herein.

[0178] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0179] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, an optical disk), and includes a number of instructions for enabling a terminal device or a server (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0181] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying a speaking object, characterized in that: The method comprises: Determine a similarity value of a speaker in a target audio file; the target audio file includes: a plurality of frame files obtained after being divided according to a preset frame length; the similarity value includes: M similarity values ​​corresponding to each frame file and M speakers; M is a positive integer greater than 1; For any frame file among the multiple frame files, perform the following operations: Based on the similarity value corresponding to any frame file and in combination with the association value of a previous frame file of any frame file, recursively obtain the association values ​​of the M speaking objects corresponding to any frame file; Based on a preset dynamic programming model, the association values ​​of the M speaking objects corresponding to any frame file are processed in combination with the association values ​​of the speaking objects corresponding to the previous and next frame files of any frame file, so as to determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file; The speaking object corresponding to the maximum value in the decision value is determined as the speaking object of any frame file, so as to identify the speaking objects corresponding to all the frame files in the target audio file.

2. The method according to claim 1, characterized in that The method of processing the association values ​​of the M speaking objects corresponding to any frame file based on the preset dynamic programming model and combining the association values ​​of the speaking objects corresponding to the preceding and following frame files of any frame file to determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file includes: Determine whether any of the frame files is the last frame file of the plurality of frame files, the plurality of frame files being arranged in a division order; If any of the frame files is the last frame file, determining a first preset dynamic programming sub-model from the preset dynamic programming model, and determining decision values ​​corresponding to the M speaking objects corresponding to the any frame file according to the association values ​​of the M speaking objects corresponding to the any frame file and the first preset sub-dynamic programming model; and, If it is determined that any one of the frame files is not the last frame file, a second preset dynamic programming sub-model is determined from the preset dynamic programming model, and decision values ​​corresponding to the M speaking objects corresponding to the any one of the frame files are determined according to the association values ​​of the M speaking objects corresponding to the any one of the files and the second preset sub-dynamic programming model.

3. The method according to claim 2, characterized in that The determining, according to the association values ​​of the M speaking objects corresponding to any frame file and the first preset sub-dynamic programming model, the decision values ​​corresponding to the M speaking objects corresponding to any frame file includes: Inputting the association values ​​of the M speaking objects corresponding to the last frame file into the first preset sub-dynamic programming model to obtain the decision values ​​corresponding to the M speaking objects corresponding to the last frame file; The first preset sub-dynamic programming model is used to output the association values ​​of the M speaking objects as decision values ​​corresponding to the M speaking objects corresponding to the last frame file.

4. The method according to claim 2, characterized in that The determining, according to the association values ​​of the M speaking objects corresponding to any frame file and the second preset sub-dynamic programming model, the decision values ​​corresponding to the M speaking objects corresponding to any frame file comprises: Inputting the association value of any speaking object corresponding to any frame file, the similarity of the speaking object of the frame file after the frame file, and the index value and the preset weight value corresponding to whether the speaking object of the frame file after the frame file is the same as the speaking object into the second preset sub-dynamic programming model, obtaining the decision value corresponding to any speaking object corresponding to any frame file, so as to determine the decision values ​​corresponding to the M speaking objects corresponding to any frame file; Among them, the second preset sub-dynamic programming model is used to add the association value of any speaking object and the similarity of the speaking object of the subsequent frame file to obtain a first processing value; to determine the second processing value according to the indicator value and the preset weight value, and to determine the decision value according to the first processing value and the second processing value.

5. The method according to claim 1, characterized in that Before determining the similarity value of the speaking object of the target audio file, the method further includes: Determine attribute information of the audio file, wherein the attribute information is used to indicate whether the audio file is recorded; If it is determined according to the attribute information that the audio file is a recorded file, dividing the audio file based on the preset frame length to determine a target audio file; and, If it is determined that the audio file is a file being recorded, a first file of the file being recorded that has been recorded is divided based on the preset frame length to determine a first audio file; and the first audio file is used as a target audio file to determine the target audio file.

6. The method according to claim 5, characterized in that The method further comprises: If the speaker corresponding to each frame file in the first audio file has been identified, dividing the second file of the recording file that has been recorded based on the preset frame length to determine the second audio file; Using the second audio file as a target audio file to determine the target audio file; The second file is used to represent an audio file in which the recording of the file being recorded has been completed and the speaker has not been identified.

7. A device for identifying a speaking object, characterized in that: include: A first determining unit, configured to determine a similarity value of a speaking object in a target audio file; The target audio file includes: a plurality of frame files obtained after being divided and processed according to a preset frame length; the similarity value includes: M similarity values ​​corresponding to each frame file and M speaking objects; M is a positive integer greater than 1; A second determination unit is configured to perform the following operations for any frame file among the multiple frame files: based on the similarity value corresponding to the any frame file, combined with the association value of the previous frame file of the any frame file, recursively obtain the association value of the M speaking objects corresponding to the any frame file; based on a preset dynamic programming model, combined with the association values ​​of the speaking objects corresponding to the previous and next frame files of the any frame file, process the association value of the M speaking objects corresponding to the any frame file, and determine the decision value corresponding to the M speaking objects corresponding to the any frame file; The identification unit is used to determine the speaking object corresponding to the maximum value in the decision value as the speaking object of any frame file, so as to identify the speaking objects corresponding to all the frame files in the target audio file.

8. A device for identifying a speaking object, characterized in that: The device for identifying a speaking object comprises: a memory, a processor, and a program for identifying a speaking object stored in the memory and executable on the processor. When the program for identifying a speaking object is executed by the processor, the steps of the method for identifying a speaking object according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for identifying a speaking object according to any one of claims 1 to 6 are implemented.

Images