Information processing device, information processing method, and computer-readable recording medium

Abstract

This information processing device includes a search unit that searches for whether or not second audio data similar to sound source data stored in a database is included in first audio data that is recorded, a generating unit for generating a synchronization adjustment unit that, in a case in which the second audio data similar to the sound source data is found, adjusts time lag and audio level of sound source data corresponding to the second audio data, and synchronizes with the first audio data, and a noise removal unit that adjusts the sound source data using the synchronization adjustment unit and removes the adjusted sound source data from the first audio data.

Description

Information processing device, information processing method, and computer-readable recording medium

[0001] The present disclosure relates to an information processing device, an information processing method, and a computer-readable recording medium that remove noise from speech.

[0002] A technique has been disclosed for removing noise from recorded speech when the recorded speech contains noise after the speech has been recorded.

[0003] As a related technique, Patent Document 1 discloses a vehicle imaging and recording device that can more reliably record audio related to an accident. The vehicle imaging and recording device of Patent Document 1 performs cancellation processing to reduce electronic device audio output from electronic devices that is included in audio around the vehicle.

[0004] Japanese Patent Application Laid-Open No. 2020-024740

[0005] However, with conventional techniques, when recorded audio contains noise such as music output from devices such as a radio or television, or background music (BGM), it is difficult to accurately remove the noise from the recorded audio.

[0006] That is, if the recorded speech contains noise that includes speech, the noise that includes speech will also be determined to be speech, making it difficult to remove the noise that includes speech from the recorded speech.

[0007] An example of an objective of the present disclosure is to accurately remove noise from recorded speech.

[0008] In order to achieve the above object, an information processing device according to one aspect of the present disclosure is characterized by having: a search unit that searches whether recorded first audio data contains second audio data similar to audio source data stored in a database; a generation unit that, when second audio data similar to the audio source data is found, generates a synchronization adjustment unit that adjusts the time lag and audio level of the audio source data corresponding to the second audio data and synchronizes it with the first audio data; and a noise removal unit that adjusts the audio source data using the synchronization adjustment unit and removes the adjusted audio source data from the first audio data.

[0009] In addition, in order to achieve the above object, an information processing method according to one aspect of the present disclosure is characterized in that an information processing device searches whether recorded first audio data contains second audio data similar to audio source data stored in a database, and if second audio data similar to the audio source data is found, generates a synchronization adjustment unit that adjusts the time lag and audio level of the audio source data corresponding to the second audio data and synchronizes it with the first audio data, adjusts the audio source data using the synchronization adjustment unit, and removes the adjusted audio source data from the first audio data.

[0010] Furthermore, in order to achieve the above object, a computer-readable recording medium according to one aspect of the present disclosure is characterized in that it records a program including instructions for causing a computer to search whether recorded first audio data contains second audio data similar to audio source data stored in a database, and if second audio data similar to the audio source data is found, to generate a synchronization adjustment unit for adjusting the time lag and audio level of the audio source data corresponding to the second audio data and synchronizing it with the first audio data, and to adjust the audio source data using the synchronization adjustment unit and remove the adjusted audio source data from the first audio data.

[0011] As described above, according to the present disclosure, noise can be removed from recorded speech with high accuracy.

[0012] FIG. 1 is a diagram illustrating an example of an information processing device. FIG. 2 is a diagram illustrating an example of the relationship between first audio data and sound source data. FIG. 3 is a diagram illustrating an example of a system having an information processing device. FIG. 4 is a diagram illustrating an example of the operation of a data processing unit. FIG. 5 is a diagram illustrating an example of the operation of a search unit. FIG. 6 is a diagram illustrating an example of the operation of a generation unit. FIG. 7 is a diagram illustrating an example of the operation of an information processing device in an embodiment. FIG. 8 is a diagram illustrating an example of an audition user interface. FIG. 9 is a diagram illustrating an example of an audition user interface. FIG. 10 is a diagram illustrating an example of an audition user interface. FIG. 11 is a diagram illustrating an example of a computer that realizes the information processing device in the embodiment and modified examples.

[0013] Hereinafter, embodiments will be described with reference to the drawings. In the drawings described below, elements having the same or corresponding functions are denoted by the same reference numerals, and repeated description thereof may be omitted.

[0014] (Embodiment) The configuration of an information processing device in an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining an example of an information processing device.

[0015] 1 is a device for accurately removing noise from recorded speech. As shown in FIG. 1, the information processing device 10 includes a search unit 11, a generation unit 12, and a noise removal unit 13.

[0016] The search unit 11 searches whether or not the recorded first voice data includes second voice data similar to the voice source data stored in the database.

[0017] The sound source data is, for example, data of music or the like output from a device capable of outputting sound. The music may be, for example, music, in-store announcements, radio broadcast sound sources, commercial (CM) sound sources, or the like.

[0018] Fig. 2 is a diagram for explaining an example of the relationship between first audio data and audio source data. In the example of Fig. 2, the vertical axis represents audio level (LEVEL) and the horizontal axis represents time (TIME). Fig. 2A is a diagram showing the waveform of the first audio data X[t] (= target audio data + second audio data). Fig. 2B is a diagram showing the waveform of the audio source data R[t].

[0019] When second audio data similar to the audio source data is searched for, the generating unit 12 generates a synchronization adjusting unit for adjusting the time lag and audio level of the audio source data corresponding to the second audio data and synchronizing it with the first audio data.

[0020] The synchronization adjustment unit is, for example, a filter that synchronizes sound source data similar to the second sound data with the second sound data. The filter is preferably configured with, for example, an adaptive filter. However, the filter is not limited to an adaptive filter.

[0021] The noise elimination unit 13 adjusts the sound source data using the synchronization adjustment unit, and removes the adjusted sound source data (data similar to the second sound data (noise)) from the first sound data.

[0022] FIG. 2C is a diagram showing the waveform of the estimated target voice data S[t] (=first voice data X[t]−adjusted voice source data AR[t]).

[0023] In this way, in the embodiment, even if the recorded first audio data contains noise whose sound source is the above-mentioned music piece, the noise is removed from the first audio data based on sound source data similar to the second audio data, so that the noise can be removed from the first audio data with high accuracy.

[0024] In conventional technology, when noise originating from the above-mentioned music is determined to be speech, it is difficult to remove the noise from the recorded first speech data. However, by adjusting speech source data similar to the second speech data using a synchronization adjustment unit and removing the adjusted speech source data from the first speech data, it is possible to accurately remove noise from the first speech data.

[0025] Furthermore, in the embodiment, it is possible to prevent over-suppression of the target signal due to erroneous noise estimation, which occurs in the conventional technology, and therefore it is possible to prevent degradation of the first audio data.

[0026] [System Configuration] The configuration of the information processing device 10 in this embodiment will be described in more detail with reference to Fig. 3. Fig. 3 is a diagram illustrating an example of a system including an information processing device.

[0027] As shown in FIG. 3, the system 100 according to the embodiment includes an information processing device 10, an input device 20, a storage device 30, and an output device 40.

[0028] The information processing device 10 is, for example, a central processing unit (CPU), a programmable device such as a field-programmable gate array (FPGA), a graphics processing unit (GPU), or a circuit equipped with one or more of these, a personal computer, a mobile terminal, or other information processing device. Note that a server computer may also be used as the information processing device 10.

[0029] In the example of FIG. 3, the input device 20 and the output device 40 are provided outside the information processing device 10, but if the information processing device 10 is a personal computer, a mobile terminal, or the like, they may be provided inside the information processing device 10.

[0030] Furthermore, when the information processing device 10 is a server computer, the information processing device 10 has an input device 20, a storage device 30, and an output device 40 electrically connected via a communication network.

[0031] The input device 20 is, for example, a device such as a touch panel, a mouse, a keyboard, etc. If the information processing device 10 is a server computer, a touch panel, a mouse, a keyboard, etc. of a personal computer, a mobile terminal, etc. may be used as the input device 20.

[0032] The storage device 30 is a database, a server computer, a circuit having a memory, etc. The storage device 30 may be provided inside the information processing device 10 or may be provided externally.

[0033] The output device 40 acquires output information (described later) converted into an outputtable format, and outputs generated images, sounds, and the like based on the output information. The output device 40 is, for example, an image display device using a liquid crystal, an organic electroluminescence (EL) display, or a cathode ray tube (CRT). Furthermore, the image display device may also include an audio output device such as a speaker. The output device 40 may also be a printing device such as a printer.

[0034] When the information processing device 10 is a server computer, a monitor of a personal computer, a mobile terminal, or the like may be used as the output device 40 .

[0035] 3 includes a data processing unit 14, a search unit 11, a generation unit 12, a noise removal unit 13, and an output information generation unit 15. The information processing unit 10 in FIG.

[0036] The data processing unit 14 sequentially acquires the first audio data for each preset period T. Specifically, as shown in Fig. 4, the data processing unit 14 sequentially acquires the first audio data for each period T. Fig. 4 is a diagram for explaining an example of the operation of the data processing unit.

[0037] Note that a voice section containing voice and a section other than the voice section not containing voice (noise section) may be detected from the first voice data using processing such as VAD (Voice Activity Detection).

[0038] The search unit 11 first uses the first audio data acquired for each period T as a query and sequentially inputs the queries into the music search system. Next, the search unit 11 uses the music search system to search for sound source data stored in the storage device 30 that is similar to the query (music search process).

[0039] The music search process may be performed using music search application software such as Shazam (registered trademark), SoundHound (registered trademark), or OTO-Mii (registered trademark).

[0040] Next, the search unit 11 generates query sound source information by associating query identification information that identifies the query, query period information that indicates the period of the query, sound source identification information that identifies searched sound source data that is similar to the query, and sound source period information that indicates the period of the searched sound source data that is similar to the query, and stores the generated query sound source information in a storage unit (not shown).

[0041] The query period is, for example, the period T shown in Fig. 4 and Fig. 5. The storage unit may be provided inside or outside the information processing device 10.

[0042] Fig. 5 is a diagram illustrating an example of the operation of the search unit. In the example of Fig. 5, the query period T is set to 1 [second], and queries Q1, Q2, Q3, Q4 ... Qk, Qk+1, Qk+2 ... Qn ... are sequentially input to the music search system, and the result of executing the music search process is shown. However, the query period T is not limited to 1 [second].

[0043] In the example of Figure 5, queries Q1, Q2, Qk, Qk+1, and Qn respectively searched for 2 minutes 20 seconds (2:20) to 2 minutes 21 seconds (2:21) of song A, 2 minutes 21 seconds (2:21) to 2 minutes 22 seconds (2:22) of song A, 2 minutes 30 seconds (2:30) to 2 minutes 31 seconds (2:31) of song A, 2 minutes 31 seconds (2:31) to 2 minutes 32 seconds (2:32) of song A, and 1 minute 01 second (1:01) to 1 minute 02 seconds (1:02) of song B.

[0044] In the case of query Q1 in Figure 5, query sound source information is generated that associates query identification information "Q1", query period information of query Q1 "0:00-0:01" (first period), sound source identification information "song A" that identifies the searched sound source data corresponding to query Q1, and sound source period information "2:20-2:21" that represents the period of the searched sound source data corresponding to the query.

[0045] In addition, in the case of query Q2 in Figure 5, query sound source information is generated that associates query identification information "Q2", query period information of query Q2 "0:01-0:02", sound source identification information "song A" that identifies the searched sound source data corresponding to query Q2, and sound source period information "2:21-2:22" that represents the period of the searched sound source data corresponding to the query.

[0046] Note that query sound source information is generated in the same manner for queries Qk, Qk+1, and Qn.

[0047] 5, in queries Q3, Q4, and Qk+2, the audio level of the first audio is high, so no music can be found. In such a case, for example, in the case of query Q3, query audio source information is generated that associates query identification information "Q3," query period information "0:02-0:03" of query Q3, audio source identification information "Null" that identifies the searched audio source data corresponding to query Q3, and audio source period information "Null" that indicates the period of the searched audio source data corresponding to the query. Here, "Null" is information indicating that no similar audio source was found.

[0048] Note that query sound source information is generated in the same manner for queries Q4 and Qk+2.

[0049] Next, the search unit 11 determines a period of the same sound source data based on the sound source identification information and the sound source period information. Next, the search unit 11 associates the period of the same sound source data with the sound source data corresponding to the period of the same sound source data and stores them in the storage unit.

[0050] In the example of FIG. 5, a period TA of song A and a period TB of song B are determined, and the sound source data of songs A and B corresponding to the periods TA and TB are stored in a storage unit (not shown).

[0051] When second audio data similar to the audio source data is searched for, the generation unit 12 generates a synchronization adjustment unit for adjusting the time lag and audio level of the audio source data, which is used to synchronize the audio source data with the first audio data.

[0052] Specifically, the generation unit 12 uses the query for searching the sound source data to generate, for example, an adaptive filter for synchronizing the sound source data with the first voice data X[t]. ｔ When the sound source identification information is not "Null", [n] can be expressed as in Equation 1. E[t] represents audio data.

[0053]

[0054] Here, μ indicates a step size, and is given a value such as 0.05. However, in the period TB1 shown in FIG. 6, the filter before Qn cannot be calculated using this method. In such a case, in this period, A calculated using Qn is used. ｔ A filter that outputs the average value taken in the time direction of [n] is used.

[0055] Fig. 6 is a diagram illustrating an example of the operation of the generation unit. In the example of Fig. 6, queries Q1 and Q2 are used to generate a synchronization adjustment unit A1 for synchronizing sound source data corresponding to a period TA1. Furthermore, queries Qk and Qk+1 are used to generate a synchronization adjustment unit A2 for synchronizing sound source data corresponding to a period TA2. Furthermore, query Qn is used to generate a synchronization adjustment unit B1 for synchronizing sound source data corresponding to a period TB1.

[0056] The noise elimination unit 13 first adjusts the sound source data using the synchronization adjustment unit. Next, the noise elimination unit 13 removes the adjusted sound source data (data similar to noise) from the first sound data. Note that when there are multiple pieces of target sound data, the noise elimination unit 13 connects the multiple pieces of target sound data in a time series to generate sound data.

[0057] Specifically, the synchronization adjustment unit is an adaptive filter A ｔ [n], by subtracting the adjusted sound source data AR[t] from the first sound data X[t], the target sound data S[t] can be expressed as in Equation 2. Note that t in Equation 2 represents time, and n in Equation 2 represents the number of delay blocks in the adaptive filter.

[0058]

[0059] In the example of Fig. 6, the sound source data corresponding to song A in period TA1 is adjusted based on synchronization adjustment unit A1. Also, the sound source data corresponding to song A in period TA2 is adjusted based on synchronization adjustment unit A2. Furthermore, the sound source data corresponding to song B in period TB1 is adjusted based on synchronization adjustment unit B1.

[0060] Next, the target voice data S in the period TA1 １ [t], target voice data S in period TA2 ２ [t], target voice data S in period TB1 ３ [t] is calculated (estimated) based on Equation 3. １ [t], S ２ [t], S ３ [t] can be expressed as in Equation 3.

[0061]

[0062] X in number 3 １ [t], X ２ [t], X ３ [t] represent the first audio data in periods TA1, TA2, and TB1, respectively. １ [t-n], R ２ [t-n], R ３ [tn] represent sound source data corresponding to periods TA1, TA2, and TB1, respectively.

[0063] Then, the target voice data S １ [t], S ２ [t], S ３ By connecting [t], the entire audio data is generated.

[0064] The output information generation unit 15 combines at least one or more of the first audio data (= target audio data + second audio data), audio source data, and target audio data (= first audio data X[t] - adjusted audio source data) to generate output information to be displayed on the output device 40. Thereafter, the output information generation unit 15 outputs the output information to the output device 40.

[0065] [Device Operation] Next, the operation of the information processing device in the embodiment will be described with reference to FIG. 7. FIG. 7 is a diagram for explaining an example of the operation of the information processing device in the embodiment. In the following description, the diagram will be referenced as appropriate. Furthermore, in the embodiment, an information processing method is implemented by operating the information processing device. Therefore, the description of the information processing method in the embodiment will be replaced with the description of the operation of the information processing device below.

[0066] The data processing unit 14 sequentially acquires the first voice data for each preset period (step A1). Specifically, the data processing unit 14 sequentially acquires the first voice data for each period T, as shown in FIG.

[0067] Next, the search unit 11 uses the first audio data for each period T as a query and sequentially inputs the queries into the music search system (step A2). Next, the search unit 11 uses the music search system to search for sound source data stored in the storage device 30 that is similar to the query (music search process) (step A3).

[0068] Next, the search unit 11 generates query sound source information by associating query identification information that identifies the query, query period information that indicates the period of the query, sound source identification information that identifies searched sound source data that is similar to the query, and sound source period information that indicates the period of the searched sound source data that is similar to the query, and stores the query sound source information in a memory unit (not shown) (step A4).

[0069] Next, the search unit 11 determines the period of the same sound source data based on the sound source identification information and the sound source period information (step A5). Next, the search unit 11 associates the period of the same sound source data with the sound source data corresponding to the period of the same sound source data and stores them in the storage unit (step A6).

[0070] Next, when second audio data similar to the audio source data is searched for, the generation unit 12 generates a synchronization adjustment unit for adjusting the time lag and audio level of the audio source data, which is used to synchronize the audio source data with the first audio data (step A7).

[0071] Specifically, in step A7, the generation unit 12 uses the query used to search for the sound source data to generate, for example, an adaptive filter as a synchronization adjustment unit for synchronizing the sound source data with the second audio data.

[0072] Next, the noise elimination unit 13 first adjusts the sound source data using the synchronization adjustment unit (step A8). Next, the noise elimination unit 13 generates sound data by removing the adjusted sound source data (data similar to noise) from the first sound data (step A9). However, if multiple pieces of target sound data are generated in step A9, the noise elimination unit 13 generates sound data by connecting the target sound data in chronological order.

[0073] Next, the output information generation unit 15 generates output information to be displayed on the output device 40 by combining at least one of the first audio data (= target audio data + second audio data), audio source data, and target audio data (= first audio data X[t] - adjusted audio source data), and outputs the output information to the output device 40 (step A10).

[0074] [Effects of the embodiment] As described above, according to the embodiment, even if the recorded first audio data contains noise generated from the above-mentioned music as the audio source, the noise is removed from the first audio data based on audio source data similar to the second audio data, so that the noise can be removed from the first audio data with high accuracy.

[0075] In conventional technology, when noise originating from the above-mentioned music is determined to be speech, it is difficult to remove the noise from the recorded first speech data. However, by adjusting similar speech source data to the second speech data using a synchronization adjustment unit and removing the adjusted speech source data from the first speech data, it is possible to accurately remove noise from the first speech data.

[0076] Furthermore, in the embodiment, it is possible to prevent over-suppression of the target signal due to erroneous noise estimation, which occurs in the conventional technology, and therefore it is possible to prevent degradation of the first audio data.

[0077] The information processing device can be used to remove noise contained in, for example, audio from security cameras, audio from drive recorders, audio from emergency calls to emergency services, audio from call centers when dealing with customers, audio from online conferences, and audio for meeting minutes.

[0078] It can be used to remove background music (to avoid copyright infringement) when uploading to video services such as YouTube (registered trademark).

[0079] [Program] The program in the embodiment may be a program that causes a computer to execute steps A1 to A10 shown in Fig. 7. By installing and executing this program on a computer, the information processing device and information processing method in the embodiment can be realized. In this case, the processor of the computer functions as the data processing unit 14, the search unit 11, the generation unit 12, the noise reduction unit 13, and the output information generation unit 15 and performs processing.

[0080] The program in the embodiment may be executed by a computer system constructed by a plurality of computers, in which case, for example, each computer may function as one of the data processing unit 14, the search unit 11, the generation unit 12, the noise reduction unit 13, and the output information generation unit 15.

[0081] (Modification) In the modification, a user interface used by a user when inspecting a recording will be described. Figures 8, 9 and 10 are diagrams for explaining an example of the inspection user interface.

[0082] The listening user interface 80 is a user interface (screen) displayed on the output device 40. The listening user interface 80 has at least a screen 81, a screen 82, a file selection button 83, a play button 84, a music search button 85, and a noise removal button 86. The listening user interface 80 is operated by a user using the input device 20.

[0083] A waveform generated based on the first voice data or the estimated target voice data is displayed on a screen 81. A waveform generated based on the sound source data or the adjusted sound source data is displayed on a screen 82.

[0084] The file selection button 83 is used when the user wants to display the waveform of the audio data that he or she wants to listen to. Specifically, when the user selects the file selection button 83, a list of audio data (files) is displayed. The user then selects the audio data that he or she wants to listen to from one or more pieces of audio data. As a result, the waveform corresponding to the selected audio data (first audio data) is displayed on the screen 81. For example, a waveform like that shown on the screen 81 in FIG. 8 is displayed.

[0085] The play button 84 is used when the user listens to the first audio data. Specifically, when the user selects the play button 84, the information processing device 10 executes a general audio playback process based on the audio data (first audio data) corresponding to the waveform displayed on the screen 81, and the played audio is output from the output device 40 (speaker).

[0086] The music search button 85 is used by the user when searching for second audio data. In other words, when music is playing in the background of the audio being played, the user uses this button to search for the music.

[0087] Specifically, when the user selects the song search button 85, a song search process is executed to search for sound source data similar to the second audio data. In other words, the processes of steps A1 to A6 described above are executed. Next, the waveform of the searched similar sound source data is displayed on the screen 82. For example, a waveform like that shown on the screen 82 in FIG. 9 is displayed.

[0088] Note that the song title and duration may also be displayed as shown in Fig. 9. Also, as shown in Fig. 9, a switching point (broken line) may be displayed to indicate the switching point where song A switches to song B. The switching point may be detected automatically or may be determined manually by the user.

[0089] The noise removal button 86 is used when the user wants to remove the second audio data. The noise removal button 86 becomes active and selectable when audio source data similar to the second audio data is found.

[0090] Specifically, when the user selects the noise removal button 86, the processes of steps A7 to A9 described above are executed. Next, a waveform corresponding to the speech data after noise removal (the speech data of the estimated target) is displayed on the screen 81. For example, a waveform such as that shown on the screen 81 in Fig. 10 is displayed. After the waveform is displayed, the display of the noise removal button 86 changes to "Undo" as shown in Fig. 10. When the undo button 86' in Fig. 10 is selected, the current state transitions to the initial state, and the initial screen of the listening user interface 80 is displayed.

[0091] [Physical Configuration] A computer that realizes an information processing device by executing a program in the embodiment and modified examples will now be described with reference to Fig. 11. Fig. 11 is a diagram for explaining an example of a computer that realizes an information processing device in the embodiment and modified examples.

[0092] 11 , the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other. Note that the computer 110 may include a GPU or an FPGA in addition to or instead of the CPU 111.

[0093] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0094] The program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0095] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0096] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0097] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0098] Note that the information processing device 10 in the embodiment and modified examples can be realized not by a computer on which a program is installed, but by hardware corresponding to each unit, such as an electronic circuit. Furthermore, the information processing device 10 may be partially realized by a program and the remaining unit by hardware. In the embodiment, the computer is not limited to the computer shown in FIG. 11.

[0099] (Supplementary Note 1) An information processing device having: a search unit that searches whether recorded first audio data includes second audio data similar to audio source data stored in a database; a generation unit that, when second audio data similar to the audio source data is searched for, generates a synchronization adjustment unit that adjusts a time lag and an audio level of the audio source data corresponding to the second audio data and synchronizes it with the first audio data; and a noise reduction unit that adjusts the audio source data using the synchronization adjustment unit and removes the adjusted audio source data from the first audio data.

[0100] (Supplementary Note 2) The information processing device according to Supplementary Note 1 further comprises a data processing unit that sequentially acquires the first audio data for each preset period, wherein the search unit uses the first audio data acquired for each period as a query, sequentially inputs the queries into a music search system, and searches for the audio data similar to the query, generates query audio information by associating query identification information that identifies the query, query period information that indicates a period of the query, audio source identification information that identifies the searched audio data similar to the query, and audio source period information that indicates a period of the searched audio data similar to the query, and determines a period of the same audio data based on the audio source identification information and the audio source period information, and associates the period of the same audio data with audio data corresponding to the period of the same audio data.

[0101] (Supplementary Note 3) The information processing device according to Supplementary Note 2, wherein the generation unit generates the synchronization adjustment unit that synchronizes the sound source data with the first audio data, using the query obtained by searching the sound source data.

[0102] (Supplementary Note 4) The information processing device according to Supplementary Note 3, wherein the synchronization adjustment unit is an adaptive filter.

[0103] (Supplementary Note 5) An information processing method, in which an information processing device searches whether recorded first audio data includes second audio data similar to audio source data stored in a database, and if second audio data similar to the audio source data is found, generates a synchronization adjustment unit that adjusts a time lag and an audio level of the audio source data corresponding to the second audio data and synchronizes it with the first audio data, adjusts the audio source data using the synchronization adjustment unit, and removes the adjusted audio source data from the first audio data.

[0104] (Supplementary Note 6) The information processing method according to Supplementary Note 5, wherein the information processing device further: sequentially acquires the first audio data for each preset period; in the search: uses the first audio data acquired for each period as a query, sequentially inputs the queries into a music search system, and searches for the audio data similar to the query; generates query audio information by associating query identification information that identifies the query, query period information that indicates a period of the query, audio source identification information that identifies the searched audio data that is similar to the query, and audio source period information that indicates a period of the searched audio data that is similar to the query; determines a period of the same audio data based on the audio source identification information and the audio source period information, and associates the period of the same audio data with audio data corresponding to the period of the same audio data.

[0105] (Supplementary Note 7) The information processing method according to Supplementary Note 6, wherein in the generating, the synchronization adjustment unit is generated which synchronizes the sound source data with the first audio data using the query obtained by searching the sound source data.

[0106] (Supplementary Note 8) The information processing method according to Supplementary Note 7, wherein the synchronization adjustment unit is an adaptive filter.

[0107] (Supplementary Note 9) A computer-readable recording medium having recorded thereon a program for causing a computer to execute the following processes: searching whether recorded first audio data includes second audio data similar to audio source data stored in a database; if second audio data similar to the audio source data is found, generating a synchronization adjustment unit for adjusting the time lag and audio level of the audio source data corresponding to the second audio data and synchronizing it with the first audio data; adjusting the audio source data using the synchronization adjustment unit; and removing the adjusted audio source data from the first audio data.

[0108] (Supplementary Note 10) The computer-readable recording medium according to Supplementary Note 9, further causing the computer to execute a process of sequentially acquiring the first audio data for each preset period; in the search, using the first audio data acquired for each period as a query, sequentially inputting the queries into a music search system to search for the audio data similar to the query; generating query audio source information by associating query identification information that identifies the query, query period information that indicates a period of the query, audio source identification information that identifies the searched audio source data similar to the query, and audio source period information that indicates a period of the searched audio source data similar to the query; determining a period of the same audio source data based on the audio source identification information and the audio source period information, and associating the period of the same audio source data with audio source data corresponding to the period of the same audio source data.

[0109] (Supplementary Note 11) The computer-readable recording medium according to Supplementary Note 10, which executes a process of generating the synchronization adjustment unit that synchronizes the sound source data with the first audio data using the query that was used to search for the sound source data.

[0110] (Supplementary Note 12) The computer-readable storage medium according to Supplementary Note 11, wherein the synchronization adjustment unit is an adaptive filter.

[0111] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0112] This application claims priority based on Japanese Patent Application No. 2023-139655, filed on August 30, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0113] According to the above description, it is possible to remove noise from recorded speech with high accuracy, and this is useful in fields where speech analysis is required.

[0114] REFERENCE SIGNS LIST 10 Information processing device 11 Search unit 12 Generation unit 13 Noise removal unit 14 Data processing unit 15 Output information generation unit 20 Input device 30 Storage device 40 Output device 100 System 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. An information processing device having: a search means for searching whether recorded first voice data contains second voice data similar to voice source data stored in a database; a generation means for generating a synchronization adjustment section for adjusting the time lag and voice level of the voice source data corresponding to the second voice data and synchronizing it with the first voice data when second voice data similar to the voice source data is searched for; and a noise removal means for adjusting the voice source data using the synchronization adjustment section and removing the adjusted voice source data from the first voice data.

2. The information processing device of claim 1, further comprising a data processing means for sequentially acquiring the first audio data for each preset period, wherein the search means: uses the first audio data acquired for each period as a query, sequentially inputs the queries into a music search system to search for the audio source data similar to the query; generates query audio source information by associating query identification information that identifies the query, query period information that indicates the period of the query, audio source identification information that identifies the searched audio source data that is similar to the query, and audio source period information that indicates the period of the searched audio source data that is similar to the query; determines a period of the same audio source data based on the audio source identification information and the audio source period information, and associates the period of the same audio source data with audio source data corresponding to the period of the same audio source data.

3. The information processing device according to claim 2, wherein the generating means generates the synchronization adjustment section that synchronizes the sound source data with the first voice data using the query by which the sound source data was searched.

4. The information processing device according to claim 3, wherein the synchronization adjustment unit is an adaptive filter.

5. An information processing method, comprising: an information processing device searching whether recorded first voice data contains second voice data similar to voice source data stored in a database; if second voice data similar to the voice source data is found, generating a synchronization adjustment unit for adjusting the time lag and voice level of the voice source data corresponding to the second voice data and synchronizing it with the first voice data; adjusting the voice source data using the synchronization adjustment unit; and removing the adjusted voice source data from the first voice data.

6. A computer-readable recording medium having recorded thereon a program which causes a computer to: search whether recorded first voice data contains second voice data similar to voice source data stored in a database; if second voice data similar to the voice source data is found, generate a synchronization adjustment unit for adjusting the time lag and voice level of the voice source data corresponding to the second voice data and synchronizing it with the first voice data; adjust the voice source data using the synchronization adjustment unit; and remove the adjusted voice source data from the first voice data.

Images