Sound signal processing system, sound signal processing method, training data creation method, machine learning model generation method, inspection method, and repair method, processing unit for sound signal processing, and control device for sound signal processing system

Abstract

Taking into account the sound of the aircraft in flight, if a characteristic sound is emitted from the object based on the sound detected at the object's location, information regarding the sound source location of that characteristic sound will be appropriately acquired. [Solution] An embodiment of the present invention provides a sound signal processing system comprising: a plurality of sensors mounted on an aircraft moving to the location of an object, which receive a first sound at the location and output a first sound signal; a removal processing unit that removes a first component corresponding to the aircraft's flight sound from the first sound signal output by each sensor and generates a second sound signal for each sensor; an information generation unit that generates information regarding the sound source location of a characteristic sound when a characteristic sound is emitted from the object, based on the second sound signal for each sensor; and a transmission unit that transmits information regarding the sound source location. The removal processing unit includes a machine learning model that receives a first sound signal and outputs a second sound signal from which the first component has been removed.

Description

[Technical Field]

[0001] The present invention relates to an audio signal processing system and an audio signal processing method that receive sound at the location where an object is placed for the purpose of inspecting the object, and process the sound signal. Furthermore, the present invention also relates to a method for creating training data necessary for generating a machine learning model, and a method for generating a machine learning model using the training data, when a machine learning model is used in the above-described sound signal processing method. Furthermore, the present invention relates to an inspection method for inspecting an object using the above-described sound signal processing method, and a repair method for repairing abnormal locations found in the object during inspection. Moreover, the present invention also relates to a processing unit for sound signal processing and a control device for a sound signal processing system. [Background technology]

[0002] In inspection work that identifies the presence or absence of abnormalities in an object and the location of the abnormality based on sounds detected at the location where the object is placed, there are cases where it is not possible to approach the object for various reasons. In such cases, for example, a sound sensor can be mounted on an aerial vehicle such as a drone (UAV), and the aerial vehicle can be moved toward the object to detect sounds emanating from the object. With this type of acoustic inspection technology, if the detected sound is an abnormal sound, the source of that abnormal sound (sound source) can be identified as the location of the abnormality, thereby identifying the presence or absence of an abnormality and the location of the abnormality.

[0003] However, when using the acoustic inspection technology described above, the sound detected by the sound sensor may include noises emitted by the aircraft during flight, specifically motor noise and wind noise. In such cases, these noises may act as disturbances, potentially preventing the proper detection of abnormal sounds that should be detected. To address this, shielding members may be provided on the aircraft to suppress the reception of disturbing noises by the sensor (see, for example, Patent Document 1). [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Patent No. 7336621 [Overview of the project] [Problems that the invention aims to solve]

[0005] However, even if a shielding member is provided on the aircraft as described in Patent Document 1, it is difficult to completely eliminate noise originating from the aircraft. As a result, the sensor may still detect sounds containing noise, and accurate information regarding abnormalities in the object being inspected may not be obtained from such detected sounds.

[0006] The present invention has been made in view of the above circumstances and aims to solve the problems of the prior art described above. Specifically, the present invention aims to provide an audio signal processing system and an audio signal processing method that, taking into account the flight noise of an aircraft, can appropriately acquire information regarding the sound source location of a characteristic sound if a characteristic sound is emitted from the object, based on the sound detected at the object's location. [Means for solving the problem]

[0007] To achieve the above objectives, the present invention has the configurations described in [1] to

[11] below. [1] A sound signal processing system comprising: a plurality of sensors mounted on an aircraft moving to the location of an object, which receive a first sound at the location and output a first sound signal; a removal processing unit that removes a first component corresponding to the aircraft's flight sound from the first sound signal output by each sensor and generates a second sound signal for each sensor; an information generation unit that generates information regarding the sound source location of a characteristic sound when a characteristic sound is emitted from the object, based on the second sound signal for each sensor; and a transmission unit that transmits information regarding the sound source location, wherein the removal processing unit includes a machine learning model that receives a first sound signal and outputs a second sound signal from which the first component has been removed. [2] The sound signal processing system according to [1], comprising an aperture synthesis unit that performs aperture synthesis using a second sound signal for each sensor, and an information generation unit that generates information indicating the sound source location identified based on the second sound signals for each sensor that have been aperture synthesized. [3] The sound signal processing system according to [1], which is mounted on an aircraft and includes an image acquisition unit that acquires an image of an object, and an information generation unit that generates a composite image as information about the sound source position, in which related images relating to the sound source position are placed in a region corresponding to the sound source position in the image of the object acquired by the image acquisition unit. [4] A sound signal processing method comprising: a step of multiple sensors mounted on an aircraft moving to the location of an object receiving a first sound at the location and outputting a first sound signal; a step of a removal processing unit removing a first component corresponding to the aircraft's flight sound from the first sound signal output by each sensor to generate a second sound signal for each sensor; and a step of generating information regarding the sound source location of a characteristic sound when a characteristic sound is emitted from the object, wherein the removal processing unit comprises a machine learning model that receives a first sound signal as input and outputs a second sound signal from which the first component has been removed. A method for creating training data to generate a machine learning model used in any of the sound signal processing systems described in [5] [1] to [3], wherein a learning unit creates training data to be used for machine learning to generate a machine learning model based on sound signals obtained by receiving sound from an aircraft and sound signals of a composite sound including sound from an aircraft and sound other than sound from an aircraft. A method for generating a machine learning model, wherein the learning unit performs machine learning using the learning data created by the learning data creation method described in [6] [5] to generate a machine learning model. An inspection method for inspecting an object using the sound signal processing method described in [7] [4], wherein the sound source location is identified as the location of an abnormality in the object based on information regarding the sound source location of a characteristic sound. A repair method that involves performing repair work on the location of the abnormality in the object identified by the inspection method described in [8] [7]. [9] A processing unit for audio signal processing that generates information regarding the sound source position of a characteristic sound from a first audio signal including the first sound received at the placement location by a plurality of sensors mounted on a flying object that moves to the placement location of the object, the processing unit comprising: a removal processing unit that removes a first component corresponding to the flight sound from the first audio signal output by each sensor to generate a second audio signal for each sensor; and an information generation unit that generates information regarding the sound source position of the characteristic sound when the characteristic sound is generated from the object based on the second audio signal for each sensor, wherein the removal processing unit comprises a machine learning model that receives the first audio signal and outputs a second audio signal from which the first component has been removed from the first audio signal.

[10] A control device for an audio signal processing system, comprising: a flying object that moves to the placement location of the object; a communication unit that communicates with the processing unit described in [9] above; a control unit that controls the flying object via the communication unit; and an output unit that acquires information regarding the sound source position output from the processing unit via the communication unit and outputs the information in a predetermined format, wherein the information regarding the sound source position is information obtained from a first audio signal received by a plurality of sensors mounted on the flying object by the processing unit.

[11] A control device for an audio signal processing system, comprising: a communication unit that communicates with a flying object that moves to the placement location of the object; a control unit that controls the flying object via the communication unit; the processing unit described in [9] above; and an information output unit that outputs information regarding the sound source position output from the processing unit in a predetermined format.

Advantages of the Invention

[0008] According to the present invention, considering the flight sound of the flying object, based on the sound detected at the placement location of the object, when a characteristic sound is generated from the object, information regarding the sound source position of the characteristic sound can be appropriately acquired. Further, based on the above information, the presence or absence of an abnormality in the object is inspected, and when an abnormality is found, a repair operation is performed on the abnormality occurrence location, so that the above abnormality can be appropriately dealt with.

Brief Description of the Drawings

[0009] [Figure 1]It is a diagram showing the configuration of a sound signal processing system according to an embodiment of the present invention. [Figure 2] It is a diagram showing the function of a sound signal processing system according to an embodiment of the present invention. [Figure 3] It is an explanatory diagram of a process for removing a component corresponding to the flight sound of an aircraft from a first sound signal. [Figure 4] It is a diagram showing an example of visualization information indicating a sound source position. [Figure 5] It is a diagram showing the procedure for creating learning data and machine learning. [Figure 6] It is an explanatory diagram of a method for cutting out a plurality of waveforms for learning from the waveform of flight sound. [Figure 7] It is a diagram related to the input and output of a machine learning model and the adjustment of the machine learning model. [Figure 8] It is a diagram showing the flow of a sound signal processing method according to an embodiment of the present invention. [Figure 9] It is a diagram showing the configuration of a sound signal processing system according to a first modification of the present invention. [Figure 10] It is a diagram showing the configuration of a sound signal processing system according to a second modification of the present invention. [Figure 11] It is a diagram showing the configuration of a sound signal processing system according to a third modification of the present invention.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, an embodiment of the present invention (hereinafter referred to as the present embodiment) will be described with reference to the drawings. In the drawings referred to below, for the sake of easy understanding of the explanation, each device is shown in a somewhat simplified and schematic manner. Also, the dimensions of each device shown in the drawings, the intervals and distances between devices, etc. may be different from the actual ones. Also, hereinafter, when explaining the position, orientation, state, etc. of each device, unless otherwise specified, the position, orientation, state, etc. when the respective devices are in use will be explained. Furthermore, in this invention, the concept of "device" includes not only a single device that performs a specific function on its own, but also a combination of multiple devices that exist independently and in a distributed manner, yet cooperate (work together) to perform a specific function. Furthermore, in this specification, the terms “same,” “identical,” and “equal” include a range of errors that are generally acceptable in the art to which this invention pertains. Furthermore, in this specification, a numerical range represented by "~" means a range that includes the numbers written before and after "~" as the lower and upper limits, respectively.

[0011] <<Overview of the audio signal processing work according to this embodiment>> The sound signal processing operation according to this embodiment is performed for the purpose of inspecting an object. Specifically, it receives sound (hereinafter also referred to as the first sound) at the location where the object is placed, and analyzes the received sound signal (hereinafter referred to as the first sound signal) to identify whether or not there is an abnormality in the object, and if there is an abnormality, it identifies the location where the abnormality occurred in the object.

[0012] According to this embodiment, when a characteristic sound is emitted from an object, information regarding the sound source location of the characteristic sound can be appropriately acquired based on the sound detected at the object's location, taking into account the sound of the aircraft in flight. This provides an audio signal processing system and an audio signal processing method that can appropriately acquire information regarding the presence or absence of abnormalities in the object and the location of such abnormalities. Furthermore, according to this embodiment, when a machine learning model is used in sound signal processing work, a method for creating training data necessary for generating a machine learning model, and a method for generating a machine learning model using the training data can be provided. Furthermore, according to this embodiment, it is possible to provide an inspection method for inspecting an object using sound signal processing, and a repair method for repairing any abnormal parts of the object found during the inspection.

[0013] Here, the object of inspection is the article, material, equipment, device, facilities, etc. to be inspected. In this embodiment, the object of inspection is a tank that contains gas and gas-using equipment, a gas generating device, and pipes and ducts that constitute the gas flow path. The abnormality of the object of inspection is gas leakage in the object of inspection.

[0014] Furthermore, the object being inspected is not particularly limited, as long as it is something that can be inspected through sound signal processing. Also, the abnormality of the object is not limited to gas leakage within the object, but may also include, for example, gas intake within the object (more specifically, gas intake from the surrounding area into the object).

[0015] The location of the object is set to a place where sounds originating from the object can be received during sound signal processing, specifically, the area (range) near the object. Sounds originating from the object are, for example, characteristic sounds when an abnormality occurs in the object, i.e., abnormal sounds, more specifically, gas leak sounds when a gas leak occurs in the object. Furthermore, the fact that the object is emitting abnormal sounds (characteristic sounds) means that a sound source for the abnormal sound is formed in the object. The location of the sound source for the abnormal sound is the location where the abnormality occurred, more specifically, the location of the gas leak, and more specifically, the location where defects such as cracks, fractures, and corrosion have occurred in the object.

[0016] In this embodiment, during the process of inspecting the object through sound signal processing, information regarding the location of the sound source of abnormal sounds is acquired, and based on the acquired information, the sound source location is identified as the location of the abnormality in the object (i.e., the location of the gas leak). Information regarding the location of the sound source of abnormal sounds can be acquired by performing known signal processing, such as aperture synthesis, on the first sound signal described above.

[0017] Furthermore, in this embodiment, if the location of the abnormality (gas leak) in the object is identified by the inspection method described above, repair work is carried out appropriately at that location. This allows appropriate measures to be taken for the object where the abnormality (gas leak) occurred, and the object can continue to be used. The content of the repair work can be determined according to the extent of the abnormality in the object, specifically the degree of damage caused by the abnormality.

[0018] For example, if the object in question is located at a high altitude, scaffolding is required to manually check for gas leaks and their locations (locations of abnormalities), and setting up scaffolding takes time and money. Furthermore, due to the risk of accidents caused by gas leaks from the object, users may not be able to approach the object. For these reasons, unmanned aerial vehicles such as drones or UAVs are sometimes used to inspect objects. These vehicles are moved (more specifically, flown) to the location of the object, and the first sound is received by sensors such as microphones (more specifically, acoustic sensors) mounted on the vehicle.

[0019] However, as explained in the background technology section, the first sound received during the flight of an aircraft may unintentionally contain components corresponding to the aircraft's flight noise (hereinafter also referred to as the first component). In particular, if the aircraft has multiple rotors, such as a multicopter, the flight noise may include wind noise (wing noise) generated by the rotation of the rotors, as well as the operating noise of the motors that drive the rotors, which may be received by the sensor.

[0020] Furthermore, when the first sound is received at the location of the object, the aforementioned aircraft noise is included in the first sound, and this acts as a disturbance, making it difficult to properly receive (detect) abnormal sounds in the event of an abnormality in the object. Therefore, when identifying the location of the sound source of abnormal sounds in the object, i.e., the location of a gas leak, based on the first sound signal obtained by receiving the first sound, there is a possibility that the location may not be accurately identified due to the influence of aircraft noise.

[0021] In particular, in this embodiment, the first sound is received by multiple sensors, and the sound source location is determined by aperture synthesis of the first sound signals obtained from each sensor. Aperture synthesis is a method that can improve the S / N ratio (i.e., signal-to-noise ratio), but when attempting to determine the sound source location using sensors mounted on an aircraft, as mentioned above, the sound of the aircraft's flight is also received by the sensors, and its components are mixed into the first sound signal. As a result, if the first sound signals output from each sensor are simply aperture synthesized, the flight sound is also amplified, which may lead to the identification of a sound source location that is different from the actual sound source location (i.e., the source of the abnormal sound).

[0022] In contrast, in this embodiment, the first component corresponding to the flight sound of the aircraft is removed from the first sound signal obtained by receiving the first sound, before the first sound signal is aperture-synthesized. In other words, by subtracting the extracted flight sound component from the first sound signal received by the sensor mounted on the aircraft, the sound from which the flight sound has been removed, i.e., the sound originating from the object, is reproduced.

[0023] In this embodiment, the presence or absence of an abnormality in the object and the location of the abnormality are identified based on the sound signal after the first component has been removed (hereinafter referred to as the second sound signal). This makes it possible to eliminate the influence of aircraft noise when identifying the presence or absence of an abnormality in the object and the location of the abnormality based on the sound at the object's location. As a result, it is possible to avoid situations where an abnormality (i.e., gas leakage) is mistakenly determined to be present in the object due to aircraft noise, even though there is actually no abnormality in the object.

[0024] The above effects are particularly effective when identifying the sound source location by aperture synthesis. Specifically, for high-frequency abnormal sounds such as gas leak sounds, the phase shift of the sound received by the sensor is easily canceled out by aperture synthesis. On the other hand, for sounds with long wavelengths and low frequencies, such as aircraft noise (more specifically, sounds with undulation), the above phase shift is difficult to cancel out, and the sound components may remain even after aperture synthesis. In contrast, in this embodiment, the aircraft noise component (first component) is removed from the sound signal (first sound signal) received by the sensor, thereby resolving the above-mentioned problem.

[0025] Furthermore, in addition to the first component corresponding to the flight sound of the aircraft, other components may be added to the first sound signal. For example, components of ambient noise that are constantly occurring at the location where the object is placed, specifically, components corresponding to rotational noises such as motor noise and equipment operating noises that occur while the object is in operation, may be additionally removed from the first sound signal.

[0026] <<Example Configuration of the Sound Signal Processing System According to This Embodiment>> As described above, the sound signal processing system according to this embodiment (hereinafter referred to as the sound signal processing system 10) is used for inspecting objects, more specifically for gas leak inspection. As shown in Figure 1, the sound signal processing system 10 includes a microphone array 14 consisting of multiple microphones 12, an imaging camera 16, a processing unit 27, and a control device 40. The microphone array 14, imaging camera 16, and processing unit 27 are mounted on an unmanned aircraft 18 and move together with the aircraft 18.

[0027] The aircraft 18 is, for example, a known multirotor, having one or more rotors, and flies by utilizing the rotation of the rotors. The aircraft 18 also has GPS (Global Positioning System) functionality and is equipped with a barometer and magnetic compass sensor (not shown). As a result, in this embodiment, the horizontal position (specifically, longitude and latitude), altitude, and direction of the aircraft 18 can be determined at each point in time during flight.

[0028] The multiple microphones 12 constituting the microphone array 14 correspond to multiple sensors and are arranged in a row along a predetermined direction (for example, the width direction of the aircraft 18). Each microphone 12 receives sound and outputs a signal (sound signal) corresponding to the received sound to the processing unit 27. From the sound signal output from each microphone 12, the waveform of the sound (sound wave) received by that microphone 12 can be identified. The sound signal output by the microphone 12 may be an analog signal or a digital signal converted from an analog signal. Furthermore, it is preferable that the microphones 12 are sensors that can easily distinguish from the flight sound of the aircraft 18 and that can receive sound in a frequency band that can sensitively measure abnormal sounds emitted by the target object. Generally, the frequency of sound (flight sound) emitted by an unmanned aircraft 18 during flight is 30 kHz or less. In addition, the aircraft 18 itself may be equipped with a distance sensor that uses ultrasound with a frequency of about 40 kHz to measure the altitude and distance to surrounding objects during flight. Taking this into consideration, it is preferable to use an acoustic sensor capable of receiving sound in a frequency band of 50kHz to 70kHz as the microphone 12.

[0029] The imaging camera 16 corresponds to a photographic device and is a camera having a photographic lens and an image sensor (not shown), and has a configuration similar to that of a known digital camera. When the aircraft 18 is stationary (hovering) in the air, the imaging camera 16 photographs a predetermined location on the surface facing the aircraft 18 and acquires the captured image P2. The imaging camera 16 then digitizes the captured image P2 and transmits the image data to the processing unit 27. The captured image P2 may be a still image or a moving image. The imaging camera 16 also has a function to identify the position of the field of view at the time of shooting, and can write the identified position information of the field of view to the data file of the captured image P2 and associate it with the captured image P2. The position information of the field of view is information indicating the longitude, latitude, and altitude for each of the four vertices of the rectangle that defines the field of view, or for the center or reference point within the rectangle.

[0030] The processing unit 27 is comprised of a small computer (an example of a computer) such as a Raspberry Pi mounted on the aircraft 18, and as shown in Figure 1, it has a processor 21, memory 22, storage 23, and a communication interface 24. The processor 21 is composed of, for example, a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an NPU (Neural Network Processing Unit), or an ASIC (Application Specific Integrated Circuit). The memory 22 is composed of semiconductor memory such as ROM (Read Only Memory) and RAM (Random Access Memory). The storage 23 consists of, for example, flash memory, HDD (Hard Disc Drive), SSD (Solid State Drive), FD (Flexible Disc), MO disk (Magneto-Optical disc), CD (Compact Disc), DVD (Digital Versatile Disc), SD card (Secure Digital card), or USB memory (Universal Serial Bus memory). The communication interface 24 is comprised of, for example, a network interface card or a communication interface board. Furthermore, the computer comprising the processing unit 27 stores a program for processing sound signals.

[0031] The processing unit 27 is connected to the microphone 12 and the imaging camera 16 by wire or wireless connection, and receives sound signals from the microphone 12 and captured image P2 data from the imaging camera 16.

[0032] The processing unit 27 then performs predetermined processing on the received sound signal and the captured image P2 data. The predetermined processing includes processing to analyze, aggregate, process, and correct the signal and data to be processed, as well as processing to generate new information and data or to rewrite existing information and data. The predetermined processing also includes processing to transmit the generated or rewritten information and data to other devices, and processing to control other devices based on the generated or rewritten information and data. Furthermore, the predetermined processing includes processing to perform machine learning using the acquired data as training data to generate a learning model.

[0033] The control device 40 is composed of a computer or information processing equipment and functions as a remote controller that communicates with the aircraft 18 via a network and controls the aircraft 18. That is, the control device 40 is operated by the user and transmits control data corresponding to the user's operation to the flight controller of the aircraft 18 via the communication unit 36, thereby controlling the flight path, flight speed, attitude and state of the aircraft 18 during flight, etc. The function of controlling the aircraft 18 for these items is executed by the control unit (more specifically, the processor that constitutes the control unit) provided in the control device 40. In addition, in this embodiment, the control device 40 also functions as a communication terminal that communicates with the processing unit 27 via the network via the communication unit 36 ​​and sends and receives information to and from the processing unit 27. The network may be either wired or wireless, but a wireless network is more preferable because it does not depend on the location of the object. Furthermore, the control device 40 is equipped with a display 42, which can display and output information transmitted from the processing unit 27. The information that the control device 40 can display on the display 42 includes the visualization information described later (specifically, the composite image Pg). The control device 40 also stores a program for image output. This program allows the user to view the composite image, which is visualization information, in a format predetermined by the user, including the type of image (including whether it is a still image or a video), how it looks, the shooting location, the shooting time, and whether or not it needs to be saved.

[0034] <<About the functions of the audio signal processing system>> Next, the configuration of the sound signal processing system 10 will be described from a functional standpoint. As shown in Figure 2, the sound signal processing system 10 comprises a sound receiving unit 25, an image acquisition unit 26, a processing unit 27, an output unit 28, and a learning unit 29. These functional units are realized through the cooperation of hardware equipment constituting the sound signal processing system 10 (specifically, the microphone array 14, the imaging camera 16, the processing unit 27, and the control device 40) and programs as software stored in the processing unit 27 and the control device 40, etc. The following describes each of the functional parts.

[0035] (Sound receiving section) As shown in Figure 2, the sound receiving unit 25 is composed of a microphone array 14 and a signal processing unit 30. In a gas leak inspection of an object, the aircraft 18 flies to the location of the object and, while stationary (hovering) at that location, each of the multiple microphones 12 making up the microphone array 14 receives the first sound at that location and outputs a first sound signal. At this time, if there is a damaged part in the object and an abnormal sound (characteristic sound) is generated due to gas leakage from the damaged part, the first sound signal will include a component corresponding to that abnormal sound. Also, as mentioned above, the first sound signal will include a first component corresponding to the flight sound of the aircraft 18.

[0036] The signal processing unit 30, when transferring the first sound signal output from each microphone 12 to the processing unit 27, performs necessary signal processing on the first sound signal, specifically signal amplification, filtering, and A / D conversion. The signal processing unit 30 may be located in the sound receiving unit 25 or in the processing unit 27. If the signal processing unit 30 is located in the processing unit 27, the signal processing unit 30 can be configured through the cooperation of the processor 21 of the computer constituting the processing unit 27 and the sound signal processing program stored in the computer.

[0037] (Image acquisition unit) The image acquisition unit 26 includes an imaging camera 16 and acquires an image of the target object as the captured image P2. Specifically, in a gas leak inspection of a target object, the aircraft 18 flies to the location where the target object is located, and while it is stationary (hovering) at that location, the imaging camera 16 photographs the target object. At this time, the field of view of the imaging camera 16 includes the area of ​​the outer surface of the target object that has been set as the inspection target area. The captured image P2 is converted into data by the image acquisition unit 26 and transmitted to the processing unit 27 as image data. Furthermore, the image data at this stage is imbued with positional information of the field of view at the time of shooting. By imbuing the data of the captured image P2 with this positional information of the field of view, it becomes possible to identify the coordinate position of each part (more specifically, each pixel) in the captured image P2, specifically its latitude, longitude, and altitude.

[0038] (processing) As shown in Figure 2, the processing unit 27 includes a removal processing unit 32, an aperture synthesis unit 33, an information generation unit 34, and a transmission unit 35. These functional units are configured, for example, through the cooperation of the processor 21 of the computer that constitutes the processing unit 27 shown in Figure 1, and a program for sound signal processing stored in the computer.

[0039] The removal processing unit 32 receives the first sound signal output from each microphone 12 and processed by the signal processing unit 30, and performs a removal process for each microphone 12 to remove the first component from each first sound signal. In other words, as shown in Figure 3, the removal process removes the first component, which is the component of the flight sound, from the first sound signal, which contains the first component. Then, as the removal process is performed for each of the multiple microphones 12, a sound signal from which the first component has been removed (hereinafter referred to as the second sound signal) is generated for each microphone 12, as shown in Figure 3. In this embodiment, the removal processing unit 32 generates the second sound signal from which the first component has been removed from the first sound signal output from each microphone 12 using a machine learning model M generated by machine learning. The machine learning model M is a mathematical model that takes the first sound signal as input and outputs a second sound signal from which the first component has been removed from the first sound signal. Here, the machine learning model M is generated, for example, by an external or internal computer performing machine learning. Furthermore, for example, the learning unit 29 shown in Figure 2 is generated by performing machine learning. The machine learning model M and the machine learning method used to generate it will be described later.

[0040] The aperture synthesis unit 33 performs aperture synthesis using the second sound signal generated for each microphone 12 by the removal processing unit 32. Based on the results of aperture synthesis using the second sound signal, the aperture synthesis unit 33 identifies the location of the abnormal sound (specifically, the gas leak sound) in the target object, i.e., the sound source location. Furthermore, the aperture synthesis unit 33 identifies the latitude, longitude, altitude, and direction of the sound source location from the relationship between the sound source location and the position of the aircraft 18. Furthermore, known methods can be used for the aperture synthesis technique of multiple sound signals, and for identifying the sound source location based on the results of the aperture synthesis.

[0041] The information generation unit 34 generates information regarding the sound source location of an abnormal sound (characteristic sound) when such an abnormal sound is emitted from an object, based on the second tone signal from each microphone 12. In this embodiment, the information generation unit 34 generates information indicating or suggesting the sound source location identified by the aperture synthesis unit 33 performing aperture synthesis using the second tone signal from each microphone 12.

[0042] Specifically, the information generation unit 34 generates visualization information indicating the sound source location identified based on the second sound signal for each aperture-synthesized microphone 12. In this embodiment, as shown in Figure 4, the visualization information is a composite image Pg in which the related image P1 relating to the sound source location is placed in the region corresponding to the sound source location in the captured image P2 of the object acquired by the image acquisition unit 26. As shown in Figure 4, the related image P1 is a map image that represents, for example, the distribution of sound intensity (e.g., sound pressure) at the identified sound source location and its surroundings by changes in color scheme and color intensity. The distribution of sound intensity (sound pressure) can be calculated by known methods based on the second sound signal for each aperture-synthesized microphone 12.

[0043] Then, by combining the related image P1 and the image of the object P2, a composite image Pg as visualization information is generated, such that the part of the related image P1 with the highest sound intensity, i.e., the part corresponding to the sound source location, overlaps with the region in the image of the object P2 that corresponds to the sound source location. Note that the region in the image of the object P2 that corresponds to the sound source location can be determined by identifying the positional relationship between the sound source location and the field of view at the time of shooting, based on the positional information of the sound source location and the positional information of the field of view of the imaging camera 16.

[0044] Furthermore, the related image P1 is not limited to a map image showing the sound source location and the distribution of sound intensity around it, but may be any information that allows for visual identification of the sound source location in the captured image P2 of the object, such as an indicator or mark such as an arrow pointing to the sound source location.

[0045] The transmitting unit 35 transmits information regarding the sound source location generated by the information generation unit 34, i.e., the visualization information described above, to the output unit 28, and more precisely, transmits it to the control device 40 after communicating with it. In this embodiment, the transmitting unit 35 transmits information to the control device 40 via a wireless network, but is not limited to this; it may also transmit information to the control device 40 via a wired network. Furthermore, if the processing unit 27 and the output unit 28 are configured by a single computer, as in the second modified example described later, the transmitting unit 35 may be included as part of the signal processing functions of that computer. In addition, the storage device described later may be used as the transmitting unit 35. In this case, information can be transmitted by physically loading the storage device into the control device 40.

[0046] (Output section) The output unit 28 is comprised of, for example, a control device 40, which, as shown in Figure 2, has a communication unit 36 ​​and an information output unit 37. The communication unit 36 ​​receives information sent from the transmission unit 35, i.e., the visualization information described above. The information output unit 37 expands the visualization information received by the communication unit 36 ​​and displays the composite image Pg, which is the visualization information, on the display 42 of the control device 40. This allows, for example, a user operating the control device 40 to look at the visualization information and identify the sound source location of abnormal sounds emitted from the object. In other words, the user can determine whether or not there is a gas leak in the object through the composite image Pg, and if a gas leak is occurring, they can confirm the damaged part of the object (location of the abnormality), which is the sound source location of the gas leak sound. Furthermore, regarding the composite image Pg, which is the visualization information mentioned above, the user can pre-select the type of image (including whether it is a still image or a video), its appearance, the shooting location, the shooting time, or whether or not it needs to be saved, etc., via the control device 40. The information output unit 37 can then output the composite image Pg in a predetermined format based on the items selected by the user. Furthermore, if the processing unit 27 and the output unit 28 are configured by a single computer, as in the second modified example described later, the communication unit 36 ​​and the information output unit 37 may be included as part of the signal processing functions provided by that computer.

[0047] (Learning Department) The learning unit 29 generates a machine learning model M that the removal processing unit 32 uses to remove the first component from the first tone signal and generate the second tone signal. Specifically, it generates the machine learning model M by performing machine learning.

[0048] In performing machine learning, the learning unit 29 creates training data, specifically training data, to be used for machine learning to generate a machine learning model M. The training data is created based on sound signals obtained by receiving the flight sound of the aircraft 18 and sound signals obtained by receiving a composite sound that includes both the flight sound and other sounds. Subsequently, the learning unit 29 performs machine learning using the created training data to generate a machine learning model M.

[0049] In this embodiment, the learning unit 29 is comprised of a computer not mounted on the aircraft 18. This computer includes, for example, a processor, memory, storage, and communication interfaces (not shown). A processor is composed of components such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), MCU (Micro Controller Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), NPU (Neural Network Processing Unit), or ASIC (Application Specific Integrated Circuit). Memory is composed of semiconductor memory such as ROM (Read Only Memory) and RAM (Random Access Memory). Storage consists of, for example, flash memory, HDD (Hard Disc Drive), SSD (Solid State Drive), FD (Flexible Disc), MO disk (Magneto-Optical disc), CD (Compact Disc), DVD (Digital Versatile Disc), SD card (Secure Digital card), or USB memory (Universal Serial Bus memory). The communication interface is comprised of, for example, a network interface card or a communication interface board. Furthermore, the learning unit 29 stores a program for generating a machine learning model M and / or a program for creating training data. These programs work in cooperation with the computer hardware that constitutes the learning unit 29 to realize the functions of generating the machine learning model M and / or creating training data. Furthermore, the computers constituting the learning unit 29 may be a single computer, multiple computers connected to a network, or a virtual computer on an external cloud. If the computers constituting the learning unit 29 are multiple computers connected to a network or a virtual computer on an external cloud, the program for generating the machine learning model M and the program for creating the training data may be stored on different computers, and the corresponding programs may be executed on each computer.

[0050] The procedure for creating training data and performing machine learning will be explained with reference to Figure 5. First, the aircraft 18 is flown in a place without reverberation, and the aircraft sound, which is not superimposed with sounds other than the aircraft sound (hereinafter also referred to as noise), is received, its waveform S is obtained, and this waveform S is input to the learning unit 29 (S001).

[0051] Next, the learning unit 29 takes the input flight sound waveform S and determines N t pieces(N tWaveforms S1, S2, ··· S (where is a natural number greater than or equal to 2) Nt are cut out (S002). Referring to FIG. 6, the method of cutting out N t waveforms will be described. Let the length of the time during which the waveform S of the flight sound is observed (hereinafter also referred to as the time of the waveform S of the flight sound) be T m , and let the period time of the flight sound (wing sound) be T w . Here, the time T m of the waveform S of the flight sound is, as shown in FIG. 6, a time longer than the period time T w of the flight sound.

[0052] The learning unit 29 cuts out N p waveforms for learning data at a predetermined time length T L for each predetermined pitch T t . Each of the cut-out waveforms S1, S2, ··· S Nt is created based on the sound signal obtained by receiving the flight sound. Here, the length of the predetermined time T L is preferably set to a length of 1 / 4 or less of the period time T w of the flight sound. Also, since it is not known from which point the waveform of the flight sound waveform S starts at the period time T w of the flight sound, it is necessary to create as much learning data as possible. From this perspective, it is preferable to set the pitch T p to as short a value as possible.

[0053] Also, the number N t of waveforms cut out from one waveform S of the flight sound is set to the maximum value that satisfies the following formula (1). In other words, the number N w of waveforms is set so that at least one period of the period time T t of the flight sound can be covered. JPEG2026047548000002.jpg1793

[0054] Next, the learning unit 29 superimposes noise on each of the cut-out Nt waveforms S1, S2, ··· S Nt to obtain noisy waveforms L₁, L₂, ··· L NtCreate (S003). Here, noisy waveforms L1, L2, ...L Nt This corresponds to training data and is created based on sound signals of composite sounds including flight sounds and other sounds (noise). Note that the noisy waveforms L1, L2, ...L Nt When creating the waveforms S1, S2, ...S before adding noise Nt The amplitude of the signal may be reduced (for example, to 0.5 or 0.2 times the original amplitude) before the noise is superimposed.

[0055] Each waveform S1, S2,...S Nt When adding noise, a pre-defined type of noise is added. Specifically, the noise added is either white noise, which uniformly contains all frequency components, or pink noise (1 / f noise), in which the sound energy is inversely proportional to the frequency.

[0056] Subsequently, the learning unit 29 processes the waveforms S1, S2, ...S created in step S002. Nt And, the noisy waveforms L1, L2, ...L created in S003 Nt This is used as training data, specifically as the training dataset, to perform machine learning (S004). This machine learning process generates noisy waveforms L1, L2, ...L Nt When input, the waveforms of sounds other than the flight sound included in the noisy waveform (noise waveforms) NW1, NW2, ...NW Nt A machine learning model M is generated that outputs the following.

[0057] As described above, the machine learning model M is used by the removal processing unit 32 to remove the first component in the first sound signal output from each microphone 12, that is, the frequency component corresponding to the flight sound of the aircraft 18. In this embodiment, the learning unit 29 performs machine learning on each of the multiple microphones 12 that make up the microphone array 14 and generates an individual machine learning model M for each microphone 12.

[0058] In other words, in this embodiment, when the removal processing unit 32 removes the first component from the first tone signal output from each microphone 12, it uses the model corresponding to each microphone 12 from among the machine learning models M generated separately for each microphone 12. This makes it possible to generate a machine learning model M for each microphone 12, taking into account the characteristics and properties of each microphone 12. As a result, when removing the first component from the first tone signal using the machine learning model M, the first component can be appropriately removed, taking into account the characteristics and properties of each microphone 12.

[0059] Furthermore, the case is not limited to one where a machine learning model M is generated for each microphone 12; a single machine learning model M may be generated and shared among multiple microphones 12.

[0060] To explain the machine learning model M in more detail, the machine learning model M according to this embodiment is composed of a neural network model. This neural network model is trained to perceive the waveform (sound wave) superimposed on the sound of the aircraft as noise, and to output a waveform in which the noise waveform of the aircraft has been removed from the input waveform.

[0061] In this embodiment, machine learning is used to remove the flight sound waveform from the noisy waveform because the flight sound waveform can change depending on the type of aircraft 18 and the environment in which the aircraft 18 flies. In other words, even if one attempts to extract sounds in a predetermined frequency band using general, conventional filtering, it is not always possible to correctly remove the flight sound. Therefore, in this embodiment, a machine learning model M is used to remove the flight sound (more specifically, components corresponding to the flight sound).

[0062] The type of neural network used to generate the machine learning model M is not particularly limited and may include, for example, a deep learning neural network, a convolutional neural network, or a recurrent neural network. Furthermore, the machine learning algorithm is not limited to neural networks and may include attention, transformer, generative adversarial networks, Boltzmann machines, matrix factoryization, factoryization machines, MWay factoryization machines, field-recognized factoryization machines, Bayesian networks, decision trees, random forests, or other machine learning algorithms.

[0063] Furthermore, the learning unit 29 may also include a learning adjustment unit 38, as shown in Figure 7. The learning adjustment unit 38 is realized through the cooperation of the computer constituting the learning unit 29 and a program as software stored in that computer. The learning adjustment unit 38 adjusts various parameters of the neural network model that constitutes the machine learning model M. Specifically, as shown in Figure 7, the machine learning model M is given noisy waveforms L1, L2, ...L Nt When input is received, the machine learning model M generates waveforms O1, O2, ... O as noise waveforms. Nt The noise waveforms NW1, NW2, ...NW are identified. Meanwhile, the learning adjustment unit 38 receives noise waveforms NW1, NW2, ...NW nt The input is NW1, NW2, ...NW nt And the noise waveforms O1, O2, ...O identified by the machine learning model M Nt The two waveforms are compared. If the difference between the waveforms is greater than a predetermined value, the learning adjustment unit 38 adjusts the parameters of the machine learning model M. This series of steps is repeated until the difference is less than or equal to the predetermined value. In this way, the machine learning model M is adjusted.

[0064] Furthermore, the machine learning process performed by the learning unit 29 (i.e., the generation of the machine learning model M) and the adjustment of the machine learning model M by the learning adjustment unit 38 may be performed at any time. Also, the adjustment of the machine learning model M by the learning adjustment unit 38 may be performed after the generation of the machine learning model M, as a form of retraining. In this case, the noisy waveforms L1, L2, ...L Nt Alternatively, in the gas leak inspection of the object, the first tone signal obtained by each microphone 12 that actually receives sound may be used.

[0065] <<Regarding the sound signal processing method according to this embodiment>> A sound signal processing method using the sound signal processing system 10 according to this embodiment will be described with reference to Figure 8. Note that the procedure shown in Figure 8 is merely an example, and new steps may be added or the order of the steps may be changed without departing from the spirit of the present invention.

[0066] The sound signal processing method according to this embodiment is used in the inspection of an object, for example, in the inspection of a gas leak in the object (hereinafter referred to as the gas leak inspection flow). That is, each step in the gas leak inspection flow corresponds to each step that constitutes the sound signal processing method according to this embodiment. Here, gas leakage includes not only cases where gas is leaking from a damaged area, but also cases where outside air is being drawn in from a damaged area.

[0067] The gas leak inspection flow proceeds as shown in Figure 8. Specifically, first, after activating each part of the sound signal processing system 10, the user remotely controls the aircraft 18 using the control device 40 to move it to the location of the target object (S011). When the aircraft 18 is stationary (more specifically, hovering) at the designated location of the target object, the sound signal processing system 10 starts processing sound signals for gas leak inspection.

[0068] Specifically, first, each of the multiple microphones 12 included in the microphone array 14 mounted on the aircraft 18 receives a first sound at the location of the target object and outputs a first sound signal (S012). At this time, the signal processing unit 30 provided in the sound receiving unit 25 may perform necessary signal processing on the first sound signal output by each microphone 12. After step S012 is performed, or in parallel with step S012, the image acquisition unit 26 mounted on the aircraft 18 photographs the target object and acquires an image of the target object (captured image P2) (S013). At this time, it is preferable to also acquire location information of the shooting location that is added at the time of shooting. The first sound signal from each microphone 12 (including the signal after signal processing by the signal processing unit 30) and the data of the captured image P2 of the target object are received by the processing unit 27.

[0069] Subsequently, the processing unit 27, which includes the removal processing unit 32, removes a first component corresponding to the flight sound of the aircraft 18 from the first sound signal received by each microphone 12, and generates a second sound signal for each microphone 12 (S014). Specifically, the removal processing unit 32 inputs the first sound signal to the aforementioned machine learning model M, and the machine learning model M removes the first component from the first sound signal to generate a second sound signal for each microphone 12.

[0070] Before step S014 is performed, the learning unit 29 creates training data to be used for machine learning to generate the machine learning model M in the manner described above, and performs machine learning using the created training data to pre-generate the machine learning model M. At this time, the training data creation method and the machine learning model generation method of the present invention can be used.

[0071] Then, the processing unit 27, which includes the aperture synthesis unit 33, performs aperture synthesis using the second sound signal generated for each microphone 12 (S015).

[0072] Next, the processing unit 27, which includes the information generation unit 34, determines whether or not there is a gas leak in the object, in other words, whether or not an abnormal sound (gas leak sound) is being emitted from the object, based on the second sound signal synthesized at the aperture (S016). At this time, the processing unit 27, which includes the information generation unit 34, also identifies the sound source location of the abnormal sound. Furthermore, based on the determination result regarding whether or not an abnormal sound is being emitted from the object, the processing unit 27, which includes the information generation unit 34, generates a related image P1 (map image) that shows the sound source location of the abnormal sound and the distribution of sound intensity in its vicinity.

[0073] Furthermore, the processing unit 27, which includes an information generation unit 34, places the associated image P1 in the region corresponding to the sound source position in the captured image P2 of the object acquired in step S013 to generate a composite image Pg (S017). The generated composite image Pg corresponds to visualization information regarding the sound source position of the abnormal sound.

[0074] Subsequently, the processing unit 27, which includes the transmission unit 35, outputs the composite image Pg, which is the generated visualization information, to the control device 40. Specifically, the processing unit 27, which includes the transmission unit 35, transmits the composite image Pg to the control device 40, which includes the communication unit 36 ​​(S018). Upon receiving the composite image Pg visualization information, the control device 40 displays the visualization information on the display 42. The gas leak inspection flow is completed once the series of steps described above are finished.

[0075] Furthermore, in a gas leak inspection, if it is determined that an abnormal sound is being emitted from the object, the user operating the control device 40 identifies the gas leak location, which is the sound source of the abnormal sound, as the location of the abnormality in the object. At this time, the inspection method of the present invention is used. Subsequently, the user performs appropriate repair work on the abnormal areas of the object identified during the inspection. This is where the repair method of the present invention comes into play.

[0076] <<Other Embodiments>> The embodiments described above are merely examples provided to facilitate understanding of the present invention and do not limit it. In other words, the present invention can be modified or improved from the embodiments described below, without departing from its spirit. Furthermore, the present invention includes equivalents thereof.

[0077] Furthermore, in the above embodiment, aperture synthesis was performed using a second tone signal generated for each microphone 12 in order to identify the sound source location of the abnormal sound. However, the invention is not limited to this, and as long as the sound source location of the abnormal sound can be identified based on the second tone signal for each microphone 12, methods other than aperture synthesis may be used to that extent.

[0078] Furthermore, in the above embodiment, information regarding the sound source location of an abnormal sound is a composite image Pg in which a related image P1 suggesting the sound source location is placed in a captured image P2 of the object. This composite image Pg is visualization information that allows for the visual recognition of the sound source location on the object, and this visualization information is output as the inspection result. However, the invention is not limited to this, and the sound intensity (more specifically, sound pressure) at each part of the object may be calculated, and the calculation results shown in text or images in the captured image P2 of the object may also be used as information regarding the sound source location of an abnormal sound.

[0079] Furthermore, in the above embodiment, an example of a method for outputting information regarding the location of the sound source of an abnormal sound was described in which the information is displayed on the display 42 of the control device 40, but the invention is not limited to this. Methods for outputting information include playing an audio recording that represents the information (for example, an audio recording that indicates the direction of the sound source location as viewed from a reference position). Other methods for outputting information include sending the information to a recipient terminal by means of email or other means, printing the information using a printer or the like, and storing the information in a database or server or the like.

[0080] Furthermore, in the above embodiment, the microphone array 14 and the imaging camera 16 are mounted on the aircraft 18, and the computer constituting the processing unit 27 is also mounted on the aircraft 18. In addition, in the above embodiment, the control device 40, which functions as an output unit 28, is a separate device from the computer constituting the processing unit 27, is located away from the aircraft 18, and is composed of a controller that remotely operates the aircraft 18. However, the system is not limited to this, and the configuration of the sound signal processing system also includes, for example, the modified forms shown in Figures 9 to 11.

[0081] [First variation] In the first modified sound signal processing system 10X shown in Figure 9, the processing unit 27 is not mounted on the aircraft 18. In other words, in the first modified example, the processing unit 27 is composed of a computer provided separately from the small computer mounted on the aircraft 18 (hereinafter referred to as the aircraft-side computer 44). The processing unit 27, which is not mounted on the aircraft 18, has a processor 21, memory 22, storage 23, and communication interface 24, etc., similar to the above embodiment. The aircraft-side computer 44 transmits the first sound signals output from each microphone 12 (including signals processed by the signal processing unit 30) and the data of the captured image P2 by the image acquisition unit 26 to the processing unit 27. In the first modified example, some or all of the functions of the processing unit 27 (specifically, the removal processing unit 32, aperture synthesis unit 33, information generation unit 34, and transmission unit 35) are provided by the processing unit 27 that is not mounted on the aircraft. Alternatively, some of the functions of the processing unit 27 may be provided by the aircraft-side computer 44. Furthermore, the signal processing unit 30 may be provided in the aircraft-side computer 44, or it may be provided in the processing unit 27. Also, some or all of the functions of the output unit 28 may be provided by the processing unit 27 which is not mounted on the aircraft 18. Also, some or all of the functions of the learning unit 29 may be provided by the processing unit 27 which is not mounted on the aircraft 18.

[0082] [Second variation] In the sound signal processing system 10Y according to the second modified example shown in Figure 10, some or all of the functions of the processing unit 27 are performed by a computer, specifically a control unit 40, which is provided separately from the aircraft-side computer 44, similar to the first modified example. In other words, in the second modified example, the control unit 40 comprises the processing unit 27 and the output unit 28. Furthermore, as shown in Figure 10, the control unit 40 comprises a processor 51, memory 52, storage 53, and a communication interface 54, and these devices have substantially the same configuration as the devices provided by the processing unit 27 in the above embodiment. Furthermore, the signal processing unit 30 may be provided in the aircraft-side computer 44, or in the control device 40. Also, the control device 40 may provide some or all of the functions of the learning unit 29.

[0083] Furthermore, in the first and second modified examples, a recording medium such as a memory card may be detachably attached to the aircraft-side computer 44. In this case, information and data acquired by the aircraft 18 during flight, specifically the first sound signals output from each microphone 12 (including signals processed by the signal processing unit 30) and the data of images P2 captured by the image acquisition unit 26, may be temporarily stored on the storage medium. After the aircraft 18 returns, the storage medium may be removed from the aircraft-side computer 44, and the information and data stored on the storage medium may be used for various processes. That is, the first sound signals and images P2 of the target object stored on the storage medium may be used for analysis processing for gas leak inspection. Alternatively, the first sound signals stored on the storage medium may be used for machine learning to generate a machine learning model M, or for adjusting (retraining) the machine learning model M.

[0084] [Third variation] In the sound signal processing system 10Z according to the third modified example shown in Figure 11, some or all of the functions of the processing unit 27 are performed by an external server computer 46. As shown in Figure 11, the server computer 46 is equipped with a processor 61, memory 62, storage 63, and a communication interface 64, and these devices have substantially the same configuration as the devices equipped in the processing unit 27 in the above embodiment. The server computer 46 also receives the first sound signal (including the signal after processing by the signal processing unit 30) output from each microphone 12 and the image data P2 captured by the image acquisition unit 26 from the aircraft-side computer 44 through communication with the aircraft-side computer 44. The server computer 46 then uses the received first sound signal and the captured image P2 to perform analysis processing for gas leak inspection and transmits the analysis results (specifically, visualization information of the sound source location, etc.) to the control device 40 via the network. The control device 40 outputs the information sent from the server computer 46, specifically displayed on the display 42. Furthermore, the server computer 46 uses the first sound signal received from the aircraft-side computer 44 to perform machine learning to generate a machine learning model M, or to adjust the machine learning model M. Furthermore, the signal processing unit 30 may be provided in the aircraft-side computer 44, or it may be provided in the server computer 46. Also, some or all of the functions of the learning unit 29 may be provided by the server computer 46. The server computer 46 may also be a server for cloud services, for example, a server computer for ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service).

[0085] Furthermore, in the above embodiment, machine learning for generating the machine learning model M is performed by a processing unit 27 mounted on the aircraft 18. However, it is not limited to this, and other equipment besides the processing unit 27, such as an external computer used by a third party (vendor), may perform the above machine learning and generate the machine learning model M. In this case, the machine learning model M generated by the external computer should be provided to the processing unit 27 in a usable state. Specifically, for example, the machine learning model M may be stored in a usable state on a server for a cloud service. In this case, the machine learning model M may be used by accessing the above server, or the machine learning model M stored on the above server may be used via an API (Application Programming Interface). [Explanation of Symbols]

[0086] 10,10X,10Y,10Z Audio Signal Processing System 12 Microphones (sensors) 14 Microphone Array 16. Imaging camera (imaging equipment) 18 flying objects 21,51,61 processors 22,52,62 memory 23,53,63 storage 24, 54, 64 communication interfaces 25 Sound receiver 26 Image acquisition unit 27 Processing Unit 28 Output section 29. Learning Department 30 Signal Processing Unit 32 Removal Processing Unit 33 Open composite section 34 Information generation section 35 Transmitter 36 Communications Department 37 Information Output Unit 38 Learning Adjustment Unit 40 Control device 42 displays 44. Onboard computer of the aircraft 46 Server Computers M Machine Learning Model P1 Related Images P2 Photographed Images Pg composite image

Claims

1. Multiple sensors mounted on an aircraft moving to the location where the object is to be placed receive a first sound at the location and output a first sound signal, A removal processing unit removes a first component corresponding to the flight sound of the aircraft from the first sound signal output by each of the sensors, and generates a second sound signal for each sensor. An information generation unit generates information regarding the sound source location of a characteristic sound when the characteristic sound is generated from the object, based on the second sound signal for each of the sensors. The system includes a transmitting unit that transmits information regarding the sound source location, The removal processing unit includes a machine learning model that receives the first sound signal as input and outputs the second sound signal from which the first component has been removed. Audio signal processing system.

2. The system includes an aperture synthesis unit that performs aperture synthesis using the second sound signal for each of the sensors, The sound signal processing system according to claim 1, wherein the information generation unit generates information indicating the sound source location identified based on the second sound signal for each of the sensors that has been aperture-synthesized.

3. The aircraft is equipped with an image acquisition unit that acquires an image of the object, The sound signal processing system according to claim 1, wherein the information generation unit generates a composite image in which an associated image relating to the sound source position is placed in a region of the image of the object acquired by the image acquisition unit that corresponds to the sound source position, as information relating to the sound source position.

4. The process involves multiple sensors mounted on an aircraft moving to the location where the object is to be placed receiving a first sound at the location and outputting a first sound signal, The removal processing unit removes a first component corresponding to the flight sound of the aircraft from the first sound signal output by each of the sensors, and generates a second sound signal for each of the sensors. The process includes generating information regarding the sound source location of a characteristic sound when the characteristic sound is emitted from the object, based on the second sound signal for each of the sensors, The removal processing unit includes a machine learning model that receives the first sound signal and outputs a second sound signal from which the first component has been removed, in this sound signal processing method.

5. A method for creating training data for generating the machine learning model used in the sound signal processing system according to any one of claims 1 to 3, A method for creating training data, comprising: a learning unit creating training data to be used for machine learning to generate the machine learning model, based on an audio signal obtained by receiving the aircraft sound and an audio signal of a composite sound including the aircraft sound and sounds other than the aircraft sound.

6. A method for generating a machine learning model, wherein the learning unit performs machine learning using the learning data created by the learning data creation method described in claim 5 to generate the machine learning model.

7. An inspection method for inspecting an object using the sound signal processing method described in claim 4, An inspection method for identifying the sound source location as the location of an abnormality in the object, based on information regarding the sound source location of the characteristic sound.

8. A repair method for performing repair work on the location of the abnormality of the object identified by the inspection method described in claim 7.

9. A sound signal processing unit that generates information regarding the sound source location of a characteristic sound from a first sound signal, including a first sound received at the location of an object, using multiple sensors mounted on an aircraft moving to the location of the object, A removal processing unit removes a first component corresponding to the flight sound from the first sound signal output by each of the sensors and generates a second sound signal for each sensor. An information generation unit generates information regarding the sound source location of a characteristic sound when the characteristic sound is generated from the object, based on the second sound signal for each of the sensors. Equipped with, The removal processing unit includes a machine learning model that receives the first sound signal as input and outputs the second sound signal from which the first component has been removed. A processing unit for audio signal processing.

10. An aircraft that moves to the location where the object is placed, and a communication unit that communicates with the processing unit described in claim 9, The aircraft is controlled via the communication unit, The system includes an output unit that acquires information regarding the sound source location output from the processing unit via the communication unit and outputs it in a predetermined format, The information relating to the sound source location is obtained by the processing unit from a first sound signal received from a plurality of sensors mounted on the aircraft. A control device for an audio signal processing system.

11. A communication unit that communicates with an aircraft moving to the location where the target object is placed, A control unit that controls the aircraft via the communication unit, The processing unit according to claim 9, An information output unit outputs information regarding the sound source location output from the aforementioned processing unit in a predetermined format. A control device for an audio signal processing system equipped with the following features.

Images