Data acquisition device, data acquisition system, and data acquisition method

Abstract

A sound collecting device according to the present invention is provided with: an acquisition unit that acquires sound data stored in a server; a sound acquisition unit that acquires collected sound data representing collected operating sound of an object of interest; a calculation unit that calculates a first score indicating the difference in features between the acquired collected sound data and sound data; and a determination unit that determines, on the basis of the first score, whether or not there is a need to store the collected sound data in the server. The determination unit outputs the collected sound data to the server, and the server stores the collected sound data when the determination unit has determined that the first score is greater than or equal to a first prescribed value.

Description

Sound collection device, sound collection system, and sound collection method

[0001] The present disclosure relates to a sound collection device, a sound collection system, and a sound collection method.

[0002] Patent Literature 1 discloses an agent system that registers a new spoken command in a storage unit based on the collected voice of a passenger. The agent system recognizes voice including a spoken command, which is a command to control onboard equipment mounted on a vehicle in which the passenger is riding and is collected by a microphone, and interprets the meaning of the recognized voice. If the interpreted voice is interpreted as including an instruction to register a new spoken command, the agent system registers the new spoken command in the storage unit.

[0003] Japanese Patent Application Publication No. 2020-144285

[0004] However, in Patent Document 1, if the picked-up voice contains noise, there is a possibility that a voice command that is already stored in the storage unit may be misinterpreted as a new voice command. In such a case, the agent system may register the same voice command twice in the storage unit, or may misinterpret the voice command and register an incorrect voice command as a new voice command, which may result in a decrease in the accuracy of voice interpretation.

[0005] The present disclosure has been devised in view of the above-described conventional situation, and aims to provide a sound collection device, a sound collection system, and a sound collection method that efficiently collect abnormal sound data suitable for machine learning.

[0006] The present disclosure provides a sound collection device including an acquisition unit that acquires sound data stored in a server, an audio acquisition unit that acquires sound collection data obtained by collecting operating sounds of an object, a calculation unit that calculates a first score that indicates a difference in features between the acquired sound collection data and the sound data, and a determination unit that determines whether or not it is necessary to store the sound collection data in the server based on the first score, wherein the determination unit outputs the sound collection data to the server for storage when it determines that the first score is equal to or greater than a first predetermined value.

[0007] The present disclosure also provides a sound collection system including a server that stores one or more sound data, and a terminal device that can communicate with the server, wherein the server transmits the stored sound data to the terminal device, the terminal device acquires collected sound data in which operating sounds of an object are collected, calculates a first score indicating a difference in features between the transmitted sound data and the collected sound data, determines whether the calculated first score is equal to or greater than a first predetermined value, and if it is determined that the first score is equal to or greater than the first predetermined value, transmits the collected sound data to the server, and the server records the transmitted collected sound data.

[0008] The present disclosure also provides a sound collection method performed by a terminal device capable of communicating with a server that stores one or more sound data, the sound collection method comprising: acquiring collected sound data in which an operating sound of an object is collected; acquiring the sound data stored in the server; calculating a first score indicating a difference in features between the collected sound data and the sound data; determining whether the calculated first score is equal to or greater than a first predetermined value; and, if it is determined that the first score is equal to or greater than the first predetermined value, transmitting the collected sound data to the server for recording.

[0009] The present disclosure also provides a sound collection method performed by a terminal device capable of communicating with a server that stores one or more sound data, the sound collection method comprising: acquiring collected sound data in which an operating sound of an object is collected; acquiring the sound data stored in the server; calculating a first score indicating a difference in features between the collected sound data and the sound data; determining whether the calculated first score is equal to or greater than a first predetermined value; and, if it is determined that the first score is equal to or greater than the first predetermined value, determining that the collected sound data is sound stored in the server.

[0010] According to the present disclosure, abnormal sound data suitable for machine learning can be efficiently collected.

[0011] FIG. 1 is a diagram showing an example of the system configuration of an abnormal sound collecting system according to embodiment 1. FIG. 2 is a block diagram showing an example of the internal configuration of an abnormal sound collecting device and a cloud server according to embodiment 1. FIG. 3 is a flowchart showing an example of the operation procedure of the abnormal sound collecting device according to embodiment 1. FIG. 4 is a diagram explaining an example of the distance between a recorded sound and a group of normal sounds. FIG. 5 is a diagram explaining an example of the minimum abnormal sound distance between a recorded sound and a plurality of abnormal sounds. FIG. 6 is a diagram explaining an example of a determination result screen. FIG. 7 is a diagram explaining an example of an abnormal sound data screen. FIG. 8 is a diagram showing an example of an abnormal sound data table stored in an abnormal sound storage unit.

[0012] Hereinafter, with reference to the drawings as appropriate, detailed descriptions of embodiments specifically disclosing a sound collection device, a sound collection system, and a sound collection method according to the present disclosure will be described in detail. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0013] An example of the system configuration of an abnormal sound collection system 1000 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the system configuration of the abnormal sound collection system 1000 according to the first embodiment.

[0014] An abnormal sound collection system 1000, as an example of a sound collection system, is a system that uses one or more abnormal sound collection devices 10 to collect operating sounds of inspection objects (e.g., various operating devices such as electrical equipment and electronic devices), and collects and stores abnormal sound data for machine learning to automatically detect abnormal sounds emitted from the inspection objects. The abnormal sound collection system 1000 is configured to include one or more abnormal sound collection devices 10, one or more cloud servers, and a network NW. Note that the abnormal sound collection system 1000 shown in FIG. 1 is, as an example, configured to include one abnormal sound collection device 10 and one cloud server 50.

[0015] An abnormal sound collecting device 10, which is an example of a terminal device that is an example of a sound collecting device, is connected via a network NW to one or more cloud servers 50 so as to be able to communicate data with each other. The abnormal sound collecting device 10 is, for example, a tablet, a smartphone, a laptop PC (Personal Computer), or the like, and collects operating sounds from one or more inspection objects 101, 102, and 103. The abnormal sound collecting device 10 extracts abnormal sound data suitable for machine learning for abnormal sound detection that is not stored in the cloud server 50 from the collected collected sound data, and transmits the extracted abnormal sound data to the cloud server 50 for storage.

[0016] The abnormal sound collecting device 10 may receive an operation from the operator hm to collect the operation sounds of each of the multiple inspection objects 101 to 103. The abnormal sound collecting device 10 may also be installed in a location where it can collect the operation sounds of each of the multiple inspection objects 101 to 103 (for example, on a wall, the ground, a ceiling, or suspended from a wall or ceiling), and may collect the operation sounds continuously or automatically during a predetermined time period that is set in advance.

[0017] Furthermore, it goes without saying that the abnormal noise collecting device 10 may be installed at each of a plurality of bases, and a plurality of devices may be installed at each base (for example, "base A" shown in FIG. 1).

[0018] A cloud server 50, which is an example of a server, is connected via the network NW to enable data communication with each of the one or more abnormal sound collecting devices 10. The cloud server 50 stores one or more pieces of abnormal sound data for various inspection objects transmitted from each of the one or more abnormal sound collecting devices 10. The cloud server 50 also stores one or more pieces of normal sound data for various inspection objects that have been collected in advance.

[0019] Furthermore, the cloud server 50 extracts one or more pieces of abnormal sound data that match the conditions of the object to be inspected from each of the one or more pieces of stored abnormal sound data based on the condition information of the object to be inspected transmitted from the abnormal sound collecting device 10, and transmits the extracted data to the abnormal sound collecting device 10. Based on the one or more pieces of abnormal sound data transmitted from the cloud server 50 and the collected sound data that has been determined to be an abnormal sound, the abnormal sound collecting device 10 determines whether the collected sound data is abnormal sound data that should be stored in the cloud server 50 (that is, suitable for machine learning).

[0020] The network NW connects each of the one or more abnormal sound collecting devices 10 and each of the one or more cloud servers 50 via wired or wireless communication so that data can be communicated between them. The network NW may be a wired network, a wireless network, or a combination thereof. The wired network may be, for example, a wired LAN (Local Area Network) such as Ethernet (registered trademark), and the type is not particularly limited. On the other hand, the wireless network may be, for example, a wireless LAN such as Wi-Fi (registered trademark), and the type is not particularly limited.

[0021] Next, an example of the internal configuration of each of the abnormal sound collecting device 10 and the cloud server 50 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the internal configuration of each of the abnormal sound collecting device and the cloud server according to the first embodiment.

[0022] The abnormal sound collecting device 10 includes a sound collecting device 30, a display device 31, an input device 32, a communication unit 11, an I / F (Interface) 12, a processor 13, and a memory 21. Note that the sound collecting device 30, the display device 31, and the input device 32 may each be an external device that is not configured integrally with the abnormal sound collecting device 10, and may be connected to the abnormal sound collecting device 10 so as to be able to send and receive data therebetween.

[0023] The sound collection device 30 collects operation sounds of the inspection object at a predetermined sampling frequency such as 16 kHz, 48 kHz, etc. The sound collection device 30 converts the collected operation sounds into audio signals and inputs them to the processor 13 via the I / F 12. Specifically, the audio signals output from the sound collection device 30 are input to the audio processing unit 14 of the processor 13.

[0024] The sound collection device 30 may collect the operation sounds of the inspection object based on the operation of the worker hm using the input device 32, or may automatically collect the operation sounds of the inspection object based on a preset inspection schedule including the inspection date and time, the number of inspections, etc. The collection time of the operation sounds collected by the sound collection device 30 may be the time period while the worker hm continues to press the sound collection button (not shown) of the input device 32, the time period from when the worker hm presses the start button (not shown) of the input device 32 to when the worker hm presses the stop button (not shown), or a predetermined time period (for example, 1 s, 3 s, 10 s, etc.) designated in advance.

[0025] The display device 31 is configured using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display, and displays (outputs) various screens (such as judgment result screens Sc1, Sc2, and Sc3 (see FIG. 6 ) and abnormal sound data screen Sc4 (see FIG. 7 )) output from the display control unit 16 of the processor 13 via the I / F 12.

[0026] The input device 32 is configured using at least one of devices such as a touch panel, a mouse, a keyboard, a touch pad, etc. The input device 32 accepts an input operation by the worker hm (see FIG. 1 ) using the abnormal sound collecting device 10, generates a signal according to this input operation, and outputs it to the input processing unit 15 of the processor 13 via the I / F 12. Note that when the input device 32 is configured using a touch panel, the input device 32 and the display device 31 are configured integrally.

[0027] The communication unit 11, which is an example of an acquisition unit, transmits and receives data to and from the cloud server 50 via the network NW. The communication unit 11 outputs one or more pieces of normal sound data or one or more pieces of abnormal sound data transmitted from the storage device 53 of the cloud server 50 to the processor 13. The communication unit 11 also transmits the abnormal sound data output from the processor 13 to the cloud server 50.

[0028] The I / F 12 connects the processor 13 to the sound collection device 30, the display device 31, and the input device 32 so that data can be communicated between them.

[0029] The processor 13, which serves as an example of a calculation unit and a determination unit, is configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array), and controls the operation of each unit of the processor 13. The processor 13 performs various processes and controls in cooperation with the memory 21. Specifically, the processor 13 references programs and data stored in the memory 21 and executes the programs to realize the functions of each unit. The respective units referred to here are the voice processing unit 14, the input processing unit 15, the display control unit 16, the normal sound learning unit 17, the abnormal sound determination unit 18, the SN determination unit 19, and the abnormal sound storage determination unit 20.

[0030] The sound processing unit 14, which is an example of a sound acquisition unit, acquires the sound signal output from the sound collection device 30 and converts it into collected sound data in a format that can be processed by each unit of the processor 13 (e.g., the normal sound learning unit 17, the abnormal sound detection unit 18, the SN detection unit 19, and the abnormal sound storage detection unit 20). The sound processing unit 14 outputs the converted collected sound data to the abnormal sound detection unit 18.

[0031] The input processing unit 15 executes various processes based on the signals output from the input device 32 .

[0032] For example, when the input device 32 receives an operation by the operator hm to start collecting operation sounds of the inspection object, it generates a control command (signal) to start collecting operation sounds of the inspection object and outputs it to the input processing unit 15. Based on the control command (signal) output from the input device 32, the input processing unit 15 operates the sound collection device 30 to start collecting operation sounds of the inspection object, and causes the abnormal sound detection unit 18 to execute abnormal sound detection processing using the collected sound data.

[0033] Furthermore, for example, when the input device 32 receives an operation by the operator hm to continue collecting the operation sound of the inspection object, it generates a control command (signal) to continue collecting the operation sound of the inspection object and outputs it to the input processing unit 15. Based on the control command (signal) to continue collecting the operation sound of the inspection object output from the input device 32, the input processing unit 15 continues to operate the sound collection device 30 to collect the operation sound of the inspection object again, or causes the abnormal sound detection unit 18 to perform the abnormal sound detection process again using the collected sound data based on the collected operation sound of the inspection object.

[0034] The display control unit 16 generates screens corresponding to the determination results (determination result screens Sc1, Sc2, Sc3 (see FIG. 6), abnormal sound data screen Sc4 (see FIG. 7), etc.) based on the respective determination results of the abnormal sound determination unit 18, the SN determination unit 19, and the abnormal sound storage determination unit 20. The display control unit 16 outputs the generated screens (determination result screens Sc1, Sc2, Sc3 (see FIG. 6), abnormal sound data screen Sc4 (see FIG. 7), etc.) to the display device 31 for display.

[0035] The normal sound learning unit 17 acquires the collected sound data output from the sound collection device 30. The normal sound learning unit 17 analyzes the acquired collected sound data, and if it determines that the collected sound data is normal sound data to be stored in the normal sound storage unit 54 in the cloud server 50, it learns the collected sound data.

[0036] The abnormal sound detector 18 acquires one or more pieces of normal sound data from the normal sound storage unit 54 of the cloud server 50. Note that the one or more pieces of normal sound data acquired here may be normal sound data related to the inspection object (e.g., normal sound data for the same inspection object, normal sound data collected at the same location, etc.), or, if conditions for the inspection object (e.g., conditions related to the inspection object, conditions related to the location, etc.) are specified by the operator hm, the normal sound data may be normal sound data corresponding to the specified conditions.

[0037] The abnormal sound detector 18 calculates an index (distance) indicating the difference between the feature amounts of the collected sound data and the feature amounts of one or more pieces of normal sound data transmitted from the normal sound storage unit 54 of the cloud server 50. Note that the distance here is an index indicating the difference in sound based on various feature amounts such as the frequency and sound pressure of each piece of sound data, and may be calculated using absolute value distance, Euclidean distance, Mahalanobis distance, or deep learning such as an autoencoder.

[0038] The abnormal sound detector 18 determines whether the acquired sound collection data represents an abnormal sound based on the calculated distance. If the abnormal sound detector 18 determines that the acquired sound collection data represents an abnormal sound, it generates a control command requesting a determination as to whether the sound collection data is suitable for machine learning, and outputs the control command to the SN determination unit 19 or the abnormal sound storage determination unit 20. On the other hand, if the abnormal sound detector 18 determines that the acquired sound collection data does not represent an abnormal sound, it generates a control command to generate a determination result screen Sc2 notifying that the sound collection data is not an abnormal sound, and outputs the control command to the display control unit 16.

[0039] The SN determination unit 19 determines whether the SN ratio of the picked-up sound data is suitable as abnormal sound data to be used for machine learning, based on the control command output from the abnormal sound determination unit 18. Note that when the operator hm performs an operation to omit the processing by the SN determination unit 19, the processor 13 may omit the processing executed by the SN determination unit 19.

[0040] The abnormal sound storage determination unit 20 calculates an index (distance) indicating the difference between the feature amounts of the one or more pieces of abnormal sound data transmitted from the abnormal sound storage unit 55 of the cloud server 50 and the feature amounts of the collected sound data, based on the control command output from the abnormal sound determination unit 18. Note that the distance here is an index indicating the difference in sound based on various feature amounts such as the frequency and sound pressure of each piece of sound data, and may be calculated using absolute value distance, Euclidean distance, Mahalanobis distance, or deep learning such as an autoencoder.

[0041] The abnormal sound storage determination unit 20 selects a minimum abnormal sound distance that is the smallest of the distances between the feature amounts of the one or more calculated abnormal sound data and the feature amounts of the sound pickup data. Based on the selected minimum abnormal sound distance, the abnormal sound storage determination unit 20 determines whether the acquired sound pickup data (abnormal sound data) is abnormal sound data already stored in the cloud server 50. If the abnormal sound storage determination unit 20 determines that the acquired sound pickup data (abnormal sound data) is abnormal sound data already stored in the cloud server 50, it determines not to store this sound pickup data. On the other hand, if the abnormal sound storage determination unit 20 determines that the acquired sound pickup data (abnormal sound data) is not abnormal sound data already stored in the cloud server 50, it determines to store this sound pickup data in the cloud server 50.

[0042] The processing performed by the SN determination unit 19 and the processing performed by the abnormal sound storage determination unit 20 may be performed simultaneously or in parallel, or one of the processing may be performed after the other. As an example of the operation procedure of the abnormal sound collecting device 10 described below, an example will be described in which the processing by the abnormal sound storage determination unit 20 is performed first, and the processing by the SN determination unit 19 is performed based on the result of the processing by the abnormal sound storage determination unit 20.

[0043] If the SN determination unit 19 determines that the SN ratio of the collected sound data is suitable for use as abnormal sound data for machine learning, and the abnormal sound memory determination unit 20 determines that the collected sound data is not stored in the abnormal sound memory unit 55, the processor 13 associates the collected sound data with identification information that can identify the abnormal sound collection device 10 and various information related to the collected sound data, outputs the association to the communication unit 11, and causes it to be transmitted to the cloud server 50.

[0044] The various information referred to here includes information on the date and time when the sound was collected, information on the object to be inspected (for example, device identification information, model information, etc.), information on the installation state of the abnormal sound collecting device 10, information on the operating mode of the object to be inspected, etc. Note that the various information does not have to be limited to the examples described above.

[0045] The memory 21 includes at least a RAM (Random Access Memory) as a work memory used when the processor 13 executes various processes, and a ROM (Read Only Memory) that stores programs that define the various processes executed by the processor 13 and data used during execution of the programs. The RAM temporarily stores data or information generated or acquired by the processor 13. The ROM stores programs that define the various processes executed by the processor 13 and data used during execution of the programs.

[0046] The cloud server 50 includes a communication unit 51 , a processor 52 , and a storage device 53 .

[0047] The communication unit 51 transmits and receives data to and from each of the one or more abnormal sound collecting devices 10 via the network NW. The communication unit 51 transmits one or more pieces of normal sound data or one or more pieces of abnormal sound data output from the storage device 53 to the abnormal sound collecting device 10. The communication unit 51 also outputs the abnormal sound data (collected sound data) transmitted from the abnormal sound collecting device 10 to the processor 52.

[0048] The processor 52 is configured using, for example, a CPU, DSP, or FPGA, and controls the operation of each unit of the processor 52. The processor 52 performs various processes and controls in cooperation with the storage device 53. Specifically, the processor 52 references the programs and data stored in the storage device 53 and executes the programs to realize the functions of each unit.

[0049] The processor 52 extracts one or more normal sound data and one or more abnormal sound data that match the conditions of the inspection object from the normal sound storage unit 54 and the abnormal sound storage unit 55 of the storage device 53, respectively, based on the conditions of the inspection object transmitted from the abnormal sound collecting device 10, and transmits these to the abnormal sound collecting device 10. Furthermore, when the processor 52 acquires abnormal sound data (collected sound data) transmitted from the abnormal sound collecting device 10, it outputs the data to the storage device 53 for storage.

[0050] The storage device 53 includes at least a RAM serving as a work memory used when the processor 52 executes various processes, and a ROM for storing programs defining the various processes executed by the processor 52 and data used during execution of the programs. The RAM temporarily stores data or information generated or acquired by the processor 52. The ROM stores programs defining the various processes executed by the processor 52 and data used during execution of the programs. The storage device 53 also includes a normal sound storage unit 54 and an abnormal sound storage unit 55.

[0051] The normal sound storage unit 54 stores one or more normal sound data used for machine learning of normal sounds by the abnormal sound collecting device 10. The normal sound data is stored in association with identification information of the abnormal sound collecting device that collected the normal sound, various information related to the normal sound, etc. Note that the various information here refers to information on the date and time when the normal sound was collected, information related to the object to be inspected (e.g., device identification information, model information, etc.), information related to the installation status of the abnormal sound collecting device, information related to the operating mode of the object to be inspected, etc. Note that the various information does not have to be limited to the examples described above.

[0052] The abnormal sound storage unit 55 stores one or more pieces of abnormal sound data used for machine learning of normal sounds by the abnormal sound collecting device 10. The abnormal sound data is stored in association with identification information of the abnormal sound collecting device that collected the abnormal sound, various information related to the abnormal sound, etc. Note that the various information here refers to information on the date and time when the abnormal sound was collected, information related to the object to be inspected (e.g., identification information of the device, model information, etc.), identification information of the sound collecting device, base information, information related to the installation status of the abnormal sound collecting device, information related to the operating mode of the object to be inspected, etc. Note that the various information does not have to be limited to the examples described above.

[0053] Next, an example of the operation procedure of the abnormal sound collecting device 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the operation procedure of the abnormal sound collecting device 10 according to the first embodiment.

[0054] The sound collection device 30 in the abnormal sound collection device 10 collects operation sounds of one or more inspection objects based on an operator's operation on the input device 32 or a preset inspection date and time (St11). The sound collection device 30 converts the collected sounds into audio signals and outputs them to the audio processing unit 14 of the processor 13.

[0055] The input device 32 accepts an operation by the operator hm to specify the conditions of the object to be inspected (St12). The input device 32 generates a signal corresponding to the conditions of the object to be inspected input by the operator hm and outputs the signal to the processor 13. Note that the abnormal sound collecting device 10 may omit the processing of step St12 if the conditions of the object to be inspected have been set in advance.

[0056] Based on the conditions of the inspection object input to the input device, the processor 13 generates a control command requesting one or more pieces of normal sound data that meet these conditions, and transmits the control command to the cloud server 50 via the network NW. The cloud server 50 extracts one or more pieces of normal sound data that meet the conditions of the inspection object based on the control command transmitted from the abnormal sound collecting device 10. The cloud server 50 transmits the one or more pieces of extracted normal sound data to the abnormal sound collecting device 10 via the network NW. The processor 13 acquires and reads the one or more pieces of normal sound data transmitted from the cloud server 50 (St13).

[0057] The abnormal sound detector 18 of the processor 13 calculates the distance (for example, the distance D0 shown in FIG. 4 ) between the normal sound data read in step St13 and the collected sound data converted and output by the sound processor 14 (St14). The abnormal sound detector 18 determines whether the calculated distance is equal to or greater than a first threshold value (St15).

[0058] If the abnormal sound detector 18 determines in the processing of step St15 that the calculated distance is equal to or greater than the first threshold value (YES in St15), it determines that the collected sound data is a normal sound (St 16). If the abnormal sound detector 18 determines that the operation sound of the inspection object is a normal sound, the display controller 16 generates a determination result screen Sc1 (see FIG. 6 ) that notifies the user that the collected operation sound of the inspection object is an abnormal sound, and outputs the determination result screen Sc1 to the display device 31 via the I / F 12 (St18).

[0059] On the other hand, if the abnormal sound detector 18 determines in the processing of step St15 that the calculated distance is not equal to or greater than the first threshold value (NO in St15), it determines that the operation sound of the inspection object is a normal sound (St 17). If the abnormal sound detector 18 determines that the operation sound of the inspection object is a normal sound, the display controller 16 generates a determination result screen Sc2 (see FIG. 6 ) that notifies the user that the picked-up operation sound of the inspection object is a normal sound, and outputs the determination result screen Sc2 to the display device 31 via the I / F 12 (St26).

[0060] Based on the conditions of the inspection object input to the input device, the processor 13 generates a control command requesting one or more pieces of abnormal sound data that match these conditions, and transmits the control command to the cloud server 50 via the network NW. The cloud server 50 extracts one or more pieces of abnormal sound data that match the conditions of the inspection object based on the control command transmitted from the abnormal sound collecting device 10. The cloud server 50 transmits the one or more pieces of extracted abnormal sound data to the abnormal sound collecting device 10 via the network NW. The processor 13 acquires and reads the one or more pieces of abnormal sound data transmitted from the cloud server 50 (St19).

[0061] The abnormal sound memory determination unit 20 calculates the smallest abnormal sound distance (for example, distance ASD1 shown in FIG. 5 ) among the distances between the feature amounts of the one or more abnormal sound data read in step St19 and the feature amounts of the sound pickup data (St20). The abnormal sound memory determination unit 20 determines whether the calculated smallest abnormal sound distance is equal to or greater than a second threshold value (St21). It goes without saying that the abnormal sound memory determination unit 20 may calculate multiple distances and then select the smallest abnormal sound distance from the multiple calculated distances.

[0062] If the abnormal sound storage determination unit 20 determines in the processing of step St21 that the calculated minimum abnormal sound distance is equal to or greater than the second threshold value (YES in St21), it determines that the picked-up sound data is not abnormal sound data already stored in the cloud server 50. Furthermore, the SN determination unit 19 determines whether the SN ratio of the picked-up sound data is equal to or greater than a predetermined value (St22).

[0063] On the other hand, if the abnormal sound storage determination unit 20 determines in the processing of step St21 that the calculated minimum abnormal sound distance is not greater than the second threshold value (St21, NO), it determines that the collected sound data is abnormal sound data that is identical to or similar to abnormal sound data already stored in the cloud server 50, and does not store it in the cloud server 50 (St23).

[0064] If the SN determination unit 19 determines in the processing of step St22 that the SN ratio of the sound collection data is equal to or greater than a predetermined value (YES in St22), it determines that the sound collection data is abnormal sound data suitable for machine learning. The processor 13 associates the sound collection data, identification information that can identify the abnormal sound collecting device 10, and various information related to the sound collection data, transmits the associated data to the cloud server 50, and stores the associated data in the abnormal sound storage unit 55 (St24), and stores the associated data in the memory 21.

[0065] On the other hand, if the SN determination unit 19 determines in the processing of step St22 that the SN ratio of the picked-up sound data is not equal to or greater than the predetermined value (St22, NO), it determines that the picked-up sound data is not abnormal sound data suitable for machine learning used in the abnormal sound detection processing (St25).

[0066] If, in the processing of step St21, the abnormal sound storage determination unit 20 determines that the minimum abnormal sound distance is equal to or greater than the second threshold value (St21, YES), the display control unit 16 generates a determination result screen Sc3 (see FIG. 6) notifying that the picked-up operating sound is abnormal sound data already stored in the cloud server 50, and outputs and displays it on the display device 31 (St26).

[0067] In addition, after transmitting the collected sound data to the cloud server 50 and storing it, the display control unit 16 generates a completion notification (not shown) notifying the completion of the storage process of the collected sound data to the cloud server 50, and outputs it to the display device 31 (St26).

[0068] The input device 32 accepts an operator hm operation as to whether or not to continue collecting the operation sound of the inspection object. Here, the display control unit 16 may generate a selection screen (not shown) that allows the operator to select whether or not to continue collecting the operation sound of the inspection object, and output the selection screen to the display device 31 for display. The input device 32 converts the accepted operator hm operation into a corresponding signal and outputs the signal to the processor 13. The processor 13 determines whether or not to continue collecting the operation sound of the inspection object based on the signal output from the input device 32 (St27).

[0069] If the processor 13 determines in the processing of step St27 to continue collecting the operation sound of the inspection object (St27, YES), the processor 13 returns to the processing of step St11. On the other hand, if the processor 13 determines in the processing of step St27 not to continue collecting the operation sound of the inspection object (St27, NO), the processor 13 ends the operation procedure shown in FIG.

[0070] In addition, if an inspection schedule including the inspection date and time, the number of inspections, etc. has been set in advance, the processing of step St27 may determine whether or not to continue collecting the operating sounds of the object to be inspected based on this set inspection schedule.

[0071] Furthermore, the processes of steps St13 to St25 of the above-described operational procedure may be executed by the cloud server 50 or another server (not shown) connected to the abnormal sound collecting device 10 via the network NW so as to be able to communicate data with the abnormal sound collecting device 10. In such a case, the abnormal sound collecting device 10 associates the collected pickup data of the operation sounds of the inspection object with the conditions of the inspection object specified by the operator hm operation, and transmits the associated data to the cloud server 50 or another server. After executing the processes of steps St13 to St25, the cloud server 50 or another server transmits information on the determination results to the abnormal sound collecting device 10.

[0072] Furthermore, in the above-described operational procedure, if the distance calculated in the processing of step St14 is calculated as an absolute distance, the processing procedure based on the determination result in the processing of step St15 may be reversed. Specifically, in such a case, if the abnormal sound detector 18 determines in the processing of step St15 that the calculated distance (absolute distance) is equal to or greater than the first threshold (St15, YES), the abnormal sound detector 18 may determine that the collected sound data is a normal sound (St17). Furthermore, if the abnormal sound detector 18 determines in the processing of step St15 that the calculated distance (absolute distance) is not equal to or greater than the first threshold (St15, NO), the abnormal sound detector 18 may determine that the collected sound data is an abnormal sound (St16).

[0073] Similarly, if the minimum abnormal sound distance calculated in the processing of step St20 is calculated as an absolute value distance, the processing procedure based on the determination result in the processing of step St21 may be reversed. Specifically, in such a case, if the abnormal sound storage determination unit 20 determines in the processing of step St21 that the calculated minimum abnormal sound distance (absolute value distance) is equal to or greater than the second threshold (St21, YES), it may determine that the collected operation sound of the inspection object is abnormal sound data already stored in the cloud server 50 and should not be stored in the cloud server 50 (St23). Furthermore, if the abnormal sound storage determination unit 20 determines in the processing of step St21 that the calculated minimum abnormal sound distance (absolute value distance) is not equal to or greater than the second threshold (St21, NO), it determines that the collected operation sound of the inspection object is not abnormal sound data already stored in the cloud server 50. Furthermore, the SN determination unit 19 determines whether the SN ratio of the collected sound data is equal to or greater than a predetermined value (St22).

[0074] As described above, the abnormal sound collecting device 10 according to the first embodiment can store in the cloud server 50 only abnormal sound data that has been determined to be an abnormal sound and that has not already been stored in the cloud server 50. Therefore, by storing dissimilar abnormal sound data, the abnormal sound collecting device 10 can effectively suppress bias in the abnormal sound data used for machine learning and more efficiently collect abnormal sound data suitable for machine learning.

[0075] Furthermore, the abnormal sound collection device 10 according to the first embodiment can store in the cloud server 50 only abnormal sound data that is determined to be an abnormal sound when the collected operating sound of the object being inspected is an abnormal sound and that has not yet been stored in the cloud server 50, and that has an S / N ratio equal to or greater than a predetermined value (i.e., abnormal sound data that contains little noise in the collected sound data).

[0076] The calculation process of the distance D0 between the collected sound data RS and the normal sound data group NS will be described with reference to Fig. 4. Fig. 4 is a diagram illustrating an example of the distance D0 between the collected sound data RS and the normal sound data group NS.

[0077] 4 shows an example of calculating the distance D0 between the sound pickup data RS and the normal sound data group NS based on the Mahalanobis distance. Also, the graph shown in FIG. 4 shows an example of calculating the distance D0 between the sound pickup data RS and the normal sound data group NS in a two-dimensional space based on two axes (sound pressure, frequency), but it goes without saying that the number of axes (feature amounts) used to calculate the distance D0 may be three or more.

[0078] The normal sound data group NS is each of one or more normal sound data transmitted from the cloud server 50. Note that, although Fig. 4 shows an example in which one or more normal sound data are distributed in each normal sound data group NS, it goes without saying that the distance between the picked-up sound data RS and each normal sound data may be calculated.

[0079] 4, the first axis represents the feature amount of sound pressure as a first feature amount of sound, and the second axis represents the feature amount of frequency as a second feature amount of sound.

[0080] The abnormal sound detector 18 analyzes each of the feature quantities (sound pressure and frequency in the example shown in FIG. 4 ) of the sound collection data RS and the normal sound data group NS. Based on the analysis results, the abnormal sound detector 18 calculates the distance D0 between the sound collection data RS and the normal sound data group NS. The abnormal sound detector 18 determines whether the sound collection data RS is an abnormal sound based on whether the calculated distance D0 (an example of a second score) is equal to or greater than a first threshold value.

[0081] The calculation process of the minimum abnormal sound distance between the sound pickup data RS and the plurality of abnormal sound data AS1, AS2, and AS3 will be described with reference to Fig. 5. Fig. 5 is a diagram illustrating an example of the minimum abnormal sound distance between the sound pickup data RS and the plurality of abnormal sound data AS1 to AS3.

[0082] 5 shows an example of calculation of distances ASD1 to ASD3 between the sound pickup data RS and the plurality of abnormal sound data AS1 to AS3 based on the Mahalanobis distance, as an example. Also, the graph shown in Fig. 5 shows an example of calculation of distances ASD1 to ASD3 between the sound pickup data RS and the plurality of abnormal sound data AS1 to AS3 in a two-dimensional space based on two axes (sound pressure, frequency), as an example, but it goes without saying that the number of axes (feature amounts) used to calculate the distances ASD1 to ASD3 may be three or more.

[0083] 5, the first axis represents the feature amount of sound pressure as a first feature amount of sound, and the second axis represents the feature amount of frequency as a second feature amount of sound.

[0084] The abnormal sound storage determination unit 20 analyzes each of the feature quantities (sound pressure and frequency in the example shown in FIG. 5 ) of the sound collection data RS and the plurality of abnormal sound data AS1 to AS3. Based on the analysis results, the abnormal sound storage determination unit 20 calculates distances ASD1 to ASD3 between the sound collection data RS and the plurality of abnormal sound data AS1 to AS3. The abnormal sound storage determination unit 20 selects a minimum abnormal sound distance that is the smallest of the calculated distances ASD1 to ASD3. In the example shown in FIG. 5 , the minimum abnormal sound distance is distance ASD1. The abnormal sound storage determination unit 20 determines whether the sound collection data RS is abnormal sound data similar to the plurality of abnormal sound data AS1 to AS3 (i.e., abnormal sound data that has already been stored in the cloud server 50) based on whether the selected distance ASD1 is equal to or greater than a second threshold.

[0085] 6, various determination result screens that are generated based on the determination results by the abnormal sound determination unit 18 and the abnormal sound storage determination unit 20 of the processor 13 and displayed on the display device 31 will be described. Fig. 6 is a diagram illustrating each of the determination result screens Sc1 to Sc3. Note that the determination result screens Sc1 to Sc3 shown in Fig. 6 are merely examples, and the present invention is not limited to these.

[0086] When the abnormal sound detector 18 determines that the distance (for example, distance D0 shown in FIG. 4 ) between the picked-up sound data and one or more normal sounds acquired from the cloud server 50 is equal to or greater than the first threshold (i.e., the picked-up sound data is an abnormal sound), the display controller 16 generates a determination result screen Sc1 notifying the user that the picked-up sound data is an abnormal sound. The display controller 16 outputs the generated determination result screen Sc1 to the display device 31 for display. Note that the determination result screen Sc1 shown in FIG. 6 is generated, as an example, to include a message Msg1 "Abnormal sound" indicating that the picked-up sound data is an abnormal sound, but the present invention is not limited to this.

[0087] When the abnormal sound detector 18 determines that the distance between the sound pickup data and one or more normal sounds acquired from the cloud server 50 (for example, the distance D0 shown in FIG. 4 ) is not equal to or greater than the first threshold (i.e., the sound pickup data is normal sound), the display controller 16 generates a determination result screen Sc2 notifying the operator that the sound pickup data is normal sound. The display controller 16 outputs the generated determination result screen Sc2 to the display device 31 for display. Note that the determination result screen Sc2 shown in FIG. 6 is generated, as an example, to include a message Msg2 "It's a normal sound. Do you want to stop measuring?" that indicates that the sound pickup data is normal sound and asks the operator hm whether or not to continue collecting the operation sound of the object to be inspected, but the present invention is not limited to this.

[0088] When the abnormal sound storage determination unit 20 determines that the minimum abnormal sound distance (for example, the distance ASD1 shown in FIG. 5 ) between the picked-up sound data and one or more abnormal sounds acquired from the cloud server 50 is not equal to or greater than the second threshold, the display control unit 16 generates a determination result screen Sc3 that notifies the user that the picked-up sound data is the same as or similar to an abnormal sound already stored in the cloud server 50. The display control unit 16 outputs the generated determination result screen Sc3 to the display device 31 for display. Note that the determination result screen Sc3 shown in FIG. 6 is generated, as an example, to include a message Msg3 "This is a similar sound to an abnormal sound already stored," that notifies the user that the picked-up sound data is an abnormal sound and is the same as or similar to an abnormal sound already stored in the cloud server 50, but the present invention is not limited to this.

[0089] Next, an example of the abnormal sound data screen Sc4 will be described with reference to Fig. 7. Fig. 7 is a diagram illustrating an example of the abnormal sound data screen Sc4. Note that the abnormal sound data screen Sc4 shown in Fig. 7 is an example of an abnormal sound data screen that is generated when multiple microphones (an example of sound collection devices 30) are installed at one location, and is not limited to this.

[0090] In such a case, two or more microphones (sound collection devices 30) may be provided for one abnormal sound collecting device 10. Furthermore, the microphone (sound collection device 30) may be an external device connected so as to be able to send and receive data to and from the abnormal sound collecting device 10. Two or more microphones (sound collection devices 30) may be connectable to one abnormal sound collecting device 10.

[0091] The abnormal sound data screen Sc4 is generated by the display control unit 16 based on the respective determination results of the abnormal sound determination unit 18 and the abnormal sound storage determination unit 20 of the processor 13. Specifically, the respective determination results of the abnormal sound determination unit 18 and the abnormal sound storage determination unit 20 of the processor 13 are accumulated and stored in the memory 21. The display control unit 16 extracts the determination result of one or more pieces of picked-up sound data that have been determined to be an abnormal sound from the one or more determination results accumulated and stored in the memory 21. The display control unit 16 generates the abnormal sound data screen Sc4 based on various information associated with the extracted picked-up sound data, and outputs it to the display device 31 for display.

[0092] The abnormal sound data screen Sc4 includes start date and time information If1 indicating the timing when collection of the operating sound of the inspection object started, and abnormal sound information If11 and If1n indicating collected sound data determined to be abnormal sounds. The start date and time information If1 shown in FIG. 7 is "XX year, YY month, ZZ day, xx hour, yy minute, zz second." The abnormal sound information If1 shown in FIG. 7 also includes the abnormal sound number "abnormal sound 1," the collection date and time information when the abnormal sound was collected "A1 year, B1 month, C1 day, D1 hour, E1 minute, F1 second to G1 year, H1 month, I1 day, J1 hour, K1 minute, L1 second," and the identification information of the sound collection device 30, "microphone 1." The abnormal sound information If12 includes an abnormal sound number "abnormal sound n", the sound pickup date and time information when the abnormal sound was picked up "An year Bn month Cn day Dn hour En minute Fn second to Gn year Hn month In day Jn hour Kn minute Ln second", and the identification information of the sound pickup device 30 "microphone 2".

[0093] Next, the abnormal sound data table TB1 stored in the abnormal sound storage unit 55 of the cloud server 50 will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the abnormal sound data table TB1 stored in the abnormal sound storage unit 55. Note that the items in the abnormal sound data table TB1 shown in Fig. 8 are merely an example, and the present invention is not limited to these.

[0094] The abnormal sound storage unit 55 stores an abnormal sound data table TB1 that includes each piece of abnormal sound data transmitted from one or more abnormal sound collectors 10 and various information associated with the abnormal sound data. The abnormal sound data table TB1 shown in FIG. 8 includes, as an example, an item "inspection object," an item "microphone ID," an item "installation status," an item "operation mode," and waveform data of the abnormal sound data.

[0095] Here, when the conditions of the inspection object are transmitted from the abnormal sound collecting device 10, the processor 52 compares the information of each item with the conditions of the inspection object. The processor 52 extracts one or more abnormal sound data items that match the conditions of the inspection object and are associated with the information of each item, and transmits the data to the cloud server 50.

[0096] The item "inspection object" indicates information about the inspection object in which the abnormal sound was picked up. The item "inspection object" may be, for example, identification information that can identify the inspection object, the device name of the inspection object, the model name, etc.

[0097] The item "microphone ID" indicates information related to the sound collection device 30 (microphone) that collected the abnormal sound. The item "microphone ID" may be, for example, identification information that can identify the sound collection device 30, or information such as the device name or model name of the sound collection device 30. Note that the item "microphone ID" may also be information such as identification information, the device name, or model name of the abnormal sound collecting device 10 that is equipped with the sound collection device 30.

[0098] The item "sound pickup date and time" indicates information on the sound pickup date and time when the abnormal sound was picked up.

[0099] The item "Installation state" indicates the installation state of the sound collection device 30 (microphone) or the abnormal sound collecting device 10 equipped with the sound collection device 30 when the abnormal sound was collected. For example, "wall" in the item "Installation state" indicates that the abnormal sound was collected with the abnormal sound collecting device 10 installed on a wall. "Ground" in the item "Installation state" indicates that the abnormal sound data was collected with the abnormal sound collecting device 10 installed on the ground. "Hanging" in the item "Installation state" indicates that the abnormal sound data was collected with the abnormal sound collecting device 10 suspended from a ceiling, wall, etc.

[0100] The "operating mode" item indicates information about the operating state of the object under test. For example, the operating mode of an air conditioner is such that an inrush current occurs at startup, the operating state reaches its maximum when cooling or heating to the set temperature, and the operating state reaches its minimum after cooling or heating to the set temperature. Here, the operating mode may be acquired by manual input by the operator hm. Furthermore, if the object under test is an IoT (Internet of Things)-applied device and is connected to the abnormal sound collecting device 10 or the cloud server 50 so as to be able to communicate data with it, the operating mode may be acquired by referring to the operating mode of the object under test at the time and date of sound collection.

[0101] The item "abnormal sound data" indicates signal waveform data of collected sound data that has been determined to be an abnormal sound. Note that the item "abnormal sound data" shown in Fig. 8 shows an example of only the signal waveform data of collected sound data, but it may also be frequency waveform data of collected sound data that has been determined to be an abnormal sound, or the signal waveform data and frequency waveform data of collected sound data that has been determined to be an abnormal sound may be shown in association with each other.

[0102] The abnormal sound data table TB1 shown in FIG. 8 stores four abnormal sound data SW1 to SW4.

[0103] The abnormal sound data SW1 is abnormal sound data obtained by collecting the operating sound of the inspection object "001" in the "minimum" operating mode on the sound collection date and time "2021 / 03 / 15 17:00" by an abnormal sound collection device equipped with a microphone ID "001" installed on the "wall."

[0104] The abnormal sound data SW2 is abnormal sound data of the operating sound of the inspection object "001" in the "maximum" operating mode state, collected on the collection date and time "2021 / 03 / 14 13:00" by an abnormal sound collection device equipped with a microphone ID "001" installed on the "ground."

[0105] The abnormal sound data SW3 is abnormal sound data of the operating sound of the inspection object "001" in the "medium" operating mode state, collected on the collection date and time "2021 / 03 / 11 14:00" by an abnormal sound collection device equipped with a microphone ID "003" installed "suspended in mid-air."

[0106] The abnormal sound data SW4 is abnormal sound data of the operating sound of the inspection object "002" in the "minimum" operating mode state, collected on the collection date and time "2021 / 03 / 10 12:00" by an abnormal sound collection device equipped with a microphone ID "001" installed on the "wall."

[0107] As described above, the abnormal sound collecting device 10 (an example of a sound collecting device) according to the first embodiment includes a communication unit 11 (an example of an acquisition unit) that acquires abnormal sound data (an example of sound data) stored in the cloud server 50 (an example of a server), a sound processing unit 14 (an example of a sound acquisition unit) that acquires collected sound data obtained by collecting operating sounds of an inspection target (an example of an object), a processor 13 (an example of a calculation unit) that calculates a distance (an example of a first score, for example, distances ASD1 to ASD3 shown in FIG. 5 ) indicating a difference in feature amount between the acquired collected sound data and the abnormal sound data, and a processor 13 (an example of a determination unit) that determines whether or not the collected sound data needs to be stored in the cloud server 50 based on the distance (for example, distances ASD1 to ASD3). If the processor 13 determines that the distance (for example, distances ASD1 to ASD3 shown in FIG. 5 ) is equal to or greater than a second threshold value (an example of a first predetermined value), the processor 13 outputs the collected sound data to the cloud server 50 for storage.

[0108] As a result, the abnormal sound collecting device 10 according to embodiment 1 can store in the cloud server 50 only collected sound data that is determined to have a distance between the collected sound data determined to be an abnormal sound and the abnormal sound data already stored in the cloud server 50 that is equal to or greater than the second threshold (in other words, the sound feature quantity between the stored abnormal sound data differs by equal to or greater than the predetermined value indicated by the second threshold). Therefore, by collecting (storing) abnormal sound data that is not similar to data for machine learning that has already been collected (stored), the abnormal sound collecting device 10 can effectively reduce the bias in the abnormal sound data used for machine learning and more efficiently collect abnormal sound data suitable for machine learning. This also enables the cloud server 50 to prevent a shortage of storage capacity caused by collecting the same abnormal sound data.

[0109] Furthermore, as described above, the processor 13 in the abnormal sound collecting device 10 according to embodiment 1 selects a minimum abnormal sound distance (one example of the minimum score, for example, distance ASD1 shown in FIG. 5 ) that results in the smallest difference among the distances (for example, distances ASD1 to ASD3), and if it determines that the selected minimum abnormal sound distance is equal to or greater than the second threshold, outputs the collected sound data to the cloud server 50 for storage. This allows the abnormal sound collecting device 10 according to embodiment 1 to collect (store) abnormal sound data that is not similar to data for machine learning that has already been collected (stored), effectively suppressing bias in the abnormal sound data used for machine learning and more efficiently collecting abnormal sound data suitable for machine learning.

[0110] Furthermore, as described above, when the processor 13 in the abnormal sound collecting device 10 according to embodiment 1 determines that the distance (for example, distances ASD1 to ASD3) is not equal to or greater than the second threshold, it omits storing the collected sound data in the cloud server 50. This allows the abnormal sound collecting device 10 according to embodiment 1 to omit storing in the cloud server 50 abnormal sound data that is the same as or similar to data for machine learning that has already been collected (stored), and more effectively prevents the cloud server 50 from running out of storage capacity.

[0111] Furthermore, as described above, when the processor 13 in the abnormal sound collecting device 10 according to embodiment 1 determines that the distance (for example, distances ASD1 to ASD3) is equal to or greater than the second threshold, it further determines whether the signal-to-noise ratio of the collected sound data is equal to or greater than a predetermined value (an example of a second predetermined value). When the processor 13 determines that the signal-to-noise ratio is equal to or greater than the predetermined value, it outputs the collected sound data to the cloud server 50 for storage. In this way, the abnormal sound collecting device 10 according to embodiment 1 can collect only abnormal sound data that has less noise and is suitable for machine learning as abnormal sound data to be used for machine learning.

[0112] Furthermore, as described above, when the processor 13 in the abnormal sound collecting device 10 according to the first embodiment determines that the S / N ratio is not equal to or greater than a predetermined value, it omits storing the collected sound data in the cloud server 50. This allows the abnormal sound collecting device 10 according to the first embodiment to more effectively suppress the collection of abnormal sound data that is noisy and unsuitable for machine learning as abnormal sound data to be used for machine learning.

[0113] Furthermore, as described above, the communication unit 11 in the abnormal sound collecting device 10 according to embodiment 1 acquires normal sound data (an example of sound data) stored in the cloud server 50. The processor 13 calculates a distance D0 (see FIG. 4 , an example of a second score) indicating the difference in feature amount between the acquired collected sound data and the normal sound data, and if it determines that the distance D0 is equal to or greater than a first threshold value (an example of a third predetermined value), it determines that the collected sound data is abnormal sound data. This allows the abnormal sound collecting device 10 according to embodiment 1 to determine whether the collected sound data is abnormal sound data based on the distance D0 from the normal sound data.

[0114] Furthermore, as described above, when the processor 13 in the abnormal sound collecting device 10 according to the first embodiment determines that the distance D0 is not equal to or greater than the first threshold, it determines that the collected sound data is not abnormal sound data stored in the cloud server 50. This allows the abnormal sound collecting device 10 according to the first embodiment to determine whether or not the collected sound data is abnormal sound data based on the distance D0 between the normal sound data.

[0115] As described above, in the abnormal sound collecting device 10 according to embodiment 1, distances (for example, distances ASD1 to ASD3) are calculated using Euclidean distance, Mahalanobis distance, or an autoencoder. This enables the abnormal sound collecting device 10 to more effectively collect (store) abnormal sound data that is dissimilar to data for machine learning that has already been collected (stored).

[0116] As described above, in the abnormal sound collecting device 10 according to embodiment 1, the distance D0 is calculated using the Euclidean distance, the Mahalanobis distance, or an autoencoder. This allows the abnormal sound collecting device 10 to more effectively collect (store) whether the collected sound data is a normal sound or not.

[0117] As described above, the abnormal sound collecting device 10 according to the first embodiment further includes an input processing unit 15 (an example of an input unit) that can accept the conditions of the inspection object from the operator hm. The communication unit 11 acquires abnormal sound data that matches the conditions of the inspection object accepted by the input processing unit 15.

[0118] Furthermore, as described above, when the processor 13 in the abnormal sound collecting device 10 according to the first embodiment determines that the distance (for example, the distances ASD1 to ASD3) is not equal to or greater than the second threshold value, the processor 13 generates and outputs a determination result screen Sc3 (an example of a screen) that notifies the user that it is not necessary to store the collected sound data in the cloud server 50.

[0119] As described above, the abnormal sound collecting system 1000 (an example of a sound collecting device) according to the first embodiment is an abnormal sound collecting system 1000 including a cloud server 50 that stores one or more pieces of abnormal sound data, and an abnormal sound collecting device 10 (an example of a terminal device) that can communicate with the cloud server 50. The cloud server 50 transmits the stored abnormal sound data (an example of sound data) to the abnormal sound collecting device 10. The abnormal sound collecting device 10 acquires collected sound data in which operating sounds of an inspection object are collected, calculates a distance (e.g., distances ASD1 to ASD3) that indicates a difference in feature amount between the transmitted abnormal sound data and the collected sound data, determines whether the calculated distance (e.g., distances ASD1 to ASD3) is equal to or greater than a second threshold (an example of a first predetermined value), and transmits the collected sound data to the cloud server 50 if it determines that the distance (e.g., distances ASD1 to ASD3) is equal to or greater than the second threshold. The cloud server 50 records the transmitted collected sound data.

[0120] As a result, the abnormal sound collection system 1000 according to embodiment 1 can store in the cloud server 50 only collected sound data for which it is determined that the distance between the collected sound data determined to be an abnormal sound and the abnormal sound data already stored in the cloud server 50 is equal to or greater than the second threshold (in other words, the sound feature quantity between the stored abnormal sound data differs by equal to or greater than the predetermined value indicated by the second threshold). Therefore, by collecting (storing) abnormal sound data that is not similar to data for machine learning that has already been collected (stored), the abnormal sound collection system 1000 can effectively reduce the bias in the abnormal sound data used for machine learning and more efficiently collect abnormal sound data suitable for machine learning. Furthermore, as a result, the abnormal sound collection system 1000 can prevent the cloud server 50 from running out of storage capacity due to the collection of the same abnormal sound data.

[0121] As described above, the abnormal sound collecting device 10 (an example of a sound collecting device) according to the first embodiment is a sound collecting method performed by the abnormal sound collecting device 10 (an example of a terminal device) that can communicate with the cloud server 50 that stores one or more pieces of abnormal sound data (an example of sound data), and the method acquires collected sound data in which the operating sound of an object under inspection is collected, calculates a distance (an example of a first score, for example, distances ASD1 to ASD3) that indicates the difference in feature amount between the transmitted abnormal sound data and the collected sound data, determines whether the calculated distance (for example, distances ASD1 to ASD3) is equal to or greater than a second threshold value (an example of a first predetermined value), and if it is determined that the distance (for example, distances ASD1 to ASD3) is equal to or greater than the second threshold value, determines that the collected sound data is an abnormal sound (an example of a sound) stored in the cloud server 50.

[0122] As a result, when the abnormal sound collecting device 10 according to the first embodiment determines that the distance between the collected sound data determined to be an abnormal sound and the abnormal sound data already stored in the cloud server 50 is equal to or greater than the second threshold value (that is, the sound feature quantity between the collected sound data and the stored abnormal sound data differs by equal to or greater than the predetermined value indicated by the second threshold value), it can determine that the sound included in the collected sound data is an abnormal sound.

[0123] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention.

[0124] This application is based on a Japanese patent application (Patent Application No. 2021-153205) filed on September 21, 2021, the contents of which are incorporated by reference into this application.

[0125] The present disclosure is useful as a sound collection device, a sound collection system, and a sound collection method that efficiently collect abnormal sound data suitable for machine learning.

[0126] REFERENCE SIGNS LIST 10 Abnormal sound collecting device 11, 51 Communication unit 12 I / F 13, 52 Processor 14 Audio processing unit 15 Input processing unit 16 Display control unit 17 Normal sound learning unit 18 Abnormal sound determination unit 19 SN determination unit 20 Abnormal sound storage determination unit 21 Memory 30 Sound collecting device 31 Display device 32 Input device 50 Cloud server 53 Storage device 54 Normal sound storage unit 55 Abnormal sound storage unit 101, 102, 103 Inspection object 1000 Abnormal sound collecting system NW Network Sc1, Sc2, Sc3 Determination result screen

Claims

1. A first acquisition unit that acquires first data stored in a server; A second acquisition unit that acquires second data indicating the state of an object; A calculation unit that calculates a first score indicating a difference in feature amounts between the second data and the first data; A determination unit that determines whether or not it is necessary to store the second data in the server based on the first score, wherein when the determination unit determines that the first score is equal to or greater than a first predetermined value, the determination unit outputs the second data to the server for storage, A data acquisition device.

2. The first data is audio data, and the second data is sound collection data in which the operating sound of the object is collected, The data acquisition device according to Claim 1.

3. The determination unit selects a minimum score that is the minimum of the first scores, and when it is determined that the selected minimum score is equal to or greater than the first predetermined value, the determination unit outputs the sound collection data to the server for storage, The data acquisition device according to Claim 2.

4. When the determination unit determines that the first score is not equal to or greater than the first predetermined value, the determination unit omits storage of the sound collection data in the server, The data acquisition device according to Claim 2.

5. The determination unit when it is determined that the first score is equal to or greater than the first predetermined value, further determines whether or not the signal-to-noise ratio of the sound collection data is equal to or greater than a second predetermined value, and when it is determined that the signal-to-noise ratio is equal to or greater than the second predetermined value, the determination unit outputs the sound collection data to the server for storage, The data acquisition device according to Claim 2.

6. The determination unit when it is determined that the signal-to-noise ratio is not equal to or greater than the second predetermined value, omits storage of the sound collection data in the server, The data acquisition device according to Claim 5.

7. The acquisition unit acquires the audio data, the calculation unit calculates a second score indicating a difference in feature amounts between the acquired sound collection data and the audio data, and when the determination unit determines that the second score is equal to or greater than a third predetermined value, the determination unit determines that the sound collection data is abnormal sound, The data acquisition device according to Claim 2.

8. When the determination unit determines that the second score is not equal to or greater than the third predetermined value, the determination unit determines that the sound collection data is not the sound stored in the server, The data acquisition device according to Claim 7.

9. The first score is calculated using Euclidean distance, Mahalanobis distance, or an autoencoder. The data acquisition device according to claim 2.

10. The second score is calculated using Euclidean distance, Mahalanobis distance, or an autoencoder. The data acquisition device according to claim 7.

11. Further comprising an input unit capable of receiving the conditions of the object by an operator, The acquisition unit acquires the sound data that conforms to the conditions of the object received by the input unit. The data acquisition device according to claim 2.

12. When the determination unit determines that the first score is not equal to or greater than a first predetermined value, the determination unit generates and outputs a screen notifying that it is unnecessary to store the sound collection data in the server. The data acquisition device according to claim 2.

13. A system including a server that stores first data and a terminal device capable of communicating with the server, The server transmits the stored first data to the terminal device, The terminal device acquires second data indicating the state of the object, calculates a first score indicating the difference in feature amounts between the transmitted first data and the second data, determines whether or not the calculated first score is equal to or greater than a first predetermined value, when it is determined that the first score is equal to or greater than the first predetermined value, transmits the second data to the server, The server records the transmitted second data. Data acquisition system.

14. A data acquisition method performed by a terminal device capable of communicating with a server that stores first data, acquiring second data indicating the state of the object, acquiring the first data stored in the server, calculating a first score indicating the difference in feature amounts between the second data and the first data, determining whether or not the calculated first score is equal to or greater than a first predetermined value, when it is determined that the first score is equal to or greater than the first predetermined value, transmitting the second data to the server to record it. Data acquisition method.

15. A data acquisition method performed by a terminal device capable of communicating with a server that stores first data, acquiring second data indicating the state of the object, acquiring the first data stored in the server, calculating a first score indicating the difference in feature amounts between the second data and the first data, determining whether or not the calculated first score is equal to or greater than a first predetermined value, ​ When it is determined that the first score is equal to or greater than a first predetermined value, it is determined that the second data is data stored in the server. Data acquisition method.