Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program that enable an abnormal sound in a vehicle to be effectively detected.SOLUTION: According to an embodiment, an information processing device includes a microphone interface, a memory, and a processor. The microphone interface connects to a microphone that collects sounds in a vehicle. The memory stores peripheral information indicating a state that influences sounds and an acoustic model for detecting an abnormal sound in association with each other. The processor generates peripheral information related to the vehicle, acquiring, from the memory, an acoustic model corresponding to the generated peripheral information, and using the acquired acoustic model, detecting an abnormal sound from sound data indicating sounds collected by the microphone.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A system for detecting abnormal sounds, such as screams, is provided. Such a system collects sounds in a predetermined area and detects abnormal sounds from the collected sounds using a model for detecting abnormal sounds.

[0003] In addition, there may be cases where an operator such as a conductor is not present on the vehicle due to automatic driving, etc.

[0004] Therefore, there is a demand for technology that can effectively detect abnormal sounds inside a vehicle. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-182140 Summary of the Invention [Problem to be solved by the invention]

[0006] In order to solve the above problem, an information processing device, an information processing method, and a program are provided that can effectively detect abnormal sounds inside a vehicle. [Means for solving the problem]

[0007] According to an embodiment, an information processing device includes a microphone interface, a memory, and a processor. The microphone interface is connected to a microphone that collects sounds inside a vehicle. The memory stores peripheral information indicating conditions that affect the sound and an acoustic model for detecting abnormal sounds in association with each other. The processor generates peripheral information related to the vehicle and generates an acoustic model corresponding to the generated peripheral information. The aforementionedAn acoustic model is acquired from the memory, and the acquired acoustic model is used to detect abnormal sounds from sound data representing sounds collected by the microphone. The background sound in the state indicated by the surrounding information is included. The processor includes a noise removal model for removing noise. When the processor determines that the sound data includes an abnormal sound, the processor performs the following steps: The acquired acoustic model includes Remove noise from the sound data using the noise removal model. and extract abnormal sounds do. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram of a vehicle system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing a control system of the vehicle according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of an acoustic model database according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the operation of the vehicle according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of the operation of the vehicle according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described with reference to the drawings. (First embodiment) The vehicle provided in the vehicle system according to the embodiment constitutes a train that travels on a railroad track. The vehicle collects internal sounds. The vehicle detects abnormal sounds from the collected sounds. When an abnormal sound is detected, the vehicle notifies an operator such as a driver that an abnormal sound has been detected. Furthermore, in the case of autonomous driving, the vehicle stops or notifies a higher-level device that an abnormal sound has been detected. For example, abnormal sounds include screams, shouts, gunfire, collision sounds, explosions, popping sounds, and the sound of glass breaking. The content of the abnormal sound is not limited to a specific configuration.

[0010] Fig. 1 is a schematic diagram of a vehicle system 100 according to an embodiment. As shown in Fig. 1, the vehicle system 100 includes a vehicle 10 and a host device 40. The vehicle 10 and the host device 40 are connected to each other so that they can communicate with each other.

[0011] The vehicle 10 constitutes a train that runs on a railway line R. The vehicle 10 runs on the railway line R either by operation of a driver or automatically.

[0012] The vehicle 10 includes a housing 1, a camera 2, an antenna 3, a microphone 4, and the like.

[0013] The housing 1 forms the outer shape of the vehicle 10. For example, the housing 1 is formed so that it can accommodate people, luggage, etc. inside.

[0014] Camera 2 is installed in housing 1. For example, camera 2 is installed facing downward on the top of housing 1. Camera 2 captures images of the inside of housing 1 (i.e., the inside of vehicle 10). For example, camera 2 captures images of people staying in housing 1.

[0015] For example, the camera 2 is a CCD (Charge Coupled Device) camera. The camera 2 may also be equipped with a light that illuminates the inside of the housing 1. Furthermore, the camera 2 may be made up of multiple cameras.

[0016] An antenna 3 is also installed in the housing 1. The antenna 3 is an antenna for receiving signals for positioning the vehicle 10. For example, the antenna 3 is an antenna for receiving signals of the Global Navigation Satellite System (GNSS).

[0017] Furthermore, a microphone 4 is installed in the housing 1. For example, the microphone 4 is installed inside the housing 1. The microphone 4 collects sounds inside the housing 1 (i.e., inside the vehicle 10). Furthermore, the microphone 4 may be configured from a plurality of microphones.

[0018] The host device 40 manages the operation of the vehicle 10. The host device 40 provides the vehicle 10 with various information related to the operation of the vehicle 10. The host device 40 also acquires various information related to the operation from the vehicle 10. For example, the host device 40 is a server managed by a railway company.

[0019] Next, the vehicle 10 will be described. Fig. 2 shows an example of the configuration of a vehicle 10 according to an embodiment. Fig. 2 is a block diagram showing an example of the configuration of the vehicle 10. As shown in Fig. 2, the vehicle 10 includes a camera 2, an antenna 3, a microphone 4, an information processing device 5, and the like.

[0020] The information processing device 5 detects abnormal sounds from sounds inside the vehicle 10. The information processing device 5 includes a processor 11, a ROM 12, a RAM 13, an NVM 14, a communication unit 15, an operation unit 16, a display unit 17, a camera interface 18, an antenna interface 19, a microphone interface 20, and the like.

[0021] The processor 11, ROM 12, RAM 13, NVM 14, communication unit 15, operation unit 16, display unit 17, camera interface 18, antenna interface 19, and microphone interface 20 are connected to one another via a data bus or the like. The camera interface 18 is connected to a camera 2. The antenna interface 19 is connected to an antenna 3. The microphone interface 20 is connected to a microphone 4.

[0022] The vehicle 10 and the information processing device 5 may be provided with components as needed in addition to the components shown in FIG. 2, or specific components may be excluded from the vehicle 10.

[0023] The processor 11 has the function of controlling the overall operation of the vehicle 10. The processor 11 may include an internal cache and various interfaces. The processor 11 performs various processes by executing programs stored in advance in the internal memory, the ROM 12, or the NVM 14.

[0024] Some of the various functions realized by the processor 11 executing the programs may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0025] The ROM 12 is a non-volatile memory that stores in advance control programs, control data, etc. The control programs and control data stored in the ROM 12 are installed in advance in accordance with the specifications of the vehicle 10.

[0026] RAM 13 is a volatile memory. RAM 13 temporarily stores data being processed by processor 11. RAM 13 stores various application programs based on instructions from processor 11. RAM 13 may also store data necessary for executing application programs and execution results of application programs.

[0027] The NVM 14 is a nonvolatile memory to which data can be written and rewritten. The NVM 14 is configured, for example, with a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The NVM 14 stores control programs, applications, and various data according to the operational use of the vehicle 10.

[0028] The NVM 14 also stores an acoustic model database containing a plurality of acoustic models, which will be described later.

[0029] The communication unit 15 is an interface for transmitting and receiving data to and from the higher-level device 40, etc. For example, the communication unit 15 is connected to the higher-level device 40, etc. via a network. For example, the communication unit 15 is an interface that supports a wired or wireless LAN (Local Area Network) connection.

[0030] The operation unit 16 receives input of various operations from an operator (for example, a driver, etc.). The operation unit 16 transmits a signal indicating the input operation to the processor 11. The operation unit 16 may be configured as a touch panel.

[0031] The display unit 17 displays image data from the processor 11. For example, the display unit 17 is configured with a liquid crystal monitor. When the operation unit 16 is configured with a touch panel, the display unit 17 may be formed integrally with the operation unit 16.

[0032] The camera interface 18 is an interface that connects to the camera 2. The camera interface 18 transmits a signal from the processor 11 to the camera 2. The camera interface 18 also acquires a signal (such as a captured image) from the camera 2 and transmits it to the processor 11.

[0033] The antenna interface 19 is an interface that connects to the antenna 3. The antenna interface 19 measures the current position of the vehicle 10 based on a signal from the antenna 3 or the like.

[0034] For example, the antenna interface 19 measures the position based on a signal from the antenna 3. After measuring the position, the antenna interface 19 fits the measured position onto a nearby railroad track R based on map information including the position of the railroad track R. The antenna interface 19 obtains the fitted position as the current position of the vehicle 10. The antenna interface 19 transmits the obtained current position to the processor 11.

[0035] The microphone interface 20 is an interface that connects to the microphone 4. For example, the microphone interface 20 amplifies a signal that indicates a sound measured by the microphone 4 and converts it into a digital signal. The microphone interface 20 transmits the converted digital signal (sound data) to the processor 11.

[0036] The function of the antenna interface 19 may be realized by the processor 11. Furthermore, when the vehicle 10 travels by automatic driving, the information processing device 5 does not need to include the operation unit 16 and the display unit 17.

[0037] Next, the acoustic model database will be described. The acoustic model database is stored in advance in the NVM 14.

[0038] An example of the configuration of the acoustic model database is shown in Fig. 3. As shown in Fig. 3, the acoustic model database stores a plurality of acoustic models and a plurality of peripheral information in association with each other.

[0039] The surrounding information is information for selecting an acoustic model by the vehicle 10. That is, the surrounding information is information indicating conditions (for example, factors) that affect sounds inside the vehicle 10 (mainly background sounds).

[0040] Here, the surrounding area information is made up of the current position, current time, and congestion level of the vehicle 10. The congestion level is an index relating to the number of people accommodated in the cabinet 1 (for example, occupancy rate).

[0041] The surrounding information may include the contents of announcements made on the train, whether the train is accelerating, decelerating, stopping, or running at a constant speed, or the weather. The configuration of the peripheral information is not limited to a specific configuration.

[0042] Each acoustic model corresponds to a corresponding piece of peripheral information, i.e., the acoustic model is a model for detecting abnormal sounds in the state indicated by the corresponding peripheral information.

[0043] As shown in FIG. 3, the acoustic model database includes a plurality of acoustic models 30 (30a, 30b, . . . ).

[0044] Here, the acoustic model 30a will be described. The other acoustic models 30 have the same configuration as the acoustic model 30a, so their description will be omitted.

[0045] The acoustic model 30a is composed of an anomaly detection model 31a, a noise removal model 32a, and the like.

[0046] The abnormality detection model 31a is composed of an abnormal sound model 311a, a background sound model 312a, and the like.

[0047] The abnormal sound model 311a is a model of an abnormal sound in a state indicated by the peripheral information. For example, the abnormal sound model 311a is a model of sound data in which an abnormal sound is superimposed on background sound in a state indicated by the peripheral information.

[0048] The abnormal sound model 311a indicates the feature amount of the abnormal sound in the state indicated by the peripheral information. For example, the abnormal sound model 311a indicates the distribution of the feature amount of the abnormal sound (for example, GMM (Gaussian Mixture Model)).

[0049] The background sound model 312a is a model of the background sound in the state indicated by the peripheral information. The background sound model 312a indicates the feature amount of the background sound in the state indicated by the peripheral information. For example, the background sound model 312a indicates the distribution (for example, GMM) of the feature amount of the background sound.

[0050] The noise removal model 32a is a model for removing background sounds from sound data including abnormal sounds. For example, the noise removal model 32a is generated based on the difference between the background sounds in the state indicated by the peripheral information and the clean sounds. For example, the noise removal model 32a is related to the frequencies of the sounds that make up the background sounds.

[0051] The acoustic model database may be updated as needed. For example, new acoustic models may be added to the acoustic model database, and existing acoustic models may be deleted. The contents of the acoustic models may also be updated as needed.

[0052] Next, a description will be given of functions realized by the vehicle 10. The functions realized by the vehicle 10 are realized by the processor 11 executing a program stored in the internal memory, the ROM 12, the NVM 14, or the like.

[0053] First, the processor 11 has a function of acquiring sound data using the microphone 4 . For example, the processor 11 starts the microphone 4 in accordance with an operation input to the operation unit 16. The processor 11 may also start the microphone 4 in accordance with a control signal received from the higher-level device 40 via the communication unit 15.

[0054] When the microphone 4 is started, the processor 11 acquires, via the microphone interface 20, sound data indicative of the sound inside the housing 1 over a predetermined period of time. The processor 11 continuously acquires sound data using the microphone 4 .

[0055] The vehicle 10 may or may not be in operation.

[0056] The processor 11 also has a function of generating surrounding information indicating the current state of the vehicle 10 (surrounding information related to the vehicle 10).

[0057] The processor 11 acquires the current position of the vehicle 10 using the antenna 3 and the antenna interface 19. After acquiring the current position, the processor 11 acquires the current time.

[0058] After acquiring the current time, the processor 11 acquires the degree of congestion inside the vehicle 10. For example, the processor 11 uses the camera 2 to acquire a captured image of the interior of the housing 1 of the vehicle 10. After acquiring the captured image, the processor 11 identifies people from the captured image according to a predetermined image processing algorithm. After identifying the people, the processor 11 calculates the congestion level based on the number of people identified, etc.

[0059] When the congestion degree is acquired, the processor 11 generates surrounding information indicating the current position of the vehicle 10, the current time, and the congestion degree.

[0060] The processor 11 also has a function of selecting acoustic model candidates from the acoustic model database based on the generated peripheral information.

[0061] After generating the peripheral information, the processor 11 calculates the similarity between each piece of peripheral information in the acoustic model database and the generated peripheral information according to a predetermined algorithm. For example, the processor 11 calculates the similarity by comparing each element of the peripheral information. Here, the similarity is assumed to be larger as the similarity between the peripheral information becomes higher.

[0062] After calculating the similarity, the processor 11 selects candidates from the acoustic model database based on the similarity. For example, the processor 11 selects, as candidates, acoustic models corresponding to a predetermined number of peripheral information pieces with high similarity. Alternatively, the processor 11 may select, as candidates, acoustic models corresponding to peripheral information pieces with similarity greater than a predetermined threshold.

[0063] The processor 11 also has a function of selecting one acoustic model from the candidates based on the acquired sound data.

[0064] After selecting the candidates, the processor 11 matches the sound data with the background sound model of each candidate. For example, the processor 11 extracts features from the sound data according to a predetermined algorithm. After extracting the features from the sound data, the processor 11 calculates the similarity between the extracted features and each background sound model. Here, the similarity is assumed to be larger as the similarity increases.

[0065] After calculating the similarity between the sound data and each background sound model, the processor 11 identifies the background sound model with the highest similarity. After identifying the background sound model with the highest similarity, the processor 11 selects an acoustic model including the background sound model. Here, the highest similarity is defined as the first similarity.

[0066] The processor 11 also has a function of detecting abnormal sounds from sound data using the selected acoustic model.

[0067] When one acoustic model is selected from the candidates, the processor 11 matches the sound data with the abnormal sound model of the selected acoustic model. The processor 11 calculates a similarity (second similarity) by matching the sound data with the abnormal sound model. After calculating the second similarity, the processor 11 compares the first similarity with the second similarity.

[0068] If the second similarity is greater than the first similarity (that is, if the sound data is more similar to the abnormal sound model than to the background sound model), the processor 11 determines that the sound data contains an abnormal sound.

[0069] Furthermore, if the second similarity is equal to or less than the first similarity, the processor 11 determines that the sound data does not contain an abnormal sound.

[0070] Processor 11 may detect the abnormal sound further based on the captured image from camera 2. For example, processor 11 detects an abnormality (such as a gun) from the captured image according to a predetermined image processing algorithm. Processor 11 may detect the abnormal sound further based on the abnormality detected from the captured image.

[0071] The processor 11 also has a function of removing noise from the sound data. When it is determined that the sound data contains an abnormal sound, the processor 11 acquires a noise reduction model from the selected acoustic model. After acquiring the noise reduction model, the processor 11 uses the acquired noise reduction model to remove noise (such as background sound) from the sound data. That is, the processor 11 uses the acquired noise reduction model to extract the abnormal sound from the sound data.

[0072] When an abnormal sound is extracted from the sound data, the processor 11 outputs the extracted abnormal sound. For example, the processor 11 transmits the sound data of the extracted abnormal sound to the higher-level device 40 via the communication unit 15 or the like.

[0073] The processor 11 also has a function of outputting the abnormal sound detection result. For example, the processor 11 displays the abnormal sound detection result on the display unit 17. The processor 11 may also transmit the abnormal sound detection result to the higher-level device 40 via the communication unit 15.

[0074] Next, an example of the operation of the vehicle 10 will be described. FIG. 4 is a flowchart for explaining an example of the operation of the vehicle 10.

[0075] The processor 11 of the vehicle 10 acquires sound data using the microphone 4 (S11). After acquiring the sound data, the processor 11 acquires the current position of the vehicle 10 using the antenna 3 and the antenna interface 19 (S12).

[0076] After acquiring the current location of the vehicle 10, the processor 11 acquires the current time (S13). After acquiring the current time, the processor 11 acquires the congestion level (S14). After acquiring the congestion level, the processor 11 generates surrounding information indicating the current location of the vehicle 10, the current time, and the congestion level (S15).

[0077] After generating the peripheral information, the processor 11 selects acoustic model candidates from the acoustic model database based on the generated peripheral information (S16). After selecting the candidates, the processor 11 selects one acoustic model from the candidates based on the acquired sound data (S17).

[0078] When one acoustic model is selected, the processor 11 matches the abnormal sound model of the selected acoustic model with the sound data (S18). When the abnormal sound model is matched with the sound data, the processor 11 determines whether the sound data includes an abnormal sound (S19).

[0079] If it is determined that the sound data contains an abnormal sound (YES in S19), the processor 11 extracts the abnormal sound from the sound data using the noise removal model of the selected acoustic model (S20).

[0080] When it is determined that the sound data does not contain an abnormal sound (S19, NO), or when an abnormal sound is extracted from the sound data (S20), the processor 11 outputs the detection result of the abnormal sound (S21). After outputting the abnormal sound detection result, the processor 11 ends its operation.

[0081] The processor 11 may repeatedly execute steps S11 to S21.

[0082] Furthermore, the anomaly detection model does not have to include an abnormal sound model. In this case, the processor 11 determines that the sound data contains an abnormal sound when the similarity between the sound data and the background sound model is equal to or less than a predetermined threshold.

[0083] The abnormality detection model may also include multiple abnormal sound models. For example, the abnormality detection model includes an abnormal sound model for each type of abnormal sound (scream, gunshot, etc.). In this case, the processor 11 calculates the similarity between the sound data and each abnormal sound model. The processor 11 determines that the sound data includes an abnormal sound corresponding to the abnormal sound model with the highest similarity.

[0084] Furthermore, when the first similarity is equal to or less than a predetermined threshold, the processor 11 may display that fact on the display unit 17. Furthermore, the processor 11 may notify the higher-level device 40 of that fact via the communication unit 15.

[0085] Furthermore, when the first similarity is equal to or less than a predetermined threshold, the processor 11 may select an acoustic model based on an acoustic model selected for detecting an abnormal sound during a predetermined period in the past. For example, the processor 11 may select the acoustic model that was used most frequently during the predetermined period in the past. Furthermore, the processor 11 may select an acoustic model by weighting the acoustic model used during the predetermined period in the past (for example, by weighting the most recent acoustic model).

[0086] A vehicle configured as described above selects candidate acoustic models based on surrounding information. The vehicle selects one acoustic model from the candidates based on sound data. The vehicle detects abnormal sounds from the sound data based on the selected acoustic model. As a result, the vehicle can detect abnormal sounds based on an acoustic model that is suited to the state of the vehicle. Therefore, the vehicle can effectively detect abnormal sounds from sounds inside the vehicle.

[0087] (Second embodiment) Next, a second embodiment will be described. The vehicle system according to the second embodiment differs from that according to the first embodiment in that a background sound model is generated, so the other components are denoted by the same reference numerals and detailed description thereof will be omitted.

[0088] The configuration of the vehicle system 100 according to the second embodiment is similar to that according to the first embodiment, and therefore a description thereof will be omitted.

[0089] Next, a description will be given of functions realized by the vehicle 10. The functions realized by the vehicle 10 are realized by the processor 11 executing a program stored in the internal memory, the ROM 12, the NVM 14, or the like. The vehicle 10 according to the second embodiment realizes the following functions in addition to the functions according to the first embodiment.

[0090] First, the processor 11 has a function of using the microphone 4 to acquire sound data indicating background sounds.

[0091] For example, the processor 11 starts the microphone 4 in accordance with an operation input to the operation unit 16. The processor 11 may also start the microphone 4 in accordance with a control signal received from the higher-level device 40 via the communication unit 15.

[0092] Here, it is assumed that no abnormal sound is occurring inside the vehicle 10. In other words, it is assumed that background sound is occurring inside the vehicle 10.

[0093] When the microphone 4 is started, the processor 11 acquires, via the microphone interface 20, sound data indicative of background sounds over a predetermined period of time. The processor 11 has a function of generating a background sound model from sound data.

[0094] Here, it is assumed that the processor 11 generates peripheral information indicating the state in which the sound data was acquired.

[0095] For example, the processor 11 extracts features from the sound data in accordance with a predetermined algorithm. After extracting the features from the sound data, the processor 11 generates a background sound model based on the distribution of the extracted features.

[0096] After generating the background sound model, the processor 11 stores the generated background sound model in the acoustic model database in association with the peripheral information.

[0097] The processor 11 may generate a noise reduction model based on sound data. In this case, the processor 11 stores the generated noise reduction model in the acoustic model database in association with surrounding information.

[0098] Next, an example of the operation of the vehicle 10 to generate a background sound model will be described. FIG. 5 is a flowchart illustrating an example of an operation in which the vehicle 10 generates a background sound model.

[0099] The processor 11 of the vehicle 10 acquires sound data using the microphone 4 (S31). After acquiring the sound data, the processor 11 acquires the current position of the vehicle 10 using the antenna 3 and the antenna interface 19 (S32).

[0100] After acquiring the current location of the vehicle 10, the processor 11 acquires the current time (S33). After acquiring the current time, the processor 11 acquires the congestion level (S34). After acquiring the congestion level, the processor 11 generates surrounding information indicating the current location of the vehicle 10, the current time, and the congestion level (S35).

[0101] After generating the ambient information, the processor 11 generates a background sound model based on the acquired sound data (S36). After generating the background sound model, the processor 11 associates the generated background sound model with the generated ambient information and stores them in the acoustic model database (S37).

[0102] When the generated background sound model and the generated peripheral information are stored in association with each other in the acoustic model database, the processor 11 ends its operation.

[0103] The processor 11 may generate a background sound model in the same manner for a plurality of states. That is, the processor 11 may execute S31 to S37 each time the ambient information changes.

[0104] The processor 11 may also acquire sound data including abnormal sounds generated by an operator or the like, and generate an abnormal sound model.

[0105] The vehicle configured as described above generates ambient information. The vehicle also generates a background sound model from sound data. The vehicle stores the generated ambient information and the generated background sound model in association with each other. As a result, the vehicle can generate a background sound model corresponding to the ambient information.

[0106] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. (Appendix 1) a microphone interface for connecting to a microphone for collecting sounds inside the vehicle; a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other; generating surrounding information relating to the vehicle; retrieving an acoustic model corresponding to the generated peripheral information from the memory; using the acquired acoustic model to detect abnormal sounds from sound data representing sounds collected by the microphone; a processor; An information processing device comprising: (Appendix 2) The processor: Selecting candidate acoustic models based on the generated peripheral information; selecting an acoustic model from the candidates based on the sound data; Detecting abnormal sounds from the sound data using the selected acoustic model. 2. The information processing device according to claim 1. (Appendix 3) the acoustic model includes a background sound model related to characteristics of background sound; the processor matches the sound data with a background sound model included in each of the candidate acoustic models to select the acoustic model from the candidates. 3. The information processing device according to claim 2. (Appendix 4) the acoustic model includes an abnormal sound model related to a feature of the abnormal sound; the processor detects an abnormal sound from the sound data using the abnormal sound model. 4. An information processing device according to any one of appendices 1 to 3. (Appendix 5) the acoustic model includes a noise reduction model for reducing noise; When the processor determines that the sound data includes an abnormal sound, it removes noise from the sound data using the noise removal model. 5. An information processing device according to any one of appendices 1 to 4. (Appendix 6) an antenna interface connected to an antenna for receiving a signal for positioning the vehicle; The processor generates the surrounding information including a current location of the vehicle. 6. An information processing device according to any one of appendices 1 to 5. (Appendix 7) a camera interface connected to a camera that captures images of the interior of the vehicle; The processor: Calculating the degree of congestion of the vehicle from the image captured by the camera; generating the surrounding information including the congestion degree; 7. An information processing device according to any one of appendices 1 to 6. (Appendix 8) The processor generates the surrounding information including a current time. 8. An information processing device according to any one of appendices 1 to 7. (Appendix 9) The processor: Acquire sound data indicating background sound using the microphone; generating a background sound model based on the sound data; 9. An information processing device according to any one of appendices 1 to 8. (Appendix 10) 1. An information processing method executed by a processor, comprising: Uses microphones to collect sounds from inside the vehicle, generating surrounding information relating to the vehicle; Acquire an acoustic model corresponding to the generated peripheral information from a memory that stores peripheral information indicating conditions that affect the sound and an acoustic model for detecting abnormal sounds in association with each other; using the acquired acoustic model to detect abnormal sounds from sound data representing sounds collected by the microphone; Information processing methods. (Appendix 11) A program executed by a processor, the processor, A function to collect sounds inside the vehicle using a microphone; generating surrounding information related to the vehicle; a function of acquiring an acoustic model corresponding to the generated peripheral information from a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other; a function of detecting abnormal sounds from sound data indicating sounds collected by the microphone using the acquired acoustic model; A program to make this happen. [Explanation of symbols]

[0107] 1...housing, 2...camera, 3...antenna, 4...microphone, 5...information processing device, 10...vehicle, 11...processor, 12...ROM, 13...RAM, 14...NVM, 15...communication unit, 16...operation unit, 17...display unit, 18...camera interface, 19...antenna interface, 20...microphone interface, 30...acoustic model, 30a...acoustic model, 31a...anomaly detection model, 32a...noise reduction model, 40...higher-level device, 100...vehicle system, 311a...abnormal sound model, 312a...background sound model

Claims

1. a microphone interface for connecting to a microphone for collecting sounds inside the vehicle; a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other; generating surrounding information relating to the vehicle; retrieving the acoustic model corresponding to the generated peripheral information from the memory; using the acquired acoustic model to detect abnormal sounds from sound data representing sounds collected by the microphone; a processor, the acoustic model includes a noise reduction model for removing noise including background sound in the state indicated by the peripheral information, When the processor determines that the sound data includes an abnormal sound, the processor removes noise from the sound data using the noise removal model included in the acquired acoustic model to extract the abnormal sound.

2. The processor: Selecting candidate acoustic models based on the generated peripheral information; selecting the acoustic model from the candidates based on the sound data; Detecting abnormal sounds from the sound data using the selected acoustic model. The information processing device according to claim 1 .

3. the acoustic model includes a background sound model related to characteristics of background sound; the processor matches the sound data with a background sound model included in each of the candidate acoustic models to select the acoustic model from the candidates. The information processing device according to claim 2 .

4. the acoustic model includes an abnormal sound model related to a feature of the abnormal sound; the processor detects an abnormal sound from the sound data using the abnormal sound model. The information processing device according to claim 1 .

5. an antenna interface connected to an antenna for receiving a signal for positioning the vehicle; The processor generates the surrounding information including a current location of the vehicle. The information processing device according to claim 1 .

6. a camera interface connected to a camera that captures images of the interior of the vehicle; The processor: Calculating the degree of congestion of the vehicle from the image captured by the camera; generating the surrounding information including the congestion degree; The information processing device according to claim 1 .

7. The processor generates the surrounding information including a current time. The information processing device according to claim 1 .

8. The processor: Acquire sound data indicating background sound using the microphone; generating a background sound model based on the sound data; The information processing device according to claim 1 .

9. 1. An information processing method executed by a processor, comprising: Uses microphones to collect sounds from inside the vehicle, generating surrounding information relating to the vehicle; acquiring, from a memory that stores peripheral information indicating conditions that affect the sound and an acoustic model for detecting abnormal sounds in association with each other, the acoustic model corresponding to the generated peripheral information; Using the acquired acoustic model, an abnormal sound is detected from sound data representing the sound collected by the microphone; removing noise from the sound data and extracting abnormal sounds using a noise removal model for removing noise including background sounds in the state indicated by the surrounding information, the noise removal model being included in the acquired acoustic model; Information processing methods.

10. A program executed by a processor, the processor, A function to collect sounds inside the vehicle using a microphone; generating surrounding information related to the vehicle; a function of acquiring, from a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other, the acoustic model corresponding to the generated peripheral information; a function of detecting abnormal sounds from sound data indicating sounds collected by the microphone using the acquired acoustic model; a function of removing noise from the sound data and extracting abnormal sounds using a noise removal model for removing noise, including background sounds, in the state indicated by the peripheral information, which is included in the acquired acoustic model; and A program to make this happen.

Images