Information processing method, information processing device, and program

Abstract

An information processing method according to one aspect of the present disclosure includes: detecting sound output from a first speaker provided to a mobile body (S10); and determining whether the sound output from the first speaker is a predetermined sound, on the basis of the results of detecting the sound output from the first speaker (S20).

Description

Information processing method, information processing device, and program

[0001] The present disclosure relates to an information processing method, an information processing device, and a program.

[0002] Conventionally, there have been devices that are mounted on vehicles such as self-driving cars and that output warning sounds. These devices can alert people located in the vehicle or around the vehicle by sounding a warning. For example, Patent Literature 1 discloses a control device for controlling a self-driving car that is equipped with a device that outputs a warning sound. The control device disclosed in Patent Literature 1 performs control to prohibit the self-driving car from switching to a driving mode when an abnormality occurs in the warning sound output device.

[0003] JP 2023-30329 A

[0004] For example, suppose a device such as a speaker that outputs a warning sound breaks down. In such a case, the warning sound may not be output, and it may not be possible to alert pedestrians or other targets. Even if a sound is output from the speaker, it may not be possible to sufficiently alert targets due to the influence of noise, etc. In this way, it is possible that the speaker is not outputting an appropriate sound.

[0005] The present disclosure provides an information processing method and the like that can determine whether an appropriate sound is being output from a speaker.

[0006] An information processing method according to one aspect of the present disclosure detects a sound output from a first speaker equipped on a moving body, and determines whether the sound output from the first speaker is a predetermined sound based on the detection result of the sound output from the first speaker.

[0007] An information processing device according to one aspect of the present disclosure includes a detection unit that detects sound output from a first speaker provided on a moving body, and a determination unit that determines whether the sound output from the first speaker is a predetermined sound based on the detection result of the sound output from the first speaker.

[0008] A program according to one aspect of the present disclosure is a program that causes a computer to execute the information processing method.

[0009] According to the present disclosure, it is possible to provide an information processing method that can determine whether an appropriate sound is being output from a speaker.

[0010] FIG. 1 is a block diagram showing a configuration of an anomaly detection system according to a first embodiment. FIG. 2 is a block diagram showing a configuration of a terminal according to the first embodiment. FIG. 3 is a diagram for explaining anomaly detection processing by a terminal according to the first embodiment. FIG. 4 is a diagram showing a specific example of a response executed by a terminal according to the first embodiment. FIG. 5 is a sequence diagram showing a processing procedure of an anomaly detection system according to the first embodiment. FIG. 6 is a flowchart showing a processing procedure of anomaly detection processing by a terminal according to the first embodiment. FIG. 7 is a block diagram showing a configuration of a terminal according to a second embodiment. FIG. 8 is a diagram showing a specific example of an image displayed on a display device included in a PC according to the second embodiment. FIG. 9 is a flowchart showing a processing procedure of a terminal according to the second embodiment. FIG. 10 is a block diagram showing a configuration of an anomaly detection system according to a first modification. FIG. 11 is a block diagram showing a configuration of an anomaly detection system according to the second modification. FIG. 12 is a flowchart showing a processing procedure of an information processing device according to an aspect of the present disclosure.

[0011] Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0012] Note that each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, the same reference numerals are used to denote substantially the same components, and duplicated explanations may be omitted or simplified.

[0013] First Embodiment [Configuration] FIG. 1 is a block diagram showing the configuration of an abnormality detection system 1 according to this embodiment.

[0014] The abnormality detection system 1 is a system that detects whether an appropriate sound is being output from a speaker 20 provided in the vehicle 10. For example, the abnormality detection system 1 detects whether an appropriate sound is being output from a speaker 20 provided in the vehicle 10, which outputs a sound based on a command (spoken command) output from a PC 300 operated by an operator. In the abnormality detection system 1, for example, the operator checks information from a sensor such as a camera provided in the vehicle 10 to remotely monitor and remotely operate the vehicle 10. For example, the operator operates operating devices such as a steering wheel and brakes (not shown) while checking an image displayed on a display device such as a display provided in the PC 300 (for example, the display device 301 shown in FIG. 8). The PC 300 transmits information related to the operation received by the operating device to the control device 200. As a result, the control device 200 controls the vehicle 10 based on the operator's operation.

[0015] Furthermore, the operator operates the PC 300 to transmit a speech command to the control device 200 that controls the vehicle 10 via the terminal 100. The control device 200, which has acquired the speech command, causes the speaker 20 to output a sound corresponding to the speech command. Here, the terminal 100 determines whether or not a sound corresponding to the speech command is being output from the speaker 20.

[0016] The sound output by the speaker 20 is not limited to a sound corresponding to a speech command. For example, the sound output by the speaker 20 may be a voice uttered by an operator or a speech sound automatically produced by the vehicle 10. In other words, the terminal 100 may determine whether the speech sound automatically produced by the vehicle 10 is being appropriately output from the speaker 20.

[0017] In the following description, the output of sound from a speaker is also referred to as "uttering."

[0018] The abnormality detection system 1 includes a vehicle 10 , a terminal 100 , a speaker 21 , a microphone 30 , and a PC 300 .

[0019] The vehicle 10 is a vehicle that is remotely monitored and / or remotely operated by an operator. The vehicle 10 may be an autonomous vehicle that operates automatically, a vehicle that is remotely operated by an operator, or a vehicle that is driven by a driver on board the vehicle 10.

[0020] The vehicle 10 is an example of a moving body.

[0021] Furthermore, the number of vehicles 10 equipped with the abnormality detection system 1 may be one or more.

[0022] The vehicle 10 includes a control device 200 and a speaker 20 .

[0023] The control device 200 is a computer that controls the operation of the vehicle 10. The control device 200 includes, for example, a communication interface for communicating with each device, a non-volatile memory in which a program is stored, a volatile memory that is a temporary storage area for executing the program, an input / output port for transmitting and receiving signals, and a processor for executing the program. The communication interface may be realized by a connector connected to a control line for wired communication with the terminal 100, or may be realized by a wireless communication circuit for wireless communication with the terminal 100.

[0024] For example, when the control device 200 acquires a speech command, it causes the speaker 20 to output a sound corresponding to the speech command.

[0025] The speaker 20 is a speaker that outputs sound. The speaker 20 is disposed in the vehicle 10, for example, and outputs sound to the surroundings of the vehicle 10. The speaker 20 is an example of a first speaker.

[0026] The terminal 100 is a computer that acquires speech commands output from the PC 300 and outputs (transfers) the commands to the control device 200. The terminal 100 includes, for example, a communication interface for communicating with each device, a non-volatile memory in which a program is stored, a volatile memory that is a temporary storage area for executing the program, an input / output port for transmitting and receiving signals, and a processor for executing the program. The communication interface is realized, for example, by a wireless communication circuit for wireless communication with the PC 300. In this embodiment, the terminal 100 is disposed in the vehicle 10. The terminal 100 may also include a communication interface realized, for example, by a connector connected to a control line for wired communication with the control device 200.

[0027] The terminal 100 may be disposed outside the vehicle 10. The terminal 100 may include a wireless communication circuit for wireless communication with the control device 200.

[0028] The terminal 100 is an example of an information processing device.

[0029] FIG. 2 is a block diagram showing the configuration of terminal 100 according to this embodiment.

[0030] The terminal 100 includes an utterance abnormality detection unit 110, an utterance abnormality response unit 120, and a storage unit 130.

[0031] The speech abnormality detection unit 110 is a processing unit that determines whether the sound output from the speaker 20 is a predetermined sound based on the detection result of the sound output from the speaker 20. Specifically, the speech abnormality detection unit 110 detects a sound that corresponds to a speech command transmitted from a PC 300 used by an operator located outside the vehicle 10 and that is output from the speaker 20 provided in the vehicle 10. For example, the speech abnormality detection unit 110 detects a sound corresponding to a speech command by acquiring information indicating a sound detected by the microphone 30. Furthermore, the speech abnormality detection unit 110 determines whether the sound output from the speaker 20 is a predetermined sound based on the detection result of the sound output from the speaker 20. In other words, the speech abnormality detection unit 110 determines whether the predetermined sound desired by the operator is output from the speaker 20.

[0032] The speech abnormality detection unit 110 is an example of a detection unit and a determination unit. The speech command is an example of a command.

[0033] For example, the speech abnormality detection unit 110 determines whether the sound output from the speaker 20 is a predetermined sound by comparing the waveform of the sound output from the speaker 20 with sound data indicating the waveform of the predetermined sound.

[0034] Fig. 3 is a diagram for explaining the abnormality detection process of the terminal 100 according to the present embodiment. Specifically, Fig. 3 is a graph showing a specific example of sound detected by the microphone 30. The horizontal axis of the graph shown in Fig. 3 represents time, and the vertical axis represents the volume of the sound detected by the microphone 30.

[0035] For example, sound data indicating a waveform of a sound corresponding to a speech command is stored in advance in the storage unit 130. The speech abnormality detection unit 110 determines whether or not the waveform of the sound output from the speaker 20 matches the waveform indicated by the sound data stored in the storage unit 130. If they match, it determines that the sound output from the speaker 20 is a predetermined sound, and if they do not match, it determines that the sound output from the speaker 20 is not a predetermined sound.

[0036] For example, the speech abnormality detection unit 110 compares the waveform of the sound continuously detected by the microphone 30 over a unit time with the waveform of the sound data stored in the storage unit 130. For example, the speech abnormality detection unit 110 compares the waveform of the unit time (first loop) shown in FIG. 3 with the waveform of the sound data stored in the storage unit 130. If the speech abnormality detection unit 110 determines that these waveforms do not match, it compares the waveform of the sound detected by the microphone 30 slightly after the unit time (first loop) shown in FIG. 3 , for example, the waveform of the unit time (second loop), with the waveform of the sound data stored in the storage unit 130.

[0037] The speech abnormality detection unit 110 acquires a speech command transmitted from the PC 300 and outputs it to the vehicle 10 (more specifically, the control device 200), thereby causing the speaker 20 to output a sound corresponding to the speech command. Here, for example, in detecting the sound output from the speaker 20, the speech abnormality detection unit 110 detects the sound for a predetermined period after the speech command is output to the vehicle 10 (more specifically, the control device 200), and in determining whether the sound is the predetermined sound, determines whether the sound output from the speaker 20 is the predetermined sound based on the sound detection results for the predetermined period. Specifically, the speech abnormality detection unit 110 compares, for example, the waveform of the sound per unit time detected by the microphone 30 with the waveform of the sound indicated by the sound data stored in the storage unit 130, repeatedly for a predetermined time while gradually shifting the range of the waveform of the sound detected by the microphone 30 to be compared until these two waveforms match. If these two waveforms do not match even after repeated detection for a predetermined period of time, the speech abnormality detection unit 110 determines that the sound output from the speaker 20 is not the predetermined sound.

[0038] The unit time and the predetermined period may be arbitrarily determined in advance and are not particularly limited. Information indicating these times (periods) is stored in advance in the storage unit 130, for example.

[0039] The terminal 100 may include a timekeeping unit such as a real time clock (RTC) for measuring time.

[0040] The speech abnormality response unit 120 is a processing unit that takes a predetermined response based on the detection result of the speech abnormality detection unit 110. For example, the speech abnormality response unit 120 outputs the determination result of the speech abnormality detection unit 110 as to whether or not the sound is a predetermined sound to the PC 300. For example, when the speech abnormality detection unit 110 determines that the sound output from the speaker 20 is a predetermined sound, the speech abnormality response unit 120 outputs information indicating that the sound output from the speaker 20 is a predetermined sound to the PC 300, thereby causing a notification device such as a speaker and / or a display device provided in the PC 300 to notify an operator of the information indicating that the sound output from the speaker 20 is a predetermined sound. On the other hand, for example, if the speech abnormality detection unit 110 determines that the sound output from the speaker 20 is not a specified sound, the speech abnormality response unit 120 outputs information indicating that the sound output from the speaker 20 is not a specified sound to the PC 300, thereby causing a notification device provided in the PC 300 to notify the operator of the information indicating that the sound output from the speaker 20 is not a specified sound.

[0041] Furthermore, for example, the speech abnormality response unit 120 outputs to the PC 300 suggestion information indicating the type of sound to be output from the speaker 20 based on the result of the determination by the speech abnormality detection unit 110 as to whether or not the sound is a predetermined sound. For example, the speech abnormality detection unit 110 determines that the waveform of the sound output from the speaker 20 does not match the waveform indicated by the sound data stored in the storage unit 130 due to differences in volume (amplitude). In this case, for example, the speech abnormality response unit 120 transmits to the PC 300 suggestion information suggesting that the speech command be output again with the volume changed.

[0042] The aspect of the sound is, for example, the volume, but may also be the frequency of the sound, the time for which the sound is output, etc.

[0043] Furthermore, for example, the speech abnormality response unit 120 controls the running state of the vehicle 10 based on the result of the determination by the speech abnormality detection unit 110 as to whether or not the sound is a predetermined sound. For example, when the speech abnormality response unit 120 determines that the sound output from the speaker 20 is not a predetermined sound, the speech abnormality response unit 120 stops the vehicle 10 or decelerates the vehicle 10 to run.

[0044] Also, for example, the speech abnormality response unit 120 determines the type of sound to be output from the speaker 20 and / or the speaker 21 based on the judgment result by the speech abnormality detection unit 110 as to whether or not the sound is a specified sound, and outputs the sound of the determined type from the speaker 20 and / or the speaker 21.

[0045] The speaker 20 and the speaker 21 are each an example of a second speaker.

[0046] In addition, the speech anomaly response unit 120 may determine the response to be taken depending on the state of the vehicle 10 and the type of obstacle located around the vehicle 10 (for example, the type of object, such as a pedestrian or other vehicle, that may interfere with the movement of the vehicle 10).

[0047] Fig. 4 is a diagram showing a specific example of a response executed by the terminal 100 according to the present embodiment. Specifically, Fig. 4 is a table showing a specific example of a process executed by the utterance anomaly response unit 120 based on the state of the vehicle 10, the type of utterance command, and the situation of an obstacle. For example, response information such as that shown in Fig. 4 is stored in advance in the storage unit 130.

[0048] The vehicle state is information indicating the type of control by which the vehicle 10 is traveling (driving). For example, the vehicle state "remotely operated" indicates a state in which the vehicle 10 is being driven by remote control by an operator. Also, for example, the vehicle state "automated driving" indicates a state in which the vehicle 10 is being driven automatically.

[0049] The spoken command is information indicating the type (content) of the spoken command. For example, the spoken command "horn" indicates that the spoken command is a command to make the vehicle 10 sound the horn. Also, for example, the spoken command "turn right or turn left" indicates that the spoken command is a command to make the vehicle 10 sound a sound when turning right or left. Also, for example, the spoken command "warning sound when starting" indicates that the spoken command is a command to make a sound when the vehicle 10 starts moving.

[0050] An obstacle is an object located on the travel path of the vehicle 10 or around the vehicle 10. For example, "there is an obstacle on the vehicle path" indicates that an object that will obstruct the travel of the vehicle 10 is located on the travel path of the vehicle 10. Also, for example, "there is an obstacle around the vehicle" indicates that an object is located around the vehicle 10.

[0051] The obstacle may be, for example, a pedestrian or another vehicle, but the type of obstacle is not particularly limited.

[0052] Furthermore, information indicating the travel route of vehicle 10 and whether or not an obstacle is located around vehicle 10 may be acquired from control device 200 or from PC 300. Furthermore, for example, terminal 100 may acquire information indicating the travel route of vehicle 10, as well as information indicating the situation around vehicle 10 detected by a camera, LiDAR (Light Detection and Ranging), or the like, from control device 200 and / or PC 300, and determine whether or not an obstacle is located around vehicle 10 and / or the travel route of vehicle 10 based on this information.

[0053] For example, when the utterance abnormality detection unit 110 determines that the sound output from the speaker 20 is not a predetermined sound, the vehicle state is "remote control," the speech command is "horn," and the obstacle is "obstacle on vehicle path," the utterance abnormality response unit 120 stops the vehicle 10 ("safe stop" shown in FIG. 4 ) and outputs information to the PC 300 indicating that the sound output from the speaker 20 is not a predetermined sound ("notify operator" shown in FIG. 4 ). In other words, when the utterance abnormality detection unit 110 detects an abnormality, the vehicle 10 is being remotely controlled, the predetermined sound is a sound corresponding to the speech command indicating "horn," and an obstacle is located on the driving path of the vehicle 10, the utterance abnormality response unit 120 stops the vehicle 10 and notifies the PC 300 of the abnormality.

[0054] Each processing unit such as the speech abnormality detection unit 110 and the speech abnormality response unit 120 is realized by, for example, a memory and a processor such as a CPU (Central Processing Unit) that executes a control program stored in the memory. The speech abnormality detection unit 110 and the speech abnormality response unit 120 may be realized by a common memory and processor, or by separate memories and processors.

[0055] The storage unit 130 is a storage device that stores various types of information. The storage unit 130 stores, for example, sound data, correspondence information, etc. The storage unit 130 is realized by, for example, a semiconductor memory or a hard disk drive (HDD).

[0056] The storage unit 130 may be accessible by the control device 200 .

[0057] The speaker 21 is a speaker that outputs sound. The speaker 21 is disposed in the vehicle 10, for example, and outputs sound around the vehicle 10. For example, the speaker 21 plays a supporting role to the speaker 20. For example, when an abnormality is detected in the speaker 20, the speaker 21 is controlled to output sound.

[0058] The microphone 30 is a microphone for detecting sound output from the speaker 20. Of course, the microphone 30 may also detect sound output from the speaker 21. The microphone 30 also detects, for example, sound (environmental sound) emitted from the surroundings of the vehicle 10. For example, the speech abnormality detection unit 110 detects the environmental sound via the microphone 30 before causing the speaker 20 to output a sound corresponding to a speech command. For example, the speech abnormality detection unit 110 determines whether the sound output from the speaker 20 is a predetermined sound based on the acquired environmental sound as well.

[0059] The PC 300 is a computer operated by an operator. The PC 300 is used, for example, by an operator located outside the vehicle 10. For example, the PC 300 is operated by the operator to transmit a speech command to the vehicle 10 (more specifically, the terminal 100).

[0060] The PC 300 includes, for example, a personal computer including a communication interface for communicating with each device, a non-volatile memory storing a program, a volatile memory serving as a temporary storage area for executing the program, an input / output port for transmitting and receiving signals, and a processor for executing the program, as well as a display device (for example, the display device 301 shown in FIG. 8 ), a notification device, and a user interface such as a mouse and keyboard for accepting operator operations. The communication interface may be realized by a connector connected to a control line for wired communication with a base station that wirelessly communicates with the terminal 100, or by a wireless communication circuit for wireless communication with the terminal 100. The PC 300 may also be realized by a tablet terminal or the like.

[0061] The PC 300 is an example of a remote device.

[0062] [Processing Procedure] Next, the processing procedure of the anomaly detection system 1 will be described.

[0063] FIG. 5 is a sequence diagram showing the processing procedure of the abnormality detection system 1 according to this embodiment.

[0064] First, the PC 300 acquires an event that triggers transmission of a speech command (S110). The type of event may be determined arbitrarily and is not particularly limited. Examples of events include the vehicle 10 starting to move, the vehicle 10 making a right or left turn, and an obstacle being positioned around the vehicle 10. For example, this information is transmitted from the vehicle 10, and the PC 300 acquires this information to acquire the event.

[0065] Next, the PC 300 outputs a speech command (S120). Specifically, the PC 300 transmits a speech command corresponding to the acquired event to the terminal 100. For example, if the event is that the vehicle 10 starts moving, the PC 300 outputs a speech command instructing the vehicle 10 to output a sound that is determined to be output when the vehicle 10 starts moving.

[0066] Next, the terminal 100 transfers the speech command output from the PC 300 to the control device 200 (S130).

[0067] Next, the control device 200 causes the speaker 20 to speak based on the speech command transferred from the terminal 100 (S140). That is, the control device 200 causes the speaker 20 to output a sound corresponding to the speech command.

[0068] After transferring the speech command to the control device 200, the terminal 100 starts detecting sound via the microphone 30 (S150). As a result, the terminal 100 starts detecting (acquiring) sound output from the speaker 20.

[0069] Next, the terminal 100 performs an abnormality determination process (S160). Specifically, the terminal 100 determines whether the sound output from the speaker 20 is a predetermined sound.

[0070] Next, the terminal 100 outputs the determination result in step S160 to the PC 300 (S170).

[0071] Next, the PC 300 notifies the operator of the acquired determination result using the notification device (S180).

[0072] 6 is a flowchart showing the procedure of the anomaly detection process of the terminal 100 according to the present embodiment. Specifically, FIG. 6 is a flowchart showing a specific example of the process of step S160 shown in FIG. 5, which is executed by the speech anomaly detection unit 110.

[0073] First, the speech abnormality detection unit 110 determines whether a predetermined period has elapsed since outputting a speech command to the control device 200 (S210). Specifically, the speech abnormality detection unit 110 performs the sound detection process shown in step S150 in Fig. 5 for a predetermined period. In step S210, the speech abnormality detection unit 110 determines whether the predetermined period for performing the sound detection process is still elapsed.

[0074] When it is determined that the predetermined period has elapsed (Yes in S210), the speech abnormality detection unit 110 determines whether the acquired sound is normal (S220). That is, the speech abnormality detection unit 110 determines whether the acquired sound is a predetermined sound.

[0075] If the speech abnormality detection unit 110 determines that the acquired sound is not normal (No in S220), the process returns to step S210. In other words, the speech abnormality detection unit 110 detects sound for a predetermined period of time and repeatedly determines whether the acquired sound is normal.

[0076] If the speech abnormality detection unit 110 determines that the predetermined period has not elapsed (No in S210), it outputs information indicating an abnormality (determination result) to the PC 300 (S240). In other words, the speech abnormality detection unit 110 detects sound for a predetermined period of time, and repeatedly determines whether the acquired sound is normal or not. If the speech abnormality detection unit 110 never determines that the sound is normal, it outputs information indicating an abnormality to the PC 300.

[0077] On the other hand, if the speech abnormality detection unit 110 determines that the acquired sounds are normal (Yes in S220), it outputs information indicating that the sounds are normal (determination result) to the PC 300 (S230). In other words, if the sounds acquired during a predetermined period include a predetermined sound, the speech abnormality detection unit 110 outputs information indicating that the sounds are normal to the PC 300.

[0078] (Embodiment 2) Next, a description will be given of embodiment 2. Note that in the following description, differences from embodiment 1 will be mainly described, and some or all of the overlapping description may be omitted or simplified.

[0079] FIG. 7 is a block diagram showing the configuration of terminal 101 according to this embodiment.

[0080] Like the terminal 100, the terminal 101 is a computer that acquires a speech command output from the PC 300 and outputs (transfers) the command to the control device 200. Also, like the terminal 100, the terminal 101 determines whether the sound output from the speaker 20 is a predetermined sound, and takes action according to the determination result.

[0081] The terminal 101 includes an utterance abnormality detection unit 111 , an utterance abnormality response unit 121 , a storage unit 131 , an ambient sound measurement unit 140 , and an utterance sound adjustment unit 150 .

[0082] The ambient sound measurement unit 140 is a processing unit that measures (detects) sounds around the vehicle 10 (hereinafter also simply referred to as ambient sounds). For example, the ambient sound measurement unit 140 periodically measures the volume (e.g., dB value) of the ambient sounds. The ambient sound measurement unit 140 acquires the ambient sounds around the vehicle 10 via the microphone 30, for example.

[0083] In addition to the microphone 30 that detects the sounds output from the speakers 20 and 21, the vehicle 10 may further include a microphone for detecting ambient sounds.

[0084] Furthermore, the ambient sound measurement unit 140 may estimate the volume of the ambient sound around the vehicle 10 based on the volume of the ambient sound acquired from the microphone 30 and information (a speech map) that links the type of obstacle, the position of the obstacle, and the volume of the ambient sound. The speech map may be stored in the storage unit 131 or may be transmitted from another device such as the PC 300.

[0085] Speech sound adjustment unit 150 is a processing unit that adjusts the volume of the sound output from speaker 20 and / or speaker 21. Specifically, speech sound adjustment unit 150 determines the volume to be output from speaker 20 based on the volume of the ambient sound measured by ambient sound measurement unit 140, and outputs a sound corresponding to a speech command from speaker 20 at the determined volume. In other words, speech sound adjustment unit 150 adjusts the volume of the speech sound based on the volume of the ambient sound measured by ambient sound measurement unit 140, and causes speaker 20 and / or speaker 21 to speak.

[0086] Note that the speech sound adjustment unit 150 may adjust the volume based on a speech command, or may adjust the volume based on the speech command and the volume of the ambient sound measured by the ambient sound measurement unit 140. The speech command may include information indicating the volume. The predetermined volume may also be determined in advance. For example, the speech sound adjustment unit 150 may adjust (change) the volume from the predetermined volume based on the volume of the ambient sound measured by the ambient sound measurement unit 140. The information indicating the predetermined volume may be stored in the storage unit 131 in advance.

[0087] Similar to the speech abnormality detection unit 110, the speech abnormality detection unit 111 is a processing unit that determines whether the sound output from the speaker 20 is a predetermined sound based on the detection result of the sound output from the speaker 20. In the present embodiment, the speech abnormality detection unit 111 determines whether the sound output from the speaker 20 is a predetermined sound based on the detection result of the sound output from the speaker 20 and ambient sounds. For example, the speech abnormality detection unit 111 checks whether the volume of the sound output from the speaker 20 is the volume adjusted by the speech sound adjustment unit 150.

[0088] Furthermore, for example, the speech abnormality detection unit 111 determines whether the sound output from the speaker 20 has reached an obstacle based on obstacle information indicating obstacles located around the vehicle 10.

[0089] The obstacle information is information in which information indicating the type of obstacle is linked to a threshold value related to the volume. For example, the speech abnormality detection unit 111 acquires information indicating the type of obstacle, such as a pedestrian or another vehicle, located around the vehicle 10 based on information (also referred to as sensor information) detected by a camera, LiDAR, or the like, arranged on the vehicle 10. The speech abnormality detection unit 111 also acquires information indicating the volume of the sound output from the speaker 20. The information indicating the volume is acquired, for example, via the microphone 30. For example, the speech abnormality detection unit 111 determines whether the volume of the sound output from the speaker 20 is equal to or greater than a threshold value included in the obstacle information and linked to the type of obstacle indicated by the sensor information. If the speech abnormality detection unit 111 determines that the volume is equal to or greater than the threshold value, it determines that the sound output from the speaker 20 has reached the obstacle located around the vehicle 10. On the other hand, if the speech abnormality detection unit 111 determines that the sound is less than the threshold, it determines that the sound output from the speaker 20 has not reached the obstacle located around the vehicle 10. In this way, for example, the speech abnormality detection unit 111 may determine whether the sound output from the speaker 20 has reached the obstacle based on the obstacle information and the sensor information.

[0090] The determination of whether a sound has reached an obstacle may take into consideration the distance between the vehicle 10 and the obstacle. For example, the sensor information may include information indicating the distance between the vehicle 10 and the obstacle. The obstacle information may also include information indicating the type of obstacle and information indicating the relationship between a volume threshold and the distance.

[0091] The abnormal speech detection unit 111 is an example of a detection unit and a determination unit.

[0092] The speech abnormality response unit 121 is a processing unit that takes a predetermined response based on the detection result of the speech abnormality detection unit 110. For example, if the sound output from the speaker 20 is not at the volume adjusted by the speech sound adjustment unit 150, the speech abnormality response unit 121 transmits information indicating the abnormality to the PC 300.

[0093] Furthermore, for example, the speech abnormality response unit 121 outputs the determination result of the speech abnormality detection unit 111 as to whether or not an obstacle has been reached to the PC 300. The speech abnormality response unit 121 may transmit obstacle information together with the determination result of the speech abnormality detection unit 111 as to whether or not an obstacle has been reached.

[0094] 8 is a diagram showing a specific example of an image displayed on the display device 301 of the PC 300 according to the present embodiment. The example shown in FIG. 8 shows a case where a pedestrian and another vehicle are present as obstacles around the vehicle 10.

[0095] For example, an image captured by a camera installed in the vehicle 10 is displayed on the display device 301. The image is transmitted to the PC 300 by the speech abnormality response unit 121 together with the determination result of the speech abnormality detection unit 111. For example, the speech abnormality detection unit 111 determines that the sound output from the speaker 20 has reached a pedestrian, but that the sound output from the speaker 20 has not reached another vehicle. The speech abnormality response unit 121 transmits, for example, this determination result, the camera image, and information indicating the corresponding obstacle to the PC 300. As a result, in such a case, for example, the display device 301 displays an image 310 including a pedestrian to whom the sound output from the speaker 20 has reached, and for which the determination result of the speech abnormality detection unit 111 has been displayed, such as "Attention OK," and an image 311 including another vehicle to whom the sound output from the speaker 20 has not reached, and for which the determination result of the speech abnormality detection unit 111 has been displayed, such as "Attention NG."

[0096] The storage unit 131 is a storage device that stores various types of information. The storage unit 131 stores, for example, sound data and response information. The storage unit 131 also stores, for example, obstacle information. The storage unit 130 is realized by, for example, a semiconductor memory or a HDD.

[0097] FIG. 9 is a flowchart showing the processing procedure of the terminal 101 according to this embodiment.

[0098] First, the terminal 101 transmits (transfers) the speech command acquired from the PC 300 to the vehicle 10 (more specifically, the control device 200) (S310).

[0099] Next, the terminal 101 detects a sound via the microphone 30 (S320). For example, the speech abnormality detection unit 111 detects a sound via the microphone 30.

[0100] Next, the terminal 101 determines whether or not attention has been drawn to obstacles located around the vehicle 10 (S330). Specifically, the speech abnormality detection unit 111 determines whether or not the sound output from the speaker 20 has reached the obstacles, based on obstacle information indicating obstacles located around the vehicle 10.

[0101] In addition, the speech abnormality detection unit 111 may determine whether the sound output from the speaker 20 is a specified sound based on the detection result of the sound output from the speaker 20, and if it determines that the sound is not a specified sound, it may determine that the sound output from the speaker 20 has not reached an obstacle.

[0102] Furthermore, in step S320, the ambient sound measurement unit 140 may further measure the ambient sound of the vehicle 10. Furthermore, in step S330, the speech abnormality detection unit 111 may determine whether the sound output from the speaker 20 has reached an obstacle, also based on the volume of the ambient sound.

[0103] Next, the terminal 101 (more specifically, the speech abnormality response unit 121) outputs (transmits) the determination result of the speech abnormality detection unit 111 to the PC 300 (S340). As a result, for example, an image showing the determination result of the speech abnormality detection unit 111 is displayed on the display device 301 provided in the PC 300, as shown in FIG.

[0104] (Modification) The configuration of the anomaly detection system of the present disclosure is not limited to the above. Modifications of the anomaly detection system of the present disclosure will be described below.

[0105] As described above, even if the vehicle 10 is able to output a sound, the abnormality detection system of the present disclosure determines whether the sound has reached the intended recipient. For example, if the terminal 100 determines that the sound has not reached the intended recipient (e.g., if it determines that the sound output from the speaker 20 is not a predetermined sound), the terminal 100 may again assist the vehicle 10 in determining whether to speak. For example, if the terminal 100 determines that the sound has not reached the intended recipient, the terminal 10 may output an instruction to the control device 200 to output, from the speaker 20 and / or the speaker 21, an operation of the vehicle 10, such as "We are departing," or a voice (one-shot sound) such as "We are passing" to alert people around the vehicle 10. This allows, for example, people around the vehicle 10 to be reliably alerted.

[0106] For example, the above utterance such as "We are departing" is made when it is determined that the sound output from the speaker 20 is not a predetermined sound. For example, when a target such as a person is not located near a moving body such as the vehicle 10, the terminal 100 may present, on the display device 301, information indicating that the user does not need to utter again. On the other hand, when a target such as a person is located near a moving body such as the vehicle 10, the terminal 100 may present, on the display device 301, information indicating that the user may be prompted to utter again. Furthermore, in this case, for example, the terminal 100 may display, on the display device 301, information indicating that the user may be prompted to utter again, together with the information indicating that the user may be prompted to utter again, so that the operator can recognize which target such as a person caused the user to utter again. This makes it easier for the operator to determine whether to prompt the moving body to utter again.

[0107] The above-described mobile object is, for example, a vehicle 10, but may also be a robot. The mobile object may also include a sensor such as a camera for identifying the target. The mobile object may also include, for example, a display for displaying an image and / or a light-emitting device such as an LED (Light Emitting Diode). For example, when the terminal 100 receives an instruction from an operator to speak again, the mobile object may use a display and / or an output device capable of outputting light other than sound, such as an LED, to make the mobile object give a warning or other notification to the target. In this case, the terminal 100 may make the mobile object give a warning or other notification using a method other than sound in addition to sound, or may make the mobile object give a warning or other notification using a method other than sound without outputting sound. By making the mobile object give a warning or other notification using a method other than sound without outputting sound, it is possible to reduce the annoyance felt by the target from being told the same thing repeatedly.

[0108] Furthermore, for example, the terminal 100 may cause the mobile object to speak the operator's words (voice) as they are, instead of speaking again. The PC 300 may be equipped with a microphone for detecting the operator's voice. The terminal 100 may, for example, acquire information indicating such voice from the PC 300 and output the information indicating the acquired voice from the speaker 20 and / or the speaker 21. This allows the operator to appropriately alert the target in the operator's own words depending on the situation. Furthermore, for example, the terminal 100 may cause the mobile object to speak a message different from the voice message that was previously spoken, instead of speaking again.

[0109] Furthermore, for example, if it is determined that the sound output from the speaker 20 is not a predetermined sound, it may be dangerous if the vehicle 10 is stopped.

[0110] Therefore, for example, if the sound output from the speaker 20 is not a predetermined sound, the terminal 100 may repeatedly output the same sound from the speaker 20 without stopping the vehicle 10. In other words, if the sound output from the speaker 20 is not a predetermined sound, the terminal 100 may loop the same sound to the vehicle 10, such as "Turn right (or left)," "Mechanical sound (for example, when the vehicle 10 is reversing)," or "Please go ahead."

[0111] For example, if the vehicle 10 stops on the spot because the utterance "turn right (or left)" is not a predetermined sound, it may cause a nuisance to other traffic participants. On the other hand, for example, if the vehicle 10 is an autonomous driving vehicle and the vehicle 10 autonomously moves as is, it may cause a hazard to other traffic participants located around the vehicle 10.

[0112] Here, for example, when the content of the utterance output from the speaker 20 is "Turn right (or left)," and it is determined that the sound output from the speaker 20 is not a predetermined sound, the terminal 100 may request the operator to pay attention. For example, the terminal 100 may output information (gazing request information) for requesting the PC 300 to pay attention to the vehicle 10. Specifically, when it is determined that the sound output from the speaker 20 is not a predetermined sound, the terminal 100 may output, to the PC 300 used by the operator located outside the vehicle 10, the gaze request information for requesting the operator to pay attention to the vehicle 10.

[0113] This allows the vehicle 10 to travel autonomously under the watchful eye of an operator, so that if other traffic participants appear to be in danger, the operator can remotely intervene and stop the vehicle 10.

[0114] Furthermore, for example, when requesting the operator to pay attention to the utterance, the terminal 100 may also notify the operator of the utterance content indicating what was said in the utterance. For example, the terminal 100 outputs information indicating the utterance content indicating what was said in the utterance (content information) to the PC 300 along with the attention request. Specifically, when the terminal 100 determines that the sound output from the speaker 20 is not a predetermined sound, it may further output content information indicating the content of the sound output from the speaker 20 to the PC 300. This makes it easier for the operator to understand what they need to pay attention to.

[0115] Furthermore, in the above-described looped speech (also referred to as loop processing), for example, the terminal 100 may determine whether the sound output from the speaker 20 is a predetermined sound for each loop processing. Furthermore, the terminal 100 may notify the operator if it determines that the sound is not the predetermined sound consecutively in each loop processing. According to such loop processing, even if the sound is determined to be not the predetermined sound once, if it is determined to be the predetermined sound the next time, it is considered that the sound has been properly notified to the target, i.e., the target has heard the sound.

[0116] For example, in a scene where a moving object utters "I'm passing by, I'm passing by, I'm passing by" three times using this loop processing, even if the first "I'm passing by" is not audible, i.e., it is determined not to be the specified sound, there is no problem as long as the second or third "I'm passing by" can be heard, i.e., it is determined to be the specified sound.

[0117] Furthermore, for example, when a mobile object speaks while moving, if there is a communication delay, the mobile object may speak from a different location than the intended location, which may result in a contradiction between the content of the speech and the control content.

[0118] Therefore, for example, when the vehicle 10 is made to utter "I'm passing through," if the vehicle 10 is stopped for a safety stop or the like between the time when the speech command instructing the vehicle 10 to utter "I'm passing through" is transmitted from the PC 300 and the time when the vehicle 10 actually utters the command, the vehicle 10, upon receiving the speech command, is controlled not to utter if there is a contradiction between the speech content included in the speech command and the state of the vehicle 10. Specifically, when the speech content indicates that the vehicle 10 will proceed, such as "I'm passing through," the terminal 100 determines that there is a contradiction between the speech content and the state of the vehicle 10 if the vehicle 10 is not proceeding, such as when the vehicle 10 makes an emergency stop. The criteria for determining whether or not there is a contradiction may be set arbitrarily.

[0119] In this way, for example, when the terminal 100 acquires status information indicating the status of the vehicle 10 and acquires a speech command transmitted from a PC 300 used by an operator located outside the vehicle 10, the terminal 100 determines whether to output a sound corresponding to the speech command from the speaker 20 based on the status information and the speech command. For example, based on the status information and the speech command, if there is no contradiction between the status of the vehicle 10 and the content of the speech command, the terminal 100 outputs a sound based on the speech command from the speaker 20, and if there is a contradiction between the status of the vehicle 10 and the content of the speech command, the terminal 100 does not output a sound based on the speech command from the speaker 20. The status information is information indicating the status of the vehicle 10, such as going straight, stopping, turning left, or turning right. This information may be acquired from various sensors provided in the vehicle 10, such as a speed sensor that detects the speed of the vehicle 10, an acceleration sensor that detects the acceleration of the vehicle 10, and a steering angle sensor that detects the steering angle of the vehicle 10.

[0120] Next, a specific configuration of a modified example of the anomaly detection system of the present disclosure will be described. Note that the following description will focus on differences from the first and second embodiments, and some or all of the overlapping descriptions may be omitted or simplified.

[0121] FIG. 10 is a block diagram showing the configuration of an abnormality detection system 2 according to the first modification.

[0122] The abnormality detection system 2 includes a vehicle 10, a PC 300A, a speaker 22, and a microphone 31. In the abnormality detection system 2, for example, the PC 300A detects whether or not there is an abnormality in the sound output from the speaker 20. Specifically, the PC 300A determines whether or not the sound output from the speaker 20 is a predetermined sound.

[0123] For example, the PC 300A acquires from the terminal 100 sound data (e.g., voice data) indicating sounds (e.g., voice) such as ambient sounds detected by the microphone 30, and outputs a sound based on the acquired voice data from the speaker 22 connected to the PC 300A. The PC 300A detects the output sound using the microphone 31, and determines whether there is an abnormality in the output sound by comparing the voice data indicating the voice detected by the microphone 31 with the sound output from the speaker 22 (specifically, the voice data indicating the voice output from the speaker 22). For example, if the PC 300A determines that there is an abnormality, it notifies the operator of the determination result indicating the abnormality. Specifically, the terminal 100 detects ambient sounds around the vehicle 10 using the microphone 30 provided in the vehicle 10, and outputs sound data indicating the detected ambient sounds around the vehicle 10 to the PC 300A used by an operator located outside the vehicle 10. Furthermore, the PC 300A detects sound based on sound data output from the speaker 22 connected to the PC 300A using a microphone 31 connected to the PC 300A, and determines whether or not there is an abnormality in the sound output from the speaker 22 based on the sound detected by the microphone 31 and the sound data. The microphone 30 is an example of a first microphone. The PC 300A is an example of a remote device. The speaker 22 is an example of a third speaker. The microphone 31 is an example of a second microphone. For example, when determining whether or not there is an abnormality in the sound output from the speaker 22, the PC 300A determines whether or not the sound is the same as a predetermined sound. The predetermined sound determined here may be determined arbitrarily and is not particularly limited. The predetermined sound determined here may be the same as or different from the predetermined sound used to determine the sound output from the speakers 20 and / or 21.

[0124] The PC 300A includes an utterance abnormality detection unit 112 and an utterance abnormality response unit 122.

[0125] The speech abnormality detection unit 112 is a processing unit that determines whether the sound output from the speaker 20 is a predetermined sound based on the detection result of the sound output from the speaker 20. Specifically, the speech abnormality detection unit 112 acquires sound data indicating the sound output from the speaker 20 provided in the vehicle 10 from the terminal 100 that detected the sound, and determines whether the sound output from the speaker 20 is a predetermined sound based on the acquired sound data. In other words, the speech abnormality detection unit 112 determines whether the predetermined sound desired by the operator is output from the speaker 20.

[0126] Specifically, the speech abnormality detection unit 112 outputs the sound indicated by the acquired sound data from the speaker 22. Furthermore, the speech abnormality detection unit 112 acquires sound data indicating the sound from the microphone 31 that detected the sound output from the speaker 22. Furthermore, the speech abnormality detection unit 112 determines whether the sound output from the speaker 20 is a predetermined sound based on the sound data.

[0127] The PC 300A includes a storage unit in which sound data indicating sound waveforms corresponding to speech commands is stored in advance. The storage unit is realized by, for example, an HDD or SSD. For example, similar to the speech abnormality detection unit 110, the speech abnormality detection unit 112 determines whether the waveform of the sound output from the speaker 20 matches the waveform indicated by the sound data stored in the storage unit, and if they match, determines that the sound output from the speaker 20 is a predetermined sound, and if they do not match, determines that the sound output from the speaker 20 is not a predetermined sound.

[0128] The abnormal speech detection unit 112 is an example of a detection unit and a determination unit.

[0129] The speech abnormality response unit 122 is a processing unit that takes a predetermined response based on the detection result of the speech abnormality detection unit 112. For example, the speech abnormality response unit 122 presents the determination result of the speech abnormality detection unit 112 as to whether or not the sound is a predetermined sound to the operator using a display device 301 provided in the PC 300.

[0130] Each processing unit such as the speech abnormality detection unit 112 and the speech abnormality response unit 122 is realized by, for example, a processor such as a CPU and a memory in which a control program executed by the processor is recorded.

[0131] In this way, each component included in the anomaly detection system of the present disclosure may be included in any device such as a PC, a terminal, or a control device.

[0132] In addition, when the PC 300A includes the speech abnormality detection unit 112 and the speech abnormality response unit 122, the terminal 100 may or may not include the speech abnormality detection unit 110 and the speech abnormality response unit 120.

[0133] Furthermore, the speech abnormality detection unit 112 and the speech abnormality response unit 122 included in the PC 300A may execute the processes of the speech abnormality detection unit 110 and the speech abnormality response unit 120, or may execute the processes of the speech abnormality detection unit 111 and the speech abnormality response unit 121. Furthermore, the PC 300A may include a processing unit that performs the processes executed by the ambient sound measurement unit 140 and the speech sound adjustment unit 150.

[0134] FIG. 11 is a block diagram showing the configuration of an abnormality detection system 3 according to the second modification.

[0135] The abnormality detection system 3 includes the vehicle 10, a PC 300, and a speaker 23. The abnormality detection system 3 detects an abnormality in the sound output from the speaker 23, which is located in a different location from the vehicle 10.

[0136] The speaker 23 is installed, for example, at the entrance of a building such as a factory. In this example, a scene is assumed in which sound is output from the speaker 23 to alert people around the speaker 23. For example, when the vehicle 10 is in a predetermined situation, such as when the vehicle 10 moves from inside a factory to outside the factory or from outside the factory to inside the factory, the speaker 23 outputs a sound to alert people around the speaker 23. For example, the terminal 100 determines whether or not there is an abnormality in the sound data indicating the sound output from the speaker 23 (determines whether or not the sound is the predetermined sound described above), and if it determines that there is an abnormality (i.e., if it determines that the sound is not the predetermined sound), it notifies the operator of the abnormality.

[0137] In this way, the terminal 100 determines whether or not not only the sound output from the speaker 20 but also the sound output from a speaker other than the speaker 20 (for example, the speaker 23) is a predetermined sound. For example, the terminal 100 further detects the sound output from the speaker 23 disposed outside the vehicle 10, and determines whether or not there is an abnormality in the sound output from the speaker 23 based on the detection result of the sound output from the speaker 23. The speaker 23 is an example of a fourth speaker.

[0138] This allows the terminal 100 to determine whether there is an abnormality in the sound detected around the vehicle 10.

[0139] The predetermined sound used to determine the sound output from the speaker 20 provided in the vehicle 10 and the sound output from the speaker 23 disposed outside the vehicle 10 may be the same or different. The predetermined sound is represented by sound data indicating a waveform, for example, but may also be represented by data indicating the volume, sound pressure, frequency, etc.

[0140] 12 is a flowchart showing a processing procedure of an information processing device according to one aspect of the present disclosure. The information processing device is, for example, the terminal 100 or the terminal 101. Note that the information processing device may be realized by any combination of the configurations and processes of the terminal 100 and the terminal 101.

[0141] First, the information processing device detects a sound output from a first speaker provided in the moving object (S10).

[0142] Next, the information processing device determines whether the sound output from the first speaker is a predetermined sound based on the detection result of the sound output from the first speaker (S20).

[0143] The information processing device includes, for example, a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0144] (Effects, etc.) Hereinafter, examples of techniques that can be obtained from the disclosure of this specification will be given, and effects, etc. that can be obtained from the exemplified techniques will be described.

[0145] Technique 1 is an information processing method that detects a sound output from a first speaker equipped on a moving body (S10), and determines whether the sound output from the first speaker is a predetermined sound based on the detection result of the sound output from the first speaker (S20).

[0146] The moving body is, for example, a vehicle 10 .

[0147] The mobile object may be any object that can move, such as a motorcycle or a robot.

[0148] The remote device is, for example, a PC 300 .

[0149] The command is, for example, the above-mentioned spoken command.

[0150] The first speaker is, for example, the speaker 20 .

[0151] This can detect, for example, that the first speaker is broken and is not outputting a predetermined sound. Furthermore, since the sound output from the first speaker is detected and it is determined whether the sound output from the first speaker is the predetermined sound, it can also detect a situation in which, even if sound is being output from the first speaker, the sound output from the first speaker does not properly reach targets such as pedestrians or other vehicles due to loud noise around the moving object. Therefore, the information processing method according to the embodiment can determine whether an appropriate sound is being output from the speaker (first speaker). In other words, it can determine whether the sound is being output from the speaker in a manner that properly reaches targets.

[0152] Technique 2 is an information processing method described in Technique 1, in which the determination of whether the sound output from the first speaker is a predetermined sound is made by comparing the waveform of the sound output from the first speaker with sound data indicating the waveform of the predetermined sound.

[0153] This makes it easy to determine whether the sound output from the first speaker is a predetermined sound.

[0154] Technology 3 is an information processing method according to Technology 1 or 2, in which a command transmitted from a remote device used by an operator located outside the mobile body is acquired and output to the mobile body, thereby outputting a sound corresponding to the command from a first speaker, and in detecting the sound output from the first speaker, sound is detected for a predetermined period after the command is output to the mobile body, and in determining whether the sound is the predetermined sound, it is determined whether the sound output from the first speaker is the predetermined sound based on the detection result of the sound for the predetermined period.

[0155] When a moving body (more specifically, a control device 200 that controls each device such as the speaker 20 provided in the vehicle 10) acquires a command, it outputs a sound corresponding to the command from the first speaker. Therefore, the sound corresponding to the command can be output from the first speaker, for example, within a predetermined period of time after the command is output to the moving body. Therefore, for example, if a long time has passed since the command was output to the moving body, the sound corresponding to the command will not be output from the first speaker. Therefore, if the predetermined sound is not detected within the predetermined period of time, it can be determined that the sound corresponding to the command has not been properly output from the first speaker. Therefore, this makes it possible to properly determine whether the sound output from the first speaker is the predetermined sound.

[0156] Technique 4 is an information processing method according to any one of Techniques 1 to 3, which further detects sounds around the moving body, and determines whether the sound output from the first speaker is a predetermined sound based on the detection result of the sound output from the first speaker and the surrounding sounds, in determining whether the sound is a predetermined sound.

[0157] This makes it possible to appropriately determine whether or not the detected sound contains a predetermined sound, even if the detected sound contains surrounding sounds as noise.

[0158] Technique 5 is the information processing method according to any one of techniques 1 to 4, in which the result of the determination as to whether or not the sound is a predetermined sound is output to a remote device used by an operator located outside the mobile body.

[0159] This allows an operator who is not located at the mobile object to know whether or not a predetermined sound is being appropriately output from the first speaker.

[0160] Technique 6 is an information processing method according to any one of Techniques 1 to 5, in which, based on the determination result of whether the sound is a predetermined sound, suggested information indicating the type of sound to be output from the first speaker is output to a remote device used by an operator located outside the mobile body.

[0161] For example, if the sound output from the first speaker is not reaching the target properly because the surrounding noise of the moving object is too loud, the volume of the sound output from the first speaker can be increased. Therefore, in such a case, for example, suggestion information including a suggestion to increase the volume is output to the remote device, and the suggestion is notified to the operator. This allows the operator to easily understand how to make the sound corresponding to the command reach the target properly.

[0162] Technique 7 is the information processing method according to any one of techniques 1 to 6, in which the traveling state of the moving body is controlled based on the determination result of whether or not the sound is a predetermined sound.

[0163] The sound output from the first speaker may include, for example, a warning sound for pedestrians around the moving body. Therefore, if such a warning sound is not output appropriately from the first speaker, pedestrians may be in danger. Therefore, for example, if it is determined that the sound output from the first speaker is not a predetermined sound, the moving body may be stopped or slowed down. This can improve the safety of the moving body during its travel.

[0164] Technique 8 is an information processing method described in any one of Techniques 1 to 7, in which the type of sound to be output from the second speaker is determined based on the determination result of whether the sound is a predetermined sound, and the sound of the determined type is output from the second speaker.

[0165] The second speaker is, for example, the speaker 21 .

[0166] The second speaker may be the speaker 20. In other words, the second speaker may be the first speaker or a speaker different from the first speaker.

[0167] According to this, even if the sound output from the first speaker does not reach the target person properly because the sound around the moving object is loud, the sound output from the first speaker can be made to reach the target person more easily by increasing the volume, etc. Furthermore, if the second speaker is a speaker different from the first speaker, the sound corresponding to the command can be output from the second speaker even if the first speaker breaks down.

[0168] Technique 9 is an information processing method according to any one of techniques 1 to 8, further determining whether or not the sound output from the first speaker has reached an obstacle based on obstacle information indicating obstacles located around the mobile body, and outputting the determination result of whether or not the sound has reached the obstacle to a remote device used by an operator located outside the mobile body.

[0169] This allows the operator to easily determine whether the sound output from the first speaker is reaching the obstacle appropriately.

[0170] Technology 10 is an information processing method described in any of Technology 1 to Technology 9, in which, if it is determined that the sound output from the first speaker is not a specified sound, gaze request information is output to a remote device used by an operator located outside the moving body to prompt the operator to gaze at the moving body.

[0171] This allows the vehicle to travel autonomously under the watchful eye of an operator, so if traffic participants around the vehicle appear to be in danger, the operator can remotely intervene and stop the vehicle.

[0172] Technology 11 is an information processing method described in Technology 10, in which, if it is determined that the sound output from the first speaker is not a predetermined sound, content information indicating the content of the sound output from the first speaker is further output to a remote device.

[0173] This makes it easier for the operator to understand what he or she needs to pay attention to.

[0174] Technology 12 is an information processing method according to any one of technologies 1 to 11, further comprising: detecting ambient sound around the mobile body using a first microphone equipped on the mobile body; outputting sound data indicating the detected ambient sound around the mobile body to a remote device used by an operator located outside the mobile body; detecting sound based on the sound data output from a third speaker connected to the remote device using a second microphone connected to the remote device; and determining whether or not there is an abnormality in the sound output from the third speaker based on the sound detected by the second microphone and the sound data.

[0175] The first microphone is, for example, the microphone 30 described above. The remote device referred to here is, for example, the PC 300A. The third speaker is, for example, the speaker 22 described above. The second microphone is, for example, the microphone 31 described above. When determining whether or not there is an abnormality in the sound output from the third speaker, for example, a determination is made as to whether or not the sound is the same as a predetermined sound.

[0176] This allows the moving body to determine whether or not there is an abnormality in sounds coming from speakers other than the first speaker.

[0177] Technique 13 is an information processing method according to any one of techniques 1 to 12, further detecting sound output from a fourth speaker positioned other than the moving body, and determining whether or not there is an abnormality in the sound output from the fourth speaker based on the detection result of the sound output from the fourth speaker.

[0178] The fourth speaker is, for example, the speaker 23. In determining whether or not there is an abnormality in the sound output from the fourth speaker, for example, a determination is made as to whether or not the sound is the same as a predetermined sound.

[0179] This allows the moving body to determine whether or not there is an abnormality in sounds coming from speakers other than the first speaker.

[0180] Technique 14 is an information processing method according to any one of techniques 1 to 13, further acquiring status information indicating a status of the mobile body, and when acquiring a command transmitted from a remote device used by an operator located outside the mobile body, determining whether or not to output a sound corresponding to the command from the first speaker based on the status information and the command.

[0181] According to this, for example, if there is a contradiction between the content of the sound corresponding to the command (the content of the speech) and the state of the moving body, the sound corresponding to the command can be prevented from being output from the first speaker, thereby preventing traffic participants located around the moving body from misunderstanding the sound corresponding to the command.

[0182] Technology 15 is an information processing device that includes a detection unit that detects sound output from a first speaker equipped in a moving body, and a determination unit that determines whether the sound output from the first speaker is a predetermined sound based on the detection result of the sound output from the first speaker.

[0183] The information processing device is, for example, the terminal 100 or the terminal 101. The detection unit and the determination unit are realized by, for example, the speech abnormality detection unit 110 or the speech abnormality detection unit 111.

[0184] This provides the same effects as the information processing method according to the embodiment.

[0185] Technique 16 is the information processing device according to technique 15, wherein the information processing device is disposed in a mobile body.

[0186] This allows for more reliable communication between the information processing device and the mobile body compared to, for example, a case where the information processing device is located outside the mobile body together with a remote device, etc. Therefore, when the mobile body is to execute a process such as stopping the mobile body when it is determined that the sound output from the first speaker is not a predetermined sound, the mobile body can be made to execute such a process more reliably.

[0187] Technique 17 is a program that causes a computer to execute the information processing method according to any one of techniques 1 to 14.

[0188] This provides the same effects as the information processing method according to the embodiment.

[0189] Other Embodiments Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments and modifications.

[0190] For example, in the above embodiment, a process executed by a specific processing unit may be executed by another processing unit. Also, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0191] In the above-described embodiments, each component (each processing unit) may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0192] Furthermore, each component may be realized by hardware. Each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0193] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0194] In addition, this disclosure also includes forms obtained by applying various modifications that a person skilled in the art would conceive of to each embodiment and variant, or forms realized by arbitrarily combining the components and functions of each embodiment and variant within the scope that does not deviate from the intent of this disclosure.

[0195] The present disclosure is applicable to a device that is mounted on a vehicle and detects an abnormality in a speaker provided in the vehicle.

[0196] 1, 2, 3 Abnormality detection system 10 Vehicle 20, 21, 22, 23 Speaker 30, 31 Microphone 100, 101 Terminal 110, 111, 112 Speech abnormality detection unit 120, 121, 122 Speech abnormality response unit 130, 131 Storage unit 140 Ambient sound measurement unit 150 Speech sound adjustment unit 200 Control device 300, 300A PC 301 Display device 310, 311 Image

Claims

1. An information processing method comprising: detecting a sound output from a first speaker equipped in a moving object; and determining whether the sound output from the first speaker is a specified sound based on the detection result of the sound output from the first speaker.

2. The information processing method of claim 1, wherein in determining whether or not the sound is the specified sound, a waveform of the sound output from the first speaker is compared with sound data indicating the waveform of the specified sound to determine whether or not the sound output from the first speaker is the specified sound.

3. The information processing method of claim 1, further comprising: acquiring a command transmitted from a remote device used by an operator located outside the moving body and outputting the command to the moving body, thereby outputting a sound corresponding to the command from the first speaker; detecting the sound output from the first speaker by detecting sound for a predetermined period of time after the command is output to the moving body; and determining whether the sound is the predetermined sound by determining whether the sound output from the first speaker is the predetermined sound based on the detection result of the sound for the predetermined period of time.

4. The information processing method of claim 1, further comprising detecting sounds around the moving body, and determining whether the sound is the specified sound by determining whether the sound output from the first speaker is the specified sound based on the detection result of the sound output from the first speaker and the surrounding sounds.

5. The information processing method according to claim 1, further comprising outputting a result of the determination as to whether or not the sound is a predetermined sound to a remote device used by an operator located outside the moving body.

6. The information processing method according to claim 1, further comprising: outputting, based on the determination result of whether the sound is a specified sound or not, suggested information indicating the type of sound to be output from the first speaker to a remote device used by an operator located outside the mobile body.

7. The information processing method according to claim 1, further comprising controlling a running state of the moving object based on a result of determining whether the sound is a predetermined sound or not.

8. The information processing method according to claim 1, further comprising determining a style of sound to be output from the second speaker based on a result of the determination as to whether the sound is the predetermined sound, and outputting the sound in the determined style from the second speaker.

9. The information processing method according to claim 1, further comprising: determining whether or not the sound output from the first speaker has reached the obstacle based on obstacle information indicating obstacles located around the moving body; and outputting the determination result as to whether or not the sound has reached the obstacle to a remote device used by an operator located outside the moving body.

10. The information processing method of claim 1, further comprising: if it is determined that the sound output from the first speaker is not the specified sound, outputting gaze request information to a remote device used by an operator located outside the moving body to cause the operator to gaze at the moving body.

11. The information processing method according to claim 10, further comprising the step of outputting content information indicating the content of the sound output from the first speaker to the remote device when it is determined that the sound output from the first speaker is not the specified sound.

12. The information processing method of claim 1, further comprising: detecting ambient sound of the moving body using a first microphone equipped on the moving body; outputting sound data indicating the detected ambient sound of the moving body to a remote device used by an operator located outside the moving body; detecting sound based on the sound data output from a third speaker connected to the remote device using a second microphone connected to the remote device; and determining whether or not there is an abnormality in the sound output from the third speaker based on the sound detected by the second microphone and the sound data.

13. The information processing method of claim 1, further comprising: detecting sound output from a fourth speaker positioned other than the moving body; and determining whether or not there is an abnormality in the sound output from the fourth speaker based on the detection result of the sound output from the fourth speaker.

14. The information processing method of claim 1, further comprising: acquiring status information indicating a status of the moving body; and, when acquiring a command transmitted from a remote device used by an operator located outside the moving body, determining, based on the status information and the command, whether or not to output a sound corresponding to the command from the first speaker.

15. An information processing device comprising: a detection unit that detects a sound output from a first speaker equipped in a moving body; and a judgment unit that judges whether the sound output from the first speaker is a predetermined sound based on a detection result of the sound output from the first speaker.

16. The information processing device according to claim 15, wherein the information processing device is disposed in the mobile object.

17. A program for causing a computer to execute the information processing method according to any one of claims 1 to 14.

Images