Information processing device, information processing method, and program

Abstract

This information processing device (1) comprises an output control unit (10), an input unit (20), a determination unit (30), and a mode setting unit (40). The output control unit (10) outputs question information relating to whether a co-occupant is present in a moving body. The input unit (20) receives input of response information relating to a response by an occupant to the question information. The determination unit (30) determines, using the response information, whether a co-occupant is present in the moving body. The mode setting unit (40) determines, using the result of the determination by the determination unit (30), whether to apply a first mode for operating as a voice recognition device upon detecting a first keyword, or a second mode for operating as a voice recognition device even when no first keyword is detected.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Patent Document 1 (Japanese Unexamined Patent Application Publication No. 2020-148805) discloses a technique for detecting the number of people in the surroundings and making it possible to omit a trigger word for enabling voice recognition when the detected number of people in the surroundings is one.

[0003] Patent Document 2 (International Publication No. 2017 / 042906) discloses a technique for determining whether the number of speakers in a vehicle is plural or singular and changing the voice recognition method according to the number of people in the vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In causing a voice recognition device to perform voice recognition, a technique is known in which a wake-up word (= trigger word) is used to request the voice recognition device to execute a predetermined command (such as route guidance). In particular, when there are a plurality of speakers, there is a possibility that the voice recognition device may erroneously recognize an unintended voice (command) and request the voice recognition device to execute an unintended command. By using the wake-up word for voice recognition, misrecognition of the voice recognition device is suppressed.

[0006] On the other hand, when there is only one speaker, the likelihood of the speech recognition device misinterpreting the speech is reduced, making it very cumbersome to use a wake-up word every time you request the speech recognition device to execute a command.

[0007] Patent Document 1 describes acquiring audio, detecting the number of people in the surrounding area based on the audio, and, if only one person is detected, performing subsequent audio processing without recognizing a trigger word. However, it is difficult to accurately detect the number of people inside a vehicle by analyzing in-vehicle sound information, including audio, engine noise, and the voices of multiple speakers, based on that sound information.

[0008] Furthermore, Patent Document 2 describes using cameras and pressure sensors installed inside the vehicle to determine the number of passengers. However, it is difficult to accurately detect whether there is one or multiple passengers in the vehicle because passengers may not be clearly visible to the camera or may have luggage inside the vehicle. Moreover, if the number of passengers is incorrectly detected, the possibility of the voice recognition device activating unintentionally increases, which would disrupt a comfortable driving experience.

[0009] The problem that this invention aims to solve is to provide an information processing device that can enable a comfortable driving experience while suppressing erroneous speech recognition. [Means for solving the problem]

[0010] The invention described in claim 1 is, An output control unit that outputs question information regarding the presence or absence of passengers inside the mobile vehicle, An input unit into which answer information regarding the passenger's response to the aforementioned question information is entered, A determination unit that uses the aforementioned response information to determine whether or not there is a passenger inside the mobile vehicle, The information processing device includes a mode setting unit that determines, using the determination result of the determination unit, whether to apply a first mode in which it operates as a speech recognition device when a first keyword is detected, or a second mode in which it operates as a speech recognition device even when the first keyword is not detected.

[0011] The invention described in claim 11 is, A computer that makes up an information processing device, The system outputs question information regarding the presence or absence of passengers inside the vehicle. The response information regarding the passenger's answers to the aforementioned questions is entered. Using the aforementioned response information, determine whether or not there is a passenger inside the moving vehicle. This information processing method determines whether to apply a first mode, in which the device operates as a voice recognition device when a predetermined keyword is detected, or a second mode, in which the device operates as a voice recognition device even without detecting the keyword, using the result of determining whether or not there is a passenger inside the mobile vehicle.

[0012] The invention described in claim 12 is, Computers that realize information processing devices, Procedure for outputting question information regarding the presence or absence of passengers inside the vehicle. A procedure for inputting information regarding the passenger's answers to the aforementioned questions, A procedure for determining whether or not there is a passenger inside the mobile vehicle using the aforementioned response information. This program is for executing a procedure to determine whether to apply a first mode, which operates as a voice recognition device when a predetermined keyword is detected, or a second mode, which operates as a voice recognition device even if the keyword is not detected, using the result of determining whether or not there is a passenger inside the mobile vehicle. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic block diagram of an information processing device according to the first embodiment. [Figure 2] This figure shows an example of the hardware configuration of an information processing device. [Figure 3] The flowchart of the process in which the information processing apparatus according to the first embodiment applies the first mode or the second mode. [Figure 4] The block diagram showing the outline of the information processing apparatus according to the second embodiment. [Figure 5] The flowchart of the process in which the storage processing unit of the second embodiment stores the mode determination information in the storage unit. [Figure 6] The flowchart of the process in which the information processing apparatus applies the first mode or the second mode using the mode determination information. [Figure 7] The block diagram for explaining the functions of the input unit and the storage processing unit according to the fourth embodiment.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and the description will be omitted as appropriate.

[0015] In the following description, each component of each device indicates a block of a functional unit, not a configuration of a hardware unit. Each component of each device is realized by an arbitrary combination of hardware and software centered on the CPU, memory, program loaded in the memory, storage media such as a hard disk storing the program, and network connection interface of an arbitrary computer. And there are various modifications to the realization method and device.

[0016] [First Embodiment] (Information Processing Apparatus 1) FIG. 1 is a block diagram showing the outline of the information processing apparatus 1 according to the first embodiment. The information processing apparatus 1 will be described with reference to FIG. 1. In the first embodiment, the information processing apparatus 1 is provided outside the vehicle. The information processing apparatus 1 may be a so-called cloud server.

[0017] In the first embodiment, the information processing device 1 includes a voice recognition function. The information processing device 1 receives a predetermined command (for example, directions, search, etc.) as voice information and executes processing corresponding to the command. The predetermined command is, for example, "Show me the route to the nearest station" or "Find the nearest parking lot." The information processing device 1 receives the command as voice information and executes processing corresponding to the command (show the route to the nearest station, find the nearest parking lot).

[0018] The information processing device 1 operates as a speech recognition device when it detects a first keyword. In other words, when the information processing device 1 detects a first keyword, it accepts the speech information spoken after that first keyword as a predetermined command. The first keyword is a so-called wake-up word (=trigger word). The wake-up word is a keyword necessary to cause the information processing device 1 to execute a predetermined command (to execute speech recognition processing). For example, if "A" is registered as the first keyword, and the driver says "A, find a nearby parking lot," the information processing device 1 accepts "Find a nearby parking lot" (speech information spoken after the first keyword) as a predetermined command.

[0019] (Mobile device 2) The mobile body is equipped with an internal device 2. The internal device 2 may be brought in from outside the mobile body or may be pre-attached to the mobile body. In the first embodiment, the mobile body is a vehicle. The internal device 2 includes a camera unit 2a, an audio output unit 2b, an audio input unit 2c, and a position sensor unit 2d. Although not shown, the internal device 2 may include a display.

[0020] Camera unit 2a has an in-camera and an out-camera (not shown). The in-camera is facing inward and includes at least the driver's seat in its field of view. The in-camera photographs the interior of the vehicle so that the driver is visible. Alternatively, the in-camera may photograph the entire interior of the vehicle so that the occupants (driver and passengers) are visible. The out-camera is facing outward. The out-camera photographs the area outside the vehicle.

[0021] In the first embodiment, the audio output unit 2b includes a speaker (not shown). The audio output unit 2b outputs machine voice from the speaker.

[0022] The voice input unit 2c includes at least one microphone (not shown). The voice spoken by the passenger is input to the voice input unit 2c. The passenger includes the driver and other passengers.

[0023] The position sensor unit 2d acquires the position information of a moving object, for example, using GNSS (Global Navigation Satellite System).

[0024] The mobile device 2 is configured to communicate with the information processing device 1 via the communication network 101. The communication network 101 includes, for example, communication over a 4G or 5G line. The mobile device 2 is configured to include an external communication unit (not shown) for configuring the communication network 101.

[0025] (Output control unit 10) The information processing device 1 comprises an output control unit 10, an input unit 20, a determination unit 30, and a mode setting unit 40. The output control unit 10 outputs question information. In the first embodiment, the output control unit 10 outputs question information to the voice output unit 2b of the mobile device 2. The question information is information regarding the presence or absence of passengers in the mobile device. For example, the question information is information to confirm with the driver whether or not there are passengers in the mobile device, such as "Are you going out alone today?" or "Are you accompanied by anyone?". In the first embodiment, the output control unit 10 outputs the question information by voice, but it may also output it to a display on the mobile device 2.

[0026] (Input section 20) The input unit 20 receives the answer information. In the first embodiment, the input unit 20 receives the voice information input to the voice input unit 2c of the mobile device 2 as the answer information. The answer information is information about the passenger's answer to the question information. For example, if the question information is "Are you going out alone today?", the answer information would be answers such as "No, I'm going out with my family of four" or "Yes, I'm alone".

[0027] (Judgment Department 30) The decision unit 30 uses the response information to determine whether or not there is a passenger inside the vehicle. The decision unit 30 determines that there is a passenger inside the vehicle if the response information includes information that suggests there is a passenger inside the vehicle. Specifically, if the question information "Are you going out alone today?" is output from the voice output unit 2b and the voice information "No, I'm going out with my family of four" is input to the input unit 20 as the response information, the decision unit 30 determines that there is a passenger inside the vehicle.

[0028] (Mode setting unit 40, 1st mode, 2nd mode) The mode setting unit 40 determines whether to apply the first mode or the second mode using the determination result of the determination unit 30. The first mode is a mode in which the information processing device 1 operates as a speech recognition device when the first keyword is detected. In other words, the first mode is a mode in which, when the information processing device 1 detects the first keyword, the information processing device 1 accepts the speech information uttered after the first keyword as a predetermined command.

[0029] Let's explain a specific example where "A" is registered as the first keyword and the command "Find a parking lot near the mobile object" is registered as a predetermined command. In this case, when the driver says "A, find a nearby parking lot," the information processing device 1 receives the command "Find a parking lot near the mobile object" and executes the process of searching for a parking lot near the mobile object. The information processing device 1 then outputs the search results to the voice output unit 2b.

[0030] The second mode is a mode in which the information processing device 1 operates as a speech recognition device even without detecting the first keyword. In other words, the second mode is a mode in which the information processing device 1 operates as a speech recognition device even without a wake-up word. In the example above, when the driver says "Find a nearby parking lot," the information processing device 1 receives the command "Find a parking lot near the vehicle" and executes the process of searching for a parking lot near the vehicle. Thus, in the second mode, the information processing device 1 can receive a command even if the driver does not say "A" as the first keyword.

[0031] The mode setting unit 40 applies the first mode when the determination unit 30 determines that there is a passenger inside the mobile vehicle. When there is one or more passengers inside the mobile vehicle, the first mode, which requires a first keyword (wake-up word), is applied, which reduces the possibility of misrecognizing the voices of multiple speakers, thus suppressing incorrect voice recognition by the information processing device 1.

[0032] The mode setting unit 40 applies the second mode when the determination unit 30 determines that there are no passengers in the vehicle. When there are no passengers in the vehicle (i.e., only the driver), the second mode is applied, which does not require the first keyword (wake-up word), thus eliminating the need to enter the wake-up word each time. This improves the comfort of driving.

[0033] When the second mode is applied, the passenger's voice information input to the voice input unit 2c will continue to be uploaded to the information processing device 1. However, the personal information contained in the voice information may be configured not to be uploaded to the information processing device 1, or all such personal information may be deleted after it has been uploaded to the information processing device 1.

[0034] In terms of specific configurations, the mobile device 2 may have a function to filter voice information. More specifically, for example, the configuration may be such that only voice information containing specific keywords ("search," "directions," "weather," etc.) is uploaded to the information processing device 1. Alternatively, the configuration may be such that voice information not containing specific keywords is automatically deleted by the information processing device 1.

[0035] (Example hardware configuration) Figure 2 shows an example of the hardware configuration of the information processing device 1. The information processing device 1 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0036] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0037] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0038] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0039] The storage device 1040 is an auxiliary storage device implemented as a removable media such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or memory card, or as ROM (Read Only Memory), and has a recording medium. The recording medium of the storage device 1040 stores program modules that realize each function of the information processing device 1 (for example, the output control unit 10, the input unit 20, the determination unit 30, and the mode setting unit 40). The processor 1020 reads these program modules onto the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 may also function as a storage unit 3, which will be described later.

[0040] The input / output interface 1050 is an interface for connecting the information processing device 1 with various input / output devices.

[0041] The network interface 1060 is an interface for connecting the information processing device 1 to a network. This network may be, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1060 connects to the network may be wireless or wired. The information processing device 1 may communicate with the mobile device 2 via the network interface 1060.

[0042] (Example of operation 1) Figure 3 is a flowchart of the process in which the information processing device 1 of the first embodiment applies either the first or second mode. In step S10, the output control unit 10 causes the voice output unit 2b to output question information. Specifically, for example, the output control unit 10 causes the voice output unit 2b to output a machine voice saying, "Are you going out alone today?" as question information.

[0043] The timing of the output of the question information is arbitrary; it may be output when the driver gets into the vehicle, or when the driver starts the vehicle's engine. It may also be output after the driver has been driving the vehicle for some time or after a predetermined distance has been traveled. Furthermore, it may be output at the first traffic light stop after the driver starts driving (when the first stopped state after the vehicle has started moving is detected).

[0044] In step S20, the input unit 20 (voice input unit 2c) receives the answer information. Specifically, for example, if the driver answers "No" to the above question information ("Are you going out alone today?"), the voice information "No" is input to the input unit 20 via the voice input unit 2c as the answer information.

[0045] In step S30, the determination unit 30 uses the above response information to determine whether or not there is a passenger inside the vehicle. In the above case, the determination unit 30 uses the response information "no" returned by the driver to determine that there is a passenger inside the vehicle. If the determination unit 30 determines that there is a passenger inside the vehicle (YES in step S30), the process proceeds to step S40. In step S40, the mode setting unit 40 applies the first mode.

[0046] If the determination unit 30 determines that there are no passengers inside the mobile vehicle (NO in step S30), the process proceeds to step S50. In step S50, the mode setting unit 40 applies the second mode.

[0047] As described above, according to the first embodiment, the determination unit 30 uses the response information to determine whether or not there is a passenger inside the mobile vehicle. This makes it possible to accurately determine whether or not there is a passenger inside the mobile vehicle with a simple configuration.

[0048] Furthermore, the mode setting unit 40 according to the first embodiment uses the determination result of the determination unit 30 to determine whether to apply the first mode or the second mode. Specifically, the mode setting unit 40 applies the first mode if the determination unit 30 determines that there is a passenger inside the vehicle, and applies the second mode if the determination unit 30 determines that there is no passenger inside the vehicle.

[0049] This allows the system to decide whether or not to omit the wake-up word depending on whether or not there are passengers in the vehicle. By omitting the wake-up word when there are no passengers, the user can avoid the need to speak the wake-up word each time they request a specific command (such as finding the nearest station or setting a route to the nearest parking lot). This enables faster command requests to the voice recognition system. Furthermore, the elimination of the hassle of speaking the wake-up word makes driving more comfortable.

[0050] By configuring the system so that the wake-up word is not omitted when there are passengers inside the mobile vehicle, the possibility of any of the passengers' voices being incorrectly recognized by the information processing device 1 is reduced.

[0051] As described above, the information processing device 1 according to the first embodiment can achieve a comfortable driving experience while suppressing erroneous speech recognition.

[0052] [Second Embodiment] Figure 4 is a block diagram illustrating the information processing device 1 according to the second embodiment. Unlike the first embodiment, the information processing device 1 according to the second embodiment further includes a device information acquisition unit 50 and a storage processing unit 60. Furthermore, in the second embodiment, the mobile device 2 is configured to include a mobile communication unit (not shown) for constituting the mobile communication means 102.

[0053] The mobile device 2 is configured to communicate with the communication device 4 inside the mobile vehicle via the mobile communication means 102. The mobile communication means 102 is, for example, a wireless communication means, and includes WiFi® and Bluetooth®. The communication device 4 is a device of the passenger of the mobile vehicle, and is owned, managed, carried, or used by the passenger. The communication device 4 is brought into the mobile vehicle by the passenger.

[0054] (Device information acquisition unit 50) The device information acquisition unit 50 acquires multiple device information from multiple communication devices 4 within the mobile vehicle. In the second embodiment, the device information acquisition unit 50 acquires device information via the mobile vehicle device 2.

[0055] Communication devices 4 include, for example, smartphones, tablets, PCs (personal computers), and wearable devices. Device information is information used to identify each communication device 4. Device information is information related to communication devices 4. Information related to communication devices 4 includes, for example, the name registered on the communication device 4, the device model, the device ID, and the name of the device registered on the communication device 4. By using device information, multiple communication devices 4 can be identified individually.

[0056] The device information acquisition unit 50 may determine whether or not the communication device 4 is located within a moving object using radio wave strength information relating to the strength of radio waves. The radio wave strength information may be acquired using a known configuration.

[0057] The device information acquisition unit 50 may determine whether or not the communication device 4 is inside the mobile body by comparing the device information acquired in the first state with the device information acquired in the second state. The first state is the state before the mobile body starts moving. Specifically, the first state is the state before the driver starts driving. The second state is the state after a predetermined time has elapsed or after the mobile body has traveled a predetermined distance since it started moving. Specifically, the second state is the state some time after the driver has started driving.

[0058] The device information acquisition unit 50 may repeatedly acquire device information and use the device history information regarding the acquisition history of device information to determine whether or not the communication device 4 is in a moving vehicle. More specifically, for example, the device information acquisition unit 50 may determine that the communication device 4 is in a moving vehicle if the device information acquired continuously for a predetermined time (e.g., 30 seconds) after the driver starts driving is the same. The timing at which the device information acquisition unit 50 begins to continuously acquire device information may be when the driver starts the vehicle's engine, or immediately after starting to drive. The device information acquisition unit 50 may also continuously acquire device information from the time it starts acquiring device information until it has traveled a predetermined distance. Furthermore, the device information acquisition unit 50 may repeatedly acquire device information up to immediately after starting to drive.

[0059] Furthermore, if the device information acquisition unit 50 continuously acquires device information for a predetermined period of time after the driver starts driving, and that device information differs from the device information acquired immediately after the start of device information acquisition, the device information acquisition unit 50 may determine that the communication device 4 related to the device information acquired immediately after the start of device information acquisition is not in the mobile vehicle. Also, if the device information acquired immediately after the start of device information acquisition has disappeared after a predetermined period of time has elapsed, the device information acquisition unit 50 may determine that the communication device 4 related to that device information is not in the mobile vehicle.

[0060] (Memory processing unit 60) The memory processing unit 60 associates multiple device information and response information and stores them in the memory unit 3 as mode determination information. The memory processing unit 60 stores in the memory unit 3 the response information input to the input unit 20 and the combination of multiple device information detected within the mobile body when the response information is input.

[0061] (Example of operation of the second embodiment 1) Figure 5 is a flowchart of the process by which the storage processing unit 60 of the second embodiment stores mode determination information in the storage unit 3. The process by which the storage processing unit 60 stores mode determination information in the storage unit 3 will be explained using Figure 5.

[0062] In step S5, the device information acquisition unit 50 acquires multiple device information from multiple communication devices 4 within the mobile body. Steps S11 and S21 are the same as steps S10 and S20 of the first embodiment, respectively. Specifically, in step S11, the output control unit 10 causes the voice output unit 2b to output question information, and in step S21, answer information is input to the input unit 20 (voice input unit 2c).

[0063] The timing at which the device information acquisition unit 50 acquires device information is arbitrary. The device information acquisition unit 50 may acquire device information after a predetermined time has elapsed since the driver started driving the vehicle. Alternatively, the device information acquisition unit 50 may acquire device information after the driver has driven the vehicle a predetermined distance. In this case, the predetermined distance may be calculated using the position information of the moving object generated by the position sensor unit 2d. Note that the processing in step S5 may be executed after the processing in step S11.

[0064] In step S25, the memory processing unit 60 associates the multiple device information acquired by the device information acquisition unit 50 with the answer information and stores it in the memory unit 3 as mode determination information. Specifically, for example, if the question information "Are you alone today?" is output in step S11, and the answer information "No, I'm driving with a friend" is input to the input unit 20 in step S21, the memory processing unit 60 stores the combination of the answer information, the driver's device information, and the passenger's (=friend's) device information in the memory unit 3. After step S25, the process proceeds to step S31.

[0065] The processes in steps S31, S41, and S51 are the same as the processes in steps S30, S40, and S50 of the first embodiment, respectively.

[0066] (Example of operation of the second embodiment 2) Figure 6 is a flowchart of the process by which the information processing device 1 applies either the first mode or the second mode using mode determination information. The process of applying either the first mode or the second mode using mode determination information will be explained using Figure 6. In step S6, the device information acquisition unit 50 acquires multiple device information from multiple communication devices 4 within the mobile vehicle.

[0067] Similar to Operation Example 1 of the second embodiment, the timing at which the device information acquisition unit 50 acquires device information is arbitrary. The device information acquisition unit 50 may acquire device information immediately after the start of operation, or it may acquire device information after a predetermined time has elapsed since the driver started operation. Alternatively, the device information acquisition unit 50 may acquire device information after a predetermined time has elapsed since the driver started the vehicle's engine. Furthermore, the device information acquisition unit 50 may acquire device information after the driver has driven the vehicle a predetermined distance.

[0068] Next, in step S7, the mode setting unit 40 uses the multiple device information and the mode determination information previously stored in the storage unit 3 to determine whether or not to omit the question information. Specifically, the mode setting unit 40 searches the storage unit 3 to see if there is device information with the same combination as the multiple device information acquired in step S6. If there is multiple device information with the same combination as the multiple device information, the mode setting unit 40 acquires the multiple device information with the same combination and the answer information associated with it as mode determination information. The mode setting unit 40 then uses this mode determination information to determine whether or not to omit the question information. The mode setting unit 40 determines to omit the question information if the mode determination information contains information that suggests there is a passenger in the vehicle. For example, if the answer information included in the mode determination information is "No, I'm out with two friends," the mode setting unit 40 determines to omit the question information. If the mode setting unit 40 determines to omit the question information (YES in step S7), the process proceeds to step S42.

[0069] Then, in step S42, the mode setting unit 40 applies the first mode without using the determination result of the determination unit 30 (the determination result of whether or not there is a passenger inside the mobile vehicle).

[0070] If the mode setting unit 40 determines that the question information should not be omitted (NO in step S7), the process proceeds to step S12. The processing in steps S12, S22, S26, S32, S42, and S52 is the same as the processing in steps S11, S21, S25, S31, S41, and S51 of the second embodiment (Figure 5), respectively.

[0071] As described above, according to the second embodiment, the information processing device 1 further comprises a device information acquisition unit 50 and a storage processing unit 60, and the mode setting unit 40 according to the second embodiment determines whether or not to omit question information using a plurality of device information and pre-stored mode determination information. As a result, when the same device as multiple devices previously stored is brought into the vehicle (when the same driver and passenger combination is in the vehicle), the process of outputting question information can be omitted. Therefore, the driver's effort and inconvenience of having to answer question information can be reduced, and the information processing man-hours can be suppressed.

[0072] Furthermore, the device information acquisition unit 50 according to the second embodiment may use radio wave intensity information to determine whether or not the communication device 4 is inside a moving object. This makes it easy to determine whether or not the communication device 4 is inside a moving object. Therefore, it is possible to suppress the erroneous acquisition of device information relating to a communication device that is outside a moving object.

[0073] Furthermore, the device information acquisition unit 50 according to the second embodiment may determine whether or not the communication device 4 is located within the moving vehicle by comparing the device information acquired in the first state with the device information acquired in the second state. By comparing the device information acquired some time after the driver has started driving with the device information acquired before the driver has started driving, it is possible to easily determine whether or not the communication device 4 is located within the moving vehicle by determining whether or not the device information is the same. This reduces the possibility of mistakenly acquiring device information located near where the vehicle is parked.

[0074] Furthermore, the device information acquisition unit 50 according to the second embodiment may repeatedly acquire device information and use device history information relating to the acquisition history of device information to determine whether or not the communication device 4 is located in a mobile object. For example, if device information is repeatedly acquired over a predetermined period of time and all the repeatedly acquired device information belongs to the same communication device, it can be said that there is a very high probability that the communication device is located in a mobile object. By using a configuration that utilizes device history information, it is possible to easily determine whether or not the communication device 4 is located in a mobile object.

[0075] By having the device information acquisition unit 50 acquire device information after a predetermined time has elapsed since the driver started driving (or after the vehicle's engine was started), or after a predetermined distance has been traveled, it is possible to prevent the device information acquisition unit 50 from mistakenly acquiring device information for a communication device located outside the vehicle near where the vehicle is parked.

[0076] [Third Embodiment] The information processing device 1 according to the third embodiment has a configuration that is almost the same as that shown in Figure 4 (configuration of the information processing device 1 according to the second embodiment), but some of the functions of the mode setting unit 40 differ from those of the information processing device 1 according to the second embodiment. After the mode setting unit 40 according to the third embodiment applies the second mode, when the device information acquisition unit 50 detects that the number of communication devices 4 in the mobile device has increased, the mode setting unit 40 according to the third embodiment switches from the second mode to the first mode.

[0077] This allows the system to automatically switch to mode 1 even if a passenger joins the group midway through the drive. By immediately switching to mode 1, which requires a wake-up word, as soon as there are multiple speakers, the possibility of the information processing device 1 misrecognizing speech is reduced. Furthermore, even if the number of passengers increases midway through the drive, there is no need to perform the processing shown in Figure 6, as the system automatically switches from mode 2 to mode 1, thus reducing the information processing workload.

[0078] In another aspect of the third embodiment, after the mode setting unit 40 applies the first mode, when the device information acquisition unit 50 detects that the number of communication devices 4 in the mobile body has decreased, the output control unit 10 causes the question information to be output again. Specifically, when the device information acquisition unit 50 detects that the number of communication devices 4 in the mobile body has decreased, the processing from step S12 onwards in Figure 6 is executed.

[0079] This allows for a smooth transition to the second mode, even if there was at least one passenger at the beginning of the drive but later the passengers left, leaving the driver alone. Therefore, it can improve driving comfort.

[0080] [Fourth Embodiment] The information processing device 1 according to the fourth embodiment has substantially the same configuration as shown in Figure 4 (configuration of the information processing device 1 according to the second embodiment), but some of the functions of the input unit 20 and the storage processing unit 60 differ from those of the information processing device 1 according to the second embodiment.

[0081] By inputting a second keyword to the input unit 20 according to the fourth embodiment, the memory processing unit 60 causes the specific mode determination information to be erased. The second keyword is any word and may be pre-set or newly set by the driver. The specific mode determination information is any mode determination information stored in the memory unit 3.

[0082] Figure 7 is a block diagram illustrating the functions of the input unit 20 and the memory processing unit 60 according to the fourth embodiment. The block diagram in Figure 7 is almost identical to the block diagram in Figure 4, but the number of devices relative to the number of passengers is different (in Figure 4, there are two communication devices 4 for two passengers, while in Figure 7, there are two communication devices 4 for one passenger). The functions of the input unit 20 and the memory processing unit 60 according to the fourth embodiment will be explained in detail using Figures 4 and 7.

[0083] For example, suppose a driver who owns two communication devices 4 gets into a vehicle with a child (corresponding to the situation in Figure 4), and the input unit 20 receives the answer "No" to the question "Are you alone today?", and the memory processing unit 60 stores the combination of this answer information and the device information of the two communication devices 4 in the memory unit 3.

[0084] Then, the next time the driver brings the same two communication devices 4 into the vehicle without a child (corresponding to the situation in Figure 7), the previously stored mode determination information (a combination of the answer information and the device information of the two communication devices 4) is adopted by the mode setting unit 40. As a result, even though there is no passenger (child) and only the driver is present, the question information is omitted and the system automatically switches to mode 1.

[0085] However, the memory processing unit 60 has a function to erase specific mode determination information, so that previously stored mode determination information can be erased. Therefore, the correct mode determination information can be stored again.

[0086] In another aspect of the fourth embodiment, the output control unit 10 may output the question information again by inputting a third keyword to the input unit 20. Specifically, by inputting a third keyword to the input unit 20, the processing from step S12 in Figure 6 (or step S11 in Figure 5) onward may be executed. The third keyword is an arbitrary word and may be pre-set or newly set by the driver.

[0087] This allows the driver to re-execute the process even if it mistakenly inputs incorrect answer information, as shown in Figure 6 (or Figure 5). This enables switching to the correct mode (the mode intended by the driver). Furthermore, it allows the driver to re-store the correct mode determination information.

[0088] (Other variations) Multiple combinations of communication devices and answer information may be manually stored in the storage unit 3 in advance. The mode setting unit 40 may then use the multiple device information acquired by the device information acquisition unit 50 and the manually stored information of such combinations to determine whether or not to omit the question information.

[0089] The embodiments described above with reference to the drawings are examples of the present invention, and various other configurations can also be adopted.

[0090] The information processing device 1 may be mounted inside a mobile vehicle. In this embodiment, the information processing device 1 may also include a mobile vehicle device 2. In this case, the mobile vehicle device 2 (= information processing device 1) includes, in addition to the camera unit 2a, audio output unit 2b, audio input unit 2c, and position sensor unit 2d, an output control unit 10, an input unit 20, a determination unit 30, and a mode setting unit 40.

[0091] If the driver turns off the engine after applying the second mode shown in Figure 3, the process shown in Figure 3 may be executed again. At this time, the wording of the question information may or may not be changed.

[0092] Examples of reference formats are provided below. 1. An output control unit that outputs question information regarding the presence or absence of passengers inside the mobile vehicle, An input unit into which answer information regarding the passenger's response to the aforementioned question information is entered, A determination unit that uses the aforementioned response information to determine whether or not there is a passenger inside the mobile vehicle, An information processing device comprising: a mode setting unit that determines, using the determination result of the determination unit, whether to apply a first mode in which the device operates as a speech recognition device when a first keyword is detected, or a second mode in which the device operates as a speech recognition device even when the first keyword is not detected. 2. In the information processing device described in 1., The mode setting unit applies the first mode when the determination unit determines that there is a passenger inside the mobile vehicle, and applies the second mode when the determination unit determines that there is no passenger inside the mobile vehicle. 3. In the information processing device described in 1. or 2., A device information acquisition unit that acquires multiple device information for multiple communication devices within the aforementioned mobile vehicle, The system further comprises a storage processing unit that links the aforementioned multiple device information and the aforementioned response information and stores them as mode determination information, The mode setting unit determines whether or not to omit the question information using the plurality of device information and the mode determination information stored in advance. If the mode setting unit determines that the question information should be omitted, the mode setting unit applies the first mode without using the determination result of the determination unit. 4. In the information processing device described in any one of items 1 to 3, The system further includes a device information acquisition unit that acquires multiple device information items for multiple communication devices within the aforementioned mobile vehicle, An information processing device wherein, after the mode setting unit applies the second mode, when the device information acquisition unit detects that the number of communication devices in the mobile body has increased, the mode setting unit switches from the second mode to the first mode. In the information processing device described in 5.4, An information processing device in which, after the mode setting unit has applied the first mode, the device information acquisition unit detects that the number of communication devices in the mobile body has decreased, and the output control unit causes the question information to be output again. 6. In the information processing device described in any one of 3 to 5, The device information acquisition unit is an information processing device that determines whether or not the communication device is located within the mobile body using radio wave intensity information relating to the intensity of radio waves. 7. In the information processing device described in any one of 3 to 6, The state before the moving body moves is defined as the first state, and the state after a predetermined time has elapsed or after the moving body has traveled a predetermined distance is defined as the second state. The device information acquisition unit determines whether or not the communication device is located within the mobile body by comparing the device information acquired in the first state with the device information acquired in the second state. 8. In the information processing device described in any one of 3 to 7, The device information acquisition unit repeatedly acquires the device information and uses device history information relating to the acquisition history of the device information to determine whether or not the communication device is located within the mobile body. 9. In the information processing device described in any one of items 3 to 8, An information processing device in which, by inputting a second keyword into the input unit, the memory processing unit erases specific mode determination information. 10. In the information processing device described in any one of items 1 to 9, An information processing device wherein, by inputting a third keyword into the input unit, the output control unit causes the question information to output again. 11. A computer that implements an information processing device, The system outputs question information regarding the presence or absence of passengers inside the vehicle. The response information regarding the passenger's answers to the aforementioned questions is entered. Using the aforementioned response information, determine whether or not there is a passenger inside the moving vehicle. An information processing method that determines whether to apply a first mode, in which the device operates as a voice recognition device when a predetermined keyword is detected, or a second mode, in which the device operates as a voice recognition device even if the keyword is not detected, using the result of determining whether or not there is a passenger inside the mobile vehicle. 12. To the computer that will realize the information processing device, Procedure for outputting question information regarding the presence or absence of passengers inside the vehicle. A procedure for inputting information regarding the passenger's answers to the aforementioned questions, A procedure for determining whether or not there is a passenger inside the mobile vehicle using the aforementioned response information. A program for executing a procedure to determine whether to apply a first mode, which operates as a voice recognition device when a predetermined keyword is detected, or a second mode, which operates as a voice recognition device even if the keyword is not detected, using the result of determining whether or not there is a passenger inside the mobile vehicle.

[0093] This application claims priority based on Japanese Patent Application No. 2022-158479, filed on 30 September 2022, and incorporates all of its disclosures herein. [Explanation of Symbols]

[0094] 1. Information Processing Device 2 Mobile internal device 2a Camera section 2b Audio output section 2c Voice input section 2D position sensor unit 3 Storage section 4. Communication devices 10 Output control unit 20 Input section 30 Judgment Department 40 Mode setting section 50 Device Information Acquisition Unit 60 Memory Processing Unit

Claims

1. A device information acquisition unit that acquires device information about communication devices located within a mobile device, A mode setting unit sets either a first mode in which the device operates as a speech recognition device when a first keyword is detected, or a second mode in which the device operates as a speech recognition device even if the first keyword is not detected, based on the device information acquired by the device information acquisition unit. A storage processing unit that links the device information and the information regarding the presence or absence of passengers in the mobile vehicle and stores them as mode determination information, An output control unit that outputs question information regarding the presence or absence of passengers in the mobile vehicle, An input unit into which answer information regarding the passenger's response to the aforementioned question information is entered, Equipped with, The mode setting unit sets either the first mode or the second mode based on the device information and the mode determination information stored in advance. The memory processing unit is an information processing device that stores the response information, which is information regarding the presence or absence of a passenger in the mobile vehicle, and the device information as mode determination information.

2. In the information processing apparatus according to Claim 1, The output control unit is an information processing device that, after the mode setting unit has set the first mode, causes the device information acquisition unit to output the question information when it detects that the number of communication devices in the mobile body has decreased.

3. In the information processing apparatus described in Claim 1, The mode setting unit determines whether or not to omit the question information based on the device information acquired by the device information acquisition unit and the mode determination information stored in advance. The output control unit is an information processing device that does not output the question information when the mode setting unit determines that the question information should be omitted.

4. In the information processing apparatus according to any one of claims 1 to 3, An information processing device in which, by inputting a second keyword into the input unit, the storage processing unit erases specific mode determination information.

5. In the information processing apparatus according to any one of claims 1 to 3, An information processing device wherein, by inputting a third keyword into the input unit, the output control unit causes the question information to output again.

6. In the information processing apparatus according to any one of claims 1 to 3, The state before the moving body moves is defined as the first state, and the state after a predetermined time has elapsed or after the moving body has traveled a predetermined distance is defined as the second state. The device information acquisition unit determines whether or not the communication device is located within the mobile body by comparing the device information acquired in the first state with the device information acquired in the second state.

7. A computer that makes up an information processing device, To obtain device information about communication devices located within a mobile object, Based on the acquired device information, one of two modes is set: a first mode in which the device operates as a speech recognition device when a first keyword is detected, and a second mode in which the device operates as a speech recognition device even if the first keyword is not detected. The device information and the information regarding the presence or absence of passengers in the mobile vehicle are linked and stored as mode determination information. Based on the device information and the pre-stored mode determination information, either the first mode or the second mode is set. The system outputs question information regarding the presence or absence of passengers in the aforementioned mobile vehicle. The response information regarding the passenger's answers to the aforementioned questions is entered. An information processing method that links the response information, which is information regarding the presence or absence of passengers in the mobile vehicle, with the device information and stores it as mode determination information.

8. Computers that realize information processing devices, Procedure for obtaining device information about communication devices located within a mobile object. A procedure for setting either a first mode in which the device operates as a speech recognition device when a first keyword is detected, or a second mode in which the device operates as a speech recognition device even if the first keyword is not detected, based on acquired device information. A procedure for linking the device information and the information regarding the presence or absence of passengers in the mobile vehicle and storing them as mode determination information. A procedure for setting either the first mode or the second mode based on the device information and the mode determination information stored in advance, A procedure for outputting question information regarding the presence or absence of passengers in the aforementioned mobile vehicle. A procedure for inputting information regarding the passenger's answers to the aforementioned questions, A program for executing a procedure to link the response information, which is information regarding the presence or absence of passengers in the mobile vehicle, with the device information and store it as mode determination information.

Images