Server device, information processing method, and server program

Abstract

To provide a guidance control device that, even when a terminal device and a smartphone are in proximity to each other and the terminal device is not provided with a display, can appropriately provide information by using the terminal device.SOLUTION: A guidance control device comprises: an interface 1 that is connected by radio respectively to a smartphone SP that displays a guidance image and a smart speaker SK that outputs a voice for movement guidance; a determination unit 2 that, when the smartphone SP and the smart speaker SK are in a vehicle C, determines whether the guidance image is displayed; and a control unit 3 that makes an output mode of the voice for the movement guidance different according to a result of determination made by the determination unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present application relates to the technical field of a server device, an information processing method, and a server program. More specifically, the present application relates to the technical field of a server device, an information processing method, and a server program for controlling a method of guiding a moving body such as a vehicle. [Background technology]

[0002] In recent years, a home server connected to the Internet has been installed in, for example, a typical home, and various information is collected from the Internet via the home server. It has also been considered to enable operations such as turning on / off lights in the home and changing television channels by voice commands using a microphone and speaker provided in a terminal device connected to the home server. Patent Document 1 listed below is an example of a prior art document disclosing such technology. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-114744 Summary of the Invention [Problem to be solved by the invention]

[0004] On the other hand, since such terminal devices are small and portable, it would be convenient if they could be placed inside a vehicle and used as a navigation device to guide the vehicle's movement or as a device to control the vehicle's air conditioner, etc.

[0005] However, the terminal device described in Patent Document 1 does not have a display used in conventional navigation devices. This display displays, for example, images including maps and routes for travel guidance, and is an important component for such travel guidance. On the other hand, examples of portable devices that can be brought into a vehicle and have a display include so-called smartphones, but linking these smartphones and the terminal device to provide travel guidance for the vehicle has not been considered until now.

[0006] Therefore, the present application has been made in consideration of the above-mentioned problems and demands, and one example of the object of the present application is to provide a server device, an information processing method, and a server program for the server device that can accurately provide information such as vehicle travel guidance using a terminal device, even when the terminal device and the smartphone, etc. are located in close proximity to each other (more specifically, for example, inside a vehicle) and the terminal device is not equipped with a display. [Means for solving the problem]

[0007] In order to solve the above problem, the invention described in claim 1 comprises an acquisition means for respectively acquiring display device current status information indicating the current status of a display device that displays an image and output device current status information indicating the current status of an output device that outputs audio; a determination means for determining whether the display device and the output device are within a predetermined range based on the acquired display device current status information and output device current status information; and, if it is determined that the display device and the output device are within the range, a transmission means for (i) transmitting to the display device a guide image, which is the image for guidance, and an instruction to display the guide image, and (ii) transmitting to the output device an image-using guide voice, which is the guide voice for guidance using the guide image, and an instruction to output the image-using guide voice, and, if it is determined that the display device and the output device are not within the range, (iii) transmitting to the output device a non-image-using guide voice, which is the voice for guidance using only audio, the output content of which is different from the image-using guide voice, and an instruction to output the non-image-using guide voice.

[0008] In order to solve the above problem, the invention of claim 7 is an information processing method executed in a server device having an acquisition means, a determination means, and a transmission means, the information processing method comprising: an acquisition step of acquiring, by the acquisition means, display device current state information indicating the current state of a display device that displays an image, and output device current state information indicating the current state of an output device that outputs sound; a determination step of determining, by the determination means, whether or not the display device and the output device are within a predetermined range based on the acquired display device current state information and output device current state information; and, when it is determined that the display device and the output device are within the range, (i) displaying the guidance image, which is a guidance image for guidance. The method includes a transmitting step of (ii) transmitting an image and an instruction to display the guidance image to the display device by the transmitting means, and (iii) transmitting to the output device by the transmitting means an image-using guidance voice, which is a guidance voice for guidance using the guidance image, and an instruction to output the image-using guidance voice, if it is determined that the display device and the output device are not within the range, a non-image-using guidance voice, which is a voice for guidance using only the voice, the output content of which is different from the image-using guidance voice, and an instruction to output the non-image-using guidance voice.

[0009] In order to solve the above problem, the invention described in claim 8 causes a computer to function as the server device described in any one of claims 1 to 6. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of a guidance control device according to an embodiment; [Figure 2] 1 is a block diagram showing a schematic configuration of a guidance control system according to an embodiment; [Figure 3]1A and 1B are block diagrams showing the general configuration of a server device and the like according to an embodiment, where (a) is a block diagram showing the general configuration of the server device, (b) is a block diagram showing the general configuration of a smart speaker according to an embodiment, and (c) is a block diagram showing the general configuration of a smartphone according to an embodiment. [Figure 4] 10 is a flowchart illustrating a guidance control process according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Next, an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram showing a schematic configuration of a guidance control device according to the embodiment.

[0012] As shown in FIG. 1, a guidance control device S according to the embodiment is configured to include a connection means 1, a determination means 2, and a control means 3.

[0013] In this configuration, the connection means 1 can be connected to a display device SP that displays images and an output device SK that outputs audio via communication. The communication in this case may be wired communication, wireless communication, direct communication, indirect communication, or any standard to which it should conform, but for example, the connection means 1 is connected to the display device SP and the output device SK wirelessly.

[0014] Then, the determining means 2 determines whether or not the display device SP and the output device SK are within a preset range.

[0015] As a result, the control means 3 changes the audio output mode of the output device SK depending on the determination result of the determination means 2.

[0016] As described above, according to the operation of the guidance control device S of the embodiment, the output mode of the sound is changed depending on the determination result of whether the display device SP that displays images and the output device SK that outputs sound are within a predetermined range. Therefore, necessary information can be provided more accurately by sound alone depending on whether the separate display device SP and the output device SK are both within the predetermined range. [Example]

[0017] Next, specific examples corresponding to the above-described embodiments will be described with reference to the drawings. The examples described below are examples in which a so-called smart speaker and a smartphone are used in a vehicle, and the vehicle's movement is guided using a guidance control system in which the smart speaker and the smartphone are connected to a server device via a network such as the Internet. In this case, the smart speaker refers to a device that includes, for example, at least a microphone, a speaker, and a communication interface, and is capable of transmitting and receiving information, data, or messages to and from a network such as the Internet. The smartphone corresponds to an example of a display device SP according to the embodiment, and the smart speaker corresponds to an example of an output device SK according to the embodiment.

[0018] Fig. 2 is a block diagram showing the general configuration of a guidance control system according to an embodiment, Fig. 3 is a block diagram showing the general configuration of a server device etc. according to an embodiment, and Fig. 4 is a flowchart showing guidance control processing according to an embodiment. In Fig. 2 and Fig. 3, the same component numbers as those of the respective components in the guidance control device S according to the embodiment shown in Fig. 1 are used for the respective components in the embodiment.

[0019] As shown in FIG. 2, the guidance control system SS according to the embodiment is configured with a smart speaker SK present in the vehicle C driven by the user (e.g., placed in the vehicle C), a smartphone SP owned by the user, and a server device S. In this case, since the user who drives the vehicle C also owns the smartphone SP, as shown in FIG. 2, the smart speaker SK and the smartphone SP are ultimately placed in the same vehicle C. The server device S and the smart speaker SK, and the server device S and the smartphone SP, can mutually exchange messages and various data via the network NW. With this configuration, the guidance control system SS according to the embodiment executes guidance control processing according to the embodiment, which will be described in detail below. The server device S corresponds to an example of a guidance control device S according to the embodiment.

[0020] Although not shown in Fig. 2, in the guidance control system SS according to the embodiment, smartphones SP and smart speakers SK placed in a plurality of vehicles C are connected to the server device S shown in Fig. 2 via a network NW for each smartphone SP and smart speaker SK. In the guidance control system SS according to the embodiment, the guidance control process according to the embodiment is executed for each smartphone SP and smart speaker SK present in one vehicle C. At this time, even if a smart speaker SK and one smartphone SP of the same user are present in one vehicle C and are the targets of the guidance control process according to the embodiment, at a different time, another smartphone SP and smart speaker SK of the same user may be present in the same vehicle C and these may also be the targets of the guidance control process according to the embodiment.

[0021] Specifically, in the guidance control process according to the embodiment of the guidance control system SS shown in FIG. 2, first, each smart speaker SK and smartphone SP placed in a vehicle C transmits current status information, including a user ID for identifying their user, current location information indicating their current location, and device identification information that identifies the smart speaker SK or smartphone SP and enables individual devices of the same type to a server device S via a network NW, for example, at predetermined intervals. The current status information transmitted from the smartphone SP also includes, in addition to the user ID, display status information indicating whether a screen (image) for guiding the movement of the vehicle C is displayed on the display of the smartphone SP (described later). In the following description, the screen (image) for guiding the movement of the vehicle C will be simply referred to as a "guidance image." Here, the "guidance image" refers not only to a screen displaying specific guidance, such as route guidance information, but also to a simple map display or an image of a navigation application.

[0022] When the display status information indicates that the guide image is displayed on the display of the smartphone SP, the server device S provides travel guidance for the vehicle C using the guide image. In the following description, travel guidance for the vehicle C using the guide image (in other words, using the smartphone SP on which the guide image is displayed) is simply referred to as "image-based guidance." On the other hand, when the display status information indicates that the display itself is not functioning because the smartphone SP is powered off, or that the guide image is not displayed on the display (for example, an email application unrelated to the navigation application is displayed), the server device S provides travel guidance for the vehicle C using only audio from the smart speaker SK. In the following description, travel guidance for the vehicle C using only audio is simply referred to as "image-free guidance." When comparing the image-based guidance and the image-free guidance, there are various differences between the content of the two due to the latter being vehicle C travel guidance using only audio. These differences will be described in detail later.

[0023] Next, a schematic configuration of the server device S according to the embodiment will be described with reference to FIG.

[0024] First, the schematic configuration of the server device S according to the embodiment will be described with reference to Fig. 3(a). As shown in Fig. 3(a), the server device S according to the embodiment is composed of an interface 1, a processing unit 4 including a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), etc., a recording unit 5 including, for example, an HDD (Hard Disc Drive) or an SSD (Solid State Drive), etc., a display 6 including a liquid crystal display, etc., and an operation unit 7 including a keyboard, a mouse, etc.

[0025] The processing unit 4 is composed of a determination unit 2 and a control unit 3. In this case, the determination unit 2 and the control unit 3 may be configured as a hardware logic circuit including the CPU and the like constituting the processing unit 4, or may be realized in software by the CPU reading and executing a program corresponding to the processing to be executed by the server device S among the guidance control processing according to the embodiment described later.

[0026] Furthermore, in the above configuration, the interface 1 corresponds to an example of the connection means 1 according to the embodiment, the determination unit 2 corresponds to an example of the determination means 2 according to the embodiment, and the control unit 3 corresponds to an example of the control means 3 according to the embodiment. As shown by the dashed lines in Fig. 3(a), the interface 1, the determination unit 2, and the control unit 3 constitute an example of a guidance control device S according to the embodiment. Furthermore, the interface 1 corresponds to an example of the "display device identification information acquisition means" and an example of the "output device identification information acquisition means" according to the present application, and the processing unit 4 corresponds to an example of the "identity determination means" according to the present application.

[0027] In the above-described configuration of the server device S, the interface 1, under the control of the processing unit 4, controls the exchange of messages, various data, etc. between the server device S and the smart speaker SK and smartphone SP via the network NW.

[0028] The recording unit 5 also non-volatilely records a program corresponding to the process to be executed by the server device S among the guidance control processes according to the embodiment, and temporarily records data and the like required for executing the guidance control process. Furthermore, the recording unit 5 stores, in association with a user ID for identifying the user of the smart speaker SK and the smartphone SP, search count information indicating the number of searches based on requests from users, including other users, for each of multiple search results obtained by multiple search methods for points or features (e.g., facilities as destinations; the same applies below) related to past guidance control processes based on requests from the user, in association with the user ID for identifying the user of the smart speaker SK and the smartphone SP. The search count information does not necessarily include the number of searches based on a current request from the user of the smart speaker SK and the smartphone SP. Additionally, the recording unit 5 stores, in association with the user ID, search history information indicating the past search history of the user. Note that, if the user of the smart speaker SK and the user of the smartphone SP are different, the search history information may be search history information indicating the search history of at least one of the users. The search count information and the search history information may be acquired from an external server device (not shown) or the like.

[0029] On the other hand, when an operation by an administrator or an operator to instruct the operation of the server device S is performed on the operation unit 7, the operation unit 7 generates an operation signal corresponding to the operation and outputs the operation signal to the processing unit 4. As a result, the processing unit 4 and the determination unit 2 and control unit 3 of the processing unit 4 exchange necessary messages and various data, etc. with the smart speaker SK and the smartphone SP based on the operation signal, using the program, data, or information recorded in the recording unit 5, and mainly execute the processing to be executed by the server device S among the guidance control processing according to the embodiment. Furthermore, data necessary for executing the processing is presented to the administrator or the operator via the display 6.

[0030] Next, the schematic configuration of a smart speaker SK according to an embodiment will be described with reference to FIG. 3(b). As shown in FIG. 3(b), the smart speaker SK according to the embodiment includes an interface 10, a CPU 11, an indicator 12 consisting of a small liquid crystal display or an LED (Light Emission Diode), a speaker 13, a memory 14 consisting of a ROM and RAM, a microphone 15, and a sensor unit 16 consisting of a GPS (Global Positioning System) sensor. When the indicator 12 is configured using the small liquid crystal display, the liquid crystal display may only have the function of displaying a few characters, and is not capable of displaying detailed guidance maps or routes, as is the case with liquid crystal displays provided in conventional navigation devices. In this configuration, the interface 10, under the control of the CPU 11, controls the exchange of messages and various data with the server device S via the network NW. The memory 14 also stores, in a non-volatile manner, a program corresponding to the process to be executed by the smart speaker SK in the guidance control process according to the embodiment, and temporarily stores various data required for executing the guidance control process. Meanwhile, under the control of the CPU 11, when the user speaks to the smart speaker SK, the microphone 15 detects the voice and outputs corresponding voice data to the CPU 11. At this time, the utterance includes a user's speech to instruct the operation of the smart speaker SK. The microphone 15 then outputs the voice data corresponding to the utterance to the CPU 11. Based on the voice data, the CPU 11 exchanges necessary messages and data with the server device S while using the programs and data stored in the memory 14, thereby primarily executing the processes to be executed by the smart speaker SK among the guidance control processes according to the embodiment. At this time, the processes include a process of detecting the current location of the smart speaker SK using the GPS sensor constituting the sensor unit 16. Furthermore, under the control of the CPU 11, the execution status of the processes is presented to the user via the indicator 12.Furthermore, under the control of CPU 11, speaker 13 outputs (emits sound) the sound required for the process to be recognized by the user. Furthermore, when the above-mentioned non-image guidance is performed, CPU 11 outputs the sound required for the travel guidance via speaker 13. Furthermore, even when the above-mentioned image-using guidance is performed, CPU 11 outputs the sound required for the travel guidance via speaker 13. At this time, in the guidance control process according to the embodiment, various differences are provided between the content of the sound required for the image-using guidance and the content of the sound required for the above-mentioned non-image-using guidance, due to the fact that the non-image-using guidance is travel guidance that requires only sound, as described above.

[0031] Finally, the schematic configuration of the smartphone SP according to the embodiment will be described with reference to FIG. 3(c). As shown in FIG. 3(c), the smartphone SP according to the embodiment includes an interface 20, a CPU 21, a display 22 such as a liquid crystal display, an operation unit 23 such as an operation button or a touch panel, a speaker 24, a sensor unit 25 such as a GPS sensor, and a memory 26 such as a ROM and RAM. In this configuration, the interface 20 controls the exchange of messages and various data with the server device S via the network NW under the control of the CPU 21. The memory 26 also stores non-volatile programs corresponding to the processes to be executed by the smartphone SP in the guidance control process according to the embodiment, and temporarily stores various data necessary for executing the guidance control process. Meanwhile, when a user performs an operation on the operation unit 23 to instruct the operation of the smartphone SP, the operation unit 23 generates an operation signal corresponding to the operation and outputs the operation signal to the CPU 21. As a result, based on the operation signal, the CPU 21 uses the programs and data stored in the memory 26 to exchange necessary messages and data with the server device S, and mainly executes the processes to be executed by the smartphone SP among the guidance control processes according to the embodiment. At this time, the processes include a process of detecting the current location of the smartphone SP using the GPS sensor. Furthermore, when performing the image-based guidance, the CPU 21 displays a guidance image required for the travel guidance on the display 22 and outputs audio required for the travel guidance via the speaker 24.

[0032] Next, the guidance control process according to the embodiment, which is executed by the server device S, etc., will be explained using Fig. 4. In Fig. 4, the flow of each message exchanged between the server device S and the smart speaker SK, etc., is indicated by dashed arrows, and the flow of data necessary for the image use guidance and the image non-use guidance, etc., is indicated by open double-headed arrows. Furthermore, the processes in the server device S, smart speaker SK, and smartphone SP shown in Fig. 4 are started, for example, when the power of each of these server devices S, etc. is turned on.

[0033] As shown in FIG. 4, when the guidance control process according to the embodiment is started, the CPU 11 of the smart speaker SK and the CPU 21 of the smartphone SP first perform a process of detecting the current location of the smart speaker SK using the sensor unit 16 and a process of detecting the current location of the smartphone SP using the sensor unit 25, respectively. Thereafter, the CPU 11 monitors whether a preset timing has arrived for transmitting current status information, including current location information indicating the detected current location and a user ID for identifying the user of the smart speaker SK, to the server device S (step S25). The CPU 21 also monitors whether a preset timing has arrived for transmitting current status information, including the current location information indicating the detected current location, the user ID for identifying the user of the smartphone SP, and the display status information indicating whether the display 22 itself is functioning and a guidance image is being displayed, to the server device S (step S15). If the timing has not arrived during the monitoring in steps S15 and S25 (NO in steps S15 and S25), the CPU 11 and the CPU 21 continue to monitor for the arrival of the timing. On the other hand, if the timing has arrived in the monitoring of steps S15 and S25 (step S15: YES and step S25: YES), the CPU 11 and the CPU 21 transmit the current status information of the smartphone SP and the smart speaker SK to the server device S via the network NW, respectively (steps S16 and S26). The generation and transmission of the current status information in steps S15, S16, S25, and S26 are continuously performed at the above-mentioned timings at preset intervals, regardless of the execution of the guidance control process according to the embodiment. Thereafter, the CPU 21 proceeds to step S17, which will be described later. Next, the CPU 11 monitors whether or not the user of the smart speaker SK has instructed the start of the guidance control process according to the embodiment (step S27). At this time, the start of the guidance control process is instructed by, for example, the user uttering a predetermined start instruction voice indicating the start of the guidance control process via the microphone 15.If the instruction to start is received in the monitoring of step S27 (step S27: YES), the CPU 11 transmits a guidance start message indicating that the instruction to start has been received from the smart speaker SK to the server device S via the network NW (step S28). Thereafter, the CPU 11 proceeds to step S29, which will be described later. On the other hand, if the instruction to start is not received in the monitoring of step S27 (step S27: NO), the CPU 11 proceeds to other processing as the smart speaker SK.

[0034] On the other hand, when the guidance control process according to the embodiment is started, the processing unit 4 of the server device S monitors whether the current status information has been transmitted from either or both of the smartphone SP and the smart speaker SK via the network NW (step S1). If the current status information has not been received in the monitoring of step S1 (step S1: NO), the processing unit 4 continues the monitoring. On the other hand, if any current status information has been received in the monitoring of step S1 (step S1: YES), the processing unit 4 associates each received current status information with the corresponding user ID and records it in the recording unit 5 (step S2). This allows the server device S to grasp, for each user, the current positions of the smartphone SP and the smart speaker SK and whether a guide image is displayed on the smartphone SP. Note that the receiving and recording processes of the current status information in steps S1 and S2 are performed when each piece of current status information is transmitted, regardless of the execution of the guidance control process according to the embodiment.

[0035] Next, the processing unit 4 monitors whether the guidance start message is transmitted from the smart speaker SK via the network NW (step S3). If the guidance start message is not received during the monitoring in step S3 (step S3: NO), the processing unit 4 proceeds to other processing as the server device S.

[0036] On the other hand, if a guidance start message is received from the smart speaker SK during monitoring in step S3 (step S3: YES), the processing unit 4 then determines whether the user of the smart speaker SK that sent the received guidance start message is the same as the user of the smartphone SP that sent the current status information in step S16, based on the user ID included in the current status information recorded in steps S1 and S2 (step S4).If the determination in step S4 is that the users of the smart speaker SK that sent the message and the smartphone SP are not the same (step S4: NO), the processing unit 4 proceeds to step S9, which will be described later.

[0037] On the other hand, if the determination in step S4 indicates that the smart speaker SK that sent the guidance start message and the smartphone SP that sent the current status information are the same user (step S4: YES), the processing unit 4 then determines whether the distance between the smart speaker SK that sent the message and the smartphone SP is equal to or less than a predetermined threshold distance based on the current location information included in the current status information recorded in steps S1 and S2 (step S5). In this case, the threshold distance is, for example, a predetermined threshold distance as a distance at which the smart speaker SK and the smartphone SP can be considered to be present (placed) in a single vehicle C. This threshold distance is, for example, determined in advance as a design parameter, and is, for example, 2 meters. If the determination in step S5 indicates that the distance between the smart speaker SK that sent the message and the smartphone SP is greater than the threshold distance (step S5: NO), the processing unit 4 proceeds to step S9, which will be described later. On the other hand, if the determination in step S5 indicates that the distance between the smart speaker SK that is the sender of the current state information and the smartphone SP is equal to or less than the threshold distance (step S5: YES), the determination unit 2 of the processing unit 4 then determines whether a guide image is displayed on the display 22 of the smartphone SP that is the sender of the current state information, based on the display state information of the smartphone SP included in the current state information recorded in steps S1 and S2 (step S6). If the determination in step S6 indicates that a guide image is displayed on the display 22 (step S6: YES), the control unit 3 of the processing unit 4 executes image use guidance according to the embodiment using the guide image displayed on the display 22 by exchanging image data for image use guidance with the smartphone SP and audio data for image use guidance with the smart speaker SK (step S7; see steps S18 and S33 described later). Note that in the image use guidance according to the embodiment, the audio guidance required for the image use guidance is output from the speaker 13 of the smart speaker SK, as described later, rather than from the speaker 24 of the smartphone SP.Furthermore, before starting transmission of the image usage guidance audio data to the smart speaker SK, the control unit 3 transmits an image usage guidance instruction message to the smart speaker SK via the network NW, indicating that image usage guidance according to the embodiment will be performed using the smart speaker SK (step S7; see step S29 described later). Thereafter, the processing unit 4 determines whether or not to terminate the guidance control process according to the embodiment, for example, because the vehicle C driven by the user of the smartphone SP and smart speaker SK has arrived at the destination (step S8). If the determination in step S8 is that the guidance control process should not be terminated (step S8: NO), the processing unit 4 returns to step S4 and repeats the process to be executed by the server device S among the guidance control process according to the embodiment. On the other hand, if the determination in step S8 is that the guidance control process should be terminated (step S8: YES), the processing unit 4 proceeds to another process as the server device S. Note that the other process in this case also continues to include the process of receiving and recording the current status information.

[0038] On the other hand, after transmitting the current status information as the smartphone SP in step S16, the CPU 21 of the smartphone SP next checks again whether a guide image is displayed on the display 22 (step S17). If the check in step S17 finds that the guide image is displayed (step S17: YES), the CPU 21 executes the image use guide according to the embodiment using the guide image by exchanging the image use guide image data with the server device S (step S18; see step S7). Thereafter, the CPU 21 determines whether to terminate the guidance control process according to the embodiment, for example, because the vehicle C has arrived at the destination (step S19). If the determination in step S19 is that the guidance control process should not be terminated (step S19: NO), the CPU 21 returns to step S17 and repeats the process to be executed by the smartphone SP in the guidance control process according to the embodiment. On the other hand, if the determination in step S19 is that the guidance control process should be terminated (step S19: YES), the CPU 21 proceeds to another process as the smartphone SP. In this case, the other processes also include the process of generating and transmitting the current status information.

[0039] Here, in the image usage guidance according to the embodiment (steps S7, S20, and S33), for example, in a search process for a location for guidance, a preset number of search results are displayed on the display 22 of the smartphone SP in descending order of the search count indicated by the search count information, using the image usage guidance image data based on the search count information recorded in the recording unit 5 of the server device S. Furthermore, as the search process, either the most recent location or the location with the most search count in the search history of the user of the smartphone SP may be displayed on the display 22 based on the search history information recorded in the recording unit 5. Furthermore, for example, as a process for searching for a route to a destination and an actual process for guiding the user to the destination, images related to the search process and images related to the guiding process using the image usage guidance image data transmitted from the server device S are displayed on the display 22 of the smartphone SP using the image usage guidance image data. At this time, the audio required for the search process, the search process, and the guiding process is output from the speaker 13 of the smart speaker SK, as described below, using the image usage guidance audio data transmitted from the server device S.

[0040] If the CPU 21 determines in step S17 that the display status information indicates that the guide image is no longer displayed for some reason (step S17: NO), the processing unit 4 of the server device S generates a guide image display instruction message to display the guide image on the display 22, and transmits the message to the smartphone SP via the network NW (step S7). Then, the CPU 21 of the smartphone SP in which the guide image was not displayed (step S17: NO) monitors for receipt of the guide image display instruction message (step S20). If the guide image display instruction message is received (step S20: YES), the CPU 21 displays the guide image on the display 22 (step S21), and then proceeds to step S18. On the other hand, even if the guide image was not displayed, if the guide image display instruction message is not received (step S20: NO), the CPU 21 proceeds to other processing as the smartphone SP without executing image use guidance.

[0041] Next, in the determination of step S6 above, if a guide image is not displayed on the display 22 of the smartphone SP or the display 22 itself is not functioning (step S6: NO), the control unit 3 of the processing unit 4 of the server device S transmits an image non-use guide instruction message to the smart speaker SK via the network NW to perform image non-use guide according to the embodiment using only the smart speaker SK (step S9). Thereafter, the control unit 3 executes the image non-use guide according to the embodiment using the smart speaker SK by exchanging image non-use guide data with the smart speaker SK (step S10). Thereafter, the processing unit 4 proceeds to step S8 above.

[0042] On the other hand, after transmitting the guidance start message as the smart speaker SK in step S28, the CPU 11 of the smart speaker SK then monitors whether or not the image usage guidance instruction message has been received (step S29; see step S7). If the image usage guidance instruction message has been received during the monitoring in step S29 (step S29: YES), the CPU 11 executes the image usage guidance according to the embodiment by transmitting and receiving the image usage guidance audio data to and from the server device S (step S33; see steps S7 and S18).

[0043] Here, in the image use guidance according to the embodiment as a smart speaker SK, for example, in the search process for a guidance point, only the image for the search process is displayed on the display 22 of the smartphone SP as described above, and the audio for the search process is output from the speaker 13 of the smartphone SP. Similarly, in the search process and guidance process as the image use guidance according to the embodiment, the images related to the search process and the images related to the guidance process are displayed on the display 22 of the smartphone SP, and the audio required for the search process and the guidance process is output from the speaker 13 of the smart speaker SK.

[0044] Thereafter, the CPU 11 determines whether or not to terminate the guidance control process according to the embodiment, for example, because the vehicle C has arrived at the destination (step S32). If the determination in step S32 is that the guidance control process should not be terminated (step S32: NO), the CPU 11 returns to step S29 and repeats the process to be executed by the smart speaker SK among the guidance control processes according to the embodiment. On the other hand, if the determination in step S32 is that the guidance control process should be terminated (step S32: YES), the CPU 11 proceeds to other processes as the smart speaker SK. Note that the other processes in this case also include the ongoing generation and transmission processes of the current status information.

[0045] On the other hand, if the image use guide instruction message is not received during the monitoring of step S29 (step S29: NO), then CPU 11 monitors whether or not the image non-use guide instruction message has been received (step S30; see step S9). If the image non-use guide instruction message is received during the monitoring of step S30 (step S30: YES), CPU 11 executes the image non-use guide according to the embodiment by exchanging audio data for the image non-use guide with server device S (step S31; see step S10). Thereafter, CPU 11 proceeds to step S32. On the other hand, if the image non-use guide instruction message is not received during the monitoring of step S30 (step S30: NO), CPU 11 returns to step S29 and continues the operation of confirming the reception of the image use guide instruction message or the image non-use guide instruction message.

[0046] Here, in the image-free guidance according to the embodiment (see steps S10 and S31), for example, in the search process for a location for guidance, based on the search count information recorded in the recording unit 5 of the server device S, only the search result with the most searches indicated by the search count information is output by speech from the speaker 13 of the smart speaker SK. In this case, the guidance image is not used. Also, as the search process, based on the search history information recorded in the recording unit 5, only speech indicating the most recent location or the location with the most searches in the search history of the user of the smartphone SP may be output from the speaker 13. Furthermore, it may be configured to output only the search result with the highest score as a result of applying a preset weighting.

[0047] Furthermore, for example, in the process of searching for a route to a destination, it is desirable to configure the route to be determined so as to reduce the number of travel methods that are difficult to guide using only audio from the smart speaker SK, rather than using the above-mentioned image guidance. One example of a method for determining such a route is a method for determining a route that minimizes the sum of the link costs assigned to the roads that make up the route and the turn costs assigned to the intersections that make up the route. Note that such a route determination method is described in detail in, for example, Japanese Patent No. 4361595 by the applicant of the present application.

[0048] Furthermore, for example, the actual guidance process to the destination may involve using only audio from the smart speaker SK to guide the user to the location in one of the following ways i) to iv): i) The number of times audio for location guidance is output from the smart speaker SK is increased compared to the case of image-based guidance. ii) If the point to be guided is an intersection, the name of the intersection itself is output as voice from the smart speaker SK. iii) A voice indicating a feature indicating the point to be guided or a feature corresponding to the point that is actually visible from the vehicle C is output from the smart speaker SK. iv) If the point to be guided is an intersection, the smart speaker SK outputs a voice indicating the branch that the vehicle C should take at the intersection in relation to other branches at the intersection.

[0049] Specifically, as for the guidance method i) above, for example, in image-based guidance, a guidance voice is output three times when the distance to the destination point is 700 meters, 300 meters, and just before that, whereas in the non-image-based guidance of the embodiment, in addition to the three times above, a guidance voice is output even when the distance to the destination point is 1,000 meters.

[0050] As a specific example of the guidance method of iv) above, when vehicle C needs to be guided to a diagonally right branch between a right-angle branch and a straight-ahead branch at an intersection to be guided, the smart speaker SK may output a guidance voice such as, "Please enter the second road diagonally to the right from the road to turn right."

[0051] In the case of the image use guidance according to the embodiment, the above i) to iv) are not executed, or the frequency of execution is reduced compared to the non-image use guidance, or the amount of information of the guidance is reduced compared to the non-image use guidance. Therefore, in these cases, the user does not feel bothered by the execution of the above i) to iv) when the guide image can be used in combination.

[0052] Furthermore, when a preset condition is met, such as when the guidance process using only voice becomes extremely complicated (for example, when there is an intersection with many branches or when the target feature is complex, or when the target feature itself cannot be seen from the vehicle C), the control unit 3 of the processing unit 4 of the server device S can be configured to use the image-based guidance in combination by sending the guidance image display instruction message to the smartphone SP that should be in the same vehicle C as the smart speaker SK, and forcibly displaying the guidance image on its display 22 (see step S23 above). Note that "when the guidance process using only voice becomes extremely complicated" in this case refers to, for example, an intersection with many branches, when the target feature is complex, or when the target feature itself cannot be seen from the vehicle C.

[0053] In the guidance control process according to the embodiment, guidance is performed in steps S18, S31, S33, etc., but thereafter, as described above, until the end of the guidance, the generation and transmission of the current status information in steps S15, S16, S25, and S26, and the reception and recording of the current status information in steps S1 and S2 are continuously performed at the predetermined intervals. Therefore, even if image-use guidance is being performed, if it is determined that the conditions for image-use guidance are no longer met, for example, if the smartphone SP is subsequently turned off or the smartphone SP and the smart speaker SK are outside a predetermined range (for example, a positional relationship greater than a predetermined threshold distance), it is also possible to configure the system to switch to image-use guidance. Conversely, if the conditions for image-use guidance are met during image-use-free guidance, the system can also be configured to switch to image-use guidance.

[0054] As described above, according to the guidance control process of the embodiment, the audio output mode from the smart speaker SK is changed depending on the determination result of whether the smartphone SP that displays the guidance image and the smart speaker SK that outputs audio for travel guidance are within a range equal to or less than the threshold distance (see step S5 in FIG. 4). Therefore, necessary information can be provided more accurately using audio alone depending on whether the separate smartphone SP and the smart speaker SK are both within the threshold distance range.

[0055] Note that the means for determining whether the smartphone SP and the smart speaker SK are within a predetermined range is not limited to the distance-based method. For example, the predetermined range can be set as a specific space. For example, the specific space can be set as the inside of a specific automobile, and specifically, specific radio waves can be generated inside the vehicle and technically devised to prevent the radio waves from being generated outside the vehicle (for example, to prevent leakage from the vehicle). Furthermore, by providing sensors to the smartphone SP and the smart speaker SK that detect the specific radio waves and transmitting the detection results of the sensors to the server device S as part of current status information, the server device S can determine whether the smartphone SP and the smart speaker SK are both within the specific automobile.

[0056] Furthermore, if the audio output mode from the smart speaker SK is changed depending on the determination result of the identity of the user of the smartphone SP and the user of the smart speaker SK (see step S4 in Figure 4) and the determination result of whether or not a guidance image is displayed when both the smartphone SP and the smart speaker SK are within a threshold distance range (see step S5 in Figure 4), the necessary information can be accurately provided by audio alone, taking into account the identity of the user.

[0057] Furthermore, it is determined whether a guide image for search or guidance is displayed (see step S6 in Figure 4), and if the smartphone SP and smart speaker SK are located within the above-mentioned threshold distance and the audio output mode from the smart speaker SK is changed depending on the determination result of whether the guide image is displayed or not, the necessary information can be accurately applied by audio depending on whether the guide image is displayed or not.

[0058] Furthermore, when it is determined that both the smartphone SP and the smart speaker SK are within a threshold distance range and a guidance image is being displayed, if an image containing content related to the audio output mode from the smart speaker SK is displayed on the display 22 of the smartphone SP, the image can be used to provide more accurate information.

[0059] Furthermore, when a guidance image is displayed when the smartphone SP and the smart speaker SK are both within a threshold distance range, and it is determined that the user of the smartphone SP is the same as the user of the smart speaker SK, if an image containing content related to the audio output mode from the smart speaker SK is displayed on the display 22 of the smartphone SP, information can be provided more accurately using the image, taking into account the identity of the user.

[0060] Furthermore, if the audio output mode from the smart speaker SK is an output mode for navigation, even if the guide image on the smartphone SP is not displayed when performing navigation, the necessary information can be provided more accurately using audio alone.

[0061] Furthermore, if the above navigation includes a route setting process, even if the guide image on the smartphone SP is not displayed when performing navigation including a route setting process, the necessary information can be provided accurately by voice alone.

[0062] Furthermore, if the above navigation includes route guidance processing, even if the guide image on the smartphone SP is not displayed when performing navigation including route guidance processing, the necessary information can be provided accurately using only voice.

[0063] Furthermore, if the audio output mode from the smart speaker SK is a search output mode, even if the guide image on the smartphone SP is not displayed when performing a search, the necessary information can be provided more accurately through audio alone.

[0064] Furthermore, if the audio output mode from the smart speaker SK is an output mode for location search, even if the guide image on the smartphone SP is not displayed when performing a location search, the necessary information can be provided more accurately using audio alone.

[0065] In addition, in the case of image-free guidance in the embodiment, audio containing only one relevant test result is output as the search result for a location, etc., so even if the guidance image is not displayed, the search result can be accurately output in a manner that is easy for the user of vehicle C to recognize.

[0066] Furthermore, in the case of the non-image guidance in the embodiment, based on search count information indicating the past search count for each of the multiple search results obtained by multiple search methods, audio is output that includes only the search result with the largest search count among the multiple search results, so that even if the guidance image is not displayed, more general and accurate search results can be output.

[0067] On the other hand, in the case of the non-image guidance of the embodiment, if audio containing only one test result is output based on search history information indicating the past search history of at least one of the user of the smartphone SP or the user of the smart speaker SK, accurate search results suitable for either user can be output even if the guidance image is not displayed.

[0068] Furthermore, when providing location guidance by voice as in the above i) to iv), even if the guidance image is not displayed, location information can be accurately output by voice alone.

[0069] Furthermore, in the case of guidance without using images according to the embodiment, the route that vehicle C should travel is determined so that there are fewer travel methods that are difficult to guide using only the voice guidance for travel compared to when guidance is given while displaying a guidance image, and when vehicle C is guided using only voice according to the determined route, the route is determined so that there are fewer travel methods that are difficult to guide using only voice, so guidance can be given more accurately using only the voice.

[0070] Furthermore, in the embodiment, a guide image is displayed when a predetermined condition is met during the non-image guidance (see step S21 in Figure 4), so by displaying a guide image when necessary, more accurate guidance can be achieved.

[0071] In the above-mentioned examples, we have explained the image use guidance and image non-use guidance when the smartphone SP and smart speaker SK of the examples are placed in a vehicle C, but in addition to this, the present application can also be applied to the image use guidance and image non-use guidance when a portable wireless telephone (mobile phone) with the same functions as a smartphone SP and a tablet terminal with a speaker are placed in a single vehicle.

[0072] In the above-described embodiment, whether the smartphone SP and the smart speaker SK are within a range of a threshold distance or less is determined using their respective current location information (see step S5 in FIG. 4). Alternatively, whether the smartphone SP and the smart speaker SK are within a range of a threshold distance or less can be determined based on whether they are connected via so-called short-range wireless communication. In this case, the distance corresponding to the connection limit of the short-range communication can be made to correspond to the threshold distance.

[0073] Furthermore, in the voice guidance when the image is not used, an additional explanation may be added by voice to help the user understand the difference between so-called homonyms.

[0074] Furthermore, in the above-described embodiment, a case has been described in which a smart speaker SK and a smartphone SP are present inside a vehicle C, but the present application can also be applied to moving bodies such as automobiles, motorcycles, and pedestrians.

[0075] Furthermore, among steps S4 to S6 shown in FIG. 4, the determination of whether to execute image use guidance or image non-use guidance may be made only by the determination of step S5 without making the determinations of steps S4 and S6.

[0076] Furthermore, the search examples do not necessarily apply only to searches related to directions.

[0077] Furthermore, a program corresponding to the flowchart showing the guidance control processing to be executed by the server device S shown on the left side of Figure 4 can be recorded on a recording medium such as an optical disk or a hard disk, or obtained via a network such as the Internet, and then read and executed by a general-purpose microcomputer or the like, thereby causing the microcomputer or the like to function as the processing unit 4, judgment unit 3, and control unit 3 of the server device S according to the embodiment. [Explanation of symbols]

[0078] 1. Connection method (interface) 2 Judgment means (judgment section) 3 Control means (control unit) 4 Processing section 11, 21 CPUs S Guidance control device (server device) C. Mobile object (vehicle) SS Guidance and Control System SK output device (smart speaker) SP display device (smartphone)

Claims

[Claim 1] a connection means capable of connecting to a display device for displaying images and an output device for outputting audio via communication; a determination means for determining whether the display device and the output device are within a predetermined range; a control means for changing an output mode of the audio according to a result of the determination by the determination means; A guidance control device comprising:

Images