Controller and control method

Abstract

To provide a controller that proposes a result of executing an application input by one user, to another user, and allows the other user to properly approve the proposal.SOLUTION: A controller displays a first screen and a second screen, for the same application, the first screen being a user interface for one user, the second screen being a user interface for another user, executes the application, and displays the result of executing the application on the first screen. When receiving a second instruction different from a first instruction on the second screen during a period from displaying the second screen until receiving the first instruction on the second screen, the controller displays, on the second screen, a result of temporarily executing the application displayed on the first screen on the basis of the second instruction. When receiving the first instruction on the second screen, the controller displays, on the first screen, a screen to inquire as to whether to reflect the result of the temporary execution on the second screen, in the first screen.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to the control of applications.

Background Art

[0002] For example, Patent Documents 1 and 2 disclose a technique of reflecting the execution result of an application performed in response to an input made to a user interface for one user on a user interface for another user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, while the driving load determination unit determines that the driving load of the driver is high, notification based on the output from the cooperative device due to the operation by the non-driving passenger is not performed, and it is disclosed that the output is stored in the internal memory. When the driving load of the driver becomes low, it is disclosed that notification based on the output stored in the internal memory is performed. However, when the output based on the operation by the non-driving passenger, who is one user, goes against the will of the driver, who is another user, the driver needs to perform a complicated operation on the output, which imposes a burden on the driver.

[0005] In Patent Document 2, in a configuration where a processing unit is provided in the driver's seat and another processing unit is also provided in the passenger seat, the processing unit in the driver's seat can transmit and receive an operation process with the processing unit in the passenger seat. Thereby, it is disclosed that the driver can share one operation process with the passenger. However, according to this technique, the driver, who is another user, also has to perform complicated operations such as POI addition and route change, which imposes a burden on the driver.

[0006] An object of the present invention is to propose the execution result of an application input by one user to another user and enable the other user to appropriately approve the proposal.

Means for Solving the Problems

[0007] The control device displays a first screen that is a user interface for one user and a second screen that is a user interface for another user for the same application, executes the application, and displays the application execution result on the first screen. When the control device receives a second instruction different from the first instruction on the second screen during the period from when the second screen is displayed until it receives the first instruction for the second screen, it displays on the second screen the result of temporarily executing the application displayed on the first screen based on the second instruction. When the control device receives the first instruction for the second screen, it displays on the first screen an inquiry as to whether to reflect the result of the temporary execution on the second screen on the first screen.

Effects of the Invention

[0008] According to the present invention, the execution result of an application input by one user can be proposed to another user, and the other user can appropriately approve the proposal. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 3E

Figure 3F

Figure 4

Embodiments for Carrying Out the Invention

[0010] Hereinafter, an embodiment will be described with reference to the drawings. In the following description, when describing without distinguishing between elements of the same kind, common reference numerals among the reference numerals are used, and when distinguishing between elements of the same kind, reference numerals may be used.

[0011] FIG. 1 is a configuration diagram of a control device according to an embodiment.

[0012] The control device 100 includes an interface device 101, a storage device 102, and a processor 103 connected thereto. In the present embodiment, the control device 100 is a vehicle device in the vehicle 50, but the control device according to the present invention may be applied to devices other than in-vehicle devices. For example, some functions of the control device 100 may be provided in a device outside the vehicle 50 that can communicate with the in-vehicle device. In this case, the control device 100 may be a device composed of an in-vehicle device and a device outside the vehicle 50. An example of the "device outside the vehicle 50" is a server device on a communication network. The control device according to the present invention may be applied to a mobile information processing device.

[0013] The interface device 101 is an interface for various devices provided in the vehicle 50. For example, the interface device 101 is connected to a sonar 121, a camera group 122, an operation device 123 (typically a steering wheel), a drive device 124 (typically a power train), a brake control device 125 (typically a brake), a touch panel 131, an operation button group 132, and a position detection device 133 in the vehicle 50. The camera group 122 is one or more cameras. The operation button group 132 is one or more operation buttons. The position detection device 133 is a device that detects the position of the vehicle 50 based on GNSS (global navigation satellite system) signals from satellites. The sonar 121 and the camera group 122 are each an example of a sensor for detecting the situation around the vehicle 50. The touch panel 131 and the operation button group 132 are each an example of a UI (User Interface) device such as a display device and an input device. The operation button group 132 may be absent, and all operations may be performed using the touch panel 131. Also, a display device other than the touch panel 131 may be adopted, and input may be performed via the operation button group 132 or other UI devices. The structure of the touch panel 131 may be either an on-cell structure or an in-cell structure. For example, a touch panel with an on-cell structure has a display portion capable of displaying an image, and an image is displayed on the display portion.

[0014] The storage device 102 includes at least a memory among a memory and a persistent storage device. The memory may be a volatile memory or a non-volatile memory. The persistent storage device may be a non-volatile storage device (e.g., a Hard Disk Drive (HDD) or a Solid State Drive (SSD)). The storage device 102 stores computer programs and data. For example, information representing camera images, sonar detection information, and information representing a parking route described later are stored in the storage device 102. Also, as an example of a computer program, one or more application programs 154 are stored in the storage device 102. In the present embodiment, "execution of an application" may mean execution of the application program 154, or may mean execution of a partial function (e.g., route guidance or email transmission) in the application program 154. The application program 154 may be, for example, a car navigation, an automatic parking, or an email transmission application.

[0015] The processor 106 may be one or more processor devices, and at least one processor device may be a microprocessor device such as a CPU (Central Processing Unit), but may also be other types of processor devices such as a GPU (Graphics Processing Unit). At least one processor device may be single-core or multi-core, and may also be a processor core. At least one processor device may be a processor device in a broad sense such as a hardware circuit that performs part or all of the processing. This "hardware circuit" may be, for example, an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). By the processor 106 executing a computer program, functions such as a setting unit 151, an execution unit 152, and a determination unit 153 are realized. These functions 151 to 153 are examples, and a plurality of functions may be combined into one function, or one function may be divided into a plurality of functions. These functions 151 to 153 will be described in the description with reference to FIGS. 2 to 4.

[0016] FIG. 2 is a flowchart of the processing performed by the control device 100. FIGS. 3A to 3F show examples of screens displayed on the touch panel 131 when the processing shown in FIG. 2 is applied to navigation as an example of an application.

[0017] In S201, the setting unit 151 displays, on the touch panel 131, a first screen 301A that is a user interface for one user and a second screen 301B that is a user interface for another user for the same application. One user is, for example, the driver of the vehicle 50, and the other user is, for example, a passenger in the vehicle 50. Both the first screen 301A and the second screen 301B are, for example, a GUI (Graphical User Interface).

[0018] For example, the first screen 301A is a user interface for the driver of the vehicle 50, and the second screen 301B is a user interface for the passengers of the vehicle 50. When the driver's seat of the vehicle 50 is installed on the left, the first screen 301A is displayed on the left side of the touch panel 131, and the second screen 301B is displayed on the right side of the touch panel 131. When the driver's seat of the vehicle 50 is installed on the right, it is the reverse. Note that both the first screen 301A and the second screen 301B can be displayed on a display device other than the touch panel 131. For example, the first screen 301A may be displayed on the display device of the driver's mobile terminal, and the second screen 301B may be displayed on the display device of the passenger's mobile terminal.

[0019] Before the process described in FIG. 2 starts, the second screen 301B may not be displayed on the touch panel 131. Alternatively, the first screen 301A and the second screen 301B are displayed on the touch panel 131, and each screen 301 is related to a different application. In this case, the passenger can input a desired instruction for an application different from the application displayed on the first screen 301A, which is displayed on the second screen 301B, and the instruction is not reflected on the first screen 301A. That is, before the process described in FIG. 2 starts, the second screen 301B may have no relevance to the first screen 301A. When an input that triggers the start of the process shown in FIG. 2 is received for the first screen 301A or the second screen 301B, the setting unit 151 performs S201.

[0020] Also, the second screen 301B displayed in S201 is preferably a copy of the first screen 301A (see, for example, FIG. 3A). This is because it is expected that the operations of the passengers on the second screen will not easily deviate from the content displayed on the first screen 301A. The execution unit 152 executes the application and displays the application execution result on the first screen 301A. Since a copy of this first screen 301A is displayed as the second screen 301B, the passengers can input desired instructions on the second screen 301B based on the first screen 301A. According to the example shown in FIG. 3A, on the map displayed on the first screen 301A, as the application execution result, the determined route 310 and the own vehicle mark 311, which is a mark representing the current position and direction of the vehicle 50, are displayed.

[0021] In the period from when the second screen 301B is displayed until a first instruction for the second screen 301B is received (S201 to S204: YES), the execution unit 152 is configured to perform the following (a) and (b). Note that the "first instruction" is an instruction to reflect the result of a simulation temporarily executed based on a second instruction described later on the second screen 301B on the first screen. In other words, it is an instruction to propose the simulation on the second screen 301B to the first screen. For example, when the application is navigation, the first instruction is an instruction to present and propose a new route, which is the result of temporarily executing a simulation based on the second instruction described later on the second screen, on the first screen. The "second instruction" is an instruction different from the first instruction and is an instruction to simulate the operation of the application displayed on the first screen 301A on the second screen. For example, when the application is navigation, the second instruction is an operation added to the map that is a copy of the first screen, and is an instruction to search for a waypoint, add the waypoint to the route, and search for a new route passing through the added waypoint. Even if a second instruction is input to the second screen, the simulation on the second screen is not reflected on the first screen until a first instruction is input.

[0022] (a) While the execution unit 152 is in the period from when the second screen 301B is displayed until it receives the first instruction for the second screen 301B (S201 to S204: YES), every time a display change of a predetermined type is made on the first screen 301A, the display change of the predetermined type is reflected on the second screen 301B. Here, "reflecting the display change of the predetermined type on the second screen 301B" means reflecting only a part of the display changes corresponding to the predetermined type among the display changes on the first screen 301A on the second screen 301B. The display changes of the predetermined type can be defined in advance for each application.

[0023] Even if a display change other than the predetermined type is made on the first screen 301A, the display change other than the predetermined type is not reflected on the second screen 301B.

[0024] For example, when the application is a car navigation, the display change of the predetermined type is the change in the display position of the own vehicle mark 311 on the map, and the display change other than the predetermined type is the viewpoint switching of the map or the scale change of the map. This can prevent the change in the viewpoint or scale of the map on one screen from affecting the map displayed on the other screen. When the application is an automatic parking, the display change of the predetermined type is the update of the image captured by the camera group 122, and the display change other than the predetermined type is the viewpoint switching of the image or the camera switching. This can prevent the viewpoint switching of the image or the camera switching on one screen from affecting the other screen. When the application is a message sending application, there is no display change of the predetermined type, and the display change other than the predetermined type is the enlargement or reduction of the text surface and the change in the position of the cursor.

[0025] (b) When the execution unit 152 receives a second instruction for the second screen 301B in S202, in S203, it displays on the second screen 301B the result of temporarily executing the second instruction on the second screen 301B which is a copy of the first screen 301A. The execution unit 152 can simulate the operation of the application on the second screen 301B based on the second instruction, and the result of the simulation does not affect the first screen 301A. This is the temporary execution. When the execution 152 does not receive a second instruction for the second screen 301B in S202, it returns to S201 and displays the second screen 301B.

[0026] For example, when the application is a car navigation, the execution unit 152 can receive an instruction for switching the viewpoint of the image in S203. When receiving the instruction, the execution unit 152 executes, as an example of the temporary execution, the switching of the viewpoint of the image as shown in FIG. 3B, and displays the image after the viewpoint switching on the second screen 301B.

[0027] Also, when the application is a car navigation application, in S203, the execution unit 152 accepts an instruction to add a via point to the second screen 301B as an instruction for virtual execution to simulate the application displayed on the first screen 301A on the second screen 301B. In this case, the execution unit 152 executes the following processing. First, the execution unit 152 displays via point candidates on the second screen 301B. Next, as shown in FIG. 3C, when a selection of a via point is accepted via the second screen 301B, the execution unit 152 searches for a new route including the selected via point based on the route determined as the application execution result of the instruction for the first screen 301A. Next, as shown in FIG. 3D, the execution unit 152 displays the found new route 360 on the second screen 301B, and also displays a "Route Proposal" button 351 and a "Cancel" button 352 on the second screen 301B. The "Route Proposal" button 351 is an example of a GUI component for accepting a route proposal instruction, and the route proposal instruction is an example of a first instruction. The "Cancel" button 352 is an example of a GUI component for accepting an instruction to cancel the addition of a via point. If the cancellation of the addition of the via point is maintained, the process may return to S201. The addition of the via point and the search for a new route including the added via point are examples of virtual execution, and the added via point and the new route including the via value are examples of the results of virtual execution.

[0028] When the input of the first instruction is accepted on the second screen 301B (S204: YES), the execution unit 152 displays on the first screen 301A a screen for asking whether to commit to reflecting the result of the simulation virtually executed based on the second instruction on the second screen 301B (S205). For example, this inquiry screen may be displayed on the first screen 301A. The inquiry screen may be, for example, a pop-up.

[0029] For example, when the application is for navigation, the execution unit 152 determines whether it has received a press of the "Route Proposal" button 351 on the second screen 301B as a first instruction to reflect the result of the simulation temporarily executed based on the second instruction on the first screen 301A. When the "Route Proposal" button is pressed as the first instruction (S204: YES), the flow proceeds to S205. In S205, as shown in FIG. 3E, the execution unit 152 displays a pop-up 400 on the first screen 301A, which is a screen for querying whether to reflect the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen 301A. The pop-up 400 is an example of a user interface that receives whether to accept or reject reflecting the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen 301A. As shown in FIG. 4, the pop-up 400 has text for querying whether to reflect the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen, a "Yes" button 401 meaning acceptance, and a "No" button 402 meaning rejection. In addition to these texts, the pop-up 400 includes text representing the difference between the result of the temporary execution at the time when the first instruction was received via the second screen 301B and the result of the application execution before the temporary execution.

[0030] In S205, the determination unit 153 determines whether it has received acceptance of reflecting the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen 301A for the screen that queries whether to reflect the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen 301A and displayed in S205. When acceptance is received (S206: YES), in S207, the execution unit 152 reflects the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen 301. When the result of the temporary execution is reflected in the display of the first screen 301A, or when the query on the first screen 301A is rejected without acceptance, the execution unit 152 deletes the simulation displayed on the second screen 301B (S208).

[0031] According to the example shown in FIG. 4, in S206, the determination unit 153 determines whether or not it has received, via the first screen 301A, the pressing of the "Yes" button 401 that commits to reflecting the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen. When the determination unit 153 determines that the "Yes" button 401 has been pressed (S206: YES), in S207, the execution unit 152 reflects, on the first screen 301A, the result of the simulation temporarily executed based on the second instruction on the second screen 301B. When the determination unit 153 determines that the "No" button 402 has been pressed (S206: NO), the execution unit 152 deletes the simulation displayed on the second screen 301B (S208). When the application is for navigation, as shown in FIG. 3F, the route 310 displayed on the first screen 301A is overwritten with a new route 360 to continue the navigation. Note that, thus, when the result of the temporary execution is reflected in the display of the first screen 301A, or when the temporary execution is canceled, in S208, the execution unit 152 may cancel the display of the second screen 301B as shown in FIG. 3F.

[0032] As described above, according to the present embodiment, a screen (for example, the pop-up 400) for inquiring whether or not to reflect the new route searched as a result of the temporary execution is displayed on the first screen 301A. If the driver answers affirmatively to the inquiry as to whether or not to reflect the new route searched as a result of the temporary execution (for example, by pressing the "Yes" button 401 in the pop-up 400), the result of the simulation temporarily executed based on the second instruction on the second screen 301B is reflected on the first screen 301 (for example, the first screen 301A is overwritten with the new route and the route based on the new route is guided). That is, it is avoided that the result of the temporary execution is reflected on the first screen 301A against the driver's will. Further, since the answer only needs to be whether or not to approve the temporary execution, the burden on the driver is small.

[0033] When the execution unit 152 receives the first instruction for the second screen 301B, in addition to the screen that inquires whether to reflect the result of the trial execution on the second screen 301B on the first screen 301A, the difference between the result of the trial execution (for example, a new route) at the time when the first instruction is received via the second screen and the application execution result before the trial execution (for example, the current route) may be displayed on the first screen 301A. Thereby, the driver can easily understand the difference between the result of the trial execution and the application execution result before the trial execution, and thus, the burden of making a decision on whether to approve is small, and the burden on the driver is smaller. In FIG. 4, the text "The increased time is XX minutes" represents the difference, and the increased time is the difference between the predicted travel time based on the distance of the new route and the predicted travel time based on the current route, and may be calculated by the execution unit 152.

[0034] A determination unit 153 may be provided to determine whether it is approved to reflect the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen. When the determination unit 153 determines that it is approved to reflect the result of the simulation temporarily executed based on the second instruction on the second screen 301B on the first screen, the execution unit 152 may reflect the result of the temporarily executed simulation on the first screen 301A and overwrite the first screen 301A with the result temporarily executed on the second screen 301B. Thereby, the intention of the passenger can be conveyed to the driver via the first screen 301A.

[0035] The execution unit 152 may restrict reflecting the result of the simulation temporarily executed and displayed on the second screen 301B on the first screen 301A from when the second screen 301B is displayed until the first instruction for the second screen 301B is received. Thereby, it is possible to prevent the result of the trial execution contrary to the driver's intention from being reflected on the first screen 301A.

[0036] As described above, one embodiment of the present invention has been explained. However, the present invention is not limited to the above-described embodiment and includes various modifications. For example, the above-described embodiment has been explained in detail for easy understanding of the present invention, and is not necessarily limited to the one having all the configurations explained. Also, not only deletion of such a configuration is possible, but also replacement and addition of configurations are possible.

[0037] For example, the difference between the result of the trial execution and the application execution result before the trial execution is the difference between the first screen 301A and the second screen 301B at the time when the first instruction is received. Display changes other than the predetermined type on the second screen are not displayed as differences. Instead of or in addition to the example of adding a waypoint, at least one of the following may be adopted. · When an example of the trial execution according to the second instruction is deletion of the specified waypoint and search for a new route to the destination, the difference may be the shortened time as the difference between the travel time on the route after the waypoint deletion and the travel time on the route before the waypoint deletion. · When an example of the trial execution according to the second instruction is change of the destination and search for a new route to the changed destination, the difference may be the shortened time or increased time as the difference between the travel time on the route after the destination change and the travel time on the route before the destination change. · When an example of the trial execution according to the second instruction is search for a new route without changing the destination and waypoints, the difference may be the shortened time or increased time as the difference between the travel time on the new route and the travel time on the old route. · When an example of the trial execution according to the second instruction is change of the route search conditions (for example, presence or absence of using a toll road), the difference may be the shortened time or increased time as the difference between the travel time and toll on the route after the condition change and the travel time and toll on the route before the condition change, and the decreased amount or added amount.

[0038] Also, the application is not limited to car navigation. For example, at least one of the following may be adopted. · The application may be a message sending application. In this case, when the driver inputs a message on the first screen while the vehicle is stopped, the setting unit may display a second screen that is a copy of the first screen at a timing such as when starting the vehicle. When a passenger edits the message displayed on the second screen and the execution unit receives a first instruction for the second screen, the execution unit may display, on the first screen, an inquiry as to whether to commit to sending the edited message. At this time, together with the inquiry (for example, a pop-up), the execution unit may display, on the first screen, the difference between the message displayed on the first screen and the message displayed on the second screen. The difference in this case may be, for example, a character string that is not displayed on the first screen but is displayed on the second screen, or a character string that is displayed on the first screen but is not displayed on the second screen. · The application may be automatic parking. In this case, the setting unit may display a first screen and a second screen that are images of the vehicle's surroundings taken by the camera and display the same image. The second instruction for the second screen may be that a passenger designates a desired parking position for the second screen. The trial execution in the case where the application is automatic parking is the execution of a simulation that displays the route when the vehicle is automatically parked at the parking position designated based on the second instruction input by the passenger. The first instruction for the second screen may be an instruction to inquire whether to commit to the execution of the automatic parking trial-executed on the second screen. When the first instruction for the second screen is received, the execution unit may display, on the first screen, an inquiry as to whether to commit to parking the vehicle at the parking position designated on the second screen. The difference in this case may be the parking position that is not designated on the first screen but is designated as a result of the trial execution on the second screen.

Explanation of Signs

[0039] 50: Vehicle, 100: Control device, 151: Setting unit, 152: Execution unit, 153: Judgment unit, 154: Application program

Claims

1. A setting unit that displays a first screen which is a user interface for one user and a second screen which is a user interface for another user for the same application, An execution unit that executes the application and displays the application execution result on the first screen Comprising During the period from when the second screen is displayed until the first instruction for the second screen is received, When a second instruction different from the first instruction is received on the second screen, the result of temporarily executing the application displayed on the first screen based on the second instruction is displayed on the second screen, When the execution unit receives the first instruction for the second screen, it displays on the first screen a screen that inquires whether to reflect the result of the temporary execution on the second screen to the first screen Control device

2. When the execution unit receives the first instruction for the second screen, in addition to the screen that inquires whether to reflect the result of the temporary execution on the second screen to the first screen, the difference between the result of the temporary execution at the time when the first instruction is received via the second screen and the application execution result before the temporary execution is displayed on the first screen The control device according to claim 1.

3. A determination unit that determines whether it is approved to reflect the result of the temporary execution on the second screen to the first screen Further comprising When the determination unit determines that the temporary execution is approved, the execution unit reflects the result of the temporary execution on the first screen The control device according to claim 1.

4. During the period from when the second screen is displayed until the first instruction for the second screen is received, the execution unit restricts reflecting the result of the temporary execution displayed on the second screen to the first screen The control device according to claim 1.

5. When the execution unit receives the second instruction for the second screen, it displays on the second screen the result of temporarily executing the second instruction on the second screen which is a copy of the first screen The control device according to claim 1.

6. During the period from when the second screen is displayed until the first instruction for the second screen is received, The execution unit reflects a predetermined type of display change on the first screen to the second screen Do not reflect on the second screen display changes other than the predetermined type on the first screen. The control device according to claim 1.

7. For the same application, display a first screen that is a user interface for one user and a second screen that is a user interface for another user. Execute the application and display the application execution result on the first screen. When a second instruction different from the first instruction is received on the second screen during the period from when the second screen is displayed until a first instruction for the second screen is received, display on the second screen the result of temporarily executing the application displayed on the first screen based on the second instruction. When the first instruction for the second screen is received, display on the first screen a screen for inquiring whether to reflect the result of the temporary execution on the second screen on the first screen. A control method performed by a computer.

Images