Display point registration method and display system for vehicle
The vehicle display system addresses the limitation of existing systems by allowing users to register and view points of interest on a user terminal, enabling safe and convenient automatic image display when the vehicle reaches selected locations, even when the engine is off.
Patent Information
- Application Number
- PCT/JP2023/017064
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-01
- Publication Date
- 2024-11-07
AI Technical Summary
Existing in-vehicle display systems cannot register or display points of interest when the vehicle's engine is off or the user is not inside, limiting user convenience and safety in selecting and viewing registered locations.
A vehicle display system that stores display history information, including positional data of captured images, on a user terminal, allowing users to register and select points of interest even when the engine is off, using a network-connected in-vehicle device, server, and user terminal to synchronize and display images automatically.
Enables users to safely and conveniently register and view points of interest in a calm environment, allowing for automatic image display when the vehicle reaches selected locations, enhancing user experience and safety.
Smart Images

Figure JP2023017064_07112024_PF_FP_ABST
Abstract
Description
Display point registration method and vehicle display system
[0001] The present invention relates to a display point registration method and a vehicle display system.
[0002] Patent Document 1 discloses an in-vehicle display device that selects points that need to be registered as activation points for automatically activating a front camera mounted on a vehicle and presents them to a user. The in-vehicle display device disclosed in Patent Document 1 displays points on the vehicle's travel path that meet predetermined conditions as registration point candidates on a display means, stores a point selected from the displayed registration point candidates as a registration point, and displays an image captured by the front camera when the current position reaches the registration point.
[0003] JP 2012-173269 A
[0004] However, the above-mentioned in-vehicle display device is mounted on a vehicle, and when the vehicle arrives at the destination, the registration location candidates are displayed with the vehicle engine on, and the registration location is registered by displaying the image captured by the front camera. Therefore, when the vehicle engine is off or after the user gets out of the vehicle, the user cannot check the registration location candidates and select a registration location.
[0005] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a display location registration method and a vehicle display system that can register locations where images captured by an onboard camera are automatically displayed in a safer and more relaxed environment.
[0006] A display point registration method according to one aspect of the present invention stores display history information in a storage device, including a history of location information of display points, which are points at which captured images captured by an onboard camera mounted on a vehicle are displayed, displays the display history information on a user terminal carried by a user of the vehicle, and stores location information of a display point selected by the user on the user terminal in the storage device as location information of a registered point, which is a point at which the captured image is automatically displayed.
[0007] A display point registration method according to one aspect of the present invention stores display history information in a storage device, the display history information including a history of location information of a display point at which an image captured by an onboard camera mounted on a vehicle is displayed, and information indicating the display mode when the image was displayed at the display point; outputs the display history information to a terminal device that can be operated by a user of the vehicle; stores the location information included in the display history information selected on the terminal device in the storage device as location information of a registered point, which is a point at which the image is automatically displayed; and stores the display mode included in the display history information selected on the terminal device in the storage device as a registered display mode, which is the display mode when the image is automatically displayed.
[0008] According to the present invention, a user can register a location where an image captured by an in-vehicle camera is automatically displayed in a safer and more relaxed environment.
[0009] FIG. 1 is a diagram schematically illustrating the overall configuration of a vehicular display system according to an embodiment. FIG. 2 is a block diagram illustrating the configuration of a vehicular display system according to an embodiment. FIG. 3 is a timing chart illustrating correspondence between user operations, vehicle operations, and server operations when display history information is stored in a vehicular display system according to an embodiment. FIG. 4 is a timing chart illustrating correspondence between user operations, user terminal operations, and server operations when a registered point is stored in a vehicular display system according to an embodiment. FIG. 5 is a diagram illustrating an example of a screen displaying display history information and driving history information. FIG. 6 is a diagram illustrating an example of a screen displaying details of the display history information. FIG. 7 is a diagram illustrating an example of a screen for a user to select whether to store display history information and to edit the display history information. FIG. 8 is a diagram illustrating an example of a screen for a user to select a display mode. FIG. 9 is a diagram illustrating an example of a screen for a user to edit the position of a display point.
[0010] An embodiment of the present invention will now be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.
[0011] [Overview of Vehicle Display System] FIG. 1 schematically illustrates the overall configuration of a vehicle display system 10 according to one embodiment. The vehicle display system 10 is a system for presenting a user with a history of display locations where captured images captured by an onboard camera are displayed, and for registering locations where captured images are automatically displayed. The vehicle display system 10 is mainly composed of an onboard device 20 mounted on a vehicle, a server 40, and a user terminal (terminal device) 50. The onboard device 20, the server 40, and the user terminal 50 are configured to be able to communicate with each other via a network 30. The network 30 may be, for example, the Internet. The network 30 may utilize a mobile communication function such as 4G / LTE or 5G.
[0012] The in-vehicle device 20 is mounted on a vehicle. The in-vehicle device 20 associates display history information, including a history of position information of display points, which are points where images captured by the in-vehicle camera are displayed on the HMI 26, with a user ID for identifying the user of the vehicle, and transmits the information to the server 40. The server 40 stores the display history information and registration information, including position information of registered points, which are points where images captured by the in-vehicle camera are automatically displayed on the HMI 26, in association with the user ID.
[0013] The server 40 also transmits the display history information and registration information to the user terminal 50 corresponding to the user ID. The user terminal 50 displays the display history information received from the server 40 and updates the display history information based on editing operations from the user. The user terminal 50 transmits the updated display history information to the server 40 as registration information. The server 40 receives the registration information from the user terminal 50 and updates the registration information corresponding to the user ID.
[0014] Then, the server 40 transmits the registration information corresponding to the user ID to the in-vehicle device 20. The in-vehicle device 20 receives the registration information corresponding to the user ID from the server 40, and automatically displays the image captured by the in-vehicle camera on the HMI 26 when the vehicle travels through the registered point.
[0015] Next, a specific configuration of the vehicular display system 10 will be described with reference to FIG.
[0016] [Configuration of the In-Vehicle Device] First, a description will be given of the configuration of the in-vehicle device 20. As shown in Fig. 2 , the in-vehicle device 20 includes a communication device 21, a GPS (Global Positioning System) receiver 22, a camera 23, a controller 24, a storage device 25, and an HMI (Human Machine Interface) 26.
[0017] The communication device 21 communicates with the server 40 via the network 30. The communication device 21 receives predetermined data from the server 40 and transmits predetermined data to the server 40. For example, the communication device 21 may be a device equipped with a mobile communication function such as 4G / LTE or 5G, or may be a device equipped with a Wi-Fi (registered trademark) communication function.
[0018] The GPS receiver 22 receives radio waves from artificial satellites to detect the vehicle's terrestrial position information (including latitude and longitude information). However, the method of detecting the vehicle's position information is not limited to the GPS receiver 22. For example, the vehicle's position information may be estimated based on the reception status of radio waves from multiple mobile phone base stations. For example, the position may be estimated using odometry. In this case, the in-vehicle device 20 includes a steering angle sensor, a wheel speed sensor, and a gyro sensor.
[0019] The cameras 23 have imaging elements such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and capture images of the surroundings of the vehicle. There are no particular limitations on the locations where the cameras 23 are installed, but in this embodiment, four cameras 23 are installed at the front, rear, left, and right sides of the vehicle, and each camera 23 captures images of the front, rear, left, and right sides of the vehicle.
[0020] The camera 23 outputs the captured image to the controller 24. The image captured by the camera 23 may be stored in the storage device 25, and the controller 24 may acquire the image stored in the storage device 25. The camera 23 is a device that captures images of the surroundings of the vehicle, but is not a device that captures images of the vehicle itself. Therefore, when the controller 24, which will be described later, displays an overhead image, the controller 24 uses a pre-stored icon (an image that resembles a vehicle).
[0021] The storage device 25 may include non-transitory tangible storage media such as a register, a cache memory, or a memory used as a main memory, such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage device 25 stores display history information including at least a history of location information of the vehicle's location when an image captured by the camera 23 is displayed on the HMI 26 (hereinafter referred to as a display point). In other words, the display point is the vehicle's location when the display on the HMI 26 is switched to the image captured by the camera 23.
[0022] The HMI 26 is an interface device that exchanges information between the in-vehicle device 20 and the vehicle user. The HMI 26 has a display device that can be seen by the vehicle user. The HMI 26 also has an operator and a voice input device that accept user operation input to the in-vehicle device 20. The operator may be a mechanical interface device such as a button, switch, lever, dial, or keyboard, or may be a button, switch, lever, dial, or keyboard displayed on a touch panel. The HMI 26 accepts operation input of a user ID for identifying the user from the vehicle user.
[0023] The HMI 26 displays the captured image captured by the camera 23. The HMI 26 also accepts an operation input from the user to display the captured image captured by the camera 23. In this embodiment, the HMI 26 accepts an operation input from the user to specify a display mode. The display mode indicates in what mode the captured image captured by the camera 23 (front, rear, left, or right side of the vehicle) is displayed on the HMI 26. For example, the display mode includes a mode that displays a captured image including any of the front, rear, or side of the vehicle, and a mode that displays any of a surrounding image of an area including the vehicle viewed from above.
[0024] Furthermore, if various parameters such as the orientation and zoom amount of the camera 23 are variable, the display mode may further include, for example, the orientation of the camera 23, the zoom amount, the focus position, exposure compensation (sensitivity, exposure time, F-number), image processing parameters (gamma value and white balance), etc. Furthermore, if the HMI 26 can post images captured by the camera 23 on a social networking service (SNS), the display mode may further include information regarding the type of SNS application that posted the captured image. Specific examples of the display modes and details of operation inputs for the display modes will be described later with reference to FIGS. 4 to 9.
[0025] The controller 24 is connected to the communication device 21, the GPS receiver 22, the camera 23, the storage device 25, and the HMI 26, and receives information from the communication device 21, the GPS receiver 22, the camera 23, and the HMI 26. The controller 24 outputs information to the communication device 21, the GPS receiver 22, the camera 23, and the storage device 25.
[0026] The controller 24 is, for example, an electronic control unit (ECU) that includes a central processing unit (CPU), a memory, and an input / output unit, and controls the operation of the in-vehicle device 20. A predetermined computer program is installed in the controller 24. The controller 24 executes the computer program to perform various processes.
[0027] In this embodiment, the controller 24 generates an overhead image (surrounding image) of the vehicle viewed from above using images captured by the four cameras 23. Note that the method for generating the overhead image is a known technique, and therefore a detailed description thereof will be omitted. The controller 24 also stores vehicle driving history information in the storage device 25 using vehicle position information and various sensors (not shown) mounted on the vehicle. The driving history information includes at least one of the driving route traveled by the vehicle and the orientation of the vehicle when the captured image is displayed on the HMI 26.
[0028] The controller 24 recognizes the user ID accepted by the HMI 26. The controller 24 also stores display history information in the storage device 25, including at least a history of location information of display points where captured images were displayed on the HMI 26, and transmits the information to the server 40 in association with the user ID. The display history information may further include information indicating the display mode when the captured image was displayed on the HMI 26. The display history information may also further include at least one of the date and time when the captured image was displayed on the HMI 26 and the captured image.
[0029] Furthermore, when the controller 24 recognizes the user ID, it receives from the server 40 registration information including location information of a registered point, which is a point corresponding to the user ID and at least where an image is to be automatically displayed on the HMI 26. In other words, the registered point is a point where the display on the HMI 26 is automatically switched to an image captured by the camera 23. When the vehicle passes the registered point, the controller 24 automatically displays the image captured by the camera 23 on the HMI 26. In other words, when the vehicle passes the registered point, the controller 24 automatically switches the display on the HMI 26 to an image captured by the camera 23.
[0030] The registration information may further include information indicating a registration display mode, which is a display mode when the captured image is automatically displayed on the HMI 26. In this case, when the vehicle travels through a registered point, the controller 24 automatically displays the captured image captured by the camera 23 on the HMI 26 in the registration display mode. Details of the processing performed by the controller 24 will be described later with reference to Fig. 3. The various processes provided by the controller 24 may be realized by software or dedicated hardware.
[0031] [Configuration of Server] Next, a description will be given of the configuration of the server 40. As shown in FIG.
[0032] The communication device 41 communicates with the in-vehicle device 20 and the user terminal 50 via the network 30. The communication device 41 receives predetermined data from the in-vehicle device 20 and the user terminal 50, and transmits predetermined data to the in-vehicle device 20 and the user terminal 50. For example, the communication device 41 may be a device equipped with a mobile communication function such as 4G / LTE or 5G, or may be a device equipped with a Wi-Fi communication function.
[0033] The storage device 43 is a storage medium such as a hard disk drive (HDD), etc. The storage device 43 includes a user database 431, a display history information database 432, and a registration information database 433.
[0034] The user database 431 is a database in which various information related to the vehicle user is stored in association with a user ID that identifies the user.
[0035] The display history information database 432 is a database in which the display history information received from the in-vehicle device 20 is stored in association with the user ID.
[0036] The registration information database 433 is a database in which registration information received from the user terminal 50 is stored in association with a user ID. The registration information includes at least location information of a registration point, which is a point where an image captured by the camera 23 is automatically displayed on the HMI 26. The registration information may further include information indicating a registration display mode. The registration display mode is a display mode when an image captured by the camera 23 is automatically displayed on the HMI 26.
[0037] The controller 42 is connected to the communication device 41 and the storage device 43, and receives information from the communication device 41 as input to the controller 42. The controller 42 outputs information to the communication device 41 and the storage device 43. The controller 42 is a general-purpose microcomputer equipped with a CPU, memory, and input / output units. A predetermined computer program is installed in the controller 42. By executing the computer program, the controller 42 performs various processes.
[0038] The controller 42 stores the display history information received from the in-vehicle device 20 in the display history information database 432 of the storage device 43 in association with the user ID, and updates the display history information. When the controller 42 receives the display history information and there is no registration information for the corresponding user ID, the controller 42 automatically generates registration information from the display history information and stores the generated registration information in the registration information database 433 of the storage device 43 in association with the user ID.
[0039] Furthermore, the controller 42 transmits the display history information to the user terminal 50 corresponding to the user ID. When the controller 42 receives registration information from the user terminal 50, the controller 42 updates the registration information corresponding to the user ID. The controller 42 transmits the registration information corresponding to the user ID to the in-vehicle device 20. Details of the processing performed by the controller 42 will be described later with reference to Figures 3 and 4. The various processes provided by the controller 42 may be realized by software or may be realized by dedicated hardware.
[0040] [Configuration of User Terminal] Next, the configuration of the user terminal 50 will be described. The user terminal 50 is, for example, a mobile terminal such as a smartphone carried by a user. As shown in Fig. 2, the user terminal 50 includes a communication device 51, a controller 52, and an operation display unit 53. Note that the user terminal 50 is not limited to a mobile terminal carried by a user, and may be, for example, a terminal device installed in a parking space.
[0041] The communication device 51 communicates with the server 40 via the network 30. The communication device 51 receives predetermined data from the server 40 and transmits predetermined data to the server 40. For example, the communication device 51 may be a device equipped with a mobile communication function such as 4G / LTE or 5G, or may be a device equipped with a Wi-Fi communication function.
[0042] The operation display unit 53 displays information necessary for user operations and accepts input operations from the user. For example, the operation display unit 53 can be a touch panel that allows input operations to be performed according to information displayed on the operation display unit 53. Note that the operation display unit 53 may be configured so that the operation unit that accepts input operations from the user and the display unit that displays information necessary for user operations are separate units.
[0043] The controller 52 is connected to the communication device 51 and the operation display unit 53, and receives information from the communication device 51 and the operation display unit 53. The controller 52 outputs information to the communication device 51 and the operation display unit 53. The controller 52 is a general-purpose microcomputer equipped with a CPU, memory, and input / output units. A predetermined computer program is installed in the controller 52. By executing the computer program, the controller 52 performs various processes.
[0044] The controller 52 receives display history information from the server 40. The controller 52 displays the display history information received from the server 40 on the operation display unit 53. The controller 52 updates the display history information based on an editing operation from the user. The controller 52 transmits the updated display history information to the server 40 as registration information. Details of the processing performed by the controller 52 will be described later with reference to FIG. 4. The various processes provided by the controller 52 may be realized by software or dedicated hardware.
[0045] [Operation of the Vehicle Display System] Next, with reference to FIG. 3, the correspondence between the user's operation, the operation of the in-vehicle device 20, and the operation of the server 40 when the display history information is stored in the vehicle display system 10 will be described.
[0046] First, in step S101, a user in a vehicle inputs a user ID to the HMI 26 of the in-vehicle device 20. The HMI 26 of the in-vehicle device 20 accepts an operation input of the user ID from the user via, for example, a touch panel.
[0047] In step S102, the controller 24 of the in-vehicle device 20 recognizes the user ID received by the HMI 26. In step S103, the controller 24 transmits the user ID to the server 40.
[0048] Proceeding to step S104, the controller 42 of the server 40 receives the user ID from the in-vehicle device 20. The controller 42 selects registration information corresponding to the user ID from the registration information stored in the registration information database 433 of the storage device 43 and transmits it to the in-vehicle device 20. The registration information includes at least position information of a registration point, which is a point where an image captured by the camera 23 is automatically displayed on the HMI 26.
[0049] The process proceeds to step S105, where the controller 24 of the in-vehicle device 20 automatically displays on the HMI 26 an image captured by the camera 23 when the vehicle passes through a registered point. The controller 24 determines whether the vehicle is passing through a registered point based on the vehicle's position information. For example, if the vehicle's position information is within a predetermined range from the position information of the registered point, the controller 24 determines that the vehicle is passing through the registered point and automatically displays the image on the HMI 26. Furthermore, if the registration information includes information on the registration display mode, the controller 24 displays on the HMI 26 an image captured by the camera 23 at the registered point in the registration display mode. The process then proceeds to step S108.
[0050] Furthermore, as shown in step S106, if the user inputs a display operation to display the captured image on the HMI 26 of the in-vehicle device 20, the process proceeds to step S107, where the controller 24 of the in-vehicle device 20 displays the captured image on the HMI 26. Note that if the user inputs an operation input to instruct the HMI 26 of the in-vehicle device 20 to change the display mode in step S106, the controller 24 of the in-vehicle device 20 displays the captured image in the display mode instructed by the user.
[0051] The process proceeds to step S108, where the controller 42 of the in-vehicle device 20 ends the display of the captured image after a predetermined time has elapsed since the captured image was displayed, or after the vehicle has moved a predetermined distance since the captured image was displayed. Note that the controller 42 may end the display of the captured image captured by the camera 23 when receiving an operation to end the display of the captured image from the user via the HMI 26.
[0052] Proceeding to step S109, the controller 24 of the in-vehicle device 20 stores, in the storage device 25, display history information including a history of position information of a display point, which is a point where an image captured by the camera 23 is displayed on the HMI 26, and an image captured by the camera 23 at the display point. If, in step S106, the user inputs an operation to instruct the HMI 26 of the in-vehicle device 20 to change the display mode, the controller 24 stores, in the storage device 25, display history information further including information about the display mode. In this embodiment, the controller 24 stores the display history information in the storage device 25 in association with vehicle driving history information.
[0053] Proceeding to step S110, the controller 24 transmits the display history information and the vehicle driving history information to the server 40 in association with the user ID.
[0054] The process proceeds to step S111, where the controller 42 of the server 40 stores the display history information received from the in-vehicle device 20 in the display history information database 432 of the storage device 43 in association with the user ID, thereby updating the display history information. In this embodiment, the controller 42 of the server 40 stores the display history information and vehicle driving history information received from the in-vehicle device 20 in the display history information database 432 of the storage device 43 in association with the user ID. When the controller 42 receives the display history information, if there is no registration information for the corresponding user ID, the controller 42 automatically generates registration information from the display history information and stores the generated registration information in association with the user ID in the storage device 43. The process then proceeds to step S201 in FIG. 4 .
[0055] 3 ends if the user has turned off the engine and exited the vehicle (YES in step S112). If the user has not turned off the engine and exited the vehicle (NO in step S112), the process returns to step S106.
[0056] Next, with reference to FIGS. 4 to 9, the correspondence between the user's operation, the operation of the user terminal 50, and the operation of the server 40 when a registration point is registered in the vehicular display system 10 will be described.
[0057] First, in step S201, the controller 42 of the server 40 transmits the display history information to the user terminal 50 corresponding to the user ID. The controller 42 of the server 40 also transmits the driving history information to the user terminal 50 corresponding to the user ID. The timing at which the controller 42 of the server 40 transmits the display history information and driving history information to the user terminal 50 is not particularly limited, as long as the user can check the display history information and driving history information using the user terminal 50 after getting out of the vehicle.
[0058] Proceeding to step S202, the controller 52 of the user terminal 50 displays the display history information received from the server 40 on the operation display unit 53 of the user terminal 50. In this embodiment, the controller 52 of the user terminal 50 displays the display history information and driving history information received from the server 40 on the operation display unit 53 of the user terminal 50. For example, as shown in FIG. 5 , the controller 52 of the user terminal 50 displays on the operation display unit 53 a screen 531 including camera icons P1 to P3 indicating display points where captured images are displayed and a vehicle driving route 62.
[0059] The process proceeds to step S203, where the user inputs a predetermined operation, such as selecting a predetermined image, via the operation display unit 53 of the user terminal 50. When the operation display unit 53 of the user terminal 50 accepts the predetermined operation from the user, the process proceeds to step S204. Furthermore, for example, the process may proceed to step S204 when the user selects a notification displayed on the operation display unit 53; the predetermined operation is not particularly limited. In the example shown in FIG. 5, the process proceeds to step S204 when the user selects one of the camera icons P1 to P3 on the screen 531.
[0060] For example, when the user selects icon 61 in FIG. 5 , the controller 52 of the user terminal 50 may display a screen 532 showing details of the display history information and the driving history information on the operation / display unit 53, as shown in FIG. 6 . The screen 532 displays, for example, location information of the display point, the vehicle's driving route, and a list of captured images 63 captured by the camera 23 at the display point and displayed on the HMI 26. In the example shown in FIG. 6 , the display points are indicated by circular marks 631 to 633. The marks 631 to 633 correspond to the display points indicated by the camera icons P1 to P3 in FIG. 5 . In the example shown in FIG. 6 , the vehicle's driving route is indicated by a vertical line 634. The line 634 corresponds to the driving route 62 in FIG. 5 . In FIG. 6 , for example, when the user touches mark 632, a captured image 63 captured by the camera 23 at the display point indicated by mark 632 is displayed on the screen 532. When the user selects the captured image 63, the operation proceeds to step S204.
[0061] In step S204, the controller 52 of the user terminal 50 displays a selection screen 533 on the operation display unit 53 of the user terminal 50. The selection screen 533 is a screen on which the user can select whether or not to store the position information and display mode of the display point included in the display history information selected by the user in step S203 as a registered point and a registered display mode in the storage device 43 of the server 40. In other words, the selection screen 533 is a screen on which the user can select whether or not to store the display history information selected by the user in step S203 as registered information in the storage device 43 of the server 40. As shown in FIG. 7 , the selection screen 533 includes a first image 641 and a second image 642 that indicate whether or not to store the position information and display mode of the display point included in the display history information selected by the user in step S203 as the position information of the registered point and a registered display mode in the storage device 43 of the server 40. In addition, if the location information of the registered point and the registered display mode are stored in the memory device 43, the next time the vehicle travels through the registered point, the image captured by the on-board camera 23 will be automatically displayed on the HMI 26 in the registered display mode.
[0062] Furthermore, the selection screen 533 is a screen on which the user can select to edit the location information and display mode of the display point included in the display history information selected by the user in step S203. As shown in Fig. 7 , the selection screen 533 includes a third image 643 indicating that the user can edit the location information and display mode of the display point included in the display history information selected by the user in step S203.
[0063] In step S205, if the user decides to edit the location information and display mode of the display point (YES in step S205), the process proceeds to step S206. In step S206, the user selects the third image 643 on the selection screen 533. If the user does not decide to edit the location information and display mode of the display point (NO in step S205), the process proceeds to step S210.
[0064] In step S207, the controller 52 of the user terminal 50 displays a display mode editing screen 534 for editing the display mode and a display location editing screen 535 for editing the position information of the display location on the operation display unit 53 of the user terminal 50. Proceeding to step S208, the user performs an editing operation for the position information of the location and the display mode.
[0065] [Display Mode Editing Operation] An example of a display mode editing operation will be described with reference to Fig. 8. As shown in Fig. 8, the display mode editing screen 534 includes a plurality of fourth images 651 to 653 indicating display mode options.
[0066] 8 , image 651 represents a first display mode (View mode: TOP View) that displays an overhead image of the vehicle generated from images captured by four cameras 23 and an image of the rear of the vehicle captured by a camera 23 installed at the rear of the vehicle. Image 652 represents a second display mode (View mode: Side View) that displays an image of the front of the vehicle captured by a camera 23 installed at the front of the vehicle and an image of the right side of the vehicle (or the right side of the vehicle) captured by a camera 23 installed at the right side of the vehicle. Image 652 represents a third display mode (View mode: Wide View) that displays an image of a range of nearly 180 degrees in front of the vehicle captured by a camera 23 installed at the front of the vehicle.
[0067] 8, when one of the plurality of fourth images 651 to 653 is selected, the display mode editing screen 534 may include a captured image 63 that is displayed on the HMI 26 in a display mode corresponding to the selected fourth image. The captured image 63 may be an image previously captured by the camera 23 at the display location, or may be a sample image of the display mode corresponding to the selected fourth image.
[0068] Furthermore, when one of the plurality of fourth images 651 to 653 is selected, the display mode editing screen 534 may include an explanation of which position relative to the vehicle the captured image displayed on the HMI 26 in the display mode corresponding to the selected fourth image includes and what type of vehicle operation it is suitable for. For example, in the example shown in FIG. 8 , the captured image displayed in the first display mode corresponding to image 651 includes an overhead view of the vehicle and the rear of the vehicle, and is suitable for parking the vehicle. The captured image displayed in the second display mode corresponding to image 652 includes the front of the vehicle and the right (or left) side of the vehicle, and is suitable for checking the distance to the side of the vehicle, such as when parallel parking or pulling over. The captured image displayed in the third display mode corresponding to image 653 includes a range of nearly 180 degrees in front of the vehicle, and is suitable for checking safety on both sides at intersections with poor visibility.
[0069] [Editing Operation of Displayed Locations] Next, an example of an editing operation of a displayed location will be described with reference to FIG. 9 . As shown in FIG. 9 , the displayed location editing screen 535 includes a fifth image 66 for moving the location of a location by a predetermined distance. For example, as shown in FIG. 9 , the fifth image 66 includes three images: "Earlier," "Just," and "Later." If the user selects "Earlier" from the fifth image 66, the location of the displayed location is moved by a predetermined distance in the direction opposite to the vehicle's traveling direction on the vehicle's traveling route. If the user selects "Later" from the fifth image 66, the location of the displayed location is moved by a predetermined distance in the vehicle's traveling direction on the vehicle's traveling route. The predetermined distance is, for example, 10 m, but is not particularly limited. If the user selects "Just" from the fifth image 66, the location of the displayed location included in the display history information is maintained. Note that the display mode editing screen 534 may display the edited location of the displayed location on a map based on the user's operation.
[0070] 9 , the display location editing screen 535 may include a map 67 of the vicinity of the display location, and the operation display unit 53 may accept an operation to move the position of the display location on the map 67. For example, the map 67 may include a vehicle travel route 671 and a marker 672 indicating the display location, and the user may move the position of the display location by selecting and dragging the marker 672. Note that if the camera 23 captures video for a predetermined period of time before and after the display location and the captured video is stored in the storage device 43 as display history information, when an operation to move the position of the display location is accepted from the user on the display location editing screen 535, it is also possible to cut out an image captured by the camera 23 at the point after the move from the video and display it on the display location editing screen 535.
[0071] 4, the controller 52 of the user terminal 50 saves the edited display history information. The controller 52 of the user terminal 50 also displays the selection screen 533 again on the operation display unit 53 of the user terminal 50. In step S210, the user selects whether or not to register the registration information.
[0072] The process proceeds to step S211, and if the user selects to register the registration information (YES in step S211), the process proceeds to step S212. On the other hand, if the user selects not to register the registration information (NO in step S211), the process proceeds to step S214.
[0073] In step S212, the controller 52 of the user terminal 50 transmits the display history information as registration information to the server 40 in association with the user ID. When the controller 52 of the user terminal 50 receives an edit operation for the display history information from the user via the operation display unit 53, the controller 52 transmits the edited display history information as registration information to the server 40. When the controller 52 of the user terminal 50 does not receive an edit operation for the display history information from the user via the operation display unit 53, the controller 52 transmits the original display history information as registration information to the server 40.
[0074] In step S213, the controller 42 of the server 40 updates the registration information corresponding to the user ID with the registration information received from the user terminal 50, stores the updated registration information in the registration information database 433 of the storage device 43, and updates the registration information. Then, the operation of FIG. 4 is terminated.
[0075] In step S214, the controller 52 of the user terminal 50 instructs the server 40 not to register the display history information, and the operation of FIG. 4 ends.
[0076] [Effects] As described above, a display point registration method according to one embodiment is a display point registration method for a vehicular display system 10 that includes an in-vehicle device 20 mounted on a vehicle and having a display unit (HMI) 26 and an in-vehicle camera 23, a server 40, and a terminal device (user terminal) 50 carried by a user of the vehicle. The server 40 stores display history information in a storage device 43, the display history information including a history of position information of display points, which are points at which the in-vehicle device 20 displayed captured images taken by the in-vehicle camera 23 on the display unit 26. The server 40 outputs the display history information to the terminal device 50, and stores in the storage device 43 the position information of a display point selected by the user on the terminal device 50 as position information of a registered point, which is a point at which the captured image is automatically displayed on the display unit 26.
[0077] The position of the display point where the captured image is displayed can be displayed on the terminal device 50. When the vehicle engine is off or after the user has exited the vehicle, the user can check the position of the display point using the terminal device 50 and register a registered point where the captured image is automatically displayed on the display unit. The user can register the registered point in a safe and calm environment.
[0078] In addition, in the display point registration method according to one embodiment, the display history information further includes information indicating the display mode when the captured image was displayed on the display unit 26. The server 40 stores the display mode selected by the user on the terminal device 50 in the storage device 43 as a registered display mode, which is the display mode when the captured image is automatically displayed on the display unit 26.
[0079] This allows the display mode when the captured image is displayed to be displayed on the terminal device 50. When the vehicle engine is off or after the user has exited the vehicle, the user can use the terminal device 50 to check the display mode at the display location and select a registration display mode when the captured image is to be automatically displayed on the display unit 26. The user can register the display mode when the captured image is to be automatically displayed on the display unit 26 in a safe and calm environment.
[0080] Furthermore, a display point registration method according to one embodiment is a display point registration method for a vehicular display system 10 including an in-vehicle device 20 mounted on a vehicle and having a display unit (HMI) 26 and an in-vehicle camera 23, a server 40, and a terminal device (user terminal) 50 operable by a user of the vehicle. The server 40 stores display history information in a storage device 43, the display history information including a history of position information of a display point, which is a point at which the in-vehicle device 20 displayed an image captured by the in-vehicle camera 23 on the display unit 26, and information indicating a display mode when the captured image was displayed on the display unit 26. The server 40 outputs the display history information to the terminal device 50, and stores the position information included in the display history information selected on the terminal device 50 in the storage device 43 as position information of a registered point, which is a point at which the captured image is automatically displayed on the display unit 26. The server 40 also stores the display mode selected on the terminal device 50 in the storage device 43 as a registered display mode, which is a display mode when the captured image is automatically displayed on the display unit 26.
[0081] This allows the user to set the registration location and registration display mode using the terminal device 50 when the vehicle engine is off or after the user has exited the vehicle, allowing the registration location to be registered in a safer and more relaxed environment.
[0082] In addition, in a display point registration method according to one embodiment, the server 40 associates the display history information with the vehicle's driving history information and stores it in the storage device 43. The terminal device 50 displays the display history information and the driving history information, and when a predetermined operation is received from the user, displays a first image 641 and a second image 642 indicating whether or not to store the position information and display mode of the display point included in the display history information in the storage device 43 as the position information and registered display mode of the registered point.
[0083] The display history information and driving history information can be displayed on the terminal device 50. The user can use the terminal device 50 to check the display points and display modes included in the display history information and the driving history information, and select whether or not to store the display points and display modes included in the display history information as registered points and registered display modes. The user can register the registered points and registered display modes based on the display history information and driving history information in a safe and calm environment.
[0084] In addition, in one embodiment of a display point registration method, the terminal device 50 displays display history information and driving history information, and when a predetermined operation is received from the user, displays a third image 643 indicating that the location information and display mode of the display point included in the display history information will be edited.
[0085] The user can use the terminal device 50 to edit the display location and display mode included in the display history information and select whether or not to store them as a registered location and registered camera mode.
[0086] Furthermore, in a display point registration method according to an embodiment, the display history information includes captured image 63 captured at the display point by in-vehicle camera 23. When third image 643 is selected, terminal device 50 displays a plurality of fourth images 651 to 653 indicating display mode options, and when any of the plurality of fourth images 651 to 653 is selected, terminal device 50 displays captured image 63 displayed in the display mode corresponding to the selected fourth image.
[0087] The user can use the terminal device 50 to check the captured image 63 captured by the vehicle-mounted camera 23 for each display mode and select a registered display mode.
[0088] In addition, in one embodiment of the display point registration method, when one of the multiple fourth images 651 to 653 is selected, the terminal device 50 displays an explanation of which positions relative to the vehicle the captured image 63 displayed in the display mode corresponding to the selected fourth image includes and what type of vehicle operation it is suitable for.
[0089] The user can use the terminal device 50 to check the position relative to the vehicle contained in the captured image 63 taken by the on-board camera 23 for each display mode and the type of vehicle operation the captured image 63 is suitable for, and then select a registered display mode.
[0090] In addition, in the display point registration method according to one embodiment, when the third image 643 is selected, the terminal device 50 displays a fifth image 66 for moving the position of the display point by a predetermined distance. The user can use the terminal device 50 to adjust the position of the display point and register it as a registered point.
[0091] Furthermore, in a display point registration method according to an embodiment, when the third image 643 is selected, the terminal device 50 displays a map 67 of the area around the display point and accepts an operation to move the position of the display point on the map 67. The user can use the terminal device 50 to adjust the position of the display point on the map 67 and register it as a registered point.
[0092] In addition, in the display point registration method according to one embodiment, the driving history information includes at least one of the driving route traveled by the vehicle or the orientation of the vehicle when the captured image is displayed on the display unit 26. The user can check the past driving route or the orientation of the vehicle and the past display history information to register a registration point and a registration display mode.
[0093] In addition, in the display point registration method according to one embodiment, the server 40 transmits position information of the registered point and information indicating the registered display mode to the in-vehicle device 20. The in-vehicle device 20 can control the in-vehicle camera 23 based on the registered point and the registered display mode.
[0094] Furthermore, in a display location registration method according to one embodiment, display history information is stored in association with a user ID that identifies a user, and when the in-vehicle device 20 recognizes the user ID, the display history information is transmitted to the in-vehicle device 20. This allows the in-vehicle device 20 to control the in-vehicle camera 23 based on the registered location and registered display mode stored in association with the user ID, even if the user gets into a vehicle different from the vehicle they got into last time.
[0095] In addition, in the display point registration method according to an embodiment, the display history information further includes the date and time when the captured image was displayed on the display unit 26. The user can check the date and time when the captured image was displayed in the past and register a registration point and a registration display mode.
[0096] In addition, in the display point registration method according to one embodiment, the display modes include a mode for displaying a captured image including any of the front, rear, and side of the vehicle, and a surrounding image of the area including the vehicle viewed from above. A user can register a registration display mode suitable for the registration point from among the multiple display modes.
[0097] Although the embodiments of the present invention have been described above, the descriptions and drawings that form part of this disclosure should not be understood to limit the present invention. Various alternative embodiments, examples, and operating techniques will become apparent to those skilled in the art from this disclosure.
[0098] For example, the display history information may further include information indicating the display mode of the captured image displayed at the display point and information on the elapsed time for each display mode during which the vehicle-captured image was displayed on the display unit 26. In this case, the server 40 may store in the storage device 43 the display mode that was displayed for the longest time in the display history information as a registered display mode, which is the display mode when the captured image is automatically displayed on the display unit 26. This makes it possible to automatically register the registered display mode.
[0099] In the above embodiment, the location information and display mode included in the display history information selected by the user on the terminal device 50 are stored in the storage device 43 of the server 40 as the location information and registered display mode of the registered point. However, the entity that selects the display history information is not limited to the user, but may be a computer. That is, the location information and display mode included in the display history information selected by the computer may be stored in the storage device 43 of the server 40 as the location information and registered display mode of the registered point. For example, the controller 42 of the server 40 or the controller 52 of the user terminal 50 may automatically select the registered point and registered display mode from the location information and display mode of the display point included in the display history information, using past display history information of other vehicles, terrain information, road information, etc.
[0100] 20 In-vehicle device 23 Camera (in-vehicle camera) 26 Display unit (HMI) 40 Server 43 Storage device 50 Terminal device (user terminal)
Claims
1. A display point registration method for a vehicle display system equipped with an on-board device mounted on a vehicle and having a display unit and an on-board camera, a server, and a terminal device carried by a user of the vehicle, wherein the server: stores in a storage device display history information including a history of position information of a display point, which is a point at which the on-board device displayed an image captured by the on-board camera on the display unit; outputs the display history information to the terminal device; and stores in the storage device position information of a display point selected by the user on the terminal device as position information of a registered point, which is a point at which the captured image is automatically displayed on the display unit.
2. The display location registration method according to claim 1, wherein the display history information further includes information indicating a display mode when the captured image was displayed on the display unit, and the server stores in the storage device the display mode selected by the user on the terminal device as a registered display mode, which is the display mode when the captured image is automatically displayed on the display unit.
3. A display point registration method for a vehicle display system equipped with an on-board device mounted on a vehicle and having a display unit and an on-board camera, a server, and a terminal device operable by a user of the vehicle, wherein the server: stores in a storage device display history information including a history of position information of a display point, which is a point at which the on-board device displayed an image captured by the on-board camera on the display unit, and information indicating a display mode when the captured image was displayed on the display unit; outputs the display history information to the terminal device; stores in the storage device the position information included in the display history information selected in the terminal device as position information of a registered point, which is a point at which the captured image is automatically displayed on the display unit; and stores in the storage device the display mode included in the display history information selected in the terminal device as a registered display mode, which is the display mode when the captured image is automatically displayed on the display unit.
4. A display point registration method as described in claim 2 or 3, wherein the display mode includes a mode for displaying either an imaged image including the front, rear, or side of the vehicle, or an image of the surroundings viewed from above the area including the vehicle.
5. A display point registration method as claimed in any one of claims 2 to 4, wherein the server stores the display history information in association with driving history information of the vehicle, and the terminal device displays the display history information and the driving history information, and when a predetermined operation is received from the user, displays a first image and a second image indicating whether or not to store the position information and the display mode of the display point included in the display history information as the position information of the registered point and the registered display mode in the storage device.
6. The display location registration method according to claim 5, wherein the terminal device displays the display history information and the driving history information, and when a predetermined operation is received from the user, displays a third image indicating that the location information and the display mode of the display location included in the display history information are to be edited.
7. The display location registration method according to claim 6, wherein the display history information includes captured images captured by the in-vehicle camera at the display location, and the terminal device, when the third image is selected, displays a plurality of fourth images indicating the display mode options, and, when any of the plurality of fourth images is selected, displays the captured image displayed in the display mode corresponding to the selected fourth image.
8. The display point registration method according to claim 7, wherein, when one of a plurality of fourth images is selected, the terminal device displays an explanation of which position relative to the vehicle the captured image displayed in the display mode corresponding to the selected fourth image includes and what type of vehicle operation it is suitable for.
9. The display location registration method according to claim 6, wherein the terminal device, when the third image is selected, displays a fifth image for moving the position of the displayed location a predetermined distance.
10. The display location registration method according to claim 9, wherein, when the third image is selected, the terminal device displays a map of the area surrounding the display location and accepts an operation to move the position of the display location on the map.
11. The display point registration method according to claim 5, wherein the driving history information includes at least one of the route traveled by the vehicle or the orientation of the vehicle when the captured image was displayed.
12. A display point registration method according to any one of claims 2 to 11, wherein the server transmits position information of the registered point and information indicating the registration display mode to the in-vehicle device.
13. A display location registration method according to any one of claims 1 to 12, wherein the server stores the display history information in association with a user ID that identifies the user, and transmits the display history information to the in-vehicle device when the in-vehicle device recognizes the user ID.
14. A display location registration method according to any one of claims 1 to 13, wherein the display history information further includes a date and time when the captured image was displayed on the display unit.
15. The display location registration method according to claim 1, wherein the display history information further includes information indicating the display mode of the captured image displayed at the display location and information on the elapsed time for which the captured image was displayed on the display unit for each display mode, and the server stores in the storage device the display mode that was displayed for the longest time in the display history information as a registered display mode that is the display mode when the captured image is automatically displayed on the display unit.
16. A display system for a vehicle comprising an on-board device mounted on a vehicle and having a display unit and an on-board camera, a server, and a terminal device carried by a user of the vehicle, wherein the server: stores in a storage device display history information including a history of position information of a display point, which is a point at which the on-board device displayed an image captured by the on-board camera on the display unit; outputs the display history information to the terminal device; and stores in the storage device position information of a display point selected by the user on the terminal device as position information of a registered point, which is a point at which the captured image is automatically displayed on the display unit.
17. A vehicle display system comprising an on-board device mounted on a vehicle and having a display unit and an on-board camera, a server, and a terminal device operable by a user of the vehicle, wherein the server: stores display history information in a storage device, the display history information including a history of position information of a display point, which is a point at which the on-board device displayed an image captured by the on-board camera on the display unit, and information indicating a display mode when the captured image was displayed on the display unit; outputs the display history information to the terminal device; stores the position information included in the display history information selected in the terminal device in the storage device as position information of a registered point, which is a point at which the captured image is automatically displayed on the display unit; and stores the display mode included in the display history information selected in the terminal device in the storage device as a registered display mode, which is the display mode when the captured image is automatically displayed on the display unit.
Citation Information
Patent Citations
Imaging apparatus for vehicles
JP2005318519A
Map data updating system, personal digital assistant, map distribution server, navigation system, and map data updating method
JP2008139091A
On-vehicle display device
JP2012173269A
