Display system, display method

Abstract

The present disclosure relates to a display system and a display method. The display system of the present disclosure includes: a camera that captures a driving image of the traveling direction of a vehicle; a display device; one or more storage devices that store a reference deceleration representing a prescribed deceleration; and one or more processors. The one or more processors are configured to perform the following processing: obtain the vehicle speed; calculate an estimated stop position representing a stop position in the case of braking from the vehicle speed at the reference deceleration; and display the estimated stop position overlapping the driving image on the display device.

Description

Technical Field

[0001] The present disclosure relates to a technique for displaying a traveling image of a vehicle on a display device. Background Art

[0002] Conventionally, a technique has been considered in which an image captured by a camera mounted on a vehicle is displayed on a display device, and an auxiliary display is superimposed and displayed to support smooth driving operations.

[0003] For example, Japanese Unexamined Patent Application Publication No. 2004-262449 discloses a parking assistance device that, during a parking operation, captures an image of the rear of the vehicle with a camera, displays the image from the camera as a rear image on a display provided inside the vehicle, and superimposes and displays a predicted travel trajectory that changes according to the state of the steering wheel angle on the rear image, and in which an attention area that is a standard for the distance to the rear of the vehicle is superimposed and displayed on the display.

[0004] In a remote driving system or the like, it is desired that a driver of a vehicle visually recognize a traveling image displayed on a display device and perform a driving operation without actually riding in the vehicle. In such a case, the driver lacks driving sensations such as a sense of distance, a sense of speed, and a sense of acceleration. In particular, the inventors of the present disclosure have confirmed the following problem: in such a case, when the driver wants to stop the vehicle at a desired stop position, the vehicle will stop at a position that is excessively forward of the desired stop position. Summary of the Invention

[0005] An object of the present disclosure is to provide a display system and a display method that can prompt appropriate stopping at a desired stop position.

[0006] The first disclosure relates to a display system.

[0007] The display system according to the first disclosure includes: a camera that captures a traveling image in the traveling direction of the vehicle; a display device; one or more storage devices that store a reference deceleration indicating a prescribed deceleration; and one or more processors.

[0008] The one or more processors are configured to perform the following processes: acquire the vehicle speed; a stop position calculation process that calculates a predicted stop position indicating a stop position when braking is performed from the vehicle speed at the reference deceleration; and display the predicted stop position superimposed on the traveling image on the display device.

[0009] The second disclosure relates to a display system that further has the following features with respect to the display system according to the first disclosure.

[0010] The stop position calculation process includes the following process: when the vehicle speed is set to v and the reference deceleration is set to as When the constant is set to α, the predicted stopping distance x is calculated by the following formula (1): p ; and calculating a position that is the predicted stopping distance advanced along the travel path from the vehicle as the predicted stopping position,

[0011] [Formula 1]

[0012]

[0013] The third disclosure relates to a display system that further has the following features with respect to the display system of the first disclosure or the second disclosure.

[0014] The one or more processors are further configured to perform the following processing: identifying a stop line presented in the driving image and calculating the position of the stop line; and making the display mode of the predicted stopping position different between a case where the predicted stopping position is closer to the front side than the position of the stop line and a case where the predicted stopping position is farther to the rear side than the position of the stop line.

[0015] The fourth disclosure relates to a display system that further has the following features with respect to any one of the display systems of the first disclosure to the third disclosure.

[0016] The one or more processors are further configured to perform the following processing: accepting an input of a set value of the reference deceleration; and changing the reference deceleration stored in the one or more storage devices according to the set value.

[0017] The fifth disclosure relates to a display system that further has the following features with respect to any one of the display systems of the first disclosure to the third disclosure.

[0018] The one or more storage devices store proficiency information indicating proficiency related to a specific driver of the vehicle.

[0019] The one or more processors are further configured to perform the following processing: changing based on the proficiency information in such a way that the reference deceleration stored in the one or more storage devices increases as the proficiency increases.

[0020] The sixth disclosure relates to a display system that further has the following features with respect to any one of the display systems of the first disclosure to the fifth disclosure.

[0021] The reference deceleration includes a first reference deceleration, a second reference deceleration that is smaller than the first reference deceleration by a specified value, and a variable third reference deceleration that varies between the first reference deceleration and the second reference deceleration.

[0022] The one or more processors are configured to further perform the following processes: obtaining braking state information of the vehicle; determining the start of braking of the vehicle based on the braking state information; and starting from the time point of the start of braking, using the second reference deceleration as an initial value, gradually changing the third reference deceleration to the first reference deceleration according to the elapsed time or the braking state information.

[0023] The parking position calculation process includes the following processes: before the time point of the start of braking, calculating the predicted parking position based on the second reference deceleration; from the time point of the start of braking until the third reference deceleration reaches the first reference deceleration, calculating the predicted parking position based on the third reference deceleration; and after the third reference deceleration reaches the first reference deceleration, calculating the predicted parking position based on the first reference deceleration.

[0024] The seventh disclosure relates to a display method for displaying a traveling image of the traveling direction of a vehicle captured by a camera on a display device.

[0025] The display method of the seventh disclosure includes: calculating a predicted parking position representing a parking position in a case of braking with a reference deceleration representing a prescribed deceleration from the vehicle speed of the vehicle; and displaying the predicted parking position overlapping the traveling image on the display device.

[0026] The eighth disclosure relates to a display method having the following characteristics in addition to the display method of the seventh disclosure.

[0027] Calculating the predicted parking position includes: when setting the vehicle speed as v, setting the reference deceleration as a s and setting a constant as α, calculating a predicted parking distance x by the following arithmetic expression (1) p ; and calculating a position that has advanced the predicted parking distance along the traveling path from the vehicle as the predicted parking position.

[0028] [Expression 2]

[0029]

[0030] The ninth disclosure relates to a display method having the following characteristics in addition to the display method of the seventh disclosure or the eighth disclosure.

[0031] The display method of the ninth disclosure further includes: identifying a stop line presented in the traveling image and calculating the position of the stop line; and making the display mode of the predicted parking position different between a case where the predicted parking position is closer to the front side than the position of the stop line and a case where the predicted parking position is farther to the rear side than the position of the stop line.

[0032] The tenth disclosure relates to a display method having the following features in addition to the display method disclosed in any one of the seventh to ninth disclosures.

[0033] The display method of the tenth disclosure further includes: receiving an input of a set value of the reference deceleration; and changing the reference deceleration according to the set value.

[0034] The eleventh disclosure relates to a display method having the following features in addition to the display method disclosed in any one of the seventh to ninth disclosures.

[0035] The display method of the eleventh disclosure further includes: managing proficiency information indicating proficiency related to a specific driver of the vehicle; and changing the reference deceleration in such a manner that the reference deceleration increases as the proficiency increases, based on the proficiency information.

[0036] The twelfth disclosure relates to a display method having the following features in addition to the display method disclosed in any one of the seventh to eleventh disclosures.

[0037] The reference deceleration includes a first reference deceleration, a second reference deceleration smaller than the first reference deceleration by a specified value, and a variable third reference deceleration that varies between the first reference deceleration and the second reference deceleration.

[0038] The display method of the twelfth disclosure further includes: acquiring braking state information of the vehicle; judging the start of braking of the vehicle according to the braking state information; and starting from the time point of the start of braking, using the second reference deceleration as an initial value, and gradually changing the third reference deceleration to the first reference deceleration according to the elapsed time or the braking state information.

[0039] Calculating the predicted stop position includes: before the time point of the start of braking, calculating the predicted stop position based on the second reference deceleration; from the time point of the start of braking until the third reference deceleration reaches the first reference deceleration, calculating the predicted stop position based on the third reference deceleration; and after the third reference deceleration reaches the first reference deceleration, calculating the predicted stop position based on the first reference deceleration.

[0040] According to the present disclosure, a predicted parking position is superimposed and displayed on a traveling image in the traveling direction. Further, the predicted parking position is a parking position when braking is performed from the vehicle speed at a reference deceleration. Thereby, when a driver wants to stop the vehicle at a desired parking position, the vehicle can be appropriately stopped at the desired parking position. In particular, the following problem can be solved: when the driver visually recognizes the traveling image displayed on the display device without actually riding in the vehicle to perform driving operation, the vehicle stops at a position that is excessively forward of the desired parking position. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Hereinafter, with reference to the drawings, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, in which like reference numerals denote like elements, wherein:

[0042] Figure 1 is a graph showing an example of the deceleration of a typical vehicle in a case where the driver visually recognizes the traveling image displayed on the display device without actually riding in the vehicle and wants to stop the vehicle at a stop line.

[0043] Figure 2 is a block diagram for explaining an AR display function implemented by the display system of the first embodiment.

[0044] Figure 3 is shown in Figure 2 is a conceptual diagram showing an example of the predicted parking position of the vehicle calculated in the predicted parking position calculation processing unit.

[0045] Figure 4 is a conceptual diagram showing an example of the predicted parking position of the vehicle when the traveling path is a turning path.

[0046] Figure 5 is shown in Figure 2 is a conceptual diagram showing an example of the predicted parking position calculated in the coordinate transformation processing unit.

[0047] Figure 6 is a conceptual diagram showing an example of a traveling image in the traveling direction of the vehicle implemented by the display system of the first embodiment.

[0048] Figures 7A to 7C is a conceptual diagram for explaining a first effect achieved by the AR display function of the predicted parking position.

[0049] Figure 8A 、 Figure 8B is a conceptual diagram for explaining a second effect achieved by the AR display function of the predicted parking position.

[0050] Figures 9A to 9CIt is a conceptual diagram for explaining the solution to the problem through the AR display function of the predicted parking position.

[0051] Figure 10A , Figure 10B It is a conceptual diagram showing an example of another display method of the predicted parking position.

[0052] Figure 11 It is a block diagram showing the schematic configuration of the display system of the first embodiment.

[0053] Figure 12 It is a flowchart showing the display method implemented by the display system of the first embodiment.

[0054] Figure 13 It is a graph showing an example of the first embodiment.

[0055] Figure 14A , Figure 14B It is a diagram showing examples of the first embodiment with different set values of the reference deceleration for the driving operations of a specific driver.

[0056] Figure 15 It is a block diagram showing the schematic configuration of the display system of the second embodiment.

[0057] Figure 16 It is an example of a mapping diagram of the reference deceleration given for proficiency in the second embodiment.

[0058] Figure 17 It is a diagram showing an example of the third reference deceleration in the third embodiment.

[0059] Figure 18 It is an example of a mapping diagram of the third reference deceleration 113c given by the processing executed by one or more processors in the third embodiment. Detailed Embodiments

[0060] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the embodiments described below, when numerical values such as the number, quantity, amount, and range of each element are mentioned, unless otherwise explicitly stated or clearly determined to be that value in principle, the idea of the present disclosure is not limited to the mentioned value. In addition, regarding the configurations and the like described in the embodiments below, unless otherwise explicitly stated or clearly determined to be that configuration in principle, they are not necessarily essential in the idea of the present disclosure. It should be noted that the same or corresponding parts are denoted by the same reference numerals in the respective drawings, and the repeated description thereof is appropriately simplified or omitted.

[0061] 1. First Embodiment

[0062] 1-1. Introduction

[0063] When it is desired to stop the vehicle at a desired stop position such as a stop line, it can be considered that the driving operation performed by the driver of the vehicle is carried out as follows. First, the driver judges the start of braking. At this time, the driver assumes braking at a prescribed deceleration to estimate the braking distance, and judges the start of braking based on the remaining distance from the current position to the stop position. Next, the driver performs a driving operation (typically, stepping on the brake pedal) in such a way that the vehicle has a prescribed deceleration. At this time, the driver performs the driving operation while confirming the deceleration of the vehicle based on the view of the scenery in the traveling direction and the G applied to himself / herself. Finally, the driver stops the vehicle in alignment with the stop position. At this time, the driver performs the driving operation while confirming the deceleration of the vehicle so that the vehicle stops at the stop position.

[0064] Furthermore, in a remote driving system or the like, the driver of the vehicle visually recognizes the traveling image displayed on the display device without actually riding in the vehicle to perform a driving operation. The inventors of the present disclosure have confirmed the following problem: In such a case, when the driver wants to stop the vehicle at a desired stop position, the vehicle stops at a position that is overly forward of the desired stop position.

[0065] Figure 1 It is a graph showing an example of the typical deceleration of a vehicle in the case where the driver does not actually ride in the vehicle but visually recognizes the traveling image displayed on the display device to perform a driving operation and wants to stop the vehicle at the stop line. In Figure 1 As a comparison, an example of the typical deceleration of a vehicle when the driver riding in the vehicle wants to stop the vehicle at the stop line (dotted line) is also shown in the same manner. Figure 1 The deceleration of the vehicle with respect to the position of the vehicle is shown. Here, in Figure 1 the vertical axis represents the magnitude (absolute value) of the deceleration. In addition, in Figure 1 the position of the stop line (dashed line) is shown.

[0066] As Figure 1 shown, in the driving operation performed by visually recognizing the traveling image, the vehicle stops at a position that is overly forward of the stop line. It is considered that this is because in the case where the driver does not actually ride in the vehicle but visually recognizes the traveling image displayed on the display device to perform a driving operation, the driving sensations (distance sense, speed sense, acceleration sense) are lacking. In Figure 1 the example shown, in the driving operation performed by visually recognizing the traveling image, excessive deceleration is generated, and in addition, the deceleration is unstable. In addition, in the driving operation performed by visually recognizing the traveling image, there is a tendency for the timing of the start of braking to be uncertain.

[0067] In the display system of the first embodiment, in order to address the above problems, the predicted parking position when the vehicle stops is superimposed on the driving image. In addition, there are features in the way the predicted parking position is given. Hereinafter, the outline of the display system of the first embodiment will be described.

[0068] 1-2. Outline

[0069] The display system of the first embodiment provides a function of displaying the driving image of the vehicle on the display device. Here, the driving image of the vehicle includes the driving image in the traveling direction of the vehicle, and is captured by a camera equipped on the vehicle. In particular, the display system of the first embodiment superimposes the predicted parking position of the vehicle on the driving image in the traveling direction. The display of the predicted parking position of the vehicle is one of the AR (Augmented Reality) displays. Hereinafter, the function of superimposing the predicted parking position of the vehicle on the driving image will also be referred to as the "AR display function". Such a display system is considered to be adopted in a remote driving system that visually recognizes the driving image displayed on the display device to perform driving operations.

[0070] Figure 2 It is a block diagram for explaining the AR display function implemented by the display system of the first embodiment. The AR display function is composed of a predicted parking position calculation processing unit 121, a coordinate transformation processing unit 122, and a rendering processing unit 123.

[0071] First, in the predicted parking position calculation processing unit 121, the driving state information of the vehicle, the vehicle specification information, and a reference deceleration indicating a specified deceleration are acquired, and the predicted parking position of the vehicle is calculated. Here, the reference deceleration is a specified value and is stored in the storage device for management. In addition, the driving state information of the vehicle includes at least the current vehicle speed of the vehicle. In addition, as the driving state information of the vehicle, examples include acceleration / deceleration and steering angle. As the vehicle specification information, examples include vehicle weight, stability coefficient, cornering power, wheelbase, and steering gear ratio.

[0072] In the predicted parking position calculation processing unit 121, the representation of the predicted parking position of the vehicle in the spatial coordinates (spatial coordinate representation) is given as the processing result. Figure 3 shows an example of the predicted parking position 2 of the vehicle 1 calculated in the predicted parking position calculation processing unit 121. In Figure 3 the example shown, the spatial coordinates are two-dimensional orthogonal coordinates. Therefore, the predicted parking position 2 is represented by two-dimensional coordinates (x, y). As shown in Figure 3 the predicted parking position 2 is x forward from the vehicle 1 along the travel path 3 p(Predicted stopping distance) position. Here, when the vehicle 1 is turned, the travel path 3 can be a turning path. In this case, the travel path 3 can be calculated based on the driving state information and the vehicle specification information. Figure 4 An example of the predicted stopping position 2 of the vehicle 1 when the travel path 3 shown in Figure 4 is a turning path is shown.

[0073] The predicted stopping position 2 calculated in the predicted stopping position calculation processing unit 121 is characterized in that it is the stopping position when braking is performed from the current vehicle speed of the vehicle 1 at the reference deceleration. That is, the predicted stopping distance x p is the braking distance at the reference deceleration. Therefore, when the current vehicle speed of the vehicle 1 is set to v and the reference deceleration is set to a s (<0), the predicted stopping distance x is calculated by the following formula (1) p . It should be noted that α is a constant, typically -1 / 2. However, it can also be a parameter adjusted according to the environment.

[0074] [Equation 3]

[0075]

[0076] Refer to again Figure 2 . Next, in the coordinate transformation processing unit 122, based on the specification information of the camera that captures the driving image (camera specification information), the coordinate transformation of the predicted stopping position 2 calculated in the predicted stopping position calculation processing unit 121 is performed, and the representation of the predicted stopping position 2 in the screen coordinates (screen coordinate representation) is calculated. Here, as the camera specification information, the installation position, installation angle, and field of view angle of the camera are exemplified. In addition, the screen coordinates give the position on the image captured by the camera, and the position of the screen coordinates can be given in a manner corresponding to the position of the spatial coordinates.

[0077] Figure 5 An example of the predicted stopping position 2 calculated in the coordinate transformation processing unit 122 is shown. Figure 5 Shows the Figure 3 Screen coordinate representation of the predicted stopping position 2 corresponding to the spatial coordinate representation of the predicted stopping position 2 shown.

[0078] Refer to again Figure 2 . Next, in the rendering processing unit 123, a display signal for AR-displaying the predicted stopping position 2 calculated in the coordinate transformation processing unit 122 on the display device is generated. The AR display of the predicted stopping position 2 is realized by the display device displaying according to the display signal generated in the rendering processing unit 123.

[0079] Figure 6An example of a driving image 4 of the traveling direction of the vehicle 1 realized by the display system of the first embodiment is shown. Figure 6 The driving image 4 in the situation where the vehicle 1 is traveling near the stop line 5 is shown. It should be noted that in Figure 6 the example of the driving image 4 shown, the traveling path 3 is also AR - displayed. In this case, in the coordinate transformation processing unit 122 and the drawing processing unit 123, coordinate transformation of the traveling path 3 and generation of a display signal are performed. The visual recognition of the predicted stop position 2 can be improved by performing the AR display of the traveling path 3 together.

[0080] As described above, in the display system of the first embodiment, the predicted stop position 2 is overlapped and displayed on the driving image 4 in the traveling direction. In addition, in the display system of the first embodiment, the predicted stop position 2 is characterized in that it is the stop position when braking is performed at a reference deceleration indicating a prescribed deceleration from the current vehicle speed of the vehicle 1. Thus, even when the driver visually recognizes the driving image 4 displayed on the display device without actually riding in the vehicle to perform a driving operation, it can have the effect of solving the problem of stopping at a position that is overly forward of the desired stop position. Hereinafter, the effects achieved by the AR display function of the predicted stop position 2 will be described.

[0081] The first effect achieved by the AR display function of the predicted stop position 2 is that the driver can grasp the timing of starting braking. Figures 7A to 7C It is a conceptual diagram for explaining the first effect achieved by the AR display function of the predicted stop position 2. Figures 7A to 7C The situation where the driver wants the vehicle 1 to stop at the stop line 5 is shown, and three situations with different positional relationships between the predicted stop position 2 and the stop line 5 are shown Figure 7A , Figure 7B and Figure 7C .

[0082] The driver can confirm the difference between the braking distance at the reference deceleration and the remaining distance from the current position to the stop line 5 by the AR display of the predicted stop position 2. Furthermore, the driver can grasp the situation where the predicted stop position 2 becomes near the stop line 5 as shown in Figure 7B as the timing of starting braking. In addition, in the situation shown in Figure 7A , it can be grasped that braking does not need to start yet or braking starts too early, and in the situation shown in Figure 7C , it can be grasped that braking should be started immediately or braking starts too late.

[0083] The second effect achieved by the AR display function of the predicted stop position 2 is that the driver can grasp the difference between the current deceleration of the vehicle 1 and the reference deceleration. Figure 8A ,Figure 8B This is a conceptual diagram for explaining the second effect achieved by the AR display function at the predicted parking position 2. Figure 8A An example is shown where the absolute value of the current deceleration of the vehicle 1 is greater than the absolute value of the reference deceleration. Figure 8B An example is shown where the absolute value of the current deceleration of the vehicle 1 is less than the absolute value of the reference deceleration. Hereinafter, "greater" or "less" in the relationship between the deceleration of the vehicle 1 and the reference deceleration is considered with respect to the absolute value.

[0084] As Figure 8A shown, when the current deceleration of the vehicle 1 is greater than the reference deceleration, the predicted parking position 2 is displayed as approaching the vehicle 1. On the other hand, as Figure 8B shown, when the current deceleration of the vehicle 1 is less than the reference deceleration, the predicted parking position 2 is displayed as being away from the vehicle 1. That is, when the display of the predicted parking position 2 remains stationary, it indicates that the deceleration of the vehicle 1 is the reference deceleration. In this way, the driver can grasp the difference between the current deceleration of the vehicle 1 and the reference deceleration based on the change in the display of the predicted parking position 2. Furthermore, the driver can perform driving operations in such a way that the deceleration of the vehicle 1 is set to the reference deceleration by confirming the display of the predicted parking position 2.

[0085] It should be noted that the speed of approach or departure of the predicted parking position 2 in the spatial coordinate representation can be expressed by the following formula (2). Here, a (<0) is the current deceleration of the vehicle 1. In formula (2), the direction approaching the predicted parking position 2 is positive. By referring to formula (2), it can also be known that the predicted parking position 2 approaches when the current deceleration of the vehicle 1 is less than the reference deceleration, and the predicted parking position 2 departs when the current deceleration of the vehicle 1 is greater than the reference deceleration.

[0086] [Equation 4]

[0087]

[0088] By achieving the first and second effects described above, the following problem can be solved: When the driver wants the vehicle 1 to stop at a desired parking position, the vehicle 1 stops at a position that is overly ahead of the desired parking position. Referring to Figures 9A to 9C , the solution to the problem will be described. In Figures 9A to 9C , the desired parking position is the stop line 5.

[0089] First, through the first effect, the driver can set the timing of starting braking in a situation where the predicted stop position 2 is near the stop line 5. This means that as long as the vehicle 1 is braked at the reference deceleration, the vehicle 1 can be stopped near the stop line 5. That is to say, after starting braking, as long as the driver operates the vehicle 1 in a way that sets the deceleration of the vehicle 1 to the reference deceleration, the vehicle 1 can be stopped near the stop line 5.

[0090] Therefore, next, through the second effect, the driver can operate the vehicle 1 in a way that sets the deceleration of the vehicle 1 to the reference deceleration by confirming the display of the predicted stop position 2. That is to say, the driver can operate the vehicle 1 in such a way that the predicted stop position 2 is maintained near the stop line 5 as shown in Figure 9A . Thus, the vehicle 1 can be stopped near the stop line 5.

[0091] However, depending on the driving operation situation and driving operation delay of the driver, consider the situation where after starting braking, it becomes the situation shown in Figure 9B or Figure 9C . In this case, the driver can also operate the vehicle 1 in such a way that the predicted stop position 2 is near the stop line 5 by confirming the display of the predicted stop position 2. For example, in the situation shown in Figure 9B , the driver can grasp that as long as the braking of the vehicle 1 is relaxed (typically, the operation amount of the brake pedal is reduced) so that the deceleration of the vehicle 1 is less than the reference deceleration. Thus, the predicted stop position 2 moves away from the vehicle 1, and the predicted stop position 2 can be made near the stop line 5. On the other hand, in the situation shown in Figure 9C , the driver can grasp that as long as the braking of the vehicle 1 is strengthened (typically, the operation amount of the brake pedal is increased) so that the deceleration of the vehicle 1 is greater than the reference deceleration. Thus, the predicted stop position 2 moves closer to the vehicle 1, and the predicted stop position 2 can be made near the stop line 5. It should be noted that after operating the vehicle 1 in such a way that the predicted stop position 2 is near the stop line 5, the vehicle 1 is operated in such a way that the predicted stop position 2 is maintained near the stop line 5, thereby enabling the vehicle 1 to be stopped near the stop line 5.

[0092] In this way, the problem of stopping at a position that is overly forward from the desired stop position can be solved by the AR display function of the predicted stop position 2 in the first embodiment.

[0093] Furthermore, regarding the predicted stop position 2, it can be considered that the reference deceleration can also be given as the deceleration of the current vehicle 1. However, if the deceleration of the current vehicle 1 is used instead of the reference deceleration, the predicted stop position 2 is not displayed until the braking of the vehicle 1 starts. That is, the first effect cannot be achieved. In addition, in the second effect, the movement of the predicted stop position 2 also affects the jerk and becomes too sensitive. Furthermore, the driving operation corresponding to the movement of the predicted stop position 2 becomes difficult.

[0094] In addition, as another method of AR display, it is also possible to consider AR-displaying the position of the vehicle 1 after a specified time of traveling at the current vehicle speed of the vehicle 1. In this way, the first effect can also be achieved by giving the AR display. However, in such an AR display, the driver cannot grasp the deceleration of the vehicle 1 (the second effect is not achieved), so it is difficult to perform the driving operation in such a way that the AR display of the predicted stop position 2 is near the stop line 5. Furthermore, the accuracy of the stop position will decrease.

[0095] In this way, by AR-displaying the predicted stop position 2 as the stop position when braking from the current vehicle speed of the vehicle 1 at the reference deceleration, the first effect and the second effect can be fully achieved.

[0096] It should be noted that the display method of the predicted stop position 2 can also adopt other methods. Figure 10A 、 Figure 10B is a conceptual diagram showing an example of another display method of the predicted stop position 2. As another display method of the predicted stop position 2, Figure 10A and Figure 10B these two examples are shown. Figure 10A is the case where the predicted stop position 2 is set to a frame matching the length of the vehicle 1. In this way, it is possible to confirm the position of the entire vehicle body at the predicted stop position 2 by performing AR display. Figure 10B is the case where vehicle width lines 6 are displayed at regular intervals on the traveling path 3 up to the predicted stop position 2 together with the AR display of the predicted stop position 2. By performing AR display in this way, the driver can confirm the scale of the distance on the driving image. Furthermore, the driver's sense of distance can be improved.

[0097] Such an AR display can be realized by configuring the predicted stop position calculation processing unit 121 in such a way as to calculate and display the spatial coordinates of the predicted stop position 2 corresponding to the display method.

[0098] 1 - 3. Display System

[0099] Hereinafter, the configuration of the display system according to the first embodiment will be described. Figure 11FIG. 0 is a block diagram schematically showing the configuration of a display system 10 according to a first embodiment. The display system 10 includes an information processing device 100, a camera 200, a driving state detection sensor 300, a display device 400, and an input device 500. The information processing device 100 is connected so as to be able to communicate with the camera 200, the driving state detection sensor 300, the display device 400, and the input device 500. For example, electrical connection via a cable, connection via an optical communication line, connection via wireless communication through a wireless communication terminal, etc. may be cited. It should be noted that the transmission of information may also be indirectly performed via a relay device.

[0100] The camera 200 is mounted on the vehicle 1 and captures a driving image of the vehicle 1. The driving image captured by the camera 200 is transmitted to the information processing device 100.

[0101] The driving state detection sensor 300 is a sensor that detects driving state information of the vehicle 1 and outputs detection information. The driving state detection sensor 300 includes at least a sensor that detects the vehicle speed of the vehicle 1 (for example, a wheel speed sensor). In addition, as the driving state detection sensor 300, a G sensor that detects the acceleration and deceleration of the vehicle 1, a steering angle sensor that detects the steering angle of the vehicle 1, etc. are exemplified. The detection information is transmitted to the information processing device 100.

[0102] The information processing device 100 is a computer that outputs a display signal for controlling the display of the display device 400 based on the acquired information. The information processing device 100 may also be a computer that outputs a display signal as one of its functions. For example, the information processing device 100 may be a computer mounted on a remote driving device and performing remote driving processing.

[0103] The information processing device 100 includes one or more storage devices 110 and one or more processors 120.

[0104] The one or more storage devices 110 store a control program 111 that can be executed by the one or more processors 120 and control information 112 required for the processing executed by the one or more processors 120. As the one or more storage devices 110, a volatile memory, a non-volatile memory, an HDD, an SSD, etc. are exemplified. The information acquired by the information processing device 100 is stored in the storage device 110 as control information 112.

[0105] The control program 111 includes a program for generating a display signal for displaying the driving image 4 on the display device 400 and a program for generating a display signal for performing AR display of the predicted parking position 2 on the display device 400.

[0106] The control information 112 includes at least a reference deceleration 113. In addition, as the control information 112, for example, the driving image 4 acquired from the camera 200, the detection information acquired from the driving state detection sensor 300, the vehicle specification information of the vehicle 1, the camera specification information of the camera 200, and the parameter information of the control program 111 are exemplified.

[0107] One or more processors 120 read out the control program 111 and the control information 112 from one or more storage devices 110, and execute the processing according to the control program 111 based on the control information 112. As a result, a display signal for displaying the driving image and a display signal for AR-displaying the predicted stop position 2 are generated. That is, the predicted stop position calculation processing unit 121, the coordinate transformation processing unit 122, and the rendering processing unit 123 are respectively implemented by one or more processors 120.

[0108] It should be noted that one or more storage devices 110 can also be configured as devices external to the information processing device 100. For example, one or more storage devices 110 are data servers configured on a communication network. In this case, the information processing device 100 only needs to be configured to be able to acquire the information stored in one or more storage devices 110 through communication via the communication network.

[0109] The display device 400 performs display according to the display signal acquired from the information processing device 100. The display device 400 is, for example, a monitor provided in the cockpit of a remote driving system. The display device 400 performs display according to the display signal, thereby realizing the display of the driving image 4 and the AR display of the predicted stop position 2.

[0110] The input device 500 is a device that accepts a user's operation and outputs operation information corresponding to the operation. As the input device 500, for example, a touch panel, a keyboard, a switch, etc. are exemplified. Or it is an operation panel provided in the cockpit of a remote driving system. In addition, the input device 500 can also be integrally formed with the display device 400. The operation information is transmitted to the information processing device 100. In the information processing device 100, the processing corresponding to the operation information is executed by one or more processors 120 to realize setting changes made by the user (for example, changing the camera for shooting the driving image, changing the display method), parameter changes of the processing (for example, changing the vehicle specification information, the camera specification information), etc.

[0111] In particular, the input device 500 is configured to be able to input a set value of the reference deceleration 113. The input form of the set value can be appropriately given according to the environment of the application display system 10. For example, regarding the value of the reference deceleration 113, it can be in the form of being arbitrarily input or in the form of being selected from several set value options. And, one or more processors 120 are configured to perform the following processing: accept the input of the set value of the reference deceleration 113, and change the reference deceleration 113 stored in one or more storage devices 110 according to the input set value. Thereby, the reference deceleration 113 can be set according to the driver's preference and adaptability.

[0112] 1 - 4. Display method

[0113] Hereinafter, the display method implemented by the display system 10 of the first embodiment will be described. Figure 12 It is a flowchart showing the display method implemented by the display system 10 of the first embodiment. Figure 12 The shown flowchart is repeated at a prescribed cycle, and each process is executed for each prescribed cycle.

[0114] In step S100, the information processing device 100 acquires the traveling image 4 captured by the camera 200 and the detection information (including the vehicle speed of the vehicle 1) detected by the traveling state detection sensor 300.

[0115] In step S200 (parking position calculation process), one or more processors 120 calculate the predicted parking position 2. Here, one or more processors 120 calculate the predicted stopping distance x through formula (1) based on the vehicle speed acquired in step S100 and the reference deceleration 113 stored in one or more storage devices 110 p . Then, the position that has advanced the predicted stopping distance along the traveling path 3 from the vehicle 1 is calculated as the predicted parking position 2. It should be noted that the traveling path 3 is calculated based on the detection information and the vehicle specification information. For example, when the vehicle 1 is turned, the traveling path 3 can be given by a steady circular turning locus. When the vehicle 1 is not turned, the traveling path 3 can be a path that goes straight ahead of the vehicle 1.

[0116] In step S300, one or more processors 120 perform coordinate transformation of the predicted parking position 2 calculated in step S200 and calculate the screen coordinate representation of the predicted parking position 2.

[0117] In step S400, one or more processors 120 generate a display signal for displaying a driving image 4 and a display signal for AR-displaying a predicted stop position 2 calculated in step S300. Then, the information processing device 100 outputs the generated display signals, and the display device 400 performs display according to the display signals.

[0118] 1-4. Effects

[0119] As described above, according to the first embodiment, the predicted stop position 2 is overlapped and displayed on the driving image 4 in the traveling direction. In addition, the predicted stop position 2 is the stop position when braking from the current vehicle speed of the vehicle 1 at the reference deceleration 113. Thus, when the driver wants to stop the vehicle 1 at a desired stop position, the vehicle 1 can be prompted to stop appropriately at the desired stop position. In particular, when the driver visually recognizes the driving image 4 displayed on the display device without actually riding in the vehicle to perform driving operations, the problem of stopping at a position that is overly ahead of the desired stop position can be solved.

[0120] Figure 13 An example of the first embodiment is shown. Figure 13 Shows the same as Figure 1 The same curve graph. It should be noted that Figure 13 is an example when the reference deceleration 113 is set to 0.15G (1.5 m / s 2 ). As Figure 13 shown, it can be seen that the problem of stopping at a position that is overly ahead of the stop line 5 can be solved by performing the AR display of the predicted stop position 2 of the first embodiment. In addition, compared with the driving operations performed while riding in the vehicle, it is also expected that the driver can stop the vehicle 1 closer to the stop line 5.

[0121] Moreover, according to the first embodiment, an input of a set value of the reference deceleration 113 is accepted, and the reference deceleration 113 is changed according to the input set value.

[0122] The inventors of the present disclosure found that according to the driver, the set value of the reference deceleration 113 affects driving operations. Figure 14A 、 Figure 14B Examples (0.1G, 0.15G, 0.2G) of three first embodiments with different set values of the reference deceleration 113 for the case of a specific driver performing driving operations are shown. Figure 14A Shows the same as Figure 1 The same curve graph. Figure 14B For multiple implementations of each set value of different set values of the reference deceleration 113, the remaining distance from the position relative to the stop line 5 (positive when ahead of the stop line 5) is shown. As Figure 14AAs shown, it can be seen that the set value of the reference deceleration 113 affects the driving operation. In addition, as Figure 14B shown, the set value of the reference deceleration 113 affects the accuracy of the parking position (remaining distance).

[0123] In this way, by being able to set the reference deceleration 113, it is possible to cope with the driver's preferences and adaptability.

[0124] 1-5. Variation

[0125] The first embodiment can also adopt the following modified solutions.

[0126] 1-5-1. First variation

[0127] It can also be configured such that when the predicted parking position 2 is closer to the front side than the position of the stop line 5 ( Figure 7B the situation shown) and when the predicted parking position 2 is farther from the front side than the position of the stop line 5 ( Figure 7C the situation shown), the display mode of the predicted parking position 2 is different.

[0128] For example, when the predicted parking position 2 is closer to the front side than the position of the stop line 5, the color of the predicted parking position 2 is set to an emphasized color such as red. In addition to this, consider making the pattern and shape of the predicted parking position 2 different. Moreover, it can also be configured to perform a notification by sound such as sounding a buzzer when the predicted parking position 2 becomes farther from the front side than the position of the stop line 5.

[0129] The display system 10 of the first variation can be implemented by configuring one or more processors 120 to further perform the following processing: identifying the stop line 5 presented in the driving image 4 and calculating the position of the stop line 5; and making the display mode of the predicted parking position 2 different between the situation where the predicted parking position 2 is closer to the front side than the position of the stop line 5 and the situation where the predicted parking position 2 is farther from the front side than the position of the stop line 5. Here, the identification of the stop line 5 presented in the driving image 4 and the calculation of the position of the stop line 5 are realized by image recognition technology, for example.

[0130] By adopting the first variation, the driver can more easily grasp the timing of starting braking.

[0131] 1-5-2. Second variation

[0132] One or more processors 120 may also be configured to perform the following processes: receiving the load of the vehicle 1 and the riding mode of the passengers as inputs, and changing the reference deceleration 113 stored in one or more storage devices 110 according to the received load of the vehicle 1 and the riding mode of the passengers. Here, the input of the load of the vehicle 1 and the riding mode of the passengers means, for example, that precision equipment, fragile items, etc. are loaded and the passengers ride standing. In addition, the input is realized by operating the input device 500. Alternatively, it may be configured to obtain the load of the vehicle 1 and the riding mode of the passengers based on image recognition performed by a camera that captures the interior of the vehicle 1.

[0133] One or more processors 120, for example, when precision equipment or fragile items are loaded, change the reference deceleration 113 in a way that reduces it. In addition, when the passengers ride standing, change the reference deceleration 113 in a way that reduces it. Thus, as Figure 14A , Figure 14B shown, the deceleration during braking of the vehicle 1 can be made slower. Furthermore, the load can be appropriately protected when the load of the vehicle 1 is precision equipment or fragile items. In addition, the state of the passengers can be stabilized when the passengers of the vehicle 1 ride standing.

[0134] 2. Second Embodiment

[0135] Hereinafter, the second embodiment will be described. In the following description, the differences from the first embodiment will be described, and the content repeated with the first embodiment will be appropriately omitted.

[0136] Figure 15 is a block diagram showing a schematic configuration of the display system 10 according to the second embodiment. In the display system 10 according to the second embodiment, one or more storage devices 110 store proficiency information 114 as control information 112 and manage it. The proficiency information 114 is information indicating the proficiency related to a specific driver. The proficiency is, for example, indexed by the cumulative driving distance and driving skills. More specifically, the longer the cumulative driving distance, the higher the proficiency is set, or the driving skills are evaluated based on the lateral G, longitudinal G, etc. detected during driving (the smaller the change in G, the higher the driving skills, etc.), and the higher the driving skills, the higher the proficiency is set. Note that one or more storage devices 110 may also store and manage proficiency information 114 related to each of multiple drivers.

[0137] One or more processors 120 of the second embodiment perform the following processing: Based on the proficiency information 114, the reference deceleration 113 stored in one or more storage devices 110 is changed in such a manner that it increases as the proficiency increases. For example, one or more processors 120 perform processing to change the reference deceleration 113 according to a mapping diagram that gives the reference deceleration 113 for proficiency. Figure 16 An example of a mapping diagram that gives the reference deceleration 113 for proficiency is shown. When performing processing to change the reference deceleration 113 according to the Figure 16 shown mapping diagram, as the proficiency increases from S1 to S2 (> S1), the reference deceleration 113 monotonically increases from a1 to a2. On the other hand, before the proficiency becomes S1, the reference deceleration 113 becomes a1, and after the proficiency becomes S2, the reference deceleration 113 becomes a2.

[0138] Figure 16 The shown mapping diagram is an example, and an appropriate mapping diagram can also be adopted according to the environment of the display system 10 to which the second embodiment is applied. For example, a mapping diagram in which the reference deceleration 113 increases non-linearly as the proficiency increases can also be adopted. Or a mapping diagram in which the reference deceleration 113 increases stepwise discontinuously as the proficiency increases can also be adopted.

[0139] It should be noted that, in the case of managing the proficiency information 114 for each of multiple drivers, one or more storage devices 110 manage the reference deceleration 113 for each of the multiple drivers, and one or more processors 120 are configured to perform processing to change the reference deceleration 113 for each of the multiple drivers.

[0140] The smaller the reference deceleration 113 is, the longer the margin of time for the vehicle 1 to reach near the stop line 5 is. Therefore, there is a tendency for drivers with low proficiency to prefer to set the reference deceleration 113 small, and drivers with high proficiency to prefer to set the reference deceleration 113 high. Therefore, by applying the second embodiment, an appropriate reference deceleration 113 corresponding to the driver's proficiency can be given.

[0141] 3. Third Embodiment

[0142] Hereinafter, the third embodiment will be described. In the following description, the differences from the first embodiment will be described, and the content repeated with the first embodiment will be omitted as appropriate.

[0143] In the third embodiment, the reference deceleration 113 includes a first reference deceleration 113a, a second reference deceleration 113b that is smaller than the first reference deceleration 113a by a specified value, and a variable third reference deceleration 113c that varies between the first reference deceleration 113a and the second reference deceleration 113b (see Figure 17 ). The first reference deceleration 113a and the second reference deceleration 113b (or the specified value) can be given appropriate values according to the environment of the display system 10 to which the third embodiment is applied. Among them, the first reference deceleration 113a and the second reference deceleration 113b (or the specified value) can also be configured to be set by the operation of the input device 500. The third reference deceleration 113c changes by the execution of processing by one or more processors 120. In addition, the information processing device 100 is configured to be able to acquire braking state information indicating the braking state of the vehicle 1. As the braking state information, for example, the on / off state of the stop lamp switch, the amount of depression of the brake pedal, etc. are exemplified. These can also be given as the detection information of the driving state detection sensor 300. Or it can also be configured to be acquired from the vehicle 1 through communication.

[0144] In the third embodiment, one or more processors 120 execute a process of determining the start of braking of the vehicle 1 based on the braking state information. For example, it is determined that the braking of the vehicle 1 has started based on the stop lamp switch becoming on or the detection of the depression of the brake pedal. Then, one or more processors 120 execute the following process: starting from the time point of the start of braking, with the second reference deceleration 113b as the initial value, the third reference deceleration 113c gradually changes to the first reference deceleration 113a according to the progress of braking. Here, the progress of braking can be determined based on the elapsed time from the time point of the start of braking. That is, the greater the elapsed time, the more it is determined that the braking has progressed. Alternatively, the progress of braking can also be determined based on the braking state information. For example, the greater the cumulative value of the amount of depression of the brake pedal from the time point of the start of braking, the more it is determined that the braking has progressed. Figure 18 An example of a mapping diagram of the third reference deceleration 113c given by the processing executed by one or more processors 120 is shown in Figure 18 In the example shown, the third reference deceleration 113c starts with the second reference deceleration 113b as the initial value and monotonically increases to the first reference deceleration 113a as the braking progresses. However, Figure 18The third reference deceleration 113c shown is an example, and an appropriate mapping diagram can also be adopted according to the environment in which the display system 10 applying the third embodiment is used. For example, a mapping diagram in which the third reference deceleration 113c increases non-linearly to the first reference deceleration 113a as braking progresses can also be adopted. Alternatively, a mapping diagram in which the third reference deceleration 113c increases stepwise discontinuously to the first reference deceleration 113a as braking progresses can also be adopted.

[0145] In the third embodiment, in the parking position calculation process ( Figure 12 step S200), one or more processors 120 calculate the predicted parking position 2 based on the second reference deceleration 113b before the time point of the start of braking. On the other hand, from the time point of the start of braking until the third reference deceleration 113c reaches the first reference deceleration 113a, the predicted parking position 2 is calculated based on the third reference deceleration 113c. Then, after the third reference deceleration 113c reaches the first reference deceleration 113a, the predicted parking position 2 is calculated based on the first reference deceleration 113a.

[0146] As Figure 14A , Figure 14B shown, when the reference deceleration 113 increases, the rise of braking at the start of braking becomes steeper, so there is a concern that pitching of the vehicle 1 may occur. On the other hand, when the reference deceleration 113 is decreased, the rise of braking at the start of braking can be suppressed, but the driving operation after the start of braking becomes excessively slow, so the comfort may be impaired.

[0147] Therefore, by applying the third embodiment, the predicted parking position 2 calculated based on the small reference deceleration 113 (second reference deceleration 113b) is AR-displayed until the start time point of braking, and on the other hand, after the start of braking, the predicted parking position 2 calculated based on the reference deceleration 113 (third reference deceleration 113c) that gradually increases as braking progresses is AR-displayed. Thereby, the rise of braking at the start of braking can be suppressed, and the driving operation after the start of braking does not become excessively slow. Furthermore, pitching of the vehicle 1 at the start of braking can be suppressed.

Claims

1. A display system, characterized in that, Comprising: A camera that captures a driving image of the traveling direction of the vehicle; A display device; One or more storage devices that store a reference deceleration representing a prescribed deceleration; And One or more processors, The one or more processors are configured to perform the following processing: Obtain the vehicle speed of the vehicle; A parking position calculation process that calculates a predicted parking position representing the parking position when braking is performed from the vehicle speed at the reference deceleration; Cause the predicted parking position to be displayed on the display device overlapping the driving image; Identify a stop line presented in the driving image and calculate the position of the stop line; And Between a case where the predicted parking position is closer to the front side than the position of the stop line and a case where the predicted parking position is farther to the rear side than the position of the stop line, make the display mode of the predicted parking position different, The reference deceleration includes a first reference deceleration, a second reference deceleration smaller than the first reference deceleration by a prescribed value, and a variable third reference deceleration that varies between the first reference deceleration and the second reference deceleration, The one or more processors are configured to further perform the following processing: Obtain the braking state information of the vehicle; Judge the start of braking of the vehicle based on the braking state information; and From the time point of the start of braking, with the second reference deceleration as the initial value, gradually change the third reference deceleration to the first reference deceleration according to the elapsed time or the braking state information, The parking position calculation process includes the following processing: Before the time point of the start of braking, calculate the predicted parking position based on the second reference deceleration; From the time point of the start of braking until the third reference deceleration reaches the first reference deceleration, calculate the predicted parking position based on the third reference deceleration; And After the third reference deceleration reaches the first reference deceleration, calculate the predicted parking position based on the first reference deceleration.

2. The display system according to claim 1, wherein The parking position calculation process includes the following processing: When setting the vehicle speed to v, setting the reference deceleration to a s and setting a constant to α, the predicted stopping distance x is calculated by the following formula (1) p ; And Calculate a position that has advanced the predicted parking distance along the traveling path from the vehicle as the predicted parking position, 3. The display system according to claim 1 or 2, wherein The one or more processors are configured to further perform the following processing: Accept an input of a set value of the reference deceleration; and Change the reference deceleration stored in the one or more storage devices according to the set value.

4. The display system according to claim 1 or 2, wherein The one or more storage devices store proficiency information representing the proficiency related to a specific driver of the vehicle, The one or more processors are configured to further perform the following processing: Based on the proficiency information, change in such a manner that the reference deceleration stored in the one or more storage devices increases as the proficiency increases.

5. A display method for displaying a driving image of the traveling direction of a vehicle captured by a camera on a display device, wherein the display method includes: Calculating an estimated stopping position representing a stopping position in a case of braking from the vehicle speed of the vehicle at a reference deceleration representing a prescribed deceleration; Causing the estimated stopping position to be displayed on the display device so as to overlap the driving image; Identifying a stop line presented in the driving image and calculating the position of the stop line; And Making the display mode of the estimated stopping position different between a case where the estimated stopping position is closer to the front side than the position of the stop line and a case where the estimated stopping position is farther to the rear side than the position of the stop line; The reference deceleration includes a first reference deceleration, a second reference deceleration smaller than the first reference deceleration by a prescribed value, and a variable third reference deceleration that varies between the first reference deceleration and the second reference deceleration; The display method further includes: Obtaining braking state information of the vehicle; Judging the start of braking of the vehicle based on the braking state information; and From the time point of the start of braking, using the second reference deceleration as an initial value, gradually changing the third reference deceleration to the first reference deceleration according to the elapsed time or the braking state information; Calculating the estimated stopping position includes: Before the time point of the start of braking, calculating the estimated stopping position based on the second reference deceleration; From the time point of the start of braking until the third reference deceleration reaches the first reference deceleration, calculating the estimated stopping position based on the third reference deceleration; and After the third reference deceleration reaches the first reference deceleration, calculating the estimated stopping position based on the first reference deceleration.

6. The display method according to claim 5, wherein Calculating the estimated stopping position includes: When setting the vehicle speed to v, setting the reference deceleration to a s and setting a constant to α, the predicted stopping distance x is calculated by the following formula (1) p ; and Calculating a position that has advanced the estimated stopping distance along the traveling path from the vehicle as the estimated stopping position; 7. The display method according to claim 5 or 6, characterized in that, It further includes: Accepting an input of a set value of the reference deceleration; And Changing the reference deceleration according to the set value.

8. The display method according to claim 5 or 6, characterized in that It further includes: Managing proficiency information indicating proficiency related to a specific driver of the vehicle; And Based on the proficiency information, changing in such a manner that the reference deceleration increases as the proficiency increases.

Images