Display control system and display control method

Abstract

To easily reduce anxiety of a driver of a vehicle during driving.SOLUTION: The display control system 1 includes a first acquisition unit 11, a second acquisition unit 12, an extraction unit 13, a generation unit 15, and a display control unit 16. The first acquisition unit 11 acquires a surrounding image including at least an area in front of the vehicle 100 and an area lateral to the vehicle 100 on the side where the obstacle is present. Second obtainer 12 obtains the steering angle of vehicle 100. The extraction unit 13 extracts an edge of the obstacle in the peripheral image. The generation unit 15 generates an emphasized image in which the extracted edges are emphasized and a route image indicating a predicted traveling route of the vehicle 100 based on the steering angle. The display control unit 16 causes the display device 4 mounted on the vehicle 100 to display the generated highlighted image and route image in a superimposed manner.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a display control system and a display control method for controlling a display device mounted on a vehicle. [Background technology]

[0002] Patent Document 1 discloses a vehicle surroundings monitoring device. This vehicle surroundings monitoring device captures images of the sides of the vehicle, including the sides of the vehicle, detects edges parallel to the road based on the captured images, and extracts the edges of three-dimensional obstacles from the detected edges. This vehicle surroundings monitoring device also notifies the occupant of the proximity of the three-dimensional obstacle to the vehicle based on the positional relationship between the extracted edges of the three-dimensional obstacle and the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4033017 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a display control system and the like that can easily reduce the anxiety that vehicle drivers feel when driving. [Means for solving the problem]

[0005] A display control system according to one aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, an extraction unit, a generation unit, and a display control unit. The first acquisition unit acquires a peripheral image including at least a front side of a vehicle and a side of the vehicle on which an obstacle exists. The second acquisition unit acquires a steering angle of the vehicle. The extraction unit extracts edges of the obstacle in the peripheral image. The generation unit generates an enhanced image that emphasizes the extracted edges, and a route image that shows a predicted driving route of the vehicle based on the steering angle. The display control unit superimposes and displays the generated enhanced image and route image on a display device mounted on the vehicle.

[0006] A display control method according to one aspect of the present disclosure acquires a peripheral image including at least an area ahead of a vehicle and an area to the side of the vehicle where an obstacle is present. The display control method acquires a steering angle of the vehicle. The display control method extracts edges of the obstacle in the peripheral image. The display control method generates an enhanced image that emphasizes the extracted edges, and a route image that shows a predicted driving route of the vehicle based on the steering angle. The display control method superimposes and displays the generated enhanced image and route image on a display device mounted on the vehicle. [Effects of the Invention]

[0007] The display control system and the like of the present disclosure have the advantage of easily reducing the anxiety that vehicle drivers feel when driving. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an outline of a display control system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of an image displayed on a display device by the display control system according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating another example of an image displayed on a display device by the display control system according to the embodiment. [Figure 4]FIG. 4 is a flowchart showing an example of the operation of the display control system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0010] The embodiments described below are comprehensive or specific examples. The numerical values, shapes, materials, components, component installation positions and connection forms, steps, step orders, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not described in the independent claims are described as optional components.

[0011] In addition, each drawing is a schematic diagram and is not necessarily an exact illustration. In addition, the same components are denoted by the same reference numerals in each drawing.

[0012] (Embodiment) <Summary> First, the inventor's point of view will be explained.

[0013] A known technology, as disclosed in Patent Document 1 for example, extracts the edge of a three-dimensional obstacle (here, a curb on the road shoulder) based on an image captured of the side of a vehicle, including the side of the vehicle, and alerts the occupants by a warning sound or voice announcement when the vehicle approaches the extracted edge of the curb. This technology is intended to have the effect of reducing the driver's anxiety about the vehicle hitting the curb when attempting to pull over to the shoulder of the road.

[0014] However, the above technology has a problem in that it cannot completely eliminate the anxiety felt by drivers who are not good at pulling over to the shoulder. Specifically, the above technology has a problem in that, while attempting to pull over to the shoulder, the driver has to worry about when the warning sound or voice announcement will be issued, and ultimately ends up driving with anxiety.

[0015] In view of the above, the inventor has discovered a display control system and the like that can easily reduce the anxiety felt by vehicle drivers when driving, particularly when attempting to pull over to the shoulder of the road.

[0016] <Configuration> The configuration of a display control system 1 according to an embodiment will be described below. Fig. 1 is a block diagram showing an overview of the display control system 1 according to an embodiment. The display control system 1 is used in a moving object such as a vehicle 100, and is a system for assisting driving of the vehicle 100 by controlling a display device 4 mounted on the vehicle 100. In the embodiment, the display control system 1 is realized by an in-vehicle device, but it may also be realized by an external device brought in from outside the vehicle 100.

[0017] In the embodiment, vehicle 100 is described as being a so-called right-hand drive vehicle with a steering wheel provided on the front right seat. However, vehicle 100 may be a so-called left-hand drive vehicle with a steering wheel provided on the front left seat.

[0018] In the following description, directions such as "forward," "backward," "leftward," "rightward," and "upward" are all directions seen from the driver of the vehicle 100. For example, "forward" is the direction in which the driver of the vehicle 100 looks toward the windshield 101 (see FIG. 3), and other directions such as "leftward" are directions based on the driver of the vehicle 100 looking toward the windshield 101.

[0019] The display control system 1 is realized by, for example, an ECU (Electronic Control Unit). More specifically, the display control system 1 is realized by, for example, a processor included in the ECU executing a program stored in a memory included in the ECU. As shown in FIG. 1 , the display control system 1 includes a first acquisition unit 11, a second acquisition unit 12, an extraction unit 13, a determination unit 14, a generation unit 15, and a display control unit 16.

[0020] The first acquisition unit 11 acquires a peripheral image. Here, the peripheral image is an image that includes at least the area ahead of the vehicle 100 and the side of the vehicle 100 on which an obstacle A0 (see FIG. 2) is present. The obstacle A0 is an object that hinders the travel of the vehicle 100, in other words, an object that is undesirable for the vehicle 100 to come into contact with, such as a curb A11 (see FIG. 2) on the road shoulder A1 (see FIG. 2), a gutter, a wall, or a utility pole. In the embodiment, the obstacle A0 is described as being the curb A11 on the road shoulder A1. The peripheral image is an image that includes at least the area ahead of the vehicle 100 and the left side of the vehicle 100.

[0021] In the embodiment, the first acquisition unit 11 acquires a peripheral image by acquiring a detection result from the sensor 2 and generating a peripheral image based on the acquired detection result. Note that when the sensor 2 generates a peripheral image, the first acquisition unit 11 acquires the peripheral image from the sensor 2 without generating the peripheral image.

[0022] The sensor 2 is mounted on the vehicle 100 and detects the situation around the vehicle 100. Specifically, the sensor 2 detects objects such as pedestrians, other vehicles, and obstacles A0 that exist around the vehicle 100, as well as the positions of the objects. The sensor 2 is, for example, a camera (image sensor), a sonar sensor, radar, or a LiDAR (Light Detection and Ranging) sensor.

[0023] The second acquisition unit 12 acquires the steering angle of the vehicle 100. In the embodiment, the second acquisition unit 12 acquires the detection result of the steering angle sensor 3 to thereby acquire the steering angle of the vehicle 100.

[0024] The steering angle sensor 3 is provided on, for example, the steering wheel of the vehicle 100 and detects the angle of the steering wheel, that is, the steering angle of the vehicle 100.

[0025] The extraction unit 13 extracts the edge of the obstacle A0 in the peripheral image acquired by the first acquisition unit 11. In the embodiment, the extraction unit 13 extracts the edge of the curb A11 on the road shoulder A1 in the peripheral image as the edge of the obstacle A0. The edge is extracted as a linear image along a direction parallel to the road on which the vehicle 100 is traveling (for example, the longitudinal direction). For example, the extraction unit 13 detects the edge of the obstacle A0 by executing an algorithm that extracts appropriate image features from the peripheral image. Alternatively, the extraction unit 13 may extract the edge of the obstacle A0 from the peripheral image using a trained model that has been machine-learned in advance to input the peripheral image and output the edge of the obstacle A0.

[0026] Furthermore, the extraction unit 13 may extract the edges of the obstacle A0 from the peripheral image using, for example, the technology disclosed in Patent Document 1. Specifically, the extraction unit 13 detects one or more edges from the peripheral image in a direction parallel to the road on which the vehicle 100 is traveling (for example, the front-to-rear direction). Then, the extraction unit 13 may extract the edges of the obstacle A0 by excluding edges of white lines on the road from the one or more edges based on a change in brightness in a direction in which the one or more detected edges are aligned (for example, the left-to-right direction).

[0027] The determination unit 14 determines whether or not to superimpose and display an emphasis image P1 and a route image P2, which will be described later, on the display device 4. In the embodiment, when the driving mode of the vehicle 100 is shifted to the pulling-over mode, the determination unit 14 determines that the emphasis image P1 and the route image P2 should be superimposed and displayed on the display device 4.

[0028] Here, the pulling-over mode is a mode in which the vehicle 100 performs pulling-over driving to pull over an obstacle A0 (here, a curb A11 on the road shoulder A1). The shift of the driving mode of the vehicle 100 to the pulling-over mode is executed, for example, by the driver operating a switch attached to the steering wheel of the vehicle 100. Also, for example, if the display 42 of the vehicle 100, which will be described later, is a touch panel display, the shift of the driving mode of the vehicle 100 to the pulling-over mode may be executed by the driver touching an icon for the pulling-over mode displayed on the touch panel display.

[0029] The determination unit 14 may determine not to superimpose and display either the emphasized image P1 or the route image P2 on the display device 4 when the driving mode of the vehicle 100 is a mode other than the pull-over mode.

[0030] The generation unit 15 generates an enhanced image P1 that enhances the edges extracted by the extraction unit 13. The generation unit 15 also generates a route image P2 that shows a predicted driving route of the vehicle 100 based on the steering angle acquired by the second acquisition unit 12.

[0031] For example, the generation unit 15 generates, as the enhanced image P1, a straight-line image corresponding to the road-side edge of the curb A11 on the road shoulder A1 extracted by the extraction unit 13. For example, the generation unit 15 generates, as the enhanced image P1, an image of a straight line that is thicker than the line indicating the edge of the curb A11. Furthermore, for example, the generation unit 15 predicts, based on the steering angle of the vehicle 100 at the time of acquisition by the second acquisition unit 12, a trajectory that the front wheels of the vehicle 100 will follow if the steering angle is assumed to be maintained for a certain period of time, and generates, as the route image P2, a straight-line or curved image indicating the predicted trajectory.

[0032] The display control unit 16 superimposes and displays the highlighted image P1 and the route image P2 generated by the generation unit 15 on the display device 4 mounted on the vehicle 100. In the embodiment, the display control unit 16 superimposes and displays the images on the display device 4 according to the determination result of the determination unit 14. For example, when the driving mode of the vehicle 100 is shifted to the pulling-over mode, the display control unit 16 superimposes and displays the highlighted image P1 and the route image P2 on the display device 4.

[0033] If the extraction unit 13 is unable to extract the edge of the obstacle A0, the display control unit 16 may display only the route image P2 superimposed on the display device 4. Here, a situation in which the extraction unit 13 is unable to extract the edge of the obstacle A0 may occur, for example, when the obstacle A0 is not included in the peripheral image, or when the obstacle A0 is included in the peripheral image but is partially missing, or the obstacle A0 is not clearly displayed in the peripheral image, and therefore cannot be recognized as the obstacle A0.

[0034] The display device 4 is an image display device mounted on the vehicle 100. In the embodiment, the display device 4 includes a head-up display (hereinafter also referred to as “HUD”) 41 and a display 42.

[0035] The HUD 41 is a device that is disposed on the instrument panel of the vehicle 100 and projects an image showing various information onto the windshield 101 of the vehicle 100, thereby displaying the image in the field of view of the driver.

[0036] The display 42 is, for example, a liquid crystal display or an organic EL (electro-luminescence) display, and is a display present inside the vehicle 100. For example, the display 42 is a center display disposed in the center of the instrument panel of the vehicle 100. The display 42 may be a touch panel display. Note that, if the vehicle 100 is equipped with a car navigation system, the display 42 may be used as a display for the car navigation system. The display 42 may also be another display mounted on the vehicle 100, such as a meter display. Alternatively, the display 42 may be a portable display or the like that is carried into the vehicle 100 by the user of the vehicle 100 from outside.

[0037] <Example of display control> An example of display control of the highlighted image P1 and the route image P2 by the display control system 1 according to the embodiment will be described below with reference to FIGS.

[0038] Fig. 2 is a diagram showing an example of an image displayed on the display device 4 by the display control system 1 according to the embodiment. Fig. 2 shows an example of an image displayed on a display 42, which is one of the display devices 4. In the example shown in Fig. 2, a bird's-eye view image of the vehicle 100 viewed from above, a so-called around-view monitor image, is displayed on the display 42.

[0039] This bird's-eye view image is generated by combining, for example, a forward image showing the front of vehicle 100 taken by a camera (sensor 2) located at the front of vehicle 100, a rear image showing the rear of vehicle 100 taken by a camera (sensor 2) located at the rear of vehicle 100, a left image showing the left side of vehicle 100 taken by a camera (sensor 2) located on the left side of vehicle 100, and a right image showing the right side of vehicle 100 taken by a camera (sensor 2) located on the right side of vehicle 100.

[0040] As shown in Fig. 2, the bird's-eye view image displayed on the display 42 displays the vehicle 100 as seen from above, the road shoulder A1 as seen from above, and the sidewalk A2 as seen from above. Note that in the example shown in Fig. 2, detailed images of the road, images of buildings, etc. are omitted. The display control unit 16 then causes the display 42 to display an enhanced image P1 so as to be superimposed on the edge of an obstacle A0 (here, a curb A11 on the road shoulder A1). In the example shown in Fig. 2, the display control unit 16 causes the display 42 to display a linear enhanced image P1 so as to be superimposed on each of the edges of two curbs A11 extending in the front-to-rear direction.

[0041] In the example shown in Figure 2, the highlighted image P1 is not superimposed on the gaps between these curbs A11, in other words, the gaps between the curbs A11, but the highlighted image P1 may also be superimposed on the gaps.

[0042] The display control unit 16 also displays two curved route images P2 on the display 42 so as to be superimposed on the road ahead of the vehicle 100. These two curved route images represent the predicted trajectory of the left front wheel of the vehicle 100 and the predicted trajectory of the right front wheel of the vehicle 100, respectively.

[0043] The driver of the vehicle 100 can drive while knowing whether the vehicle 100 will come into contact with the curb A11 by looking at the enhanced image P1 and the route image P2 displayed on the display 42, which can easily reduce anxiety felt while driving (here, while driving close to the side of the road). In particular, in the example shown in Fig. 2, the driver can drive while looking at the enhanced image P1 and the route image P2 displayed in the bird's-eye view image, which makes it easy to know the positions of the vehicle 100 and the curb A11.

[0044] 3 is a diagram showing another example of an image displayed on the display device 4 by the display control system 1 according to the embodiment. In the example shown in FIG. 3, an enhanced image P1 and a route image P2 projected from the HUD 41 are displayed on the windshield 101 of the vehicle 100. Therefore, the driver of the vehicle 100 can see the scenery ahead of the vehicle 100 through the windshield 101, as well as the enhanced image P1 and the route image P2. Note that detailed images of roads, images of buildings, and the like are not shown in the example shown in FIG. 3.

[0045] As shown in Fig. 3, the road ahead of the vehicle 100 and a road shoulder A1 are displayed in the field of view of the driver of the vehicle 100. The display control unit 16 then displays an enhanced image P1 on the display 42 so as to be superimposed on the edge of an obstacle A0 (here, a curb A11 on the road shoulder A1). In the example shown in Fig. 3, the display control unit 16 projects a linear enhanced image P1 onto the windshield 101 so as to be superimposed on the edge of the curb A11 extending forward.

[0046] The display control unit 16 also projects two straight-line route images P2 onto the windshield 101 so as to be superimposed in front of the vehicle 100. These two straight-line images represent the predicted trajectory of the left front wheel of the vehicle 100 and the predicted trajectory of the right front wheel of the vehicle 100, respectively.

[0047] By looking at the enhanced image P1 and the route image P2 projected on the windshield 101 of the vehicle 100, the driver of the vehicle 100 can drive while knowing whether the vehicle 100 will come into contact with the curb A11, which can easily reduce anxiety felt while driving (here, while driving close to the curb). In particular, in the example shown in Fig. 3, the driver can drive while looking at the enhanced image P1 and the route image P2 without changing his / her viewpoint from the windshield 101.

[0048] The display control system 1 may perform both the control of superimposing and displaying the enhanced image P1 and the route image P2 on the display 42 and the control of superimposing and projecting the enhanced image P1 and the route image P2 on the windshield 101, or may perform only one of them.

[0049] <Operation> The operation of the display control system 1 according to the embodiment will be described below with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the operation of the display control system 1 according to the embodiment.

[0050] First, the determination unit 14 determines whether the driving mode of the vehicle 100 has transitioned to the pulling-over mode (S1). If the driving mode of the vehicle 100 has not transitioned to the pulling-over mode (S1: No), the determination unit 14 determines that neither the emphasized image P1 nor the route image P2 should be superimposed and displayed on the display device 4. On the other hand, if the driving mode of the vehicle 100 has transitioned to the pulling-over mode (S1: Yes), the determination unit 14 determines that the emphasized image P1 and the route image P2 should be superimposed and displayed on the display device 4. In this case, the display control system 1 executes step S1 and subsequent steps.

[0051] Next, the first acquisition unit 11 acquires a peripheral image (S2). The second acquisition unit 12 acquires the steering angle of the vehicle 100 (S3). Note that steps S2 and S3 do not have to be executed in this order, and may be executed in the order of steps S3 and S2, for example, or may be executed simultaneously in parallel. Also, steps S2 and S3 may be executed constantly regardless of the result of step S1.

[0052] Next, the extraction unit 13 executes a process of extracting the edge of the obstacle A0 in the peripheral image acquired by the first acquisition unit 11 (S4). If the edge of the obstacle A0 can be extracted in the peripheral image (S4: Yes), the display control system 1 executes steps S5 and S6. On the other hand, if the edge of the obstacle A0 cannot be extracted in the peripheral image (S4: No), the display control system 1 executes steps S7 and S8.

[0053] If the edge of the obstacle A0 can be extracted from the peripheral image, the generation unit 15 generates an enhanced image P1 that enhances the edge of the obstacle A0 extracted by the extraction unit 13, and also generates a route image P2 that shows the predicted driving route of the vehicle 100 based on the steering angle acquired by the second acquisition unit 12 (S5). Then, the display control unit 16 superimposes and displays the enhanced image P1 and the route image P2 generated by the generation unit 15 on the display device 4 mounted on the vehicle 100 (S6).

[0054] On the other hand, if the edge of the obstacle A0 cannot be extracted from the peripheral image, the generation unit 15 generates a route image P2 showing a predicted travel route of the vehicle 100 based on the steering angle acquired by the second acquisition unit 12 (S7). Then, the display control unit 16 superimposes and displays the route image P2 generated by the generation unit 15 on the display device 4 mounted on the vehicle 100 (S8).

[0055] Thereafter, the display control system 1 repeats the processing of steps S2 to S8 while the driving mode of the vehicle 100 is the pulling-over mode. Then, when the driving mode of the vehicle 100 transitions to a mode other than the pulling-over mode, the display control system 1 ends the processing of steps S2 to S8.

[0056] <Advantages> The advantages of the display control system 1 according to the embodiment will be described below. As described above, the display control system 1 according to the embodiment extracts the edges of the obstacle A0 in a peripheral image that includes at least the area ahead of the vehicle 100 and the area to the side of the vehicle 100 on which the obstacle A0 is located, and displays an enhanced image P1 that enhances the edges and a route image P2 that shows the predicted driving route of the vehicle 100, superimposed on the display device 4. Therefore, in the display control system 1 according to the embodiment, the driver can drive the vehicle 100 while knowing whether or not the vehicle 100 will come into contact with the obstacle A0, by looking at the enhanced image P1 and the route image P2 displayed on the display device 4. Therefore, the display control system 1 according to the embodiment has the advantage of easily reducing the anxiety that the driver feels when driving.

[0057] The display control system 1 according to the embodiment is particularly effective when the driver drives close to the side of the road. Here, close to the side of the road driving is a counterintuitive driving method in which the vehicle 100 is pulled over to the shoulder A1, and is a type of driving that relatively many drivers find difficult. Therefore, the display control system 1 according to the embodiment has the advantage that when the driver drives close to the side of the road, the driver can drive while looking at the edge of the curb A11 on the shoulder A1 and the predicted driving path of the vehicle 100, which makes it easier to reduce the anxiety felt when driving close to the side of the road.

[0058] As already mentioned, the technology disclosed in Patent Document 1 has a problem in that the driver has to worry about when the warning sound or voice announcement will be issued when driving close to other vehicles, and ultimately ends up driving with anxiety. In contrast, the display control system 1 according to the embodiment allows the driver to drive while looking at the edge of the curb A11 on the road shoulder A1 and the predicted driving route of the vehicle 100, and therefore can solve this problem.

[0059] Furthermore, the technology disclosed in Patent Document 1 has a problem in that if a warning sound or voice announcement is not issued during driving close to the curb, the vehicle 100 is not brought close enough to the curb A11. For example, there may be cases where the vehicle 100 ends driving before coming within a predetermined distance from the edge of the curb A11. In this case, no warning sound is emitted, but it cannot be said that the vehicle 100 has come close enough to the edge of the curb A11. In contrast, the display control system 1 according to the embodiment allows the driver to drive while viewing the edge of the curb A11 on the shoulder A1 and the predicted driving route of the vehicle 100, thereby solving this problem.

[0060] (Variation) While the display control system according to the present disclosure has been described above based on the above embodiment, the present disclosure is not limited to the embodiment. As long as the modifications do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art may also be included within the scope of the present disclosure.

[0061] In the above embodiment, the generation unit 15 may generate a route image P2 showing a predicted driving route further based on the inner wheel difference and the outer wheel difference of the vehicle 100. For example, the generation unit 15 may predict the trajectories of the two front wheels of the vehicle 100 based on the steering angles acquired by the second acquisition unit 12, and further correct the trajectories taking into account the inner wheel difference and the outer wheel difference, and generate a route image P2 showing the corrected trajectory.

[0062] In the above embodiment, the determination unit 14 determines whether to superimpose and display the enhanced image P1 and the route image P2 on the display device 4 based on whether the driving mode of the vehicle 100 has transitioned to the pull-over mode, but this is not limited to this. For example, the determination unit 14 may determine to superimpose and display the enhanced image P1 and the route image P2 on the display device 4 when the distance (e.g., the shortest distance) between a driving line indicating a predicted driving route of the vehicle 100 and the edge of the obstacle A0 extracted by the extraction unit 13 is below a threshold. In other words, the display control unit 16 may superimpose and display the enhanced image P1 and the route image P2 on the display device 4 when the distance between the driving line and the edge is below a threshold. The driving line here is the trajectory closer to the obstacle A0 out of two trajectories drawn by the two front wheels of the vehicle 100.

[0063] In the above embodiment, the display control unit 16 may change the display mode of the emphasized image P1 and the route image P2 depending on the distance between the driving line and the edge. For example, the display control unit 16 may change the color of the emphasized image P1 and the route image P2 so that it approaches red as the distance becomes shorter. Also, for example, the display control unit 16 may change the thickness of the lines shown in the emphasized image P1 and the route image P2 so that it becomes thicker as the distance becomes shorter. Also, for example, the display control unit 16 may blink the emphasized image P1 and the route image P2 when the distance is below a threshold.

[0064] In the above embodiment, when there is a gap between multiple obstacles A0, the display control unit 16 may superimpose and display the gap on the display device 4 in a manner different from that of the highlighted image P1. For example, the display control unit 16 may set the color of the image to be superimposed and displayed on the gap to a color different from that of the highlighted image P1.

[0065] In the above embodiment, the display control unit 16 may always superimpose and display the highlighted image P1 and the route image P2 on the image displayed by the display device 4, regardless of the determination result of the determination unit 14. In this case, the display control system 1 may not be provided with the determination unit 14.

[0066] In the above-described embodiment, the display device 4 may include only one of the HUD 41 and the display 42. When the display device 4 includes only the HUD 41, the display control system 1 only needs to execute control to project the enhanced image P1 and the route image P2 superimposed on the windshield 101, as in the example shown in Fig. 3. When the display device 4 includes only the display 42, the display control system 1 only needs to execute control to display the enhanced image P1 and the route image P2 superimposed on the display 42, as in the example shown in Fig. 2.

[0067] In the above embodiment, the display control system 1 is realized by an in-vehicle device, but this is not limiting. For example, the display control system 1 may be mounted on a server device separate from the vehicle 100, and images may be displayed on the display device 4 by wireless communication between the server device and the vehicle 100. Furthermore, the display control system 1 may have a composite configuration in which part of it is mounted on the vehicle 100 and the rest is mounted on the server device.

[0068] Furthermore, each unit included in the display control system 1 according to the above embodiment is typically realized as an LSI (Large-Scale Integration), which is an integrated circuit. These units may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0069] Furthermore, the integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connections and settings of circuit cells within LSIs to be reconfigured, may also be used.

[0070] In the above-described embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0071] Furthermore, all of the numbers used above are examples for specifically explaining the present disclosure, and the embodiments of the present disclosure are not limited to the numbers shown as examples.

[0072] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0073] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and an order other than the above may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0074] In addition, this disclosure also includes forms obtained by making various modifications to the above embodiments that a person skilled in the art would think of, and forms realized by arbitrarily combining the components and functions in the embodiments within the scope that does not deviate from the intent of this disclosure.

[0075] (summary) As described above, the display control system 1 according to the first aspect includes a first acquisition unit 11, a second acquisition unit 12, an extraction unit 13, a generation unit 15, and a display control unit 16. The first acquisition unit 11 acquires a peripheral image including at least the area ahead of the vehicle 100 and the area to the side of the vehicle 100 on which an obstacle A0 is present. The second acquisition unit 12 acquires a steering angle of the vehicle 100. The extraction unit 13 extracts edges of the obstacle A0 from the peripheral image. The generation unit 15 generates an enhanced image P1 that emphasizes the extracted edges, and a route image P2 that shows a predicted driving route of the vehicle 100 based on the steering angle. The display control unit 16 superimposes and displays the generated enhanced image P1 and route image P2 on the display device 4 mounted on the vehicle 100.

[0076] This allows the driver to drive while knowing whether or not the vehicle 100 will come into contact with the obstacle A0 by looking at the highlighted image P1 and the route image P2 displayed on the display device 4. This has the advantage of making it easier to reduce the anxiety the driver may feel while driving.

[0077] Also, for example, in the display control system 1 according to the second aspect, in the first aspect, the generation unit 15 generates a route image P2 showing a predicted driving route further based on the inner wheel difference and outer wheel difference of the vehicle 100.

[0078] This has the advantage that the accuracy of the predicted driving route of vehicle 100 shown by the route image P2 superimposed on the display device 4 is easily improved, since the driving route of vehicle 100 is predicted taking into account the inner wheel difference and outer wheel difference of vehicle 100.

[0079] Furthermore, for example, in the display control system 1 according to the third aspect, in the first or second aspect, the display device 4 is the display 42. The display control unit 16 causes the display 42 to display an enhanced image P1 and a route image P2 superimposed on a bird's-eye view image of the vehicle 100 viewed from above.

[0080] This allows the driver to drive while looking at the enhanced image P1 and route image P2 displayed in the bird's-eye view image, which has the advantage of making it easier to grasp the positions of the vehicle 100 and obstacle A0 and reducing the driver's anxiety while driving.

[0081] Furthermore, for example, in the display control system 1 according to a fourth aspect, in any one of the first to third aspects, the display device 4 is a head-up display 41. The display control unit 16 displays the highlighted image P1 and the route image P2 on the windshield 101 of the vehicle 100 in a superimposed manner.

[0082] This has the advantage that the driver can drive while looking at the enhanced image P1 and the route image P2 without changing his / her viewpoint through the windshield 101, which makes it easier to reduce the anxiety the driver may feel while driving.

[0083] Also, for example, in the display control system 1 according to the fifth aspect, in any one of the first to fourth aspects, when the display control unit 16 transitions to a mode in which the vehicle 100 performs a side-by-side driving operation to move closer to the obstacle A0, the display control unit 16 displays the highlighted image P1 and the route image P2 superimposed on the display device 4.

[0084] According to this, when the driver performs close-to-side driving, the emphasized image P1 and the route image P2 are displayed on the display device 4, which has the advantage that it becomes easier for the driver to perform close-to-side driving and the anxiety that the driver feels when driving can be reduced. Also, according to this, since the emphasized image P1 and the route image P2 are displayed on the display device 4 only when close-to-side driving is performed, there is the advantage that it becomes easier for the driver to concentrate on driving other than when performing close-to-side driving, compared to when the emphasized image P1 and the route image P2 are always displayed.

[0085] Also, for example, in the display control system 1 according to the sixth aspect, in any one of the first to fourth aspects, when the distance between the driving line indicating the driving route and the edge is below a threshold, the display control unit 16 displays the highlighted image P1 and the route image P2 superimposed on the display device 4.

[0086] This has the advantage that the highlighted image P1 and the route image P2 are not displayed on the display device 4 until there is a high possibility that the vehicle 100 will come into contact with the obstacle A0, which makes it easier to prevent the driver from being overly cautious when driving.

[0087] Also, for example, in the display control system 1 according to the seventh aspect, in any one of the first to sixth aspects, the display control unit 16 changes the display mode of at least one of the highlighted image P1 and the route image P2 depending on the distance between the driving line indicating the driving route and the edge.

[0088] This has the advantage that the display mode of at least one of the highlighted image P1 and the route image P2 is changed depending on the degree of proximity of the vehicle 100 to the obstacle A0, making it easier for the driver to grasp the degree of proximity of the vehicle 100 to the obstacle A0, and making it easier to reduce the anxiety the driver feels while driving.

[0089] Also, for example, in a display control method according to an eighth aspect, a peripheral image including at least the area ahead of the vehicle 100 and the area to the side of the vehicle 100 on which an obstacle A0 is present is acquired (S2). In the display control method, a steering angle of the vehicle 100 is acquired (S3). In the display control method, an edge of the obstacle A0 in the peripheral image is extracted (S4). In the display control method, an enhanced image P1 that enhances the extracted edge and a route image P2 that shows a predicted driving route of the vehicle 100 based on the steering angle are generated (S5). In the display control method, the generated enhanced image P1 and route image P2 are superimposed and displayed on a display device 4 mounted on the vehicle 100 (S6).

[0090] This allows the driver to drive while knowing whether or not the vehicle 100 will come into contact with the obstacle A0 by looking at the highlighted image P1 and the route image P2 displayed on the display device 4. This has the advantage of making it easier to reduce the anxiety the driver may feel while driving. [Industrial Applicability]

[0091] The present disclosure is applicable to a system for assisting a driver of a moving object such as a vehicle. [Explanation of symbols]

[0092] 1 Display control system 11 First acquisition part 12 Second acquisition part 13 Extraction part 14 Judgment Department 15 Generation part 16 Display control unit 2 sensors 3 Steering angle sensor 4 Display Devices 41 Head-up display (HUD) 42 Display 100 vehicles 101 Windshield A0 Obstacle A1 Shoulder A11 Curb A2 Sidewalk P1 enhanced image P2 Route Image

Claims

1. a first acquisition unit that acquires a surrounding image including at least a front side of the vehicle and a side of the vehicle on which an obstacle exists; a second acquisition unit that acquires a steering angle of the vehicle; an extraction unit that extracts an edge of the obstacle in the peripheral image; a generation unit that generates an enhanced image that enhances the extracted edges and a route image that shows a predicted travel route of the vehicle based on the steering angle; a display control unit that displays the generated enhanced image and the generated route image in a superimposed manner on a display device mounted on the vehicle, Display control system.

2. the generation unit generates the route image showing the predicted travel route further based on an inner wheel radius difference and an outer wheel radius difference of the vehicle. The display control system according to claim 1 .

3. The display device is a display, the display control unit causes the emphasized image and the route image to be superimposed on a bird's-eye view image of the vehicle displayed on the display, the bird's-eye view image being viewed from above the vehicle, The display control system according to claim 1 or 2.

4. the display device is a head-up display, the display control unit displays the enhanced image and the route image in a superimposed manner on a windshield of the vehicle. The display control system according to claim 1 or 2.

5. When the display control unit shifts to a mode in which a driving mode for driving the vehicle close to the obstacle is performed, the display control unit causes the highlighted image and the route image to be superimposed on the display device. The display control system according to claim 1 or 2.

6. the display control unit, when a distance between a driving line indicating the driving route and the edge is less than a threshold, causes the highlighted image and the route image to be superimposed on the display device and displayed. The display control system according to claim 1 or 2.

7. the display control unit changes a display mode of at least one of the highlighted image and the route image according to a distance between a driving line indicating the driving route and the edge. The display control system according to claim 1 or 2.

8. Acquire a surrounding image including at least a front side of the vehicle and a side side of the vehicle on which an obstacle exists; acquiring a steering angle of the vehicle; extracting an edge of the obstacle in the surrounding image; generating an enhanced image that enhances the extracted edges and a route image that indicates a predicted travel route of the vehicle based on the steering angle; The generated enhanced image and the generated route image are displayed in a superimposed manner on a display device mounted on the vehicle. Display control method.

Images