Image display control system

Abstract

To provide a technology for displaying a captured image around a vehicle at appropriate timing.SOLUTION: An image display control system comprises: an acquisition unit that acquires information about a captured image around a vehicle and a traveling situation of the vehicle; a display determining unit that determines to start displaying of the captured image on an on-vehicle display in response to an input from the crew member or a set starting condition being met; a learning unit that learns display start timing by an input of the crew member based on information regarding the traveling situation when displaying of the captured image is commenced by the input of the crew member; and a condition setting unit that sets the learned display start timing as a starting condition, or that proposes to the crew member to set the learned display start timing as the starting condition.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for displaying captured images of the surroundings of a vehicle. [Background technology]

[0002] Patent document 1 discloses a surrounding monitoring device that includes an acquisition unit that acquires a first surrounding image of a vehicle, a first display control unit that displays a selection screen on a display device for selecting a second surrounding image that has been image processed for visibility from the first surrounding image, a touch panel that accepts the selection of the second surrounding image, and a second display control unit that displays the second surrounding image that has been selected on the display device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-201115 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, the situations in which occupants want to display images of the vehicle's surroundings vary depending on the driving conditions and individual differences, but if the occupant is always required to perform an operation to display the surrounding images, this can be inconvenient for the occupant while driving.

[0005] An object of the present invention is to provide a technique for displaying captured images of the surroundings of a vehicle at appropriate timing. [Means for solving the problem]

[0006] In order to solve the above problems, an image display control system of one embodiment of the present invention comprises an acquisition unit that acquires captured images of the area around the vehicle and information regarding the vehicle's driving conditions; a display decision unit that decides to start displaying the captured images on an in-vehicle display in response to a start condition input by the occupant or a set start condition being satisfied; a learning unit that learns the display start timing based on the occupant's input based on information regarding the driving conditions when the captured images are started to be displayed by the occupant's input; and a condition setting unit that sets the learned display start timing as the start condition or suggests to the occupant that it be set. The display determination unit starts displaying the captured image from a non-displayed state when the start condition is satisfied. The condition setting unit suggests to the occupant to set the start condition when the learned timing starts displaying the captured image earlier than the set start condition. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a technique for displaying captured images of the surroundings of a vehicle at appropriate timing. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an image of the surroundings of a vehicle displayed on an in-vehicle display. [Figure 2] FIG. 1 is a diagram illustrating a functional configuration of an image display control system according to an embodiment. [Figure 3] 10 is a flowchart of a process for displaying a captured image of the periphery of a vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 is a diagram showing an image of the surroundings of a vehicle displayed on an in-vehicle display. An output unit 18, which functions as an in-vehicle display, displays a first surroundings image 18a and a second surroundings image 18b (when there is no need to distinguish between them, they are simply referred to as surrounding images). The first surroundings image 18a is generated as a plan view by combining multiple captured images, and is displayed as if viewed from above the host vehicle 2. The second surroundings image 18b displays the area ahead of the host vehicle 2. The first surroundings image 18a and the second surroundings image 18b are images captured at the same time. The image of the vehicle's surroundings may be a processed image of the area around the vehicle, or it may be the image itself; in either case, although it includes an image that has been processed, it is essentially a captured image.

[0010] FIG. 1 shows how the host vehicle 2 and an oncoming vehicle 6 pass each other on a narrow road. The second peripheral image 18b shows that the oncoming vehicle 6 is approaching, and a ditch 4 is located on the left side of the host vehicle 2. The driver drives while looking at the first peripheral image 18a to prevent the host vehicle 2 from falling into the ditch 4. In this way, driving assistance can be provided by the output unit 18 displaying an image of the surroundings of the host vehicle 2.

[0011] Incidentally, the captured image of the vehicle's surroundings is displayed on the output unit 18 when the occupant inputs a command to start display or when a predetermined start condition is satisfied. For example, in the initial setting, the predetermined start condition is satisfied only when parking is detected, for example, when the vehicle speed is 10 kilometers per hour or less. On the other hand, the occupant may want to display the captured image not only when parking, but also when passing an oncoming vehicle 6 on a narrow road as shown in FIG. 1 or when turning a sharp corner.

[0012] Therefore, the image display control system of the embodiment learns the driving situation in which the occupant started displaying the peripheral image and resets the predetermined start condition to display the peripheral image under the learned driving situation, thereby setting the peripheral image to be displayed when the driving situation is detected to be similar to the driving situation in which the occupant inputs the display start.

[0013] 2 is a diagram showing the functional configuration of the image display control system 1 according to the embodiment. Each function of the image display control system 1 can be configured in terms of hardware using circuit blocks, memory, and other LSIs, and can be realized in terms of software using system software or application programs loaded into memory. Therefore, it will be understood by those skilled in the art that each function of the image display control system 1 can be realized in various forms using only hardware, only software, or a combination of both, and is not limited to any of these.

[0014] The image display control system 1 includes an image display control device 10, a camera 12, an input unit 14, an on-board sensor 16, and an output unit 18, and is installed in a vehicle. The camera 12 captures images of the surroundings of the vehicle. There may be multiple cameras 12, and each may be capable of capturing images of the front, rear, left, and right sides of the vehicle. The camera 12 sends the captured images of the surroundings of the vehicle to the image display control device 10.

[0015] The input unit 14 is a touch panel provided on the in-vehicle display and / or a microphone that detects the voice of the occupant. The touch panel receives operational input from the occupant, and the microphone receives voice input from the occupant. This allows the occupant to input to display a peripheral image. The input unit 14 sends the occupant's input results to the image display control device 10. The occupant's input results include inputs input by operation and inputs by voice.

[0016] The on-board sensor 16 detects the driving conditions of the vehicle and transmits driving condition information to the image display control device 10. The on-board sensor 16 may be, for example, a vehicle speed sensor, a shift position detection sensor, a steering angle sensor, a millimeter wave radar, an ultrasonic sensor, or a combination of these. The camera 12 also functions as a sensor for detecting driving conditions, similar to the on-board sensor 16. These sensors for detecting driving conditions can detect the driving state and driving environment of the vehicle.

[0017] The output unit 18 includes an in-vehicle display and a speaker. The in-vehicle display may be integrated with the touch panel of the input unit 14. The output unit 18 displays a surrounding image under the control of the image display control device 10 and outputs audio to the occupants.

[0018] The image display control device 10 includes an acquisition unit 20 , a display determination unit 22 , a display control unit 24 , a driving situation analysis unit 26 , a condition holding unit 28 , a learning unit 30 , and a condition setting unit 32 .

[0019] The acquisition unit 20 acquires information via a network from the camera 12, the input unit 14, and the on-board sensor 16. The acquisition unit 20 acquires captured images of the area around the vehicle from the camera 12, acquires input information input by the occupant from the input unit 14, and acquires information related to the vehicle's driving conditions from the on-board sensor 16.

[0020] The driving situation analysis unit 26 analyzes the driving situation based on the detection results of the camera 12 and the on-board sensor 16, and calculates information about the driving situation. The information about the driving situation includes multiple parameters such as the vehicle speed, the positional relationship between the vehicle and other vehicles, the positional relationship between the vehicle and obstacles, road width, the presence or absence of parking lots, and the presence or absence of left or right turns. The driving situation information is used to determine the start conditions and for the learning process in the learning unit 30.

[0021] The display determination unit 22 determines whether to start displaying captured images of the vehicle's surroundings on the output unit 18 in response to an input from the occupant or when a predetermined start condition is satisfied. The captured images of the vehicle's surroundings may be processed and displayed, as in the first surroundings image 18a in FIG. 1. The occupant's input may be a voice input to a microphone to display the captured images, or an operation input to a touch panel to display the captured images. When the input unit 14 detects the occupant's input, the display determination unit 22 determines to display the captured images. After determining the display, the display determination unit 22 sends the time when the occupant's input instructing the display was made to the learning unit 30.

[0022] The display determination unit 22 determines to display a captured image of the vehicle's surroundings when a predetermined start condition is met. The display determination unit 22 determines whether the predetermined start condition is met based on the driving situation information calculated by the driving situation analysis unit 26. The predetermined start condition is held in the condition holding unit 28. The predetermined start condition is changeable and updated from the initial setting. The initial setting start condition is met, for example, when the vehicle speed is 10 kilometers per hour or less and the shift position is reverse. The start condition held in the condition holding unit 28 is updated based on the learning results of the learning unit 30.

[0023] The display control unit 24 controls the output unit 18 to display the captured image of the vehicle's surroundings in accordance with the decision made by the display decision unit 22. The display control unit 24 processes the captured image of the vehicle's surroundings into the first surroundings image 18a shown in Fig. 1, adds an image that highlights an obstacle to the captured image, or adds an image that indicates the predicted traveling direction of the vehicle to the captured image. The display control unit 24 displays the processed captured image on the in-vehicle display of the output unit 18.

[0024] The learning unit 30 learns the timing of display start due to an input by the occupant based on information about the driving situation when display of captured images around the vehicle is started due to an input by the occupant. As a result, a parameter indicating the driving situation when the occupant inputs the display start is derived. The timing when display of captured images around the vehicle is started due to an input by the occupant may be the timing when the occupant inputs the input by voice or touch operation, or may be the timing when the input is detected by the input unit 14.

[0025] The display start timings input by the occupant may include slowing down in a parking lot, slowing down on a narrow road, passing an oncoming vehicle on a narrow road, avoiding an obstacle, and turning right or left at a corner where an obstacle is present. Slowing down in a parking lot and slowing down on a narrow road are detected by the camera 12 and a vehicle speed sensor and are determined by parameters such as the presence or absence of a parking lot, road width, and vehicle speed. Passing an oncoming vehicle on a narrow road and avoiding an obstacle are detected by the camera 12, a millimeter-wave radar, and an ultrasonic sensor and are determined by parameters such as the positional relationship between the vehicle and other vehicles, the positional relationship between the vehicle and an obstacle, and road width. Turning right or left at a corner where an obstacle is present is detected by the camera 12, an ultrasonic sensor, and a steering angle sensor and are determined by parameters such as the positional relationship between the vehicle and an obstacle and steering angle.

[0026] The learning unit 30 learns the driving conditions at the time of input by the occupant and derives parameters that enable detection of similar driving conditions. The parameters derived by the learning unit 30 may be composed of a single parameter included in the information on the driving conditions calculated by the driving condition analysis unit 26, or may be composed of multiple types of parameters, and in either case, indicate the display start timing obtained by learning. The learning unit 30 derives optimal parameters for redetecting driving conditions similar to those at the time of input using a machine learning technique.

[0027] The condition setting unit 32 suggests to the occupant that the learned display start timing be set as a predetermined start condition. If the occupant agrees, the condition setting unit 32 sets the proposed display start timing as an additional predetermined start condition. The condition setting unit 32 may set the learned display start timing as a predetermined start condition, or may set it as is without suggesting it to the occupant. The learned display start timing is determined by parameters indicating the driving situation.

[0028] The condition setting unit 32 determines whether the learned display start timing starts displaying the captured image earlier than the currently set start condition, and if the display starts earlier, suggests to the occupant that the start condition be set, and if the display does not start earlier, does not suggest to the occupant. For example, if the currently set start condition is 10 kilometers per hour or less and the learning unit 30 learns that the display should start at 12 kilometers per hour, the learned result will be an earlier display start than the currently set start condition, so the condition setting unit 32 suggests to the occupant that the start condition be set to 12 kilometers per hour or less. Also, the farther the positional relationship between the host vehicle and other vehicles indicated in the set condition is, the earlier the display will start.

[0029] By setting the conditions in the condition setting unit 32, when a driving situation similar to that when the occupant inputs the display start command occurs, the captured image of the vehicle surroundings can be displayed on the in-vehicle display. This allows the start conditions to be set according to the occupant's request.

[0030] 3 is a flowchart of a process for displaying a captured image of the vehicle periphery. The acquisition unit 20 acquires information about the driving conditions from the camera 12 and the on-board sensor 16 (S10), and the driving condition analysis unit 26 calculates various parameters corresponding to the information about the driving conditions.

[0031] The display determination unit 22 determines whether the start conditions stored in the condition storage unit 28 are satisfied based on the parameters indicating the driving conditions calculated by the driving condition analysis unit 26 (S12). If the parameters indicating the driving conditions satisfy the start conditions (Y in S12), the display determination unit 22 determines to start display, and the display control unit 24 causes the output unit 18 to display the captured image of the vehicle's surroundings (S14), and this process ends.

[0032] If the start condition is not met (N in S12), the display determination unit 22 determines whether or not there is an instruction to start display input by the occupant based on the detection result of the input unit 14 (S16). If there is no input instruction (N in S16), this process ends without displaying.

[0033] If there is an input instruction (Y in S16), the display determination unit 22 determines to start display, and the display control unit 24 causes the output unit 18 to display the captured image of the vehicle's surroundings (S18). The learning unit 30 learns the display start timing based on the occupant's input, based on information about the driving situation when the display of the captured image of the vehicle's surroundings is started based on the occupant's input, and derives a parameter indicating the display start timing (S20).

[0034] The condition setting unit 32 compares the learned display start timing with the currently set start condition (S22) and determines whether the learned display start timing will start the display earlier than the currently set start condition (S24). If the learned display start timing will not start the display earlier than the currently set start condition (N in S24), the process ends. If the learned display start timing will start the display earlier than the currently set start condition (Y in S24), the condition setting unit 32 proposes a change to the start condition via the output unit 18 (S26). If the occupant does not approve the proposed change (N in S28), the process ends.

[0035] If the occupant approves the proposed change (Y in S28), the currently set start condition is reset to a new start condition using the learned parameter indicating the display start timing (S30), and the process ends.

[0036] The present disclosure has been described above based on examples. The present disclosure is not limited to the above examples, and various modifications such as design changes may be made based on the knowledge of those skilled in the art.

[0037] For example, in the embodiment, the image display control device 10 is provided in a vehicle, but the present invention is not limited to this. For example, some processing may be performed by an external server device, and the analysis processing by the traveling situation analysis unit 26 may be performed by the server device. In other words, the image display control device 10 may be configured by a combination of an in-vehicle device and a server device. [Explanation of symbols]

[0038] 1 Image display control system, 2 Vehicle, 4 Gutter, 6 Oncoming vehicle, 10 Image display control device, 12 Camera, 14 Input unit, 16 Onboard sensor, 18 Output unit, 18a First peripheral image, 18b Second peripheral image, 20 Acquisition unit, 22 Display determination unit, 24 Display control unit, 26 Driving situation analysis unit, 28 Condition holding unit, 30 Learning unit, 32 Condition setting unit.

Claims

[Claim 1] an acquisition unit that acquires captured images of the surroundings of the vehicle and information about the traveling conditions of the vehicle; a display determination unit that determines whether to start displaying the captured image on the in-vehicle display in response to a start condition input by an occupant or set by the occupant being satisfied; a learning unit that learns the timing at which the display of the captured image is started by an input from the occupant, based on information about a driving situation at the time when the display of the captured image is started by an input from the occupant; a condition setting unit that sets the learned timing as the start condition or suggests to an occupant that the timing be set; the display determination unit starts displaying the captured image from a non-displayed state when the start condition is satisfied, The image display control system is characterized in that the condition setting unit suggests to the occupant that the start condition be set if the learned timing starts displaying the captured image earlier than the start condition being set.

Images