Warning system and control program

The attention-calling system on vehicle windshields uses visual and auditory cues to adjust display and sound based on detected objects, addressing the challenge of directing driver attention to nearby hazards, thereby enhancing safety.

JP7584382B2Active Publication Date: 2024-11-15HONDA MOTOR CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
JP2021159822
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-09-29
Publication Date
2024-11-15
Estimated Expiration
2041-09-29

AI Technical Summary

Technical Problem

Existing systems fail to effectively and quickly draw a driver's attention to objects in the vicinity of a moving vehicle, such as other vehicles or pedestrians, using conventional detection methods like cameras and radar.

Method used

An attention-calling system that utilizes a display device on the windshield to output visual displays and sound, adjusting the display range and mode based on detected objects, including a moving object icon and surrounding icons that change directionally and in size or sound image localization to guide the driver's attention.

Benefits of technology

The system assists drivers in quickly and effectively directing their attention to relevant objects by adjusting visual and auditory cues based on object detection and recognition, enhancing safety by ensuring timely awareness of potential hazards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584382000001
    Figure 0007584382000001
  • Figure 0007584382000002
    Figure 0007584382000002
  • Figure 0007584382000003
    Figure 0007584382000003
Patent Text Reader

Abstract

To provide an attention attracting system and a control program for supporting so that a target can easily attract attention of an operator.SOLUTION: An attention attracting system 1 includes a display device 30, a display controller 12, and a target object recognition unit 11. The display controller 12 sets a display range of a surrounding icon 52 to be a whole circular area when a target object recognition unit 11 is recognizing at least the upper limit number of target objects, and reduces a size of the surrounding icon 52 if the number of the target objects which is at least the upper limit has increased under the situation in which the display range of the surrounding icon 52 is the whole circular area.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an attention calling system and a control program. [Background technology]

[0002] Conventionally, a vehicle image display system has been proposed that detects other vehicles traveling around the vehicle using a camera or radar, and displays a marking image of the other vehicles on the front window using a HUD (Head Up Display) so that it is superimposed on the other vehicles as viewed by the driver (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-021546 A Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, when an object such as another vehicle present in the vicinity of a moving body such as one's own vehicle is detected, it is desirable for the driver's attention to be quickly directed to the object. The present invention has been made in consideration of the above background, and aims to provide an attention calling system and a control program that assist a pilot in quickly directing his or her attention to an object. [Means for solving the problem]

[0005] As a first aspect for achieving the above object, there is provided an attention calling system including a display device that outputs a visual display on a windshield of a moving body that is visible to a driver of the moving body, a display control unit that causes the display device to output a predetermined attention calling display on the windshield as the visual display, and an object recognition unit that recognizes objects present in the vicinity of the moving body, wherein the display control unit causes the display device to display on the windshield the attention calling display, the attention calling display including a moving body icon that resembles the moving body and a surrounding icon whose display range is changed in a circumferential direction of a circular area surrounding the moving body icon, and when the object is recognized by the object recognition unit, and when the number of objects recognized by the object recognition unit is equal to or greater than a predetermined upper limit, the display range of the surrounding icons is the entire annular area, and when the number of objects recognized by the object recognition unit is equal to or greater than the upper limit and the display range of the surrounding icons is the entire annular area, the size of the surrounding icons is reduced when the number of objects recognized by the object recognition unit increases.

[0006] In the above-mentioned warning system, the display control unit may be configured to display the surrounding icons in the entire annular area when the object is not recognized by the object recognition unit, and to gradually reduce the display range of the surrounding icons from the entire annular area to a part of the annular area corresponding to the direction from the moving body to the object when the state transitions from when the object is not recognized by the object recognition unit to when the object is recognized by the object recognition unit.

[0007] In the above-described warning system, the display control unit may be configured to switch the display mode of the surrounding icons to a second display mode in which the entire annular area is displayed when the object recognition unit recognizes fewer than the upper limit number of objects and the display mode of the surrounding icons transitions from a state in which the display mode of the surrounding icons is a first display mode in which a portion of the annular area corresponding to the direction of the object relative to the moving body is displayed, to a state in which the object is not recognized by the object recognition unit.

[0008] In the above-mentioned attention warning system, the display control unit may be configured to perform a process of temporarily increasing the distance between the moving object icon and the surrounding icons while switching the display mode of the surrounding icons from the first display mode to the second display mode.

[0009] In the above attention drawing system, when the object is not recognized by the object recognition unit, the display control unit changes the display mode of the surrounding icons to a display range of the entire annular area. 2 When a state in which the object is not recognized by the object recognition unit continues for a predetermined time or more, the display mode of the surrounding icons is changed to the display mode of the surrounding icons with the entire annular area as a display range. 2 Display format is different 3 The display mode may be configured as follows.

[0010] The above-mentioned warning system may be configured to include an influence degree recognition unit that recognizes the influence degree of the object recognized by the object recognition unit on the moving body, and the display control unit may be configured to change the display mode of the surrounding icons depending on the influence degree of the object.

[0011] In the above-mentioned warning system, a sound image localization control unit may be provided that controls the output mode of a plurality of speakers provided on the moving body to change the position of a sound image of a predetermined attention warning sound perceived by the pilot, and the sound image localization control unit may be configured to set the position of the sound image of the attention warning sound to a predetermined range around the pilot when the object is not recognized by the object recognition unit, and to gradually decrease the range of the position of the sound image of the attention warning sound from the predetermined range in the direction from the pilot to the object when the state transitions from one in which the object is not recognized by the object recognition unit to one in which the object is recognized by the object recognition unit.

[0012] As a second aspect for achieving the above object, there is provided a control program for causing a controller, which controls the operation of a display device that acquires a recognition result from an object recognition unit that recognizes objects present around a moving body, and outputs a visual display on a windshield of the moving body that is viewable by a driver of the moving body, to execute a process of causing the display device to display a predetermined attention calling display on the windshield as the visual display in accordance with the recognition result, the control program causing the controller to cause the display device to display on the windshield the attention calling display, the attention calling display including a moving body icon that resembles the moving body and a surrounding icon whose display range is changed in the circumferential direction of a circular area surrounding the moving body icon, and when the object is recognized by the object recognition unit, a display range is set to a range including a portion corresponding to the direction of the object relative to the moving body, and the surrounding icons are displayed on the windshield by the display device, and when the number of the objects recognized by the object recognition unit is equal to or greater than a predetermined upper limit, the display range of the surrounding icons is set to the entire annular area, and when the number of the objects recognized by the object recognition unit is equal to or greater than the upper limit and the display range of the surrounding icons is set to the entire annular area, a control program is provided to execute a process of reducing the size of the surrounding icons when the number of the objects recognized by the object recognition unit increases. Effect of the Invention

[0013] the above heads up The system can assist the pilot in quickly directing his or her attention to an object. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a configuration diagram of an attention warning system and a vehicle equipped with the attention warning system. [Diagram 2] FIG. 2 is an explanatory diagram of the arrangement of the HUD and speakers in a vehicle. [Diagram 3] FIG. 3 is an explanatory diagram of an output mode of a warning display on the windshield by the HUD. [Figure 4] FIG. 4 is a first flowchart of the attention calling process. [Diagram 5] FIG. 5 is a second flowchart of the attention calling process. [Figure 6] FIG. 6 is a third flowchart of the attention calling process. [Figure 7] FIG. 7 is an explanatory diagram of a process for gradually decreasing the display range of the surrounding icons when an object existing in front of the vehicle is recognized. [Figure 8] FIG. 8 is an explanatory diagram of a process for widening the display range of the surrounding icons when the number of objects present ahead of the vehicle increases. [Figure 9] FIG. 9 is an explanatory diagram of a process for decreasing the diameter of the surrounding icons in response to an increase in the attention level. [Figure 10] FIG. 10 is an explanatory diagram of a process for changing the display mode of surrounding icons when a state transitions from one in which an object is recognized to one in which the object is not recognized. [Figure 11] FIG. 11 is an explanatory diagram of a process of gradually decreasing the range of the position of the sound image of the attention-calling sound in the direction from the driver to the object. [Figure 12] FIG. 12 is a setting table for the display mode of the surrounding icons. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] [1. Warning system and vehicle configuration] 1 to 3, the configuration of a vehicle 100 and an attention calling system 1 mounted on the vehicle 100 will be described. The vehicle 100 corresponds to a moving body in the present disclosure. As shown in FIG. 1, the attention calling system 1 includes a controller 2 and a HUD (Head Up Display) 30. The HUD 30 corresponds to a display device in the present disclosure.

[0016] The controller 2 is a control unit including a processor 10, a memory 20, an interface circuit (not shown), and the like. The HUD 30 outputs an attention-calling display 50, which is a visual display visible to a driver (operator) P shown in FIG. 2, onto a windshield 40 of the vehicle 100. With reference to FIGS. 2 and 3, the HUD 30 is disposed on a dashboard or the like below the windshield 40, and projects (outputs) the attention-calling display 50 onto the windshield 40, thereby projecting a virtual image of the attention-calling display 50 in front of the driver. The HUD 30 of this embodiment is configured to be capable of color display.

[0017] 3, the attention-calling display 50 includes a moving object icon 51 that resembles a vehicle 100, and surrounding icons 52 whose display range changes in the circumferential direction within a circular area 52r that surrounds the moving object icon 51. Details of the display mode of the surrounding icons 52 will be described later.

[0018] In addition to the attention calling system 1, the vehicle 100 is equipped with a plurality of cameras 31a to 31d, a radar 32, a plurality of sonars 33a to 33h, and a plurality of speakers 34a to 34d. As shown in Fig. 2, the camera 31a is provided on the upper part of the windshield 40 or the like, and outputs a captured image of the area in front of the vehicle 100 to the controller 2. The camera 31b is provided near the right door mirror or the like, and outputs a captured image of the area on the right side of the vehicle 100 to the controller 2, and the camera 31c is provided near the left door mirror or the like, and outputs a captured image of the area on the left side of the vehicle 100 to the controller 2. The camera 31d is provided at the rear of the vehicle 100, and outputs a captured image of the area behind the vehicle 100 to the controller 2.

[0019] 2, the radar 32 is provided at the front of the vehicle 100, detects the position of an object present in front of the vehicle 100, and outputs a position detection signal to the controller 2. The sonars 33a to 33d are provided at the front of the vehicle 100, detect the position of an object present in the vicinity of the front of the vehicle 100, and output a position detection signal to the controller 2. The sonars 33e to 33h are provided at the rear of the vehicle 100, detect the position of an object present in the vicinity of the rear of the vehicle 100, and output a position detection signal to the controller 2. The speakers 34a to 34d are provided, for example, at the driver's door 35a, passenger door 35b, right rear door 35c, and left rear door 35d of the vehicle 100, respectively.

[0020] The processor 10 reads and executes a control program 21 for the attention calling system 1 stored in the memory 20, thereby functioning as the object recognition unit 11, the display control unit 12, the sound image localization control unit 13, and the influence degree recognition unit 14. The control program 21 may be read from a recording medium (magnetic disk, optical disk, flash memory, etc.) and stored in the memory 20, or may be downloaded from an external server or the like and stored in the memory 20.

[0021] The object recognition unit 11 recognizes objects (pedestrians, other vehicles, etc.) present in front of the vehicle 100 based on images captured by the cameras 31a to 31d, object position detection signals from the radar 32, and object position detection signals from the sonars 33a to 33h. Note that, in the present embodiment, a configuration is shown in which objects present around the vehicle 100 are recognized using the cameras 31a to 31d, the radar 32, and the sonars 33a to 33h, but an object may be recognized by any one or a combination of two or more of these, rather than by all of them. Alternatively, an object may be recognized by another type of detection device.

[0022] The display control unit 12 HUD30 to control the output of the attention calling display 50, thereby changing the display mode of the surrounding icons 52. The sound image localization control unit 13 controls the output mode (output level, output timing, etc.) of the speakers 34a to 34d to change the position of the sound image of a predetermined attention calling sound perceived by the driver P. Note that the number of speakers may be more than one, and may be three or less, or five or more.

[0023] The influence degree recognition unit 14 recognizes the influence degree of the object on the vehicle 100 in four levels, from level 0 to level 3, according to the recognition state of the object by the object recognition unit 11. Level 0 is a state in which the object is not recognized by the object recognition unit 11. Levels 1 to 3 correspond to states in which the object is recognized by the object recognition unit 11, and the influence degree recognition unit 14 recognizes the influence degree by switching it from level 1 to level 2 to level 3 as the distance between the object and the vehicle 100 becomes shorter.

[0024] The influence degree recognizing unit 14 may recognize the influence degree by taking into consideration the speed at which the object approaches the vehicle 100. In this case, the influence degree recognizing unit 14 recognizes the influence degree by switching it from Level 1 to Level 2 to Level 3 as the speed at which the object approaches the vehicle 100 increases.

[0025] The display control unit 12 acquires the number of objects recognized by the object recognition unit 11 (object recognition result), and sets the attention level for the surrounding icons 52 and the attention sound to one of four levels, from level 0 to level 3, based on the number of objects and the degree of impact recognized by the impact degree recognition unit 14. When there is no object recognized by the object recognition unit 11, the display control unit 12 sets the attention level to level 0.

[0026] Furthermore, when the number of objects recognized by the object recognition unit 11 is 1 (there is a single object), the display control unit 12 sets the level of the impact degree of the object recognized by the impact degree recognition unit 14 as the attention level. For example, when the impact degree is level 2, the display control unit 12 sets the attention level to level 2.

[0027] Furthermore, when there are multiple objects recognized by the object recognition unit 11, the display control unit 12 increases the attention level as the number of objects increases. In this embodiment, the display control unit 12 sets the attention level to level 1 when the number of objects is 2, sets the attention level to level 2 when the number of objects is 3, and sets the attention level to level 3 when the number of objects is 4 or more. The number 2 in this case corresponds to the upper limit number in the present disclosure.

[0028] In addition, when there are a plurality of objects recognized by the object recognition unit 11 and the influence degree of any of the objects increases, the attention level may be increased. For example, when there are two objects and the attention level is set to level 1, if the influence degree of any of the objects increases to level 2, the attention level may be increased from level 1 to level 2.

[0029] The display control unit 12 sets the display mode (display range, diameter, color) of the surrounding icon 52 according to the attention level in accordance with the setting table shown in Fig. 12. The size of the surrounding icon 52 is determined by the diameter. For example, when two objects are recognized by the object recognition unit 11, the display control unit 12 determines the attention level to Level 1. Then, the object recognition unit 11 sets the display mode of the surrounding icon 52 to the entire circular area, medium diameter, or yellow, which corresponds to Level 1 when there are multiple objects.

[0030] [2. Attention Processing] The attention drawing processing executed by the object recognition unit 11, the display control unit 12, the sound image localization control unit 13, and the influence degree recognition unit 14 will be described below in accordance with the flowcharts shown in FIGS. 4 to 6, with reference to the explanatory diagrams shown in FIGS. 7 to 11.

[0031] In step S1 of Fig. 4, the sound image localization control unit 13 controls the output mode of the speakers 34a to 34d to output the attention warning sound so that the position of the sound image of the attention warning sound is all around the driver P (corresponding to a predetermined range around the pilot in the present disclosure). As a result, as shown in B1 of Fig. 11, the position 60 of the sound image of the attention warning sound is all around the driver P, resulting in a situation in which the driver P perceives the attention warning sound from all around. The attention warning sound in this case is set to a gentle sound that fosters a sense of security, corresponding to attention warning level 0.

[0032] In the next step S2, the display control unit 12 causes the HUD 30 to display the surrounding icons 52 with a display range covering the entire circular area. As a result, as shown in A11 of FIG. 7, a virtual image of the circular surrounding icons 52 that entirely surrounds the moving object icon 51 is visible to the driver P through the windshield 40. In this case, the surrounding icons 52 are displayed in white, which corresponds to attention call level 0, according to the setting table of FIG. 12, and the driver P can be prompted to pay attention to a wide range. The display mode of the surrounding icons 52 shown in A11 is the same as that of the first embodiment of the present disclosure. 2 This corresponds to the display mode.

[0033] In the next step S3, the object recognition unit 11 searches for an object present around the vehicle 100. In the following step S4, if the object recognition unit 11 recognizes an object present around the vehicle 100, the process proceeds to step S20 in Fig. 5. On the other hand, if the object recognition unit 11 does not recognize an object present around the vehicle 100, the object recognition unit 11 proceeds to step S5.

[0034] In step S5, the display control unit 12 starts a timer for checking whether a state in which an object existing around the vehicle 100 is not recognized continues. The timer is set to, for example, 3 seconds. In the subsequent loop processing of steps S6 to S8, the object recognition unit 11 searches for an object existing around the vehicle 100 in step S6 until the timer times out in step S8. Then, in step S7, if the object recognition unit 11 recognizes an object existing around the vehicle 100, the process proceeds to step S20 in FIG. 5, and if the object recognition unit 11 does not recognize an object existing around the vehicle 100, the process proceeds to step S8.

[0035] In step S9, the display control unit 12 switches the surrounding icons 52 to a blinking display. In this case, the lighting mode of the surrounding icons 52 is the same as that of the first embodiment of the present disclosure. 3 This corresponds to a display mode. The display control unit 12 blinks the surrounding icon 52, for example, once every few seconds. This makes it possible to urge the driver P to widen the attention range without regarding it as safe even in a situation where an object is not recognized by the object recognition unit 11. In the next step S10, the object recognition unit 11 searches for an object present in the vicinity of the vehicle 100. In the following step S11, if the object recognition unit 11 recognizes an object present in the vicinity of the vehicle 100, the process proceeds to step S20 in FIG. 5, and if the object recognition unit 11 does not recognize an object present in the vicinity of the vehicle 100, the process proceeds to step S10.

[0036] In step S20 in Fig. 5, if there are multiple recognized objects, the object recognition unit 11 advances the process to step S40 in Fig. 6, and if there is a single recognized object, the object recognition unit 11 advances the process to step S21. In step S40, the display control unit 12 determines the attention level according to the number of objects, as described above.

[0037] In the next step S41, the display control unit 12 sets the display range of the surrounding icons 52 to the entire annular area 52r (see FIG. 3), changes the diameter and color of the surrounding icons 52 according to the attention level according to the setting table in FIG. 12, and proceeds to the process of step S25 in FIG. 5. For example, when the attention level is level 2, the display control unit 12 sets the diameter of the surrounding icons 52 to "small" and the color to "orange" according to the setting table in FIG. 12.

[0038] The sound image localization control unit 13 executes the process of step S50 in parallel with the process of step S41 by the display control unit 12. In step S50, the sound image localization control unit 13 outputs an attention calling sound according to the attention calling level, with the position of the sound image being all around the driver P. In this case, the sound image localization control unit 13 performs processes such as increasing the volume of the attention calling sound or changing the tone of the attention calling sound so that the higher the attention calling level, the more effective the attention calling for the driver P becomes. In addition, the attention calling sound output in step S1 of Fig. 4 may be changed by superimposing a sound effect that evokes the attributes of the target object (four-wheeled vehicle, two-wheeled vehicle, pedestrian, etc.).

[0039] On the other hand, when the recognized object is a single object, the object recognition unit 11 advances the process from step S20 to step S21. In step S21, the influence degree recognition unit 14 recognizes the influence degree of the object. In the following step S22, the display control unit 12 determines the attention calling level according to the influence degree of the object, as described above.

[0040] In the next step S23, the display control unit 12 sets the color and diameter of the surrounding icon 52 according to the setting table in Fig. 12 depending on the attention level. For example, when the attention level is Level 1, the display control unit 12 sets the color of the surrounding icon 52 to "yellow" and the diameter to "large". In the following step S24, the display control unit 12 gradually reduces the display range of the surrounding icon 52 toward the portion 52a corresponding to the direction from the vehicle 100 to the object, as shown in Fig. 7, and proceeds to the process of step S25.

[0041] FIG. 7 illustrates a case where an object present on the right front side of the vehicle 100 is recognized by the object recognition unit 11. In FIG. Display control unit 12 The display mode of the surrounding icon 52 is gradually reduced from the state in which the display range covers the entire circular area as shown by A11 to the portion 52a as shown by A12 and A13. This makes it possible to prompt the driver P to pay attention to the right front of the vehicle 100.

[0042] The sound image localization control unit 13 executes the processes of steps S30 and S31 in parallel with the processes of steps S22 and S24. To In this case, the sound image localization control unit 13 outputs an attention calling sound having a higher notification effect as the level of the influence degree of the object increases. For example, the sound image localization control unit 13 may output an attention calling sound in which a sound effect (such as an onomatopoeia of an engine) that evokes an attribute of the object (such as a four-wheeled vehicle, a two-wheeled vehicle, a pedestrian, etc.) is superimposed on the attention calling sound that started to be output in step S1 of FIG. 4.

[0043] 11, the sound image localization control unit 13 gradually reduces the range of the position 60 of the sound image of the attention warning sound from the entire periphery of the driver P shown in B1 toward the direction G from the driver P to the object T, and proceeds to step S24. This brings the driver P into a situation where he or she perceives the attention warning sound only from the direction of the object, and it is expected that the driver P's attention will be directed to the object.

[0044] In step S25, the object recognition unit 11 searches for an object present around the vehicle 100. In the following step S26, if the object recognition unit 11 recognizes an object, the process proceeds to step S26, and if the object recognition unit 11 does not recognize an object, the process proceeds to step S1 in FIG.

[0045] In step S27, the influence degree recognition unit 14 recognizes the influence degree of the object. In the following step S28, the display control unit 12 determines an attention level according to the number of objects recognized by the object recognition unit 11 and the influence degree of the objects recognized by the influence degree recognition unit 14. Then, the display control unit 12 changes the display mode of the surrounding icons 52 according to the change in the attention level. In addition, the sound image localization control unit 13 changes the output mode of the attention sound according to the attention level. Specifically, the display control unit 12 changes the display mode of the surrounding icons 52 as follows.

[0046] (1) When the number of objects changes from singular to plural. The display control unit 12 expands the display range of the surrounding icons 52 from A21 to A22 to A23 to A24 to the entire range of the circular area, as shown in Fig. 8. A21 in Fig. 8 is the area recognized by the object recognition unit 11 as the vehicle 100. of 8 shows a situation in which one object existing in the front right is recognized, and the display control unit 12 sets the portion 52b corresponding to the direction from the vehicle 100 to the object as the display range of the surrounding icon 52. A22 in Fig. 8 shows a situation in which an object existing in the front left of the vehicle 100 is also recognized, and the display control unit 12 adds the portion 52c corresponding to the direction from the vehicle 100 to the object in the front left to the display range of the surrounding icon 52.

[0047] Then, as shown in A23, the display control unit 12 expands the portions 52a and 52b corresponding to the directions of the two objects in the circumferential direction, and sets the display range of the surrounding icon 52 to the entire annular area, as shown in A24. In this way, when the number of objects recognized by the object recognition unit 11 changes from one to multiple, the display range of the surrounding icon 52 is expanded from the portion corresponding to the direction of the object to the entire annular area, thereby encouraging the driver P to pay attention to the multiple objects.

[0048] (2) When the number of objects increases. When the number of objects recognized by the object recognition unit 11 increases, the display control unit 12 changes the display mode of the surrounding icons 52 in accordance with the setting table shown in Fig. 12. Fig. 9 illustrates a state in which a plurality of objects are recognized by the object recognition unit 11, and the display control unit 12 displays the surrounding icons 52 whose display range is the entire circular area.

[0049] 9, the display control unit 12 changes the A31 "medium diameter, yellow" corresponding to level 1 of the attention level according to the setting table shown in Fig. 12 in response to an increase in the number of objects (2 objects -> 3 objects -> 4 objects), to "small diameter, orange" of A32 corresponding to level 2, and to "very small diameter, flashing red" corresponding to level 3. In this way, by decreasing the diameter of the surrounding icon 52 to shorten the interval between the moving object icon 51 and the surrounding icon 52, and by changing the color and lighting state of the surrounding icon 52, it is possible to more strongly call the driver P's attention to the object.

[0050] Also, when the number of objects is decreased, the display control unit 12 changes the display mode of the surrounding icons 52 according to the setting table shown in Fig. 12. For example, when the number of objects is decreased (4 objects -> 3 objects -> 2 objects) from a state in which 4 objects are recognized by the object recognition unit 11, the display control unit 12 changes the display mode of the surrounding icons 52 from A33 -> A32 -> A31 in Fig. 9.

[0051] (3) When the object transitions from a state in which it is recognized to a state in which it is not recognized. When the state transitions from one in which an object is recognized by the object recognition unit 11 to one in which the object is not recognized by the object recognition unit 11, the display control unit 12 performs a process of temporarily moving the surrounding icon 52 away from the moving object icon 51, as shown in FIG. 10.

[0052] 10, A41 shows a state in which the surrounding icon 52 corresponding to an object present in the front right of the vehicle 100 is displayed. The display mode of the surrounding icon 52 shown in A41 corresponds to the first display mode of the present disclosure. In this state, when the object recognition unit 11 transitions to a state in which the object is not recognized, the display control unit 12 moves the surrounding icon 52 away from the moving object icon 51 and expands the display range in the circumferential direction, as shown in A42.

[0053] Next, as shown in A43, the display control unit 12 sets the size of the surrounding icon 52 to "large" corresponding to the attention level 0, and moves the surrounding icon 52 closer to the moving object icon 51. Then, as shown in A44, the display control unit 12 expands the display range of the surrounding icon 52 in the circumferential direction to a display mode of "large, white" corresponding to level 0. The display mode of the surrounding icon 52 shown in A44 corresponds to the second display mode of the present disclosure. In this way, by temporarily moving the surrounding icon 52 away from the moving object icon and then changing the display mode to the display mode corresponding to level 0, the driver P can intuitively perceive that the need for attention to the object has decreased.

[0054] 3. Other embodiments In the above embodiment, the attention-attracting process for the driver P includes a process of changing the display mode of the attention-attracting display 50 by the display control unit 12 and a process of changing the range of the position of the sound image of the attention-attracting sound by the sound image localization control unit 13. As another embodiment, the sound image localization control unit 13 may be omitted, and only the process of changing the display mode of the attention-attracting display 50 by the display control unit 12 may be performed.

[0055] In the above embodiment, a four-wheeled vehicle 100 was shown as the moving body of the present disclosure, but the moving body of the present disclosure may be any moving body that is operated by a pilot, for example, a two-wheeled vehicle, an aircraft, a ship, etc.

[0056] In the above embodiment, when the object recognized by the object recognition unit 11 has one object, the display control unit 12 sets an attention level according to the influence degree of the object recognized by the influence degree recognition unit 14 and changes the display mode of the attention calling display, using the setting table of the attention calling display shown in Fig. 12. As another embodiment, the influence degree recognition unit 14 may be omitted, and a configuration may be adopted in which the process of changing the display mode of the attention calling display according to the influence degree of the object is not performed.

[0057] In the above embodiment, the display control unit 12 turns on the attention display 50 (the third display) when a situation where the object is not recognized by the object recognition unit 11 continues for a predetermined time or more through the processing of steps S5 to S9 in FIG. 2 Display mode) to blinking display (first 3 The display mode was changed to the other configuration. 3 The display mode may be a display mode other than a blinking display. For example, the display mode of the attention-calling display 50 may be changed by changing the brightness, color, etc. 2 Display format is different 3 The display mode may be switched to another mode. Also, the processes in steps S5 to S9 in FIG.

[0058] In the above embodiment, a HUD 30 is shown as a display device of the present disclosure, but the display device of the present disclosure may be a display device other than a HUD as long as it has a configuration for outputting a visual display that is visible to the pilot on the windshield of a moving object.

[0059] In the above embodiment, the size of the surrounding icons 52 is determined by the diameter of the surrounding icons 52, but the size of the surrounding icons 52 may be determined by the perimeter of the surrounding icons 52, etc. Also, in the above embodiment, the surrounding icons 52 are circular, but the surrounding icons 52 may have any shape that surrounds the moving object icon 51, such as an ellipse or a rectangle.

[0060] 1 is a schematic diagram showing the configuration of the attention calling system 1 divided according to main processing contents in order to facilitate understanding of the present invention, and the configuration of the attention calling system 1 may be divided in other ways. Furthermore, the processing of each component may be executed by one hardware unit or by multiple hardware units. Furthermore, the processing of each component shown in FIGS. 4 to 6 may be executed by one program or by multiple programs.

[0061] [4. Configuration supported by the above embodiment] The above embodiment is a specific example of the following configuration.

[0062] (Configuration 1) An attention warning system including a display device that outputs a visual display on a windshield of a moving body that is visible to a driver of the moving body, a display control unit that causes the display device to output a predetermined attention warning display on the windshield as the visual display, and an object recognition unit that recognizes objects present in the vicinity of the moving body, wherein the display control unit causes the display device to display the attention warning display on the windshield, the attention warning display including a moving body icon that resembles the moving body and a surrounding icon whose display range is changed in the circumferential direction of a circular area surrounding the moving body icon, and when the object is recognized by the object recognition unit, and displays the surrounding icons on the windshield by the display device, with a display range that includes a portion that corresponds to the direction of the object relative to the moving body, and when the number of the objects recognized by the object recognition unit is equal to or greater than a predetermined upper limit, the display range of the surrounding icons is the entire annular area, and when the number of the objects recognized by the object recognition unit is equal to or greater than the upper limit and the display range of the surrounding icons is the entire annular area, the size of the surrounding icons is reduced when the number of the objects recognized by the object recognition unit increases. According to the attention calling system of configuration 1, it is possible to assist the pilot in quickly directing his / her attention to an object.

[0063] (Configuration 2) The display control unit, when the object is not recognized by the object recognition unit, sets the display range of the surrounding icons to the entire annular area, and when the state transitions from when the object is not recognized by the object recognition unit to when the object is recognized by the object recognition unit, gradually reduces the display range of the surrounding icons from the entire annular area to a part of the annular area that corresponds to the direction from the moving body to the object. According to the attention calling system of configuration 2, it is possible to urge the pilot to pay attention in the direction where the object exists.

[0064] (Configuration 3) The attention warning system described in Configuration 1 or Configuration 2, wherein the display control unit switches the display mode of the surrounding icons to a second display mode in which the entire annular area is displayed when the object recognition unit recognizes fewer than the upper limit number of objects and the display mode of the surrounding icons transitions from a state in which the object recognition unit recognizes fewer objects than the upper limit number and the display mode of the surrounding icons transitions from a state in which the display mode of the surrounding icons is a first display mode in which a portion of the annular area corresponding to the direction of the object relative to the moving body is displayed to a state in which the object recognition unit does not recognize the object. According to the attention alert system of configuration 3, when an object is not recognized, the pilot can be urged to pay more attention to the surroundings of the moving object.

[0065] (Configuration 4) The attention warning system described in Configuration 3, wherein the display control unit performs a process of temporarily increasing the distance between the moving body icon and the surrounding icons while switching the display mode of the surrounding icons from the first display mode to the second display mode. According to the attention warning system of configuration 4, the pilot can intuitively perceive that the need for attention to the object has decreased.

[0066] (Configuration 5) When the object is not recognized by the object recognition unit, the display control unit changes the display mode of the surrounding icons to a display range of the entire annular area. 2When a state in which the object is not recognized by the object recognition unit continues for a predetermined time or more, the display mode of the surrounding icons is changed to the display mode of the surrounding icons with the entire annular area as a display range. 2 Display format is different 3 The attention warning system according to any one of configurations 1 to 4, wherein the display mode is a display mode. According to the warning system of configuration 5, when a state in which an object is not recognized by the object recognition unit continues for a predetermined time or longer, the state is not considered to be safe, and the pilot is urged to expand the range of attention.

[0067] (Configuration 6) An attention warning system described in any one of configurations 1 to 5, further comprising an influence degree recognition unit that recognizes the influence degree of the object recognized by the object recognition unit on the moving body, and the display control unit changes the display mode of the surrounding icons depending on the influence degree of the object. According to the attention calling system of configuration 6, the degree of influence of the object on the moving body can be notified to the pilot by changing the display mode of the attention calling display.

[0068] (Configuration 7) An attention warning system described in any one of configurations 1 to 6, comprising a sound image localization control unit that controls the output mode of multiple speakers provided on the moving body to change the position of a sound image of a predetermined attention warning sound perceived by the pilot, wherein the sound image localization control unit, when the object is not recognized by the object recognition unit, sets the position of the sound image of the attention warning sound to a predetermined range around the pilot, and when the state transitions from a state in which the object is not recognized by the object recognition unit to a state in which the object is recognized by the object recognition unit, gradually decreases the range of the position of the sound image of the attention warning sound from the predetermined range in the direction from the pilot to the object. According to the attention warning system of configuration 7, the effect of directing the pilot's attention to the object can be increased by gradually decreasing the range of the position of the sound image of the attention warning sound in the direction from the pilot to the object.

[0069] (Configuration 8) A control program that causes a controller, which controls the operation of a display device that acquires a recognition result from an object recognition unit that recognizes objects present around a moving body and outputs a visual display visible to a driver of the moving body on a windshield of the moving body, to execute a process of causing the display device to display a predetermined attention-calling display corresponding to the recognition result as the visual display on the windshield, the control program causing the controller to cause the display device to display the attention-calling display on the windshield, the attention-calling display including a moving body icon that resembles the moving body and a surrounding icon whose display range is changed in the circumferential direction of a circular area surrounding the moving body icon, and the object recognition unit to display the attention-calling display on the windshield. a control program that, when an object is recognized, causes the display device to display the surrounding icons on the windshield, with a display range including a portion corresponding to the direction of the object relative to the moving body, and when the number of the objects recognized by the object recognition unit is equal to or greater than a predetermined upper limit, causes the display range of the surrounding icons to be the entire annular area, and when the number of the objects recognized by the object recognition unit is equal to or greater than the upper limit and the display range of the surrounding icons is the entire annular area, executes a process to reduce the size of the surrounding icons when the number of objects recognized by the object recognition unit increases. By executing the control program of configuration 8 by the controller, the same effects as those of the attention warning system of configuration 1 can be obtained. [Explanation of symbols]

[0070] 1...attention warning system, 2...controller, 10...processor, 11...object recognition unit, 12...display control unit, 13...sound image localization control unit, 14...influence degree recognition unit, 20...memory, 21...control program, 30...HUD (display device), 31a to 31d...camera, 32...radar, 33a to 33h...sonar, 34a to 34d...speaker, 40...windshield, 50...attention warning display, 51...moving object icon, 52...surrounding icon, 60...position of sound image of attention warning sound, P...driver (pilot), T...object.

Claims

1. a display device that outputs a visual display on a windshield of the vehicle that is viewable by a driver of the vehicle; a display control unit that causes the display device to output a predetermined attention-calling display onto the windshield as the visual display; An object recognition unit that recognizes objects present around the moving object; An attention calling system comprising: The display control unit is displaying, on the windshield, the attention-calling display including a moving object icon resembling the moving object and a surrounding icon whose display range is changed in a circumferential direction of an annular area surrounding the moving object icon, by the display device; When the object is recognized by the object recognition unit, the surrounding icon is displayed on the windshield by the display device, with a display range including a portion corresponding to a direction of the object relative to the moving body being set as a display range; When the number of the objects recognized by the object recognition unit is equal to or greater than a predetermined upper limit number, a display range of the surrounding icons is set to the entire annular area, When the number of the objects recognized by the object recognition unit is greater than or equal to the upper limit number and the display range of the surrounding icons is the entire annular area, if the number of the objects recognized by the object recognition unit increases, the size of the surrounding icons is reduced. Attention warning system.

2. The display control unit is When the object is not recognized by the object recognition unit, the display range of the surrounding icons is set to the entire annular area; When a state where the object is not recognized by the object recognition unit is changed to a state where the object is recognized by the object recognition unit, a display range of the surrounding icons is gradually decreased from the entire annular area to a part of the annular area corresponding to a direction from the moving body to the object. The warning system according to claim 1 .

3. The display control unit is When the object recognition unit recognizes the objects less than the upper limit number and the display mode of the surrounding icons transitions from a state in which the display mode is a first display mode in which a part of the annular area corresponding to the direction of the object relative to the moving body is set as a display range to a state in which the object is not recognized by the object recognition unit, the display mode of the surrounding icons is switched to a second display mode in which the entire annular area is set as a display range. The attention calling system according to claim 1 or 2.

4. The display control unit performs a process of temporarily increasing a distance between the moving object icon and the surrounding icons while switching a display mode of the surrounding icons from the first display mode to the second display mode. The warning system according to claim 3 .

5. When the object is not recognized by the object recognition unit, the display control unit sets the display mode of the surrounding icons to a second display mode in which the entire annular area is a display range, and when a state in which the object is not recognized by the object recognition unit continues for a predetermined time or more, sets the display mode of the surrounding icons to a third display mode different from the second display mode in which the entire annular area is a display range. The attention calling system according to claim 1 or 2.

6. an influence degree recognition unit that recognizes an influence degree of the object recognized by the object recognition unit on the moving body; The display control unit changes a display mode of the surrounding icon according to the degree of influence of the object. The attention calling system according to any one of claims 1 to 5.

7. a sound image localization control unit that controls an output mode of a plurality of speakers provided in the moving body to change a position of a sound image of a predetermined attention sound perceived by the operator; The sound image localization control unit includes: When the object is not recognized by the object recognition unit, the position of the sound image of the attention sound is set to a predetermined range around the pilot, When a state transitions from one in which the object is not recognized by the object recognition unit to one in which the object is recognized by the object recognition unit, a range of a position of a sound image of the attention-calling sound is gradually decreased from the predetermined range in a direction from the operator to the object. The attention calling system according to any one of claims 1 to 6.

8. Obtaining a recognition result from an object recognition unit that recognizes objects present around the moving body; a control program that causes a controller that controls an operation of a display device that outputs a visual display on a windshield of the moving body that is visible to a driver of the moving body to execute a process of displaying a predetermined attention-calling display on the windshield according to the recognition result as the visual display by the display device, The controller, displaying, on the windshield, the attention-calling display including a moving object icon resembling the moving object and a surrounding icon whose display range is changed in a circumferential direction of an annular area surrounding the moving object icon, by the display device; When the object is recognized by the object recognition unit, the surrounding icon is displayed on the windshield by the display device, with a display range including a portion corresponding to a direction of the object relative to the moving body being set as a display range; When the number of the objects recognized by the object recognition unit is equal to or greater than a predetermined upper limit number, a display range of the surrounding icons is set to the entire annular area, When the number of the objects recognized by the object recognition unit is greater than or equal to the upper limit number and the display range of the surrounding icons is the entire annular area, if the number of the objects recognized by the object recognition unit increases, the size of the surrounding icons is reduced. A control program that executes processing.

Citation Information

Patent Citations

  • Sound device totallizing system on vehicle

    JP1995159190A

  • Head-up display device

    JP2015104930A

  • Alarm system for vehicle

    JP2015187806A

  • Display control device and display control program

    JP2016048562A

  • Image displaying system, and method, for on-vehicle use

    JP2017021546A