Attention reminder system and attention reminder methods

Abstract

This invention provides a attention reminder system and a attention reminder method that remind an operator to pay attention to various physical objects in the surrounding environment of a moving object without distracting the operator's attention. The attention reminder system includes: a projection device that outputs a visual display on the interior structure of the moving object for visual confirmation by the operator; and an attention reminder device that reminds the operator to pay attention to objects around the moving object through the aforementioned visual display. The attention reminder device includes: a detection unit that detects objects around the moving object; and an attention reminder unit that controls the projection device to output a visual display within a predetermined display range on the interior structure of the moving object. This display range is a strip-shaped area extending in a horizontal direction orthogonal to the height direction of the moving object. The attention reminder unit outputs the visual display at a horizontal position within the display range on the interior structure that aligns with the direction of the object as observed from the operator's position.

Description

Technical Field

[0001] This invention relates to a attention reminder system and method for alerting the operator of a moving object to objects surrounding the moving object. Background Technology

[0002] It would be convenient if it could alert the driver of the vehicle to potential road users who might be easily overlooked in the surrounding environment.

[0003] Previously, there was a known vehicle image display system that displayed an outline image of a preceding vehicle on the windshield of the vehicle, overlapping the image of the preceding vehicle as seen from the operator's position (Patent Document 1).

[0004] However, in urban traffic environments with many road users, when the technology of the aforementioned conventional display systems is intended to provide drivers with attention reminders related to road users, displaying numerous outline images overlapping with each road user on the windshield may actually distract the driver.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2017-021546 Summary of the Invention

[0008] The problem the invention aims to solve

[0009] The present invention was made in view of the above circumstances, so as to draw the operator’s attention to the various physical objects in the surrounding environment of the moving body without distracting the operator’s attention.

[0010] means for solving problems

[0011] One aspect of the present invention is a attention reminder system comprising: a projection device that outputs a visual display on an interior structure of a moving body, which can be visually confirmed by an operator of the moving body; and an attention reminder device that controls the projection device to remind the operator of objects around the moving body through the visual display, wherein the attention reminder device includes: a detection unit that detects objects located in the area surrounding the moving body; and an attention reminder unit that controls the projection device to output a visual display within a predetermined display range on the interior structure of the moving body, the display range on the interior structure being a strip-shaped range extending in a horizontal direction orthogonal to the height direction of the moving body, and the attention reminder unit outputting the visual display at a horizontal position within the display range on the interior structure that is consistent with the direction of the object as observed from the operator's position.

[0012] According to another aspect of the invention, in addition to the visual display, the attention reminder unit also generates a sound for directional positioning of the object for the operator via a speaker disposed in the room of the moving body.

[0013] According to another aspect of the invention, the projection device outputs the visual display by projecting light of a predetermined shape onto the interior structure.

[0014] According to another aspect of the invention, the interior structure is the front glass, front triangular window glass, door glass and / or rear glass of the moving body.

[0015] According to another aspect of the invention, at least one of the projection devices outputs the visual display by projecting light of a predetermined shape onto the front glass of the moving body.

[0016] According to another aspect of the invention, the projection device that outputs the visual display on the front glass consists of a plurality of light sources arranged horizontally below the front glass.

[0017] According to another aspect of the present invention, the attention reminder unit displays a circular display on the display device indicating the direction relative to the moving body, and outputs a visual display at the arc portion of the circular display corresponding to the direction of the object.

[0018] According to another aspect of the invention, the object is a physical object whose probability of contact with the moving body is a predetermined value or higher.

[0019] According to another aspect of the invention, the attention reminder unit displays the visual display in a manner corresponding to the contact probability.

[0020] Another aspect of the present invention is a computer-executed attention reminder method, wherein the attention reminder method comprises the following steps: detecting an object located in the area surrounding a moving body; and controlling a projection device disposed on the moving body to output a visual display on a predetermined display range on an interior structure of the moving body, the display range on the interior structure being a strip-shaped range extending in a horizontal direction orthogonal to the height direction of the moving body, wherein in the step of outputting the visual display, the visual display is output at a horizontal position within the display range on the interior structure that is consistent with the direction of the object as observed from the position of the operator of the moving body.

[0021] The effects of the invention

[0022] According to the present invention, it is possible to draw the operator's attention to various physical objects in the surrounding environment of the moving body without distracting the operator's attention. Attached Figure Description

[0023] Figure 1 This is a diagram showing the structure at the front of the passenger compartment of a vehicle equipped with a attention alert system according to an embodiment of the present invention.

[0024] Figure 2 This is a diagram illustrating the structure of the attention reminder system according to the first embodiment of the present invention.

[0025] Figure 3 It shows through Figure 2 The figure shows an example of a visual display on the windshield that serves as a attention reminder system.

[0026] Figure 4 It is shown Figure 2 The flowchart shown illustrates the steps of the action in the attention reminder system.

[0027] Figure 5 This is a diagram illustrating the structure of the attention reminder system according to the second embodiment of the present invention.

[0028] Figure 6 It is shown Figure 5 The diagram shows an example of cognitive proficiency information stored in a storage device by the cognitive skill evaluation unit of the attention reminder system.

[0029] Figure 7 It shows through Figure 5 The figure shows an example of a visual display on the windshield that serves as a attention reminder system.

[0030] Figure 8 It is shown Figure 5 The flowchart shown illustrates the steps of the action in the attention reminder system.

[0031] Figure 9 This is a diagram illustrating the structure of the attention reminder system according to the third embodiment of the present invention.

[0032] Figure 10 It shows through Figure 9 The figure shows an example of a visual display on the windshield that serves as a attention reminder system.

[0033] Figure 11 It is shown Figure 10 The diagram shows an example of how the visual display changes over time.

[0034] Figure 12 It is shown Figure 9 The flowchart shown illustrates the steps of the action in the attention reminder system.

[0035] Figure 13 This is a diagram illustrating the structure of the attention reminder system according to the fourth embodiment of the present invention.

[0036] Figure 14 It shows the mounting Figure 13 The diagram shows an example of the configuration of a projection device inside the vehicle's cabin for a attention alert system.

[0037] Figure 15 It is shown in Figure 13 The diagram shows an example of a vehicle-centric angular region defined in the attention alert system.

[0038] Figure 16 It is shown in Figure 13 The diagram shows an example of an object radar display on the HUD in a attention alert system.

[0039] Figure 17 This diagram illustrates examples of various display methods in object radar displays.

[0040] Figure 18 It is shown Figure 13 The flowchart shown illustrates the steps of the action in the attention reminder system.

[0041] Label Explanation

[0042] 100, 200, 201, 204… Vehicle, 102… Driver's seat, 103… Passenger seat, 104… Steering wheel, 105… Windshield, 106… Instrument panel, 110… Rearview mirror, 112… Left rearview mirror, 114… Right rearview mirror, 120, 300, 400, 500… Attention warning system, 122… Forward beam headlights, 124, 324, 424, 502… Attention warning devices, 130, 302, 402… 508…Processing unit; 131, 303, 403…Storage unit; 132, 512…Detection unit; 133, 304, 404, 514…Attention reminder unit; 140…Front object sensor; 142…Interior speaker; 144…Interior camera; 146…Brake pedal sensor; 148…Accelerator pedal sensor; 150…Rear object sensor; 152…Right side object sensor; 154…Left side object sensor; 156… HUD, 158…Left door glass, 160…Right door glass, 162…Rear glass, 202…Motorcycle, 203…Pedestrian, 306, 406…Behavior detection department, 308…Classification department, 310…Cognitive skill evaluation department, 312…Cognitive proficiency information, 410…Reaction skill evaluation department, 412…Risk calculation department, 414…Reaction proficiency information, 504…Left and right projection devices, 506…Rear projection device, 520…Object radar display , 522… vehicle icon, 524… circle display, 526… arrow, HA… display range, AD, AD-10, AD-12, AD-14, AD-16, AD-18, AD-20, AD-22, VI, VI-11, VI-12, VI-13, VI-14, VI-21, VI-22, VI-23, VI-24, VI-31, VI-32, VI-33, VI-34… visual display. Detailed Implementation

[0043] The embodiments will now be described with reference to the accompanying drawings.

[0044] [First Implementation]

[0045] First, the first embodiment of the present invention will be described. Figure 1 This is a diagram showing the structure at the front of the passenger compartment of a vehicle 100 equipped with a attention reminder system according to an embodiment of the present invention. Furthermore, Figure 2 This diagram illustrates the structure of the attention reminder system 120 according to the first embodiment. Furthermore, the structure of other parts of the vehicle 100 will be described later in relation to other embodiments.

[0046] In this embodiment, vehicle 100 is a four-wheeled automobile. Additionally, in Figure 1In the diagram, the Z-axis represents the height of vehicle 100, the X-axis represents the width of vehicle 100, and the Y-axis represents the length (or straight-line direction) of vehicle 100. Hereinafter, directions orthogonal to the height direction (Z-axis direction) (including the X-axis and Y-axis directions) will also be referred to as the horizontal direction.

[0047] Reference Figure 1 The vehicle 100 includes a driver's seat 102 and a passenger seat 103 within its interior. Additionally, the vehicle 100 includes a steering wheel 104, a windshield 105 separating the exterior from the interior, and an instrument panel 106. Furthermore, the vehicle 100 is equipped with an interior rearview mirror 110, a left rearview mirror (or left door rearview mirror) 112, and a right rearview mirror (or right door rearview mirror) 114 for the operator to visually confirm the rear and rear sides of the vehicle 100.

[0048] Reference Figure 2 The attention alert system 120 consists of a forward projection device 122 and an attention alert device 124. The forward projection device 122 is installed inside the vehicle 100 and outputs a visual display on the windshield 105 for the operator to visually confirm. The attention alert device 124 controls the forward projection device 122 and alerts the operator to objects through the aforementioned visual display. The forward projection device 122 outputs the visual display by projecting light of a predetermined shape (e.g., a rectangular shape) onto the windshield 105.

[0049] Reference Figure 1 In this embodiment, the forward projection device 122 is a light source array consisting of multiple light sources of the instrument panel 106 arranged along the width direction below the windshield 105, such as an LED array capable of projecting multiple colors of light. The forward projection device 122 projects light onto the windshield 105 under the control of the attention reminder device 124, displaying a visual display VI below the visible upper limit line L1 of the windshield 105 to guide the driver's gaze. The visible upper limit line L1 indicates the upper limit position of the windshield 105 in the vertical direction (vehicle height direction) where an image can be displayed. The position of the upper limit indicated by the visible upper limit line L1 is stipulated by law. For example, the position of this upper limit is sometimes also restricted by domestic laws or domestic rules.

[0050] The visual display VI is formed by light of a predetermined size projected onto the front glass 105. For example, the visual display VI is a virtual image formed in the operator's field of vision by the reflected light of the projected light. The position and size (horizontal length) of the visual display VI on the front glass 105 can be determined, for example, by selecting a light source from among the plurality of light sources constituting the front projection device 122 for projection onto the front glass 105.

[0051] The display range HA of the visual display VI extends below the upper visible line L1 of the windshield 105. The display range HA is a strip-shaped range on the windshield 105 having a defined width (e.g., ***) and extending along the width direction of the vehicle 100. The display range HA can also be a strip-shaped range on the windshield 105 extending in a horizontal direction orthogonal to the height direction of the vehicle 100.

[0052] Furthermore, the aforementioned light source array or LED array is one example of the front projection device 122, and the structure of the front projection device 122 is not limited to this. The front projection device 122 can have any structure as long as it can project a visual display VI of a predetermined size or larger within the aforementioned display range HA. For example, the front projection device 122 can also be a projector, displaying a visual display VI of a predetermined size or larger as an image within the display range HA.

[0053] Reference Figure 2 The attention reminder device 124 includes a processing unit 130 and a storage unit 131. The storage unit 131 may be, for example, a volatile and / or non-volatile semiconductor memory, and / or a hard disk drive. The processing unit 130 may be, for example, a computer equipped with a processor such as a CPU (Central Processing Unit). The processing unit 130 may also have a structure including a ROM (Read Only Memory) with a program written to it, RAM (Random Access Memory) for temporary data storage, etc. Furthermore, the processing unit 130 includes a detection unit 132 and an attention reminder unit 133 as functional elements or units.

[0054] These functional elements of the processing device 130 are implemented, for example, by executing a program by the processing device 130, which is a computer. Furthermore, the aforementioned computer program can be pre-stored in any computer-readable storage medium. Alternatively, all or part of the aforementioned functional elements of the processing device 130 can be constituted by hardware comprising one or more electronic circuit components.

[0055] The detection unit 132 detects objects present in the area surrounding the vehicle 100. An object is, for example, a defined physical object whose probability of contact with the vehicle 100, calculated based on its relative speed to the vehicle 100, is a predetermined value or higher.

[0056] Specifically, the detection unit 132 acquires images (or images) from a forward object sensor 140 installed on the vehicle 100 to detect objects in front of the vehicle 100. The forward object sensor 140 may be, for example, a camera, a laser, or a lidar. The detection unit 132 detects specified objects based on the forward image obtained from the forward object sensor 140, for example, detecting objects whose probability of contact with the vehicle 100, calculated based on their relative speed to the vehicle 100, is a specified value or higher. In this embodiment, the specified objects include traffic participants such as vehicles and pedestrians, as well as fixed objects such as road signs, utility poles, and mailboxes. The detection unit 132 sends the location information of each detected object to the attention alert unit 133.

[0057] Attention alert unit 133 controls the forward projection device 122 to output a visual display within the display range HA on the windshield 105 of the vehicle 100. Specifically, attention alert unit 133 displays a visual display VI at a horizontal position within the display range HA, which is the same as the horizontal position on the windshield 105, when the object is viewed from the operator's position.

[0058] More specifically, the attention reminder unit 133 calculates the intersection point of the operator's line of sight with the windshield 105 when the operator observes the objects, based on the position of the eyes of a standard operator seated in the control seat 102 and the position information of the objects received from the detection unit 132. Then, the attention reminder unit 133 displays a visual display VI at a horizontal position within a display range HA that corresponds to the horizontal position of each calculated intersection point.

[0059] Furthermore, when the attention reminder unit 133 detects an object in the area surrounding the vehicle 100 at a position that is outside the predetermined forward field of vision defined with the position of the operator sitting in the driver's seat of the vehicle 100 as the center, it generates a sound in the vehicle interior for the operator to locate the direction of the object by means of a plurality of interior speakers 142 distributed in the vehicle interior.

[0060] The aforementioned forward field of vision is the range of vision that an operator sitting in the control seat can naturally visually confirm while facing forward, for example, the range of vision in front of the operator sitting in the control seat in the straight direction of the vehicle 100. Figure 1 The field of view (Y direction shown) is centered on the line of sight and within a defined field of view angle (e.g., 45 degrees). The attention reminder unit 133 can determine whether an object has been detected at a position that has fallen out of the defined forward field of view based on the position information of each object received from the detection unit 132.

[0061] The attention alert system 120 with the above-described structure does not display an image that overlaps with the actual object on the windshield as in the past. Instead, it displays a visual display VI on the windshield 105 at a horizontal position within a strip-shaped display area HA extending along the vehicle width, which is the same horizontal position as the object as observed from the operator's position. Therefore, in the attention alert system 120, the chance of the operator observing the object overlapping with the visual display VI is reduced. Thus, it can alert the operator to various traffic participants present in the surrounding environment of the vehicle 100 in a way that makes the operator's attention more focused.

[0062] Furthermore, the forward projection device 122 outputs a visual display VI by projecting light of a predetermined shape (e.g., a rectangular shape) onto the front glass 105. Thus, the operator can be alerted to the object through a simple visual display VI without distracting their attention.

[0063] Furthermore, the forward projection device 122 can be composed of multiple light sources arranged along the width of the vehicle below the windshield 105, so the attention reminder system 120 can remind the operator to pay attention to the object through a simple structure.

[0064] Furthermore, when the attention alert unit 133 detects an object in the area surrounding the vehicle 100 at a position outside the predetermined forward field of vision defined centered on the position of the operator seated in the control seat, it generates an audible sound inside the vehicle to orient the object. Therefore, according to the attention alert system 120, for objects detected in areas where the operator may need to move their face during visual confirmation, the operator can orient the object by sound, thus alerting the operator to the object without distracting their attention.

[0065] Furthermore, in the attention alert system 120, the object is a physical object whose probability of contact with the vehicle 100 is above a predetermined value. That is, the attention alert system 120 filters out physical objects that may pose a risk to driving the vehicle 100 and displays a visual display VI indicating their direction, thus alerting the operator to the object in a way that makes the operator's attention more focused.

[0066] Figure 3 This figure illustrates an example of a visual display VI displayed on the front glass 105 via an attention alert system 120. Figure 3The figure shows the view of the area in front of vehicle 100 as observed by the driver of vehicle 100 through the windshield 105. As shown, in the area in front of an intersection in an urban area, there is a vehicle 200 parked near the left side of the intersection, a vehicle 201 approaching from the left side of the intersection, and a motorcycle 202 approaching from the right side of the intersection. In addition, there is a pedestrian 203 crossing the lane in which vehicle 100 is traveling, and a vehicle 204 traveling in the opposite lane of the lane.

[0067] At this time, the detection unit 132 detects vehicles 201 and 204, motorcycle 202, and pedestrian 203 crossing the road, in addition to the parked vehicle 200, as objects that may come into contact with the vehicle 100. Then, the attention reminder unit 133 displays visual displays VI-11, VI-12, VI-13, and VI-14 (shaded rectangles in the illustration) at the horizontal positions corresponding to the detected objects, namely vehicles 201 and 204, motorcycle 202, and pedestrian 203, respectively, within the display range HA on the windshield 105 (within the rectangle shown by the dotted line in the illustration).

[0068] Figure 4 This is a flowchart illustrating the steps of the actions in the attention reminder system 120. Figure 2 When the detection unit 132 detects at least one object, it begins to operate according to each detected object. Figure 4 The processing shown.

[0069] When processing begins, firstly, the attention reminder unit 133, based on the position information of each object detected by the detection unit 132, displays a visual display VI (S100) on the front glass 105 within the display range HA, at the horizontal position corresponding to the detected object, through the front projection device 122.

[0070] Next, the attention reminder unit 133 determines, based on the position information of the object received from the detection unit 132, whether the object was detected at a position outside the predetermined forward field of vision (S102). Then, if the object was detected at a position outside the predetermined forward field of vision (S102, "Yes"), the attention reminder unit 133 outputs a sound for directional positioning of the object to the operator via the indoor speaker 142 (S104).

[0071] Next, the attention reminder unit 133 determines whether the object is no longer detected by the detection unit 132 (S106). Then, when the object is no longer detected (S106, "Yes"), the attention reminder unit 133 ends this process. On the other hand, when the object is detected again (S106, "No"), the attention reminder unit 133 returns to step S100 and repeats the process.

[0072] [Second Implementation]

[0073] Next, the second embodiment of the present invention will be described. Figure 5 This is a diagram illustrating the structure of the attention reminder system 300 according to the second embodiment of the present invention. This attention reminder system 300 replaces the attention reminder system 120 and is mounted on the system described in the first embodiment. Figure 1 The vehicle shown is 100. Additionally, in... Figure 5 In China, use and Figure 2 The same label shown is used to indicate the same label as the one shown. Figure 2 The structural elements shown are the same as those described above. Figure 2 Explanation.

[0074] The attention reminder system 300 consists of a front projection device 122 and an attention reminder device 324. The front projection device 122 outputs a visual display on the front glass 105 that can be visually confirmed by the operator. The attention reminder device 324 controls the front projection device 122 and reminds the operator to pay attention to the object through the aforementioned visual display.

[0075] The attention reminder device 324 has the same as Figure 2 The attention reminder device 124 shown has the same structure, but differs in that it includes a processing device 302 and a storage device 303 instead of a processing device 130 and a storage device 131. The storage device 303 is configured, for example, by a volatile and / or non-volatile semiconductor memory and / or a hard disk device, and stores cognitive proficiency information 312 through the cognitive skill evaluation unit 310 of the processing device 302, which will be described later.

[0076] Processing device 302 has the same structure as processing device 130, but it differs in that it has an attention reminder unit 304 as a functional element or unit instead of attention reminder unit 133. In addition, processing device 302 also has a behavior detection unit 306, a classification unit 308 and a cognitive skill evaluation unit 310 as functional elements or units, which is different from processing device 130.

[0077] The functional elements possessed by the processing device 302 are the same as those of the processing device 130, and are implemented, for example, by the processing device 302, which is a computer, executing a program. Furthermore, the aforementioned computer program can be pre-stored in any computer-readable storage medium. Alternatively, all or part of the aforementioned functional elements possessed by the processing device 302 can be constructed from hardware comprising one or more electronic circuit components.

[0078] The behavior detection unit 306 detects the operator's eye movements and facial orientation movements (hereinafter also referred to as facial movements) via an indoor camera 144 installed in the vehicle interior of the vehicle 100. The indoor camera 144 is a so-called driver monitoring camera (DMC) that observes the operator.

[0079] The classification unit 308 classifies the detected objects into any one of several categories, based at least on the type of the object and the traffic scene in which the object is located. Here, the category of the object can refer not only to the type of the object, but also to distinctions such as the location, time, and / or movement of the object in the current traffic scene.

[0080] The categories of objects refer to the distinctions between pedestrians, vehicles, obstacles, etc. Pedestrians can be further subdivided into adults, children, the elderly, etc., and vehicles can be subdivided into cars, motorcycles, buses, bicycles, etc. Obstacles can be further subdivided into balls, fallen objects, disabled vehicles, etc.

[0081] The traffic scenarios described above distinguish between straight roads, railway crossings, intersections, and corners. Intersections can be further subdivided into intersections with parked vehicles, intersections without parked vehicles, intersections with poor visibility due to buildings, and intersections with good visibility. Furthermore, traffic scenarios can also be subdivided based on the number of traffic participants and / or their average speed.

[0082] Categories categorized as described above could include, for example, "a child appearing from behind a parked vehicle at an intersection," "a bicycle appearing from behind a building at an intersection with poor visibility," and "a motorcycle traveling straight in the opposite lane at an intersection," etc.

[0083] The cognitive skill evaluation unit 310 calculates cognitive proficiency, representing the level of familiarity with objects belonging to each of the aforementioned categories, and stores the calculated cognitive proficiency for each category as cognitive proficiency information 312 in the storage device 303. Here, the level of familiarity with an object refers, for example, to the degree of planning of actions (gaze or facial orientation) leading to the recognition of that object. The planning of actions leading to the recognition depends not only on the category of the object, i.e., the type of object, but also on the location, timing, and / or movement of the object in the current traffic scene. Therefore, the cognitive skill evaluation unit 310 stores the cognitive proficiency calculated for each object in association with the category of the object as cognitive proficiency information 312 representing the cognitive proficiency of each category in the storage device 303.

[0084] Specifically, the cognitive skill evaluation unit 310 calculates the operator's cognitive proficiency for the category to which the object is classified based on whether the operator moves their face toward the object before moving their gaze toward it when visually identifying it. This is because, typically, when an operator intends to plan visually identify the presence of traffic participants based on some degree of prediction, they tend to move their face toward the object before moving their gaze toward the object itself, where the traffic participants are likely to be.

[0085] Furthermore, the cognitive skill evaluation unit 310 calculates the cognitive proficiency of the category of the object being used for proficiency calculation based on whether the object being used for proficiency calculation is a physical object that the gaze passes through when the gaze moves to other objects, and whether the movement of the gaze is accompanied by a pre-existing facial movement.

[0086] In the above, the behavior detection unit 306 can determine whether the operator has visually identified the object based on whether the operator's gaze has lingered on the object for a specified time or more.

[0087] More specifically, the cognitive skills assessment unit 310 calculates the cognitive proficiency for the category of the object as follows.

[0088] a) For the category of objects whose gaze lingers for a specified time or more after the preceding facial orientation movement, the cognitive proficiency is set to 1.

[0089] b) For the category of the object that the gaze passes through during the subsequent facial orientation movement, set the cognitive proficiency to 2.

[0090] c) For categories of objects whose gaze lingers for a specified time or more after a gaze movement without prior facial orientation movement, the cognitive proficiency is set to 3.

[0091] d) For the category of objects that the gaze passes through during a gaze movement that does not accompany a prior facial orientation movement, set the cognitive proficiency to 4.

[0092] e) For categories of objects that are neither stopped in sight nor passed by sight, set the proficiency level to 5.

[0093] The attention reminder unit 304, like the attention reminder unit 133, controls the front projection device 122 to direct light towards... Figure 1 The vehicle 100 shown outputs a visual display VI on the display range HA of the windshield 105. However, unlike the attention reminder unit 133, the attention reminder unit 304, when outputting the visual display VI, refers to the cognitive proficiency information 312 stored in the storage device 303. Based on the cognitive proficiency information 312, it displays the visual display VI indicating the directional position of each detected object in a manner corresponding to the operator's cognitive proficiency calculated for the category of the object on the display range HA of the windshield 105.

[0094] Here, displaying a visual display VI in a manner corresponding to the operator's cognitive proficiency means, for example, displaying a visual display VI as a projection light according to the color, brightness, and / or size corresponding to the operator's cognitive proficiency.

[0095] Figure 6 This is an example of cognitive proficiency information 312 stored in storage device 303 by the cognitive skill evaluation unit 310. Figure 6 The cognitive proficiency information of an operator is displayed in tabular form. The cognitive skill assessment unit 310, according to each operator, [details omitted]. Figure 6 The cognitive proficiency information shown is stored in storage device 303. Figure 6 In the table shown, the columns from left to right are named column 1, column 2, ..., column 6. Similarly, the rows from top to bottom are named row 1, row 2, ...

[0096] Figure 6 The cognitive proficiency information shown in the first column indicates the category used when classifying objects in the classification section 308. Figure 6 The categories shown in the first column are recorded in descriptive form, but can also be classified using combinations of coded symbols that distinguish between the presence or absence of obstructions, whether it is an intersecting road or the road of the vehicle, and whether it is a pedestrian or a vehicle, in accordance with existing technology.

[0097] Figure 6Columns 2 through 6 show the cognitive proficiency calculated by the cognitive skill evaluation unit 310 for the five most recent measurements of objects categorized into various classes. That is, Figure 6 The table in rows 3, columns 2 through 6 shows the operator’s cognitive proficiency calculated for each of the last 5 encounters with vehicles classified as “vehicles appearing from unobstructed intersections”.

[0098] Figure 7 The attention reminder section 304 is based on Figure 6 This is an example of a visual display VI on the front glass 105 that shows cognitive proficiency information. Figure 7 The traffic scene in front of vehicle 100 shown is similar to Figure 3 The traffic scenario shown is the same. Here, classification unit 308 classifies vehicle 201 as "vehicles emerging from an intersection with obstructions" and vehicle 204 as "vehicles traveling straight in the opposite lane of their own road at an intersection". In addition, classification unit 308 classifies motorcycle 202 and pedestrian 203 as "vehicles emerging from an intersection without obstructions" and "pedestrians crossing their own road at an intersection", respectively.

[0099] When the attention reminder unit 304 displays the visual display VI for vehicles 201, 204, motorcycles 202, and pedestrians 203, which are objects detected by the detection unit 132, on the display range HA of the windshield 105, it first refers to the data stored in the storage device 303. Figure 6 The cognitive proficiency information shown is 312.

[0100] Then, the attention reminder unit 304 displays the visual display VI for the object classified into that category in a more conspicuous manner, based on the cognitive proficiency of each category shown in the cognitive proficiency information 312, for example, as the cognitive proficiency increases from 1 to 5. Here, "displaying in a more conspicuous manner" means, for example, that the hue of the visual display VI changes from cool colors to warm colors, becoming more eye-catching, as the cognitive proficiency increases from 1 to 5. However, the way the visual display VI changes according to cognitive proficiency is not limited to hue; it can also be brightness (tone), flicker speed, size, etc. In this embodiment, for example, as the cognitive proficiency value increases level by level from 1 to 5, the visual display VI is displayed in blue, green, yellow, orange, and red.

[0101] exist Figure 6 In the example shown, the most frequent value of the cognitive proficiency for the operator's category "vehicles appearing from an intersection with obstructions" in the last 5 instances is 4. Therefore, the attention reminder unit 304... Figure 7Within the display range HA of the windshield 105 shown, a visual display VI-21 for vehicles 201 classified into that category is displayed in orange. Furthermore, the most frequent cognitive proficiency value for the category "vehicles appearing from unobstructed intersections" is 3, therefore, the attention alert unit 304... Figure 7 Within the display area HA of the front glass 105 shown, the visual display VI-22 for the motorcycle 202 classified as belonging to this category is displayed in yellow.

[0102] Similarly, the most frequent cognitive proficiency value for the category "pedestrians crossing the road at intersections" in the last 5 instances is 2. Therefore, attention reminder 304 is required. Figure 7 Within the display range HA of the windshield 105 shown, a visual display VI-23 for pedestrians 203 classified as belonging to this category is displayed in green. Furthermore, the most frequent value of the cognitive proficiency for the category "vehicles traveling straight in the oncoming lane at an intersection" is 1 in the most recent 5 readings; therefore, the attention alert unit 304 activates the visual display VI-23 in green. Figure 7 Within the display area HA of the front windshield 105 shown, a visual display VI-24 for vehicles 204 classified as belonging to this category is displayed in blue.

[0103] In addition, in the above, when selecting the display color as the display method for the visual display VI, the most frequent value of the cognitive proficiency of the last 5 times for each category was used, but it is not limited to this. For example, the value obtained by rounding the average of the 5 times to the nearest integer, or the minimum or maximum value of the last 5 times can also be used.

[0104] Like the attention alert system 120, the attention alert system 300, with the aforementioned structure, displays a visual display VI at a horizontal position corresponding to the object within a strip-shaped display area HA on the windshield 105. Therefore, like the attention alert system 120, the attention alert system 300 can alert the operator to various traffic participants present in the surrounding environment of the vehicle 100 without distracting the operator.

[0105] Furthermore, in the attention alert system 300, the visual display VI, particularly for objects, is displayed in a manner corresponding to the operator's cognitive proficiency regarding the category of that object. Therefore, in the attention alert system 300, the operator can visually identify each object with a priority corresponding to their cognitive proficiency through the displayed visual display VI, thus alerting the operator to various traffic participants present in the surrounding environment of the vehicle 100 in a more effective way for the operator's driving actions.

[0106] Furthermore, in the attention reminder system 300, for the operator, the cognitive proficiency of each category of the object is calculated based on the presence or absence of facial movements before the planned gaze movement indicating cognition. Therefore, cognitive proficiency can be calculated with simple processing.

[0107] Furthermore, in the attention alert system 300, the proficiency in recognizing the category of an object (i.e., the object passed by the eye) is determined based on whether the eye movement accompanies a planned visual confirmation of a specific object. Therefore, in the attention alert system 300, even for objects that are not visual confirmation objects, the proficiency in recognizing them can be assessed based on the trend of the operator's eye movement path and reflected in the display method of the visual display.

[0108] Figure 8 This is a flowchart illustrating the steps of the actions in the attention reminder system 300. Figure 5 When the detection unit 132 detects at least one object, it begins to operate according to each detected object. Figure 8 The processing shown.

[0109] When processing begins, firstly, the classification unit 308 classifies the detected objects into categories (S200). Next, the attention reminder unit 304 refers to the cognitive proficiency information 312 stored in the storage device 303 (S202) and displays a visual display VI at the horizontal position corresponding to the object within the display range HA in a manner corresponding to the cognitive proficiency (S204).

[0110] Next, the attention reminder unit 304 determines whether the object is no longer detected by the detection unit 132 (S206). Then, if the object is detected again (S206, "No"), the attention reminder unit 304 returns to step S204 and repeats the process. On the other hand, if the object is no longer detected (S206, "Yes"), the cognitive skill evaluation unit 310 calculates the operator's cognitive proficiency for the object based on the operator's behavior during the period up to now, that is, the operator's eye movement and facial orientation movement detected by the behavior detection unit 306 (S208).

[0111] Next, the cognitive skill evaluation unit 310 stores the calculated cognitive proficiency in association with the category of the object as cognitive proficiency information in the storage device 303. Specifically, for example, the cognitive skill evaluation unit 310 adds the calculated cognitive proficiency to the cognitive proficiency information 312 stored in the storage device 303, updating the cognitive proficiency information 312 (S210). After the cognitive skill evaluation unit 310 updates the cognitive proficiency information 312, the processing device 302 ends the processing.

[0112] [Third Implementation]

[0113] Next, the third embodiment of the present invention will be described. Figure 9 This is a diagram illustrating the structure of the attention reminder system 400 according to the third embodiment of the present invention. This attention reminder system 400, instead of the attention reminder system 120, is mounted on the system described in the first embodiment. Figure 1 The vehicle shown is 100. Additionally, in... Figure 9 In China, use and Figure 2 and Figure 5 The same label shown is used to indicate the same label as the one shown. Figure 2 and Figure 5 The structural elements shown are the same as those described above. Figure 2 and Figure 5 Explanation.

[0114] The attention reminder system 400 consists of a front projection device 122 and an attention reminder device 424. The front projection device 122 outputs a visual display on the front glass 105 that can be visually confirmed by the operator. The attention reminder device 424 controls the front projection device 122 and reminds the operator to pay attention to the object through the aforementioned visual display.

[0115] The attention reminder device 424 has the same as Figure 5 The attention reminder device 324 shown has the same structure, but it differs in that it has a processing device 402 and a storage device 403 instead of a processing device 302 and a storage device 303. The storage device 403 has the same structure as the storage device 303, and the reaction proficiency information 414 is stored by the reaction skill evaluation unit 410 of the processing device 402, which will be described later.

[0116] Processing device 402 has the same structure as processing device 302, but differs in that it has an attention reminder unit 404 and a behavior detection unit 406 as functional elements or units instead of an attention reminder unit 304 and a behavior detection unit 306. Furthermore, processing device 402 differs from processing device 302 in that it does not have a cognitive skill evaluation unit 310 but instead has a reaction skill evaluation unit 410 and a risk calculation unit 412.

[0117] The processing device 402 possesses the same functional elements as the processing device 302, for example, by executing a program through the processing device 402, which is a computer. Furthermore, the aforementioned computer program can be pre-stored in any computer-readable storage medium. Alternatively, all or part of the aforementioned functional elements of the processing device 402 can be constructed from hardware comprising one or more electronic circuit components.

[0118] The risk calculation unit 412 calculates a risk level, representing the degree of risk of contact with the vehicle 100, for each object detected by the detection unit 132 based on the category classified by the classification unit 308. This risk level differs from the contact probability calculated by the detection unit 132 for each detected object based on at least the relative speed with the vehicle 100. For example, actual accidents are classified into the aforementioned categories, and the ratio of the number of occurrences in each category to the total number of accidents is compared with a predetermined threshold range and correlated with a multi-stage evaluation value. In this embodiment, the risk level is evaluated in 10 stages, with risk level 1 having the lowest accident occurrence rate and risk level 10 having the highest accident occurrence probability.

[0119] The risk calculation unit 412 can, for example, calculate the risk level of each detected object by referring to risk information representing the risk level of each category that is pre-stored in the storage device 403. Alternatively, the risk calculation unit 412 can use a risk calculation model that has been learned through machine learning to calculate the risk level of the category of the object being classified as the object of risk calculation and the probability of an accident of that object.

[0120] Like behavior detection unit 306, behavior detection unit 406 uses indoor camera 144 to detect the operator's eye movement and facial movement, and to determine whether the operator has visually identified the object. Specifically, behavior detection unit 406 determines that the operator has visually identified the object if the operator's gaze lingers on the object for more than a predetermined time.

[0121] The behavior detection unit 406 also measures the actual reaction time from the moment the operator visually confirms the object detected by the detection unit 132 until a certain maneuvering operation is initiated on the vehicle 100. In this embodiment, the behavior detection unit 406 detects the operation of the brake pedal or accelerator pedal as the aforementioned maneuvering operation using the brake pedal sensor 146 and / or the accelerator pedal sensor 148.

[0122] The reaction skill evaluation unit 410 calculates the reaction delay time for each object detected by the detection unit 132 by subtracting the predetermined standard reaction time for the category to which the object is classified from the actual reaction time measured by the behavior detection unit 406. Then, based on the calculated reaction delay time, the reaction skill evaluation unit 410 calculates the operator's reaction proficiency for that category and stores the reaction proficiency for each category as reaction proficiency information 414 in the storage device 403.

[0123] For example, reaction proficiency is evaluated in 5 stages. The smaller the reaction delay time (including negative numbers), the smaller the reaction proficiency value. For example, the reaction skill evaluation unit 410 can use a set of thresholds predetermined for reaction delay time to correspond the calculated reaction delay time with any one of the reaction proficiency values ​​between 0 and 4.

[0124] Note: Reminder 404 and Figure 2 Similarly, the attention reminder unit 133 controls the forward projection device 122 to output a visual display VI to a horizontally extending strip-shaped display area HA on the windshield 105 of the vehicle 100. However, the attention reminder unit 404 differs from the attention reminder unit 133 in that, when displaying the visual display VI indicating the directional position of each object detected by the detection unit 132 on the display area HA of the windshield 105, the visual display VI is displayed in a manner that corresponds at least to the risk level calculated by the risk calculation unit 412 for that object.

[0125] For example, the closer the risk level of a category is to 10, the more prominently the attention reminder unit 404 displays the visual display VI for the object classified into that category. Here, "displaying in a more prominent manner" means, for example, that as the risk level approaches 10, the hue of the visual display VI changes from cool colors to warm colors, becoming more eye-catching. However, the way the visual display VI changes according to cognitive proficiency is not limited to hue; it can also be brightness (tone), flashing speed, size, etc.

[0126] Specifically, for example, the attention alert unit 404 uses 10 color codes from 1 to 10 to represent 10 colors ranging from cool to warm colors. Then, the attention alert unit 404 at least maps the risk level value from 1 to 10 calculated for the category of the detected object to the color code with the same value, and displays the visual display VI for that object using the color corresponding to the mapped color code. The relationship between the color code and the color is defined, for example, that as the color code increases sequentially from 1 to 10, the displayed color of the visual display VI becomes purple, blue-violet, blue, blue-green, green, yellow-green, yellow, yellow-orange, orange, and red.

[0127] Therefore, for objects with higher risk levels, more conspicuous visual displays (VIs) are provided, thus guiding the operator's gaze to the objects with higher risk levels.

[0128] In this embodiment, the attention reminder unit 404 considers the operator's reaction skills for each type of object in addition to the aforementioned risk level when determining the display method of the visual display VI. Specifically, when displaying the visual display VI for the object in the display range HA, the attention reminder unit 404 refers to the operator's reaction proficiency information 414 stored in the storage device 403 by the reaction skill evaluation unit 410.

[0129] Then, the attention reminder unit 404 adds the operator's reaction proficiency (as described above, in this embodiment, a value of 0 to 4) for the category of the object, as shown in the reaction proficiency information, to the aforementioned risk level to calculate the danger level. Then, the attention reminder unit 404 displays a visual display VI for the object based on a color code with the same value as the calculated danger level. Furthermore, for objects with a calculated danger level exceeding 10, a visual display VI for that object can be displayed using a color corresponding to the maximum value 10 of the color code.

[0130] Furthermore, after the attention reminder unit 404 displays the visual display VI for each detected object in the display mode determined as described above (display color in this embodiment), when it is determined that the operator has not visually recognized the object, the level of prominence of the visual display VI for that object increases along with time.

[0131] Specifically, for example, the attention reminder unit 404 measures the elapsed time after the visual display VI for the object is displayed. Every predetermined time (e.g., 1 second), the visual display VI is changed to a color with a higher color code than the current one, thereby increasing the visibility of the visual display VI. The increase in the visibility level of the visual display VI stops when the color code increases to the maximum value of 10, or when the behavior detection unit 406 detects that the operator has visually confirmed the object.

[0132] Figure 10 This illustrates an example of a visual display VI shown on the front glass 105 by the attention reminder unit 404. Figure 10 The traffic scene in front of vehicle 100 and the classification of various objects shown are as follows: Figure 7 The examples are the same. Figure 10 This shows the state immediately after the visual displays (VIs) for each object have begun. At this point, the colors of the visual displays (VIs) for each object—specifically, the colors of VI-31, VI-32, VI-33, and VI-34 for vehicle 201, motorcycle 202, pedestrian 203, and vehicle 204—are determined based on the risk level of the categories in which vehicle 201, motorcycle 202, pedestrian 203, and vehicle 204 are classified, as described above. Figure 10In the example shown, the risk level of the categories for which vehicles 201, motorcycles 202, pedestrians 203, and vehicles 204 are classified is added to the reaction proficiency to obtain the danger level, for example, 9, 4, 6, 6. Correspondingly, visual displays VI-31, VI-32, VI-33, and VI-34 are displayed in orange, blue-green, yellow-green, and yellow-green.

[0133] Furthermore, the colors of these visual displays VI-31, VI-32, VI-33, and VI-34 change in tandem with time until the operator visually identifies the vehicle 201, motorcycle 202, pedestrian 203, and vehicle 204 corresponding to these visual displays. As an example, Figure 11 This diagram illustrates the change in the display mode of the visual display VI-34 for vehicle 204 when time has elapsed without the operator visually confirming the vehicle 204.

[0134] exist Figure 11 In the diagram, from left to right, are shown the visual display VI-34 at the start of the display within the display area HA, and the visual displays VI-34 1 second, 2 seconds, 3 seconds, and 4 seconds after the start of the display. Figure 11 Three seconds after the display begins, the operator visually confirms vehicle 204, which corresponds to visual display VI-34.

[0135] Since the danger level of vehicle 204 is 6 (as described above), the initial color code of visual display VI-34 is 6 (yellow-green). Then, the attention alert unit 404 changes the color code of visual display VI-34 to a higher level every second: after 1 second, it changes to color code 7 (yellow); after 2 seconds, it changes to color code 8 (yellow-orange); and after 3 seconds, it changes to color code 9 (orange). Then, after 3 seconds, the attention alert unit 404, in accordance with the operator's visual confirmation of vehicle 204, stops increasing the color code of visual display VI-34. Thus, after 4 seconds, visual display VI-34 continues to display based on color code 9 (orange).

[0136] Like the attention alert system 120, the attention alert system 400, with the aforementioned structure, displays a visual display VI at a horizontal position corresponding to the object within a strip-shaped display area HA on the windshield 105. Therefore, like the attention alert system 120, the attention alert system 400 can alert the operator to various traffic participants present in the surrounding environment of the vehicle 100 without distracting the operator.

[0137] Furthermore, in the attention alert system 400, in particular, the visual display VI for objects is displayed at least in a manner corresponding to the risk level of the object's category. Therefore, in the attention alert system 300, the operator can, for example, visually identify each object with a priority corresponding to the degree of contact risk with the vehicle 100, and the operator can be alerted to various objects in the surrounding environment of the vehicle 100 in a more effective way for the operator's driving actions.

[0138] Furthermore, in the attention alert system 400, the display method of the visual display VI for the object is determined based on the operator's reaction proficiency for that object category. Therefore, in the attention alert system 300, for objects where the operator's actual reaction time from visual confirmation to operation is slower than the standard reaction time, a more conspicuous visual display VI can be displayed. Thus, the operator's attention to various objects in the surrounding environment of the vehicle 100 can be alerted with a priority corresponding to the speed of the operator's reaction action.

[0139] Furthermore, in the attention alert system 400, when the attention alert unit 404 determines that the operator has not visually identified the object, it increases the level of conspicuousness of the visual display VI for that object over time (in this embodiment, for example, by changing the display color to a warmer color). Thus, the attention alert system 400 prevents a prolonged period from elapsed when the operator has not visually identified the object. As a result, in the attention alert system of structure 2, delays in the operator's response to the appearing object can be avoided.

[0140] Furthermore, in the attention reminder system 400, if the operator's gaze lingers on an object for more than a specified time, it is determined that the operator has visually confirmed the object. Therefore, it is possible to determine whether there is visual confirmation of the object with a simple process.

[0141] Figure 12 This is a flowchart illustrating the steps of the actions in the attention reminder system 400. Figure 9 When the detection unit 132 detects at least one object, it begins to operate according to each detected object. Figure 12 The processing shown.

[0142] When processing begins, firstly, the classification unit 308 classifies the detected objects into categories (S300). Furthermore, the risk calculation unit 412 calculates the risk level based on the category into which the object is classified (S302).

[0143] Furthermore, the attention reminder unit 404 reads the reaction proficiency information 414 stored in the storage device 403 to determine the reaction proficiency level for the category of the object (S304). Next, the attention reminder unit 404 sets the display mode of the visual display VI for the object based on the risk level and reaction proficiency level (S306). Then, the attention reminder unit 404 displays the visual display VI at the horizontal position corresponding to the object within the display range HA, using the aforementioned display mode (S308).

[0144] Next, the attention reminder unit 404 determines whether the object is no longer detected by the detection unit 132 (S310). Then, when the object is no longer detected (S310, "Yes"), the attention reminder unit 404 ends the process.

[0145] On the other hand, when the object is detected again (S310, "No"), the attention reminder unit 404 determines whether the object has been visually confirmed (S312). Then, when the object has been visually confirmed (S312, "Yes"), the attention reminder unit 404 does not change the display mode setting of the visual display VI and returns to step S308 to repeat the process.

[0146] On the other hand, when the object is still not visually identified (S312, "No"), the attention reminder unit 404 determines whether the operator has visually identified the object based on the operator's gaze movement detected by the behavior detection unit 406 (S314). Then, when the operator has not visually identified the object (S314, "No"), the attention reminder unit 404 sets the display mode of the visual display VI to a display mode with a higher visibility level than the current display mode (S316), and then returns to step S308 to repeat the process. As described above, in this embodiment, for example, by changing the display color used for the visual display VI to a color closer to a warm color (e.g., red), the visibility level of the display mode of the visual display VI is increased by 1 level.

[0147] On the other hand, when the operator visually confirms the object in step S314 (S314, "Yes"), the reaction skill evaluation unit 410 obtains the actual reaction time measured by the behavior detection unit 406 (S318). Next, the reaction skill evaluation unit 410 calculates the reaction proficiency for the object based on the obtained actual reaction time, associates the calculated reaction proficiency with the category of the object, and adds the reaction proficiency information 414 stored in the storage device 403 (S320). After that, the processing device 402 does not change the display mode setting of the visual display VI and returns to step S308 to reprocess.

[0148] [Fourth Implementation]

[0149] Next, the fourth embodiment of the present invention will be described. Figure 13 This diagram illustrates the structure of the attention alert system 500 according to the fourth embodiment of the present invention. In the attention alert system 120 of the first embodiment described above, the object guiding the operator's gaze is an object existing in front of the vehicle 100. In contrast, the attention alert system 500 of this embodiment considers all directions of 360 degrees centered on the vehicle 100 as objects, notifying the operator of the existence of objects around the vehicle 100.

[0150] In addition, Figure 13 In China, use and Figure 2 The same label as the one in the text is shown. Figure 2 The structural elements shown are the same as those described above. Figure 2 Explanation.

[0151] The attention reminder system 500 includes a front projection device 122, left and right projection devices 504 and a rear projection device 506 that output visual displays on the interior structure of the vehicle 100 for visual confirmation by the operator of the vehicle 100, and an attention reminder device 502 that controls these projection devices and reminds the operator to pay attention to objects around the vehicle 100 through the aforementioned visual displays.

[0152] Figure 14 This is a diagram showing an example of the configuration of a projection device inside the vehicle 100. Additionally, in Figure 14 In China, use and Figure 1 The same label as the one in the text is shown. Figure 1 The structural elements shown are the same as those described above. Figure 1 Explanation.

[0153] A forward-facing projector 122 is installed on the dashboard 106 at the front of the vehicle compartment. The forward-facing projector 122 is a reference... Figure 1 The described front projection device 122 displays a visual display VI within the display range HA of the windshield 105, which is an interior structure. A HUD (Head-Up Display) 156, which displays on the windshield 105 according to the prior art, is also provided on the instrument panel 106.

[0154] The left and right projection devices 504 and the rear projection device 506 are respectively disposed on the inner front (front of the vehicle interior) and inner rear of the roof of the vehicle 100. The left and right projection devices 504 and the rear projection device 506 are, for example, projectors. The left and right projection devices 504 and the rear projection device 506 project visual displays within a defined display range of the left and right interior structures located in the vehicle interior and the rear interior structure located in the vehicle interior, respectively. In this embodiment, the interior structures projected by the left and right projection devices 504 are the left door glass 158 and the right door glass 160. Furthermore, the interior structures projected by the rear projection device 506 are, for example, the rear window 162. Additionally, when the vehicle 100 has front triangular window glass on the left and right sides, these front triangular window glass can also be included in the interior structures for which the left and right projection devices 504 output visual displays.

[0155] The aforementioned specified display range on the interior structure for the left and right projection devices 504 and the rear projection device 506 to display visual displays, for example, with Figure 1 Similarly, the display range HA on the windshield 105 in the front projection device 122 shown is a strip-shaped range extending in a horizontal direction orthogonal to the height direction of the vehicle 100.

[0156] The visual displays projected onto the interior structure by the left and right projection devices 504 and the rear projection device 506 are also projections of a predetermined size, similar to the visual display VI displayed on the front glass 105 by the front projection device 122. Hereinafter, the visual displays projected onto the interior structure by the left and right projection devices 504 and the rear projection device 506 are also referred to as visual display VI, similar to the visual display projected onto the front glass 105 by the front projection device 122.

[0157] Reference Figure 13 The attention reminder device 502 includes a processing unit 508 and a storage unit 510. The storage unit 510 may be, for example, a volatile and / or non-volatile semiconductor memory, and / or a hard disk drive. The processing unit 508 may be, for example, a computer equipped with a processor such as a CPU. The processing unit 508 may also have a structure with a ROM for writing programs, RAM for temporarily storing data, etc. Furthermore, the processing unit 508 includes a detection unit 512 and an attention reminder unit 514 as functional elements or units.

[0158] These functional elements of the processing device 508 are implemented, for example, by executing a program by the processing device 508, which is a computer. Furthermore, the aforementioned computer program can be pre-stored in any computer-readable storage medium. Alternatively, all or part of the aforementioned functional elements of the processing device 508 can be constructed from hardware comprising one or more electronic circuit components.

[0159] The detection unit 512 detects objects located in the area surrounding the vehicle 100. Specifically, the detection unit 512 detects objects located in the front, rear, right, and left areas of the vehicle 100, respectively, using a front object sensor 140, a rear object sensor 150, a right-side object sensor 152, and a left-side object sensor 154 installed on the vehicle 100. The rear object sensor 150, right-side object sensor 152, and left-side object sensor 154 are, like the front object sensor 140, for example, cameras, lasers, and / or lidar.

[0160] The detection unit 512 acquires images of the front, rear, right, and left areas of the vehicle 100 from the front object sensor 140, rear object sensor 150, right object sensor 152, and left object sensor 154, respectively. Based on these images, the detection unit 512 detects designated objects, such as objects whose probability of contact with the vehicle 100, calculated based on their relative speed to the vehicle 100, is a predetermined value or higher. In this embodiment, similar to the first embodiment, the designated objects include traffic participants such as vehicles and pedestrians, as well as obstacles. The detection unit 512 sends the location information of each detected object to the attention alert unit 514.

[0161] Here, the front object sensor 140, the rear object sensor 150, the right-side object sensor 152, and the left-side object sensor 154 can be installed, for example, on the front bumper, the rear bumper, the right rearview mirror body, and the left rearview mirror body of the vehicle 100.

[0162] The attention reminder unit 514 notifies the operator of the direction of objects detected by the detection unit 512 that exist in the area surrounding the vehicle 100 via a visual display. Specifically, the attention reminder unit 514 controls the front projection device 122, the left and right projection devices 504, and the rear projection device 506 to output a visual display VI at a horizontal position on the interior structure of the vehicle 100, within a predetermined display range, that corresponds to the direction of the object observed from the operator's position.

[0163] More specifically, the attention reminder unit 514 divides the area around the vehicle 100 into several angular regions, and outputs the aforementioned visual display VI through the front projection device 122, the left and right projection devices 504 or the rear projection device 506, depending on which region the object detected by the detection unit 512 exists in.

[0164] Figure 15 This is an example of an angular region centered on vehicle 100. Additionally, in Figure 15 In China, use Figure 1 The labels in the text are shown as... Figure 1 The structural elements shown are the same as those described above. Figure 1 Explanation.

[0165] exist Figure 15 In the example shown, the central frontal area (CFA) is defined, allowing the operator seated in vehicle 100 to naturally make visual confirmations with their face facing forward. The left frontal area (LFA) and right frontal area (RFA) are also defined, allowing the operator to make visual confirmations by turning their face approximately 45 degrees to the left or right. Furthermore, in Figure 15 In the example shown, the left side region LA and right side region RA of the vehicle 100 are defined so that the operator can visually confirm it by slightly twisting the body to the left or right.

[0166] In addition, Figure 15 In the example, a rear area REA is defined that the operator can visually confirm via the interior rearview mirror 110, the left rearview mirror 112, and the right rearview mirror 114. The rear area REA is further divided into the interior rearview mirror area RMA, which the operator can visually confirm via the interior rearview mirror 110; the left rearview mirror area LFMA, which the operator can visually confirm via the left rearview mirror 112; and the right rearview mirror area RFMA, which the operator can visually confirm via the right rearview mirror 114.

[0167] Figure 15 The remaining areas shown are the left blind spot (LBA) and the right blind spot (RBA), which are areas where the operator cannot visually confirm the object.

[0168] The attention reminder unit 514, similar to the attention reminder unit 133 in the first embodiment, displays a visual display VI indicating the direction of the object on the display range HA of the front glass 105 via the front light projection device 122 for objects detected by the detection unit 512 in the central front region CFA, left front side region LFA, and right front side region RFA.

[0169] In addition, the attention reminder unit 514 displays the direction of the object detected by the detection unit 512 in the left side area LA and the right side area RA on the left door glass 158 and the right door glass 160 respectively by the left and right projection devices 504.

[0170] In addition, the attention reminder unit 514 displays a visual display VI indicating the direction of the object detected by the detection unit 512 in the rear area REA on the rear glass 162 through the rear projection device 506.

[0171] Therefore, the attention reminder unit 514 can take the area around the vehicle 100, excluding the left blind spot LBA and right blind spot RBA, which are areas where the operator cannot visually confirm objects, as the object, and inform the operator of the direction of the object that may come into contact with the vehicle 100, remind the operator of the object, and guide the operator's line of sight.

[0172] The attention reminder unit 514 also displays a circular display on the display device indicating the direction relative to the vehicle 100. Then, the attention reminder unit 514 outputs a visual display on the arc portion of the circular display corresponding to the direction of the object. In this embodiment, the display device is a HUD 156.

[0173] Specifically, the attention alert unit 514 considers the entire area around the vehicle 100, including the left blind spot area (LBA) and the right blind spot area (RBA), as the target, and displays a radar display indicating the direction of the target detected by the detection unit 512 on the HUD 156. Hereinafter, the radar display shown by the attention alert unit 514 on the HUD 156 to indicate the direction of the target will be referred to as the target radar display.

[0174] Figure 16 This diagram illustrates an example of an object radar display shown by the attention alert unit 514 on the HUD 156. The object radar display 520 consists of a vehicle icon 522 representing the vehicle 100 and a circular display 524 representing a full 360-degree circumference centered on the vehicle 100. The attention alert unit 514 outputs a visual display corresponding to the direction of the object at the arc of the circular display 524. Specifically, the attention alert unit 514 performs a visual display AD on the arc portion of the circular display 524 corresponding to a certain direction of the object detected by the detection unit 512. Figure 16 The upward-pointing arrow 526 shown in the circular display 524 indicates the straight-line direction of the vehicle 100. Arrow 526 is shown for illustration only in this embodiment, but it may also form part of the object radar display 520.

[0175] exist Figure 16 In the example shown, a visual display AD is shown in the arc portion of the circular display 524 extending approximately 30 degrees to the right from the straight direction of the vehicle icon 522 (in the direction indicated by arrow 526). This visual display AD shows that there is an object that may come into contact with the vehicle 100 30 degrees to the right in front of the vehicle 100.

[0176] Figure 17 This is a diagram showing the various display modes in the object radar display 520 based on the attention reminder unit 514 in tabular form. Figure 17The first layer (top layer) of the table shown is an example of displaying multiple detected objects independently using the same visual display method. In the illustrated example, visual displays AD-10 and AD-12, displayed by circle display 524, show the presence of objects near the right 30 degrees and the left 15 degrees of the vehicle 100, respectively.

[0177] Figure 17 The example of the second layer of the table shown is similar to the first layer, independently displaying each visual display for multiple objects, but displaying each visual display in a manner corresponding to the risk level of the corresponding object, for example, in a color corresponding to the probability of contact with vehicle 100. In the illustrated example, for example, visual display AD-14 is displayed in green, and visual display AD-16 is displayed in yellow. Furthermore, the probability of contact with vehicle 100 can be calculated, for example, based on the relative speed between the object detected by detection unit 512 and vehicle 100. The attention alert unit 514 can determine the display color of the visual display for the detected object based on predetermined correspondence information representing the correspondence between the range of the calculated relative speed and the display color used for the visual display.

[0178] Figure 17 The example of layer 3 in the table shown is similar to layer 2 in that it independently displays visual displays of multiple objects in a manner corresponding to their risk levels. However, unlike the example of layer 2, in the example of layer 3, the risk level is represented by the size of the visual display, rather than by the color of the visual display. In the example figure, for example, visual displays AD-18 and AD-20 are displayed by strip-shaped arcs with central angles of approximately 10 degrees and 20 degrees, respectively. Thus, it can be seen that the object shown by the larger visual display AD-20 has a higher probability of contact than the object shown by the smaller visual display AD-18. In addition, the probability of contact with vehicle 100 can be calculated, for example, based on the relative speed between the object detected by detection unit 512 and vehicle 100. The attention reminder unit 514 can determine the size of the visual display for the detected object based on predetermined correspondence information representing the correspondence between the range of the calculated contact probability and the central angle of the strip-shaped arc used for the visual display.

[0179] Figure 17The example of the fourth layer (bottom layer) of the table shown is an example of displaying a visual display around the entire circumference of the circular display 524 when a predetermined number or more objects are detected around the vehicle 100. In the example shown, for example, the visual display AD-22 is displayed extending along the entire circumference of the circular display 524, thereby notifying that three or more objects, as a predetermined number, have been detected around the vehicle 100. Furthermore, the predetermined number is not limited to three and can be any number. Additionally, the visual display AD-22 can also be displayed in a color corresponding to the highest contact probability among the detected multiple objects.

[0180] Like the attention alert system 120, the attention alert system 500, with the aforementioned structure, displays a visual display VI at a horizontal position corresponding to the object within a strip-shaped display area HA on the windshield 105. Therefore, like the attention alert system 120, the attention alert system 500 can alert the operator to various traffic participants present in the surrounding environment of the vehicle 100 without distracting the operator.

[0181] Furthermore, in the attention alert system 500, particularly for objects located to the side and rear of the vehicle 100, a visual display VI indicating the direction of the object as observed by the operator is displayed within a display area extending horizontally along the interior structure. Therefore, the attention alert system 500 can alert the operator to the direction of objects in the entire surrounding area of ​​the vehicle 100, which can be visually confirmed by the operator.

[0182] Furthermore, in the attention alert system 500, the attention alert unit 514, in addition to visual display, generates sound for directional positioning of objects for the operator via an interior speaker 142 installed in the vehicle cabin. Thus, the attention alert system 500 can alert the operator to objects from both visual and auditory perspectives.

[0183] Furthermore, the front projection device 122, the left and right projection devices 504, and the rear projection device 506 output visual displays (VIs) by projecting light of a predetermined shape onto interior structures such as the front glass 105. Thus, in the attention alert system 500, visual displays can be presented in a manner intuitively understandable to the operator.

[0184] Furthermore, in the attention alert system 500, the interior structures for displaying the visual display VI on the front projection device 122, left and right projection devices 504, and rear projection device 506 are the front windshield 105, left door glass 158, right door glass 160, and / or rear glass 162 of the vehicle 100. If the vehicle 100 has a front quarter window, it can also be the front quarter window. That is, in the attention alert system 500, visual displays are output on these interior structures that allow visual confirmation of the exterior of the vehicle 100 through the interior structures. Therefore, the operator can immediately visually confirm the object while being alerted to its presence.

[0185] Furthermore, in the attention alert system 500, the forward projection device 122 outputs a visual display VI by projecting a light of a predetermined shape onto the windshield 105. Thus, according to the attention alert system 500, at least on the windshield 105, a visual display indicating the direction of an object is displayed, overlapping with the scene in front of the vehicle that the operator needs to pay the most attention to. Therefore, the operator's attention can be effectively alerted to objects that require special attention when operating the vehicle 100.

[0186] Furthermore, the forward projection device 122 that outputs the visual display VI on the windshield 105 is composed of multiple light sources arranged horizontally (in the vehicle width direction) below the windshield 105. Thus, a forward projection device that outputs the visual display VI on the windshield 105 can be implemented with a simple structure.

[0187] Furthermore, in the attention alert system 500, the object radar display 520 in the HUD 156, which is displayed as a circle, notifies the operator of the presence of objects covering the entire circumference of the vehicle 100, including blind spots for the operator. Therefore, in the attention alert system 500, the operator can easily grasp the presence of objects throughout the entire surrounding area of ​​the vehicle 100 via the circle display, and can easily grasp the overall situation of the area surrounding the vehicle 100.

[0188] Furthermore, the object radar display 520 displays a visual display AD indicating the direction of the object in a manner corresponding to the probability of contact between the object and the vehicle 100. Therefore, the operator can visually identify each object with a priority corresponding to the degree of contact risk, and can alert the operator to various objects in the surrounding environment of the moving body in a more effective way for the operator's driving actions.

[0189] Furthermore, when the number of objects exceeds a predetermined number, the object radar display 520 displays a visual display AD extending along the circumference of the circular display 524, thus preventing the operator from being distracted by each and every single object. Therefore, in the attention alert system 500, the operator can be aware of the presence of objects around the vehicle 100 without being distracted.

[0190] Furthermore, in the attention alert system 500, the object is a physical object whose probability of contact with the vehicle 100 is above a predetermined value. Therefore, the attention alert system 500 filters out physical objects that may pose a risk to driving the vehicle 100 and displays a visual display indicating their direction, thus alerting the operator to the object without distracting the operator.

[0191] Figure 18 This is a flowchart illustrating the steps of the actions in the attention reminder system 500. Figure 13 When the detection unit 512 shown detects at least one object, it begins to operate according to each detected object. Figure 18 The processing shown.

[0192] When processing begins, firstly, the attention reminder unit 514 displays a visual display VI at a horizontal position on an indoor structure aligned with the direction of the detected object (S400). Furthermore, the attention reminder unit 514 displays a visual display AD indicating the direction of the detected object in the object radar display 520 of the HUD 156 (S402). At this time, as... Figure 17 As shown, the attention reminder unit 514 can display a visual display AD in a manner corresponding to the contact probability of the object detected by the detection unit 512 coming into contact with the vehicle 100. Alternatively, the attention reminder unit 514 may display a visual display AD extending along the entire circumference of the circular display 524 when the number of objects detected by the detection unit 512 exceeds a predetermined number.

[0193] Furthermore, the attention reminder unit 514 determines whether the area where the object is detected is the center front area (CFA) (S404). Then, when the area where the object is detected is not the center front area (CFA) (S404, "No"), the sound for directional positioning of the object is output to the operator through the indoor speaker 142 (S406).

[0194] Next, the attention reminder unit 514 determines whether the object is no longer detected by the detection unit 512 (S408). Then, when the object is no longer detected (408, "Yes"), the attention reminder unit 514 ends the process.

[0195] On the other hand, in step S404, when the area of ​​the object detected is the center front region (CFA) (S404, "Yes"), the attention reminder unit 514 does not output a sound and moves the processing to step S408. Furthermore, on the other hand, in step S408, when the object is detected again (S408, "No"), the attention reminder unit 514 returns to step S400 to repeat the processing.

[0196] Furthermore, the present invention is not limited to the structure of the above-described embodiments, and can be implemented in various ways without departing from its spirit.

[0197] For example, in the above embodiments, the physical objects that can be objects are not limited to moving physical objects such as vehicles or pedestrians, but can also be traffic infrastructure elements such as railway crossings, signal lights, electronic traffic bulletin boards, and traffic signs.

[0198] Furthermore, the mobile bodies equipped with attention alert systems 120, 300, 400, and 500 are not limited to vehicles, but can be any mobile body, such as aircraft or ships, whose movement can be operated by an operator.

[0199] Furthermore, the technical features present in any one of the attention alert systems 120, 300, 400, and 500 can also be present in any of the other attention alert systems. For example, the visual display VI implemented in attention alert systems 300 and 400 on the front glass 105 in a manner corresponding to cognitive proficiency, risk level, and / or reaction proficiency can also be used in the visual display VI of the left door glass 158, right door glass 160, and rear glass 162 of attention alert system 500.

[0200] For example, the attention reminder system 500 in the attention reminder device 502 has a function of Figure 5 The behavior detection unit 306, classification unit 308, and cognitive skill evaluation unit 310 shown, and / or through having Figure 9 The behavior detection unit 406, risk calculation unit 412, and reaction skill evaluation unit 410 shown can output a visual display VI in a manner corresponding to the above-mentioned cognitive proficiency, risk level, and / or reaction proficiency.

[0201] Furthermore, in the attention reminder system 500, the attention reminder unit 514 may also be... Figure 15 When the interior rearview mirror area RMA, left rearview mirror area LFMA, and right rearview mirror area RFMA detect an object, they output a visual display to the specified display range that can be defined on the mirror surface of the interior rearview mirror 110, left rearview mirror 112, and right rearview mirror 114, respectively.

[0202] Furthermore, in the above embodiments, the visual display VI is not limited to a rectangle, but can be any shape. Such a shape can be any polygon or line segment, such as a circle or triangle.

[0203] Furthermore, the attention reminder system 500 does not necessarily require three projection devices: a front projection device 122, left and right projection devices 504, and a rear projection device 506. At least one projection device is sufficient, and a single projection device can be used to output visual displays (VIs) to all specified interior structures such as the front glass 105, left door glass 158, right door glass 160, and rear glass 162.

[0204] In addition, the left and right projection devices 504 and / or the rear projection device 506 can also be light source arrays arranged below the left door glass 158, the right door glass 160, and the rear glass 162, which are the objects of display, just like the front projection device 122.

[0205] Furthermore, the front glass 105, left door glass 158, right door glass 160, and rear glass 162 do not necessarily have to be glass; they can be made of any material that allows the operator to visually confirm objects around the vehicle 100 with transparency (light transmittance).

[0206] [Structures supported by the above embodiments]

[0207] The above implementation supports the following structures.

[0208] (Structure 1) A attention reminder system comprising: a projection device that outputs a visual display on an interior structure of a mobile body, which can be visually confirmed by an operator of the mobile body; and an attention reminder device that controls the projection device to remind the operator of objects around the mobile body through the visual display, wherein the attention reminder device includes: a detection unit that detects objects located in the area surrounding the mobile body; and an attention reminder unit that controls the projection device to output a visual display on a predetermined display range on the interior structure of the mobile body, the display range on the interior structure being a strip-shaped range extending in a horizontal direction orthogonal to the height direction of the mobile body, and the attention reminder unit outputting the visual display at a horizontal position within the display range on the interior structure that is consistent with the direction of the object as observed from the operator's position.

[0209] In the attention alert system of structure 1, for objects present around the moving body, a visual display VI indicating the direction of the object as observed by the operator is displayed in a display range extending horizontally along the interior structure, thus alerting the operator to objects present around the moving body.

[0210] (Structure 2) Based on the attention reminder system described in Structure 1, in addition to the visual display, the attention reminder unit also generates a sound for directional positioning of the object for the operator through a speaker installed in the room of the moving body.

[0211] In the attention reminder system of Structure 2, the operator can be reminded of the object from both visual and auditory perspectives.

[0212] (Structure 3) Based on the attention reminder system described in Structure 1 or 2, the light projection device outputs the visual display by projecting light of a prescribed shape onto the interior structure.

[0213] In the attention alert system of Structure 3, visual prompts can be displayed in a way that is intuitive for the operator.

[0214] (Structure 4) Based on the attention reminder system described in any one of Structures 1 to 3, the interior structure is the front glass, front triangular window glass, door glass and / or rear glass of the moving body.

[0215] In the attention alert system of structure 4, visual displays are output on the interior structures outside the moving object that can be visually confirmed through them. Therefore, the operator can visually confirm the object while being alerted to its presence.

[0216] (Structure 5) Based on the attention reminder system described in any one of the structures 1 to 4, at least one of the projection devices outputs the visual display by projecting light of a predetermined shape onto the front glass of the moving body.

[0217] In the attention alert system of structure 5, at least on the front glass, a visual display indicating the direction of the object is displayed, overlapping the scene in front of the moving object that the operator needs to pay the most attention to. Therefore, the operator's attention can be effectively alerted to the object that should be paid special attention to when manipulating the moving object.

[0218] (Structure 6) Based on the attention reminder system described in Structure 5, the light projection device that outputs the visual display on the front glass is composed of a plurality of light sources arranged horizontally below the front glass.

[0219] In the attention reminder system of structure 6, a projection device that outputs a visual display on the front glass can be realized with a simple structure.

[0220] (Structure 7) Based on the attention reminder system described in any one of Structures 1 to 6, the attention reminder unit displays a circular display indicating the direction relative to the moving body on the display device, and outputs a visual display at the arc portion of the circular display corresponding to the direction of the object.

[0221] In the attention alert system of Structure 7, the operator can easily grasp the presence of objects in the entire surrounding area of ​​the moving object through the circular display.

[0222] (Structure 8) Based on the attention reminder system described in any one of the structures 1 to 7, the object is a physical object whose probability of contact with the moving body is a predetermined value or higher.

[0223] According to the attention reminder system of structure 8, objects that may pose a risk to the driving of the moving body are screened and their directions are shown in a visual display. Therefore, it is possible to remind the operator to pay attention to the object without distracting the operator.

[0224] (Structure 9) Based on the attention reminder system described in Structure 8, the attention reminder unit displays the visual display in a manner corresponding to the contact probability.

[0225] In the attention alert system of structure 9, the operator can visually identify objects with a priority corresponding to the degree of contact risk, and can alert the operator to various objects in the surrounding environment of the moving body in a more effective way for the operator's driving actions.

[0226] (Structure 10) A method for alerting attention, which is executed by a computer, wherein the method for alerting attention includes the following steps: detecting an object located in the area surrounding a moving body; and controlling a projection device disposed on the moving body to output a visual display on a predetermined display range on an interior structure of the moving body, the display range on the interior structure being a strip-shaped range extending in a horizontal direction orthogonal to the height direction of the moving body, wherein in the step of outputting the visual display, the visual display is output at a horizontal position within the display range on the interior structure that is consistent with the direction of the object as observed from the position of the operator of the moving body.

[0227] According to the attention reminder method of structure 10, the structure of the attention reminder system of structure 1 can be implemented by a computer executing a program.

Claims

1. A attention reminder system, comprising: A projection device that outputs a visual display on the interior structure of the mobile body, which can be visually confirmed by the operator of the mobile body; and The attention-reminder device controls the projection device to alert the operator to objects around the moving object through the visual display. in, The attention reminder device has the following features: The detection unit detects objects located in the area surrounding the moving body; and The attention and reminder unit controls the projection device to output a visual display within a predetermined display range on the indoor structure of the moving body. The surrounding area is divided into multiple angular regions. The angular region has: The operator, seated at the control station of the mobile body, can visually confirm the central front area with their face turned forward; The operator, seated in the control seat, can visually confirm the left and right front regions by turning his face about 45 degrees to the left and right. The operator, seated in the control seat, can visually confirm the left and right sides by turning his body to the left and right. The operator, seated in the control seat, can visually confirm the area of ​​the rearview mirror inside the vehicle via the rearview mirror inside the vehicle of the moving body. The operator, seated at the control station, can visually confirm the left rearview mirror area via the left rearview mirror of the moving body. as well as The operator, seated at the control station, can visually confirm the area of ​​the right rearview mirror via the right rearview mirror of the moving body. The display range on the interior structure located in front of the moving body inside the vehicle is a strip-shaped range extending in a horizontal direction orthogonal to the height direction of the moving body. The attention reminder unit outputs a visual display at a horizontal position within the display range on the interior structure in front of it, consistent with the direction of the object as observed from the operator's position. The visual display indicates the direction of the object detected by the detection unit in the central front region, the left front side region, and the right front side region. The visual display, indicating the orientation of the objects detected by the detection unit in the left and right regions, is output to the interior structures located on the left and right sides of the vehicle interior, respectively. The visual display, indicating the direction of the object detected by the detection unit in the rearview mirror area, the left rearview mirror area, and the right rearview mirror area, is output to the rear interior structure of the vehicle interior.

2. The attention reminder system according to claim 1, wherein, In addition to the visual display, the attention reminder unit also generates a sound for directional positioning of the object for the operator via a speaker installed in the interior of the moving body.

3. The attention reminder system according to claim 1, wherein, The projection device outputs the visual display by projecting light of a predetermined shape onto the interior structure.

4. The attention reminder system according to claim 1, wherein, The interior structure located in front is the front glass of the moving body. The interior structures located on the left and right sides are the front triangular window glass and / or door glass of the moving body. The interior structure located at the rear is the rear glass of the moving body.

5. The attention reminder system according to claim 1, wherein, At least one of the projection devices outputs the visual display by projecting light of a predetermined shape onto the front glass of the moving body.

6. The attention reminder system according to claim 5, wherein, The projection device that outputs the visual display on the front glass consists of a plurality of light sources arranged horizontally below the front glass.

7. The attention reminder system according to claim 1, wherein, The attention reminder unit displays a circular display on the display device indicating the direction relative to the moving body, and outputs a visual display at the arc portion of the circular display corresponding to the direction of the object.

8. The attention reminder system according to claim 1, wherein, The object is a physical object whose probability of contact with the moving body is a predetermined value or higher.

9. The attention reminder system according to claim 8, wherein, The attention reminder unit displays the visual information in a manner corresponding to the contact probability.

10. A method for reminding attention, which is a computer-executed method for reminding attention, wherein, The attention reminder method comprises the following steps: Detect objects located in the area surrounding the moving object; and The projection device installed on the mobile body is controlled to output a visual display within a specified display range on the interior structure of the mobile body. The surrounding area is divided into multiple angular regions. The angular region has: The operator, seated at the control station of the mobile body, can visually confirm the central front area with their face turned forward; The operator, seated in the control seat, can visually confirm the left and right front regions by turning his face about 45 degrees to the left and right. The operator, seated in the control seat, can visually confirm the left and right sides by turning his body to the left and right. The operator, seated in the control seat, can visually confirm the area of ​​the rearview mirror inside the vehicle via the rearview mirror inside the vehicle of the moving body. The operator, seated at the control station, can visually confirm the left rearview mirror area via the left rearview mirror of the moving body. as well as The operator, seated at the control station, can visually confirm the area of ​​the right rearview mirror via the right rearview mirror of the moving body. The display range on the interior structure located in front of the moving body inside the vehicle is a strip-shaped range extending in a horizontal direction orthogonal to the height direction of the moving body. In the step of outputting the visual display, the visual display is output at a horizontal position within the display range on the interior structure located in front, consistent with the direction of the object observed from the position of the operator of the moving body. The visual display indicates the direction of the object detected in the central front region, the left front side region, and the right front side region. The visual display indicating the direction of the object detected in the left side region and the right side region is output to the left and right interior structures in the vehicle interior, respectively. The visual display indicating the direction of the object detected in the rearview mirror region, the left rearview mirror region, and the right rearview mirror region is output to the rear interior structure in the vehicle interior.

Images