Display control apparatus, method for controlling display control apparatus, and storage medium

Abstract

To allow an occupant of a vehicle to easily recognize that a leaving risk object having a risk of leaving a display target group so as to approach the vehicle is present in the group, as compared with a case where only a group image is displayed on a display unit.SOLUTION: The display control device 100 includes the object recognizing unit 11 that recognizes an object in the vehicle 1 on the basis of the detection result of the in-vehicle external sensor 2, the display target specifying unit 12 that specifies an object forming a group among a plurality of objects present in the vehicle 1 as a target object, and the display control unit 13 that displays a group image representing the group on the HMI5 and does not display the target object on the HMI5. The display target specifying unit 12 specifies, as a leaving risk object, a target object that satisfies a leaving risk condition regarding a risk of leaving the group so as to approach the vehicle 1 among the target objects, and the display control unit 13 causes the HMI5 to display the group image and the leaving risk object when the leaving risk object is specified from among the target objects.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a display control device, a control method for a display control device, and a display control program. [Background technology]

[0002] Patent document 1 discloses a display control device that displays people present on a road displayed on a display unit with a single display symbol if there are more than a predetermined number of people within a predetermined range. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-184878 Summary of the Invention [Problem to be solved by the invention]

[0004] In a display control device that displays an image to a vehicle occupant, multiple objects may be displayed together with a single display symbol, as in the above-described conventional technology. However, if multiple objects are displayed as a group with a single display symbol, it becomes difficult for the vehicle occupant to recognize the movement of each individual object within the group. [Means for solving the problem]

[0005] A display control device according to one aspect of the present disclosure includes an object recognition unit that recognizes objects around the vehicle based on the detection results of an external sensor mounted on the vehicle, a display target identification unit that identifies objects that form a group among multiple objects present around the vehicle as target objects, and a display control unit that displays a group image representing the group on the display unit but does not display the target objects on the display unit, wherein the display target identification unit identifies as departure risk objects those target objects that satisfy a departure risk condition regarding the risk of leaving the group so as to approach the vehicle, and when a departure risk object is identified from among the target objects, the display control unit displays the group image and the departure risk object on the display unit.

[0006] In a display control device according to one aspect of the present disclosure, a target object that satisfies a departure risk condition is identified as a departure risk object. When a departure risk object is identified from among the target objects, a group image and the departure risk object are displayed on the display unit. This makes it easier for vehicle occupants to recognize that there is a departure risk object in the group that may approach the vehicle, compared to when only the group image is displayed on the display unit.

[0007] In one embodiment, the object recognition unit recognizes an obstacle that hinders the movement of the target object toward the vehicle based on the detection result of the external sensor, and the display target identification unit may identify, among the target objects, a target object whose movement toward the vehicle is not hindered by an obstacle as a departure risk object that satisfies the departure risk condition. In this case, it is easier for an occupant of the vehicle to recognize, among the target objects, a target object whose movement toward the vehicle is not hindered by an obstacle as a departure risk object.

[0008] In one embodiment, the display target identification unit may identify, among the three or more target objects, a target object that is positioned so that no other target objects intervene between it and the vehicle as a departure risk object that satisfies the departure risk condition. In this case, it is easier for a vehicle occupant to recognize, among the three or more target objects, a target object that is positioned so that no other target objects intervene between it and the vehicle as a departure risk object.

[0009] Another aspect of the present disclosure is a control method for a display control device that displays an image on an in-vehicle display unit, the control method for the display control device comprising: an object recognition step in which the display control device recognizes objects around the vehicle based on detection results from an external sensor on the vehicle; a display target identification step in which the display control device identifies objects that form a group from among a plurality of objects present around the vehicle as target objects; and a display control step in which the display control device displays a group image representing the group on the display unit while not displaying the target objects on the display unit, wherein in the display target identification step, a target object that satisfies a departure risk condition regarding the risk of leaving the group so as to approach the vehicle is identified as a departure risk object, and in the display control step, if a departure risk object is identified from among the target objects, the group image and the departure risk object are displayed on the display unit.

[0010] According to a control method for a display control device according to another aspect of the present disclosure, a target object that satisfies a departure risk condition is identified as a departure risk object. When a departure risk object is identified from among the target objects, a group image and the departure risk object are displayed on the display unit. This makes it easier for vehicle occupants to recognize that there is a departure risk object in the group that may leave the group and approach the vehicle, compared to when only the group image is displayed on the display unit.

[0011] Yet another aspect of the present disclosure is a display control program that causes a computer to function as a display control device that displays an image on an in-vehicle display unit, wherein the display control program causes the computer to function as an object recognition unit that recognizes objects around the vehicle based on detection results from an in-vehicle external sensor, a display target identification unit that identifies objects that form a group among multiple objects present around the vehicle as target objects, and a display control unit that displays a group image representing the group on the display unit but does not display the target objects on the display unit, wherein the display target identification unit identifies, among the target objects, a target object that satisfies a departure risk condition regarding the risk of leaving the group so as to approach the vehicle as a departure risk object, and when a departure risk object is identified from among the target objects, the display control unit displays the group image and the departure risk object on the display unit.

[0012] According to a display control program according to yet another aspect of the present disclosure, a target object that satisfies a departure risk condition is identified as a departure risk object. When a departure risk object is identified from among the target objects, a group image and the departure risk object are displayed on the display unit. This makes it easier for vehicle occupants to recognize that there is a departure risk object in the group that may leave the group and approach the vehicle, compared to when only the group image is displayed on the display unit. [Effects of the Invention]

[0013] According to some aspects of the present disclosure, vehicle occupants can more easily recognize that there is a departure risk object in the group that may leave the group and approach the vehicle, compared to when only the group image is displayed on the display unit. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a display control device according to an embodiment; [Figure 2] 10A and 10B are diagrams for explaining a first example of identifying a target object and a departure risk object. [Figure 3] FIG. 10 is a diagram for explaining a first display example of a target object and a departure risk object. [Figure 4] 10A and 10B are diagrams for explaining a second example of identifying a target object and a departure risk object. [Figure 5] FIG. 10 is a diagram for explaining a second display example of a target object and a departure risk object. [Figure 6] FIG. 10 is a diagram for explaining a third display example of a target object and a departure risk object. [Figure 7] 10A and 10B are diagrams for explaining a third example of identifying a target object and a departure risk object. [Figure 8] 10A and 10B are diagrams for explaining a fourth example of identifying a target object and a detachment risk object. [Figure 9] 10 is a flowchart illustrating an example of a display control process. [Figure 10] 10 is a flowchart illustrating an example of a display target specification process. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0016] Fig. 1 is a block diagram showing a display control device according to an embodiment. The display control device 100 shown in Fig. 1 is a device that uses external sensors 2 mounted on a vehicle 1, such as a passenger car or a freight vehicle, to recognize objects present around the vehicle 1 and display images to the occupants of the vehicle 1. The display control device 100 includes an ECU (Electronic Control Unit) 10, the external sensors 2, a GNSS receiver 3, a map database 4, and an HMI (Human Machine Interface) 5. The display control device 100 may be applied to a vehicle capable of executing driving assistance control or a vehicle capable of executing autonomous driving control.

[0017] [Configuration of display control device] As shown in FIG. 1 , the display control device 100 includes an ECU 10 that performs overall control of the device. The ECU 10 is an electronic control unit having a central processing unit (CPU) and storage units such as read-only memory (ROM), random access memory (RAM), and electrically erasable programmable read-only memory (EEPROM). The ECU 10 realizes various functions by, for example, executing programs stored in the storage units in the CPU. The ECU 10 may be configured as a unit integrated with an external sensor 2, or as a unit separate from the external sensor 2.

[0018] The ECU 10 is connected to an external sensor 2, a GNSS receiver 3, a map database 4, and an HMI (display) 5. Note that the GNSS receiver 3 and the map database 4 are not essential.

[0019] The external sensor 2 is a detection device that detects the situation around the vehicle 1. The external sensor 2 includes at least one of a camera and a radar sensor. The camera is an imaging device that captures an image in front of the vehicle 1. The camera is provided, for example, on the back side of the windshield of the vehicle 1, and captures an image in front of the vehicle 1. The camera has a function of measuring the relative distance to an object (pedestrian, etc.) around the vehicle 1. The camera may be a camera with a distance measurement function, such as a stereo camera. The camera may be a monocular camera that can measure the distance to an object in a captured image by performing predetermined image processing on the captured image. The camera transmits captured image information to the ECU 10.

[0020] A radar sensor is a detection device that uses radio waves (e.g., millimeter waves) or light to detect objects around the vehicle 1. The radar sensor detects objects by transmitting radio waves or light to the area around the vehicle 1 and receiving the radio waves or light reflected by the objects. Examples of radar sensors include LiDAR (Light Detection And Ranging) and millimeter wave radar. The radar sensor has the function of measuring the relative distance to objects (pedestrians, etc.) around the vehicle 1. The radar sensor transmits information about the detected objects to the ECU 10.

[0021] The external sensor 2 has a predetermined detection range. The detection range is a range in which the external sensor 2 can detect objects. Here, the external sensor 2 is configured, as an example, to be able to detect objects in all directions around the vehicle 1 when viewed from above (see FIG. 2, etc.). The detection range of the external sensor 2 is not limited to this example, and the external sensor 2 may be configured to be able to detect objects within a certain range around the vehicle 1 when viewed from above. The certain range around the vehicle 1 may be, for example, a sector-shaped area of ​​a predetermined radius centered on the position of the external sensor 2 when viewed from above.

[0022] The GNSS receiver 3 receives signals from positioning satellites to measure the vehicle position (e.g., the latitude and longitude of the vehicle 1), which is the current position of the vehicle 1. The GNSS receiver 3 transmits information about the measured vehicle position to the ECU 10.

[0023] The map database 4 is a storage device that stores map information. The map database 4 is formed, for example, in a storage medium such as a hard disk drive (HDD) mounted on the vehicle 1. The map information may include information on the positions of intersections on a map, information on the shape of the intersections, information on sidewalks along the roadway, and demarcations that separate the roadway from the sidewalk. Demarcations refer to, for example, guardrails, shrubbery, fences, curbs, etc. The map database 4 may be formed in a server that can communicate with the vehicle 1.

[0024] The HMI 5 is an interface for inputting and outputting information between the display control device 100 and an occupant of the vehicle 1. The HMI 5 has a display (display unit) provided in the cabin of the vehicle 1. The HMI 5 outputs an image to the display in response to a control signal from the ECU 10. The display may be a multi-information display (MID) or a head-up display (HUD).

[0025] Next, a description will be given of the functional configuration of the ECU 10. The ECU 10 has an object recognition unit 11, a display target specification unit 12, and a display control unit 13.

[0026] The object recognition unit 11 recognizes objects around the vehicle 1 based on the detection results of the external sensor 2. The object recognition unit 11 can recognize objects around the vehicle 1 within the detection range of the external sensor 2. The object recognition unit 11 may recognize the shape and type of object based on an image captured by a camera using a known image recognition process. The object recognition unit 11 may recognize the shape and type of object based on object information from a radar sensor using a known object recognition process. The objects recognized by the object recognition unit 11 may be, for example, other vehicles, pedestrians, bicycles, small animals, obstacles that obstruct the movement of objects around the vehicle 1, etc. Examples of obstacles that obstruct the movement of objects around the vehicle 1 include partitions (guardrails, shrubbery, fences, etc.) that separate the roadway from the sidewalk, the outer walls of buildings facing the roadway, etc. The object recognition unit 11 may recognize the relative position, relative distance, relative speed, movement direction, etc. of the object with respect to the vehicle 1. The object recognition unit 11 may recognize the relative positions, relative distances, relative speeds, and moving directions of the objects among a plurality of objects around the vehicle 1.

[0027] The object recognition unit 11 can recognize objects that form a group among multiple objects present around the vehicle 1 based on the detection results of the external sensor 2. The object here refers to traffic participants, and is not limited to pedestrians but also includes cyclists, etc. The object may be a small animal. The object recognition unit 11 recognizes objects that form a group based on the positions of the multiple objects. A group may be composed of at least three or more objects. The object recognition unit 11 recognizes objects that form a group based on, for example, the relative distances between the multiple objects. The object recognition unit 11 recognizes objects that are separated by a predetermined distance or less as objects that form a group. The object recognition unit 11 may also recognize objects that are separated by a predetermined distance or less from the center of gravity of the group as objects that form a group.

[0028] The object recognition unit 11 has a function of determining whether an object is located at the back or in front of the group. The object recognition unit 11 may recognize the positional relationship between objects in the group based on, for example, the degree of overlap between multiple objects in an image captured by a camera. The object recognition unit 11 may also recognize the positional relationship between objects in the group using, for example, the results of measuring the distances to multiple objects reached by radio waves or light from a radar sensor.

[0029] The display target identification unit 12 identifies, as target objects, objects that form a group among a plurality of objects present around the vehicle 1. The target objects are objects that are subject to display control as candidates to be displayed on the display of the HMI 5 for the occupants of the vehicle 1. As an example, the display target identification unit 12 identifies, as target objects, objects that are recognized by the object recognition unit 11 as objects that form a group among a plurality of objects present around the vehicle 1 within the distance threshold based on the objects recognized by the object recognition unit 11 and a predetermined distance threshold. The distance threshold is a threshold of the relative distance between the vehicle 1 and the objects used to identify the target object from the objects recognized by the object recognition unit 11. The display target identification unit 12 may store the distance threshold in a storage unit of the ECU 10, for example.

[0030] The target objects forming the group are contained within a virtual predetermined closed area in a planar view of the vehicle 1. The predetermined closed area may be an area defined by a predetermined figure that surrounds the multiple target objects forming a group in a planar view of the vehicle 1. The shape of the figure of the closed area may be, for example, a circle, an ellipse, a regular polygon, a polygon, or the like in a planar view of the vehicle 1.

[0031] The display target identification unit 12 identifies, among the target objects, a target object that satisfies a departure risk condition as a departure risk object. The departure risk condition is a condition regarding the risk of leaving the group so as to approach the vehicle 1. The departure risk condition can also be said to be a condition corresponding to the possibility that at least one of the multiple target objects forming the group has a higher possibility of jumping out from the group toward the vehicle 1 than the other target objects.

[0032] When the object recognition unit 11 recognizes an obstacle that obstructs the movement of multiple target objects forming a group toward the vehicle 1, it can be said that a target object among the multiple target objects whose movement toward the vehicle 1 is not obstructed by the obstacle satisfies the departure risk condition. This is because a target object whose movement toward the vehicle 1 is not obstructed by the obstacle is more likely to jump out of the group toward the vehicle 1 than other target objects whose movement toward the vehicle 1 is obstructed by the obstacle. The display target identification unit 12 identifies a target object among the target objects whose movement toward the vehicle 1 is not obstructed by the obstacle as a departure risk object that satisfies the departure risk condition.

[0033] Of three or more target objects forming a group, a target object that is positioned so that no other target objects intervene between it and vehicle 1 can be said to satisfy the departure risk condition. This is because a target object that is positioned so that no other target objects intervene between it and vehicle 1 is more likely to jump out of the group toward vehicle 1 than a target object that has other target objects intervene between it and vehicle 1. The display target identification unit 12 identifies, among the target objects, a target object whose movement toward vehicle 1 is not hindered by an obstacle as a departure risk object that satisfies the departure risk condition. The display target identification unit identifies, among three or more target objects, a target object that is positioned so that no other target objects intervene between it and vehicle 1 as a departure risk object that satisfies the departure risk condition.

[0034] The display control unit 13 displays a group image representing the group on the display of the HMI 5, and does not display the target object on the display of the HMI 5. When a departure risk object is identified from among the target objects, the display control unit 13 displays the group image and the departure risk object on the display of the HMI 5.

[0035] A group image is a graphic that represents multiple target objects that form a group to be displayed as a group, and refers to an image that is displayed to the occupants of the vehicle 1 in place of individual target objects. Therefore, as a general rule, target objects corresponding to the group image are not displayed on the display of the HMI 5. However, if a departure risk object is identified from among the target objects, the target object identified as the departure risk object is displayed on the display of the HMI 5 as an exception. The group image may be a predetermined graphic that corresponds to a closed area surrounding multiple target objects that form a group. The predetermined graphic may, for example, be a graphic that is the same shape as the closed area surrounding the target objects, or may be a simplified graphic such as a rectangle that includes part or all of the closed area surrounding the target objects.

[0036] The identification and display of target objects and objects at risk of departing will be specifically described.

[0037] As a first example, the display target identification unit 12 identifies, among the target objects, a target object whose movement toward the vehicle 1 is not hindered by an obstacle as a departure risk object that satisfies the departure risk condition. For example, the display target identification unit 12 may identify, among multiple target objects forming a group, a target object that exists in an area closer to the vehicle 1 than the obstacle as a departure risk object that satisfies the departure risk condition. FIG. 2 is a diagram illustrating a first identification example of target objects and departure risk objects. FIG. 3 is a diagram illustrating a first display example of target objects and departure risk objects. As shown in FIG. 2, pedestrians 21-26 are present around the vehicle 1 shown in a planar view. The vehicle 1 is traveling, for example, on a roadway. The pedestrians 21, 22, 24-26 are present on the sidewalk, and the pedestrian 23 is present on the roadway side of the guardrail 50 that separates the roadway from the sidewalk. It is assumed that the pedestrians 21-26 are present within the detection range of the external sensor 2 mounted on the vehicle 1 and are present within a predetermined distance threshold.

[0038] In the situation shown in FIG. 2, the object recognition unit 11 recognizes the positions of pedestrians 21 to 25 based on the detection results of the external sensor 2. The object recognition unit 11 calculates the distances between each of the pedestrians 21 to 25 based on the positions of the pedestrians 21 to 25. The object recognition unit 11 recognizes the pedestrians 21 to 25 as objects forming a group based on the distances between the pedestrians. In the example of FIG. 2, the distances between pedestrians 21 and 22, between pedestrians 22 and 23, between pedestrians 23 and 24, between pedestrians 24 and 25, and between pedestrians 25 and 21 are each assumed to be within a predetermined distance. The distances between these other combinations of pedestrians may also be within the predetermined distance. In this case, the object recognition unit 11 recognizes a region G10 corresponding to a group composed of the pedestrians 21 to 25. For example, if pedestrian 26 had been recognized by the object recognition unit 11 before the situation shown in FIG. 2 occurred, it may be treated as being included in region G10. The predetermined distance may be set appropriately through simulation or the like. The object recognition unit 11 may change the predetermined distance depending on the number of detected pedestrians. For example, the object recognition unit 11 may shorten the predetermined distance as the number of detected pedestrians increases.

[0039] The shape of region G10 is set as a polygonal region that encloses pedestrians 21-23, who are recognized as being located in front of vehicle 1, and pedestrians 24 and 25, who are recognized as being located behind vehicle 1. As an example, region G10 may be a region whose vertices are the positions of pedestrians 21-23, and whose outer edge forms a part of a rectangle on the side opposite vehicle 1 with respect to pedestrians 21-23. The part of a rectangle refers to a shape that connects a side extending away from pedestrian 21 in the width direction of vehicle 1, a side extending away from pedestrian 23 in the direction of travel of vehicle 1, and a pair of opposite sides parallel to these two sides. The length of the side may be a predetermined fixed value or may be a length calculated based on the distance between pedestrians 21-25. For example, the length of the side extending from pedestrian 21 in the width direction of vehicle 1 may be a predetermined coefficient multiplied by the length from pedestrian 21 to pedestrian 23 in the width direction. The length of the side extending from the position of pedestrian 23 in the traveling direction of vehicle 1 may be a predetermined coefficient multiplied by the length from the position of pedestrian 21 to the position of pedestrian 23 in the traveling direction.

[0040] The display target identification unit 12 identifies, as target objects, pedestrians 21 to 26 forming a group among pedestrians 21 to 26 present around the vehicle 1. The display target identification unit 12 identifies, as departure risk objects, pedestrians 21 to 26 among the pedestrians 21 to 26 belonging to the area G10 who satisfy a departure risk condition related to the risk of leaving the area G10 so as to approach the vehicle 1. Here, the pedestrian 23 present in front of the guardrail 50 as seen from the vehicle 1 is considered to be at risk of approaching the vehicle 1. The pedestrian 23 is a target object among the pedestrians 21 to 26 who are multiple target objects and whose movement toward the vehicle 1 is not hindered by the guardrail 50. The guardrail 50 is an obstacle that hinders the movement of the pedestrians 21 to 26, who are multiple target objects, toward the vehicle 1. The pedestrian 23 is an object that satisfies a departure risk condition related to the risk of leaving the area G10 so as to approach the vehicle 1. The display target identification unit 12 identifies the pedestrian 23 as a departure risk object. The display target identification unit 12 can identify the pedestrian 23 in Fig. 2 as a departure risk object based on, for example, a distance measurement result between the guardrail 50 and the pedestrian 23 by a radar sensor. The display target identification unit 12 may also identify the pedestrian 23 in Fig. 2 as a departure risk object based on the positional relationship between the guardrail 50 and the pedestrian 23 in an image captured by a camera. The display target identification unit 12 generates a group image G11 in Fig. 3 so as to correspond to the area G10.

[0041] Because pedestrian 23, a departure risk object, has been identified from among pedestrians 21-26, the display control unit 13 displays a group image G11 corresponding to area G10 and the pedestrian 23, a departure risk object, on the display of the HMI 5. As shown in FIG. 3 , the group image G11 represents a virtual area surrounding the group of pedestrians 21-26 on the display of the HMI 5, and may be, for example, a substantially rectangular shape that entirely encompasses area G10. The group image G11 may have rounded corners, and may represent a group of pedestrians by displaying a pedestrian icon that is larger than a normal pedestrian image. For example, the display control unit 13 displays a pedestrian image representing pedestrian 23, a departure risk object, superimposed on the group image G11 so as to correspond to the coordinates of pedestrian 23 in the three-dimensional space around the vehicle 1 on the display of the HMI 5. As a result, pedestrians 21, 22, 24 to 26 behind the guardrail 50 are grouped together as a group image G11 and are not displayed on the display of the HMI 5. On the other hand, pedestrian 23 in front of the guardrail 50 is displayed independently on the display of the HMI 5 as a separation risk object. This makes it easier for the occupants of the vehicle 1 to recognize the presence of pedestrian 23.

[0042] As a second example, the display target identification unit 12 may identify, among three or more target objects, a target object that is positioned so that no other target objects intervene between it and the vehicle 1, as a departure risk object that satisfies the departure risk condition.

[0043] Fig. 4 is a diagram for explaining a second identification example of a target object and a departure risk object. Fig. 5 is a diagram for explaining a second display example of a target object and a departure risk object. As shown in Fig. 4, pedestrians 21-26 are present on the sidewalk around a vehicle 1 shown in a plan view. In the example of Fig. 4, unlike the example of Fig. 2, there is no guardrail 50 separating the roadway and the sidewalk.

[0044] 4, the object recognition unit 11 recognizes the positions of pedestrians 21 to 25, which are three or more target objects, based on the detection results of the external sensor 2. The object recognition unit 11 recognizes an area G20 corresponding to one group made up of the pedestrians 21 to 25. The display target identification unit 12 identifies the pedestrians 21 to 26 that form the group, among the pedestrians 21 to 26 present around the vehicle 1, as target objects.

[0045] The display target specification unit 12 generates the group image G21 of Fig. 5 and the group image G22 of Fig. 6 so as to correspond to the region G20. The shape of the region G20 here is the same as the shape of the region G10. The shape of the region G20 may be different from the shape of the region G10.

[0046] The display target identification unit 12 identifies, as a departure risk object, a pedestrian who satisfies a departure risk condition regarding the risk of leaving the area G20 so as to approach the vehicle 1, among the pedestrians 21 to 26 belonging to the area G20. Here, the pedestrians 21 to 23 who are positioned so that no other pedestrians are between them and the vehicle 1 are considered to be at risk of approaching the vehicle 1. The pedestrians 21 to 23 are objects that satisfy the departure risk condition regarding the risk of leaving the area G20 so as to approach the vehicle 1. The display target identification unit 12 identifies, as a departure risk object that satisfies the departure risk condition, a pedestrian who is positioned so that no other pedestrians are between them and the vehicle 1, among the three or more pedestrians 21 to 26 belonging to the area G20.

[0047] As an example, the display target identification unit 12 may assume line segments (each of the dashed-dotted line segments in FIG. 4 ) connecting each of the multiple pedestrians 21 to 25 with the vehicle 1, and determine whether the pedestrian is an "object positioned such that no other object is interposed between the pedestrian and the vehicle 1" based on the length of the intersection of the line segment with the polygonal region G20 described above. For example, if the length of the portion where the line segment intersects with the region G20 (the portion of each of the dashed-dotted line segments in FIG. 4 that is inside the region G20) is less than a predetermined length threshold, the display target identification unit 12 may determine that the target object is an "object positioned such that no other object is interposed between the target object and the vehicle 1." For example, if the length of the portion where the line segment intersects with the region G20 (the portion of each of the dashed-dotted line segments in FIG. 4 that is inside the region G20) is equal to or greater than a predetermined length threshold, the display target identification unit 12 may determine that the target object is not an "object positioned such that no other object is interposed between the target object and the vehicle 1."

[0048] The predetermined length threshold is a length threshold for determining whether an object is "positioned so that no other objects are interposed between the object and the vehicle 1." In other words, the predetermined length threshold is a threshold for distinguishing whether the target object related to the line segment is located at the forefront or the back of the group as viewed from the vehicle 1. In the example of FIG. 4, when the positions of the pedestrians 21-23 are on the outer edge of the region G20, the predetermined length threshold can be set to a value greater than 0 and smaller than the length from the outer edge of the region G20 to the positions of the pedestrians 24 and 25 (the length on each of the dashed dotted lines in FIG. 4). When the positions of the pedestrians 21-23 are inside the outer edge of the region G20, the predetermined length threshold can be set to a value greater than the length from the outer edge of the region G20 to the positions of the pedestrians 21-23 (the length on each of the dashed dotted lines in FIG. 4) and smaller than the length from the outer edge of the region G20 to the positions of the pedestrians 24 and 25 (the length on each of the dashed dotted lines in FIG. 4).

[0049] 4, the length of the portion where each line segment for pedestrians 21 to 23 intersects with region G20 is less than a predetermined length threshold, so the display target specification unit 12 determines that pedestrians 21 to 23 are "objects that are positioned such that no other object is interposed between them and vehicle 1." The length of the portion where each line segment for pedestrians 24 and 25 intersects with region G20 is equal to or greater than a predetermined length threshold, so the display target specification unit 12 determines that pedestrians 24 and 25 are not "objects that are positioned such that no other object is interposed between them and vehicle 1."

[0050] Because pedestrians 21-23, who are departure risk objects, have been identified from among the pedestrians 21-26, the display control unit 13 displays a group image G21 corresponding to the area G20 and the pedestrians 21-23, who are departure risk objects, on the display of the HMI 5. As shown in FIG. 5, the group image G21, like the group image G11, represents a virtual area surrounding the group of pedestrians 21-26 on the display of the HMI 5. For example, the display control unit 13 displays pedestrian images representing the pedestrians 21-23, who are departure risk objects, superimposed on the group image G21 so as to correspond to the coordinates of the pedestrians 21-23 in the three-dimensional space around the vehicle 1 on the display of the HMI 5. As a result, pedestrians 24-26, who are target objects with other target objects (pedestrians 21-23) between them and the vehicle 1, are grouped together in the group image G21 and are not displayed on the display of the HMI 5. On the other hand, pedestrians 21-23, which are target objects positioned so that there are no other target objects between them and the vehicle 1, are displayed independently as departure risk objects on the display of the HMI 5. This makes it easier for the occupants of the vehicle 1 to recognize the presence of the pedestrians 21-23.

[0051] FIG. 6 is a diagram for explaining a third display example of a target object and a departure risk object. In the example of FIG. 6, compared to the examples of FIGS. 4 and 5, a pedestrian 24 approaching the vehicle 1 at a speed equal to or greater than a predetermined speed threshold is further identified as a departure risk object that poses a risk of approaching the vehicle 1. The pedestrian 24 is an object that satisfies the departure risk condition regarding the risk of leaving the area G20 to approach the vehicle 1. The predetermined speed threshold is a threshold of the object's speed for determining whether or not to identify the object as a departure risk object. The predetermined speed threshold may be set in advance for each type of object. When the object is a pedestrian, the predetermined speed threshold can be set to a speed that is faster than a normal walking speed and is equivalent to a running speed.

[0052] Because pedestrians 21-23 and 24, which are departure risk objects, have been identified from among pedestrians 21-26, the display control unit 13 displays the group image G22 and the departure risk objects pedestrians 21-23 and 24 on the display of the HMI 5. As shown in FIG. 6, the group image G22, like the group image G21, represents a virtual area surrounding the group of pedestrians 21-26 on the display of the HMI 5. In the group image G22, for example, taking into consideration the visibility of pedestrian 24, which is located near the center of the group image G22, a pedestrian icon that is larger than that of a normal pedestrian image may be displayed in an inconspicuous manner. Although the pedestrian icon is shown with a dashed line in FIG. 6 for convenience, it does not have to be displayed. For example, the display control unit 13 displays on the display of the HMI 5 pedestrian images representing the pedestrians 21-23, 24, who are departure risk objects, superimposed on the group image G22 so as to correspond to the coordinates of the pedestrians 21-23, 24 in the three-dimensional space around the vehicle 1. As a result, in addition to the pedestrians 21-23, who are objects positioned so that no other objects intervene between them and the vehicle 1, the pedestrian 24, who is at risk of approaching the vehicle 1, is displayed independently as a departure risk object. This makes it easier for the occupants of the vehicle 1 to recognize the presence of the pedestrians 21-24.

[0053] FIG. 7 is a diagram illustrating a third example of identifying target objects and departure risk objects. In the example of FIG. 7, multiple pedestrians 21-27 are present on the sidewalk around vehicle 1, and a sidewalk curb 51 is installed to separate the roadway from the sidewalk. A store entrance / exit 52 is provided midway along sidewalk curb 51. For example, at store entrance / exit 52, a portion of sidewalk curb 51 is lower than the other portions, allowing vehicle 1 to travel through. The width of the store entrance / exit 52 (the distance between a pair of two-dot chain lines in the figure) is, for example, approximately the width of a passenger car. In such a situation, display target identification unit 12 may identify pedestrians 21-24, 26, and 27 as target objects forming a continuous group (area G30). In addition, even if a guardrail or hedges is used instead of the sidewalk curbstone 51 and ends at the store entrance / exit 52, the display target identification unit 12 may identify the pedestrians 21 to 24, 26, and 27 as target objects that form one continuous group (area G30).

[0054] FIG. 8 is a diagram illustrating a fourth example of identifying target objects and departure risk objects. In the example of FIG. 8, an intersection 53 is present ahead of the vehicle 1. A plurality of pedestrians 21, 22, and 26 are present on the sidewalk to the right of the intersection 53, and a plurality of pedestrians 28 and 29 are present on the sidewalk to the right of the intersection 53. A crossroad 54 extends to the right of the intersection 53. The width of the crossroad 54 (the distance between a pair of solid lines in the figure) is, for example, equal to or greater than the width of two passenger cars. In such a situation, the display object identification unit 12 may identify the pedestrians 21, 22, and 26 and the pedestrians 28 and 29 as target objects that separately form two discontinuous groups (areas G31 and G32).

[0055] 7 and 8, the display target identification unit 12 may recognize the store entrance / exit 52 or the crossing road 54 based on the detection result of the external sensor 2. The display target identification unit 12 may also recognize the store entrance / exit 52 or the crossing road 54 based on the current vehicle position of the vehicle 1 measured by the GNSS receiving unit 3 and the map information in the map database 4.

[0056] [Display control device processing] Next, an example of the processing of the display control device 100 will be described with reference to the flowcharts of Fig. 9 and Fig. 10. Fig. 9 is a flowchart showing an example of the display control processing. The processing (steps) shown in Fig. 9 is repeatedly executed at predetermined calculation intervals, for example, when the power supply of the ECU 10 is ON.

[0057] 9, in S11, the ECU 10 of the display control device 100 causes the object recognition unit 11 to recognize objects around the vehicle 1 (object recognition step). The object recognition unit 11 recognizes objects that exist around the vehicle 1 within the detection range of the external sensor 2, for example, based on the detection result of the external sensor 2.

[0058] In S12, the ECU 10 sets a distance threshold value using the display object specification unit 12. The display object specification unit 12 sets the distance threshold value according to parameters stored in the ECU 10 in advance, for example.

[0059] In S13, the ECU 10 determines whether or not an object exists within the distance threshold using the display target identification unit 12. The display target identification unit 12 compares the relative distance of the object from the vehicle 1 recognized by the object recognition unit 11 with the distance threshold. If the relative distance is equal to or less than the distance threshold, the display target identification unit 12 determines that an object exists within the distance threshold. If the relative distance is greater than the distance threshold, the display target identification unit 12 determines that an object does not exist within the distance threshold.

[0060] If it is determined that an object exists within the distance threshold (S13: YES), the ECU 10 proceeds to the process of S14. If it is determined that an object does not exist within the distance threshold (S13: NO), the ECU 10 ends the process of Fig. 9. Note that the processes of S12 and S13 are not essential. If the processes of S12 and S13 are omitted, the ECU 10 may proceed to S14 after the process of S11.

[0061] In S14, the ECU 10 causes the display target identification unit 12 to identify, as target objects, objects that form a group among a plurality of objects present around the vehicle (display target identification step). As an example, for each of a plurality of objects present within the detection range of the external sensor 2, the display target identification unit 12 identifies, as target objects, objects that form a group among objects whose relative distance to the vehicle 1 is equal to or less than a distance threshold.

[0062] In S15, the ECU 10 causes the display target identification unit 12 to identify, among the target objects, target objects that satisfy the departure risk condition as departure risk objects (display target identification step). The display target identification unit 12 identifies, among the target objects, target objects that satisfy the departure risk condition related to the risk of leaving the group so as to approach the vehicle 1 as departure risk objects. The ECU 10 executes, for example, the processing shown in FIG. 10.

[0063] Fig. 10 is a flowchart showing an example of the display target specification process. The process (display target specification step) shown in Fig. 10 is repeatedly executed at predetermined calculation intervals, for example, when the power supply of the ECU 10 is ON.

[0064] 10, in S21, the ECU 10 of the display control device 100 determines, using the display target identification unit 12, whether or not there is an obstacle that obstructs the movement of the target object toward the vehicle. The display target identification unit 12 recognizes, as an obstacle, an object (e.g., a guardrail 50) that is positioned and shaped to obstruct the movement of the target object toward the vehicle 1, among the objects around the vehicle 1, based on, for example, the type, position, and shape of the object recognized by the object recognition unit 11. The obstacle may be a stationary object or a moving object.

[0065] In S22, the ECU 10 causes the display object identification unit 12 to identify, among the target objects, a target object whose movement toward the vehicle 1 is not hindered by an obstacle, as a departure risk object that satisfies the departure risk condition. In the example of FIG. 2, the display object identification unit 12 identifies, among the pedestrians 21 to 26 that are target objects that form the area G10, the pedestrian 23 that exists in the area 50a that is closer to the vehicle 1 than the guardrail 50, as a departure risk object that satisfies the departure risk condition. Thereafter, the ECU 10 ends the processing of FIG. 10 and proceeds to the processing of S16 of FIG. 9.

[0066] On the other hand, in S23, the ECU 10 determines whether or not three or more target objects exist using the display object specification unit 12. If it is determined that three or more target objects exist (S23: YES), the ECU 10 proceeds to the process of S24. If it is determined that three or more target objects do not exist (S23: NO), the ECU 10 proceeds to the process of S26.

[0067] In S24, the ECU 10 determines, for each of the three or more target objects, whether or not the target object is located such that no other target objects are interposed between the target object and the vehicle, using the display target identification unit 12. As an example, the display target identification unit 12 may assume a line segment connecting each of the multiple target objects to the vehicle 1, and determine, based on the length of the intersection of the line segment with the polygonal region G20, whether or not the target object related to the line segment is located such that no other target objects are interposed between the target object and the vehicle 1.

[0068] For example, if the length of the portion where the line segment intersects with area G20 (the portion of each dot-dash line segment in FIG. 4 that is inside area G20) is less than a predetermined length threshold, the display target identification unit 12 determines that the target object related to the line segment is an object that is positioned such that no other target object is interposed between it and the vehicle 1. For example, if the length of the portion where the line segment intersects with area G20 (the portion of each dot-dash line segment in FIG. 4 that is inside area G20) is equal to or greater than a predetermined length threshold, the display target identification unit 12 determines that the target object related to the line segment is not an object that is positioned such that no other target object is interposed between it and the vehicle 1.

[0069] For each of the three or more target objects, if it is determined that the target object is located such that no other target objects are interposed between it and the vehicle (S24: YES), the ECU 10 proceeds to the process of S25. For each of the three or more target objects, if it is determined that the target object is not located such that no other target objects are interposed between it and the vehicle (S24: NO), the ECU 10 proceeds to the process of S26.

[0070] In S25, the ECU 10 causes the display target identification unit 12 to identify, among the three or more target objects, target objects that are positioned so that no other target objects intervene between them and the vehicle, as departure risk objects that satisfy the departure risk condition. In the example of Fig. 4, the display target identification unit 12 identifies, among the pedestrians 21-26 belonging to the area G20, pedestrians 21-23 that are positioned so that no other target objects intervene between them and the vehicle 1, as departure risk objects that satisfy the departure risk condition. Thereafter, the ECU 10 ends the processing of Fig. 10 and proceeds to the processing of S16 of Fig. 9.

[0071] On the other hand, in S26, the ECU 10 does not specify the target object as a departure risk object that satisfies the departure risk condition by the display object specification unit 12. After that, the ECU 10 ends the processing of FIG. 10 and proceeds to the processing of S16 of FIG.

[0072] Returning to the processing of FIG. 9, in S16, the ECU 10 determines whether or not a separation risk object has been identified from among the target objects by the display control unit 13 (display control step). The display control unit 13 determines whether or not a separation risk object has been identified from among the target objects, depending on the result of identification of the separation risk object by the display target identification unit 12. If it is determined that a separation risk object has been identified from among the target objects (S16: YES), the ECU 10 proceeds to processing of S17. If it is determined that a separation risk object has not been identified from among the target objects (S16: NO), the ECU 10 proceeds to processing of S18.

[0073] In S17, the ECU 10 causes the display control unit 13 to display the group image and the separation risk object on the display unit (display control step). The display control unit 13 displays the separation risk object on the display of the HMI 5, for example, superimposed on the group image. Thereafter, the ECU 10 ends the processing of FIG. 9.

[0074] Meanwhile, in S18, the ECU 10 causes the display control unit 13 to display the group image on the display unit (display control step). Because no separation risk object has been identified, the display control unit 13 causes only the group image to be displayed on the display of the HMI 5. Thereafter, the ECU 10 ends the processing of FIG.

[0075] [Display control program] The display control program causes the ECU 10 (computer) to function (operate) as the object recognition unit 11, the display target identification unit 12, and the display control unit 13. The display control program is provided by a non-transitory recording medium such as a ROM or a semiconductor memory. The display control program may also be provided via communication such as a network.

[0076] According to the display control device 100, the control method for the display control device, and the display control program described above, a pedestrian (target object) that satisfies the departure risk condition is identified as a departure risk object. When a departure risk object is identified from the target objects, the group image and the departure risk object are displayed on the HMI 5. As a result, in the example of FIG. 3, the HMI 5 displays not only the group image G11 but also the pedestrian 23. In the example of FIG. 5, the HMI 5 displays not only the group image G21 but also the pedestrians 21-23. In the example of FIG. 6, the HMI 5 displays not only the group image G22 but also the pedestrians 21-24. This makes it easier for the occupants of the vehicle 1 to recognize that pedestrians 23, 21-23, and 21-24, who are departure risk objects that may leave the group images G11, G21, and G22 so as to approach the vehicle 1, are present in the group, compared to when only the group images G11, G21, and G22 are displayed on the HMI 5.

[0077] According to the display control device 100, the control method for the display control device, and the display control program, for example, in FIG. 2, a guardrail 50 that blocks the movement of pedestrians 21, 22, 24-26 toward the vehicle 1 is recognized based on the detection results of the external sensor 2. Among the pedestrians 21-26, pedestrian 23, whose movement toward the vehicle 1 is not blocked by the guardrail 50, is identified as a departure risk object that satisfies the departure risk condition. As a result, as shown in FIG. 3, the HMI 5 displays not only the group image G11 but also the pedestrian 23. This makes it easier for the occupants of the vehicle 1 to recognize pedestrian 23, among the pedestrians 21-26, whose movement toward the vehicle 1 is not blocked by the guardrail 50, as a departure risk object.

[0078] According to the display control device 100, the control method for the display control device, and the display control program, for example, in FIG. 4, three or more target objects, pedestrians 21-26, are recognized. Among the three or more pedestrians 21-26, pedestrians 21-23, who are positioned so that no other target objects intervene between them and the vehicle 1, are identified as departure-risk objects that satisfy the departure-risk condition. As a result, as shown in FIG. 5, the HMI 5 displays not only the group image G21 but also the pedestrians 21-23. In other words, by grouping the target objects at the back as a group, the target objects at the front of the group can be displayed independently without displaying them. This makes it easier for the occupants of the vehicle 1 to recognize pedestrians 21-23, who are positioned so that no other target objects intervene between them and the vehicle 1, as departure-risk objects.

[0079] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments.

[0080] In the above embodiment, FIGS. 2 and 3 are shown as examples in which the display target identification unit 12 identifies, as a departure risk object that satisfies the departure risk condition, an object that is present in an area closer to the vehicle 1 than an obstacle among the objects belonging to the display target group. However, this is not limiting. The display target identification unit 12 may identify, as a departure risk object that satisfies the departure risk condition, an object whose movement toward the vehicle 1 is not hindered by an obstacle. Furthermore, the obstacle is not limited to a guardrail 50, but may be a shrubbery or a glass-paned exterior wall on the first floor of a building (in cases where a facility such as a showroom is located inside the room behind the glass). Furthermore, the obstacle is not limited to an object extending along the traveling direction of the vehicle 1, but may be an object extending along the width direction of the vehicle 1 or an object extending in a curved line.

[0081] In the above embodiment, FIGS. 4 to 6 are shown as examples of identifying target objects among three or more target objects that are positioned so that no other target objects intervene between them and the vehicle 1 as departure risk objects that satisfy the departure risk condition. However, this is not limiting. For example, a line segment connecting each of the pedestrians 24 and 25 to the vehicle 1 is assumed, and it is determined that pedestrians 23 and 22 are interposed between the vehicle 1 and the target object based on the length of the intersection of the line segment with the region G10 or G20. However, other methods may be used to determine whether a target object is positioned so that no other target objects intervene between it and the vehicle 1. For example, the display target identification unit 12 may determine whether a target object is positioned so that no other target objects intervene between it and the vehicle 1 based on the degree of overlap between the pedestrians 23 and 22 and the pedestrians 24 and 25 as viewed from the vehicle 1, based on an image captured by the camera of the external sensor 2.

[0082] In the above embodiment, the display target identification unit 12 defines polygonal regions G10 and G20 whose vertices are the positions of the target objects located on the vehicle 1 side among the multiple target objects as the closed regions surrounding the target objects, but the shape of the closed regions in a planar view is not limited to these examples. The closed region may be a regular polygon whose vertices are the positions of the target objects located on the vehicle 1 side among the multiple target objects, or may be a circle or ellipse that passes through the positions of the target objects located on the vehicle 1 side among the multiple target objects. [Explanation of symbols]

[0083] 1...vehicle, 2...external sensor, 10...ECU (computer), 11...object recognition unit, 12...display target identification unit, 13...display control unit, G11, G21, G22...group images, 50a...area, 100...display control device

Claims

1. an object recognition unit that recognizes objects around the vehicle based on detection results from an external sensor mounted on the vehicle; a display object specifying unit that specifies, as target objects, objects that form a group among the plurality of objects present around the vehicle; a display control unit that causes a group image representing the group to be displayed on a display unit, and that does not cause the target object to be displayed on the display unit; the display target specification unit specifies, among the target objects, the target object that satisfies a departure risk condition regarding a risk of leaving the group so as to approach the vehicle, as a departure risk object; The display control unit is a display control device that, when the departure risk object is identified from the target objects, causes the group image and the departure risk object to be displayed on the display unit.

2. the object recognition unit recognizes an obstacle that obstructs the movement of the target object toward the vehicle based on the detection result of the external sensor; The display control device according to claim 1, wherein the display target identification unit identifies, among the target objects, a target object whose movement toward the vehicle is not hindered by the obstacle as the departure risk object that satisfies the departure risk condition.

3. The display control device described in claim 1 or 2, wherein the display target identification unit identifies, among three or more target objects, a target object that is positioned so that no other target objects are interposed between it and the vehicle as a departure risk object that satisfies the departure risk condition.

4. A control method for a display control device that displays an image on an in-vehicle display unit, comprising: an object recognition step of recognizing objects around the vehicle by the display control device based on detection results of an external sensor mounted on the vehicle; a display object specifying step of specifying, by the display control device, objects that form a group among the plurality of objects present around the vehicle as target objects; a display control step of causing the display control device to display a group image representing the group on a display unit and not display the target object on the display unit, In the display target identification step, the target object that satisfies a departure risk condition regarding a risk of leaving the group so as to approach the vehicle is identified as a departure risk object among the target objects; A control method for a display control device, in which, in the display control step, if the departure risk object is identified from the target objects, the group image and the departure risk object are displayed on the display unit.

5. A display control program that causes a computer to function as a display control device that displays an image on an in-vehicle display unit, the display control program comprising: an object recognition unit that recognizes objects around the vehicle based on detection results from an external sensor mounted on the vehicle; a display object specifying unit that specifies, as target objects, objects that form a group among the plurality of objects present around the vehicle; and a display control unit that causes a group image representing the group to be displayed on a display unit and that does not cause the target object to be displayed on the display unit; the display target specification unit specifies, among the target objects, the target object that satisfies a departure risk condition regarding a risk of leaving the group so as to approach the vehicle, as a departure risk object; The display control program causes the display unit to display the group image and the departure risk object when the departure risk object is identified from the target objects.

Images