Display control device, control method for display control device, and display control program

Abstract

To allow an occupant of a vehicle to easily recognize that a leaving risk object having a risk of leaving a display target group so as to approach the vehicle is present in the group, as compared with a case where only a group image is displayed on a display unit.SOLUTION: The display control device 100 includes the object recognizing unit 11 that recognizes an object in the vehicle 1 on the basis of the detection result of the in-vehicle external sensor 2, the display target specifying unit 12 that specifies an object forming a group among a plurality of objects present in the vehicle 1 as a target object, and the display control unit 13 that displays a group image representing the group on the HMI5 and does not display the target object on the HMI5. The display target specifying unit 12 specifies, as a leaving risk object, a target object that satisfies a leaving risk condition regarding a risk of leaving the group so as to approach the vehicle 1 among the target objects, and the display control unit 13 causes the HMI5 to display the group image and the leaving risk object when the leaving risk object is specified from among the target objects.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a display control device, a control method for a display control device, and a display control program. [Background Art]

[0002] Patent Document 1 discloses a display control device that, when a plurality of people present on a road displayed on a display unit is equal to or more than a predetermined number that is a plurality of people within a predetermined range, displays the people present within the predetermined range with a single display symbol. [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2015-184878 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] In a display control device that displays images to an occupant of a vehicle, a plurality of objects may be collectively displayed with a single display symbol as in the above-described conventional technology. However, for example, when a plurality of objects are displayed as a group with a single display symbol, it becomes difficult for the vehicle occupant to recognize the movement of an individual object among the objects in the group. [Means for Solving the Problem]

[0005] A display control device according to one aspect of the present disclosure includes: an object recognition unit that recognizes objects around a vehicle based on detection results from an external on-board sensor; a display target identification unit that identifies objects forming a group from among a plurality of objects present around the vehicle as target objects; and a display control unit that displays a group image representing the group on a display unit and does not display the target objects on the display unit. The display target identification unit identifies target objects that satisfy separation risk conditions regarding the risk of separating from the group to approach the vehicle as separation risk objects, and if a separation risk object is identified from among the target objects, the display control unit displays the group image and the separation risk object on the display unit.

[0006] In a display control device according to one aspect of this disclosure, objects that meet the detachment risk conditions are identified as detachment risk objects. When a detachment risk object is identified from among the objects, the group image and the detachment risk object are displayed on the display unit. This makes it easier for the vehicle occupant to recognize that there is a detachment risk object in the group that may detach from the group and approach the vehicle, compared to when only the group image is displayed on the display unit.

[0007] In one embodiment, the object recognition unit recognizes obstacles that hinder the movement of the target object towards the vehicle based on the detection results of an external sensor, and the display target identification unit identifies target objects that are not hindered in their movement towards the vehicle by obstacles as objects with a risk of separation that satisfy the separation risk conditions. In this case, it becomes easier for the vehicle occupants to recognize target objects that are not hindered in their movement towards the vehicle by obstacles as objects with a risk of separation.

[0008] In one embodiment, the display target identification unit may identify, among three or more target objects, an object that is positioned so that no other target objects are interposed between it and the vehicle as an object with a risk of separation that satisfies the separation risk condition. In this case, it becomes easier for the vehicle occupants to recognize the object that is positioned so that no other target objects are interposed between it and the vehicle as an object with a risk of separation.

[0009] Another aspect of the present disclosure is a control method for a display control device that causes an image to be displayed on an in-vehicle display unit, the control method for the display control device comprising: an object recognition step in which the display control device recognizes objects around a vehicle based on detection results from an external in-vehicle sensor; a display target identification step in which the display control device identifies objects that form a group from among a plurality of objects present around the vehicle as target objects; and a display control step in which the display control device displays a group image representing the group on the display unit and does not display the target objects on the display unit, wherein in the display target identification step, target objects that satisfy the separation risk conditions relating to the risk of separating from the group to approach the vehicle are identified as separation risk objects, and in the display control step, if a separation risk object is identified from among the target objects, the group image and the separation risk object are displayed on the display unit.

[0010] According to a control method for a display control device in another aspect of this disclosure, an object that satisfies the separation risk conditions is identified as a separation risk object. When a separation risk object is identified from among the objects, the group image and the separation risk object are displayed on the display unit. This makes it easier for the vehicle occupant to recognize that there is a separation risk object in the group that may separate from the group and approach the vehicle, compared to when only the group image is displayed on the display unit.

[0011] A further aspect of the present disclosure is a display control program that causes a computer to function as a display control device for displaying an image on an in-vehicle display unit, wherein the display control program causes the computer to function as an object recognition unit that recognizes objects around a vehicle based on detection results from an external in-vehicle sensor, a display target identification unit that identifies objects forming a group from among a plurality of objects present around the vehicle as target objects, and a display control unit that displays a group image representing the group on the display unit and does not display the target objects on the display unit, wherein the display target identification unit identifies target objects that satisfy separation risk conditions regarding the risk of separating from the group to approach the vehicle as separation risk objects, and the display control unit, when a separation risk object is identified from among the target objects, displays the group image and the separation risk object on the display unit.

[0012] According to a display control program in yet another aspect of this disclosure, objects that meet the separation risk conditions are identified as separation risk objects. When a separation risk object is identified from among the objects, the group image and the separation risk object are displayed on the display unit. This makes it easier for the vehicle occupants to recognize that there are separation risk objects in the group that may separate from the group and approach the vehicle, compared to when only the group image is displayed on the display unit. [Effects of the Invention]

[0013] According to some aspects of this disclosure, compared to the case where only a group image is displayed on the display unit, it becomes easier for the vehicle occupants to recognize the presence of detachment risk objects within the group that may detach from the group and approach the vehicle. [Brief explanation of the drawing]

[0014] [Figure 1] This is a block diagram of a display control device according to an embodiment. [Figure 2] This diagram illustrates the first specific example of the target object and the object at risk of detachment. [Figure 3] This is a diagram illustrating the first example of displaying the target object and the object at risk of detachment. [Figure 4] This diagram illustrates the second example of identifying the target object and the object at risk of detachment. [Figure 5] This is a diagram illustrating a second example of how to display the target object and the object at risk of detachment. [Figure 6] This is a diagram illustrating a third example of how to display the target object and the object at risk of detachment. [Figure 7] This diagram illustrates the third example of a target object and an object at risk of detachment. [Figure 8] This diagram illustrates the fourth example of the identification of the target object and the object at risk of detachment. [Figure 9] This flowchart shows an example of display control processing. [Figure 10] This flowchart shows an example of the process for identifying the items to be displayed. MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0016] Figure 1 is a block diagram showing a display control device according to an embodiment. The display control device 100 shown in Figure 1 is a device that recognizes objects existing around a vehicle 1 such as a passenger car or a freight car using an external sensor 2 mounted on the vehicle 1, and displays an image to an occupant of the vehicle 1. The display control device 100 includes an ECU [Electronic Control Unit] 10, an external sensor 2, a GNSS receiving unit 3, a map database 4, and an HMI [Human Machine Interface] 5. The display control device 100 may be applied to a vehicle capable of executing driving assistance control or a vehicle capable of executing automatic driving control.

[0017] [Configuration of Display Control Device] As shown in Figure 1, the display control device 100 includes the ECU 10 that comprehensively manages the device. The ECU 10 is an electronic control unit including a CPU [Central Processing Unit], a storage unit such as a ROM [Read Only Memory], a RAM [Random Access Memory], and an EEPROM [Electrically Erasable Programmable Read-Only Memory]. In the ECU 10, for example, various functions are implemented by causing the CPU to execute programs stored in the storage unit. The ECU 10 may be composed of a plurality of electronic units. The ECU 10 may be configured as an integrated unit with the external sensor 2, or may be configured as a separate unit from the external sensor 2.

[0018] The ECU 10 is connected to the external sensor 2, the GNSS receiving unit 3, the map database 4, and the HMI 5 (display unit). Note that the GNSS receiving unit 3 and the map database 4 are not essential components.

[0019] External sensor 2 is a detection device that detects the surrounding conditions of vehicle 1. External sensor 2 includes at least one of a camera and a radar sensor. The camera is an imaging device that captures images of the area in front of vehicle 1. The camera is installed, for example, behind the windshield of vehicle 1 and captures images of the area in front of vehicle 1. The camera has a function to measure the relative distance to objects (pedestrians, etc.) around vehicle 1. The camera may be a camera with a distance-measuring function, such as a stereo camera. The camera may be a monocular camera capable of measuring the distance of objects in the captured image by performing predetermined image processing on the captured image. The camera transmits the captured image information to the ECU 10.

[0020] A radar sensor is a detection device that uses radio waves (e.g., millimeter waves) or light to detect objects around a vehicle 1. The radar sensor detects objects by transmitting radio waves or light around the vehicle 1 and receiving the radio waves or light reflected by the object. Examples of radar sensors include LiDAR (Light Detection and Ranging) and millimeter-wave radar. The radar sensor has the function of measuring the relative distance to objects (pedestrians, etc.) around the vehicle 1. The radar sensor transmits the detected object information to the ECU 10.

[0021] The external sensor 2 has a predetermined detection range. The detection range is the range in which the external sensor 2 can detect objects. Here, the external sensor 2 is configured to detect objects in all directions around the vehicle 1 in a plan view of the vehicle 1 (see Figure 2, etc.). The detection range of the external sensor 2 is not limited to this example, and may be configured to detect objects within a certain range around the vehicle 1 in a plan view of the vehicle 1. The certain range around the vehicle 1 may be, for example, a sector-shaped area with a predetermined radius centered on the position of the external sensor 2 in a plan view of the vehicle 1.

[0022] The GNSS receiver 3 receives signals from positioning satellites to measure the vehicle's current position (e.g., the latitude and longitude of the vehicle 1). The GNSS receiver 3 transmits the measured vehicle position information to the ECU 10.

[0023] The map database 4 is a storage device that stores map information. The map database 4 is formed in a storage medium such as an HDD [Hard Disk Drive] installed in the vehicle 1. The map information may include information on the location of intersections on a map, information on the shape of intersections, information on sidewalks provided along roadways, and demarcation structures that separate sidewalks from roadways. Demarcation structures mean, for example, guardrails, plantings, fences, curbs, etc. The map database 4 may be formed on a server that can communicate with the vehicle 1.

[0024] HMI5 is an interface for inputting and outputting information between the display control device 100 and the occupants of vehicle 1. HMI5 has a display (display unit) installed inside the cabin of vehicle 1. HMI5 outputs an image from the display in response to a control signal from ECU 10. The display may be a MID (Multi-Information Display) or a HUD (Head-Up Display).

[0025] Next, the functional configuration of the ECU10 will be described. The ECU10 includes an object recognition unit 11, a display target identification unit 12, and a display control unit 13.

[0026] The object recognition unit 11 recognizes objects around the vehicle 1 based on the detection results of the external sensor 2. The object recognition unit 11 can recognize objects that exist around the vehicle 1 within the detection range of the external sensor 2. The object recognition unit 11 may recognize the shape and type of an object by known image recognition processing based on the image captured by the camera. The object recognition unit 11 may recognize the shape and type of an object by known object recognition processing based on the object information from the radar sensor. The objects recognized by the object recognition unit 11 may be, for example, other vehicles, pedestrians, bicycles, small animals, obstacles that obstruct the movement of objects around the vehicle 1, etc. Examples of obstacles that obstruct the movement of objects around the vehicle 1 include partitions that separate the roadway and the sidewalk (guardrails, plantings, fences, etc.), the outer walls of buildings facing the roadway, etc. The object recognition unit 11 may recognize the relative position, relative distance, relative speed, and direction of movement of an object relative to the vehicle 1. The object recognition unit 11 may recognize the relative position, relative distance, relative speed, and direction of movement of multiple objects surrounding the vehicle 1.

[0027] The object recognition unit 11 can recognize objects that form a group from among multiple objects present around the vehicle 1, based on the detection results of the external sensor 2. Here, "objects" refer to traffic participants and are not limited to pedestrians, but also include cyclists, etc. The objects may also be small animals. The object recognition unit 11 recognizes objects that form a group based on the positions of multiple objects. A group may consist of at least three or more objects. The object recognition unit 11 recognizes objects that form a group, for example, based on the relative distance between multiple objects. The object recognition unit 11 recognizes objects that form a group if the distance between multiple objects is less than or equal to a predetermined distance. The object recognition unit 11 may also recognize objects that form a group if their distance from the center of gravity of the group is less than or equal to a predetermined distance.

[0028] The object recognition unit 11 has a function to determine whether an object is located at the back of a group or at the front of the group. The object recognition unit 11 may recognize the positional relationship between objects in a group based on, for example, the degree of overlap of multiple objects in the image captured by the camera. The object recognition unit 11 may also recognize the positional relationship between objects in a group using, for example, the distance measurement results to multiple objects reached by radio waves or light from a radar sensor.

[0029] The display target identification unit 12 identifies objects that form a group from among multiple objects present around the vehicle 1 as target objects. Target objects are objects that are subject to display control as candidates to be displayed on the HMI 5 display for the occupants of the vehicle 1. As an example, the display target identification unit 12 identifies objects that the object recognition unit 11 recognizes as forming a group from among multiple objects present around the vehicle 1 within a distance threshold, based on objects recognized by the object recognition unit 11 and a predetermined distance threshold, as target objects. The distance threshold is a threshold for the relative distance between the vehicle 1 and the objects used to identify the target objects from the objects recognized by the object recognition unit 11. The display target identification unit 12 may, for example, store the distance threshold in the storage unit of the ECU 10.

[0030] The objects forming a group are contained within a predetermined virtual closed region in a plan view of vehicle 1. The predetermined closed region may be defined by a predetermined figure that surrounds a group of objects in a plan view of vehicle 1. The shape of the figure in the closed region can be, for example, a circle, an ellipse, a regular polygon, a polygon, etc., in a plan view of vehicle 1.

[0031] The display target identification unit 12 identifies target objects that satisfy the separation risk condition as separation risk objects. The separation risk condition is a condition relating to the risk of separating from the group so as to approach vehicle 1. The separation risk condition can also be described as a condition in which at least one of the multiple target objects forming the group has a higher probability than the other target objects of flying out of the group towards vehicle 1.

[0032] When the object recognition unit 11 recognizes an obstacle that prevents multiple objects forming a group from moving towards vehicle 1, the objects among the multiple objects whose movement towards vehicle 1 is not obstructed by the obstacle can be said to satisfy the escape risk condition. This is because objects whose movement towards vehicle 1 is not obstructed by the obstacle are more likely to jump out of the group towards vehicle 1 than other objects whose movement towards vehicle 1 is obstructed by the obstacle. The display object identification unit 12 identifies the objects among the objects whose movement towards vehicle 1 is not obstructed by the obstacle as escape risk objects that satisfy the escape risk condition.

[0033] Among three or more target objects forming a group, an object positioned so that no other object is interposed between it and vehicle 1 can be said to satisfy the detachment risk condition. This is because an object positioned so that no other object is interposed between it and vehicle 1 is more likely to fly out of the group towards vehicle 1 than an object that has other object interposed between it and vehicle 1. The display target identification unit 12 identifies an object that is not hindered by an obstacle from moving towards vehicle 1 as a detachment risk object that satisfies the detachment risk condition. The display target identification unit identifies an object among three or more target objects so that no other object is interposed between it and vehicle 1 as a detachment risk object that satisfies the detachment risk condition.

[0034] The display control unit 13 displays a group image representing the group on the HMI5's display, but does not display the target object on the HMI5's display. If a target object with a risk of separation is identified from among the target objects, the display control unit 13 displays both the group image and the target object with a risk of separation on the HMI5's display.

[0035] A group image is a graphic representation of multiple objects forming a group to be displayed, and is an image displayed to the occupants of vehicle 1 in place of individual, independent objects. Therefore, the objects corresponding to the group image are not displayed on the HMI 5 display in principle. However, if an object with a risk of separation is identified among the objects, that object will be displayed on the HMI 5 display as an exception. The group image may also be a predetermined graphic that corresponds to a closed region surrounding the multiple objects forming the group. The predetermined graphic may be, for example, a graphic with the same shape as the closed region surrounding the objects, or a simplified graphic such as a rectangle that includes part or all of the closed region surrounding the objects.

[0036] This section will provide a detailed explanation of how to identify and label target objects and objects at risk of detachment.

[0037] As a first example, the display target identification unit 12 identifies target objects that are not obstructed by obstacles in their movement towards vehicle 1 as objects with a risk of separation that satisfy the separation risk condition. The display target identification unit 12 may also identify, for example, target objects that are in a region closer to vehicle 1 than obstacles among a group of target objects forming a single group as objects with a risk of separation that satisfy the separation risk condition. Figure 2 is a diagram illustrating a first example of the identification of target objects and objects with a risk of separation. Figure 3 is a diagram illustrating a first example of the display of target objects and objects with a risk of separation. As shown in Figure 2, pedestrians 21-26 are present around vehicle 1 shown in plan view. Vehicle 1 is, for example, traveling on a roadway. Pedestrians 21, 22, 24-26 are on the sidewalk, and pedestrian 23 is on the roadway side of the guardrail 50 that separates the roadway and the sidewalk. Pedestrians 21-26 are within the detection range of the external sensor 2 mounted on vehicle 1 and are within a predetermined distance threshold.

[0038] In the situation shown in Figure 2, the object recognition unit 11 recognizes the positions of pedestrians 21-25 based on the detection results of the external sensor 2. Based on the positions of pedestrians 21-25, the object recognition unit 11 calculates the distance between each of the pedestrians 21-25. Based on the distances between the pedestrians, the object recognition unit 11 recognizes pedestrians 21-25 as an object forming a single group. In the example in Figure 2, it is assumed that the distances between pedestrians 21 and 22, 22 and 23, 23 and 24, 24 and 25, and 25 and 21 are all less than or equal to a predetermined distance. The distances of other combinations of pedestrians may also be less than or equal to the predetermined distance. In this case, the object recognition unit 11 recognizes a region G10 that corresponds to a single group composed of pedestrians 21-25. For example, if pedestrian 26 was recognized by the object recognition unit 11 before the situation in Figure 2 occurred, it may be treated as being included in region G10. The predetermined distance can be set appropriately through simulation or other means. The object recognition unit 11 may change the predetermined distance according to the number of pedestrians detected. For example, the object recognition unit 11 may shorten the predetermined distance as the number of detected pedestrians increases.

[0039] The shape of region G10 is set as a polygonal region enclosing pedestrians 21-23, which are recognized as being located in front of vehicle 1, and pedestrians 24 and 25, which are recognized as being located behind vehicle 1. For example, region G10 may be a region with the positions of pedestrians 21-23 as its vertices, and enclosed by an outer edge that forms part of a rectangle on the side opposite to vehicle 1 with respect to pedestrians 21-23. Part of the rectangle refers to a shape formed by connecting a side extending away from pedestrian 21 in the width direction of vehicle 1, a side extending away from pedestrian 23 in the direction of vehicle 1's movement, and a pair of opposite sides parallel to these two sides. The length of the side may be a predetermined fixed value, or it may be a length calculated according to the distance between each of the pedestrians 21-25. For example, the length of the side extending from pedestrian 21 in the width direction of vehicle 1 may be a predetermined coefficient multiple of the length from pedestrian 21 to pedestrian 23 in the width direction. The length of the side extending from the position of pedestrian 23 in the direction of travel of vehicle 1 may be a predetermined coefficient multiple of the length from the position of pedestrian 21 to the position of pedestrian 23 in the direction of travel.

[0040] The display target identification unit 12 identifies pedestrians 21-26, who form a group, as target objects from among the pedestrians 21-26 present around vehicle 1. The display target identification unit 12 identifies pedestrians 21-26 belonging to area G10 that satisfy the departure risk conditions regarding the risk of leaving area G10 in a way that approaches vehicle 1 as departure risk objects. In this case, pedestrian 23, who is located in front of the guardrail 50 as seen from vehicle 1, is considered to pose a risk of approaching vehicle 1. Pedestrian 23 is one of the target objects among the pedestrians 21-26 whose movement towards vehicle 1 is not obstructed by the guardrail 50. The guardrail 50 is an obstacle that obstructs the movement of the pedestrians 21-26, which are multiple target objects, towards vehicle 1. Pedestrian 23 is an object that satisfies the departure risk conditions regarding the risk of leaving area G10 in a way that approaches vehicle 1. The display target identification unit 12 identifies pedestrian 23 as a departure risk object. The display target identification unit 12 can identify the pedestrian 23 in Figure 2 as an object at risk of escaping, for example, based on the distance measurement result by the radar sensor between the guardrail 50 and the pedestrian 23. The display target identification unit 12 may also identify the pedestrian 23 in Figure 2 as an object at risk of escaping, based on the positional relationship between the guardrail 50 and the pedestrian 23 in the camera-captured image. The display target identification unit 12 generates the group image G11 in Figure 3 to correspond to region G10.

[0041] The display control unit 13 identifies pedestrian 23, who is a pedestrian risk object, from among pedestrians 21-26, and displays the group image G11 corresponding to region G10 and pedestrian 23, the pedestrian risk object, on the HMI5's display. As shown in Figure 3, the group image G11 represents a virtual region on the HMI5's display that surrounds the group of pedestrians 21-26, and can be, for example, a roughly rectangular shape that includes region G10 as a whole. The group image G11 may have rounded corners, and may also indicate that it is a group of pedestrians by displaying pedestrian icons that are larger than normal pedestrian images. The display control unit 13, for example, superimposes a pedestrian image representing pedestrian 23, the pedestrian risk object, onto the group image G11 and displays it on the HMI5's display, so that it corresponds to the coordinates of pedestrian 23 in the three-dimensional space around vehicle 1. As a result, pedestrians 21, 22, and 24-26, who are behind the guardrail 50, are grouped together as a group image G11 and are therefore not displayed on the HMI5 display. On the other hand, pedestrian 23, who is in front of the guardrail 50, is displayed independently on the HMI5 display as a detachment risk object. This makes it easier for the occupants of vehicle 1 to recognize the presence of pedestrian 23.

[0042] As a second example, the display target identification unit 12 may identify, among three or more target objects, an object that is positioned so that no other target objects are interposed between it and the vehicle 1 as a detachment risk object that satisfies the detachment risk condition.

[0043] Figure 4 is a diagram illustrating a second example of identifying the target object and the object at risk of detachment. Figure 5 is a diagram illustrating a second example of displaying the target object and the object at risk of detachment. As shown in Figure 4, pedestrians 21-26 are present on the sidewalk around vehicle 1 shown in a plan view. In the example in Figure 4, unlike the example in Figure 2, there is no guardrail 50 separating the roadway and the sidewalk.

[0044] In the situation shown in Figure 4, the object recognition unit 11 recognizes the positions of three or more target objects, namely pedestrians 21-25, based on the detection results of the external sensor 2. The object recognition unit 11 recognizes a region G20 that corresponds to a single group composed of pedestrians 21-25. The display target identification unit 12 identifies the pedestrians 21-26 that form a group among the pedestrians 21-26 present around the vehicle 1 as target objects.

[0045] The display target identification unit 12 generates group image G21 in Figure 5 and group image G22 in Figure 6 to correspond to region G20. The shape of region G20 here is the same as the shape of region G10. The shape of region G20 may be different from the shape of region G10.

[0046] The display target identification unit 12 identifies pedestrians 21 to 26 belonging to area G20 who meet the departure risk conditions regarding the risk of leaving area G20 in a manner that approaches vehicle 1 as departure risk objects. In this case, pedestrians 21 to 23 who are positioned so that no other pedestrians are between them and vehicle 1 are considered to be at risk of approaching vehicle 1. Pedestrians 21 to 23 are objects that meet the departure risk conditions regarding the risk of leaving area G20 in a manner that approaches vehicle 1. The display target identification unit 12 identifies pedestrians 21 to 23 who are positioned so that no other pedestrians are between them and vehicle 1, out of three or more pedestrians 21 to 26 belonging to area G20, as departure risk objects that meet the departure risk conditions.

[0047] The display target identification unit 12 may, for example, assume line segments (each dashed line segment in Figure 4) connecting each of the multiple pedestrians 21 to 25 to the vehicle 1, and determine whether the pedestrian is an "object positioned so that no other object is interposed between it and the vehicle 1" based on the length of the intersection of the line segment and the polygonal region G20 described above. For example, if the length of the part of the line segment that intersects region G20 (the part of each dashed line segment in Figure 4 that is inside region G20) is less than a predetermined length threshold, the display target identification unit 12 may determine that the target object is an "object positioned so that no other object is interposed between it and the vehicle 1". For example, if the length of the part of the line segment that intersects region G20 (the part of each dashed line segment in Figure 4 that is inside region G20) is greater than or equal to a predetermined length threshold, the display target identification unit 12 may determine that the target object is not an "object positioned so that no other object is interposed between it and the vehicle 1".

[0048] The predetermined length threshold is a length threshold used to determine whether an object is "located such that no other objects are interposed between it and vehicle 1". In other words, the predetermined length threshold is a threshold used to distinguish whether the object related to the line segment is located on the closest side or furthest side of the group from vehicle 1. In the example in Figure 4, if the positions of pedestrians 21-23 are on the outer edge of region G20, the predetermined length threshold can be a value greater than 0 and smaller than the length from the outer edge of region G20 to the positions of pedestrians 24 and 25 (the length on each dashed line in Figure 4). If the positions of pedestrians 21-23 are inside the outer edge of region G20, the predetermined length threshold can be a value greater than the length from the outer edge of region G20 to the positions of pedestrians 21-23 (the length on each dashed line in Figure 4) and smaller than the length from the outer edge of region G20 to the positions of pedestrians 24 and 25 (the length on each dashed line in Figure 4).

[0049] In the example shown in Figure 4, the length of the portion where each line segment for pedestrians 21-23 intersects region G20 is less than a predetermined length threshold, so the display target identification unit 12 determines that pedestrians 21-23 are "objects positioned so that no other objects are interposed between them and vehicle 1". The length of the portion where each line segment for pedestrians 24 and 25 intersects region G20 is greater than or equal to a predetermined length threshold, so the display target identification unit 12 determines that pedestrians 24 and 25 are not "objects positioned so that no other objects are interposed between them and vehicle 1".

[0050] The display control unit 13 identifies pedestrians 21-23, who are considered pedestrian risk objects, from among the pedestrians 21-26, and displays the group image G21 corresponding to region G20 and the pedestrian risk objects 21-23 on the HMI5 display. As shown in Figure 5, group image G21, like group image G11, represents a virtual region on the HMI5 display that surrounds the group of pedestrians 21-26. The display control unit 13 superimposes pedestrian images representing the pedestrian risk objects 21-23 onto group image G21 and displays them on the HMI5 display, for example, so that they correspond to the coordinates of pedestrians 21-23 in the three-dimensional space around vehicle 1. As a result, pedestrians 24-26, who are subject objects with other subject objects (pedestrians 21-23) between them and vehicle 1, are grouped together in group image G21 and are therefore not displayed on the HMI5 display. On the other hand, pedestrians 21-23, which are positioned so that no other objects are interposed between them and vehicle 1, are displayed independently on the HMI5's display as objects with a risk of separation. This makes it easier for the occupants of vehicle 1 to recognize the presence of pedestrians 21-23.

[0051] Figure 6 is a diagram illustrating a third example of the representation of the target object and the object at risk of detachment. In the example in Figure 6, compared to the examples in Figures 4 and 5, a pedestrian 24 approaching vehicle 1 at a predetermined speed threshold or higher is further identified as a detachment risk object that poses a risk of approaching vehicle 1. Pedestrian 24 is an object that satisfies the detachment risk condition regarding the risk of leaving area G20 in order to approach vehicle 1. The predetermined speed threshold is the speed threshold of an object used to determine whether or not to identify it as a detachment risk object. The predetermined speed threshold may be set in advance for each type of object. If the object is a pedestrian, the predetermined speed threshold can be set to a speed faster than a typical walking speed, and can be set to a speed equivalent to running.

[0052] The display control unit 13 identifies pedestrians 21-23 and 24, who are considered to be at risk of separation, from among the pedestrians 21-26, and therefore displays the group image G22 and the pedestrians 21-23 and 24, who are considered to be at risk of separation, on the display of the HMI 5. As shown in Figure 6, the group image G22, like the group image G21, represents a virtual area on the display of the HMI 5 that surrounds the group of pedestrians 21-26. In the group image G22, for example, considering the visibility of pedestrian 24, which is located near the center of the group image G22, a larger pedestrian icon than a normal pedestrian image may be displayed in an inconspicuous manner. The pedestrian icon is shown by a dashed line for convenience in Figure 6, but it does not have to be displayed. The display control unit 13, for example, superimposes images of pedestrians 21-23 and 24, which are pedestrians at risk of escaping, onto the group image G22 and displays them on the HMI 5's display, so that they correspond to the coordinates of pedestrians 21-23 and 24 in the three-dimensional space around the vehicle 1. As a result, in addition to pedestrians 21-23, which are positioned so that there are no other objects between them and the vehicle 1, pedestrian 24, which poses a risk of approaching the vehicle 1, is displayed independently as a pedestrian at risk of escaping. This makes it easier for the occupants of the vehicle 1 to recognize the presence of pedestrians 21-24.

[0053] Figure 7 is a diagram illustrating a third example of the identification of target objects and objects at risk of separation. In the example in Figure 7, several pedestrians 21-27 are present on the sidewalk around vehicle 1, and a sidewalk curb 51 is installed to separate the roadway from the sidewalk. A store entrance 52 is located in the middle of the sidewalk curb 51. For example, at the store entrance 52, a portion of the sidewalk curb 51 is lower than other portions, allowing vehicle 1 to pass through. The width of the portion of the store entrance 52 (the distance between the pair of dashed lines in the figure) is, for example, about the width of a single passenger car. In such a situation, the display target identification unit 12 may identify pedestrians 21-24, 26, and 27 as target objects forming a continuous group (region G30). Furthermore, instead of the sidewalk curb 51, the display target identification section 12 may identify pedestrians 21-24, 26, and 27 as objects forming a continuous group (region G30) even if the guardrail or planting is interrupted at the store entrance 52.

[0054] Figure 8 is a diagram illustrating a fourth example of identifying target objects and objects at risk of separation. In the example in Figure 8, there is a crossroads 53 in front of vehicle 1. There are several pedestrians 21, 22, and 26 on the sidewalk to the right of the crossroads 53, and several pedestrians 28 and 29 on the sidewalk to the right of the crossroads 53. A crossing road 54 extends to the right of the crossroads 53. The width of the crossing road 54 (the distance between the pair of solid lines in the figure) is, for example, greater than the width of two passenger cars. In such a situation, the display target identification unit 12 may identify pedestrians 21, 22, and 26, and pedestrians 28 and 29, as target objects that separately form two non-contiguous groups (regions G31 and G32).

[0055] In the examples shown in Figures 7 and 8, the display target identification unit 12 may recognize the store entrance 52 or the intersecting road 54 based on the detection results of the external sensor 2. Alternatively, the display target identification unit 12 may recognize the store entrance 52 or the intersecting road 54 based on the current vehicle position of the vehicle 1 measured by the GNSS receiver 3 and the map information in the map database 4.

[0056] [Processing of the display control device] Next, an example of the processing of the display control device 100 will be explained with reference to the flowcharts in Figures 9 and 10. Figure 9 is a flowchart of an example of the display control processing. The processing (steps) shown in Figure 9 are repeatedly executed at predetermined calculation cycles, for example, when the power supply of the ECU 10 is ON.

[0057] As shown in Figure 9, in S11, the ECU 10 of the display control device 100 performs object recognition of objects around the vehicle 1 using the object recognition unit 11 (object recognition step). The object recognition unit 11 recognizes objects that exist around the vehicle 1 within the detection range of the external sensor 2, for example, based on the detection results of the external sensor 2.

[0058] In S12, the ECU 10 sets the distance threshold using the display target identification unit 12. The display target identification unit 12 sets the distance threshold according to parameters stored in the ECU 10 in advance, for example.

[0059] In S13, the ECU 10 uses the display target identification unit 12 to determine whether or not an object exists within the distance threshold. The display target identification unit 12 compares the relative distance of the object from the vehicle 1 recognized by the object recognition unit 11 with the distance threshold. The display target identification unit 12 determines that an object exists within the distance threshold if the relative distance is less than or equal to the distance threshold. The display target identification unit 12 determines that an object does not exist within the distance threshold if the relative distance is greater than the distance threshold.

[0060] If it is determined that an object exists within the distance threshold (S13: YES), ECU10 proceeds to process S14. If it is determined that an object does not exist within the distance threshold (S13: NO), ECU10 terminates the process shown in Figure 9. Note that processes S12 and S13 are not mandatory. If processes S12 and S13 are omitted, ECU10 may proceed to S14 after processing S11.

[0061] In S14, the ECU 10 uses the display target identification unit 12 to identify objects that form a group from among multiple objects present around the vehicle as target objects (display target identification step). As an example, the display target identification unit 12 identifies objects that form a group from among multiple objects present within the detection range of the external sensor 2, whose relative distance to the vehicle 1 is less than or equal to a distance threshold, as target objects.

[0062] In S15, the ECU 10 uses the display target identification unit 12 to identify target objects that meet the separation risk conditions as separation risk objects (display target identification step). The display target identification unit 12 identifies target objects that meet the separation risk conditions regarding the risk of separating from the group so as to approach vehicle 1 as separation risk objects. The ECU 10 then executes the process shown in Figure 10, for example.

[0063] Figure 10 is a flowchart showing an example of the display target identification process. The process shown in Figure 10 (display target identification step) is repeatedly executed at predetermined calculation cycles, for example, when the power supply of the ECU 10 is ON.

[0064] As shown in Figure 10, in S21, the ECU 10 of the display control device 100 determines, using the display target identification unit 12, whether or not there is an obstacle that would hinder the movement of the target object towards the vehicle. The display target identification unit 12 recognizes, for example, an object (e.g., a guardrail 50) among the objects around the vehicle 1 as an obstacle, based on the type, position, and shape of the object recognized by the object recognition unit 11, etc., which would hinder the movement of the target object towards the vehicle 1. The obstacle may be a stationary object or a moving object.

[0065] In S22, the ECU 10 uses the display target identification unit 12 to identify target objects that are not obstructed by obstacles in their movement towards vehicle 1 as objects with a risk of separation that satisfy the separation risk condition. In the example in Figure 2, the display target identification unit 12 identifies pedestrian 23, who is located in area 50a, which is closer to vehicle 1 than the guardrail 50, as an object with a risk of separation that satisfies the separation risk condition, among the pedestrians 21 to 26 that form area G10. After that, the ECU 10 finishes the process in Figure 10 and moves on to the process in S16 in Figure 9.

[0066] Meanwhile, in S23, the ECU 10 uses the display target identification unit 12 to determine whether or not there are three or more target objects. If it is determined that there are three or more target objects (S23: YES), the ECU 10 proceeds to the process in S24. If it is determined that there are no three or more target objects (S23: NO), the ECU 10 proceeds to the process in S26.

[0067] In S24, the ECU 10 uses the display target identification unit 12 to determine, for each of the three or more target objects, whether or not the target object is positioned so that no other target objects are interposed between it and the vehicle. For example, the display target identification unit 12 may assume a line segment connecting each of the multiple target objects to the vehicle 1, and determine whether or not the target object related to the line segment is positioned so that no other target objects are interposed between it and the vehicle 1, based on the length of the intersection between the line segment and the polygonal region G20 described above.

[0068] The display target identification unit 12 determines, for example, that if the length of the portion of the line segment that intersects region G20 (the portion of each dashed line segment in Figure 4 that lies inside region G20) is less than a predetermined length threshold, the target object related to the line segment is an object that is positioned so that no other object is interposed between it and the vehicle 1. The display target identification unit 12 determines, for example, that if the length of the portion of the line segment that intersects region G20 (the portion of each dashed line segment in Figure 4 that lies inside region G20) is greater than or equal to a predetermined length threshold, the target object related to the line segment is not an object that is positioned so that no other object is interposed between it and the vehicle 1.

[0069] If, for each of the three or more target objects, it is determined that the target object is positioned so that no other target objects are interposed between it and the vehicle (S24:YES), the ECU10 proceeds to process S25. If, for each of the three or more target objects, it is determined that the target object is not positioned so that no other target objects are interposed between it and the vehicle (S24:NO), the ECU10 proceeds to process S26.

[0070] In S25, the ECU 10 uses the display target identification unit 12 to identify, among three or more target objects, those that are positioned so that no other target objects are interposed between them and the vehicle, as objects with a risk of separation that satisfy the separation risk condition. In the example in Figure 4, the display target identification unit 12 identifies pedestrians 21 to 23, among pedestrians 21 to 26 belonging to area G20, as objects with a risk of separation that satisfy the separation risk condition, since they are positioned so that no other target objects are interposed between them and the vehicle 1. After that, the ECU 10 finishes the process in Figure 10 and moves on to the process in S16 in Figure 9.

[0071] On the other hand, in S26, the ECU 10 does not identify the target object as a detachment risk object that satisfies the detachment risk conditions using the display target identification unit 12. Subsequently, the ECU 10 terminates the process shown in Figure 10 and proceeds to the process shown in S16 of Figure 9.

[0072] Returning to the process in Figure 9, in S16, the ECU 10 determines whether or not a detachment risk object has been identified from the target objects by the display control unit 13 (display control step). The display control unit 13 determines whether or not a detachment risk object has been identified from the target objects according to the result of the detachment risk object identification by the display target identification unit 12. If it is determined that a detachment risk object has been identified from the target objects (S16: YES), the ECU 10 proceeds to the process in S17. If it is determined that no detachment risk object has been identified from the target objects (S16: NO), the ECU 10 proceeds to the process in S18.

[0073] In S17, the ECU 10 displays the group image and the detachment risk object on the display unit via the display control unit 13 (display control step). The display control unit 13, for example, displays the detachment risk object superimposed on the group image on the HMI 5's display. After that, the ECU 10 terminates the process shown in Figure 9.

[0074] Meanwhile, in S18, the ECU 10 displays the group image on the display unit via the display control unit 13 (display control step). Since the detachable risk object has not been identified, the display control unit 13 displays only the group image on the HMI 5's display. After that, the ECU 10 terminates the process shown in Figure 9.

[0075] [Display control program] The display control program causes the ECU 10 (computer) to function (operate) as the object recognition unit 11, the display target identification unit 12, and the display control unit 13 described above. The display control program is provided, for example, by a non-temporary recording medium such as ROM or semiconductor memory. Alternatively, the display control program may be provided via communication such as a network.

[0076] According to the display control device 100, the control method for the display control device, and the display control program described above, pedestrians (target objects) that meet the escape risk conditions are identified as escape risk objects. When escape risk objects are identified from among the target objects, the group image and the escape risk objects are displayed on the HMI5. As a result, in the example in Figure 3, not only the group image G11 but also pedestrian 23 is displayed on the HMI5. In the example in Figure 5, not only the group image G21 but also pedestrians 21-23 are displayed on the HMI5. In the example in Figure 6, not only the group image G22 but also pedestrians 21-24 are displayed on the HMI5. This makes it easier for the occupants of vehicle 1 to recognize that pedestrians 23, 21-23, and 21-24, which are escape risk objects that may leave the group images G11, G21, and G22 in order to approach vehicle 1, are present in the group, compared to the case where only the group images G11, G21, and G22 are displayed on the HMI5.

[0077] According to the display control device 100, the control method for the display control device, and the display control program, for example, in Figure 2, based on the detection results of the external sensor 2, the guardrail 50 that obstructs the movement of pedestrians 21, 22, 24-26 towards vehicle 1 is recognized. Among the pedestrians 21-26, pedestrian 23 whose movement towards vehicle 1 is not obstructed by the guardrail 50 is identified as an escape risk object that meets the escape risk conditions. As a result, as shown in Figure 3, the HMI 5 displays not only the group image G11 but also pedestrian 23. This makes it easier for the occupants of vehicle 1 to recognize pedestrian 23, whose movement towards vehicle 1 is not obstructed by the guardrail 50, as an escape risk object.

[0078] According to the display control device 100, the control method for the display control device, and the display control program, for example, in Figure 4, three or more target objects, namely pedestrians 21-26, are recognized. Of the three or more pedestrians 21-26, pedestrians 21-23 that are positioned so that no other target objects are interposed between them and vehicle 1 are identified as escape risk objects that satisfy the escape risk condition. As a result, as shown in Figure 5, not only the group image G21 but also pedestrians 21-23 are displayed on the HMI 5. In other words, by grouping the target objects in the background, it is possible to display the target objects in the foreground of the group independently without displaying the target objects in the background. This makes it easier for the occupants of vehicle 1 to recognize pedestrians 21-23, which are positioned so that no other target objects are interposed between them and vehicle 1, as escape risk objects.

[0079] Although embodiments of the present disclosure have been described above, the present disclosure is not limited to the embodiments described above.

[0080] In the above embodiment, Figures 2 and 3 show an example in which the display target identification unit 12 identifies objects belonging to the display target group that are in an area closer to the vehicle 1 than to the obstacle as objects with a risk of separation that satisfy the separation risk condition, but it is not limited to this example. The display target identification unit 12 only needs to identify objects that are not prevented from moving towards the vehicle 1 by the obstacle as objects with a risk of separation that satisfy the separation risk condition. Furthermore, the obstacle is not limited to the guardrail 50, but may be a planted area, or the glass exterior wall of the first floor of a building (for example, if there is a showroom or other facility inside the room beyond the glass). Also, the obstacle is not limited to an object that extends along the direction of travel of the vehicle 1, but may be an object that extends along the width direction of the vehicle 1, or an object that extends in a curved shape.

[0081] In the above embodiment, Figures 4 to 6 show an example of identifying an object among three or more target objects that is positioned so that no other target objects are interposed between it and the vehicle 1 as an object with a risk of separation that satisfies the separation risk condition. However, the embodiment is not limited to this example. For example, by assuming line segments connecting each of pedestrians 24 and 25 to the vehicle 1, it was determined that pedestrians 23 and 22 are interposed between the vehicle 1 as other objects based on the length of the intersection of these line segments and regions G10 and G20. However, other methods may be used to determine whether or not an object is positioned so that no other target objects are interposed between it and the vehicle 1. For example, the display target identification unit 12 may determine whether or not an object is positioned so that no other target objects are interposed between it and the vehicle 1 based on the image captured by the camera of the external sensor 2, according to the degree to which pedestrians 23 and 22 and pedestrians 24 and 25 overlap as seen from the vehicle 1.

[0082] In the above embodiment, the display target identification unit 12 defines polygonal regions G10 and G20 as closed regions surrounding the target object, with the vertices being the positions of the target objects located on the vehicle 1 side among the multiple target objects. However, the shape of the closed region in plan view is not limited to these examples. The closed region may be a regular polygon with the vertices being the positions of the target objects located on the vehicle 1 side among the multiple target objects, or it may be a circle or ellipse passing through the positions of the target objects located on the vehicle 1 side among the multiple target objects. [Explanation of Symbols]

[0083] 1...Vehicle, 2...External sensor, 10...ECU (computer), 11...Object recognition unit, 12...Display target identification unit, 13...Display control unit, G11, G21, G22...Group image, 50a...Region, 100...Display control device.

Claims

1. An object recognition unit recognizes objects around the vehicle based on the detection results of external sensors on board the vehicle, A display target identification unit that identifies a group of objects from among the multiple objects present around the vehicle as target objects, The system includes a display control unit that displays a group image representing the group on the display unit, but does not display the target object on the display unit, The display target identification unit identifies the target objects that satisfy the departure risk conditions regarding the risk of leaving the group so as to approach the vehicle as departure risk objects. The display control unit, when the object at risk of separation is identified from the target objects, causes the group image and the object at risk of separation to be displayed on the display unit.

2. The object recognition unit recognizes, based on the detection results of the external sensor, an obstacle that hinders the movement of the target object towards the vehicle. The display control device according to claim 1, wherein the display target identification unit identifies, among the target objects, those target objects whose movement towards the vehicle is not obstructed by the obstacle, as the object with a risk of separation that satisfies the separation risk condition.

3. The display control device according to claim 1 or 2, wherein the display target identification unit identifies, among the three or more target objects, the target object that is positioned so that no other target objects are interposed between it and the vehicle, as the target object that satisfies the target object that meets

4. A control method for a display control device that displays an image on an in-vehicle display unit, An object recognition step in which the display control device recognizes objects around the vehicle based on the detection results of an external sensor on board the vehicle, The display control device performs a display target identification step of identifying a group of objects that form a cluster among a plurality of objects present around the vehicle as target objects, The display control device includes a display control step of displaying a group image representing the group on the display unit while not displaying the target object on the display unit, In the step of identifying the object to be displayed, the object that satisfies the departure risk conditions regarding the risk of leaving the group so as to approach the vehicle is identified as a departure risk object. A control method for a display control device, wherein, in the display control step, if the object at risk of separation is identified from the target objects, the group image and the object at risk of separation are displayed on the display unit.

5. A display control program that causes a computer to function as a display control device for displaying an image on an in-vehicle display unit, wherein the display control program causes the computer to function as a display control device for displaying an image on an in-vehicle display unit, An object recognition unit recognizes objects around the vehicle based on the detection results of external sensors on board the vehicle. A display target identification unit that identifies a group of objects from among the multiple objects present around the vehicle as target objects, and The display control unit functions as a unit that displays a group image representing the group on the display unit, but does not display the target object on the display unit. The display target identification unit identifies the target objects that satisfy the departure risk conditions regarding the risk of leaving the group so as to approach the vehicle as departure risk objects. The display control unit is a display control program that, when a detachment risk object is identified from the target objects, causes the group image and the detachment risk object to be displayed on the display unit.

Images