Vehicle control device and vehicle control system

Abstract

To improve availability while securing safety in a vehicle control system for performing drive support and automatic drive in backlight environment.SOLUTION: A vehicle control device mounted on a vehicle includes: a solar position calculation unit for calculating position information of the sun at a predetermined time; an invisible area calculation unit for calculating an invisible area in which it is difficult to visually recognize a visual recognition target object from the position information of the sun and the position information of the visual recognition target object; and an invisible area output unit for outputting the calculated invisible area. The vehicle control device further includes an invisible area acquisition unit for acquiring information on the invisible area, and a vehicle control information output unit for identifying the invisible area on a route of the vehicle based on the acquired information on the invisible area to output control information on the vehicle based on the identified invisible area.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004]

[0001] The present invention relates to a vehicle control device and a vehicle control system.

Background Art

[0002] A vehicle control system mounted on a vehicle for performing driving support or autonomous driving recognizes the situation around the vehicle based on information acquired from a sensor device, and performs driving support or autonomous driving based on the situation around it. In recognizing the surrounding situation by utilizing a camera device which is one of the sensor devices, there is a performance limit such that part of the captured image becomes white due to backlight, making it impossible to recognize objects around the vehicle. When this performance limit occurs when recognizing the color of a signal by utilizing the camera device, it may lead to dangerous situations such as accidents. The vehicle control system is required to avoid such dangerous situations in a backlight environment. To ensure safety, the vehicle control system is required to avoid an environment where the performance limit occurs, or to take measures so that there is no problem even if the performance limit occurs.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 discloses a technique for searching for a route in which the angular difference between the direction of the forward camera's shooting and the direction of the sun's position remains outside a predetermined range. The predetermined range is the range of angular difference in which it is estimated that the surrounding environment cannot be recognized due to backlighting. Patent Document 1 ensures safety by avoiding dangerous areas where the camera's shooting direction and the sun's direction fall within the predetermined range. However, following Patent Document 1 would significantly reduce the usability of the vehicle control system, for example, by causing all roads traveling westward in the evening to fall outside the range.

[0005] Patent Document 2 discloses a technology that defines a dangerous area where traffic lights are difficult to recognize due to backlighting, etc., and determines whether the camera's line of sight to the traffic lights outside of that dangerous area is obstructed by surrounding vehicles. Patent Document 2 mentions a dangerous area where traffic lights cannot be recognized, but it does not disclose how to define the location of this dangerous area. To ensure safety, it is desirable to define the dangerous area more than necessary, but defining it too broadly will reduce the usability of the vehicle control system. Furthermore, the position of the sun changes constantly, and the dangerous area due to backlighting also changes constantly. Patent Document 2 mentions the association between the dangerous area and the time of day, but it does not disclose how to define the location of the dangerous area due to backlighting in accordance with the time of day. The area that may be dangerous in the evening may be defined to be sufficiently broad, but this will reduce the usability of the vehicle control system.

[0006] This invention was made to solve the above problems and takes into consideration an object that must be seen by a sensor device mounted on a vehicle or the driver of the vehicle (hereinafter referred to as the "seeing entity") (hereinafter referred to as the "object to be seen"). An example of an object to be seen is a traffic light. [Means for solving the problem]

[0007] In this invention, a vehicle control system or a vehicle control device included in the vehicle control system uses positional information of an object to be observed and a predetermined timeThe system calculates the position of the sun. Based on the calculated sun position information and the position information of the object to be seen, it calculates the area on the road where the object to be seen is difficult to see due to backlighting (hereinafter referred to as the "invisibility zone"). Furthermore, safety is ensured by controlling the vehicle's movement or backlighting countermeasures based on the calculated invisibility zone. For example, the vehicle's speed is adjusted so that the vehicle's position at a time when the object to be seen must be seen, such as when entering an intersection, does not fall within the invisibility zone. For example, if it is determined that the sun is to the right of the object to be seen from the perspective of the observer within the invisibility zone, the sun visor is controlled so that it aligns with the sun's position from the perspective of the observer.

[0008] In this invention, it is possible to choose not to calculate the area of ​​obstruction of visibility by determining whether sunlight is blocked based on weather information and map information. For example, the area of ​​obstruction of visibility is not calculated when it is cloudy, rainy, or when it is determined that a tall building or the like is behind the object to be seen. [Effects of the Invention]

[0009] According to the present invention, by accurately calculating a dangerous area where the target object is difficult to see due to backlighting, given a predetermined time and the target object, it is possible to improve the availability of the vehicle control system while ensuring its safety in backlit environments. Furthermore, by taking countermeasures based on the information of that dangerous area, the safety of the vehicle control system can be improved.

[0010] Further features related to the present invention will become apparent from the description herein and the accompanying drawings. Problems, configurations, and effects other than those described above will be revealed by the following description of embodiments. [Brief explanation of the drawing]

[0011] [Figure 1] A functional block diagram showing the configuration of a vehicle control system, including a vehicle control device according to a first embodiment of the present invention. [Figure 2] An example of information from a data set of objects that can be observed. [Figure 3]An example of information on a non-visible area data group. [Figure 4] Processing flow of the sun position calculation unit. [Figure 5] Processing flow of the non-visible area calculation unit. [Figure 6] An example of a visible object and the surrounding area of the visible object. [Figure 7] An example of a method for calculating a non-visible area. [Figure 8] Processing flow of the vehicle control information generation unit. [Figure 9] Specific example of a part of the processing flow of the vehicle control information generation unit (relating to the backlight countermeasure device). [Figure 10] Specific example of a part of the processing flow of the vehicle control information generation unit (relating to adjusting the stop position of the vehicle). [Figure 11] Specific example of a part of the processing flow of the vehicle control information generation unit (relating to adjusting the speed of the vehicle). [Figure 12] Functional block diagram showing the configuration of a vehicle control system according to the third embodiment of the present invention. [Embodiments for Carrying Out the Invention]

[0012] Hereinafter, a first embodiment of a vehicle control device will be described with reference to FIGS. 1 to 11.

[0013] [First Embodiment] (System Configuration) FIG. 1 is a functional block diagram showing the configuration of a vehicle control system 101 including a vehicle control device 110 according to the first embodiment of the present invention. The vehicle control system 101 is mounted on a vehicle 102. The vehicle control system 101 is a system that recognizes the situation of the driving road and surrounding vehicles around the vehicle 102 and performs driving support and driving control of the vehicle 102. The vehicle control system 101 includes a vehicle control device 110, a map information management device 111, a sensor device group 112, an actuator group 113, and a backlight countermeasure device group 114. The vehicle control device 110, the map information management device 111, the sensor device group 112, the actuator group 113, and the backlight countermeasure device group 114 are connected to each other by an in-vehicle network.

[0014] The vehicle control device 110 is an ECU (Electronic Control Unit). Based on various input information provided from a map information management device 111, a sensor device group 112, etc., the vehicle control device 110 generates driving support or driving control information for automatic driving of the vehicle 102, and generates control information for the backlight countermeasure device group 114. Then, the vehicle control device 110 outputs control information to an actuator group 113, the backlight countermeasure device group 114, etc.

[0015] The vehicle control device 110 includes a processing unit 120 and a storage unit 121. The processing unit 120 is configured to include, for example, a CPU (Central Processing Unit) which is a central processing unit. However, in addition to the CPU, it may be configured to include a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc., or may be configured by any one of them. The storage unit 121 is configured to include, for example, a storage device such as an HDD (Hard Disk Drive), a flash memory, a ROM (Read Only Memory), and a memory such as a RAM (Random Access Memory). In the storage unit 121, programs processed by the processing unit 120, data groups necessary for the processing, etc. are stored. Also, the storage unit 121 is used for the purpose of temporarily storing data necessary for program operations as the main memory when the processing unit 120 executes a program.

[0016] The map information management device 111 is a device that manages and provides digital map information about the area around the vehicle 102, information about the vehicle 102's travel route, and information about objects that can be seen. The map information management device 111 is composed of, for example, a navigation device. The map information management device 111 includes, for example, digital road map data for a predetermined area including the area around the vehicle 102, and is configured to identify the current position of the vehicle 102 on the map, i.e., the road and lane that the vehicle 102 is traveling on, based on the position information of the vehicle 102 output from the sensor device group 112. In addition, the map information management device 111 includes information about objects that can be seen, and these objects are associated with, for example, various object information (traffic lights, landmarks, etc.) in the digital road map data. Furthermore, the map information management device 111 outputs the identified current position of the vehicle 102, the map data of its surroundings, route information, and information about objects that can be seen to the vehicle control device 110 via the in-vehicle network.

[0017] The sensor group 112 is a collection of devices that detect the surrounding conditions of the vehicle 102 and various states of the vehicle 102. Examples of the sensor group 112 include cameras, millimeter-wave radar, LiDAR, sonar, and wheel odometry. The sensor group 112 outputs observation information from sensing the area around the vehicle 102, as well as information on environmental elements such as obstacles, road markings, signs, and signals identified based on that observation information, to the vehicle control device 110 via the in-vehicle network. "Obstacles" include, for example, other vehicles other than the vehicle 102, pedestrians, objects that have fallen onto the road, and the roadside. In addition, the sensor group 112 detects the vehicle 102's position information, driving speed, steering angle, accelerator operation amount, brake operation amount, etc., and outputs them to the vehicle control device 110 via the in-vehicle network.

[0018] The actuator group 113 is a group of devices that control control elements such as steering, brakes, and accelerators that determine the movement of the vehicle. The actuator group 113 controls the movement of the vehicle based on operation information from the driver, such as the steering wheel, brake pedal, and accelerator pedal, and control command values ​​output from the vehicle control device 110.

[0019] The group of devices 114 for countering glare are in-vehicle devices that block or reduce sunlight (direct sunlight) directed at the subject of vision in the vehicle within a specified area of ​​blindness. The group of devices 114 for countering glare includes, for example, an electric lens hood for the sensor device group 112, an electric sun visor for the driver of the vehicle 102, and a dimmable glass device that changes the transmittance of sunlight.

[0020] The processing unit 120 included in the vehicle control device 110 has the following functions: map information acquisition unit 130, sun position calculation unit 131, blind area calculation unit 132, blind area output unit 133, vehicle surrounding situation recognition unit 134, blind area acquisition unit 135, vehicle control information generation unit 136, and vehicle control information output unit 137. The processing unit 120 achieves these functions by executing a predetermined operation program stored in the memory unit 121.

[0021] The map information acquisition unit 130 acquires information from the map information management device 111 via the in-vehicle network and stores it in the map data group 141 and the visible object data group 142 of the storage unit 121.

[0022] The solar position calculation unit 131 calculates the position of each visible object along the vehicle 102's route and the position information of the sun at a predetermined time, based on the map data group 141 and the visible object data group 142. The predetermined time is, for example, the time when the vehicle 102 is expected to pass around a visible object, based on the speed of the vehicle 102 obtained from the sensor data group 144 and the distance along the route to the visible object obtained from the map data group 141. Alternatively, it is the time when the vehicle 102 is expected to pass around a visible object, based on the distance along the route to the visible object and the legal maximum speed information of the road along the route, based on the map data group 141. Alternatively, it is the current time. Alternatively, it is the time after a predetermined time has elapsed from the current time. Furthermore, the predetermined time may be expressed as a single time, or as time information with a width composed of two values: a start time and an end time. The solar position calculation unit 131 calculates a single position if the predetermined time is a single time, and if the time information is expressed as two values, a start time and an end time, it expresses the range of positions the sun can take within that time.

[0023] The invisible area calculation unit 132 calculates an invisible area at a predetermined time and at a predetermined location of a predetermined object, based on the position information of the sun calculated by the sun position calculation unit 131 and the position information of the object to be seen included in the object to be seen data group 142. The invisible area calculation unit 132 then stores the result in the invisible area data group 143.

[0024] The invisible area output unit 133 outputs the information stored in the invisible area data group 143. If the output is to the same vehicle control device 110, the output may be a pointer to the invisible area data group 143 instead of the invisible area information itself.

[0025] The vehicle surroundings situation recognition unit 134 recognizes the conditions of the road and other vehicles around the vehicle 102 based on information from the map data group 141 and the sensor data group 144. The vehicle surroundings situation recognition unit 134 is implemented using known technologies in driver assistance and autonomous driving.

[0026] The invisible area acquisition unit 135 acquires information about the invisible area from the invisible area output unit 133 and stores it in the invisible area data group 143 if necessary. If the invisible area output unit 133 outputs pointer information for the invisible area data group 143, it is sufficient to simply retain that pointer information.

[0027] The vehicle control information generation unit 136 first generates driving control information necessary for driver assistance and autonomous driving based on the conditions around the vehicle 102 recognized by the vehicle surrounding conditions recognition unit 134, and stores it in the driving control data group 145. The generation of this driving control information is achieved using known technologies in driver assistance and autonomous driving.

[0028] Furthermore, the vehicle control information generation unit 136 updates the driving control information included in the driving control data group 145 based on the conditions around the vehicle 102, the map data group 141, and the blind area data group 143. Specifically, it determines whether the vehicle 102 will pass through a blind area based on the route information included in the map data group 141 and the driving control data group 145. If it determines that the vehicle will pass through, it considers driving control to prevent dangerous situations in the blind area and updates the driving control data group 145. For example, if the vehicle 102 is to stop in a blind area and must see a target object at that time, the driving control data group 145 is updated to stop before or beyond the blind area. For example, if the vehicle 102 traveling at a predetermined speed must see a target object in a blind area, the driving control data group 145 is updated to accelerate or decelerate the speed of the vehicle 102.

[0029] In addition, if the vehicle control information generation unit 136 determines that the vehicle 102 is passing through an area where visibility is impossible, it may generate control information to control the group of backlight countermeasure devices 114 so that the viewer can see the target object by blocking or reducing the backlight in the area where visibility is impossible. In this case, the vehicle control information generation unit 136 stores the generated control information in the group of backlight countermeasure device control data 146. For example, if the viewer is the driver of the vehicle 102, and the sun is to the right of the target object as viewed from the viewer, subject Generate information to control the movement of the electric sun visor so that it overlaps with the area on the right side of the object.

[0030] The processing of the vehicle control information generation unit 136 is not limited to the above description. For example, instead of updating the information of the driving control data group 145 necessary for driver assistance or autonomous driving based on information of the blind area after generating the information of the driving control data group 145, the information of the driving control data group 145 may be generated considering the blind area from the beginning. In addition, other processing necessary for safe driver assistance or autonomous driving may be added based on information of the blind area.

[0031] The vehicle control information output unit 137 outputs the driving control information included in the driving control data group 145 to the actuator group 113. The vehicle control information output unit 137 also outputs the control information included in the backlight countermeasure device control data group 146 to the backlight countermeasure device group 114. In other words, the vehicle control information output unit 137 outputs information to activate the backlight countermeasure device group 114 as vehicle control information.

[0032] The memory unit 121 includes map data group 141, visible object data group 142, invisible area data group 143, sensor data group 144, driving control data group 145, and backlight countermeasure device control data group 146.

[0033] Map data set 141 is a collection of digital map information about the area around vehicle 102 and data related to the vehicle's travel route, output by the map information management device 111. Map data set 141 includes road information, lane information, sign information, traffic light information, landmark information, etc. This information is represented using known techniques.

[0034] The object data group 142 is a collection of data about objects to be seen output by the map information management device 111. The object data group 142 includes information such as the ID, location, size, direction, type, and corresponding road of the object to be seen. The type indicates what kind of object the object to be seen is, for example, a traffic light. The corresponding road is information about the road that corresponds to the object to be seen. For example, if the object to be seen is a traffic light, the traffic light in the straight direction at an intersection will be seen, and the traffic light in the orthogonal direction will not be seen. In this way, it includes information that associates objects to be seen in order to identify them according to the road. The configuration of the object data group 142 is not limited to Figure 1 and the above description. For example, the object data group 142 may be included in the map data group 141. For example, it may be expressed by adding flag information indicating that an object to be seen to various object information such as traffic lights included in the map data group 141.

[0035] The invisible area data group 143 is a collection of data relating to invisible areas. The invisible area data group 143 includes, for example, the ID of the invisible area, the ID of the corresponding object to be seen, and position information of the vertices that constitute the invisible area. The information in the invisible area data group 143 is generated and stored by the invisible area calculation unit 132.

[0036] The sensor data group 144 is a collection of data relating to detection information from the sensor device group 112. The detection information includes, for example, information on environmental elements such as obstacles, road markings, signs, and signals identified based on the sensing observation information of the sensor device group 112, as well as information such as the position of the vehicle 102, driving speed, steering angle, accelerator operation amount, and brake operation amount.

[0037] The driving control data group 145 is a collection of data related to planning information for controlling the movement of the vehicle 102, and includes the planned trajectory of the vehicle 102, control command values ​​to be output to the actuator group 113, etc. The information in the driving control data group 145 is generated and stored by the vehicle control information generation unit 136.

[0038] The backlight countermeasure device control data group 146 is a collection of data related to the control of the backlight countermeasure device group 114, and includes control command values ​​to be output to the backlight countermeasure device group 114. The information in the backlight countermeasure device control data group 146 is generated and stored by the vehicle control information generation unit 136.

[0039] (Example of data structure) Figure 2 shows an example of information related to the data set 142 of the objects to be observed.

[0040] Column 201 is an ID that uniquely identifies the object being observed. Column 202 is a value that indicates the type of object being observed. Examples of values ​​indicated in Column 202 include those corresponding to traffic lights, signs, electronic display boards, etc.

[0041] Columns 203, 204, and 205 contain information indicating the position, size, and orientation of the object being observed, respectively. Column 203 consists, for example, of horizontal position, vertical position information, and altitude information in map data group 141. Column 203 may be expressed in the same coordinate system as the coordinate system represented in map data group 141, or in a different coordinate system that is one-to-one associated with that coordinate system. Column 204 consists of information regarding the width, depth, and height of the object being observed. Column 205 is expressed as an angle relative to a predetermined direction or coordinate axis in map data group 141.

[0042] Column 206 contains information identifying the road associated with the object being observed, and is set to the road ID stored in the map data set 141. Column 206 may be set to multiple values ​​to accommodate cases where multiple roads are associated, or a lane ID may be set as a value to identify the associated lane.

[0043] Information regarding the object to be observed is not limited to what is shown in Figure 2. For example, in the case of a traffic light, information on sub-types that identify pedestrian signals, signals with right-turn arrows, etc., may be included. For example, in order to associate the object to be observed with the object information of each object in the map data group 141, information on the ID of each object included in the map data group 141 may be included. Furthermore, as mentioned above, instead of preparing the table shown in Figure 2, the same information as in Figure 2 may be expressed by adding a flag to the various object information included in the map data group 141 to indicate that it is an object to be observed.

[0044] Figure 3 shows an example of information regarding the invisible area data group 143. Column 301 is an ID for uniquely identifying the invisible area. Column 302 is set to one of the values ​​present in column 201 of the visible object data group 142, associating the visible object with the invisible area. In Figure 2, if, instead of preparing the visible object data group 142, flag information indicating that an object is a visible object is added to the various object information in the map data group 141, the ID of that various object information is set here.

[0045] Column 303 is the number of vertices included in each invisible region. Columns 304-307 are the position information of each vertex when the number of vertices is 4. Columns 304-307 consist, for example, of horizontal position information and altitude information in map data group 141. Columns 304-307 may be expressed in the same coordinate system as the coordinate system represented in map data group 141, or in a different coordinate system that is one-to-one associated with that coordinate system.

[0046] The information regarding the invisible area data set 143 is not limited to what is shown in Figure 3. For example, if there are 5 or more invisible areas... Having a vertex Assuming it is a polygon, it may include information about five or more vertices.

[0047] (Calculation of areas that cannot be seen) The process of calculating the invisible area, performed by the solar position calculation unit 131 and the invisible area calculation unit 132, will be explained using Figures 4 to 7.

[0048] Figure 4 shows the processing flow of the solar position calculation unit 131. The solar position calculation unit 131 refers to the map data group 141 and the visible object data group 142 to obtain all visible objects along the route of the vehicle 102 (step S401). For example, if the route information of the vehicle 102 is given as a list of road IDs, the objects may be obtained by searching the list of road IDs and column 206 of the visible object data group 142. The same search is possible even if the route information is a list of lane IDs and column 206 of the visible object data group 142 is a lane ID. Alternatively, if the route information of the vehicle 102 is given as a list of road IDs, the location information of the road area may be obtained from the list of road IDs and the map data group 141, and the visible objects located within a predetermined distance from that location may be obtained.

[0049] Then, the solar position calculation unit 131 selects one of the acquired visible objects (step S402). The following steps S403 to S404 are performed on the visible object selected here (hereinafter referred to as the selected visible object).

[0050] The solar position calculation unit 131 sets the time when the vehicle 102 will see the selected target object (step S403). For example, it obtains the speed of the vehicle 102 from the sensor data group 144 and predicts and sets the time when the vehicle 102 will pass near the selected target object based on that speed or the average speed calculated therefrom and the route information of the vehicle 102 included in the map data group 141. Alternatively, it may predict and set the time when the vehicle 102 will pass near the selected target object based on the route information of the vehicle 102 included in the map data group 141 and the legal maximum speed information of the roads along that route. Alternatively, it may set the current time assuming that the selected target object is in the vicinity of the vehicle 102. Alternatively, it may set a time after a predetermined time has elapsed from the current time. Furthermore, the time may be expressed as a single time, or it may be expressed as time information with a range based on two pieces of information: a start time and an end time.

[0051] Next, the solar position calculation unit 131 calculates the position of the sun based on this time information and the position information of the selected object to be observed (step S404). This calculation method is implemented using known techniques, and the calculated position of the sun is expressed, for example, in terms of elevation angle and azimuth. Furthermore, the representation of the solar position information calculated in step S404 differs depending on the representation of the time information in step S403. If the time information is expressed as a single time, the position of the sun corresponding to that time is expressed as a single position. If the time information is expressed as information with a range of start and end times, the position of the sun is also expressed as a range of possible positions for the sun between the start time and the end time.

[0052] Then, the solar position calculation unit 131 checks if there are any other visible objects for which the processing in steps S403 to S404 has not yet been performed (step S405). If there are any (YES in step S405), it returns to step S402 and selects another visible object. If there are no other visible objects (NO in step S405), the processing of the solar position calculation unit 131 is terminated.

[0053] Incidentally, steps S403 to S404 do not necessarily have to be performed for all objects to be observed. When multiple objects to be observed are in close proximity, the times set in steps S403 to S404 for these objects will be approximately the same, and furthermore, the positions of the sun calculated based on those times are likely to be approximately the same. For this reason, if a selected object to be observed is in close proximity to another object for which the processing in steps S403 to S404 has already been completed, the information calculated for the other objects may be duplicated instead of performing steps S403 to S404.

[0054] Figure 5 shows the processing flow of the invisible area calculation unit 132. The unvisible area calculation unit 132 selects one more visible object from among the visible objects processed by the solar position calculation unit 131 (step S501). Subsequent steps S502 to S504 are performed on the visible object selected here (hereinafter referred to as the selected visible object).

[0055] The obscured area calculation unit 132 determines whether sunlight is blocked when viewing the selected object to be viewed (step S502). For example, if the map information management device 111 outputs weather information for the region included in the map data in addition to the map data group 141 as dynamic information, it may determine from the weather information whether it is cloudy or rainy and determine that sunlight is blocked. In other words, it may obtain weather information for the region to which the selected object to be viewed belongs at a predetermined time and determine from the weather information that the effect of backlighting is small, and decide not to calculate an obscured area. Alternatively, it may determine whether sunlight is blocked based on the position and size of landmark information such as high-rise buildings included in the map data group 141 and their positional relationship with the selected object to be viewed. In other words, the obscured area calculation unit 132 may determine not to calculate an obscured area if it determines from the position information of the sun, the position information of the selected object to be viewed, and the map information that there is an obstacle that blocks sunlight between the selected object to be viewed and the sun. For example, if a large landmark such as a skyscraper exists and the landmark is located between the selected object to be observed and the sun and is in close proximity, it may be determined that sunlight is blocked. If the unobservable area calculation unit 132 determines that sunlight is blocked (YES in step S502), it moves to step S505. If it determines that sunlight is not blocked (NO in step S502), it moves to step S503.

[0056] In step S503, the invisible area calculation unit 132 sets an area surrounding the selected object to be seen. Based on the position information of the selected object to be seen, the invisible area calculation unit 132 sets an area surrounding the selected object to be seen, including the space around the selected object, for the area in space occupied by the selected object to be seen, and calculates the invisible area from the position information of the sun and the area surrounding the object to be seen. Specifically, taking backlighting into consideration, it sets an area including the space around the selected object to be seen, for the space occupied by the selected object to be seen. Details of the area surrounding the object to be seen are described below.

[0057] A subject may have difficulty seeing a selected object due to backlighting if the sun is present around the object when the subject sees it. In other words, this occurs when the line connecting the sun and the subject passes around the selected object. Therefore, a region around the object is introduced, and if the line connecting the sun and the subject passes through this region, the subject is considered to have difficulty seeing the selected object. The size of the region around the object may be calculated by multiplying the size of the selected object by a predetermined coefficient, or by adding a predetermined value to the size of the selected object. Details of the region around the object will be described later using Figure 6.

[0058] The invisible area calculation unit 132 calculates the invisible area (step S504) based on the area around the object to be seen set in step S503 and the position information of the sun calculated in step S404. An example of this calculation method will be described later with reference to Figure 7. The invisible area calculated here is stored in the invisible area data group 143.

[0059] In step S505, the invisible area calculation unit 132 checks if there are any objects to be seen that have not yet been processed in steps S502 to S504. If there are (YES in step S505), it returns to step S501 and selects another object to be seen. If there are no such objects (NO in step S505), the invisible area calculation unit 132 terminates its processing.

[0060] Figure 6 shows a traffic light, which is an example of a visible object, and the corresponding visible object. objectThis diagram shows the surrounding area. The visible object 601 is a traffic light, and information regarding its position and size is included in the visible object data group 142. In step S503, the invisible area calculation unit 132 generates a visible object surrounding area 602 for the visible object 601. The size of the visible object surrounding area 602 may be calculated by multiplying the size of the visible object 601 by a predetermined coefficient, or by adding a predetermined size. The center of the visible object 601 is configured to be the same as the center of the visible object surrounding area 602.

[0061] The space surrounding the object 601 being viewed can be considered as a plane based on the plane from which the viewer sees, as shown in Figure 6, or as a three-dimensional space including depth. Figure 7 illustrates the case where the space is considered as a plane based on the plane from which it is seen. The concept of considering the space as a three-dimensional space will be supplemented as needed in the explanation of Figure 7.

[0062] Using Figure 7, an example of a specific method for calculating the invisible area will be explained, using the visible object 601 and the area surrounding the visible object 602 as examples.

[0063] Figure 7 is a top view of road 701 on which the object to be observed 601 is installed. The object to be observed 601 and its corresponding surrounding area 602 are defined. When viewed from the near side of road 701 (bottom side of Figure 7), the sun 702 is located beyond the object to be observed 601. The position of the sun 702 (elevation angle θ1, azimuth θ2) is calculated by the sun position calculation unit 131.

[0064] The invisible area calculation unit 132 calculates the invisible area 703 based on the position information of the area 602 surrounding the object to be seen and the sun 702. An example of the method for calculating this invisible area 703 is described below.

[0065] The vertices of the area 602 surrounding the object to be seen are vertices 704, 705, 706, and 707, starting from the upper left and moving counterclockwise. First, a line is drawn from vertex 704 toward the sun with an elevation angle θ1 and an azimuth θ2. This line is the line from vertex 704 toward the sun. Next, this line is extended in the opposite direction, and the point where it intersects with the surface of the road 701 is calculated, which becomes vertex 708. This can be calculated based on the position information (including altitude information) of any point on the road included in the map data set 141, and the position information (including altitude information) and size of the area 602 surrounding the object to be seen, which was set in step S503. In the same way that vertex 708 corresponding to vertex 704 is calculated, vertex 709 corresponding to vertex 705, vertex 710 corresponding to vertex 706, and vertex 711 corresponding to vertex 707 are calculated. The area 703 that cannot be seen is given as a rectangle with these vertices 708 to 711 as its vertices. The example in Figure 7 assumes that the area surrounding the object being observed is a rectangle, but it can be easily calculated for other polygons in the same way by calculating each vertex of the area surrounding the object being observed and the corresponding vertex on road 701.

[0066] Even when road 701 is uphill or downhill, the area of ​​obscured visibility 703 can be easily calculated. This is because the map data set 141 includes elevation information at each point along the road, and this information is taken into consideration.

[0067] If the position information of the sun 702 is expressed as a range of possible positions for the sun between a predetermined start time and end time, the unobservable region 703 corresponding to each position of the sun 702 within that position range can be easily calculated by superimposing these unobservable regions 703.

[0068] If the area 602 surrounding the object to be seen is a three-dimensional object including depth, the area 703 that cannot be seen can be easily calculated by considering the shape when that three-dimensional object is projected onto the road 701 from the sun 702. For example, if the sun's azimuth is parallel to the direction of travel on the road 701, vertices 704 and 707 can be selected as the two vertices at the upper front end of the area 602 surrounding the object to be seen, viewed from the front side of the road 701, and vertices 705 and 706 can be selected as the two vertices at the lower rear end of the area 602 surrounding the object to be seen, viewed from the front side of the road 701. Furthermore, if necessary, points on the road 701 corresponding to other vertices or points on edges of the area 602 surrounding the object to be seen can be calculated, and the area 703 that cannot be seen can be calculated based on the results.

[0069] (Calculation of the invisible area considering the height information of the subject of the view) Figure 7 illustrates the method for calculating the area of ​​no visibility 703 on the surface of the road 701. Here, we will also mention the method for calculating the area of ​​no visibility that takes into account the height information of the viewing subject.

[0070] The height information of the primary observer refers to the eye level of the driver of vehicle 102, or the height information of the sensor group 112 from which it is installed. These can be expressed, for example, as the height from the tire contact surface of vehicle 102.

[0071] Figure 7 shows how the invisible area 703 was calculated by determining the point where the extension of a line drawn from the area 602 surrounding the object to be seen in the direction of the sun intersects with the surface of the road 701. Here, instead of the surface of the road 701, we consider a surface that is parallel to the road 701 and is located at a position obtained by adding the height information from the tire contact surface of the vehicle 102 to the surface of the road 701. In the explanation of Figure 7, instead of the point where it intersects with the surface of the road 701, the invisible area is calculated by determining the vertices 708 to 711 that intersect with this surface. In other words, the invisible area calculation unit 132 calculates the invisible area based on the height information of the position of the viewing subject who sees the object to be seen in the vehicle. The invisible area 703 calculated by this method is an invisible area corresponding to the height at which the viewing subject sees it, and is a more accurate area that takes into account what the viewing subject sees.

[0072] (Generation of vehicle control information) The processing of the vehicle control information generation unit 136 will be explained with reference to Figures 8 to 11. Figure 8 is a diagram showing the overall processing flow of the vehicle control information generation unit 136.

[0073] In step S801, the vehicle control information generation unit 136 generates driving control information based on the information from the map data group 141, the vehicle information obtained from the sensor device group 112, and the conditions around the vehicle 102 recognized by the vehicle surroundings situation recognition unit 134, and stores it in the driving control data group 145. The processing in step S801 is implemented using known technologies related to driver assistance and autonomous driving.

[0074] In step S802, the vehicle control information generation unit 136 refers to the route information, the invisible area data group 143, and the driving control data group 145 included in the map data group 141 to determine whether the vehicle 102 may pass through an invisible area. For example, it may refer to the driving control data group 145 to determine if an invisible area exists on the planned trajectory of the vehicle 102, or it may refer to the route information to determine if an invisible area exists on the path of the vehicle 102. If it is determined that there is a possibility of passing through an invisible area (YES in step S802), the process moves to step S803. If it is determined that there is no possibility of passing through an invisible area (NO in step S802), the processing of the vehicle control information generation unit 136 ends.

[0075] In step S803, the vehicle control information generation unit 136 modifies the driving control information included in the driving control data group 145 based on the area that cannot be seen. Alternatively, the vehicle control information generation unit 136 generates control information for controlling the backlight countermeasure device group 114 based on the area that cannot be seen and stores it in the backlight countermeasure device control data group 146. An example of this step S803 will be described later using Figures 9 to 11. Then, the processing of the vehicle control information generation unit 136 ends. Hereafter, an example of the process in step S803 will be explained with reference to Figures 9 to 11.

[0076] Figure 9 shows a processing flow for controlling the backlight countermeasure device group 114 as an example of step S803. This process is performed so that the viewing subject can see the target object even in the area where visibility is impossible, by blocking the sunlight entering the viewing subject of the vehicle 102 with the backlight countermeasure device group 114.

[0077] In step S901, the vehicle control information generation unit 136 calculates the positional relationship between the sun and the object to be seen when the viewing subject of vehicle 102 sees it within the invisible area. For example, within the invisible area, it calculates a positional relationship such that the sun is to the right of the object to be seen from the perspective of the viewing subject. The vehicle When entering an area where visibility is impaired, The sun, From the perspective of the observer, it is initially located above the object being observed, The vehicle As you drive The sun Time changes, such as moving from above the object being viewed to below it, can also be calculated here.

[0078] In step S902, the vehicle control information generation unit 136 generates control information for the backlight countermeasure device group 114 according to the positional relationship calculated in step S901. For example, if the sun is to the right of the object to be seen when viewed from the perspective of the observer within the area of ​​non-visibility, the unit generates information to control the backlight countermeasure device group 114 to be positioned in the area to the right of the object to be seen within the observer's field of view, and stores this information in the backlight countermeasure device control data group 146. Then, the process in step S803 is completed.

[0079] Figure 10 shows a processing flow related to the driving control of vehicle 102 as an example of step S803. This process is performed to avoid problems that occur when the vehicle 102 is stopped, by adjusting its stopping position, which makes it difficult to see the target object.

[0080] In step S1001, the vehicle control information generation unit 136 determines whether, if the vehicle 102 stops within a blind spot, it may become difficult for the vehicle control system 101 to perform appropriate processing regarding driver assistance and autonomous driving.

[0081] Here, let's consider the case where the object to be seen is a traffic light as an example. Whether or not to consider this traffic light use case can be determined according to the information in column 202 of the object to be seen data set. A case where a problem may occur is, for example, when the stop line corresponding to the traffic light is located within a blind spot. This can be easily determined by utilizing the information in map data set 141. In this case, if vehicle 102 stops in the blind spot, the color of the traffic light will be difficult to see due to backlighting, so when should vehicle 102 start moving? mosquito This makes the decision difficult.

[0082] On the other hand, a case in which no problem occurs is, for example, when the stop line corresponding to the traffic light and the area where visibility is impaired are a certain distance apart. This can also be easily determined based on the information in the map data set 141. In this case, the vehicle 102 can decide whether or not to start moving based on the situation of surrounding vehicles on the road, regardless of the color of the traffic light, and therefore no problem may occur.

[0083] If it is determined that a problem may occur (YES in step S1001), proceed to step S1002. If it is determined that no problem will occur (NO in step S1001), terminate the process in step S803.

[0084] In step S1002, the vehicle control information generation unit 136 modifies the driving control information included in the driving control data group 145 so that the vehicle 102 does not stop within the blind spot area. For example, if it is determined that the vehicle must stop within the blind spot area, the driving control information is modified to stop before reaching the blind spot area. Alternatively, if it is determined that the vehicle 102 can stop on the other side of the blind spot area, the driving control information is modified to stop on the other side of the blind spot area. Alternatively, after confirming that the vehicle 102 does not need to stop within or around the blind spot area, the driving control information is modified to pass through the blind spot area. Then, the vehicle control information generation unit 136 completes the process in step S803.

[0085] Figure 11 shows a processing flow related to the driving control of vehicle 102 as an example of step S803. This process is performed to avoid problems that occur when the vehicle 102 is traveling at a predetermined speed, by adjusting the vehicle's speed, which makes it difficult to see the target object.

[0086] In step S1101, the vehicle control information generation unit 136 determines whether a problem may arise where the vehicle control system 101 cannot properly perform driver assistance or autonomous driving processing if the vehicle 102 passes through an area where visibility is impaired at a predetermined speed. The predetermined speed is a speed assumed based on the current speed of the vehicle 102 obtainable from the sensor device group 112, the legal maximum speed on the road obtainable from the map data group 141, the driving speed of surrounding vehicles of the vehicle 102 as recognized by the vehicle surrounding situation recognition unit 134, etc. Furthermore, the predetermined speed does not need to be uniquely defined; the range of the predetermined speed may be defined using two pieces of information: the legal minimum speed and the legal maximum speed. In this case, in the following explanation, similar processing can be easily achieved by repeating the same processing for each value within that range.

[0087] Here, as in Figure 10, let's consider the case where the object to be seen is a traffic light as an example. A problem may arise, for example, when the position of vehicle 102 at the time of deciding whether or not to enter the intersection after seeing the color of the traffic light, overlaps with the area where visibility is impossible at a given speed. This can be easily determined from the distance along the path from the area where visibility is impossible to the traffic light, which can be obtained from the map data group 141 and the area where visibility is impossible data group 143, and from the given speed. In this case, because the color of the traffic light is difficult to see due to backlighting, it becomes difficult for vehicle 102 to decide whether or not to enter the intersection.

[0088] On the other hand, a case in which no problem occurs is, for example, when the position of vehicle 102 at the time of visually checking the traffic light color and deciding whether or not to enter the intersection does not overlap with the area where visibility is impossible at a predetermined speed. In this case, the traffic light color can be recognized at the necessary time, and appropriate processing is possible.

[0089] If it is determined that a problem may occur (YES in step S1101), proceed to step S1102. If it is determined that no problem will occur (NO in step S1101), terminate the process in step S803.

[0090] In step S1102, the vehicle control information generation unit 136 modifies the driving control information included in the driving control data group 145 and adjusts the speed of vehicle 102 within and around the blind spot area. For example, the speed of vehicle 102 is reduced so that the position of vehicle 102 at the time when the driver decides whether or not to enter the intersection after seeing the color of the traffic light is on the other side of the blind spot area (i.e., closer to the traffic light than the blind spot area). Alternatively, the speed of vehicle 102 is increased so that the position of vehicle 102 at that time is on the near side of the blind spot area (i.e., further away from the traffic light than the blind spot area). Then, the vehicle control information generation unit 136 completes the process in step S803.

[0091] Figures 9 to 11 all illustrate examples of the processing in step S803, but the processing performed in step S803 is not limited to any of the figures in Figures 9 to 11. For example, Figures 9 to 11 may be combined to perform multiple processes simultaneously. For example, in addition to Figures 9 to 11, some modifications may be made to the driving control data group 145 based on information from areas that are not visible in order to improve the safety of the vehicle control system 101.

[0092] As explained in Figures 1 to 11, the vehicle control device 110 calculates the invisible area corresponding to the object to be seen and outputs information to control the actuator group 113 and the backlight countermeasure device group 114 based on the invisible area. This makes it possible to improve the availability of driver assistance and autonomous driving in backlit environments while ensuring safety.

[0093] [Second Embodiment] (System Configuration) A second embodiment of the present invention is comprised of a part of the vehicle control device 110 in the first embodiment. In the second embodiment, the vehicle control device 110 is configured such that the processing unit 120 in Figure 1 includes a map information acquisition unit 130, a sun position calculation unit 131, an invisible area calculation unit 132, and an invisible area output unit 133, and the storage unit 121 further includes a map data group 141, an invisible target object data group 142, and an invisible area data group 143.

[0094] (Processing according to the second embodiment) In the second embodiment, the map information acquisition unit 130, the sun position calculation unit 131, and the invisible area calculation unit 132 perform the same processing as in the first embodiment. The invisible area output unit 133 outputs information about the invisible area to other vehicle control devices in the vehicle 102. The other vehicle control devices perform processing such as generating vehicle driving control based on the invisible area. In other words, in the second embodiment, the vehicle control device 110 is responsible for calculating the invisible area based on the map data group 141 and the visible target object data group 142, and outputting information about the invisible area.

[0095] [Third Embodiment] (System Configuration) A third embodiment of the present invention will be described with reference to Figure 12.

[0096] Figure 12 is a functional block diagram showing the configuration of a vehicle control system 1201 according to a third embodiment of the present invention. The vehicle control system 1201 comprises a server device 1202, a vehicle 102, and a vehicle control device 110 mounted on the vehicle 102. The server device 1202 is a device located in a remote location independent of the location of the vehicle 102, and the server device 1202 and the vehicle 102 are connected via a network 1203. The vehicle control system 1201 achieves the same functions as the first embodiment of the present invention through the server device 1202 and the vehicle control device 110 mounted on the vehicle 102.

[0097] The server device 1202 is an information processing device and comprises a processing unit 1210 and a storage unit 1211. The processing unit 1210 is configured to include a CPU (Central Processing Unit), which is a central processing unit. However, in addition to the CPU, it may also include a GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), ASIC (Application Specific Integrated Circuit), etc., or it may be configured with any one of these. The storage unit 1211 is configured to include, for example, a storage device such as an HDD (Hard Disk Drive), flash memory, ROM (Read Only Memory), or memory such as RAM (Random Access Memory). The storage unit 1211 stores programs processed by the processing unit 1210 and data sets necessary for that processing. It is also used as main memory when the processing unit 1210 executes a program, temporarily storing data necessary for program calculations.

[0098] The processing unit 1210 included in the server device 1202 has the following functions: a server-side map information management unit 1221, a sun position calculation unit 131, an invisible area calculation unit 132, and an invisible area output unit 133. The processing unit 1210 achieves these functions by executing a predetermined operation program stored in the storage unit 1211.

[0099] The server-side map information management unit 1221 manages the map data group 141 and the visible object data group 142 in the storage unit 1211, and outputs this information to other functions of the processing unit 1210 and other devices.

[0100] The solar position calculation unit 131 and the invisible area calculation unit 132 are the same as those in the first embodiment of the present invention. Unlike the first embodiment, the invisible area output unit 133 outputs information about the invisible area to the vehicle control device 110 via the network 1203. The storage unit 1211 includes a map data group 141, a data group of objects to be seen 142, and a data group of invisible areas 143. These have the same role as the data groups held by the storage unit 121 of the vehicle control device 110.

[0101] Vehicle 102 includes, as in the first embodiment, a vehicle control device 110, a map information management device 111, a sensor device group 112, an actuator group 113, and a backlight countermeasure device group 114. However, in the third embodiment, the map information management device 111 is not necessarily included.

[0102] Unlike the first embodiment, the processing unit 120 included in the vehicle control device 110 includes a map information acquisition unit 130, a vehicle surrounding situation recognition unit 134, an unvisible area acquisition unit 135, a vehicle control information generation unit 136, and a vehicle control information output unit 137.

[0103] The map information acquisition unit 130 acquires information regarding the map data group 141 and the visible object data group 142, similar to the first embodiment, and stores it in the map data group 141 and the visible object data group 142 of the storage unit 121. The map information acquisition unit 130 may acquire the information from the map information management device 111, similar to the first embodiment, or it may acquire it from the server-side map information management unit 1221 via the network 1203.

[0104] Unlike the first embodiment, the invisible area acquisition unit 135 acquires information on invisible areas output by the invisible area output unit 133 via the network 1203 and stores it in the invisible area data group 143. The vehicle surrounding situation recognition unit 134, the vehicle control information generation unit 136, and the vehicle control information output unit 137 are the same as in the first embodiment of the present invention.

[0105] The storage unit 121 included in the vehicle control device 110 contains map data group 141, visible target object data group 142, invisible area data group 143, sensor data group 144, driving control data group 145, and backlight countermeasure device control data group 146. These data groups are the same as those in the first embodiment.

[0106] In the third embodiment of the present invention, unlike the first embodiment, the server device 1202 is responsible for calculating the blind spot area, and the vehicle control device 110 is responsible for generating and correcting the vehicle 102's driving control information based on the blind spot area, and for generating control information for the backlight countermeasure device group 114. The characteristic of the third embodiment of the present invention is that the server device 1202 is responsible for calculating the blind spot area. For example, by calculating the blind spot area at the current time or at a predetermined time after a predetermined period of time has elapsed, without depending on the information of the vehicle 102, it becomes possible to distribute blind spot area information to a large number of vehicles 102.

[0107] Although embodiments of the present invention have been described in detail above, the present invention is not limited to the embodiments described above, and various design modifications can be made without departing from the spirit of the invention as described in the claims. For example, the embodiments described above are described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations. [Explanation of symbols]

[0108] 101: Vehicle control system, 102: Vehicle, 110: Vehicle control device, 111: Map information management device, 112: Sensor device group, 113: Actuator group, 114: Backlight countermeasure device group, 120: Processing unit, 121: Memory unit, 130: Map information acquisition unit, 131: Sun position calculation unit, 132: Invisible area calculation unit, 133: Invisible area output unit, 134: Vehicle surrounding situation recognition unit, 135: Invisible area acquisition unit, 136: Vehicle control information generation unit, 137: Vehicle control information output unit, 141: Map data group, 142: Data group for visible objects, 143: Data group for areas not visible, 144: Sensor data group, 145: Driving control data group, 146: Data group for devices to counter backlighting, 601: Objects to be seen, 602: Area around objects to be seen, 701: Road, 702: Sun, 703: Area not visible, 1201: Vehicle control system, 1202: Server device, 1203: Network, 1210: Processing unit, 1211: Storage unit, 1221: Server-side map information management unit

Claims

1. A vehicle control device installed in a vehicle, A solar position calculation unit that calculates the position information of the sun at a predetermined time, An invisible area calculation unit calculates an invisible area on a map where the object to be seen is difficult to see, based on the position information of the sun and the position information of the object to be seen. An invisible area output unit that outputs the invisible area calculated by the invisible area calculation unit, An invisible area acquisition unit that acquires information about the invisible area from the invisible area output unit, A vehicle control information generation unit identifies the invisible area on the vehicle's path based on the acquired information on the invisible area, A vehicle control information output unit that outputs vehicle control information based on the identified area of ​​no visibility, Equipped with, The vehicle control device is characterized in that, when the vehicle stops within the specified blind spot area, the vehicle control information output unit outputs information to stop the vehicle either before or beyond the blind spot area as vehicle control information.

2. A vehicle control device mounted on a vehicle, A solar position calculation unit that calculates the position information of the sun at a predetermined time, An invisible area calculation unit calculates an invisible area on a map where the object to be seen is difficult to see, based on the position information of the sun and the position information of the object to be seen. An invisible area output unit that outputs the invisible area calculated by the invisible area calculation unit, An invisible area acquisition unit that acquires information about the invisible area from the invisible area output unit, A vehicle control information generation unit identifies the invisible area on the vehicle's path based on the acquired information on the invisible area, A vehicle control information output unit that outputs vehicle control information based on the identified area of ​​no visibility, Equipped with, The vehicle control information output unit is When the vehicle passes through the identified area of ​​blindness, A vehicle control device characterized by outputting information as vehicle control information that causes the vehicle to adjust its speed so that the position of the vehicle at the time it needs to see the object to be seen does not overlap with the area where visibility is impossible.

3. A vehicle control system comprising a server device installed in a location independent of the vehicle, and a vehicle control device mounted on the vehicle, The server device comprises: a solar position calculation unit that calculates the position information of the sun at a predetermined time; an invisible area calculation unit that calculates an invisible area on a map where the object to be seen is difficult to see, based on the position information of the sun and the position information of the object to be seen; and an invisible area output unit that outputs information of the invisible area. The vehicle control device comprises: an invisible area acquisition unit that acquires information about the invisible area from the invisible area output unit; a vehicle control information generation unit that identifies the invisible area on the vehicle's path based on the acquired information about the invisible area; and a vehicle control information output unit that outputs vehicle control information based on the identified invisible area. The vehicle control system is characterized in that, when the vehicle stops within the identified blind spot area, the vehicle control information output unit outputs information to stop the vehicle either before or beyond the blind spot area as vehicle control information.

4. A vehicle control system comprising a server device installed in a location independent of the vehicle and a vehicle control device mounted on the vehicle, The server device comprises: a solar position calculation unit that calculates the position information of the sun at a predetermined time; an invisible area calculation unit that calculates an invisible area on a map where the object to be seen is difficult to see, based on the position information of the sun and the position information of the object to be seen; and an invisible area output unit that outputs information of the invisible area. The vehicle control device comprises: an invisible area acquisition unit that acquires information about the invisible area from the invisible area output unit; a vehicle control information generation unit that identifies the invisible area on the vehicle's path based on the acquired information about the invisible area; and a vehicle control information output unit that outputs vehicle control information based on the identified invisible area. The vehicle control system is characterized in that, when the vehicle passes through the identified blind area, the vehicle control information output unit outputs information as vehicle control information that causes the vehicle to adjust its speed so that the position of the vehicle at the time when the vehicle needs to see the object to be seen does not overlap with the blind area.

Images