Device for acquiring information on stationary objects, program, and method for acquiring information on stationary objects

Abstract

A stationary object information acquisition device (100) mounted to a vehicle (2) is provided with: an image acquisition unit (111) that acquires image data (122) of an image captured by a sensor unit (31) mounted to the vehicle (2); an identification unit (112) for identifying, on the basis of the image data (122), stationary object information (123) which includes stationary object position information indicating the position of a stationary object calculated on the basis of the image data (122) and / or includes stationary object image data corresponding to an image where at least one stationary object selected from among a self-light-emitting body, a sign, a delineator, and a guardrail is present or to one part of the image; and a transmission unit (113) for transmitting, to a storage unit (220), the stationary object information (123) and vehicle position information (124) which is of the vehicle (2), which has been acquired from a position information acquisition unit 32 mounted to the vehicle (2), and which is at a time when an image, corresponding to the image data 122 having the stationary object information (123) identified therein, has been captured.

Description

Technical Field

[0001] The present disclosure relates to a stationary object information acquisition device, a program, and a stationary object information acquisition method.

Background Art

[0002] In recent years, based on the situation around a vehicle, an ADB (Adaptive Driving Beam) technology has been proposed that performs light shielding or light reduction on high-reflection objects such as the position of a preceding vehicle and the position of a sign or the like that shields light from an oncoming vehicle. For example, Patent Document 1 describes detecting a vehicle ahead and controlling the light distribution forward.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, the light distribution control of ADB is performed based on the target information sent from the vehicle. Each target is detected by a specific algorithm based on the data acquired by a sensor such as a camera. However, depending on the accuracy of the data or the detection accuracy of the algorithm, there may be cases where a target that exists is not detected (overdetection), or a target that does not exist is detected (false detection).

[0005] By the way, when there are stationary objects with high brightness such as street lights and signs on the road, these stationary objects may be misrecognized as a preceding vehicle. Also, it is possible that the headlamp of a preceding vehicle may be misrecognized as a street light or the like. If information on stationary objects such as street lights and signs on the road can be collected, it can be usefully applied to reducing the possibility of the above misrecognition, etc., which is beneficial.

[0006] This disclosure aims to collect information on stationary objects such as streetlights and signs on roads. [Means for solving the problem]

[0007] A static object information acquisition device according to one aspect of this disclosure is: An image acquisition unit that acquires image data of images captured by a sensor unit mounted on the vehicle, A unit for identifying stationary object information based on image data, which includes at least one of the following: an image containing one or more stationary objects from among self-illuminating objects, signs, delineators, and guardrails, or a part of said image; and stationary object position information indicating the position of said stationary object calculated based on said image data. A transmission unit transmits to a storage unit the vehicle location information of the vehicle, which is obtained from a location information acquisition unit mounted on the vehicle, the vehicle location information when an image corresponding to the image data in which the stationary object information is identified is captured, and the stationary object information. It is equipped with and mounted on the vehicle.

[0008] A program relating to one aspect of this disclosure is: A program that is executed in a computer device installed in a vehicle, which is equipped with a processor, The program is provided to the processor: Image acquisition step: acquires image data of an image captured by a sensor unit mounted on the vehicle, Identification step of identifying stationary object information based on the image data, which includes at least one of the following: an image containing one or more stationary objects from among a self-illuminating object, a sign, a delineator, and a guardrail, or a stationary object image data corresponding to a part of said image, and stationary object position information indicating the position of said stationary object calculated based on said image data; A transmission step of transmitting to a storage unit the vehicle location information of the vehicle obtained from a location information acquisition unit mounted on the vehicle, the vehicle location information when an image corresponding to the image data in which the stationary object information is identified is captured, and the stationary object information. Make it run.

[0009] A method for acquiring static object information relating to one aspect of this disclosure is: A method for acquiring stationary object information, which is performed in a computer device mounted on a vehicle and includes a processor, The method for acquiring stationary object information involves the processor, Image acquisition step: acquires image data of an image captured by a sensor unit mounted on the vehicle, Identification step of identifying stationary object information based on the image data, which includes at least one of the following: an image containing one or more stationary objects from among a self-illuminating object, a sign, a delineator, and a guardrail, or a stationary object image data corresponding to a part of said image, and stationary object position information indicating the position of said stationary object calculated based on said image data; A transmission step of transmitting to a storage unit the vehicle location information of the vehicle obtained from a location information acquisition unit mounted on the vehicle, the vehicle location information when an image corresponding to the image data in which the stationary object information is identified is captured, and the stationary object information. This includes causing the execution of the command. [Effects of the Invention]

[0010] According to this disclosure, it is possible to collect information on stationary objects such as streetlights and signs on roads. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing an example of a system including a stationary object information acquisition device according to one embodiment of the present disclosure. [Figure 2] This is a block diagram showing an example of a system including a stationary object information acquisition device according to one embodiment of the present disclosure. [Figure 3] This flowchart shows an example of a method for acquiring information on a stationary object according to one embodiment of the present disclosure. [Figure 4] Figure 3 is a flowchart illustrating an example of the process for identifying stationary object information. [Figure 5] It is a schematic diagram for explaining the stationary object position information. [Figure 6] It is a schematic diagram showing the imaging timing and each image data acquired at each imaging timing. [Figure 7] It is a flowchart showing an example of the determination process of the stationary object information. [Figure 8] It is a schematic diagram showing an example of the reference image data used in the identification process of the stationary object information. [Figure 9] It is a schematic diagram showing an example of the image data used in the identification process of the stationary object information.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, the present invention will be described based on embodiments with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing are denoted by the same reference numerals, and duplicate explanations will be omitted as appropriate. Also, the embodiments are illustrative and not restrictive of the invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention.

[0013] (System) First, a system 1 including a stationary object information acquisition device 100 according to an embodiment of the present disclosure will be described with reference to FIGS. 1 and 2. FIG. 1 is a schematic diagram showing the system 1 according to an embodiment of the present disclosure. As shown in FIG. 1, the system 1 includes a stationary object information storage device 200 and a plurality of vehicles 2 such as vehicles 2A and 2B on which the stationary object information acquisition device 100 is mounted respectively. The stationary object information storage device 200 and each vehicle 2 can be communicatively connected to each other by wireless communication.

[0014] The stationary object information acquisition device 100 acquires stationary object information about stationary objects and transmits the stationary object information to the stationary object information storage device 200. The stationary object information storage device 200 stores the stationary object information received from each stationary object information acquisition device 100, for example. The stationary object information storage device 200 also analyzes the received stationary object information to improve the accuracy of the stationary object information, acquire more detailed information, or create a light distribution pattern based on the stationary object information. The stationary object information storage device 200 also transmits this improved accuracy stationary object information to each vehicle 2 in response to requests from each vehicle 2. Each vehicle 2 can, for example, use the improved accuracy stationary object information received from the stationary object information storage device 200 to improve the accuracy and efficiency of target detection, or to appropriately control the light distribution of the headlights.

[0015] In this embodiment, "stationary object" refers to an object that is fixed to the road and has high brightness, and specifically, it is one or more of the following: self-illuminating objects (e.g., streetlights and traffic signals), signs, delineators, and guardrails. In other words, the stationary object information acquisition device 100 in this embodiment acquires stationary object information relating to the various stationary objects listed as specific examples above. In other embodiments, the stationary object information acquisition device 100 may be configured to identify as a stationary object any object not included in the specific examples above, that is fixed to the road, has high brightness, and may affect the detection of targets.

[0016] Figure 2 is a block diagram showing a system 1 according to one embodiment of the present disclosure. The vehicle 2 comprises a vehicle ECU (Electronic Control Unit) 10, a storage unit 20, a sensor unit 31, a location information acquisition unit 32, an illuminance sensor 33, and a stationary object information acquisition device 100. The vehicle 2 can also communicate with the stationary object information storage device 200 via wireless communication over a communication network 3. The means of wireless communication are not particularly limited and may include, for example, a mobile communication system such as a telematics system for automobiles, collaboration with a smartphone, or the use of in-vehicle Wi-Fi.

[0017] The vehicle ECU 10 controls various operations of the vehicle 2, such as driving. The vehicle ECU 10 includes a processor such as an ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or general-purpose CPU (Central Processing Unit). The storage unit 20 includes, for example, a ROM (Read Only Memory) in which various vehicle control programs are stored, and a RAM (Random Access Memory) in which various vehicle control data is temporarily stored. The processor of the vehicle ECU 10 loads data specified from the various vehicle control programs stored in the ROM onto the RAM and controls various operations of the vehicle 2 in cooperation with the RAM.

[0018] The sensor unit 31 outputs image data of an image taken of the outside of the vehicle 2. The sensor unit 31 includes, for example, one or more sensors from among a visible light camera, LiDAR, and millimeter-wave radar. The image data output by the LiDAR and millimeter-wave radar may be three-dimensional image data. The location information acquisition unit 32 outputs vehicle location information indicating the current location of the vehicle 2. The location information acquisition unit 32 includes, for example, a GPS (Global Positioning System) sensor. The illuminance sensor 33 detects and outputs the illuminance around the vehicle 2.

[0019] The stationary object information acquisition device 100 comprises a control unit 110 and a storage unit 120. The control unit 110 is configured, for example, by a processor such as a CPU. The control unit 110 may be configured, for example, as part of a lighting ECU that controls the operation of lighting devices such as headlights in a vehicle 2. Alternatively, the control unit 110 may be configured, for example, as part of a vehicle ECU 10. The storage unit 120 is configured, for example, by a ROM or RAM. The storage unit 120 may be configured as part of a storage unit 20 or a storage device provided for the lighting ECU.

[0020] The control unit 110 functions as the image acquisition unit 111, the identification unit 112, the transmission / reception unit 113, and the determination unit 114 by reading the program 121 stored in the storage unit 120. Some of these functions may be implemented by the vehicle ECU 10 or the lighting ECU. In this configuration, the vehicle ECU 10 or the lighting ECU constitutes part of the stationary object information acquisition device 100. The program 121 may also be recorded on a non-temporary computer-readable medium.

[0021] The image acquisition unit 111 acquires image data 122 of the image captured by the sensor unit 31. The acquired image data 122 is stored in the storage unit 120. The image acquisition unit 111 also acquires vehicle position information 124 (i.e., imaging position information indicating the image capture location) from the position information acquisition unit 32, which corresponds to the acquired image data 122, when the image was captured. Preferably, the vehicle position information 124 includes information indicating the orientation of the vehicle 2 when the image was captured. The vehicle position information 124 may also include information indicating the position of the vehicle in the vehicle width direction. The position of the vehicle in the vehicle width direction can be calculated, for example, by detecting the driving lane and using that driving lane as a reference. The acquired vehicle position information 124 is stored in the storage unit 120. The vehicle position information 124 is stored in the storage unit 120, for example, associated with the corresponding image data 122.

[0022] The image acquisition unit 111 may acquire time information indicating the time when the image was captured. The time information may include information indicating the year, month, and day the image was captured. The image acquisition unit 111 may also acquire illumination information regarding whether or not the headlights of the vehicle 2 were illuminated when the image was captured. The time information and illumination information are stored in the storage unit 120, for example, associated with the corresponding image data 122.

[0023] Furthermore, the image acquisition unit 111 may acquire image data 122 captured when the illuminance sensor 33 outputs a signal indicating that the illuminance is above a predetermined value (for example, 1000 lux), as reference image data. Here, illuminance above a predetermined value is, for example, illuminance above a value that would be judged as daytime. That is, the image acquisition unit 111 may store image data 122 of an image captured during daytime as reference image data in the storage unit 120. Alternatively, the image acquisition unit 111 may acquire illuminance information from the illuminance sensor 33 indicating the illuminance around the vehicle 2 when the image was captured, and store the image data 122 and the illuminance information in association in the storage unit 120. In this case, image data 122 in which the illuminance indicated by the associated illuminance information is above a predetermined value may become reference image data.

[0024] The identification unit 112 identifies stationary object information 123 based on the image data 122. The stationary object information 123 identified by the identification unit 112 is stored in the storage unit 120. Here, "stationary object information" refers to information that includes at least one of the following: stationary object image data corresponding to an image in which a stationary object exists or a part of said image, and stationary object position information indicating the position of the stationary object calculated based on the image data 122.

[0025] The identification unit 112, for example, detects stationary objects in an image by image analysis and includes the image data 122 of the image in which the stationary object was detected as stationary object image data in the stationary object information 123. The identification unit 112 also identifies the region containing the stationary object in the image in which the stationary object was detected as the stationary object region and includes the data corresponding to the stationary object region, which is a part of the image, as stationary object image data in the stationary object information 123. The identification unit 112 also calculates the position of the stationary object based on the image in which the stationary object was detected and includes stationary object position information indicating the position of the stationary object in the stationary object information 123. The stationary object position information may be, for example, information indicating the position of the stationary object in the image (for example, the coordinates and size of the position where the stationary object is located in the image), or it may indicate the distance and direction from the image acquisition position to the stationary object. The identification unit 112 may also identify the type of stationary object and include information indicating the type in the stationary object information 123.

[0026] The transmitting / receiving unit 113 transmits and receives information to and from the vehicle ECU 10 and the stationary object information storage device 200. In other words, the transmitting / receiving unit 113 functions as both a transmitter and a receiver. The transmitting / receiving unit 113 transmits stationary object information 123 and vehicle position information 124 corresponding to the stationary object information 123 (when an image corresponding to the image data 122 in which the stationary object information 123 is identified is captured) to the stationary object information storage device 200, which includes a storage unit 220. The transmitting / receiving unit 113 can also transmit reference image data to the stationary object information storage device 200. Furthermore, the transmitting / receiving unit 113 can transmit and receive other information to and from the stationary object information storage device 200 as needed.

[0027] The determination unit 114 determines whether or not a stationary object exists at the location indicated by the stationary object location information calculated by the identification unit 112, based on reference image data captured at the same location as the image capture location of the image data 122 used to calculate the stationary object location. The reference image data used by the determination unit 114 for determination is, for example, the image data 122 of an image captured when a vehicle 2 passes through the location indicated by the vehicle location information 124 corresponding to the image data 122 for which the stationary object information was identified by the identification unit 112, and when the illuminance sensor 33 outputs a signal indicating that the illuminance is above a predetermined value, and is acquired by the image acquisition unit 111.

[0028] The stationary object information storage device 200 comprises a control unit 210 and a storage unit 220. In this embodiment, the stationary object information storage device 200 is a computer device that aggregates and stores information transmitted from multiple vehicles 2, and is installed, for example, in a data center. The control unit 210 is composed of, for example, a processor such as a CPU. The storage unit 220 is composed of, for example, ROM or RAM.

[0029] The control unit 210 functions as a transmitting / receiving unit 211, a recording unit 212, and a determination unit 213 by reading the program 221 stored in the storage unit 220. The program 221 may be recorded on a non-temporary computer-readable medium.

[0030] The transmitting / receiving unit 211 transmits and receives information to and from the vehicle ECU 10 and the stationary object information acquisition device 100. The transmitting / receiving unit 211 receives stationary object information 123 transmitted from the transmitting / receiving unit 113 and vehicle position information 124 corresponding to the stationary object information 123. The transmitting / receiving unit 211 can also receive reference image data from the stationary object information acquisition device 100. In addition, the transmitting / receiving unit 211 can transmit and receive other information to and from the vehicle ECU 10 and the stationary object information acquisition device 100 as needed.

[0031] The recording unit 212 associates the stationary object information 123 received by the transmitting / receiving unit 211 with the vehicle position information 124 corresponding to the stationary object information 123 and records it in the stationary object database 222. The recording unit 212 may update the stationary object database 222 based on the result of the determination by the determination unit 213.

[0032] The stationary object database 222 records vehicle position information 124 and stationary object information 123 in association. In the stationary object database 222, for example, multiple stationary object image data may be recorded for a single imaging position indicated by the vehicle position information 124. In addition, the stationary object database 222 may record stationary object image data and reference image data in association with the same imaging position. The stationary object database 222 may record information such as the position, size, distance and direction from the imaging position, and type of stationary object in association with the imaging position.

[0033] The determination unit 213 uses a different algorithm than the one used by the identification unit 112 to identify the stationary object information 123 to determine whether or not the stationary object information 123 contains a stationary object. Preferably, the algorithm used by the determination unit 213 is one that has a higher accuracy in detecting stationary objects than the algorithm used by the identification unit 112.

[0034] The determination unit 213 determines, for example, whether or not a still object is included in the image, using the image corresponding to the still object image data included in the still object information 123. The determination unit 213 may also use the image corresponding to the still object image data to identify detailed information such as the position and size of the still object in the image, the distance and direction from the image acquisition position to the still object, and the type of the still object.

[0035] As an alternative example of the system 1 described above, a stationary object information storage device 200 may be mounted on the vehicle 2. In this case, a control unit 210 and a storage unit 220 may be provided separately from the vehicle ECU 10, control unit 110, storage unit 20, and storage unit 120. On the other hand, the control unit 210 may be configured as part of one or more of the following: the lighting ECU, the vehicle ECU 10, and the control unit 110. Furthermore, some of the functions listed for the control unit 210 may be implemented by the vehicle ECU 10 or the lighting ECU. Also, the storage unit 220 may be configured as part of one or more of the following: storage unit 20, storage unit 120, or a storage device provided for the lighting ECU. When the stationary object information storage device 200 is mounted on the vehicle 2, the stationary object information acquisition device 100 and the stationary object information storage device 200 are configured to be connectable by wireless or wired communication.

[0036] (Stationary object information acquisition method) Next, a method for acquiring stationary object information by the stationary object information acquisition device 100 according to this embodiment will be described. The method for acquiring stationary object information according to this embodiment is executed, for example, by the control unit 110 of the stationary object information acquisition device 100 which has read the program 121. In the following description, the case in which the stationary object information acquisition device 100 identifies stationary object information 123 using an image captured by a visible camera will be used as an example, but this disclosure is not limited thereto. The stationary object information acquisition device 100 may also identify stationary object information 123 using an image output by, for example, a millimeter-wave radar or LiDAR.

[0037] Figure 3 is a flowchart showing an example of a method for acquiring stationary object information according to this embodiment. The order of the processes constituting each flowchart described herein is not limited to the extent that no contradictions or inconsistencies occur in the processing content, and they may be executed in parallel.

[0038] First, in step S10, the control unit 110 acquires image data, etc. Specifically, the control unit 110 acquires image data of the image captured by the visible camera. The control unit 110 also acquires vehicle position information 124 corresponding to the image data.

[0039] Furthermore, in step S10, it is preferable that the control unit 110 also acquires one or more of the following: time information indicating the time when the image was captured, illumination information indicating whether or not the headlights of the vehicle 2 were illuminated when the image was captured, and illumination information indicating the illumination around the vehicle 2 when the image was captured. By acquiring this information, it becomes possible to appropriately compare each image, and as a result, the detection accuracy of stationary objects can be improved.

[0040] Here, the visible camera is controlled by the vehicle ECU to, for example, image the outside of the vehicle 2 at predetermined time intervals. The control unit 110 preferably acquires image data 122 of images captured at predetermined time intervals by thinning the data so that the time intervals are longer than the time interval at which the images were captured (for example, 0.1 to 1 second) or the imaging positions are at predetermined distance intervals (for example, 1 to 10 m intervals). Thinning the image data 122 can suppress the need for a large storage capacity in the storage unit 120. In addition, it reduces the number of targets for the specific processing in step S30 described later, thus reducing the burden on the control unit 110. The control unit 110 may, for example, acquire all of the image data 122 of images captured at predetermined time intervals and temporarily store it in the storage unit 120, and then thin the image data 122 at a predetermined timing, such as before the specific processing in step S30.

[0041] Furthermore, the image acquisition unit 111 may thin out the image data 122 based on whether or not the image was taken in a place that the vehicle 2 usually travels. Specifically, the image acquisition unit 111 may thin out the image data 122 of images taken on roads where the number of times the vehicle has traveled in a predetermined period in the past is less than a predetermined number (for example, once or less in the past month). This is because identifying stationary objects in places where the vehicle 2 does not usually travel is not particularly useful to the user of the vehicle 2. In particular, if the stationary object information storage device 200 is installed in the vehicle 2, it is preferable to thin out the image data 122 based on the number of times the imaging location has been traveled in a predetermined period.

[0042] Next, if the vehicle 2 is in the first state (Yes in step S20), the control unit 110 performs the identification process for the stationary object information 123 in step S30. On the other hand, if the vehicle 2 is not in the first state (No in step S20), the control unit 110 waits to execute the identification process in step S30 until the vehicle 2 is in the first state.

[0043] Here, "first state" refers to a state in which the processing load on the vehicle ECU 10 or lighting ECU is considered to be low. "First state" includes, for example, a stopped state or a slow-moving state (for example, driving at a speed of 10 km / h or less). When the control unit 110 is configured as part of the vehicle ECU 10 or lighting ECU, configuring it to execute the specific processing in step S30 when the vehicle 2 is in the first state reduces the load on the vehicle ECU 10 or lighting ECU. Note that if the control unit 110 is configured independently of the vehicle ECU 10 and lighting ECU, the determination in step S20 does not need to be performed.

[0044] In step S30, the control unit 110 performs a identification process to identify stationary object information 123 based on the image data 122. Details of the identification process will be described later with reference to Figure 4, etc.

[0045] Next, if the vehicle 2 is in the second state (Yes in step S40), the control unit 110 transmits stationary object information 123 and vehicle position information 124 corresponding to the stationary object information 123 to the stationary object information storage device 200 equipped with a storage unit 220 in step S50, and then terminates. In addition, time information, lighting information, illumination information, etc. may also be transmitted in step S50. On the other hand, if the vehicle 2 is not in the second state (No in step S40), the control unit 110 waits to execute the transmission process in step S50 until the vehicle 2 is in the second state.

[0046] Here, the "second state" refers to a state in which the processing load on the vehicle ECU 10 or the lighting ECU is considered to be low. The "second state" includes, for example, a stopped state or a slow-moving state (for example, driving at a speed of 10 km / h or less). When the control unit 110 is configured as part of the vehicle ECU 10 or the lighting ECU, configuring it to execute the transmission process in step S50 when the vehicle 2 is in the second state reduces the load on the vehicle ECU 10 or the lighting ECU. Note that if the control unit 110 is configured independently of the vehicle ECU 10 and the lighting ECU, the determination in step S40 does not need to be performed.

[0047] The stationary object information 123 transmitted in step S50 may be stationary object image data of an image in which a stationary object has been identified, stationary object position information calculated from the image, or both. If the transmitted stationary object information 123 includes stationary object image data, the stationary object information storage device 200 can further examine the stationary object image data to obtain more accurate information. On the other hand, if the transmitted stationary object information 123 does not include stationary object image data, it is advantageous in that the amount of data to be transmitted is reduced.

[0048] The process for identifying stationary object information 123 in step S30 will be described in detail below. Figure 4 is a flowchart showing an example of the process for identifying stationary object information 123. In step S31, the control unit 110 detects light points in the image. The detection of light points can be performed using conventionally known techniques, for example, by brightness analysis of the image.

[0049] Furthermore, in step S32, the control unit 110 performs pattern recognition processing on the image. The pattern recognition method can be a conventionally known method; for example, a machine learning model may be used to detect stationary objects, or a clustering method may be used to detect stationary objects.

[0050] Next, in step S33, the control unit 110 determines whether or not there are stationary objects in the image based on the results of the processing in step S31 and / or step S32. If it is determined that there are no stationary objects in the image (No in step S33), in step S35, the control unit 110 deletes the image data 122 corresponding to that image from the storage unit 120 and terminates.

[0051] If it is determined that there is a stationary object in the image (Yes in step S33), in step S34, the control unit 110 identifies the stationary object region or the location of the stationary object in the image. By identifying the stationary object region and using the data of the portion of the image that includes the stationary object region as stationary object image data, the data capacity when transmitted to the stationary object information storage device 200 can be reduced. In this case, it is preferable to also identify information indicating the location of the stationary object region in the original image and include it in the stationary object information 123. Alternatively, the stationary object image data may be obtained by processing the data amount of the region excluding the stationary object region to reduce its size.

[0052] The position of a stationary object is, for example, the position of a stationary object in an image. The position of a stationary object can be determined, for example, using an arbitrary coordinate system set for the image. The position of a stationary object may indicate, for example, the center point of the stationary object, or the position of the outer edge of the stationary object. Furthermore, it is preferable that the position of a stationary object includes information about its size, which is determined using the above coordinate system.

[0053] Figure 5 is a schematic diagram illustrating stationary object position information. In the image shown in Figure 5, sign O1 and streetlights O2-O4 are identified as stationary objects. In this case, for example, the positions of the regions Z1-Z4, which include sign O1 and streetlights O2-O4 respectively, can be defined using coordinates defined by the x and y axes to represent stationary object position information. There are no particular restrictions on how the coordinates are set; for example, the center of the image may be used as the origin. Also, in the example in Figure 5, the support posts of sign O1 and streetlights O2-O4 are not included in the regions Z1-Z4, but the region including these support posts may be used as the stationary object position.

[0054] Furthermore, the stationary object position identified in step S34 may indicate the distance and direction from the image acquisition position to the stationary object. The distance and direction from the acquisition position to the stationary object may be calculated, for example, using depth information if the image data 122 includes depth information. Alternatively, it may be calculated by comparing it with other image data 122 acquired in the vicinity of the acquisition position, or by using data acquired from millimeter-wave radar or LiDAR.

[0055] Once the location of a stationary object is identified, the image data 122 may be deleted from the storage unit 120, or it may be included in the stationary object information 123 in association with the stationary object location information. After step S34, the process proceeds to step S40 in Figure 3.

[0056] When the specific processing shown in Figure 4 is performed based on image data 122 captured during the daytime when the illuminance is above a predetermined value, it becomes easier to grasp the contours of structures in the image and to obtain color information of structures from the image, thereby improving the accuracy of detecting stationary objects by pattern recognition processing.

[0057] The process of identifying stationary object information 123 may be performed by comparing multiple image data 122 captured at the same location or at locations close to each other. In addition, in step S34, the control unit 110 may identify the type of stationary object based on the results of step S31 and / or step S32 and include the type information in the stationary object information 123. Below, using Figure 6, an example of a method for identifying whether a stationary object is a self-illuminating object and an example of detecting a stationary object by comparing multiple image data 122 will be explained.

[0058] Figure 6 is a schematic diagram showing the image acquisition timing and the images 122A to D acquired at each acquisition timing. In the example in Figure 6, the visible camera captures images of the front of vehicle 2 at times T1, T2, T3, and T4, and outputs image data 122 of images 122A to D. The intervals F1 to F4 between each time point are all the same. That is, images 122A to D are image data captured at regular time intervals. The headlights are on at times T1, T3, and T4, and off at time T2. Between times T1 and T4, vehicle 2 is traveling forward at a predetermined speed.

[0059] Whether a stationary object in an image corresponding to image data 122 is a self-illuminating object can be determined, for example, based on the image data 122 of at least two images taken before and after the switching timing of the headlights mounted on vehicle 2. In Figure 6, light points LP1 and LP2 are detected in image 122A. On the other hand, in image 122B, light point LP1 is detected, but no light point is detected at the position where light point LP2 is presumed to be detected (indicated by the dotted line). Furthermore, light point LP2 is detected again in image 122C. From these findings, it can be determined that light point LP2 is a light point detected due to reflection of light from the headlights, and that light point LP2 is not due to a self-illuminating object. In addition, light point LP1, which is also detected in image 122B taken when the headlights are not on, can be determined to be due to a self-illuminating object.

[0060] The detection of stationary objects by comparing multiple images can be performed, for example, by comparing images taken at the same location or in close proximity to each other. In Figure 6, image 122C' is an image taken at the same location as image 122C, prior to image 122C. In image 122C, in addition to light spots LP1 and LP2, light spots LP3 and LP4 are detected. On the other hand, in image 122C', light spots LP1 and LP2 are detected, but light spots LP3 and LP4 are not. If light spots LP3 and LP4 were stationary objects, they would also be detected in image 122C', but in reality, light spots LP3 and LP4 are not detected in image 122C'. Therefore, it can be determined that light spots LP3 and LP4 are not stationary objects. Furthermore, it can be determined that light spots LP1 and LP2, detected at the same location in both image 122C and image data C', are due to stationary objects.

[0061] Thus, when image data 122 of multiple images taken at the same location exist, the light spots detected in each image may include not only light spots caused by stationary objects but also light spots caused by moving objects such as vehicles. For example, by comparing the positions of each detected light spot across multiple images, if the position of the light spot does not change or the relative position between the light spots does not change, it can be determined that it is a light spot caused by a stationary object, and if the position of the light spot has changed significantly, it can be determined that it is a light spot caused by a moving object, thereby identifying the stationary object.

[0062] In other words, if a light point estimated to be a stationary object is identified based on an image 122C' taken at a certain location, and if that light point is also identified in an image 122C taken at the same location as image 122C' when vehicle 2 passes over that location again after image 122C' was taken, then the light point can be considered a stationary object and stationary object information 123 can be identified.

[0063] Furthermore, since other vehicles operating at night have their lights on, light spots caused by the lights of other vehicles are more easily detected in images taken at night. Therefore, from the viewpoint of improving the accuracy of detecting stationary objects, it is preferable to compare images taken at night with other images taken at the same location, and it is even more preferable to compare them with other images taken at the same location during the day.

[0064] Furthermore, the detection of stationary objects by comparing multiple image data 122 may be performed by comparing the multiple image data 122 in the time series in which the images were captured, and based on the amount of movement of each light point between the multiple images and the travel speed of the vehicle 2. For example, in the example in Figure 6, the amount of movement of light points LP3 and LP4 between images 122C and 122D is greater than the amount of movement of light points LP1 and LP2. If the amount of movement of light points LP3 and LP4 is greater than the amount of movement estimated from the travel speed of the vehicle 2, it can be considered that light points LP3 and LP4 are moving in the direction of the vehicle 2, and light points LP3 and LP4 can be identified as being caused by a moving object. Also, if the amount of movement of light points LP1 and LP2 is equal to the amount of movement estimated from the travel speed of the vehicle 2, light points LP1 and LP2 can be identified as being caused by a stationary object. Furthermore, if the amount of movement of light points LP1 and LP2 is smaller than the amount of movement estimated from the travel speed of vehicle 2, it is considered that light points LP1 and LP2 are moving in the same direction as the travel direction of vehicle 2, and therefore light points LP1 and LP2 are identified as being caused by a moving object.

[0065] The following describes a determination process to improve the accuracy of the stationary object information 123 by reconfirming whether or not a stationary object exists in the image data 122 for which stationary object information 123 has already been identified. Figure 7 is a flowchart showing an example of the determination process for stationary object information 123.

[0066] In step S131, the control unit 110 acquires reference image data. Specifically, the control unit 110 acquires the image data 122 of the image captured when the vehicle 2 passes again over the image capture position corresponding to the image data 122 for which the stationary object information 123 was identified, and when the illuminance sensor 33 outputs a signal indicating that the illuminance is above a predetermined value, as reference image data.

[0067] Next, in step S132, the control unit 110 identifies the position of a stationary object in the reference image indicated by the reference image data. The identification of the position of a stationary object in step S132 can be performed, for example, by the same process as in steps S31 to S34.

[0068] Next, in step S133, the control unit 110 determines whether the stationary object position in the reference image matches the stationary object position in the target image, which is an image for which the stationary object information 123 has already been identified. If they match (Yes in step S133), the control unit 110 determines that the identified stationary object information 123 is correct and terminates the process.

[0069] On the other hand, if there is no match (No in step S133), in step S134, the control unit 110 updates the stationary object information 123 and terminates. In step S134, for example, the control unit 110 determines that the stationary object positions that match between the reference image and the target image are correct, and updates the stationary object information 123 as incorrect for stationary object positions that do not match between the reference image and the target image.

[0070] Figure 8 is a schematic diagram showing an example of a reference image 122E used in the judgment process shown in Figure 7. Figure 9 is a schematic diagram showing an example of a target image 122F used in the judgment process shown in Figure 7. In this example, the reference image 122E is an image taken during the daytime, and the target image 122F is an image taken at night.

[0071] In reference image 122E, the process in step S132 identifies sign O1 and streetlights O2-O4 as stationary objects. Furthermore, the preceding vehicle C1 is stopped with its hazard lights on, and its rear lights BL1 and BL2 are illuminated. As a result, the process in step S132 incorrectly identifies rear lights BL1 and BL2 as stationary objects. Additionally, the oncoming vehicle C2 has its headlights HL1 and HL2 off because it is daytime. Therefore, headlights HL1 and HL2 are not identified as stationary objects.

[0072] In the target image 122F, the processing in step S30 identifies sign O1 and streetlights O2-O4 as stationary objects, and their surroundings are identified as stationary object regions Z1-Z4. Also, because it is nighttime, the preceding vehicle C1 has its taillights on, and its rear lights BL3 and BL4 are illuminated. As a result, the processing in step S30 incorrectly identifies the rear lights BL3 and BL4 as stationary objects. Similarly, the oncoming vehicle C4 also has its headlights HL3 and HL4 on, and the headlights HL3 and HL4 are incorrectly identified as stationary objects. Although not shown in the diagram, the areas around the rear lights BL3 and BL4, as well as the headlights HL3 and HL4, are also identified as stationary object regions.

[0073] Sign O1 and streetlights O2-O4 are located in the same position in both reference image 122E and target image 122F. Therefore, sign O1 and streetlights O2-O4 are determined to be stationary objects. On the other hand, rear lights BL1-BL4 and headlights HL3-HL4 are present in only one of the images. Therefore, rear lights BL1-BL4 and headlights HL3-HL4 are determined not to be stationary objects. As a result, in step S134, the stationary object information 123 is updated, determining that rear lights BL3-BL4 and headlights HL3-HL4 are not stationary objects.

[0074] Furthermore, the present invention is not limited to the embodiments described above, and can be freely modified and improved as appropriate. In addition, the material, shape, dimensions, numerical values, form, number, and placement of each component in the embodiments described above are arbitrary and not limited as long as they can achieve the present invention.

[0075] This application is based on Japanese Patent Application No. 2021-117823, filed on 16 July 2021, the contents of which are incorporated herein by reference.

Claims

1. An image acquisition unit that acquires image data of images captured by a sensor unit mounted on the vehicle, A unit that identifies stationary object information, which includes at least one of the following: an image containing one or more stationary objects from among self-illuminating objects, signs, delineators, and guardrails, or stationary object image data corresponding to a part of said image; and stationary object position information indicating the position of said stationary object calculated based on said image data, based on said image data and lighting information regarding whether or not the vehicle's headlights were illuminated when the image corresponding to said image data was captured; A transmission unit transmits to a storage unit the vehicle location information of the vehicle, which is obtained from a location information acquisition unit mounted on the vehicle, the vehicle location information when an image corresponding to the image data in which the stationary object information is identified is captured, and the stationary object information. Equipped with, The aforementioned identification unit is a stationary object information acquisition device mounted on a vehicle, which determines whether or not a light point present in an image captured by the sensor unit is caused by a stationary object, based on the amount of movement of a light point between a plurality of images captured by the sensor unit and the vehicle's travel speed.

2. The aforementioned identification unit identifies the stationary object information when the vehicle is in the first state, The first state includes the vehicle being stopped or moving slowly. The stationary object information acquisition device according to claim 1.

3. The transmitting unit transmits the vehicle position information and the stationary object information to the storage unit when the vehicle is in the second state. The second state includes the state in which the vehicle is stopped or moving slowly. The stationary object information acquisition device according to claim 2.

4. The specified unit is capable of identifying the stationary object region, which is the region in the image that contains the stationary object. The transmitting unit transmits data corresponding to the portion of the image that includes the area of ​​the stationary object as the stationary object image data. A stationary object information acquisition device according to any one of claims 1 to 3.

5. The identification unit is capable of identifying stationary object position information that indicates the position of the stationary object calculated based on the image data, The stationary object information transmitted by the transmitting unit includes the stationary object's position information. A stationary object information acquisition device according to any one of claims 1 to 3.

6. When the vehicle passes the position indicated by the vehicle position information at the time the image corresponding to the image data in which the stationary object information is identified is captured, and the illuminance sensor mounted on the vehicle and detecting the illuminance around the vehicle outputs a signal indicating that the illuminance is above a predetermined value, the transmission unit transmits the image data of the image captured by the sensor unit to the storage unit as reference image data. The stationary object information acquisition device according to claim 5.

7. When the vehicle passes the position indicated by the vehicle position information at the time the image corresponding to the image data in which the stationary object information has been identified is captured, and the illuminance sensor mounted on the vehicle and detecting the illuminance around the vehicle outputs a signal indicating that the illuminance is above a predetermined value, the image acquisition unit acquires the image data of the image captured by the sensor unit as reference image data. The system further includes a determination unit that determines, based on the reference image data, whether or not the stationary object exists at the location indicated by the stationary object position information calculated based on the image data corresponding to the reference image data. The stationary object information acquisition device according to claim 6.

8. The identifying unit is capable of determining whether a stationary object present in an image corresponding to the image data is a self-illuminating object, based on the image data of at least two images captured by the sensor unit before and after the switching timing of the headlights mounted on the vehicle. The stationary object information transmitted by the transmitting unit includes information regarding whether or not the stationary object is a self-illuminating object. A stationary object information acquisition device according to any one of claims 1 to 3.

9. The identifying unit identifies a light point estimated to be the stationary object based on a first image corresponding to a first image captured at a first position, and if the light point is present in a second image corresponding to a second image captured at the first position when the vehicle passes the first position again after the first image has been captured, the light point is considered the stationary object and the stationary object information is identified. A stationary object information acquisition device according to any one of claims 1 to 3.

10. A program that is executed in a computer device installed in a vehicle, which is equipped with a processor, The program is provided to the processor: Image acquisition step: acquires image data of an image captured by a sensor unit mounted on the vehicle, Identification step of identifying stationary object information, which includes at least one of the following: an image containing one or more stationary objects from among self-illuminating objects, signs, delineators, and guardrails, or stationary object image data corresponding to a part of said image, and stationary object position information indicating the position of said stationary object calculated based on said image data, based on said image data and lighting information regarding whether or not the vehicle's headlights were illuminated when the image corresponding to said image data was captured; A transmission step of transmitting to a storage unit the vehicle location information of the vehicle obtained from a location information acquisition unit mounted on the vehicle, the vehicle location information when an image corresponding to the image data in which the stationary object information is identified is captured, and the stationary object information. A program that executes, In the aforementioned specific step, the processor is programmed to determine, based on the amount of movement of light points between a plurality of images captured by the sensor unit and the vehicle's travel speed, whether or not the light points present in the images captured by the sensor unit are due to a stationary object.

11. A method for acquiring stationary object information, which is performed in a computer device mounted on a vehicle and includes a processor, The method for acquiring stationary object information involves the processor, Image acquisition step: acquires image data of an image captured by a sensor unit mounted on the vehicle, Identification step of identifying stationary object information, which includes at least one of the following: an image containing one or more stationary objects from among self-illuminating objects, signs, delineators, and guardrails, or stationary object image data corresponding to a part of said image, and stationary object position information indicating the position of said stationary object calculated based on said image data, based on said image data and lighting information regarding whether or not the vehicle's headlights were illuminated when the image corresponding to said image data was captured; A transmission step of transmitting to a storage unit the vehicle location information of the vehicle obtained from a location information acquisition unit mounted on the vehicle, the vehicle location information when an image corresponding to the image data in which the stationary object information is identified is captured, and the stationary object information. This includes causing the execution of A method for acquiring stationary object information, wherein in the specified step, the processor determines whether or not the light points present in the images captured by the sensor unit are due to a stationary object, based on the amount of movement of light points between a plurality of images captured by the sensor unit and the vehicle's travel speed.

Images