Stationary object information acquisition device, program, and stationary object information acquisition method

Abstract

A stationary object information acquisition device (100) mounted to a vehicle (2) is provided with: an image acquisition unit (111) that acquires image data (122) of an image captured by a sensor unit (31) mounted to the vehicle (2); an identification unit (112) for identifying, on the basis of the image data (122), stationary object information (123) which includes stationary object position information indicating the position of a stationary object calculated on the basis of the image data (122) and / or includes stationary object image data corresponding to an image where at least one stationary object selected from among a self-light-emitting body, a sign, a delineator, and a guardrail is present or to one part of the image; and a transmission unit (113) for transmitting, to a storage unit (220), the stationary object information (123) and vehicle position information (124) which is of the vehicle (2), which has been acquired from a position information acquisition unit 32 mounted to the vehicle (2), and which is at a time when an image, corresponding to the image data 122 having the stationary object information (123) identified therein, has been captured.

Description

Stationary object information acquisition device, program, and stationary object information acquisition method

[0001] The present disclosure relates to a stationary object information acquisition device, a program, and a stationary object information acquisition method.

[0002] In recent years, adaptive driving beam (ADB) technology has been proposed that blocks light from the positions of preceding and oncoming vehicles and blocks or reduces light from the positions of highly reflective objects such as signs, based on the situation around the vehicle. For example, Patent Document 1 describes a technology that detects a preceding vehicle and controls the light distribution forward.

[0003] Japanese Patent Application Publication No. 2011-246023

[0004] Generally, ADB light distribution control is performed based on target information sent from a vehicle. Each target is detected by a specific algorithm based on data acquired by a sensor such as a camera. However, depending on the accuracy of the data or the detection accuracy of the algorithm, a target may not be detected even though it exists (overdetection) or may be detected even though it does not exist (false detection).

[0005] However, if there are bright stationary objects on the road, such as street lights or signs, these stationary objects may be mistakenly recognized as a vehicle ahead. Also, the headlights of a vehicle ahead may be mistakenly recognized as a street light. If information on stationary objects on the road, such as street lights or signs, can be collected, it would be useful because it can be used to reduce the possibility of such misrecognition.

[0006] The present disclosure aims to collect stationary object information of stationary objects such as street lights and signs on roads.

[0007] A stationary object information acquisition device according to one aspect of the present disclosure is mounted on a vehicle and includes: an image acquisition unit that acquires image data of an image captured by a sensor unit mounted on the vehicle; an identification unit that identifies, based on the image data, stationary object information including at least one of: still object image data corresponding to an image in which one or more types of stationary objects, including a self-luminous object, a sign, a delineator, and a guardrail, or a portion of the image; and stationary object position information indicating the position of the stationary object calculated based on the image data; and a transmission unit that transmits, to a memory unit, vehicle position information of the vehicle acquired from a position information acquisition unit mounted on the vehicle, the vehicle position information at the time when an image corresponding to the image data in which the stationary object information was identified was captured, and the stationary object information.

[0008] A program according to one aspect of the present disclosure is a program that has a processor and is executed on a computer device mounted on a vehicle, and the program causes the processor to execute the following steps: an image acquisition step of acquiring image data of an image captured by a sensor unit mounted on the vehicle; an identification step of identifying, based on the image data, stationary object information including at least one of still object image data corresponding to an image in which one or more types of still objects selected from the group consisting of self-luminous objects, signs, delineators, and guardrails are present or a portion of the image, and stationary object position information indicating the position of the stationary object calculated based on the image data; and a transmission step of transmitting, to a memory unit, vehicle position information of the vehicle acquired from a position information acquisition unit mounted on the vehicle, the vehicle position information at the time when an image corresponding to the image data in which the stationary object information was identified, and the stationary object information.

[0009] A stationary object information acquisition method according to one aspect of the present disclosure is a stationary object information acquisition method that is executed in a computer device equipped with a processor and mounted on a vehicle, and the stationary object information acquisition method includes causing the processor to execute the following steps: an image acquisition step of acquiring image data of an image captured by a sensor unit mounted on the vehicle; an identification step of identifying, based on the image data, stationary object information including at least one of: still object image data corresponding to an image in which one or more types of stationary objects selected from the group consisting of self-luminous objects, signs, delineators, and guardrails are present, or a portion of the image; and stationary object position information indicating the position of the stationary object calculated based on the image data; and a transmission step of transmitting, to a memory unit, vehicle position information of the vehicle acquired from a position information acquisition unit mounted on the vehicle, the vehicle position information at the time when an image corresponding to the image data in which the stationary object information was identified, and the stationary object information.

[0010] According to the present disclosure, it is possible to collect stationary object information of stationary objects such as street lights and signs on roads.

[0011] FIG. 1 is a schematic diagram showing an example of a system including a stationary object information acquisition device according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of a system including a stationary object information acquisition device according to an embodiment of the present disclosure. FIG. 3 is a flowchart showing an example of a method for acquiring stationary object information according to an embodiment of the present disclosure. FIG. 4 is a flowchart showing an example of a process for specifying the stationary object information shown in FIG. 3. FIG. 5 is a schematic diagram for explaining stationary object position information. FIG. 6 is a schematic diagram showing image capture timings and image data acquired at each image capture timing. FIG. 7 is a flowchart showing an example of a process for determining stationary object information. FIG. 8 is a schematic diagram showing an example of reference image data used in the process for specifying stationary object information. FIG. 9 is a schematic diagram showing an example of image data used in the process for specifying stationary object information.

[0012] The present invention will be described below based on embodiments with reference to the drawings. The same or equivalent components, parts, and processes shown in each drawing will be given the same reference numerals, and redundant explanations will be omitted where appropriate. Furthermore, the embodiments are illustrative and do not limit the invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention.

[0013] (System) First, a system 1 including a stationary object information acquisition device 100 according to an embodiment of the present disclosure will be described using Figures 1 and 2. Figure 1 is a schematic diagram showing the system 1 according to an embodiment of the present disclosure. As shown in Figure 1, the system 1 includes a stationary object information storage device 200 and a plurality of vehicles 2, such as vehicles 2A and 2B, each equipped with a stationary object information acquisition device 100. The stationary object information storage device 200 and each vehicle 2 can be communicatively connected to each other via wireless communication.

[0014] The stationary object information acquisition device 100 acquires stationary object information related to stationary objects and transmits the stationary object information to the stationary object information storage device 200. The stationary object information storage device 200, for example, stores the stationary object information received from each stationary object information acquisition device 100. The stationary object information storage device 200 also analyzes the received stationary object information to improve the accuracy of the stationary object information, acquire more detailed information, and create a light distribution pattern based on the stationary object information. The stationary object information storage device 200 also transmits the improved accuracy of the stationary object information, etc. to each vehicle 2 in response to a request from each vehicle 2. Each vehicle 2 can, for example, use the improved accuracy of the stationary object information, etc. received from the stationary object information storage device 200 to improve the accuracy and efficiency of target detection and appropriately control the light distribution of its headlights.

[0015] Here, in this embodiment, a "stationary object" refers to an object that is fixed to the road and has high brightness, and specifically refers to one or more of self-luminous objects (e.g., street lights, traffic lights, etc.), signs, delineators, and guardrails. That is, the stationary object information acquisition device 100 in this embodiment acquires stationary object information regarding the various stationary objects listed as the specific examples above. Note that, in another embodiment, the stationary object information acquisition device 100 may be configured to be able to identify, as a stationary object, an object that is not included in the specific examples above, is fixed to the road, has high brightness, and may affect target detection.

[0016] 2 is a block diagram showing a system 1 according to an embodiment of the present disclosure. The vehicle 2 includes a vehicle ECU (Electronic Control Unit) 10, a storage unit 20, a sensor unit 31, a position information acquisition unit 32, an illuminance sensor 33, and a stationary object information acquisition device 100. The vehicle 2 can be communicatively connected to the stationary object information storage device 200 by wireless communication via a communication network 3. The means of wireless communication is not particularly limited, and may be, for example, a mobile communication system such as a telematics system for automobiles, cooperation with a smartphone, or in-vehicle Wi-Fi.

[0017] The vehicle ECU 10 controls various operations such as driving of the vehicle 2. The vehicle ECU 10 includes a processor such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a general-purpose CPU (Central Processing Unit). The storage unit 20 includes, for example, a ROM (Read Only Memory) in which various vehicle control programs are stored, and a RAM (Random Access Memory) in which various vehicle control data are temporarily stored. The processor of the vehicle ECU 10 loads data specified by the various vehicle control programs stored in the ROM onto the RAM, and controls various operations of the vehicle 2 in cooperation with the RAM.

[0018] The sensor unit 31 outputs image data of an image captured outside the vehicle 2. The sensor unit 31 includes, for example, one or more sensors selected from a visible light camera, LiDAR, and millimeter-wave radar. The image data output by the LiDAR or millimeter-wave radar may be three-dimensional image data. The position information acquisition unit 32 outputs vehicle position information indicating the current position of the vehicle 2. The position information acquisition unit 32 includes, for example, a GPS (Global Positioning System) sensor. The illuminance sensor 33 detects and outputs illuminance around the vehicle 2.

[0019] The stationary object information acquisition device 100 includes a control unit 110 and a storage unit 120. The control unit 110 is configured, for example, by a processor such as a CPU. The control unit 110 can be configured, for example, as part of a lamp ECU that controls the operation of lamps such as headlights in the vehicle 2. The control unit 110 may also be configured, for example, as part of the vehicle ECU 10. The storage unit 120 is configured, for example, by a ROM, a RAM, etc. The storage unit 120 may be configured as part of a storage device provided for the storage unit 20 or the lamp ECU.

[0020] The control unit 110 functions as an image acquisition unit 111, an identification unit 112, a transmission / reception unit 113, and a determination unit 114 by reading the program 121 stored in the storage unit 120. Note that some of these functions may be realized by the vehicle ECU 10 or the lamp ECU. In such a configuration, the vehicle ECU 10 or the lamp ECU constitutes part of the stationary object information acquisition device 100. The program 121 may also be recorded on a non-transitory computer-readable medium.

[0021] The image acquisition unit 111 acquires image data 122 of an image captured by the sensor unit 31. The acquired image data 122 is stored in the storage unit 120. The image acquisition unit 111 also acquires, from the position information acquisition unit 32, vehicle position information 124 (i.e., image capture position information indicating the image capture position) at the time when an image corresponding to the acquired image data 122 was captured. The vehicle position information 124 preferably includes information indicating the orientation of the vehicle 2 at the time the image was captured. The vehicle position information 124 may also include information indicating the position of the vehicle in the vehicle width direction. The position of the vehicle in the vehicle width direction can be calculated, for example, by detecting a driving lane and using the driving lane as a reference. The acquired vehicle position information 124 is stored in the storage unit 120. The vehicle position information 124 is, for example, associated with the corresponding image data 122 and stored in the storage unit 120.

[0022] The image acquisition unit 111 may acquire time information indicating the time when the image was captured. The time information may include information indicating the date when the image was captured. The image acquisition unit 111 may also acquire lighting information regarding whether the headlights of the vehicle 2 were on when the image was captured. The time information and the lighting information are stored in the storage unit 120, for example, in association with the corresponding image data 122.

[0023] Furthermore, the image acquisition unit 111 may acquire, as reference image data, image data 122 captured when the illuminance sensor 33 outputs a signal indicating that the illuminance is equal to or greater than a predetermined value (e.g., 1000 lux). Here, the illuminance equal to or greater than the predetermined value refers to, for example, illuminance equal to or greater than a value determined to be daytime. That is, the image acquisition unit 111 may store, in the storage unit 120, image data 122 of an image captured during the day as reference image data. Furthermore, the image acquisition unit 111 may acquire, from the illuminance sensor 33, illuminance information indicating the illuminance around the vehicle 2 when the image was captured, and associate the image data 122 with the illuminance information and store them in the storage unit 120. In this case, image data 122 in which the illuminance indicated by the associated illuminance information is equal to or greater than a predetermined value may be the reference image data.

[0024] The identification unit 112 identifies stationary object information 123 based on the image data 122. The stationary object information 123 identified by the identification unit 112 is stored in the storage unit 120. Here, the "stationary object information" refers to information including at least one of still object image data corresponding to an image in which a stationary object exists or a part of the image, and stationary object position information indicating the position of the stationary object calculated based on the image data 122.

[0025] The identification unit 112, for example, detects a still object in an image by image analysis and includes image data 122 of the image in which the still object is detected as still object image data in the still object information 123. Furthermore, the identification unit 112, for example, identifies an area in the image in which the still object is detected that includes the still object as a still object area, and includes data corresponding to the still object area, which is a part of the image, as still object image data in the still object information 123. Furthermore, the identification unit 112, for example, calculates the position of the still object based on the image in which the still object is detected, and includes still object position information indicating the position of the still object in the still object information 123. The still object position information may, for example, be information indicating the position of the still object in the image (e.g., coordinates or size of the position where the still object is located in the image), or may indicate the distance or direction from the capture position of the image to the still object. Furthermore, the identification unit 112 may identify the type of the still object and include information indicating the type in the still object information 123.

[0026] The transmitter / receiver unit 113 transmits and receives information between the vehicle ECU 10 and the stationary object information storage device 200. That is, the transmitter / receiver unit 113 functions as a transmitter and a receiver. The transmitter / receiver unit 113 transmits stationary object information 123 and vehicle position information 124 corresponding to the stationary object information 123 (at the time when an image corresponding to the image data 122 from which the stationary object information 123 was identified was captured) to the stationary object information storage device 200, which includes a storage unit 220. The transmitter / receiver unit 113 can also transmit reference image data to the stationary object information storage device 200. The transmitter / receiver unit 113 can also transmit and receive other information to and from the stationary object information storage device 200 as necessary.

[0027] The determination unit 114 determines whether or not a stationary object exists at the position indicated by the stationary object position information calculated by the identification unit 112, based on reference image data captured at the same position as the image capturing position of the image corresponding to the image data 122 used to calculate the stationary object position. The reference image data used by the determination unit 114 for the determination is, for example, image data 122 of an image captured when the vehicle 2 passes through the position indicated by the vehicle position information 124 corresponding to the image data 122 for which the stationary object information has been identified by the identification unit 112, and the illuminance sensor 33 outputs a signal indicating that the illuminance is equal to or greater than a predetermined value, and is acquired by the image acquisition unit 111.

[0028] The stationary object information storage device 200 includes a control unit 210 and a storage unit 220. In this embodiment, the stationary object information storage device 200 is a computer device that aggregates and stores information transmitted from multiple vehicles 2, and is installed, for example, in a data center. The control unit 210 is configured, for example, by a processor such as a CPU. The storage unit 220 is configured, for example, by a ROM, a RAM, etc.

[0029] The control unit 210 functions as a transmission / reception unit 211, a recording unit 212, and a determination unit 213 by reading a program 221 stored in the storage unit 220. The program 221 may be recorded on a non-transitory computer-readable medium.

[0030] The transmitting / receiving unit 211 transmits and receives information to and from the vehicle ECU 10 and the stationary object information acquisition device 100. The transmitting / receiving unit 211 receives the stationary object information 123 transmitted from the transmitting / receiving unit 113 and the vehicle position information 124 corresponding to the stationary object information 123. The transmitting / receiving unit 211 can also receive reference image data from the stationary object information acquisition device 100. The transmitting / receiving unit 211 can also transmit and receive other information to and from the vehicle ECU 10 and the stationary object information acquisition device 100 as necessary.

[0031] The recording unit 212 associates the stationary object information 123 received by the transmitting / receiving unit 211 with the vehicle position information 124 corresponding to the stationary object information 123 and records the information in the stationary object database 222. The recording unit 212 can update the stationary object database 222 based on the result of the determination by the determination unit 213.

[0032] The stationary object database 222 stores vehicle position information 124 and stationary object information 123 in association with each other. In the stationary object database 222, for example, a plurality of still object image data may be recorded for one imaging position indicated by the vehicle position information 124. Furthermore, the stationary object database 222 may store still object image data and reference image data that are captured at the same imaging position in association with each other. The stationary object database 222 may store information such as the position, size, distance and direction from the imaging position, and type of stationary object in association with the imaging position.

[0033] The determination unit 213 determines whether or not a stationary object is included in the stationary object information 123 using an algorithm different from the algorithm used by the identification unit 112 to identify the stationary object information 123. The algorithm used by the determination unit 213 is preferably an algorithm that has higher stationary object detection accuracy than the algorithm used by the identification unit 112, for example.

[0034] The determination unit 213 determines whether or not a still object is included in an image, for example, by using an image corresponding to still object image data included in the still object information 123. The determination unit 213 may use the image corresponding to the still object image data to identify detailed information such as the position and size of the still object in the image, the distance and direction from the image capture position of the image to the still object, and the type of the still object.

[0035] As another example of the system 1 described above, the stationary object information storage device 200 may be mounted on the vehicle 2. In this case, the control unit 210 and the storage unit 220 may be provided separately from the vehicle ECU 10, the control unit 110, the storage unit 20, and the storage unit 120. On the other hand, the control unit 210 may be configured as part of, for example, one or more of the lamp ECU, the vehicle ECU 10, and the control unit 110. Some of the functions of the control unit 210 may be implemented by the vehicle ECU 10 or the lamp ECU. The storage unit 220 may be configured as part of, for example, one or more of the storage units 20, 120, or storage devices provided for the lamp ECU. When the stationary object information storage device 200 is mounted on the vehicle 2, the stationary object information acquisition device 100 and the stationary object information storage device 200 are configured to be connectable via wireless communication or wired communication.

[0036] (Stationary Object Information Acquisition Method) Next, a stationary object information acquisition method by the stationary object information acquisition device 100 according to this embodiment will be described. The stationary object information acquisition method according to this embodiment is executed, for example, by the control unit 110 of the stationary object information acquisition device 100 that has loaded the program 121. In the following description, an example will be given in which the stationary object information acquisition device 100 identifies the stationary object information 123 using an image captured by a visible camera, but the present disclosure is not limited to this. The stationary object information acquisition device 100 may identify the stationary object information 123 using an image output by, for example, millimeter-wave radar or LiDAR.

[0037] 3 is a flowchart showing an example of a method for acquiring stationary object information according to this embodiment. Note that the order of the processes constituting each flowchart described in this specification may be random, and may be executed in parallel, as long as no contradictions or inconsistencies occur in the process content.

[0038] First, in step S10, the control unit 110 acquires image data, etc. Specifically, the control unit 110 acquires image data of an image captured by a visible light camera. The control unit 110 also acquires vehicle position information 124 corresponding to the image data.

[0039] In step S10, the control unit 110 also preferably acquires one or more of time information indicating the time when the image was captured, lighting information indicating whether the headlights of the vehicle 2 were on when the image was captured, and illuminance information indicating the illuminance around the vehicle 2 when the image was captured. By acquiring this information, it becomes possible to appropriately compare the images, and as a result, the accuracy of detecting stationary objects can be improved.

[0040] Here, the visible camera is controlled by the vehicle ECU to capture images of the exterior of the vehicle 2 at predetermined time intervals, for example. The control unit 110 preferably thins out and acquires the image data 122 of images captured at predetermined time intervals, for example, at a time interval longer than the image capture time (e.g., 0.1 to 1 second) or at predetermined distance intervals (e.g., 1 to 10 meters). Thinning out the image data 122 can prevent the storage unit 120 from becoming too large. Furthermore, this reduces the number of targets for the identification process in step S30, which will be described later, thereby reducing the burden on the control unit 110. Note that the control unit 110 may, for example, acquire all of the image data 122 of images captured at predetermined time intervals and temporarily store them in the storage unit 120, and then thin out the image data 122 at a predetermined timing, such as before the identification process in step S30.

[0041] Furthermore, the image acquisition unit 111 may thin out the image data 122 based on whether the image was captured in a location where the vehicle 2 usually travels frequently. Specifically, the image acquisition unit 111 may thin out the image data 122 of an image captured on a road where the number of times the vehicle 2 has traveled in a predetermined period of time is less than a predetermined number (for example, less than once in the past month). This is because identifying a stationary object in a location where the vehicle 2 does not usually travel is not very useful to the user of the vehicle 2. In particular, when the stationary object information storage device 200 is installed in the vehicle 2, it is preferable to thin out the image data 122 based on the number of times the vehicle 2 has traveled through the image capture position in a predetermined period of time.

[0042] Next, if the vehicle 2 is in the first state (Yes in step S20), the control unit 110 executes, in step S30, a process for identifying the stationary object information 123. On the other hand, if the vehicle 2 is not in the first state (No in step S20), the control unit 110 waits to execute the process for identifying the vehicle 2 in step S30 until the vehicle 2 enters the first state.

[0043] Here, the "first state" refers to a state in which the processing load on the vehicle ECU 10 or the lamp ECU is considered to be small. The "first state" includes, for example, a stopped state or a slow-moving state (e.g., traveling at a speed of 10 km / h or less). In the case where the control unit 110 is configured as a part of the vehicle ECU 10 or the lamp ECU, configuring the control unit 110 to execute the specific processing of step S30 when the vehicle 2 is in the first state reduces the load on the vehicle ECU 10 or the lamp ECU. Note that, in the case where the control unit 110 is configured independently of the vehicle ECU 10 and the lamp ECU, the determination of step S20 does not need to be executed.

[0044] In step S30, the control unit 110 executes a specification process for specifying the stationary object information 123 based on the image data 122. Details of the specification process will be described later with reference to FIG.

[0045] Next, if the vehicle 2 is in the second state (Yes in step S40), the control unit 110 transmits the stationary object information 123 and the vehicle position information 124 corresponding to the stationary object information 123 to the stationary object information storage device 200 including the storage unit 220 in step S50, and then ends the process. In step S50, time information, lighting information, illuminance information, and the like may also be transmitted. On the other hand, if the vehicle 2 is not in the second state (No in step S40), the control unit 110 waits to execute the transmission process in step S50 until the vehicle 2 enters the second state.

[0046] Here, the "second state" refers to a state in which the processing load on the vehicle ECU 10 or the lamp ECU is considered to be small. The "second state" includes, for example, a stopped state or a slow-moving state (e.g., traveling at a speed of 10 km / h or less). In the case where the control unit 110 is configured as a part of the vehicle ECU 10 or the lamp ECU, the load on the vehicle ECU 10 or the lamp ECU can be reduced by configuring the control unit 110 to execute the transmission process of step S50 when the vehicle 2 is in the second state. Note that, in the case where the control unit 110 is configured independently of the vehicle ECU 10 and the lamp ECU, the determination of step S40 does not need to be executed.

[0047] The still object information 123 transmitted in step S50 may be still object image data of an image identified as containing a still object, or may be still object position information calculated from the image, or both. If the transmitted still object information 123 includes still object image data, the still object image data can be further examined in the still object information storage device 200, enabling more accurate information to be obtained. On the other hand, if the transmitted still object information 123 does not include still object image data, this is advantageous in that the volume of data to be transmitted is reduced.

[0048] The process of identifying the stationary object information 123 in step S30 will be described in detail below. Fig. 4 is a flowchart showing an example of the process of identifying the stationary object information 123. In step S31, the control unit 110 detects light spots in the image. The detection of light spots can be performed using a conventionally known technique, for example, by analyzing the brightness of the image.

[0049] In step S32, the control unit 110 performs pattern recognition processing on the image. The pattern recognition method can be a conventionally known method, and for example, a machine learning model may be used to detect stationary objects, or a clustering method may be used to detect stationary objects.

[0050] Next, in step S33, control unit 110 determines whether or not there is a stationary object in the image based on the results of the processing in step S31 and / or step S32. If it is determined that there is no stationary object in the image (No in step S33), in step S35, control unit 110 deletes image data 122 corresponding to that image from storage unit 120, and ends the process.

[0051] If it is determined that a still object is present in the image (Yes in step S33), in step S34, the control unit 110 identifies a still object region or a still object position in the image. By identifying the still object region and treating the data of the portion of the image that includes the still object region as still object image data, the data volume when transmitted to the still object information storage device 200 can be reduced. In this case, it is preferable to also identify information indicating the position of the still object region in the original image and include it in the still object information 123. Alternatively, the still object image data may be processed to reduce the data volume of the region excluding the still object region.

[0052] The stationary object position is, for example, the position of a stationary object in an image. The stationary object position can be specified, for example, using an arbitrary coordinate system set in the image. The stationary object position may indicate, for example, the center point of the stationary object or the position of the outer edge of the stationary object. Furthermore, the stationary object position preferably includes information regarding the size specified using the coordinate system.

[0053] FIG. 5 is a schematic diagram for explaining stationary object position information. In the image shown in FIG. 5, sign O1 and street lights O2 to O4 are identified as stationary objects. In this case, for example, the positions of areas Z1 to Z4 including sign O1 and street lights O2 to O4, respectively, can be defined using coordinates defined by the x-axis and y-axis, which can serve as stationary object position information. Note that there are no particular limitations on the method for setting the coordinates; for example, the center of the image can be used as the origin. Furthermore, in the example of FIG. 5, areas Z1 to Z4 do not include the support poles of sign O1 and street lights O2 to O4, but areas including these support poles can also be used as stationary object positions.

[0054] The stationary object position identified in step S34 may indicate the distance or direction from the image capture position to the stationary object. For example, if the image data 122 includes depth information, the distance or direction from the image capture position to the stationary object may be calculated using the depth information. Alternatively, the distance or direction may be calculated by comparing the image data 122 with other image data 122 captured near the image capture position, or may be calculated using data acquired from millimeter-wave radar or LiDAR.

[0055] When the stationary object position is identified, the image data 122 may be deleted from the storage unit 120, or may be associated with the stationary object position information and included in the stationary object information 123. After step S34, the process proceeds to step S40 in FIG.

[0056] When the identification process shown in FIG. 4 is performed based on image data 122 captured during the day when the illuminance is equal to or greater than a predetermined value, it becomes easier to grasp the contours of structures in the image and to obtain color information of the structures from the image, thereby improving the accuracy of detecting stationary objects using pattern recognition processing.

[0057] The process of identifying the stationary object information 123 may be performed by comparing multiple sets of image data 122 captured at the same location or at locations nearby each other. In addition, in step S34, the control unit 110 may identify the type of the stationary object based on the results of step S31 and / or step S32, and include type information in the stationary object information 123. Below, an example of a method for identifying whether a stationary object is a self-luminous object and an example of detecting a stationary object by comparing multiple sets of image data 122 will be described with reference to FIG. 6 .

[0058] FIG. 6 is a schematic diagram showing the image capture timings and the images 122A-D captured at each capture timing. In the example of FIG. 6, the visible light camera captures images of the area ahead of the vehicle 2 at times T1, T2, T3, and T4, and outputs image data 122 of the images 122A-D. The intervals F1-F4 between each time are all the same. That is, the images 122A-D are image data captured at regular time intervals. Furthermore, the headlights are on at times T1, T3, and T4, and off at time T2. Between times T1 and T4, the vehicle 2 is traveling forward at a predetermined speed.

[0059] Whether a stationary object present in an image corresponding to image data 122 is a self-luminous object can be determined, for example, based on the image data 122 of at least two images captured before and after the timing of switching on and off the headlights mounted on vehicle 2. In FIG. 6 , light points LP1 and LP2 are detected in image 122A. Meanwhile, light point LP1 is detected in image 122B, but no light point is detected at the position where light point LP2 is estimated to be detected (the position indicated by the dotted line). Furthermore, light point LP2 is detected again in image 122C. From these findings, it can be determined that light point LP2 is detected as a light point resulting from reflection of light from the headlights, and that light point LP2 is not caused by a self-luminous object. Furthermore, light point LP1, which is also detected in image 122B captured when the headlights are not on, can be determined to be caused by a self-luminous object.

[0060] Detecting stationary objects by comparing multiple images can be performed, for example, by comparing images captured at the same location or adjacent locations. In FIG. 6, image 122C' is an image captured at the same location as image 122C, prior to image 122C. In image 122C, light points LP1 and LP2 are detected, as well as light points LP3 and LP4. Meanwhile, in image 122C', light points LP1 and LP2 are detected, but light points LP3 and LP4 are not. If light points LP3 and LP4 were stationary objects, they would also be detected in image 122C', but in reality, light points LP3 and LP4 are not detected in image 122C'. Therefore, it can be determined that light points LP3 and LP4 are not stationary objects. Furthermore, it can be determined that light points LP1 and LP2 detected at the same location in both image 122C and image data C' are due to stationary objects.

[0061] In this way, when there is image data 122 of a plurality of images taken at the same point, the light spots detected from each image may include not only light spots caused by stationary objects but also light spots caused by moving objects such as vehicles. For example, by comparing the positions of each detected light spot between a plurality of images, if the position of the light spot does not change or the relative position between the light spots does not change, it is determined that the light spot is caused by a stationary object, and if the position of the light spot has changed significantly, it is determined that the light spot is caused by a moving object, and it is also possible to identify a stationary object.

[0062] In other words, a light spot that is estimated to be a stationary object is identified based on image 122C' captured at a certain position, and if the light spot is also identified from image 122C captured at the same position as image 122C' when vehicle 2 passes that position again after image 122C' is captured, the light spot can be considered to be a stationary object and stationary object information 123 can be identified.

[0063] Since other vehicles traveling at night have their headlights on, it is easier to detect light spots caused by the headlights of other vehicles from images captured at night. Therefore, from the perspective of improving the accuracy of detecting stationary objects, it is preferable to compare images captured at night with other images captured at the same position, and it is even more preferable to compare them with other images captured at the same position during the day.

[0064] Furthermore, detection of stationary objects by comparing multiple image data 122 may be performed by comparing the multiple image data 122 in the time series in which the images were captured, and based on the amount of movement of each light spot between the multiple images and the traveling speed of vehicle 2. For example, in the example of FIG. 6, the amount of movement of light spots LP3 and LP4 between images 122C and 122D is greater than the amount of movement of light spots LP1 and LP2. If the amount of movement of light spots LP3 and LP4 is greater than the amount of movement estimated from the traveling speed of vehicle 2, light spots LP3 and LP4 are considered to be moving toward vehicle 2, and light spots LP3 and LP4 can be identified as being caused by a moving object. Furthermore, if the amount of movement of light spots LP1 and LP2 is equal to the amount of movement estimated from the traveling speed of vehicle 2, light spots LP1 and LP2 can be identified as being caused by a stationary object. Furthermore, if the amount of movement of light points LP1 and LP2 is smaller than the amount of movement estimated from the traveling speed of vehicle 2, light points LP1 and LP2 are considered to be moving in the same direction as the traveling direction of vehicle 2, and therefore light points LP1 and LP2 are identified as being caused by a moving object.

[0065] The following describes a determination process for reconfirming whether or not a stationary object exists in image data 122 for which stationary object information 123 has already been specified, thereby improving the accuracy of the stationary object information 123. FIG. 7 is a flowchart showing an example of the determination process for the stationary object information 123.

[0066] In step S131, the control unit 110 acquires reference image data. Specifically, the control unit 110 acquires, as reference image data, the image data 122 of an image captured when the vehicle 2 passes again through the imaging position of the image corresponding to the image data 122 for which the stationary object information 123 has been specified and the illuminance sensor 33 outputs a signal indicating that the illuminance is equal to or greater than a predetermined value.

[0067] Next, in step S132, the control unit 110 identifies the position of a stationary object in the reference image represented by the reference image data. The identification of the position of the stationary object in step S132 can be performed, for example, by the same processing as in steps S31 to S34.

[0068] Next, in step S133, the control unit 110 determines whether the position of the still object in the reference image matches the position of the still object in the target image, which is an image for which the still object information 123 has been specified. If they match (Yes in step S133), the control unit 110 determines that the specified still object information 123 is correct, and ends the process.

[0069] On the other hand, if there is no match (No in step S133), in step S134, the control unit 110 updates the stationary object information 123 and ends the process. In step S134, for example, the control unit 110 determines that the stationary object positions whose positions match between the reference image and the target image are correct, and determines that the stationary object positions whose positions do not match between the reference image and the target image are incorrect, and updates the stationary object information 123.

[0070] Fig. 8 is a schematic diagram showing an example of a reference image 122E used in the determination process shown in Fig. 7. Fig. 9 is a schematic diagram showing an example of a target image 122F used in the determination process shown in Fig. 7. In this example, the reference image 122E is an image captured during the day, and the target image 122F is an image captured at night.

[0071] In reference image 122E, sign O1 and street lights O2-O4 have been identified as stationary objects by the processing of step S132. Furthermore, preceding vehicle C1 is stopped with its hazard lights on and its rear lamps BL1 and BL2 are on. As a result, it is assumed that the processing of step S132 has erroneously detected the rear lamps BL1 and BL2 as stationary objects. Furthermore, oncoming vehicle C2 has its headlights HL1 and HL2 turned off because it is daytime. Therefore, it is assumed that the headlights HL1 and HL2 have not been identified as stationary objects.

[0072] In the target image 122F, the processing of step S30 identifies the sign O1 and street lights O2-O4 as stationary objects, and the areas surrounding each of them are identified as stationary object regions Z1-Z4. Furthermore, because it is nighttime, the preceding vehicle C1 has its taillights on, and its rear lamps BL3 and BL4 are on. As a result, the processing of step S30 erroneously detects the rear lamps BL3 and BL4 as stationary objects. Similarly, the oncoming vehicle C4 also has its headlights HL3 and HL4 on, and the headlights HL3 and HL4 are erroneously detected as stationary objects. Although not shown, the rear lamps BL3 and BL4 and the areas surrounding the headlights HL3 and HL4 are also identified as stationary object regions.

[0073] The sign O1 and the street lights O2-O4 are present in the same position in both the reference image 122E and the target image 122F. Therefore, the sign O1 and the street lights O2-O4 are determined to be stationary objects. On the other hand, the rear lights BL1-BL4 and the headlights HL3-HL4 are present in only one of the images. Therefore, the rear lights BL1-BL4 and the headlights HL3-HL4 are determined to be non-stationary objects. As a result, in step S134, the stationary object information 123 is updated to indicate that the rear lights BL3-BL4 and the headlights HL3-HL4 are not stationary objects.

[0074] The present invention is not limited to the above-described embodiments, and can be freely modified, improved, etc. The material, shape, dimensions, numerical values, form, number, location, etc. of each component in the above-described embodiments are arbitrary and not limited as long as the present invention can be achieved.

[0075] This application is based on Japanese Patent Application No. 2021-117823, filed on July 16, 2021, the contents of which are incorporated herein by reference.

Claims

1. an image acquisition unit that acquires image data of an image captured by a sensor unit mounted on the vehicle; an identification unit that identifies stationary object information based on the image data, the stationary object information including at least one of stationary object image data corresponding to an image or a part of the image in which one or more types of stationary object among a self-luminous object, a sign, a delineator, and a guardrail are present, and stationary object position information indicating the position of the stationary object calculated based on the image data; a transmission unit that transmits to a storage unit vehicle position information of the vehicle acquired from a position information acquisition unit mounted on the vehicle, the vehicle position information at the time when an image corresponding to the image data in which the stationary object information is specified is captured, and the stationary object information; A stationary object information acquisition device mounted on a vehicle.

2. the identification unit is configured to identify the stationary object information when the vehicle is in a first state, The first state includes the vehicle being stopped or moving slowly. The stationary object information acquisition device according to claim 1 .

3. the transmission unit transmits the vehicle position information and the stationary object information to the storage unit when the vehicle is in a second state; The second state includes the vehicle being stopped or moving slowly. The stationary object information acquisition device according to claim 2 .

4. the identification unit is capable of identifying a stationary object region in the image, the stationary object being a region including the stationary object; the transmission unit transmits data corresponding to a portion of the image including the still object region as the still object image data. The stationary object information acquisition device according to any one of claims 1 to 3.

5. the identification unit is capable of identifying stationary object position information indicating a position of the stationary object calculated based on the image data, The stationary object information transmitted by the transmission unit includes the stationary object position information. The stationary object information acquisition device according to any one of claims 1 to 3.

6. When the vehicle passes through a position indicated by the vehicle position information at the time when an image corresponding to the image data in which the still object information is specified was captured, and an illuminance sensor mounted on the vehicle and detecting the illuminance around the vehicle outputs a signal indicating that the illuminance is equal to or greater than a predetermined value, the transmission unit transmits the image data of the image captured by the sensor unit to the storage unit as reference image data. The stationary object information acquisition device according to claim 5 .

7. When the vehicle passes through a position indicated by the vehicle position information at the time when an image corresponding to the image data in which the still object information is specified was captured, and an illuminance sensor mounted on the vehicle and detecting illuminance around the vehicle outputs a signal indicating that the illuminance is equal to or greater than a predetermined value, the image acquisition unit acquires image data of the image captured by the sensor unit as reference image data, a determination unit that determines, based on the reference image data, whether or not the stationary object is present at a position indicated by the stationary object position information calculated based on the image data corresponding to the reference image data, The stationary object information acquisition device according to claim 6.

8. the identification unit is capable of identifying whether a stationary object present in an image corresponding to the image data is a self-luminous object based on image data of at least two images captured by the sensor unit before and after a timing of switching between turning on and off a headlight mounted on the vehicle, The stationary object information transmitted by the transmission unit includes information regarding whether the stationary object is a self-luminous object. The stationary object information acquisition device according to any one of claims 1 to 3.

9. the identification unit identifies a light spot estimated to be the stationary object based on a first image corresponding to first image data captured at a first position, and when the light spot is present in a second image corresponding to second image data captured at the first position when the vehicle passes the first position again after the first image is captured, the identification unit regards the light spot as the stationary object and identifies the stationary object information; The stationary object information acquisition device according to any one of claims 1 to 3.

10. A program to be executed on a computer device equipped with a processor and mounted on a vehicle, The program causes the processor to: an image acquiring step of acquiring image data of an image captured by a sensor unit mounted on the vehicle; a step of identifying stationary object information based on the image data, the stationary object information including at least one of stationary object image data corresponding to an image or a part of the image in which one or more types of stationary object among a self-luminous object, a sign, a delineator, and a guardrail are present, and stationary object position information indicating a position of the stationary object calculated based on the image data; a transmitting step of transmitting, to a storage unit, vehicle position information of the vehicle acquired from a position information acquiring unit mounted on the vehicle, the vehicle position information at the time when an image corresponding to the image data in which the stationary object information is specified was captured, and the stationary object information; A program to execute.

11. A stationary object information acquisition method executed in a computer device equipped with a processor and mounted on a vehicle, comprising: The stationary object information acquisition method includes: an image acquiring step of acquiring image data of an image captured by a sensor unit mounted on the vehicle; a step of identifying stationary object information based on the image data, the stationary object information including at least one of stationary object image data corresponding to an image or a part of the image in which one or more types of stationary object among a self-luminous object, a sign, a delineator, and a guardrail are present, and stationary object position information indicating a position of the stationary object calculated based on the image data; a transmitting step of transmitting, to a storage unit, vehicle position information of the vehicle acquired from a position information acquiring unit mounted on the vehicle, the vehicle position information at the time when an image corresponding to the image data in which the stationary object information is specified was captured, and the stationary object information; A stationary object information acquisition method comprising: