Data collection device, system, method, vehicle control device and storage medium

Abstract

The present invention provides a data collection device, system, method, vehicle control device, and storage medium that can easily collect learning data for a determination model for determining obstacles on the road. The data collection device in one embodiment includes: an acquisition unit that acquires images captured around a first vehicle; a first determination unit that determines, based on information indicating the first vehicle's driving history, whether the first vehicle has changed lanes during a determination period; and a collection unit that, if the first determination unit determines that the first vehicle has not changed lanes, collects the images included in the determination period as learning data for a determination model for determining obstacles on the road.

Description

Technical Field

[0001] The present invention relates to a data collection device, a vehicle control device, a data collection system, a data collection method and a storage medium. Background Art

[0002] Conventionally, there is known a technology for determining the presence or absence of obstacles on a road based on an image of the road captured by a vehicle-mounted camera (for example, see Japanese Patent Application Laid-Open No. 2020-87037). Summary of the Invention

[0003] Problems to be solved by the invention

[0004] Conventional technology requires preparing a large amount of learning data to generate a model for determining the presence of obstacles. For example, manually selecting images without obstacles from a large number of road images and labeling them requires labor and time to prepare such learning data. Furthermore, when learning images containing obstacles, the low frequency of obstacle appearance makes it difficult to prepare a large amount of learning data.

[0005] The present invention was completed in consideration of such a situation, and one of its purposes is to provide a data collection device, a vehicle control device, a data collection system, a data collection method and a storage medium that can easily collect learning data for a determination model for determining obstacles on the road.

[0006] Solutions to Problems

[0007] The data collection device, vehicle control device, data collection system, data collection method, and storage medium of the present invention employ the following structures.

[0008] (1) A data collection device according to one embodiment of the present invention comprises: an acquisition unit that acquires an image obtained by photographing the periphery of a first vehicle; a first determination unit that determines whether the first vehicle has changed lanes during a determination period based on information indicating a driving history of the first vehicle; and a collection unit that, when the first determination unit determines that the first vehicle has not changed lanes, collects the image included in the determination period as learning data for a determination model for determining obstacles on a road.

[0009] According to the aspect (2), in the data collection device according to the aspect (1), the first determination unit determines whether the first vehicle has changed lanes based on information on the movement of a direction indicator of the first vehicle.

[0010] The scheme (3) is based on the data collection device of the above-mentioned scheme (1) or (2), wherein the first judgment unit determines whether a second vehicle traveling in a lane different from the driving lane of the first vehicle has changed lanes during the judgment period, and the collection unit collects the images contained in the judgment period as the learning data when the first judgment unit determines that the second vehicle has not changed lanes.

[0011] According to the scheme (4), based on the data collection device of the scheme (3) above, the first determination unit determines whether the second vehicle has changed lanes based on the movement information of the direction indicator of the second vehicle recognized from the image obtained by the acquisition unit.

[0012] The method (5) is a data collection device according to any one of the methods (1) to (4) above, further comprising a calculation unit for calculating the determination period based on the mileage information of the first vehicle and the shooting conditions of the camera mounted on the first vehicle.

[0013] The invention according to (6) is the data collection device according to any one of the above-mentioned inventions (1) to (5), further comprising a learning unit configured to learn the learning data collected by the collection unit to generate the determination model.

[0014] The scheme (7) is based on the data collection device of any one of the above schemes (1) to (5) and further includes a communication unit, which sends the learning data collected by the collection unit to an external learning device, and receives the judgment model generated by learning the learning data by the learning device from the learning device.

[0015] (8) A vehicle control device according to another embodiment of the present invention comprises: a data collection device according to the embodiment (6) or (7) above; and a second determination unit that uses the determination model to determine whether the first vehicle can travel in the driving lane.

[0016] (9) A data collection system according to another embodiment of the present invention includes: the data collection device according to any one of the above-mentioned embodiments (1) to (6); and a camera mounted on the first vehicle.

[0017] (10) A data collection system according to another embodiment of the present invention includes: the data collection device according to the embodiment (7) above; a camera mounted on the first vehicle; and the learning device.

[0018] (11) The data collection method of another embodiment of the present invention causes a computer to perform the following processing: obtaining an image obtained by photographing the periphery of a first vehicle; determining whether the first vehicle has changed lanes during a determination period based on information representing the driving history of the first vehicle; and if it is determined that the first vehicle has not changed lanes, collecting the image contained in the determination period as learning data for a determination model for determining obstacles on the road.

[0019] (12) A storage medium of another embodiment of the present invention stores a program that causes a computer to perform the following processing: obtaining an image obtained by photographing the periphery of a first vehicle; determining whether the first vehicle has changed lanes during a determination period based on information representing the driving history of the first vehicle; and if it is determined that the first vehicle has not changed lanes, collecting the image contained in the determination period as learning data for a determination model for determining obstacles on the road.

[0020] Effects of the Invention

[0021] According to the above-mentioned configurations (1) to (12), learning data for determining obstacles on the road can be easily collected.

[0022] According to the solution of (5) above, the accuracy of the learning data can be further improved.

[0023] According to the above-mentioned aspects (6) to (8), a determination model for determining obstacles on the road can be generated, and using this determination model, it is possible to accurately determine whether or not the vehicle can travel on the road. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a diagram showing an example of the configuration of the data collection system 1 according to the first embodiment.

[0025] Figure 2 This is a diagram showing an example of the configuration of the data collection device 100 according to the first embodiment.

[0026] Figure 3 This is a flowchart showing an example of data collection processing by the data collection device 100 according to the first embodiment.

[0027] Figure 4A This is a diagram for explaining an example of calculation processing of the determination period by the determination period calculation unit 114 according to the first embodiment.

[0028] Figure 4B This is a diagram for explaining an example of calculation processing of the determination period by the determination period calculation unit 114 according to the first embodiment.

[0029] Figure 5A This is a diagram for explaining an example of a lane change determination process of the host vehicle M by the lane change determination unit 115 according to the first embodiment.

[0030] Figure 5B This is a diagram for explaining an example of a lane change determination process of the host vehicle M by the lane change determination unit 115 according to the first embodiment.

[0031] Figure 6 This is a diagram for explaining an example of a lane change determination process of another vehicle m1 by the lane change determination unit 115 according to the first embodiment.

[0032] Figure 7 This is a diagram showing an example of the configuration of a data collection system 2 according to the second embodiment.

[0033] Figure 8 This is a diagram showing an example of the configuration of a data collection device 100A according to the second embodiment. DETAILED DESCRIPTION

[0034] Hereinafter, embodiments of a data collection device, a vehicle control device, a data collection system, a data collection method, and a storage medium according to the present invention will be described with reference to the accompanying drawings.

[0035] <First embodiment>

[0036] [Overall structure]

[0037] Figure 1 This diagram shows an example of the configuration of a data collection system 1 according to the first embodiment. The data collection system is installed in a vehicle M. The data collection system 1 includes, for example, a camera 10; wheel speed sensors 20-1 to 20-4, as examples of devices for acquiring mileage information; a speed calculation device 21; a steering angle sensor 30; and a yaw rate sensor 40; turn indicators (directional indicators) 50-1 to 50-4; and a data collection device 100. The vehicle M can be a vehicle with an autonomous driving function or a vehicle driven manually. Furthermore, there are no particular limitations on its drive mechanism; various vehicles, such as engine vehicles, hybrid vehicles, electric vehicles, and fuel cell vehicles, can be included in the vehicle M. Hereinafter, when the wheel speed sensors are not distinguished, they are simply referred to as wheel speed sensors 20. Hereinafter, when the turn indicators are not distinguished, they are simply referred to as turn indicators 50. The vehicle M is an example of a "first vehicle."

[0038] The camera 10 is, for example, a digital camera that utilizes a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is installed at any location of the vehicle M. When shooting the front, the camera 10 is installed at the top of the front windshield, the back of the rearview mirror inside the vehicle, etc. When shooting the rear, the camera 10 is installed at the top of the side windshield, the rear door, etc. When shooting the side, the camera 10 is installed at the rearview mirror on the door, etc. The camera 10, for example, periodically and repeatedly shoots the surroundings of the vehicle M to obtain surrounding images. The following describes the case where the camera 10 shoots the front of the vehicle M as an example.

[0039] Odometer information refers to the result of estimating the position and posture of a moving object based on the output of a device (e.g., a sensor) attached to the moving object to measure its behavior. In the case of a vehicle, the "sensors" mentioned above include some or all of the following: the wheel speed sensor 20 for measuring wheel speed, the speed calculation device 21 for calculating the vehicle speed based on the output of the wheel speed sensor 20, the steering angle sensor 30 for detecting the steering wheel angle (or the angle of the steering mechanism), the yaw rate sensor 40 for detecting the vehicle's rotational speed about the vertical axis, and other similar sensors. Sensors for detecting the rotation angle of a transmission or a travel motor can also be used as speed sensors.

[0040] Wheel speed sensors 20 are mounted on each wheel of the vehicle M. Each wheel speed sensor 20 outputs a pulse signal each time the wheel rotates a predetermined angle. A speed calculation device 21 calculates the speed of each wheel by counting the pulse signals input from each wheel speed sensor 20. Furthermore, the speed calculation device 21 calculates the speed of the vehicle M by averaging the speeds of the driven wheels, for example.

[0041] The data collection device 100 collects learning data for generating a determination model based on images of the surroundings of the vehicle M (e.g., images in front of the vehicle) captured by the camera 10. The determination model determines whether there are obstacles on the road. Alternatively, the determination model determines whether there are obstacles on the road and whether the vehicle can travel on the road. An obstacle is any object or phenomenon that hinders the vehicle's travel. Examples of obstacles include any fallen objects, damaged areas on the road, vehicles stopped due to an accident, people, animals, temporary signs indicating that the vehicle cannot travel, and road construction.

[0042] Figure 2This diagram shows an example of the configuration of a data collection device 100 according to the first embodiment. The data collection device 100 includes, for example, a control unit 110 and a storage unit 130. The control unit 110 includes, for example, a first acquisition unit 111, a second acquisition unit 112, a third acquisition unit 113, a determination period calculation unit 114, a lane change determination unit 115, a learned data collection unit 116, a learning unit 117, and a driving possibility determination unit 118. The first acquisition unit 111 is an example of an "acquisition unit." The determination period calculation unit 114 is an example of a "calculation unit." The lane change determination unit 115 is an example of a "first determination unit." The learned data collection unit 116 is an example of a "collection unit." The learning unit 117 is an example of a "learning unit." The driving possibility determination unit 118 is an example of a "second determination unit." The data collection device 100 including the driving possibility determination unit 118 is an example of a "vehicle control device."

[0043] The components of the control unit 110 are implemented by, for example, a hardware processor (computer) such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components can be implemented by hardware (including circuitry) such as LSI (LargeScale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc., or they can be implemented by the collaboration of software and hardware. The program can be pre-stored in a storage device such as an HDD (Hard Disk Drive) or a flash memory (a storage device having a non-temporary storage medium), or it can be stored in a removable storage medium such as a DVD or CD-ROM (a non-temporary storage medium), and installed by assembling the storage medium in a drive device.

[0044] The first acquisition unit 111 acquires a surrounding image D1 obtained by capturing the surroundings of the host vehicle M from the camera 10 , and stores the image in the storage unit 130 .

[0045] The second acquisition unit 112 acquires the output values ​​of the speed calculation device 21, the steering angle sensor 30, and the yaw angular velocity sensor 40, synthesizes them to generate mileage information D2 of the host vehicle M, and stores it in the storage unit 130. The mileage information D2 can be information represented by six degrees of freedom, or in practice, three degrees of freedom, including translational movement on the X and Y axes and rotational movement centered on the Z axis. Various calculation methods are known for generating the mileage information D2, but as one example, a calculation method known as a unicycle model can be employed. This calculation method uses, for example, the output values ​​of the speed calculation device 21 and the output values ​​of the steering angle sensor 30 as input values. The output mileage information represents, for example, the X- and Y-direction positions and orientations of the host vehicle M at a given moment.

[0046] The third acquisition unit 113 acquires information related to the lighting / flashing of the turn signal lamp 50 from the turn signal lamp 50 and stores the information as turn signal lamp information D3 in the storage unit 130. Alternatively, the third acquisition unit 113 may acquire the turn signal lamp information D3 from a turn signal lamp switch (turn signal lamp lever) (not shown) that receives an instruction to light up / flash the turn signal lamp 50 from a passenger of the host vehicle M. Alternatively, when the host vehicle M is traveling under automatic driving control, the third acquisition unit 113 may acquire the turn signal lamp information D3 from an automatic driving control device (not shown) that controls the lighting / flashing of the turn signal lamp 50.

[0047] The determination period calculation unit 114 calculates the determination period for determining whether or not a lane change has occurred. The determination period calculation unit 114 calculates the determination period based on, for example, the camera 10's imaging conditions, such as resolution and field of view (FOV), and the mileage information D2 (accumulated mileage distance). For example, the determination period calculation unit 114 sets the accumulated mileage distance to be longer for higher resolutions. For example, the determination period calculation unit 114 sets the accumulated mileage distance to be longer for smaller angles of the FOV. The determination period calculation unit 114 calculates the determination period based on the set accumulated mileage distance and the vehicle speed of the host vehicle M.

[0048] Specifically, the determination period calculation unit 114 calculates the determination period based on the mileage information of the host vehicle M (first vehicle) and the imaging conditions of the camera 10 mounted on the host vehicle M (first vehicle). The details of the processing of the determination period calculation unit 114 will be described later.

[0049] The lane change determination unit 115 determines whether the host vehicle M has changed lanes during the determination period calculated by the determination period calculation unit 114 based on information indicating the driving history of the host vehicle M. Alternatively, the lane change determination unit 115 determines whether the host vehicle M and another vehicle have changed lanes during the determination period. The other vehicle is a vehicle traveling in another lane in the same direction as the host vehicle M. The other vehicle is an example of a "second vehicle."

[0050] Regarding the host vehicle M, the lane change determination unit 115 determines whether the host vehicle M has changed lanes based on, for example, the turn signal information D3 obtained from the turn signal 50. Alternatively, the lane change determination unit 115 may determine whether the host vehicle M has changed lanes based on the surrounding image D1 obtained from the camera 10. Alternatively, the lane change determination unit 115 may determine whether the host vehicle M has changed lanes based on both the turn signal information D3 and the surrounding image D1. For example, the lane change determination unit 115 may determine that the host vehicle M has changed lanes if at least one of the determination results based on the turn signal information D3 and the determination results based on the surrounding image D1 indicates that the host vehicle M has changed lanes.

[0051] Regarding other vehicles, the lane change determination unit 115 determines whether the other vehicles have changed lanes based on information related to the lighting / flashing of the other vehicles' turn indicators included in the surrounding image D1. It should be noted that the lane change determination unit 115 may also determine whether the host vehicle M or other vehicles have stopped.

[0052] Specifically, the lane change determination unit 115 determines whether the host vehicle M (first vehicle) has changed lanes during the determination period based on information indicating the driving history of the host vehicle M (first vehicle). The lane change determination unit 115 determines whether the host vehicle M (first vehicle) has changed lanes based on information regarding the movement of the turn signal indicator of the host vehicle M (first vehicle). Furthermore, the lane change determination unit 115 determines whether another vehicle (second vehicle) traveling in a lane different from the driving lane of the host vehicle M (first vehicle) has changed lanes during the determination period. The lane change determination unit 115 determines whether the other vehicle (second vehicle) has changed lanes based on information regarding the movement of the turn signal indicator of the other vehicle (second vehicle) identified from the surrounding image D1.

[0053] If the lane change determination unit 115 determines that the host vehicle M has not changed lanes, the learning data collection unit 116 uses the surrounding images during the determination period as learning data because the likelihood of an obstacle being included is low. The learning data collection unit 116 collects the surrounding images as learning data D4 and stores the learning data in the storage unit 130. On the other hand, if the lane change determination unit 115 determines that the host vehicle M has changed lanes, the learning data collection unit 116 does not use the surrounding images during the determination period as learning data because the likelihood of an obstacle being included is high.

[0054] Furthermore, if the lane change determination unit 115 determines that the other vehicle has not changed lanes, the learning data collection unit 116 uses the surrounding images during the determination period as learning data because the likelihood of an obstacle being included is low. The surrounding images are collected as learning data D4 and stored in the storage unit 130. On the other hand, if the lane change determination unit 115 determines that the other vehicle has changed lanes, the learning data collection unit 116 does not use the surrounding images during the determination period as learning data because the likelihood of an obstacle being included is high.

[0055] Alternatively, if the lane change determination unit 115 determines that neither the host vehicle M nor the other vehicle has changed lanes, the learning data collection unit 116 uses the surrounding images during the determination period as learning data because the likelihood of an obstacle being included is low, and collects the surrounding images as learning data D4, which is stored in the storage unit 130. On the other hand, if the lane change determination unit 115 determines that at least one of the host vehicle M and the other vehicle has changed lanes, the learning data collection unit 116 does not use the surrounding images during the determination period as learning data because the likelihood of an obstacle being included is high.

[0056] Specifically, when the lane change determination unit 115 determines that the host vehicle M (first vehicle) has not changed lanes, the learning data collection unit 116 collects the surrounding images included during the determination period as learning data for the determination model for determining obstacles on the road. Furthermore, when the lane change determination unit 115 determines that the other vehicle (second vehicle) has not changed lanes, the learning data collection unit 116 collects the surrounding images included during the determination period as learning data.

[0057] The learning unit 117 uses machine learning methods such as deep learning to learn the learning data D4 collected by the learning data collection unit 116, thereby generating a determination model MD. The learning unit 117 stores the generated determination model MD in the storage unit 130. The determination model MD is a model that determines the presence or absence of obstacles on the road. For example, when a surrounding image is input, the determination model MD outputs information indicating the presence or absence of obstacles on the road contained in the surrounding image. For example, a convolutional neural network (CNN) can be used as the determination model MD.

[0058] That is, the learning unit 117 learns the learning data collected by the learning data collecting unit 116 and generates the determination model MD.

[0059] The driving permission determination unit 118 determines whether the road on which the host vehicle M is traveling is drivable based on the surrounding images captured by the camera 10 during driving and the determination model MD. If the output of the surrounding images input to the determination model MD indicates the absence of obstacles, the driving permission determination unit 118 determines that the road is drivable. On the other hand, if the output of the surrounding images input to the determination model MD indicates the presence of obstacles, the driving permission determination unit 118 determines that the road is not drivable. For example, if the host vehicle M is driving under automatic driving control, the automatic driving control device can consider the determination result of the driving permission determination unit 118 when setting the lane and trajectory of the host vehicle M.

[0060] That is, the travel permission determination unit 118 determines whether the host vehicle M (first vehicle) can travel in the travel lane using the determination model MD.

[0061] The storage unit 130 stores, for example, a surrounding image D1, mileage information D2, blinker information D3, learning data D4, a determination model MD, etc. The storage unit 130 is a storage device such as an HDD, a RAM (Random Access Memory), or a flash memory.

[0062] The data collection process will be described below using a flowchart. Figure 3 This is a flowchart showing an example of data collection processing by the data collection device 100 according to the first embodiment. Figure 3 The processing in the flowchart shown is repeatedly executed while the vehicle M is traveling on a road targeted for data collection. Note that the processing steps in the flowchart described below may be reversed in order, and a plurality of processing steps may be executed in parallel.

[0063] First, the first acquisition unit 111 acquires the surrounding image D1 captured by the camera 10 and stores it in the storage unit 130 (step S101). The second acquisition unit 112 acquires the output values ​​of the speed calculation device 21, the steering angle sensor 30, and the yaw rate sensor 40, synthesizes these values ​​to generate mileage information D2 of the host vehicle M, and stores it in the storage unit 130 (step S103). Furthermore, the third acquisition unit 113 acquires the direction indicator information D3 from the direction indicator 50 and stores it in the storage unit 130 (step S105). The processes of steps S101, S103, and S105 are continuously executed in parallel while the host vehicle M is traveling on the road targeted for data collection.

[0064] Next, the determination period calculation unit 114 calculates a determination period for determining whether or not a lane change has occurred (step S107 ). The determination period calculation unit 114 calculates the determination period based on the imaging conditions of the camera 10 and the mileage information D2 . Figure 4A and 4B This is a diagram for explaining an example of calculation processing of the determination period by the determination period calculation unit 114 according to the first embodiment. Figure 4A This shows an example in which the vehicle M is traveling in the lane L1 at a speed V while capturing surrounding images with the camera 10 in a low-resolution or FOV (wide-angle) capture mode. Figure 4B An example is shown in which the vehicle M travels in the lane L1 at a speed V while capturing surrounding images with the camera 10 in a high-resolution or FOV (narrow angle) capturing mode.

[0065] like Figure 4A As shown, when the camera 10 is shooting in a low-resolution or FOV (wide-angle) shooting mode, the range relatively close to the vehicle M becomes the shooting object, and the range far away from the vehicle M does not become the shooting object. Therefore, the range close to the vehicle M is reflected in the peripheral image PA captured by the camera 10, and the distant range is not included in the shooting object, or even if it is included in the shooting object image, it is not clear and its detailed content cannot be grasped. As a result, the possibility of distant obstacles being reflected in the peripheral image captured under such shooting conditions becomes low. In this case, the cumulative distance of mileage (reference distance L1) is set to be relatively short. The judgment period calculation unit 114 calculates the judgment period Δt by, for example, dividing the reference distance L1 pre-set according to the shooting conditions by the current speed V calculated based on the mileage information D2.

[0066] On the other hand, Figure 4BAs shown, when the camera 10 is shooting in a high-resolution or FOV (angle) shooting mode, in addition to the range relatively close to the vehicle M, the range far away from the vehicle M is also included in the shooting object. Therefore, the range far away from the vehicle M is also reflected in the peripheral image PB shot by the camera 10. As a result, the possibility of distant obstacles being reflected in the peripheral image PB shot under such shooting conditions becomes high. In this case, the cumulative distance of mileage (reference distance L2) is set to be relatively longer than the reference distance L1. The judgment period calculation unit 114 calculates the judgment period Δt", for example, by dividing the reference distance L2 pre-set according to the shooting conditions by the current speed V calculated based on the mileage information D2.

[0067] Next, the lane change determination unit 115 determines whether the vehicle M and / or other vehicles have changed lanes during the determination period calculated by the determination period calculation unit 114 (step S109). If the lane change determination unit 115 determines that the vehicle M has not changed lanes during the determination period, the learning data collection unit 116 uses the surrounding images during the determination period as learning data and stores them as learning data D4 in the storage unit 130 (step S111). On the other hand, if the lane change determination unit 115 determines that the vehicle M has changed lanes during the determination period, the learning data collection unit 116 excludes the surrounding images during the determination period from being used as learning data due to the high probability that they contain obstacles (step S113).

[0068] Alternatively, if the lane change determination unit 115 determines that neither the host vehicle M nor the other vehicle has changed lanes during the determination period, the learning data collection unit 116 uses the surrounding images during the determination period as learning data and stores them as learning data D4 in the storage unit 130 (step S111). On the other hand, if the lane change determination unit 115 determines that at least one of the host vehicle M and the other vehicle has changed lanes during the determination period, the learning data collection unit 116 excludes the surrounding images during the determination period from being used as learning data because there is a high probability that the surrounding images during the determination period contain an obstacle (step S113).

[0069] Figure 5A and Figure 5B This figure illustrates an example of lane change determination processing of the host vehicle M by the lane change determination unit 115 of the first embodiment. At a certain time T during driving, the lane change determination unit 115 refers to the past direction indicator information D3 stored in the storage unit 130 for the determination period Δt (the period from time T-Δt to time T) calculated by the determination period calculation unit 114, and determines whether the host vehicle M has changed lanes. Figure 5AIn the example shown, the vehicle M travels straight in lane L1 during the determination period Δt without changing lanes. In this case, the learning data collection unit 116 uses the surrounding images captured during the determination period Δt as learning data and stores them in the storage unit 130 as learning data D4.

[0070] On the other hand, Figure 5B In the example shown, the vehicle M changes lanes from lane L2 to lane L1 to avoid obstacle OB located in lane L2 during the determination period Δt. In this case, the learning data collection unit 116 does not use the surrounding images captured during the determination period Δt (for example, the surrounding image P including obstacle OB captured at time T-Δt). T-Δt etc.) are excluded as learning data.

[0071] Figure 6 This is a diagram for explaining an example of a lane change determination process of another vehicle m1 by the lane change determination unit 115 according to the first embodiment. Figure 6 This example shows a situation where the host vehicle M is traveling straight in lane L3, and another vehicle m1 changes lanes from lane L2 to lane L1 to avoid an obstacle OB located in lane L2. At a certain time T during driving, the lane change determination unit 115 determines whether the other vehicle m1 has changed lanes based on the past surrounding image D1 stored in the storage unit 130, for the determination period Δt calculated by the determination period calculation unit 114. For example, the lane change determination unit 115 performs image analysis on the surrounding image D1 and extracts information related to the lighting / flashing of the turn signal indicator WS of the other vehicle m1 reflected in the surrounding image D1. If the extracted information related to the lighting / flashing of the turn signal WS does not indicate that the other vehicle m1 has changed lanes, the lane change determination unit 115 determines that the other vehicle m1 has not changed lanes. In this case, the learning data collection unit 116 uses the surrounding images captured during the determination period Δt as learning data and stores this information as learning data D4 in the storage unit 130. On the other hand, if the extracted information on the lighting / flashing of the direction indicator WS indicates that the other vehicle m1 has changed lanes, the lane change determination unit 115 determines that the other vehicle m1 has changed lanes. In this case, the learning data collection unit 116 does not use the surrounding images captured within the determination period Δt (for example, the surrounding image P including the obstacle OB captured at time T-Δt). T-Δt The image group (such as the image group) is excluded as learning data. Thus, the processing of this flowchart is completed.

[0072] According to the data collection system 1 and the data collection device 100 of the first embodiment described above, it is possible to easily collect learning data for a determination model for determining obstacles on the road. In addition, the learning data collected as described above includes images of objects that do not become obstacles to driving (objects that do not cause lane changes), such as corrugated paper and manholes. Therefore, by using the learning data collected in this way, the determination accuracy of the determination model can be further improved. In addition, by calculating the determination period based on the shooting conditions and mileage information of the camera 10, the accuracy of the learning data can be further improved. In addition, using the learning data collected as described above, a determination model for determining obstacles on the road can be generated, and using this determination model, it is possible to determine whether it is possible to drive on the road with high accuracy.

[0073] <Second embodiment>

[0074] Hereinafter, a second embodiment will be described. Figure 7 This diagram shows an example of the configuration of a data collection system 2 according to the second embodiment. Unlike the first embodiment, the data collection device 100A of the data collection system 2 does not have a learning function (learning unit 117 of the data collection device 100). Instead, a learning device 200 configured as a cloud server performs learning processing to generate a decision model. Therefore, the following description focuses on the differences from the first embodiment, and descriptions of common features with the first embodiment are omitted. In the description of the second embodiment, components identical to those in the first embodiment are denoted by the same reference numerals.

[0075] Figure 8 : This is a diagram showing an example of the structure of a data collection device 100A according to the second embodiment. In addition to a first acquisition unit 111, a second acquisition unit 112, a third acquisition unit 113, a determination period calculation unit 114, a lane change determination unit 115, a learning data collection unit 116, and a driving possibility determination unit 118, the control unit 110 further includes a communication unit 119. The communication unit 119 transmits the learning data D4 to an external learning device 200 via a network NW. The network NW includes, for example, a WAN (Wide Area Network), a LAN (Local Area Network), a cellular network, a wireless base station, the Internet, and the like. The learning device 200 includes a communication interface (not shown) for connecting to the network NW. The communication unit 119 is an example of a "communication unit."

[0076] The learning device 200 acquires the learning data D4 transmitted from the communication unit 119 via the network NW, learns the acquired learning data D4, and generates a determination model MD. The learning device 200 transmits the generated determination model MD to the data collection device 100A via the network NW. The data collection device 100A acquires the determination model MD generated by the learning device 200 via the network NW as described above and stores it in the storage unit 130.

[0077] That is, the communication unit 119 transmits the learning data D4 collected by the learning data collection unit 116 to the external learning device 200 , and receives from the learning device 200 the determination model MD generated by the learning device 200 learning the learning data D4 .

[0078] The configuration of other functional units of the data collection device 100A of the host vehicle M is the same as that of the first embodiment, and therefore detailed description thereof will be omitted.

[0079] According to the data collection system 2 and the data collection device 100A of the second embodiment described above, similar to the data collection system 1 of the first embodiment, learning data for a determination model for determining obstacles on a road can be easily collected.

[0080] The above-described embodiment can also be expressed as follows.

[0081] A data collection device, wherein:

[0082] The data collection device comprises:

[0083] a storage device storing a program; and

[0084] Hardware processor,

[0085] The hardware processor executes the program to perform the following processing:

[0086] obtaining an image obtained by photographing the periphery of the first vehicle;

[0087] determining whether the first vehicle has changed lanes within a determination period based on information indicating the driving history of the first vehicle; and

[0088] When it is determined that the first vehicle has not changed lanes, the image included in the determination period is collected as learning data for a determination model for determining obstacles on the road.

[0089] While specific embodiments of the present invention have been described above, the present invention is not limited to these embodiments at all, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A data collection device, wherein: The data collection device comprises: an acquisition unit that acquires an image obtained by photographing the periphery of the first vehicle; a calculation unit that calculates a determination period based on mileage information of the first vehicle and an imaging condition of a camera mounted on the first vehicle; a first determination unit that determines whether the first vehicle has changed lanes within the determination period based on information indicating a driving history of the first vehicle; as well as A collecting unit collects the image included in the determination period as learning data for a determination model for determining an obstacle on the road when the first determination unit determines that the first vehicle has not changed lanes.

2. The data collection device according to claim 1, wherein: The first determination unit determines whether the first vehicle has changed lanes based on information regarding the movement of a direction indicator of the first vehicle.

3. The data collection device according to claim 1 or 2, wherein: The first determination unit determines whether a second vehicle traveling in a lane different from the driving lane of the first vehicle has changed lanes during the determination period. The collecting unit collects the image included in a period of the determination as the learning data when the first determining unit determines that the second vehicle has not changed lanes.

4. The data collection device according to claim 3, wherein: The first determination unit determines whether the second vehicle has changed lanes based on movement information of a direction indicator of the second vehicle recognized from the image acquired by the acquisition unit.

5. The data collection device according to claim 1 or 2, wherein: The data collection device further includes a learning unit configured to learn the learning data collected by the collection unit to generate the determination model.

6. The data collection device according to claim 1 or 2, wherein: The data collection device further includes a communication unit that transmits the learning data collected by the collection unit to an external learning device and receives from the learning device the determination model generated by learning the learning data by the learning device.

7. A vehicle control device, wherein: The vehicle control device comprises: The data collection device according to claim 5 or 6; and The second determination unit determines whether the first vehicle can travel in the driving lane using the determination model.

8. A data collection system, wherein: The data collection system comprises: The data collection device according to any one of claims 1 to 5; and A camera mounted on the first vehicle.

9. A data collection system, wherein: The data collection system comprises: The data collection device according to claim 6; a camera mounted on the first vehicle; and The learning device.

10. A data collection method, wherein: The data collection method enables the computer to perform the following processing: obtaining an image obtained by photographing the periphery of the first vehicle; calculating a determination period based on the mileage information of the first vehicle and a photographing condition of a camera mounted on the first vehicle; determining whether the first vehicle has changed lanes within the determination period based on information indicating the driving history of the first vehicle; as well as When it is determined that the first vehicle has not changed lanes, the image included in the determination period is collected as learning data for a determination model for determining obstacles on the road.

11. A storage medium storing a program, wherein: The program causes the computer to perform the following processing: obtaining an image obtained by photographing the periphery of the first vehicle; calculating a determination period based on the mileage information of the first vehicle and a photographing condition of a camera mounted on the first vehicle; determining whether the first vehicle has changed lanes within the determination period based on information indicating the driving history of the first vehicle; as well as When it is determined that the first vehicle has not changed lanes, the image included in the determination period is collected as learning data for a determination model for determining obstacles on the road.

Images