Preceding vehicle identification device, preceding vehicle identification method, and program

Abstract

To provide a new technique for detecting a preceding vehicle by using a sensor provided in a vehicle.SOLUTION: A preceding vehicle identification device according to the present disclosure acquires forward information including information on an area of a road included in a captured image obtained by capturing an image in front of a target vehicle and information on one or more vehicles on the road, identifies a reference point of the target vehicle in the captured image using the forward information, identifies a relative distance and a relative direction with respect to the reference point for each vehicle indicated in the forward information, and identifies a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each vehicle. The preceding vehicle is a vehicle closest to the target vehicle among vehicles traveling on the same lane as the target vehicle.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a leading vehicle identification device, a leading vehicle identification method, and a program. [Background technology]

[0002] Technologies have been developed that use sensors installed in vehicles traveling on roads to analyze the situation around the vehicle. For example, Patent Document 1 discloses a technology that detects the lane in which the vehicle is traveling based on the yaw rate detected by a yaw rate sensor, and then detects a preceding vehicle on the detected lane. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-335223 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 does not mention a method for detecting a preceding vehicle without using a yaw rate sensor. The present disclosure has been made in consideration of this problem, and one of its purposes is to provide a new technology for detecting a preceding vehicle using a sensor provided in a vehicle. [Means for solving the problem]

[0005] A preceding vehicle identification device according to the present disclosure includes an acquisition means for acquiring forward information including information on a road area included in a captured image obtained by capturing an image of the area ahead of a target vehicle and information on one or more vehicles on the road, a reference point identification means for identifying a reference point of the target vehicle in the captured image using the forward information, a relative position identification means for identifying a relative distance and a relative direction of each vehicle shown in the forward information with respect to the reference point, and a preceding vehicle identification means for identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each vehicle. The preceding vehicle is the vehicle closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle.

[0006] A leading vehicle identification method executed by a computer according to the present disclosure includes: an acquisition step of acquiring forward information including information on a road area included in a captured image obtained by capturing an image of a road ahead of a target vehicle and information on one or more vehicles on the road; a reference point identification step of identifying a reference point of the target vehicle in the captured image using the forward information; a relative position identification step of identifying a relative distance and a relative direction of each vehicle shown in the forward information relative to the reference point; and a leading vehicle identification step of identifying a leading vehicle of the target vehicle based on the relative distance and the relative direction identified for each vehicle. The leading vehicle is the vehicle closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle.

[0007] The program according to the present disclosure causes a computer to execute the following steps: acquiring forward information including information about a road area included in a captured image obtained by capturing an image of a road ahead of a target vehicle and information about one or more vehicles on the road; identifying a reference point of the target vehicle in the captured image using the forward information; identifying a relative position of each vehicle shown in the forward information relative to the reference point; and identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each vehicle. The preceding vehicle is the vehicle closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle. [Effects of the Invention]

[0008] According to the present disclosure, a new technique for detecting a preceding vehicle using a sensor provided in a vehicle is provided. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram illustrating an example of an outline of the operation of the leading vehicle identification device; [Figure 2] FIG. 2 is a block diagram illustrating a functional configuration of a leading vehicle identification device. [Figure 3] FIG. 2 is a block diagram illustrating a hardware configuration of a computer that realizes the preceding vehicle identification device. [Figure 4] 10 is a flowchart illustrating a flow of processing executed by a leading vehicle identification device. [Figure 5] FIG. 10 is a diagram illustrating an example of the configuration of forward information. [Figure 6] FIG. 10 is a diagram illustrating an example of a reference point. [Figure 7] FIG. 2 is a second diagram illustrating an example of an outline of the operation of the leading vehicle identification device. [Figure 8] FIG. 2 is a second block diagram illustrating an example of the functional configuration of the leading vehicle identification device. [Figure 9]10 is a second flowchart illustrating the flow of processing executed by the leading vehicle identification device. [Figure 10] 10 is a flowchart illustrating a process for detecting a cut-in and a cut-out. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and duplicate explanations will be omitted as necessary for clarity. Furthermore, unless otherwise specified, predetermined values ​​such as predetermined values ​​and threshold values ​​are stored in advance in a storage device accessible from a device that uses the values. Furthermore, unless otherwise specified, the storage unit is composed of one or any number of storage devices.

[0011] [Embodiment 1] <Summary> Fig. 1 is a diagram illustrating an example of an outline of the operation of the leading vehicle identification device 2000. Here, Fig. 1 is a diagram for facilitating understanding of the outline of the leading vehicle identification device 2000, and the operation of the leading vehicle identification device 2000 is not limited to that shown in Fig. 1.

[0012] The preceding vehicle identification device 2000 uses information obtained from the captured image 30 to identify a vehicle preceding the target vehicle 10. The preceding vehicle of the target vehicle 10 is the vehicle closest to the target vehicle 10 among the vehicles traveling in the same lane as the target vehicle 10 traveling in (hereinafter referred to as the target lane).

[0013] The captured image 30 is an image generated by the camera 20 capturing an image of the area ahead of the target vehicle 10. The area ahead of the target vehicle 10 can also be described as the direction of travel of the target vehicle 10. For example, the camera 20 is configured to repeatedly capture images of the area ahead of the target vehicle 10, thereby generating a sequence of image frames that are time-series data of the captured images 30. In this case, the captured image 30 is one of a plurality of image frames that make up the sequence of image frames.

[0014] The camera 20 is provided so as to capture an image of the area ahead of the target vehicle 10. For example, the camera 20 is provided on the windshield of the target vehicle 10. Note that the camera 20 only needs to be configured to capture an image of the scenery ahead of the target vehicle 10, and the image capturing direction of the camera 20 does not need to completely coincide with the traveling direction of the target vehicle 10.

[0015] For example, the camera 20 is a camera used as a drive recorder. Alternatively, for example, the camera 20 may be a camera used to control the traveling of the target vehicle 10. The control of the traveling of the target vehicle 10 is, for example, automatic control of the steering, accelerator, brake, or the like.

[0016] In order to identify a vehicle ahead of the target vehicle 10, the leading vehicle identification device 2000 acquires forward information 50 generated using the captured image 30. The forward information 50 indicates road information 60, which is information about an image area of ​​a road included in the captured image 30 (hereinafter referred to as a road area), and vehicle information 70, which is information about an image area of ​​a vehicle included in the captured image 30 (hereinafter referred to as a vehicle area). Note that the forward information 50 may be generated by the leading vehicle identification device 2000 itself.

[0017] The leading vehicle identification device 2000 uses the forward information 50 to identify a reference point of the target vehicle 10 in the captured image 30. Furthermore, the leading vehicle identification device 2000 uses the forward information 50 to identify the relative distance and relative direction of each vehicle with respect to the reference point. Hereinafter, the combination of the relative position and the relative direction will also be referred to as the relative position.

[0018] Based on the relative position determined for each vehicle, the leading vehicle identification device 2000 identifies a vehicle preceding the target vehicle 10. Specifically, the leading vehicle identification device 2000 identifies one or more vehicles traveling in the target lane, and identifies the vehicle with the smallest relative distance from the identified vehicles as the leading vehicle.

[0019] <Examples of effects> According to the leading vehicle identification device 2000, a vehicle ahead of the target vehicle 10 is identified using forward information 50 obtained from a captured image 30 in which an image of the area ahead of the target vehicle 10 is captured. Therefore, the leading vehicle identification device 2000 provides a technique that can identify a vehicle ahead of the target vehicle 10 without using a yaw rate sensor.

[0020] More specifically, the leading vehicle identification device 2000 identifies a reference point of the target vehicle 10 in the captured image 30 using the forward information 50, and further identifies the relative position (relative distance and relative direction) of each vehicle with respect to the reference point. Then, the leading vehicle identification device 2000 identifies the leading vehicle based on the relative position of each vehicle.

[0021] According to this method, it is possible to identify a vehicle traveling in the same lane as the target vehicle 10 without performing lane detection using white line detection from the captured image 30. Therefore, the leading vehicle identification device 2000 can identify the leading vehicle even in a situation where it is difficult to detect the lane from the captured image 30. Examples of situations where it is difficult to detect the lane from the captured image 30 include a situation where the lane markings on the road are faint or the road is wet due to rain.

[0022] The leading vehicle identification device 2000 of this embodiment will be described in more detail below.

[0023] <Example of functional configuration> FIG. 2 is a block diagram illustrating an example of the functional configuration of the leading vehicle identification device 2000. In the example of FIG. 2, the leading vehicle identification device 2000 includes an acquisition unit 2020, a reference point identification unit 2040, a relative position identification unit 2060, and a leading vehicle identification unit 2080. The acquisition unit 2020 acquires forward information 50. The reference point identification unit 2040 identifies a reference point of the target vehicle 10 in the captured image 30 using the forward information 50. The relative position identification unit 2060 identifies the relative position of each vehicle (a combination of relative distance and relative direction) using the forward information 50. The leading vehicle identification unit 2080 identifies the vehicle preceding the target vehicle 10 based on the relative position of each vehicle.

[0024] <Example of hardware configuration> Each functional component of the preceding vehicle identification device 2000 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the preceding vehicle identification device 2000 is realized by a combination of hardware and software will be further described.

[0025] FIG. 3 is a block diagram illustrating an example of the hardware configuration of a computer 1000 that realizes the leading vehicle identification device 2000. The computer 1000 is any computer. For example, the computer 1000 is a stationary computer such as a PC (Personal Computer) or a server machine. Alternatively, the computer 1000 may be a portable computer such as a smartphone or a tablet terminal. The computer 1000 may be provided inside the camera 20. The computer 1000 may be a dedicated computer designed to realize the leading vehicle identification device 2000, or may be a general-purpose computer.

[0026] For example, by installing a predetermined application in the computer 1000, each function of the leading vehicle identification device 2000 is realized by the computer 1000. The application is configured by a program for realizing each functional component of the leading vehicle identification device 2000.

[0027] The program may be acquired by any method. For example, the program may be acquired from a storage medium (such as a DVD (Digital Versatile Disc) or a USB (Universal Serial Bus) memory) on which the program is stored. Alternatively, the program may be acquired by downloading the program from a server device that manages the storage device on which the program is stored.

[0028] The computer 1000 has a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. However, the method for connecting the processor 1040 and the like to each other is not limited to a bus connection.

[0029] The processor 1040 is a variety of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device realized using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, a read only memory (ROM), or the like.

[0030] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, the input / output interface 1100 is connected to an input device such as a keyboard and an output device such as a display device.

[0031] The network interface 1120 is an interface for connecting the computer 1000 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0032] The storage device 1080 stores a program (a program that realizes the above-mentioned application) that realizes each functional component of the leading vehicle identification device 2000. The processor 1040 reads this program into the memory 1060 and executes it, thereby realizing each functional component of the leading vehicle identification device 2000.

[0033] The leading vehicle identification device 2000 may be realized by one computer 1000 or by multiple computers 1000. In the latter case, the configurations of the computers 1000 do not need to be the same, and may be different from each other.

[0034] <Processing flow> 4 is a flowchart illustrating the flow of processing executed by the preceding vehicle identification device 2000. The acquisition unit 2020 acquires forward information 50 (S102). The reference point identification unit 2040 identifies a reference point using the forward information 50 (S104). The relative position identification unit 2060 identifies the relative position of each vehicle using the forward information 50 (S106). The preceding vehicle identification unit 2080 identifies a vehicle preceding the target vehicle 10 based on the relative position of each vehicle (S108).

[0035] As described above, image frames constituting the image frame sequence can be used as the captured images 30. When image frames are used as the captured images 30, for example, the leading vehicle identification device 2000 acquires forward information 50 generated for each image frame (captured image 30) constituting the image frame sequence, thereby identifying a leading vehicle for each image frame constituting the image frame sequence.

[0036] The image frame sequence may be made up of all image frames generated by the camera 20, or may be made up of a portion of the image frames generated by the camera 20. In the latter case, for example, image frames are extracted from the image frames generated by the camera 20 at a rate of one in a predetermined number (for example, one in 30 frames), and the image frame sequence is made up of the extracted image frames.

[0037] <Acquisition of forward information 50: S102> The acquisition unit 2020 acquires forward information 50 (S102). The forward information 50 is generated by a forward information generation device. The forward information generation device may be realized as a separate entity from the leading vehicle identification device 2000, or may be realized as an entity integrated with the leading vehicle identification device 2000. The latter means that the leading vehicle identification device 2000 also functions as a forward information generation device.

[0038] When the leading vehicle identification device 2000 also functions as a forward information generation device, the acquisition unit 2020 acquires forward information 50 generated by the leading vehicle identification device 2000 itself. For example, the leading vehicle identification device 2000 is configured to generate the forward information 50 and store the generated forward information 50 in an arbitrary storage unit (for example, the storage device 1080). The acquisition unit 2020 acquires the forward information 50 from the storage unit.

[0039] It is assumed that the forward information generation device is realized as a separate entity from the leading vehicle identification device 2000. In this case, the forward information generation device is configured, for example, to transmit the forward information 50 to the leading vehicle identification device 2000. In this case, the acquisition unit 2020 acquires the forward information 50 by receiving the forward information 50 transmitted from the forward information generation device. Alternatively, for example, the forward information generation device is configured to store the forward information 50 in an arbitrary storage unit. In this case, the acquisition unit 2020 acquires the forward information 50 by reading out the forward information 50 from the storage unit in which the forward information 50 is stored.

[0040] <<How to generate forward information 50>> The following describes a method for generating the forward information 50. As described above, the forward information 50 indicates the road information 60 and the vehicle information .

[0041] For example, the forward direction information generating device divides the captured image 30 into image regions that represent multiple objects by performing segmentation processing such as panoptic segmentation on the captured image 30. The forward direction information generating device extracts road regions from the image regions obtained by the division and generates road information 60.

[0042] For example, the forward information generating device identifies a circumscribing rectangle of a road area and generates road information 60 that indicates the position and dimensions of the circumscribing rectangle. The position of the circumscribing rectangle is represented, for example, by the coordinates of a specific position (upper left corner or center) of the circumscribing rectangle. The dimensions of the circumscribing rectangle are represented, for example, by a combination of the width and height of the circumscribing rectangle.

[0043] Here, not only an image area representing a road in the traveling direction of the target vehicle 10 but also an image area representing a road in a direction different from the traveling direction of the target vehicle 10 (for example, a direction opposite to the traveling direction) can be detected from the captured image 30. In other words, a plurality of road areas can be detected from the captured image 30.

[0044] When multiple road areas are detected from the captured image 30, the forward information generation device identifies a road area including the target lane from the multiple road areas obtained from the captured image 30 (i.e., a road area representing a road in the traveling direction of the target vehicle 10), and generates road information 60 for the identified road area. Hereinafter, the road area including the target lane will be referred to as the target road area.

[0045] For example, the forward facing information generating device identifies the largest road area among multiple road areas obtained from the captured image 30 as the target road area. Alternatively, for example, the forward facing information generating device calculates the horizontal center coordinate of the captured image 30, and identifies the road area that includes this center coordinate in the horizontal direction as the target road area. Here, if the upper left corner of the captured image 30 is set as the origin and the width of the captured image 30 is set as W, the horizontal center coordinate of the captured image 30 is x=W.

[0046] Furthermore, the forward facing information generating device generates vehicle information 70 for a vehicle region, which is an image region representing a vehicle, among the multiple image regions obtained by segmentation. For example, the forward facing information generating device identifies a circumscribing rectangle of the vehicle region and generates vehicle information 70 indicating the position of the circumscribing rectangle. The vehicle information 70 includes information about each of one or more vehicles detected from the captured image 30. The vehicle information 70 may further indicate the dimensions of the circumscribing rectangle in addition to the position of the circumscribing rectangle.

[0047] Note that the method of detecting road areas and vehicle areas from the captured image 30 is not limited to the method using segmentation. For example, a road area can be detected from the captured image 30 using any machine learning model (such as a neural network) that has been trained in advance to detect road areas from an image. Similarly, a vehicle area can be detected from the captured image 30 using any machine learning model (such as a neural network) that has been trained in advance to detect vehicle areas from an image.

[0048] Fig. 5 is a diagram illustrating an example of the configuration of forward information 50. In Fig. 5, the forward information 50 indicates road information 60 and vehicle information 70. The road information 60 indicates a position 62 and dimensions 64. The position 62 indicates the position of the circumscribing rectangle of the target road area. The dimensions 64 indicate the width and height of the circumscribing rectangle of the target road area.

[0049] The vehicle information 70 indicates an identifier 72, a position 74, and dimensions 76 for each vehicle. The identifier 72 is an identifier uniquely assigned to each vehicle detected from the captured image 30 by the forward information generating device. The position 74 indicates the position of the vehicle's circumscribing rectangle. The dimensions 76 indicate the width and height of the vehicle's circumscribing rectangle.

[0050] Here, when each image frame constituting the image frame sequence is treated as a captured image 30, it is preferable to assign the same identifier to identical vehicles detected from the multiple captured images 30. The process of assigning the same identifier to identical objects detected from multiple different images can be realized by, for example, an identification process using tracking or the like.

[0051] <<Acquisition of captured image 30>> When the leading vehicle identification device 2000 also functions as a forward information generation device, the acquisition unit 2020 acquires the captured image 30. There are various methods for acquiring the image generated by the camera. For example, the camera 20 is configured to transmit the captured image 30 to the leading vehicle identification device 2000. In this case, the acquisition unit 2020 acquires the captured image 30 by receiving the captured image 30 transmitted by the camera 20. Alternatively, for example, the camera 20 is configured to store the captured image 30 in an arbitrary storage unit. The storage unit may be provided inside the camera 20 or outside the camera 20. The acquisition unit 2020 acquires the captured image 30 from the storage unit by, for example, accessing the storage unit.

[0052] The acquisition unit 2020 may acquire an image frame sequence and extract one or more captured images 30 from the acquired image frame sequence, thereby acquiring the captured images 30. The method of acquiring the image frame sequence can be the same as the method of acquiring the captured images 30 described above.

[0053] <Identifying the reference point: S104> The reference point identification unit 2040 uses the forward information 50 to identify the reference point of the target vehicle 10 in the captured image 30 (S104). The reference point is identified based on the road information 60, for example, as follows.

[0054] 6 is a diagram illustrating an example of a reference point. In Fig. 6, the upper left corner of the captured image 30 is the origin (0, 0). The width and height of the entire captured image 30 are W and H, respectively.

[0055] First, the reference point identification unit 2040 identifies the center coordinates (hereinafter referred to as horizontal center coordinates) of the circumscribing rectangle 80 of the target road area in the horizontal direction using the road information 60. The horizontal direction can also be expressed as the x direction.

[0056] For example, assume that the road information 60 indicates the position (x, y) = (x1, y1) of the upper left corner of the circumscribing rectangle 80 and the dimensions (w, h) = (w1, h1) of the circumscribing rectangle 80. Here, w represents the width and h represents the height. In this case, the reference point identification unit 2040 calculates the horizontal center coordinate (x = x1 + w / 2) by adding w / 2, which is half the width w of the circumscribing rectangle 80, to x1, which is the x coordinate of the upper left corner of the circumscribing rectangle 80. However, the horizontal center coordinate may be directly indicated by the road information 60 as a value representing the position of the circumscribing rectangle 80.

[0057] Next, the reference point identification unit 2040 identifies the point on the bottom side of the captured image 30 whose horizontal coordinate coincides with the horizontal center coordinate as the reference point 100. That is, the reference point 100 is (x1+w / 2, H).

[0058] The reference point identification unit 2040 may calculate the reference point 100 by further considering the size of the width of the circumscribing rectangle 80, which is the circumscribing rectangle of the target road area, relative to the width of the entire captured image 30. For example, the reference point identification unit 2040 calculates the width of the range (hereinafter referred to as the empty range) obtained by excluding the horizontal range of the circumscribing rectangle 80 from the horizontal range of the entire captured image 30. Since the width of the entire captured image 30 is W and the width of the circumscribing rectangle 80 is w1, the width of the empty range is W-w1. Then, the reference point identification unit 2040 calculates the x coordinate of the reference point 100 based on the width W-w1 of the empty range and the horizontal center coordinate x1+w / 2.

[0059] For example, the reference point identification unit 2040 calculates the x coordinate of the reference point 100 as follows: First, the reference point identification unit 2040 determines whether the width W-w1 of the open area is greater than 1 / n times the width W of the entire captured image 30. In other words, the reference point identification unit 2040 determines whether W-w1 > W / n, where n is a predetermined positive integer.

[0060] If W-w1 > W / n, the reference point identification unit 2040 moves the x coordinate of the reference point horizontally by (W-w1) / 2 from x1+w / 2. On the other hand, if W-w1 > W / n does not hold, the reference point identification unit 2040 moves the x coordinate of the reference point horizontally by W-w1 from x1+w / 2.

[0061] Here, the x coordinate of the reference point is moved so as to approach the vacant area. That is, if the vacant area is located on the left side of the road (i.e., if the circumscribing rectangle 80 is located on the right side of the captured image 30), the x coordinate of the reference point is moved leftward from x1+w / 2. On the other hand, if the vacant area is located on the right side of the road (i.e., if the circumscribing rectangle 80 is located leftward of the captured image 30), the x coordinate of the reference point is moved rightward from x1+w / 2.

[0062] <Relative position determination: S106> The relative position identification unit 2060 identifies the relative position of each vehicle using the forward information 50 (S106). As described above, the relative position is a combination of the relative distance and the relative direction.

[0063] The relative position identification unit 2060 calculates the relative distance from the reference point for each vehicle whose vehicle information 70 is indicated in the forward information 50. Specifically, the relative position identification unit 2060 calculates, for each vehicle, the distance between the position 74 of that vehicle and the reference point 100 as the relative distance.

[0064] Furthermore, for each vehicle whose vehicle information 70 is indicated in the forward information 50, the relative position identification unit 2060 calculates a straight line connecting the vehicle's position 74 and the reference point 100, and identifies the relative direction of the vehicle based on the straight line. For example, the relative direction is expressed by the slope (a in y=ax+b) of the line connecting the position 74 and the reference point 100. Alternatively, the relative direction can be expressed by the angle between the line connecting the position 74 and the reference point 100 and a line representing the reference direction. Here, any line passing through the reference point can be used as the line representing the reference direction. An example of a line passing through the reference point is a line that passes through the reference point and is parallel to the height direction of the captured image 30.

[0065] <Identifying the preceding vehicle: S114> The leading vehicle identification unit 2080 identifies a leading vehicle from among the vehicles whose vehicle information 70 is indicated in the forward information 50 (S114). First, the leading vehicle identification unit 2080 identifies vehicles traveling in the target lane based on the relative direction identified for each vehicle. Then, the leading vehicle identification unit 2080 identifies the vehicle with the shortest relative distance from among the vehicles traveling in the target lane as the leading vehicle.

[0066] Whether a vehicle is traveling in the target lane can be determined, for example, using a predetermined numerical range of the relative direction. That is, a numerical range of the relative direction that a vehicle traveling in the target lane can take is determined in advance. The leading vehicle identification unit 2080 determines, for each vehicle, whether the relative direction of the vehicle is within this predetermined numerical range. If the relative direction of the vehicle is within the predetermined numerical range, the leading vehicle identification unit 2080 determines that the vehicle is traveling in the target lane. On the other hand, if the relative direction of the vehicle is not within the predetermined numerical range, the leading vehicle identification unit 2080 determines that the vehicle is not traveling in the target lane.

[0067] <Output of processing results> The leading vehicle identification device 2000 outputs information indicating the processing result (hereinafter referred to as result information). The result information is information that can identify the leading vehicle. For example, the result information indicates an identifier 72 of the leading vehicle. Alternatively, for example, the result information may indicate a position 74 of the leading vehicle.

[0068] When the leading vehicle identification device 2000 identifies a leading vehicle for each of the plurality of pieces of forward information 50, the result information further indicates an identifier of the forward information 50. For example, the identifier of the forward information 50 is the identifier of the captured image 30 used to generate the forward information 50. The identifier of the captured image 30 is, for example, the file name of the captured image 30 or the frame number of the captured image 30 in the image frame sequence.

[0069] [Embodiment 2] 7 is a second diagram illustrating an outline of the operation of leading vehicle identification device 2000. Here, FIG. 7 is a diagram for facilitating understanding of the outline of leading vehicle identification device 2000, and the operation of leading vehicle identification device 2000 is not limited to that shown in FIG.

[0070] The leading vehicle identification device 2000 of the second embodiment acquires forward information 50 for each of the plurality of captured images 30 constituting the image frame sequence 40, thereby detecting a leading vehicle for each captured image 30. Then, the leading vehicle identification device 2000 of the second embodiment detects a predetermined action by a vehicle other than the target vehicle 10 based on the detected change in the leading vehicle. For example, the predetermined action is a cut-in or a cut-out.

[0071] Cut-in is an action in which "a vehicle changes from not being a leading vehicle to being a leading vehicle by changing lanes from a lane other than the target lane to the target lane." Cut-out is an action in which "a vehicle changes from being a leading vehicle to not being a leading vehicle by changing lanes from the target lane to a lane other than the target lane."

[0072] <Examples of effects> According to the preceding vehicle identification device 2000 of this embodiment, a vehicle preceding the target vehicle 10 is identified from each image frame constituting the image frame sequence 40. Then, based on a change in the preceding vehicle, it is detected that a specific action (cut-in, cut-out, etc.) has been performed by a vehicle other than the target vehicle 10. Therefore, according to the preceding vehicle identification device 2000, it is possible to easily and automatically detect from the image frame sequence 40 that a specific action has been performed by a vehicle other than the target vehicle 10.

[0073] Detection of specific actions such as cut-ins and cut-outs is useful for analyzing the circumstances of a traffic accident, for example. Actions such as cut-ins and cut-outs can be the cause of a traffic accident. Therefore, by analyzing the image frame sequence 40 and detecting specific actions such as cut-ins and cut-outs, information on actions that could be the cause of a traffic accident can be easily collected.

[0074] It is also possible to use portions of the image frame sequence 40 that represent specific actions such as cut-ins and cut-outs (hereinafter referred to as partial frame sequences) for training a machine learning model. Here, training a machine learning model requires a large amount of training data. However, obtaining a large amount of training data is laborious.

[0075] For example, suppose that a partial frame sequence representing a cut-in or cut-out must be manually extracted from an image frame sequence 40. In this case, the worker must use video editing software or the like to play back the image frame sequence 40 and detect the cut-in, cut-out, or other actions. This type of work requires time and effort.

[0076] In this regard, the leading vehicle identification device 2000 can easily identify the portions that represent cut-ins and cut-outs. Therefore, the leading vehicle identification device 2000 makes it easy to generate training data that is used to train a machine learning model.

[0077] The leading vehicle identification device 2000 of this embodiment will be described in more detail below.

[0078] <Example of functional configuration> Fig. 8 is a second block diagram illustrating an example of the functional configuration of the preceding vehicle identification device 2000. In the example of Fig. 2, the preceding vehicle identification device 2000 includes an acquisition unit 2020, a reference point identification unit 2040, a relative position identification unit 2060, a preceding vehicle identification unit 2080, and an action detection unit 2100. The action detection unit 2100 detects a specific action by a vehicle other than the target vehicle 10 based on a change in the preceding vehicle.

[0079] <Example of hardware configuration> The hardware configuration of the leading vehicle identification device 2000 of the second embodiment can be represented in Fig. 3, similar to the hardware configuration of the leading vehicle identification device 2000 of the first embodiment. However, the storage device 1080 of the second embodiment stores a program that realizes the functional configuration of the leading vehicle identification device 2000 of the second embodiment.

[0080] <Processing flow> 9 is a second flowchart illustrating the flow of processing executed by the leading vehicle identification device 2000. The acquisition unit 2020 acquires forward information 50 for each captured image 30 constituting the image frame sequence 40 (S202). Hereinafter, the i-th captured image 30 in the image frame sequence 40 will be referred to as captured image 30-i. Furthermore, the forward information 50 generated for the captured image 30-i will be referred to as forward information 50-i.

[0081] S204 to S210 constitute a loop process L1 that is performed for each piece of forward information 50-i. Here, the initial value of i is 1. The value of i is incremented by 1 at the end of each iteration. Furthermore, the total number of captured images 30 that make up the image frame sequence 40 is represented as N (N is an integer equal to or greater than 2).

[0082] In S204, the leading vehicle identification device 2000 determines whether i is equal to or less than N. If i is greater than N, the execution of the loop process L1 is ended. On the other hand, if i is equal to or less than N, S206 is executed.

[0083] In S206, the leading vehicle identification device 2000 identifies the leading vehicle in the captured image 30-i by using the forward information 50-i. The process executed in S206 corresponds to the series of processes (S104 to S108) executed by the leading vehicle identification device 2000 of the first embodiment by using the forward information 50.

[0084] The action detection unit 2100 detects a specific action based on the result of identifying the preceding vehicle in S206 (S208). For example, if cut-in and cut-out are treated as specific actions, the action detection unit 2100 attempts to detect cut-in and cut-out in S208. The detection of these multiple specific actions may be performed sequentially or in parallel.

[0085] S210 is the end of the loop L1, where the value of i is incremented by 1 and the next iteration of the loop L1 is executed.

[0086] <Acquisition of forward information 50: S202> The acquisition unit 2020 acquires the forward information 50 generated for each captured image 30 that constitutes the image frame sequence 40. Here, the method for acquiring the forward information 50 for one captured image 30 is the same as that described in the first embodiment.

[0087] The acquisition unit 2020 may acquire all of the forward information 50 obtained from the image frame sequence 40 at once, or may acquire the forward information 50 in multiple batches. In the latter case, for example, the forward information generation device is configured to generate forward information 50 for the captured image 30 each time the captured image 30 is generated by the camera 20. In other words, the forward information generation device is configured to generate forward information 50 from the captured image 30 in real time. The acquisition unit 2020 acquires new forward information 50 each time the new forward information 50 is generated.

[0088] <Detection of specific actions: S208> The action detection unit 2100 detects a specific action based on the result of identifying the preceding vehicle (S208). As described above, the specific action is, for example, a cut-in or a cut-out. A method for detecting a cut-in or a cut-out will be described below.

[0089] 10 is a flowchart illustrating the process of detecting cut-ins and cut-outs. The motion detection unit 2100 determines whether the current leading vehicle is the same as the previous leading vehicle (S302). Here, the current leading vehicle is the leading vehicle for the captured image 30-i, and the previous leading vehicle is the leading vehicle for the captured image 30-(i-1).

[0090] If the current leading vehicle is the same as the previous leading vehicle (S302: YES), the motion detection unit 2100 determines that neither cut-in nor cut-out has occurred (S304). On the other hand, if the current leading vehicle is different from the previous leading vehicle (S302: NO), the motion detection unit 2100 determines whether the current leading vehicle has moved into the target lane from a lane other than the target lane (S306).

[0091] If the current leading vehicle has moved from a lane other than the target lane into the target lane (S306: YES), the motion detection unit 2100 determines that a cut-in has been performed by the current leading vehicle (S308). On the other hand, if the current leading vehicle has not moved from a lane other than the target lane into the target lane (S306: NO), the motion detection unit 2100 determines that a cut-out has been performed by the previous leading vehicle (S310).

[0092] Here, lane movement by a vehicle can be identified based on the history of the vehicle's relative direction. For example, a straight line that passes through a reference point and is parallel to the vertical direction of the captured image 30 is treated as a reference line representing the reference direction. If the relative direction of a certain vehicle changes toward the reference direction and, as a result, the relative direction of the certain vehicle falls within a predetermined range representing that the vehicle is traveling in the target lane, the motion detection unit 2100 determines that the vehicle has moved from a lane other than the target lane to the target lane. On the other hand, if the relative direction of a certain vehicle changes away from the reference direction and, as a result, the relative direction of the certain vehicle is no longer within the predetermined range representing that the vehicle is traveling in the target lane, the motion detection unit 2100 determines that the vehicle has moved from the target lane to a lane other than the target lane.

[0093] 10, cut-in detection is performed before cut-out detection. However, cut-out detection may be performed before cut-in detection. Furthermore, the series of processes for detecting cut-in and the series of processes for detecting cut-out may be performed independently of each other.

[0094] Here, it is conceivable that the target vehicle 10 itself may change lanes. Therefore, if it is determined that the current front vehicle is different from the previous front vehicle (S302: NO), the action detection unit 2100 may determine whether the target vehicle 10 has changed lanes. If it is determined that the target vehicle 10 has changed lanes, the action detection unit 2100 determines that neither cut-in nor cut-out has occurred (S304). On the other hand, if it is determined that the target vehicle 10 has not changed lanes, the action detection unit 2100 executes S306.

[0095] <<Other specific operations>> The specific operation is not limited to the two, cut-in and cut-out. Other specific operations include, for example, an operation in which "the preceding vehicle becomes a distant vehicle." A distant vehicle is a vehicle that "is traveling in the target lane and is the closest vehicle to the target vehicle 10, but is traveling at a position far away from the target vehicle 10." Even if a vehicle is traveling in the target lane and is the closest vehicle to the target vehicle 10, if it is traveling at a position far away from the target vehicle 10, it is unlikely to affect the traveling of the target vehicle 10. Therefore, for example, the preceding vehicle identification device 2000 treats such a vehicle as a distant vehicle and distinguishes it from the preceding vehicle.

[0096] For example, if the current leading vehicle and the previous leading vehicle are the same (S302: YES), the action detection unit 2100 determines whether the relative distance of the current leading vehicle is equal to or greater than a threshold. If the relative distance of the current leading vehicle is equal to or greater than the threshold, the action detection unit 2100 determines that an action of "becoming a distant vehicle" has been performed. Note that the vehicle that has performed the action of becoming a leading vehicle is both the previous leading vehicle and the current leading vehicle.

[0097] <Output of result information> The leading vehicle identification device 2000 outputs result information. For example, the result information indicates information about one or more specific actions detected from the image frame sequence 40. The information about the specific action indicates, for example, the type of the detected specific action, the identifier of the vehicle that performed the specific action, and the time when the specific action was performed. The time when the specific action was performed is represented, for example, by the time when the captured image 30 in which the specific action was detected was generated.

[0098] Furthermore, the result information may indicate a period during which the specific action was performed, instead of or in addition to the time point at which the specific action was performed. The period during which the specific action was performed is set, for example, as a period from a predetermined time before to a predetermined time after the time point at which the specific action was performed.

[0099] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0100] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0101] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0102] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) an acquisition means for acquiring forward information including information about a road area included in a captured image obtained by capturing an image of a road ahead of a target vehicle and information about one or more vehicles on the road; a reference point specifying means for specifying a reference point of the target vehicle in the captured image using the forward information; a relative position specifying means for specifying a relative distance and a relative direction of each vehicle shown in the forward information with respect to the reference point; a preceding vehicle identification means for identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each of the vehicles, The preceding vehicle is the vehicle that is closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle. (Appendix 2) the forward information indicates a width and a position of an image area representing a road ahead of the target vehicle; 2. The preceding vehicle identification device according to claim 1, wherein the reference point identification means identifies the horizontal position of the reference point based on the width and position of an image area representing the road ahead of the target vehicle. (Appendix 3) 3. The preceding vehicle identification device according to claim 1, wherein the relative position identification means identifies the inclination of a line passing through the position of the vehicle and the reference point as the relative direction of the vehicle. (Appendix 4) The preceding vehicle identification device described in Appendix 1 or 2, wherein the preceding vehicle identification means identifies the vehicle whose relative direction is within a predetermined range as the vehicle traveling in the same lane as the target vehicle. (Appendix 5) the acquisition means acquires the forward information corresponding to each of the plurality of captured images generated by repeatedly capturing images of the forward area of ​​the target vehicle, the preceding vehicle identification means identifies the preceding vehicle for each of the plurality of pieces of forward information; 3. The preceding vehicle identification device according to claim 1, further comprising: a motion detection means for detecting a predetermined motion of the vehicle based on a change in the identified preceding vehicle. (Appendix 6) The preceding vehicle identification device described in Appendix 5, wherein the action detection means detects as the predetermined action an action in which the target vehicle becomes the preceding vehicle by moving from a lane in which the target vehicle is not traveling to a lane in which the target vehicle is traveling, or an action in which the target vehicle ceases to be the preceding vehicle by moving from a lane in which the target vehicle is traveling to a lane in which the target vehicle is not traveling. (Appendix 7) The preceding vehicle identification device described in Appendix 6, wherein the operation detection means detects, based on the change in the relative direction of the vehicle, that the vehicle has moved from a lane in which the target vehicle is not traveling to a lane in which the target vehicle is traveling, or that the vehicle has moved from a lane in which the target vehicle is traveling to a lane in which the target vehicle is not traveling. (Appendix 8) an acquisition step of acquiring forward information including information about a region of a road included in a captured image obtained by capturing an image of a region ahead of the target vehicle and information about one or more vehicles on the road; a reference point specifying step of specifying a reference point of the target vehicle in the captured image using the forward information; a relative position specifying step of specifying a relative distance and a relative direction of each vehicle shown in the forward information with respect to the reference point; a preceding vehicle identifying step of identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each of the vehicles, A computer-implemented method for identifying a preceding vehicle, wherein the preceding vehicle is the vehicle closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle. (Appendix 9) the forward information indicates a width and a position of an image area representing a road ahead of the target vehicle; 9. The preceding vehicle identification method according to claim 8, wherein in the reference point identification step, the horizontal position of the reference point is identified based on the width and position of an image area representing the road ahead of the target vehicle. (Appendix 10) an acquisition step of acquiring forward information including information about a region of a road included in a captured image obtained by capturing an image of a region ahead of the target vehicle and information about one or more vehicles on the road; a reference point specifying step of specifying a reference point of the target vehicle in the captured image using the forward information; a relative position specifying step of specifying a relative distance and a relative direction of each vehicle shown in the forward information with respect to the reference point; a preceding vehicle identifying step of identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each of the vehicles; The preceding vehicle is the vehicle closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle.

[0103] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 3 to 7 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Note 8 (method) in the same dependency relationship as Supplementary Note 3 to 7. Some or all of the elements (e.g., configurations and functions) described in Supplementary Note 2 to 7 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Note 10 (program) in the same dependency relationship as Supplementary Note 2 to 7. [Explanation of symbols]

[0104] 10 Target vehicles 20 Camera 30 Captured images 40 image frame sequences 50 Forward information 60 Road Information 62 positions 64 Dimensions 70 Vehicle Information 72 Identifier 74 position 76 Dimensions 80 circumscribed rectangle 100 reference points 1000 computers 1020 Bus 1040 processor 1060 memory 1080 storage device 1100 Input / Output Interface 1120 Network Interface 2000 Leading vehicle identification device 2020 Acquisition Department 2040 Reference point identification part 2060 Relative position determination unit 2080 Leading Vehicle Identification Department 2100 Motion detection unit

Claims

1. an acquisition means for acquiring forward information including information about a road area included in a captured image obtained by capturing an image of a road ahead of a target vehicle and information about one or more vehicles on the road; a reference point specifying means for specifying a reference point of the target vehicle in the captured image using the forward information; a relative position specifying means for specifying a relative distance and a relative direction of each vehicle shown in the forward information with respect to the reference point; a preceding vehicle identification means for identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each of the vehicles, The preceding vehicle is the vehicle that is closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle.

2. the forward information indicates a width and a position of an image area representing a road ahead of the target vehicle; 2. The preceding vehicle identification device according to claim 1, wherein the reference point identification means identifies the horizontal position of the reference point based on the width and position of an image area representing the road ahead of the target vehicle.

3. 3. The preceding vehicle identification device according to claim 1, wherein the relative position identification means identifies, as the relative direction of the vehicle, a gradient of a line passing through the position of the vehicle and the reference point.

4. 3. The preceding vehicle identification device according to claim 1, wherein the preceding vehicle identification means identifies the vehicle whose relative direction is within a predetermined range as the vehicle traveling in the same lane as the target vehicle.

5. the acquisition means acquires the forward information corresponding to each of the plurality of captured images generated by repeatedly capturing images of the forward area of ​​the target vehicle, the preceding vehicle identification means identifies the preceding vehicle for each of the plurality of pieces of forward information; 3. The preceding vehicle identification device according to claim 1, further comprising: a motion detection unit that detects a predetermined motion of the preceding vehicle based on a change in the identified preceding vehicle.

6. 6. The preceding vehicle identification device according to claim 5, wherein the action detection means detects as the predetermined action an action in which the target vehicle becomes the preceding vehicle by moving from a lane in which the target vehicle is not traveling to a lane in which the target vehicle is traveling, or an action in which the target vehicle ceases to be the preceding vehicle by moving from a lane in which the target vehicle is traveling to a lane in which the target vehicle is not traveling.

7. 7. The preceding vehicle identification device according to claim 6, wherein the movement detection means detects, based on the change in the relative direction of the vehicle, that the vehicle has moved from a lane in which the target vehicle is not traveling to a lane in which the target vehicle is traveling, or that the vehicle has moved from a lane in which the target vehicle is traveling to a lane in which the target vehicle is not traveling.

8. an acquisition step of acquiring forward information including information about a road area included in a captured image obtained by capturing an image of a road ahead of the target vehicle and information about one or more vehicles on the road; a reference point specifying step of specifying a reference point of the target vehicle in the captured image using the forward information; a relative position specifying step of specifying a relative distance and a relative direction of each vehicle shown in the forward information with respect to the reference point; a preceding vehicle identifying step of identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each of the vehicles, A computer-implemented method for identifying a preceding vehicle, wherein the preceding vehicle is the vehicle closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle.

9. the forward information indicates a width and a position of an image area representing a road ahead of the target vehicle; 9. The method for identifying a leading vehicle according to claim 8, wherein the horizontal position of the reference point is identified based on the width and position of an image area representing a road ahead of the target vehicle in the reference point identifying step.

10. an acquisition step of acquiring forward information including information about a road area included in a captured image obtained by capturing an image of a road ahead of the target vehicle and information about one or more vehicles on the road; a reference point specifying step of specifying a reference point of the target vehicle in the captured image using the forward information; a relative position specifying step of specifying a relative distance and a relative direction of each vehicle shown in the forward information with respect to the reference point; a preceding vehicle identifying step of identifying a preceding vehicle of the target vehicle based on the relative distance and the relative direction identified for each of the vehicles; The preceding vehicle is the vehicle closest to the target vehicle among the vehicles traveling in the same lane as the target vehicle.

Images