Lane detection method, image processing device, and lane detection program

Abstract

To provide a lane detection method, an image processing device, and a lane detection program that detect lanes on a road with high accuracy.SOLUTION: A lane detection method includes: an image acquisition step of acquiring a captured image from a camera; a vehicle detection step of detecting a vehicle from the captured image; a coordinate conversion step of converting the vehicle's position into road coordinates; a vehicle trajectory calculation step of calculating a vehicle trajectory from the position information of the road coordinates; an accumulation step of accumulating the vehicle trajectories for M vehicles (M is an integer equal to or greater than 1); a trajectory difference calculation step of calculating the difference in the trajectory between a lane detection line that intersects the vehicle trajectory and is set in the captured image and the maximum and the minimum values in the road coordinates at the intersection of the vehicle trajectory; and a determination step of accumulating the vehicle trajectories for M vehicles per a road divided area that divides into sections where the different lane exists, determining whether or not the trajectory difference of the accumulated vehicle trajectory for M vehicles is less than a length L, and determining that the road divided area includes one lane when it is determined that the trajectory difference is less than the length L.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a lane detection method, an image processing device, and a lane detection program. [Background technology]

[0002] Patent Document 1 discloses a computer-implemented method for generating a lane configuration of a road for automated driving. The computer-implemented method includes receiving sensory data describing a set of trajectories traveled by a plurality of vehicles navigating through a road segment of a road over a period of time. The computer-implemented method also includes determining a mainstream distribution of the set of trajectories by analyzing the set of trajectories using a set of rules, and determining a distribution of one or more lanes within the road segment based on the determined mainstream distribution of the trajectories. The computer-implemented method also includes, for each of the one or more lanes, calculating a lane reference line within the lane associated with the lane based on the trajectories within the lane. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7141370 specification Summary of the Invention [Problem to be solved by the invention]

[0004] There is known a technology for detecting information of vehicles traveling on a road using a camera installed on a highway or a national road, and analyzing the traffic volume or traffic conditions on the road. Conventionally, the imaging range of a camera installed on a highway or a national road and subjected to image processing is fixed, but in recent years, cameras that allow a monitor to change the imaging range by remote control according to the situation of vehicles traveling on the road have been often used for image processing. For example, when an accident occurs on a road, a monitor remotely controls the camera and zooms in to capture an image of a vehicle involved in the accident. If the imaging range of the camera changes, the lane position and number of lanes captured change, and there is a problem that information of vehicles traveling on the road cannot be accurately detected by image processing.

[0005] Patent Document 1 discloses that the trajectory of a vehicle is analyzed using analysis rules to estimate the lane. However, the specific method of the analysis rules is not disclosed, and there is still room for further study on a method for detecting the lane of a road captured by a camera with high accuracy.

[0006] The present disclosure has been devised in view of the above-mentioned conventional situation, and has an object to detect road lanes with high accuracy. [Means for solving the problem]

[0007] The present disclosure provides a lane detection method including an image acquisition step of acquiring an image from a camera, a vehicle detection step of detecting a vehicle from the captured image, a coordinate conversion step of converting the position of the vehicle into road coordinates, a trajectory calculation step of calculating a vehicle trajectory from position information of the road coordinates, an accumulation step of accumulating the vehicle trajectories for M vehicles (M: an integer of 1 or more), a trajectory difference calculation step of calculating a trajectory difference between a lane determination line set in the captured image intersecting the vehicle trajectory and a maximum and minimum value in road coordinates of an intersection with the vehicle trajectory, and a determination step of accumulating the vehicle trajectories for M vehicles for each road division area obtained by dividing an area in which different lanes exist, determining whether the trajectory difference of the vehicle trajectories accumulated for M vehicles is less than length L, and determining that the road division area includes one lane if it is determined that the trajectory difference is less than length L.

[0008] The present disclosure also provides an image processing device having a vehicle detection unit that acquires an image from a camera and detects a vehicle from the captured image, a coordinate conversion unit that converts the position of the vehicle into road coordinates, a vehicle storage unit that calculates a vehicle trajectory from position information of the road coordinates and accumulates the vehicle trajectories for M vehicles, and a lane determination unit that calculates a trajectory difference between a lane determination line set in the captured image that intersects with the vehicle trajectory and a maximum and minimum value in road coordinates of the intersection with the vehicle trajectory, accumulates the vehicle trajectories for M vehicles for each road division area that divides an area where different lanes exist, determines whether the trajectory difference of the vehicle trajectories accumulated for M vehicles is less than length L, and if it is determined that the trajectory difference is less than length L, determines that the road division area includes one lane.

[0009] The present disclosure also provides a lane detection program that causes a computer to execute the above-described lane detection method.

[0010] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0011] According to the present disclosure, road lanes can be detected with high accuracy. [Brief description of the drawings]

[0012] [Figure 1] FIG. 13 is a diagram showing captured images when the imaging range of a camera is changed. [Diagram 2] A system configuration diagram of a lane detection system according to the present embodiment. [Diagram 3] An example of a single white line being detected on a two-lane road. [Figure 4] A diagram showing an example where white lines were not detected on a two-lane road [Diagram 5]An example of a three-lane road where only one white line was detected. [Figure 6] An example of two white lines being detected on a three-lane road. [Figure 7] An example of a single white line being detected on a two-lane curved road. [Figure 8] A diagram showing an example where white lines were not detected on a two-lane curved road [Figure 9] An example of a curved three-lane road where only one white line was detected. [Figure 10] An example of two white lines being detected on a three-lane curved road. [Figure 11] A flowchart showing an example of a process for determining lanes when the imaging range of a camera changes. [Figure 12] A flowchart showing an example of image processing area setting processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, with reference to the drawings as appropriate, an embodiment specifically disclosing a lane detection method and an image processing device according to the present disclosure will be described in detail. However, more detailed description than necessary may be omitted. For example, detailed description of already well-known matters and duplicated description of substantially the same configuration may be omitted. This is to avoid the following description becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0014] First, a captured image when the imaging range of the camera is changed will be described with reference to Fig. 1. Fig. 1 is a diagram showing a captured image when the imaging range of the camera is changed.

[0015] The captured image IM1 is an image captured by a camera fixedly installed diagonally above the road. In the example shown in Fig. 1, the road to be captured has three lanes. The captured image IM1 is an image captured by a camera set with an imaging range capable of detecting vehicles traveling on each of the three lanes. The imaging range of the camera is set, for example, by an operator who installs the camera with a direction and angle of view that allows all lanes to be captured as shown in the captured image IM1 as an initial setting.

[0016] The captured image IM2 is an image of the same road as the captured image IM1 captured by the same camera as the captured image IM1. The captured image IM2 is an image of a part of the road shown in the captured image IM1. In other words, the captured image IM2 is an image captured by changing the imaging range of the camera so as to enlarge and capture a part of the road. Only two of the three lanes are shown in the captured image IM2.

[0017] As shown in FIG. 1, when the imaging range of the camera changes, the number of lanes and the positions of the lanes shown in the captured image change. Conventional image processing devices detect vehicles using lane information set in an image captured with an initial imaging range, so there is a problem that when the imaging range of the camera changes, the vehicle cannot be detected or is detected erroneously. The imaging range of the camera changes due to remote control by a monitor who checks the occurrence of events such as vehicle stoppage, low speed, traffic jam, avoidance, or wrong-way driving, so when the imaging range deviates from the initial setting, vehicle detection must be stopped. In order to accurately detect vehicles traveling on the road for each lane even when the imaging range of the camera changes, it is necessary to detect the lanes of the road every time the imaging range of the camera changes.

[0018] Next, a system configuration of the lane detection system 1 according to the present embodiment will be described with reference to Fig. 2. Fig. 2 is a system configuration diagram of the lane detection system 1 according to the present embodiment.

[0019] The lane detection system 1 includes a camera 10 and an image processing device 20 .

[0020] The camera 10 is a camera fixedly installed on a road to be imaged. The camera 10 is, for example, a PTZ camera. The camera 10 has the functions of horizontal rotation (pan), vertical rotation (tilt), and zoom. The camera 10 has a processor (not shown) and a memory (not shown). When a security officer gives an instruction to operate the camera remotely, the processor automatically controls the pan, tilt, and zoom operations using a program stored in the memory. Hereinafter, the pan, tilt, and zoom operations are referred to as PTZ operations. The imaging range of the camera 10 changes at any time due to the PTZ operation. The camera 10 transmits the captured image to the communication I / F 21 of the image processing device 20.

[0021] The image processing device 20 is a device that detects lanes from a captured image acquired by the camera 10. The image processing device 20 is, for example, a personal computer or a server device, etc. The image processing device 20 has a communication I / F 21, a memory 22, an input device 23, and a processor 24.

[0022] The communication I / F 21 is an interface circuit that performs wireless or wired communication between the camera 10 and the image processing device 20. The I / F is an interface. Note that communication with the camera 10 may be via a network. The communication method by the communication I / F 21 is wired communication or wireless communication. Specifically, the wireless communication may be mobile communication such as Wide Area Network, Local Area Network, Long Term Evolution, 4G, 5G, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark) communication), or communication for mobile phones. Note that the wireless communication method is not limited to the above-mentioned examples.

[0023] The memory 22 is configured using, for example, a Random Access Memory (hereinafter referred to as "RAM") and a Read Only Memory (hereinafter referred to as "ROM"), and temporarily stores programs necessary for the operation of the image processing device 20, as well as data generated during operation. The RAM is, for example, a work memory used during the operation of the image processing device 20. The ROM stores and holds, for example, programs for controlling the image processing device 20 in advance. The memory 22 temporarily saves captured images acquired from the camera 10.

[0024] The input device 23 is a device that accepts input of information (such as the height from the road where the camera 10 is installed, the depression angle, and the angle of view) necessary for the coordinate conversion coefficient calculation unit 26 to calculate the coordinate conversion coefficient. The input device 23 is composed of, for example, a keyboard, a mouse, or a touch pen. Furthermore, the information necessary for the coordinate conversion coefficient calculation unit 26 to calculate the coordinate conversion coefficient may be acquired from the camera 10, in which case the input device 23 may be omitted. Furthermore, the input device 23 may be installed outside the image processing device 20.

[0025] The processor 24 is, for example, a central processing unit, a digital signal processor, a graphic processing unit, or a field programmable gate array. The processor 24 functions as a controller that manages the overall operation of the image processing device 20. The processor 24 performs control processing for managing the operation of each part of the image processing device 20, data input / output processing between each part of the image processing device 20, data calculation processing, and data storage processing. The processor 24 operates according to a program stored in the memory 22. The processor 24 uses the memory 22 during operation, and temporarily stores data generated or acquired by the processor 24 in the memory 22. The processor 24 realizes the functions of the operation determination unit 25, the coordinate conversion coefficient calculation unit 26, the white line detection unit 27, the vehicle detection unit 28, the vehicle tracking unit 29, the coordinate conversion unit 30, the image processing area setting unit 31, the lane determination unit 32, the vehicle trajectory accumulation unit 33, the traffic volume measurement unit 34, and the event determination unit 35 by using the program and data stored in the memory 22.

[0026] The operation determination unit 25 acquires information related to PTZ operation from the camera 10. The information related to PTZ operation is, for example, values ​​changed from the initial setting states of pan, tilt, and zoom. The operation determination unit 25 uses the information related to PTZ operation acquired from the camera 10 to determine whether the camera 10 has performed a PTZ operation from the initial setting state. The operation determination unit 25 may acquire a captured image from the memory 22 and determine that the camera 10 has performed a PTZ operation.

[0027] The coordinate conversion coefficient calculation unit 26 calculates coordinate conversion coefficients using information received from the input device 23. The coordinate conversion coefficients are coefficients for converting coordinates of a captured image (hereinafter referred to as "image coordinates") into coordinates of a road (hereinafter referred to as "road coordinates").

[0028] The white line detection unit 27 acquires the captured image from the memory 22. The white line detection unit 27 detects white lines indicating lane boundaries of the road from the acquired captured image using a known technique such as image recognition. The white line detection unit 27 outputs the detection result of the detected white lines to the lane determination unit 32.

[0029] The vehicle detection unit 28 acquires the captured image from the memory 22. The vehicle detection unit 28 detects vehicles from the acquired captured image using a known technique such as image recognition. The vehicle detection unit 28 outputs the vehicle detection result to the vehicle tracking unit 29.

[0030] The vehicle tracking unit 29 matches the vehicles between the two captured images and tracks the vehicles using the vehicle detection result obtained from the vehicle detection unit 28 and the area information obtained from the image processing area setting unit 31. The vehicle tracking unit 29 outputs the vehicle tracking result to the coordinate conversion unit 30 and the vehicle trajectory accumulation unit 33.

[0031] The coordinate conversion unit 30 converts the detection result of the vehicle by the vehicle detection unit 28 from image coordinates to road coordinates to calculate the size and position of the vehicle. The coordinate conversion unit 30 outputs the calculation results to the image processing area setting unit 31, the lane determination unit 32, the vehicle trajectory accumulation unit 33, the traffic volume measurement unit 34 and the event determination unit 35.

[0032] The image processing area setting unit 31 determines a lane boundary for each lane using the position information accumulated by the vehicle trajectory accumulation unit 33 when tracking the vehicle and the result of the lane determination by the lane determination unit 32. The image processing area setting unit 31 sets the coordinates of the captured image that indicate the position of the determined lane boundary. The image processing area setting unit 31 sets a lane number for each determined lane boundary.

[0033] The lane determination unit 32 classifies the vehicle trajectories accumulated by the vehicle trajectory accumulation unit 33 into N (N: an integer equal to or greater than 1) categories (hereinafter referred to as “N categories”) to determine lanes, using at least one of the position information accumulated by the vehicle trajectory accumulation unit 33 when tracking the vehicle and the position information of the white lines detected by the white line detection unit 27. The lane determination unit 32 outputs the determination result to the image processing area setting unit 31.

[0034] The vehicle trajectory storage unit 33 stores the position information of the image coordinates of the vehicle output by the vehicle tracking unit 29 and the position information of the road coordinates of the vehicle converted from the image coordinates to the road coordinates by the coordinate conversion unit 30. The vehicle trajectory storage unit 33 outputs the result of storing the position information to the lane determination unit 32.

[0035] The traffic volume measurement unit 34 measures the speed and number of vehicles using the vehicle information converted from image coordinates to road coordinates by the coordinate conversion unit 30, and calculates the traffic volume.

[0036] The event determination unit 35 measures the speed and number of vehicles using the vehicle information converted from image coordinates to road coordinates, and determines events such as vehicle stoppage, low speed, congestion, avoidance, or wrong-way driving.

[0037] Next, a processed image of a straight road will be described with reference to Figs. 3 to 6. Fig. 3 is a diagram showing an example where one white line is detected on a two-lane road. Fig. 4 is a diagram showing an example where no white line is detected on a two-lane road. Fig. 5 is a diagram showing an example where only one white line is detected on a three-lane road. Fig. 6 is a diagram showing an example where two white lines are detected on a three-lane road. Hereinafter, an image captured by camera 10 and subjected to various processes by processor 24 to determine lane boundary lines will be referred to as a processed image.

[0038] Hereinafter, the axis parallel to the side of the processed image in the direction in which the lanes are arranged is referred to as the X-axis, and the axis parallel to the side of the periphery of the processed image and perpendicular to the X-axis is referred to as the Y-axis. These directional expressions are used for convenience of explanation and are not intended to limit the posture of the structure during actual use. Furthermore, the X coordinate in the processed image is referred to as the image X coordinate, and the Y coordinate is referred to as the image Y coordinate. The coordinate converted from the image X coordinate to the road coordinate by the coordinate conversion unit 30 is referred to as the road X coordinate, and the coordinate converted from the image Y coordinate to the road coordinate is referred to as the road Y coordinate.

[0039] A processed image IM3 shown in FIG. 3 is a processed image of a captured image of a straight road including two lanes.

[0040] Line L1 indicates the upper limit of the Y coordinate (hereinafter referred to as the "image Y coordinate upper limit") of the resolution of the captured image that can be analyzed by processor 24. This may be determined, for example, by calculating the resolution from the physical length of line L1 in road coordinates and the number of horizontal pixels of camera 10. Also, the line indicating the lower limit of the image Y coordinate of the captured image that can be analyzed by processor 24 is referred to as the image Y coordinate lower end. In the example shown in FIG. 3, the line with the smallest image Y coordinate of processed image IM3 is set as the image Y coordinate lower end. Processor 24 processes the area between the image Y coordinate upper limit and the image Y coordinate lower end in the processed image.

[0041] The line L8 is a white line detected by the white line detection unit 27.

[0042] The line L3 is a line indicating the upper limit of the image Y coordinate of the detected white line (line L8) (hereinafter referred to as "image Y coordinate white line upper limit").

[0043] Lines L2 and L4 are lines for determining lanes (hereinafter referred to as "lane determination lines"). Of the two lane determination lines, line L2 with the larger image Y coordinate is referred to as the lane determination upper line, and line L4 with the smaller image Y coordinate is referred to as the lane determination lower line. When the white line detection unit 27 detects a white line, the lane determination unit 32 sets a lane determination upper line in the processed image IM3 in the area between the image Y coordinate upper limit and the image Y coordinate white line upper limit. Also, when the white line detection unit 27 detects a white line, the lane determination unit 32 sets a lane determination lower line in the processed image IM3 in the area between the image Y coordinate white line upper limit and the image Y coordinate lower end. The lane determination lines are approximately parallel to the X axis.

[0044] When the white line detection unit 27 detects a white line (line L8), the lane determination unit 32 divides the processed image IM3 in the Y-axis direction using the detected white line, and sets a road division area in the processed image IM3. The lane determination unit 32 sets an area R1, whose image X coordinate is smaller than the image X coordinate of the line L8, in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit, as the road division area 1. The lane determination unit 32 sets an area R2, whose image X coordinate is larger than the image X coordinate of the line L8, in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit, as the road division area 2. The road division area is an area set in the processed image. When the lane determination unit 32 determines that there are multiple lanes, the road division area is set for each determined lane, and is an area that includes the lane.

[0045] The vehicle trajectory accumulation unit 33 accumulates the trajectories of M vehicles (hereinafter referred to as "vehicle trajectories") (M: an integer equal to or greater than 1) for each road division area (i.e., for each road division area 1 and road division area 2) based on the results of vehicle tracking by the vehicle tracking unit 29. The number M may be arbitrarily set by a user or the like who uses the lane detection system 1. For example, in this embodiment, the vehicle trajectory accumulation unit 33 accumulates the trajectories of five vehicles as M vehicles. Note that M vehicles is not limited to five vehicles.

[0046] When storing the five vehicle trajectories, the vehicle trajectory storage unit 33 calculates and stores the image X coordinates of the intersections between the vehicle trajectories and the lane determination upper line (line L2) and the intersections between the vehicle trajectories and the lane determination lower line (line L4).

[0047] Furthermore, the vehicle trajectory storage unit 33 calculates and stores road X coordinates obtained by converting the image X coordinates of the intersections between the vehicle trajectory and the lane determination line (line L4) into road coordinates.

[0048] After five vehicles have passed through each road division area, the lane determination unit 32 calculates the maximum and minimum road X coordinates of the intersection between the accumulated vehicle trajectories of the five vehicles and the lane determination lower line. Hereinafter, the difference between the maximum and minimum values ​​is referred to as a trajectory difference. The lane determination unit 32 determines whether the trajectory difference is less than a length L [m] (L: positive number). The length L is, for example, 1.75. Hereinafter, the description will be given assuming L=1.75 as an example, but is not limited thereto. If the lane determination unit 32 determines that the trajectory difference is less than 1.75 [m], it determines that the road division area in which the five vehicle trajectories have been accumulated includes one lane.

[0049] Furthermore, if the lane determination unit 32 determines that the trajectory difference is not less than 1.75 [m], that is, is 1.75 [m] or more, it determines that the road division area in which the five vehicle trajectories are accumulated includes two or more lanes. The case in which the road division area in which the five vehicle trajectories are accumulated includes two or more lanes will be described with reference to FIGS.

[0050] When the trajectory difference is less than 1.75 [m] in all road division areas, the lane determining unit 32 ends setting of the road division areas.

[0051] The accumulated vehicle trajectories are referred to as a vehicle trajectory group. In the example shown in FIG. 3, the trajectory difference W1 of the vehicle trajectory group P1 in the region R1 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R1 includes one lane. In addition, the trajectory difference W2 of the vehicle trajectory group P2 in the region R2 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R2 includes one lane. The lane determination unit 32 outputs the lane determination result to the image processing region setting unit 31.

[0052] The image processing area setting unit 31 calculates the average value of the image X coordinate of the intersection between the vehicle trajectory group P1 and the lane determination upper line (line L2). The image processing area setting unit 31 converts the calculated average value of the image X coordinate into a road X coordinate. The image processing area setting unit 31 converts the road X coordinate obtained by adding 1.75 [m] to the converted road X coordinate into an image X coordinate, and obtains a point on the lane determination upper line (point S2 shown in FIG. 3). Next, the image processing area setting unit 31 converts the road X coordinate obtained by subtracting 1.75 [m] from the converted road X coordinate into an image X coordinate, and obtains a point on the lane determination upper line (point S1 shown in FIG. 3). The image processing area setting unit 31 performs the same process on the vehicle trajectory group P2, and obtains points S2 and S3. At this time, if point S2 has already been determined from vehicle trajectory group P1, the average value of point S2 determined from vehicle trajectory group P1 and point S2 determined from vehicle trajectory group P2 is set as the final point S2.

[0053] Next, the image processing area setting unit 31 calculates the average value of the image X coordinate of the intersection between the vehicle trajectory group P2 and the lane determination lower line (line L4). The image processing area setting unit 31 converts the calculated average value of the image X coordinate into a road X coordinate. The image processing area setting unit 31 converts the road X coordinate obtained by adding 1.75 [m] to the converted image X coordinate into an image X coordinate, and obtains a point on the lane determination lower line (point S5 shown in FIG. 3). Next, the image processing area setting unit 31 converts the road X coordinate obtained by subtracting 1.75 [m] from the converted road X coordinate into an image X coordinate, and obtains a point on the lane determination lower line (point S4 shown in FIG. 3). The image processing area setting unit 31 performs the same process on the vehicle trajectory group P2, and obtains points S5 and S6. At this time, if point S5 has already been determined from vehicle trajectory group P1, the average value of point S5 determined from vehicle trajectory group P1 and point S5 determined from vehicle trajectory group P2 is set as the final point S5.

[0054] The image processing area setting unit 31 determines the lane boundary line by connecting the two points obtained on the lane determination upper line and the lane determination lower line. The two points are selected by adding 1.75 [m] to the road X coordinate or by subtracting 1.75 [m] from the road X coordinate. That is, in the example shown in FIG. 3, the image processing area setting unit 31 determines the straight line (line L5) passing through points S1 and S4 as the lane boundary line. The image processing area setting unit 31 determines the straight line (line L6) passing through points S2 and S5 as the lane boundary line. The image processing area setting unit 31 determines the straight line (line L7) passing through points S3 and S6 as the lane boundary line.

[0055] After setting the lane boundary lines, the image processing area setting unit 31 sets the lane number based on the traveling direction of the vehicle so that the lane on the shoulder side is numbered as 1. In the example shown in Fig. 3, the area between lines L5 and L6 is "lane 1", and the area between lines L6 and L7 is "lane 2".

[0056] The image processing area setting unit 31 sets the target accuracy to an error in lane width when determining lane boundary lines to be within L / 2 (that is, within 1.75 [m]).

[0057] As described above, the image processing device 20 can divide the area of ​​the captured image using the detected white lines and determine the lane markings for each area. Since the image processing device 20 determines the lane markings for each divided area, it is possible to prevent erroneous lane markings and suppress lane marking errors.

[0058] A processed image IM4 shown in FIG. 4 is a processed image obtained by capturing a captured image of a straight road including two lanes.

[0059] Line L100 is the upper Y coordinate limit of the image. Line L9 is the upper line for lane determination. Line L10 is the lower line for lane determination.

[0060] In the processed image IM4, the white line detection unit 27 did not detect a white line. When a white line is not detected, the lane determination unit 32 sets a line L9, which is a lane determination upper line, in an arbitrary region in the upper part of the processed image IM4 (for example, an area in the processed image IM4 where the Y coordinate is greater than the Y coordinate of the midpoint of a side parallel to the Y coordinate). When a white line is not detected, the lane determination unit 32 sets a line L10, which is a lane determination lower line, in an arbitrary region in the lower part of the processed image IM4 (for example, an area in the processed image IM4 where the Y coordinate is smaller than the Y coordinate of the midpoint of a side parallel to the Y coordinate).

[0061] The vehicle trajectory storage unit 33 stores the vehicle trajectories of five vehicles. The lane determination unit 32 calculates the trajectory difference W5 based on the vehicle trajectories of the five vehicles. Since the trajectory difference W5 is 1.75 [m] or more, the lane determination unit 32 determines that there are two or more lanes in the processed image IM4.

[0062] The lane determination unit 32 sets the area dividing line by drawing a line L14 from the lower end of the image Y coordinate to the lane determination upper line (line L9) on the image X coordinate of the midpoint of the trajectory difference W5 set in the processed image IM4. As a method for setting the area dividing line, the lane determination unit 32 may calculate the intermediate value between the maximum and minimum values ​​of the image X coordinate of the intersection point between the vehicle trajectory and the lane determination upper line, and the intermediate value between the maximum and minimum values ​​of the image X coordinate of the intersection point between the vehicle trajectory and the lane determination lower line, and set the line connecting the intermediate values ​​as the area dividing line. The lane determination unit 32 sets the area R3, in the area between the lane determination upper line (line L9) and the lower end of the image Y coordinate, whose image X coordinate is smaller than the area dividing line (line L14), as the road division area 1. The lane determining unit 32 sets, as the road divided area 2, an area R4 between the lane determination upper line (line L9) and the bottom end of the image Y coordinate, where the image X coordinate is larger than the area dividing line (line L14).

[0063] The vehicle trajectory storage unit 33 adds the vehicle trajectories already stored for each road division area (that is, road division area 1 and road division area 2) and stores the vehicle trajectories until there are five vehicle trajectories in total.

[0064] The lane determination unit 32 calculates the trajectory difference W3 of the vehicle trajectory group P3 in the road division region 1 (region R3). The trajectory difference W3 is less than 1.75 [m], and the lane determination unit 32 determines that the region R3 includes one lane. The lane determination unit 32 calculates the trajectory difference W4 of the vehicle trajectory group P4 in the road division region 2 (region R4). The trajectory difference W4 is about 0.3 [m], and satisfies the condition that the trajectory difference is less than 1.75 [m], and the lane determination unit 32 determines that the region R4 includes one lane. The lane determination unit 32 outputs the lane determination result to the image processing region setting unit 31.

[0065] The image processing area setting unit 31 sets points S7, S8, S9, S10, S11, and S12 in the same manner as described with reference to FIG.

[0066] The image processing area setting unit 31 determines the line (line L11) passing through point S7 and point S10 as the lane boundary line. The image processing area setting unit 31 determines the line (line L12) passing through point S8 and point S11 as the lane boundary line. The image processing area setting unit 31 determines the line (line L13) passing through point S9 and point S12 as the lane boundary line.

[0067] The image processing area setting unit 31 sets the area between the lines L11 and L12 as "lane 1" and the area between the lines L12 and L13 as "lane 2" in terms of lane numbers.

[0068] As a result, even in cases where the image processing device 20 is unable to detect white lines, it is possible to determine the lane for each area by dividing the captured image into a plurality of areas using the vehicle trajectory.

[0069] A processed image IM5 shown in Fig. 5 is a processed image of a captured image of a straight road including three lanes. Processed image IM5 is an example in which three lanes exist but only one white line is detected.

[0070] Line L15 is the upper Y coordinate limit of the image. Line L16 is the upper lane determination line. Line L17 is the upper Y coordinate limit of the white line of the image. Line L18 is the lower lane determination line.

[0071] The white line detection unit 27 detects the line L24 as a white line. The lane determination unit 32 sets an area R5, whose image X coordinate is smaller than the image X coordinate of the line L24, in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit, as the road division area 1. The lane determination unit 32 sets an area R6, whose image X coordinate is larger than the image X coordinate of the line L24, in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit, as the road division area 2.

[0072] The vehicle trajectory accumulation unit 33 accumulates the trajectories of five vehicles for each road division area (that is, for each road division area 1 and road division area 2) based on the results of vehicle tracking by the vehicle tracking unit 29.

[0073] The lane determining unit 32 calculates the trajectory difference W9 based on the vehicle trajectories of the five vehicles in the road division region 1. Because the trajectory difference W9 is 1.75 [m] or more, the lane determining unit 32 determines that there are two or more lanes in the region R5.

[0074] The lane determination unit 32 sets a region dividing line by drawing a line L23 from the bottom of the image Y coordinate to the upper limit of the white line in the image Y coordinate (line L17) in the image X coordinate of the midpoint of the trajectory difference W9 set in the region R5. The lane determination unit 32 may also draw a line from the bottom of the image Y coordinate to the lane determination upper line (line L16) as the region dividing line. The lane determination unit 32 sets a region R7, in which the image X coordinate is smaller than the region dividing line (line L23) in the region between the upper limit of the white line in the image Y coordinate (line L17) and the bottom of the image Y coordinate, as the road divided region 1-1. The lane determination unit 32 sets a region R8, in which the image X coordinate is larger than the region dividing line (line L23) in the region between the upper limit of the white line in the image Y coordinate (line L17) and the bottom of the image Y coordinate, as the road divided region 1-2.

[0075] The vehicle trajectory storage unit 33 stores the vehicle trajectories in addition to the already stored vehicle trajectories until the vehicle trajectories in each of the areas R7 and R8 total five vehicle trajectories.

[0076] The lane determination unit 32 calculates the trajectory difference W6 of the vehicle trajectory group P5 in the region R7. The trajectory difference W6 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R7 includes one lane. The lane determination unit 32 calculates the trajectory difference W7 of the vehicle trajectory group P6 in the region R8. The trajectory difference W7 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R8 includes one lane. The lane determination unit 32 calculates the trajectory difference W8 of the vehicle region group P7 in the region R6. The trajectory difference W8 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R6 includes one lane.

[0077] From the above, the lane determination unit 32 determines that the processed image IM5 includes a total of three lanes. In this manner, the lane determination unit 32 continues creating road division regions until the trajectory difference in all road division regions is less than 1.75 [m]. The lane determination unit 32 outputs the lane determination result to the image processing region setting unit 31.

[0078] The image processing area setting unit 31 sets points S13, S14, S15, S16, S17, S18, S19, and S20 in the same manner as described with reference to FIG.

[0079] The image processing area setting unit 31 determines the line (line L19) passing through points S13 and S17 as the lane boundary line. The image processing area setting unit 31 determines the line (line L20) passing through points S14 and S18 as the lane boundary line. The image processing area setting unit 31 determines the line (line L21) passing through points S15 and S19 as the lane boundary line. The image processing area setting unit 31 determines the line (line L22) passing through points S16 and S20 as the lane boundary line.

[0080] The image processing area setting unit 31 sets the lane numbers as follows: the area between lines L19 and L20 is "Lane 1," the area between lines L20 and L21 is "Lane 2," and the area between lines L21 and L22 is "Lane 3."

[0081] A processed image IM6 shown in FIG. 6 is a processed image obtained by capturing a photograph of a straight road including three lanes.

[0082] Line L25 is the upper Y coordinate limit of the image. Line L26 is the upper lane determination line. Line L27 is the upper Y coordinate limit of the white line of the image. Line L28 is the lower lane determination line.

[0083] The white line detection unit 27 detects the lines L29 and L30 as white lines. The lane determination unit 32 sets an area R9, whose image X coordinate is smaller than the image X coordinate of the line L29, as a road division area 1 in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit. The lane determination unit 32 sets an area R10, whose image X coordinate is larger than the image X coordinate of the line L29 and smaller than the image X coordinate of the line L30, as a road division area 2 in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit. The lane determination unit 32 sets an area R11, whose image X coordinate is larger than the image X coordinate of the line L30, as a road division area 3 in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit.

[0084] The vehicle trajectory accumulation unit 33 accumulates the vehicle trajectories of five vehicles for each road division area (that is, for each road division area 1, road division area 2, and road division area 3) based on the results of vehicle tracking by the vehicle tracking unit 29.

[0085] The lane determining unit 32 calculates the trajectory difference W10 based on the trajectories of the five vehicles in the region R9. Since the trajectory difference W10 is approximately 0.3 [m] and satisfies the condition that the trajectory difference is less than 1.75 [m], the lane determining unit 32 determines that the region R9 includes one lane.

[0086] The lane determining unit 32 calculates the trajectory difference W11 based on the trajectories of the five vehicles in the region R10. Since the trajectory difference W11 is approximately 0.3 [m] and satisfies the condition that the trajectory difference is less than 1.75 [m], the lane determining unit 32 determines that the region R10 includes one lane.

[0087] The lane determining unit 32 calculates the trajectory difference W12 based on the trajectories of the five vehicles in the region R11. Since the trajectory difference W12 is approximately 0.3 [m] and satisfies the condition that the trajectory difference is less than 1.75 [m], the lane determining unit 32 determines that the region R11 includes one lane.

[0088] From the above, the lane determining unit 32 determines that a total of three lanes are included in the processed image IM6. The lane determining unit 32 outputs the lane determination result to the image processing area setting unit 31.

[0089] The image processing area setting unit 31 sets points S21, S22, S23, S24, S25, S26, S27, and S28 in the same manner as described with reference to FIG.

[0090] The image processing area setting unit 31 determines the line (line L31) passing through points S21 and S25 as the lane boundary line. The image processing area setting unit 31 determines the line (line L32) passing through points S22 and S26 as the lane boundary line. The image processing area setting unit 31 determines the line (line L33) passing through points S23 and S27 as the lane boundary line. The image processing area setting unit 31 determines the line (line L34) passing through points S24 and S28 as the lane boundary line.

[0091] The image processing area setting unit 31 sets the lane numbers as follows: the area between lines L31 and L32 is "lane 1", the area between lines L32 and L33 is "lane 2", and the area between lines L33 and L34 is "lane 3".

[0092] Next, a processed image of a road including a curve (hereinafter referred to as a "curved road") will be described with reference to Figs. 7 to 10. Fig. 7 is a diagram showing an example where one white line is detected on a two-lane curved road. Fig. 8 is a diagram showing an example where no white lines are detected on a two-lane curved road. Fig. 9 is a diagram showing an example where only one white line is detected on a three-lane curved road. Fig. 10 is a diagram showing an example where two white lines are detected on a three-lane curved road.

[0093] A processed image IM7 shown in FIG. 7 is a processed image of a captured image of a curved road including two lanes.

[0094] In the case of a curved road, the lane determination unit 32 sets two or more lane determination upper lines. In addition, in the case of a curved road, the lane determination unit 32 sets two or more lane determination lower lines. In the example shown in Fig. 7, the lane determination unit 32 sets two lane determination upper lines and two lane determination lower lines. Note that the number of lane determination lines set by the lane determination unit 32 may be arbitrarily set depending on the shape and length of the curve.

[0095] The lane determination unit 32 arbitrarily sets two lane determination upper line 1 (line L36) and lane determination upper line 2 (line L37) in the area between the image Y coordinate upper limit (line L35) and the image Y coordinate white line upper limit (line L38). The lane determination unit 32 arbitrarily sets two lane determination lower line 1 (line L39) and lane determination lower line 2 (line L40) in the area between the image Y coordinate white line upper limit (line L38) and the image Y coordinate lower end.

[0096] The white line detection unit 27 detects the line L44 as a white line. The lane determination unit 32 sets an area R12, whose image X coordinate is smaller than the line L44, between the upper limit of the white line in the image Y coordinate and the lower end of the image Y coordinate, as the road division area 1. The lane determination unit 32 sets an area R13, whose image X coordinate is larger than the image X coordinate of the line L44, between the upper limit of the white line in the image Y coordinate and the lower end of the image Y coordinate, as the road division area 2.

[0097] The vehicle trajectory accumulation unit 33 accumulates five vehicle trajectories for each road division area (that is, for each road division area 1 and road division area 2) based on the results of vehicle tracking by the vehicle tracking unit 29.

[0098] The lane determination unit 32 calculates the trajectory difference W13 of the vehicle trajectory group P11 in the region R12. The trajectory difference W13 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R12 includes one lane. The lane determination unit 32 calculates the trajectory difference W14 of the vehicle trajectory group P12 in the region R13. The trajectory difference W14 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R13 includes one lane.

[0099] The image processing area setting unit 31 sets points S29, S30, S31, S32, S33, S34, S35, S36, S37, S38, S39, and S40 for curved roads in the same manner as described with reference to FIG.

[0100] The image processing area setting unit 31 connects the four points found on the lane determination upper line 1 (line L36), lane determination upper line 2 (line L37), lane determination lower line 1 (line L39), and lane determination lower line 2 (line L40) to determine the lane boundary line. The method of selecting the four points is the same as the method of selecting the two points described in FIG. 3. The image processing area setting unit 31 draws a line L44 passing through points S29, S32, S35, and S38, and determines it as the lane boundary line. The image processing area setting unit 31 may draw a curved line or a straight line using a known technique such as a fitting technique. The image processing area setting unit 31 draws a line L42 passing through points S30, S33, S36, and S39, and determines it as the lane boundary line. The image processing area setting unit 31 draws a line L43 passing through points S31, S34, S37, and S40, and determines it as the lane boundary line.

[0101] The image processing area setting unit 31 sets the area between the lines L44 and L42 as "lane 1" in terms of lane numbers, and the area between the lines L42 and L43 as "lane 2."

[0102] A processed image IM8 shown in FIG. 8 is a processed image of a captured image of a curved road including two lanes.

[0103] Line L45 is the upper Y coordinate limit of the image. Line L46 is the lane determination upper line 1. Line L47 is the lane determination upper line 2. Line L48 is the lane determination lower line 1. Line L49 is the lane determination lower line 2.

[0104] In the processed image IM8, the white line detection unit 27 did not detect a white line. The vehicle trajectory storage unit 33 stores the vehicle trajectories of five vehicles in the entire processed image IM8. The lane determination unit 32 calculates the trajectory difference W15 based on the vehicle trajectories of the five vehicles. Since the trajectory difference W15 is 1.75 [m] or more, the lane determination unit 32 determines that there are two or more lanes in the processed image IM8.

[0105] The lane determining unit 32 draws a line L53 from the lane determination lower line 1 (line L48) to the lower end of the image Y coordinate at the image X coordinate of the midpoint of the trajectory difference W15, thereby setting an area dividing line.

[0106] The lane determination unit 32 sets an area R14 between the lane determination lower line 1 (line L48) and the bottom end of the image Y coordinate, where the image X coordinate is smaller than the area dividing line (line L53), as the road division area 1. The lane determination unit 32 sets an area R15 between the lane determination lower line 1 (line L48) and the bottom end of the image Y coordinate, where the image X coordinate is larger than the area dividing line (line L53), as the road division area 2.

[0107] The vehicle trajectory storage unit 33 adds the vehicle trajectories already stored for each road division area (that is, road division area 1 and road division area 2) and stores the vehicle trajectories until there are five vehicle trajectories in total.

[0108] The lane determination unit 32 calculates the trajectory difference W16 of the vehicle trajectory group P13 in the region R14. The trajectory difference W16 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R14 includes one lane. The lane determination unit 32 calculates the trajectory difference W17 of the vehicle trajectory group P14 in the region R15. The trajectory difference W17 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R15 includes one lane.

[0109] Image processing area setting unit 31 sets points S41, S42, S43, S44, S45, S46, S47, S48, S49, S50, S51, and S52 in the same manner as described with reference to FIG.

[0110] The image processing area setting unit 31 determines the line (line L50) passing through points S41, S44, S47, and S50 as the lane boundary line. The image processing area setting unit 31 determines the line (line L51) passing through points S42, S45, S48, and S51 as the lane boundary line. The image processing area setting unit 31 determines the line (line L52) passing through points S43, S46, S49, and S52 as the lane boundary line.

[0111] The image processing area setting unit 31 sets the area between the lines L50 and L51 as "lane 1" and the area between the lines L51 and L52 as "lane 2" in terms of lane numbers.

[0112] A processed image IM9 shown in Fig. 9 is a processed image of a captured image of a curved road including three lanes. Processed image IM9 is an example in which three lanes exist but only one white line is detected.

[0113] Line L54 is the image Y coordinate upper limit. Line L55 is lane determination upper line 1. Line L56 is lane determination upper line 2. Line L57 is the image Y coordinate white line upper limit. Line L58 ​​is lane determination lower line 1. Line L59 is lane determination lower line 2.

[0114] The white line detection unit 27 detects the line L63 as a white line. The lane determination unit 32 sets an area R16, whose image X coordinate is smaller than the image X coordinate of the line L63, in the area between the image Y coordinate white line upper limit (line L57) and the image Y coordinate lower end, as the road division area 1. The lane determination unit 32 sets an area R19, whose image X coordinate is larger than the image X coordinate of the line L63, in the area between the image Y coordinate white line upper limit (line L57) and the image Y coordinate lower end, as the road division area 2.

[0115] The vehicle trajectory accumulation unit 33 accumulates the trajectories of five vehicles for each road division area (that is, for each road division area 1 and road division area 2) based on the results of vehicle tracking by the vehicle tracking unit 29.

[0116] The lane determining unit 32 calculates the trajectory difference W21 based on the vehicle trajectories of the five vehicles in the road division region 1. Because the trajectory difference W21 is 1.75 [m] or more, the lane determining unit 32 determines that there are two or more lanes in the region R16.

[0117] The lane determining unit 32 draws a line L64 from the lane determination lower line 1 (line L58) to the lower end of the image Y coordinate at the image X coordinate of the midpoint of the trajectory difference W21 set in the region R16, thereby setting a region dividing line.

[0118] The lane determination unit 32 sets an area R17, in which the image X coordinate is smaller than the area dividing line (line L64) in the area between the lane determination lower line 1 (line L58) and the bottom end of the image Y coordinate of the road division area 1, as the road division area 1-1. The lane determination unit 32 sets an area R18, in which the image X coordinate is larger than the area dividing line (line L64) in the area between the lane determination lower line 1 (line L58) and the bottom end of the image Y coordinate of the road division area 1, as the road division area 1-2.

[0119] The vehicle trajectory storage unit 33 stores the vehicle trajectories in addition to the already stored vehicle trajectories until the vehicle trajectories in each of the areas R17 and R18 total five vehicle trajectories.

[0120] The lane determination unit 32 calculates the trajectory difference W18 of the vehicle trajectory group P15 in the region R17. The trajectory difference W18 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R17 includes one lane. The lane determination unit 32 calculates the trajectory difference W19 of the vehicle trajectory group P16 in the region R18. The trajectory difference W19 is about 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determination unit 32 determines that the region R18 includes one lane. The lane determination unit 32 calculates the trajectory difference W20 of the vehicle trajectory group P17 in the region R19. The trajectory difference W20 is approximately 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], so the lane determining unit 32 determines that the region R19 includes one lane.

[0121] From the above, the lane determining unit 32 determines that the processed image IM9 includes a total of three lanes.

[0122] Image processing area setting unit 31 sets points S53, S54, S55, S56, S57, S58, S59, S60, S61, S62, S63, S64, S65, S66, S67, and S68 in the same manner as described in Figure 3.

[0123] The image processing area setting unit 31 determines the line L60 passing through points S53, S57, S61, and S65 as the lane boundary line. The image processing area setting unit 31 determines the line L61 passing through points S54, S58, S62, and S66 as the lane boundary line. The image processing area setting unit 31 determines the line L62 passing through points S55, S59, S63, and S67 as the lane boundary line. The image processing area setting unit 31 determines the line L77 passing through points S56, S60, S64, and S68 as the lane boundary line.

[0124] The image processing area setting unit 31 sets the lane numbers as follows: the area between lines L60 and L61 as "lane 1", the area between lines L61 and L62 as "lane 2", and the area between lines L62 and L77 as "lane 3".

[0125] A processed image IM10 shown in FIG. 10 is a processed image of a captured image of a curved road including three lanes.

[0126] The white line detection unit 27 detects the lines L75 and L76 as white lines. The lane determination unit 32 sets an area R20, whose image X coordinate is smaller than the image X coordinate of the line L75, as a road division area 1 in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit. The lane determination unit 32 sets an area R21, whose image X coordinate is larger than the image X coordinate of the line L75 and smaller than the image X coordinate of the line L76, as a road division area 2 in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit. The lane determination unit 32 sets an area R22, whose image X coordinate is larger than the image X coordinate of the line L76, as a road division area 3 in the area between the image Y coordinate white line upper limit and the image Y coordinate lower limit.

[0127] The vehicle trajectory accumulation unit 33 accumulates the vehicle trajectories of five vehicles for each road division area (that is, for each road division area 1, road division area 2, and road division area 3) based on the results of vehicle tracking by the vehicle tracking unit 29.

[0128] The lane determining unit 32 calculates the trajectory difference W210 based on the vehicle trajectories of the five vehicles in the region R20. Since the trajectory difference W210 is approximately 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], the lane determining unit 32 determines that the region R20 includes one lane.

[0129] The lane determining unit 32 calculates the trajectory difference W22 based on the vehicle trajectories of the five vehicles in the region R21. Since the trajectory difference W22 is approximately 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], the lane determining unit 32 determines that the region R21 includes one lane.

[0130] The lane determining unit 32 calculates the trajectory difference W23 based on the trajectories of the five vehicles in the region R22. Since the trajectory difference W23 is approximately 0.3 [m], which satisfies the condition that the trajectory difference is less than 1.75 [m], the lane determining unit 32 determines that the region R22 includes one lane.

[0131] From the above, the lane determining unit 32 determines that a total of three lanes are included in the processed image IM10. The lane determining unit 32 outputs the lane determination result to the image processing area setting unit 31.

[0132] Image processing area setting unit 31 sets points S69, S70, S71, S72, S73, S74, S75, S76, S77, S78, S79, S80, S81, S82, S83, and S84 in the same manner as described in Figure 3.

[0133] The image processing area setting unit 31 determines the line L71 passing through points S69, S73, S77, and S81 as the lane boundary line. The image processing area setting unit 31 determines the line L72 passing through points S70, S74, S78, and S82 as the lane boundary line. The image processing area setting unit 31 determines the line L73 passing through points S71, S75, S79, and S83 as the lane boundary line. The image processing area setting unit 31 determines the line L74 passing through points S72, S76, S80, and S84 as the lane boundary line.

[0134] The image processing area setting unit 31 sets the lane numbers as follows: the area between lines L71 and L72 is set as "lane 1", the area between lines L72 and L73 is set as "lane 2", and the area between lines L73 and L74 is set as "lane 3".

[0135] Next, an example of a process for determining a lane when the imaging range of the camera changes will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of a process for determining a lane when the imaging range of the camera changes.

[0136] The coordinate conversion coefficient calculation unit 26 acquires information about the captured image from the input device 23 (step St100). The information about the captured image includes the height of the position where the camera 10 is installed, the depression angle and the angle of view of the camera 10, and the like.

[0137] The coordinate transformation coefficient calculation unit 26 calculates coordinate transformation coefficients based on the information acquired in the process of step St100 (step St101). The coordinate transformation coefficient calculation unit 26 outputs the calculated coordinate transformation coefficients to the coordinate transformation unit 30.

[0138] The white line detection unit 27 acquires the captured image temporarily stored in the memory 22 (step St102).

[0139] The white line detection unit 27 detects white lines, which are lane boundaries, from the captured image acquired in the process of step St102 (step St103). The white line detection unit 27 outputs the detection result of the process of step St103 to the lane determination unit 32.

[0140] The processor 24 executes an image processing area setting process (step St105). The image processing area setting process is a process for determining lane boundary lines and determining the lanes of the road shown in the captured image, and will be described in detail with reference to FIG.

[0141] The vehicle detection unit 28 acquires the captured image from the memory 22 (step St106).

[0142] The vehicle detection unit 28 detects a vehicle (step St107). The vehicle detection unit 28 outputs the result of detecting the vehicle to the vehicle tracking unit 29.

[0143] The vehicle tracking unit 29 tracks the vehicle using the result of the process in step St107 (step St108). The vehicle tracking unit 29 outputs the result of tracking the vehicle to the coordinate conversion unit 30.

[0144] The coordinate conversion unit 30 converts the result of the process in step St108 by using the coordinate conversion coefficient calculated by the coordinate conversion coefficient calculation unit 26 in the process in step St101, and calculates the size and position of the vehicle (step St109). The coordinate conversion unit 30 outputs the result of the process in step St109 to the traffic volume measurement unit 34 and the event determination unit 35.

[0145] The traffic volume measurement unit 34 measures the traffic volume based on the information received from the coordinate conversion unit 30 (step St110).

[0146] The event determination unit 35 determines an event occurring on the road based on the information received from the coordinate conversion unit 30 (step St111). An event is, for example, a vehicle traveling at a low speed, a rear-end collision occurring, or a traffic jam occurring.

[0147] The operation determination unit 25 executes a determination of an operation (for example, a PTZ operation) that changes the imaging range of the camera 10 from the information of the captured image acquired in the process of step St100 (step St112).

[0148] The operation determination unit 25 determines whether or not the camera 10 has operated (step St113).

[0149] When it is determined that the camera 10 is not operating (step St113, NO), the operation determination unit 25 causes the vehicle detection unit 28 to execute the process of step St106 again.

[0150] When it is determined that the camera 10 has moved (step St113, YES), the movement determination unit 25 causes the coordinate transformation coefficient calculation unit 26 to execute the process of step St100 again.

[0151] Next, an example of the image processing area setting process will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the image processing area setting process.

[0152] When the white line detection unit 27 detects a white line (step St201, YES), the lane determination unit 32 classifies the captured image into N categories using the results of the white line detection by the white line detection unit 27 and sets road division regions (step St202). After the process of step St202, the processor 24 advances the process to step St203. On the other hand, when the white line detection unit 27 does not detect a white line (step St201, NO), the processor 24 advances the process to step St203.

[0153] The vehicle detection unit 28 acquires the captured image from the memory 22 (step St203).

[0154] The vehicle detection unit 28 detects vehicles from the captured image acquired in the process of step St203 (step St204). The vehicle detection unit 28 outputs the result of the process of step St204 to the vehicle tracking unit 29.

[0155] The vehicle tracking unit 29 tracks the vehicle using the result of the process in step St204 (step St205). The vehicle tracking unit 29 outputs the result of tracking the vehicle to the coordinate conversion unit 30.

[0156] The coordinate conversion unit 30 converts the result of the process in step St205 into coordinates to calculate the size and position of the vehicle (step St206). The coordinate conversion unit 30 outputs the result of the process in step St206 to the vehicle trajectory accumulation unit 33.

[0157] The vehicle trajectory storage unit 33 stores the vehicle trajectory using the result of the process of step St206 (step St207). The vehicle trajectory storage unit 33 outputs the result of the process of step St207 to the lane determination unit 32.

[0158] The lane determining unit 32 determines whether M vehicle trajectories have been accumulated for the entire captured image or for each road division area (step St208).

[0159] When determining that M vehicle trajectories have not been accumulated (step St208, NO), the lane determination unit 32 causes the vehicle detection unit 28 to execute the process of step St203 again.

[0160] When it is determined that M vehicle trajectories have been accumulated (step St208, YES), the lane determination unit 32 determines whether or not the trajectory difference of the vehicle trajectory group is equal to or greater than L [m] (step St209).

[0161] When the lane determining unit 32 determines that the trajectory difference between the vehicle trajectory groups is equal to or greater than L [m] (step St209, YES), it sets a road division area (step St210). The method of setting the road division area in the process of step St210 has been described with reference to FIG. 4 and the like, and therefore will not be described here.

[0162] When the lane determining unit 32 determines that the trajectory difference of the vehicle trajectory group is not equal to or greater than L [m], that is, the trajectory difference of the vehicle trajectory group is less than L [m] (step St209, NO), the lane determining unit 32 determines that the entire captured image or the road division area includes one lane (step St211). The lane determining unit 32 outputs the processing result of step St211 to the image processing area setting unit 31.

[0163] The image processing region setting unit 31 sets lane boundary lines using the results of the process in step St211 (step St212). The image processing region setting unit 31 determines the lanes based on the lane boundary lines set in the process in step St212.

[0164] The image processing region setting unit 31 determines whether or not there is a lane adjacent to the determined lane (step St213).

[0165] When the image processing region setting unit 31 determines that there is a lane adjacent to the determined lane (step St213, YES), it corrects the lane boundary line (step St214). After the process of step St214, the image processing region setting unit 31 proceeds to the process of step St215.

[0166] When it is determined that there is no lane adjacent to the determined lane (step St213, NO) or when the lane boundary line is corrected (step St214), the image processing area setting unit 31 sets a processing area of ​​the captured image (hereinafter referred to as "image processing area") (step St215). The image processing area is the area of ​​the lane through which the vehicle passes.

[0167] As described above, the lane detection system 1 according to this embodiment can reset the position of the lane shown in the captured image when the imaging range of the camera 10 changes. In other words, the lane detection system 1 can set the position of the lane each time the imaging range of the camera 10 changes as the camera 10 operates, so that it can continuously measure the number of vehicles traveling on the road and determine events that occur on the road, such as vehicles stopping, going slowly, getting stuck, avoiding, or driving in the wrong direction, without stopping. In addition, the lane detection system 1 can prevent erroneous measurement of the number of vehicles per lane due to a change in the imaging range of the camera 10 operating.

[0168] In addition, the lane detection system 1 divides the captured image into multiple regions using the detected white lines or vehicle trajectories and determines the lanes for each region, thereby preventing erroneous determinations such as errors in lane setting or the wrong number of lanes, and can detect the lanes on the road with high accuracy.

[0169] Summary of the Disclosure The above description of the embodiments discloses the following techniques.

[0170] <Technology 1> The lane detection method of this embodiment includes an image acquisition step of acquiring an image from a camera, a vehicle detection step of detecting a vehicle from the captured image, a coordinate conversion step of converting the position of the vehicle into road coordinates, a trajectory calculation step of calculating a vehicle trajectory from position information of the road coordinates, an accumulation step of accumulating vehicle trajectories for M vehicles (M: an integer of 1 or more), a trajectory difference calculation step of calculating a trajectory difference between a lane determination line set in the captured image that intersects with the vehicle trajectory and a maximum and minimum value in road coordinates of the intersection with the vehicle trajectory, and a determination step of accumulating vehicle trajectories for M vehicles for each road division area that divides an area where different lanes exist, determining whether the trajectory difference of the accumulated vehicle trajectories for M vehicles is less than length L, and if it is determined that the trajectory difference is less than length L, determining that the road division area includes one lane.

[0171] As a result, the lane detection method according to the present embodiment can detect road lanes with high accuracy by dividing areas of a captured image using vehicle trajectories and determining lanes for each area.

[0172] <Technology 2> The lane detection method described in Technology 1 further includes a division step of determining whether the trajectory difference is equal to or greater than a length L, and if it is determined that the trajectory difference is equal to or greater than the length L, setting a road division area that divides an area in which different lanes exist.

[0173] As a result, the lane detection method according to the present embodiment can detect road lanes with high accuracy by dividing areas of a captured image using vehicle trajectories and determining lanes for each area.

[0174] <Technology 3> In the lane detection method described in Technology 1 or Technology 2, if it is determined in the determination step that the trajectory difference is not less than the length L, a road division area is set by dividing the road division area in which the trajectory difference is not less than the length L into two.

[0175] As a result, in the lane detection method according to the present embodiment, when one road division area includes two lanes, the road division area can be further divided by using the vehicle trajectory.

[0176] <Technology 4> In the lane detection method according to any one of the techniques 1 to 3, road division areas are set until it is determined that the trajectory differences of all road division areas are less than the length L.

[0177] As a result, the lane detection method according to the present embodiment can prevent the number of lanes from being set erroneously, and can detect the lanes of a road with high accuracy.

[0178] <Technology 5> In the lane detection method according to any one of the first to fourth techniques, when a white line is detected from a captured image, the white line is used to set a road division area.

[0179] As a result, the lane detection method according to the present embodiment can set road division areas based on the detection results of the white lines and determine the lanes.

[0180] <Technology 6> In the lane detection method described in any one of Techniques 1 to 5, the speed and number of vehicles traveling in the determined lane are measured using information on the size and position of the vehicle converted from image coordinates to road coordinates, and the traffic volume is calculated.

[0181] As a result, the lane detection method according to the present embodiment can continue to calculate the traffic volume even if the imaging range of the camera changes by calculating the traffic volume using the determined lane.

[0182] <Technology 7> In the lane detection method described in any one of Techniques 1 to 6, events such as a vehicle being stopped, a vehicle traveling at a low speed, a traffic jam occurring, a vehicle avoiding a traffic jam, or a vehicle traveling in the wrong direction are determined based on the speed and number of vehicles.

[0183] As a result, the lane detection method according to the present embodiment can continuously determine events occurring on the road using the determined lane.

[0184] <Technology 8> In the lane detection method according to any one of Techniques 1 to 7, a camera is operated, and it is determined whether or not an imaging direction or an imaging range of the camera has changed. If it is determined that an imaging direction or an imaging range of the camera has changed, an image acquisition step, a vehicle detection step, a coordinate conversion step, a trajectory calculation step, an accumulation step, a trajectory difference calculation step, a division step, and a determination step are executed.

[0185] As a result, the lane detection method according to the present embodiment can execute a process for determining lanes when the camera operates and changes the imaging direction or imaging range of the camera. As a result, the lane detection method can continuously measure the number of vehicles or determine events occurring on the road without stopping by determining the lanes every time the camera operates. In addition, the lane detection method can prevent erroneous measurement of the number of vehicles due to changes in the imaging range caused by the camera operating, and can accurately measure the number of vehicles.

[0186] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0187] The technology disclosed herein is useful as a lane detection method and an image processing device for detecting road lanes with high accuracy. [Explanation of symbols]

[0188] 1 Lane Detection System 10. Camera 20 Image processing device 21 Communication I / F 22 Memory 23 Input Devices 24 processors 25 Operation judgment section 26 Coordinate conversion coefficient calculation section 27 White line detection unit 28 Vehicle detection unit 29 Vehicle Tracking Division 30 Coordinate conversion section 31 Image processing area setting section 32 Lane Judging Section 33 Vehicle trajectory storage unit 34 Traffic Volume Measurement Section 35 Event Judgment Section

Claims

1. an image acquisition step of acquiring a captured image from a camera; a vehicle detection step of detecting a vehicle from the captured image; a coordinate conversion step of converting the position of the vehicle into road coordinates; a trajectory calculation step of calculating a vehicle trajectory from the position information of the road coordinates; a storage step of storing the vehicle trajectories for M vehicles (M: an integer of 1 or more); a trajectory difference calculation step of calculating a trajectory difference between a maximum value and a minimum value in road coordinates of an intersection between a lane determination line set in the captured image so as to intersect with the vehicle trajectory and the vehicle trajectory; a determination step of accumulating the vehicle trajectories for M vehicles for each road division region obtained by dividing an area in which different lanes exist, determining whether or not the trajectory difference of the accumulated vehicle trajectories for M vehicles is less than a length L, and determining that the road division region includes one lane if it is determined that the trajectory difference is less than the length L. Lane detection methods.

2. a dividing step of determining whether the trajectory difference is equal to or greater than a length L, and if it is determined that the trajectory difference is equal to or greater than the length L, setting the road division area that divides an area in which different lanes exist; The lane detection method according to claim 1 .

3. When it is determined in the determining step that the trajectory difference is not less than the length L, a road division area is set by dividing the road division area in which the trajectory difference is not less than the length L into two. The lane detection method according to claim 1 .

4. The road division areas are set until it is determined that the trajectory differences of all the road division areas are less than the length L. The lane detection method according to claim 3 .

5. when a white line is detected from the captured image, the road division area is set using the white line. The lane detection method according to claim 1 .

6. Using the information on the size and position of the vehicle converted from the image coordinates to the road coordinates, the speed and number of vehicles traveling on the determined lane are measured, and traffic volume is calculated. The lane detection method according to claim 1 .

7. From the speed and the number of the vehicles, it is determined whether the vehicle is stopped, the vehicle is traveling at a low speed, a traffic jam is occurring, the vehicle is avoiding the traffic, or the vehicle is traveling in the wrong direction. The lane detection method according to claim 6.

8. the camera is operated, and it is determined whether or not an imaging direction or an imaging range of the camera has changed, and when it is determined that an imaging direction or an imaging range of the camera has changed, the image acquisition step, the vehicle detection step, the coordinate conversion step, the trajectory calculation step, the accumulation step, the trajectory difference calculation step, the division step, and the determination step are executed; The lane detection method according to claim 2 .

9. a vehicle detection unit that acquires an image captured by the camera and detects a vehicle from the image captured; a coordinate conversion unit that converts the position of the vehicle into road coordinates; a vehicle storage unit that calculates a vehicle trajectory from the position information of the road coordinates and stores the vehicle trajectories for M vehicles; a lane determination unit that calculates a trajectory difference between a maximum value and a minimum value in road coordinates of an intersection between a lane determination line set in the captured image that intersects with a vehicle trajectory dividing an area where different lanes exist and the vehicle trajectory, accumulates the vehicle trajectories for M vehicles for each road division area, determines whether the trajectory difference of the accumulated vehicle trajectories for M vehicles is less than a length L, and determines that the road division area includes one lane if it is determined that the trajectory difference is less than the length L. Image processing device.

10. A method for detecting a lane according to claim 1, Lane detection program.

Images