Detection device, detection method, detection program and recording medium

Abstract

To perform calibration corresponding to a change even if a distance between an imaging part and a road face is changed due to a change in the weight of a moving body and a change in the pneumatic pressure of wheels.SOLUTION: A moving amount detection part 321a detects a pixel moving amount on the basis of a captured imaged image which is captured by an imaging part 210 with respect to a stop line being a feature part on a road face having a well-known length along a traveling direction of a moving body MV. A specification part 322A specifies a white line region in the captured image. Then, when it is determined that the moving body MV linearly travels on the flat road face at a constant velocity, the calibration part 323A calibrates a distance on the road face corresponding to one pixel related to a distance between the imaging part 210 and the road face, on the basis of the well-known length along the traveling direction of the moving body MV on the stop line, in addition to the pixel moving amount and a specified result of the white line region in the captured image.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a detection device, a detection method, a detection program, and a recording medium on which the detection program is recorded. [Background technology]

[0002] The slip ratio λ, which is a normalized difference between the wheel speed of a moving body such as a vehicle and the moving body speed, and the friction coefficient μ, which is a normalized grip force between the wheel and the road surface, have the relationship (hereinafter referred to as "μ-λ characteristics") shown in Figures 1 and 2. Here, Figure 1 shows the μ-λ characteristics during driving, and Figure 2 shows the relationship between the μ-λ characteristics during braking.

[0003] In Figures 1 and 2, the μ-λ characteristics on a dry road surface are shown by a solid line, the μ-λ characteristics on a wet road surface are shown by a dashed line, and the μ-λ characteristics on a frozen road surface are shown by a dashed double-dot line.

[0004] In the change in the friction coefficient μ with the increase in the slip ratio λ during driving shown in Figure 1, the state where the slip ratio is smaller than the maximum friction coefficient μ is the stable region where the moving body can run stably. On the other hand, when the slip ratio is larger than the maximum friction coefficient μ, the grip force decreases and in the worst case, it becomes an unstable region where slipping or locking occurs.

[0005] In the change in the friction coefficient μ with the increase in the slip ratio λ during braking shown in Figure 2, the state where the slip ratio is larger than the slip ratio where the friction coefficient μ is the minimum is a stable region. On the other hand, the state where the slip ratio is smaller than the slip ratio where the friction coefficient μ is the minimum is an unstable region.

[0006] If the slip ratio λ is controlled to be within the range below which the absolute value of the friction coefficient μ becomes maximum, the moving body can maintain stable running. On the other hand, if the unstable state continues and the tires spin or lock, the driving, braking, and steering of the vehicle cannot be controlled.

[0007] For this reason, in order to avoid the risk of accidents, internal combustion engine vehicles are equipped with ABS (Antilock Brake System) and other systems that mainly use brake hydraulic control and engine control. Such ABS systems determine the slip state and control the engine drive torque and brake hydraulic braking torque to bring the vehicle closer to a stable region. Also, for electric vehicles, anti-slip control has been proposed that estimates the slip ratio λ and appropriately controls the motor torque to maintain the vehicle within the stable region. Thus, detecting the slip ratio λ is extremely important in understanding the driving state.

[0008] Now, in order to calculate the slip ratio λ using the following equation (1), the wheel radius r, the rotational angular velocity ω, and the moving body velocity v are required. λ=(r·ω-v) / Max(r·ω,v) …(1)

[0009] Here, Max(r·ω,v) indicates the larger of (r·ω) and v. During driving, (r·ω) is greater than v, so Max(r·ω,v)=r·ω. On the other hand, during braking, v is greater than (r·ω), so Max(r·ω,v)=v.

[0010] For example, if the moving object is an automobile, the wheel radius r is considered to be constant if the air pressure in the wheels is sufficient. Also, the rotational angular velocity ω can be detected from the pulse output of an encoder mounted on the wheel, or, in the case of an electric vehicle, from the signal output of a resolver connected to the motor.

[0011] On the other hand, the following methods (a) to (c) are generally used to detect the moving object speed v. (a) Calculated from the rotational speed of the non-driven wheels (b) Calculate by integrating the acceleration value detected by the acceleration sensor (c) Calculation based on the results of detection by an optical sensor

[0012] Here, in the method (a), the brakes are applied to all the wheels, so it is not possible to detect the moving object speed v during braking. Also, in the case of a four-wheel drive vehicle, there are no non-driven wheels, so it is not possible to detect the moving object speed v.

[0013] In addition, in the method (b), since the output of the acceleration sensor is integrated, the offset present in the output of the acceleration sensor is accumulated, and as a result, the moving object velocity v cannot be detected accurately.

[0014] Method (c), which does not fundamentally have the drawbacks of methods (a) and (b), is expected. The technology described in Patent Document 1 has been proposed as a technology that employs method (c) (hereinafter, referred to as the "conventional example"). In this conventional technology, the vehicle's running or stopped state is determined from an image signal that captures a scene in the vicinity of the vehicle. Also, the vehicle speed (i.e., moving body speed) of the vehicle is detected based on the dimensions of the road markings in the image and the dimensions of the road markings on the road. Note that, in the conventional technology, the vehicle speed is detected only when a road marking is present in the image, and only the running / stopped state is determined when no road marking is present in the image. [Prior art documents] [Patent documents]

[0015] [Patent Document 1] JP 2009-205642 A Summary of the Invention [Problem to be solved by the invention]

[0016] In the above-mentioned conventional technology, if there is no road marking in the captured image, the vehicle speed is not detected. As a result, even if the vehicle speed changes while there is no road marking in the captured image, the vehicle speed detected at the last time when there was a road marking in the captured image is estimated as the current vehicle speed. For this reason, it is difficult to say that the vehicle speed at each point in time can be detected with high accuracy.

[0017] In addition, in the conventional technology, the imaging device is attached to the vehicle body, so the distance between the imaging device and the road surface changes depending on the number of passengers, the amount of luggage loaded, and the tire air pressure. As a result, the imaging magnification changes, and the size of the road markings in the captured image also changes.

[0018] 3(A), for example, imaging unit 210 includes imaging lens system 211 (focal length: f) and square imaging surface 212 (length of one side: D), and the distance from imaging lens system 211 to imaging surface 212 is "d." In this case, when imaging a road surface LD directly below, if the distance from imaging lens system 211 to road surface LD (hereinafter also referred to as "distance from imaging unit 210 to road surface LD") is "h," the optical magnification m of imaging by imaging unit 210 is expressed by the following formula (2). m = 1 / ((h / f)-1) … (2) The pixel configuration on the imaging surface 212 is NU×NU (for example, 30×30).

[0019] As a result, the length H of one side of the square area to be imaged on the road surface LD is given by the following equation (3). H=D / m=D·((h / f)-1) …(3)

[0020] For example, if f = 8 [mm], D = 1.7 [mm], and h = 495 [mm], then H = 103.5 [mm].

[0021] Now, as shown in Figure 3(B), the weight of the car becomes heavier than in Figure 3(A), and the distance h becomes shorter. * In this case, the optical magnification of the image captured by the image capture unit 210 is changed from "m" in the state of FIG. 3(A) to "m * " changes to m * =1 / ((h * / f)-1) …(4)

[0022] As a result, the length of one side of the square area to be imaged changes from "H" in the state of FIG. 3(A) to "H * " changes to H * =D / m * =D·((h * / f)-1) …(5)

[0023] For example, h * = 445 [mm], then H * = 92.9 [mm], which is a change of about 10% from H = 103.5 [mm] in the state of Figure 3 (A) described above. As a result, the size of the area on the road surface corresponding to the size of one pixel also changes by about 10% from the state of Figure 3 (A) described above.

[0024] However, when the movement distance of an image is obtained based on the correlation between captured images per unit time and the vehicle speed in the longitudinal and lateral directions is calculated, if the size of the area on the road surface corresponding to the size of one pixel in the captured image changes, the calculated vehicle speed also changes. In other words, if the size of the area on the road surface corresponding to the size of one pixel changes by about 10%, the calculated speed value also changes by about 10%, resulting in an error.

[0025] The effect of dynamic errors due to road surface variations can be reduced by averaging processing, but changes in vehicle weight and tire air pressure cause offsets, so the effect of errors cannot be suppressed by averaging processing.

[0026] For this reason, there is a demand for a technology that can detect vehicle speed with high accuracy even when changes in vehicle weight or wheel air pressure occur. Meeting such a demand is one of the problems that the present invention is to solve.

[0027] The present invention has been made in consideration of the above, and aims to provide a detection device and a detection method that can perform calibration in response to changes in the distance between the road surface and the imaging unit, even if the distance changes due to changes in vehicle weight or wheel air pressure. [Means for solving the problem]

[0028] The invention described in claim 1 is a detection device characterized by comprising: an acquisition unit that acquires multiple images including a road surface feature having a known length, imaged along the direction of travel at a predetermined time interval by an imaging unit mounted on a moving body; and an output unit that calculates the number of image areas of a predetermined width corresponding to the known length based on the multiple images acquired by the acquisition unit, and outputs distance information related to the distance on the road surface corresponding to the predetermined width based on the calculated number and the known length. The invention described in claim 4 is a detection device characterized by comprising: an acquisition unit that acquires multiple images including road surface features imaged along the direction of travel at predetermined time intervals by an imaging unit mounted on a moving body; a first output unit that calibrates information indicating the distance on the road surface corresponding to the width of a predetermined image area in the image obtained based on the multiple images acquired by the acquisition unit based on distance information output in the past, and outputs the calibration result as new distance information; and a second output unit that outputs the speed of the moving body using the new distance information.

[0029] The invention described in claim 5 is a detection method used in a detection device having an acquisition unit and an output unit, characterized in that the detection method comprises: an acquisition step in which the acquisition unit acquires a plurality of images including a road surface feature having a known length, imaged along the direction of travel at predetermined time intervals by an imaging unit mounted on a moving body; and a distance information output step in which the output unit calculates the number of image areas of a predetermined width corresponding to the known length based on the plurality of images acquired in the acquisition step, and outputs distance information related to the distance on the road surface corresponding to the predetermined width based on the calculated number and the known length.

[0030] The invention described in claim 6 is a detection program for causing a computer included in a detection device to execute the detection method described in claim 5.

[0031] The invention described in claim 7 is a recording medium having the detection program described in claim 6 recorded thereon so as to be readable by a computer included in a detection device. [Brief description of the drawings]

[0032] [Figure 1] FIG. 11 is a diagram showing the relationship between the slip ratio and the friction coefficient during driving. [Diagram 2] FIG. 11 is a diagram showing the relationship between the slip ratio and the friction coefficient during braking. [Diagram 3] 11 is a diagram for explaining a change in a speed calculation value due to a change in vehicle weight, etc.; FIG. [Figure 4] 1 is a diagram showing a configuration of a detection device according to a first embodiment of the present invention. [Diagram 5] 5 is a diagram for explaining the operation of the movement amount detection unit in FIG. 4. [Figure 6] FIG. 2 is a diagram for explaining a calibration environment in the first embodiment. [Figure 7] 5 is a diagram for explaining the operation of the determination unit in FIG. 4. [Figure 8] 5 is a diagram for explaining a calibration operation by the calibration unit in FIG. 4. [Figure 9] FIG. 4 is a diagram showing the configuration of a detection device according to a second embodiment of the present invention. [Figure 10] 10 is a diagram for explaining the relative positions of the two imaging units in FIG. 9. FIG. [Figure 11] FIG. 11 is a diagram for explaining a calibration environment in a second embodiment. [Figure 12] 10 is a diagram for explaining the operation of the determination unit in FIG. 9. [Figure 13] 10 is a diagram for explaining a calibration operation by the calibration unit in FIG. 9. [Figure 14] 1 is a diagram showing a configuration of a detection device according to a first embodiment of the present invention. [Figure 15] 15 is a flowchart for explaining a pixel movement amount detection process executed by the control unit of FIG. 14. [Figure 16]15 is a flowchart for explaining a vehicle speed output process executed by the control unit of FIG. 14. [Figure 17] 15 is a flowchart for explaining a process for identifying a white line area executed by the control unit of FIG. 14. [Figure 18] 15 is a flowchart for explaining a monitoring process of a calibration condition executed by the control unit of FIG. 14. [Figure 19] 15 is a flowchart for explaining a pixel distance calibration process executed by the control unit of FIG. 14. [Figure 20] FIG. 11 is a diagram showing the configuration of a detection device according to a second embodiment of the present invention. [Figure 21] 21 is a flowchart for explaining a pixel movement amount detection process executed by the control unit of FIG. 20. [Figure 22] 21 is a flowchart for explaining a process for identifying a white line area executed by the control unit of FIG. 20. [Diagram 23] 21 is a flowchart illustrating a pixel distance calibration process executed by the control unit of FIG. 20. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Hereinafter, an embodiment of the present invention will be described with reference to Figures 4 to 13. In the following description and drawings, the same or equivalent elements are given the same reference numerals, and duplicated description will be omitted.

[0034] [First embodiment] First, a first embodiment of the present invention will be described with reference to FIGS.

[0035] <Configuration> A configuration of a detection device 300A according to the first embodiment is shown in a block diagram in Fig. 4. As shown in Fig. 4, the detection device 300A is connected to an imaging unit 210, a navigation device 220, and an ECU (Electronic Control Unit) 230. The detection device 300A, the imaging unit 210, the navigation device 220, and the ECU 230 are mounted on a moving object MV.

[0036] The imaging unit 210 is mounted at a fixed position on the moving body MV, and captures an image of the road surface directly below the fixed position, as described above with reference to Fig. 3. The imaging unit 210 periodically captures an image of the road surface at a periodic time TP. Data of the image captured in this manner (hereinafter referred to as "captured image data") is sent to the detection device 300A.

[0037] The "periodic time TP" is determined in advance based on experiments, simulations, etc., from the viewpoint of ensuring that two images captured separated by the periodic time TP contain a common area on the road surface, even if the moving body MV is traveling at high speed.

[0038] The navigation device 220 provides the user with driving operation support for the moving body MV based on map information and current position information. When the moving body MV is within a predetermined distance from an intersection in the driving direction, the navigation device 220 notifies the detection device 300A of this fact.

[0039] The "predetermined distance" is determined in advance based on experiments, simulations, experience, etc., from the standpoint of providing adequate advance notice for imaging from the start to the end along the direction of travel regarding the stop line before an intersection, which is adopted as a road surface feature in the first embodiment.

[0040] The ECU 230 controls the traveling of the moving body MV and provides traveling information to the user based on sensor detection information acquired from various sensors such as a wheel rotation speed sensor, an acceleration sensor, a steering angle sensor, and an inclination sensor. The ECU 230 transmits the acceleration information, steering angle information, and inclination information to the detection device 300A.

[0041] The ECU 230 further acquires the moving body speed (traveling speed information) sent from the detection device 300A, and uses the acquired moving body speed for travel control of the moving body MV.

[0042] The detection device 300A includes an acquisition section 310A and a first output section 320A as shown in FIG.

[0043] The acquiring section 310A described above receives captured image data sent from the imaging section 210. Then, the acquiring section 310A sends the captured image data to the first output section 320A.

[0044] The first output unit 320A receives the captured image data sent from the acquisition unit 310A. The first output unit 320A detects the number of pixel movements (hereinafter referred to as "pixel movements") at the same road surface position between two images captured with a periodic time TP in between, and calibrates the distance on the road surface corresponding to one pixel (hereinafter referred to as "pixel distance") when a calibration condition described later is satisfied. The detected pixel movements and the calibrated pixel distance are output to the second output unit 330. The configuration of the first output unit 320A having such a function will be described in detail later.

[0045] The second output unit 330 calculates the moving object speed as the traveling information of the moving object MV based on the latest calibrated pixel distance and the periodic time TP every time the second output unit 330 receives the pixel movement amount sent from the first output unit 320A. Then, the second output unit 330 outputs the calculated moving object speed to the ECU 230.

[0046] Configuration of first output section 320A Next, the configuration of the above-mentioned first output section 320A will be described.

[0047] 4, the first output section 320A includes a movement amount detection section 321A and an identification section 322A. The first output section 320A also includes a calibration section 323A.

[0048] The movement amount detection unit 321A receives the captured image data sent from the acquisition unit 310A. Then, the movement amount detection unit 321A detects the pixel movement amount based on the captured image data by a so-called displacement amount search method. The pixel movement amount detected in this way is sent to the calibration unit 323A and the second output unit 330.

[0049] The operation of the movement amount detection unit 321A will be described later in detail.

[0050] The above-mentioned identification unit 322A receives the captured image data sent from the acquisition unit 310A. Then, the identification unit 322A identifies the white line area in the image obtained from the captured image data. Information on the white line area thus identified (hereinafter, referred to as "white line area information") is sent to the calibration unit 323A.

[0051] The operation of the specification unit 322A will be described later in detail.

[0052] The calibration unit 323A receives the pixel movement amount sent from the movement amount detection unit 321A and the white line area information sent from the identification unit 322A. When the calibration unit 323A determines that the moving object MV is traveling straight ahead at a constant speed on a flat road surface based on the acceleration information, steering angle information, and tilt information sent from the ECU 230, the calibration unit 323A calibrates the pixel distance based on the pixel movement amount, the white line area information, and the fact that the length of the stop line drawn on the road in the travel direction is approximately 45 [cm]. The pixel distance calibrated in this way is sent to the second output unit 330.

[0053] The operation of the calibration unit 323A will be described in detail later.

[0054] <Operation> Next, the operation of the detection device 300A calibrated as described above will be described, focusing on the processing performed by each element of the detection device 300A.

[0055] It is assumed that the imaging unit 210 has already started operation and is sequentially sending image data of road surface images captured at the periodic time TP to the detection device 300A. The navigation device 220 has already started operation and is configured to send a notification to the detection device 300A when the moving object MV is within a predetermined distance from an intersection in the traveling direction. The ECU 230 has already started operation and is sending acceleration information, steering angle information, and tilt information to the detection device 300A (see FIG. 4).

[0056] Furthermore, in the detection device 300A, it is assumed that the pixel distance has already been calibrated a number of times, and the calibration unit 323A holds a calibration history including the calibration times of a predetermined number of recent calibrations and a provisional pixel distance, which will be described later. It is also assumed that the second output unit 330 holds the most recently calibrated pixel distance. Here, during the period until the first calibration of the pixel distance is performed, the second output unit 330 holds an average pixel distance.

[0057] The "predetermined number of times" is determined in advance based on experiments, simulations, experience, etc., from the viewpoint of performing averaging processing for suppressing the influence of fluctuations in the length of the stop line in the traveling direction, etc., on the calibration result of the pixel distance. Also, the "average pixel distance" is determined in advance corresponding to the moving body MV, based on experiments, simulations, experience, etc.

[0058] <<Image data acquisition process>> In the detection device 300A, the acquisition section 310A receives the captured image data sent from the imaging section 210. Then, the acquisition section 310A sends the captured image data to the movement amount detection section 321A and the identification section 322A of the first output section 320A (see FIG. 4).

[0059] Detecting pixel movement amount Next, a process of detecting the pixel movement amount by the movement amount detection unit 321A will be described.

[0060] Upon receiving the captured image data sent from the acquisition unit 310A, the movement amount detection unit 321A detects the amount of displacement of the position of a common feature area between a current image obtained from the current captured image data and a previous image obtained from the previous captured image data as a pixel movement amount. Then, the movement amount detection unit 321A sends the detected pixel movement amount to the calibration unit 323A and the second output unit 330 (see FIG. 4).

[0061] When there are multiple common feature regions, the movement amount detection section 321A uses the average value of the amounts of displacement of the multiple feature regions as the amount of pixel movement.

[0062] 5 shows an example of movement amount detection by the movement amount detection unit 321A. In the example of FIG. 5, a moving object MV moves straight along the X direction, and at time T j The X-axis position at j At time T j+1 (=T j +TP) X-direction position is X j+1 In addition, the example of FIG. 5 is an example of the case where the time T j Image captured at time T j+1 In the example shown in FIG. 5, the common feature regions in the captured image in the first embodiment are feature region A and feature region B. In the example shown in FIG. 5, the displacement amount of feature region A is “ΔX jA " and the displacement of feature area B is "ΔX jB " Therefore, the pixel movement amount ΔX j is calculated using the following formula (6). ΔX j =(ΔX jA +ΔX jB ) / 2 …(6)

[0063] 《Output process of moving object speed v》 Next, the output process of the moving object velocity v by the second output unit 330 will be described.

[0064] Upon receiving the pixel movement amount sent from the movement amount detection unit 321A, the second output unit 330 calculates the moving object speed v based on the pixel movement amount, the held pixel distance (i.e., the most recently calibrated pixel distance) and the period time TP. Then, the second output unit 330 outputs the calculated moving object speed v to the ECU 230.

[0065] Here, when the pixel movement amount is "PN" and the pixel distance is "PD", the second output unit 330 calculates the moving object speed v by the following equation (7). v = PN PD / TP … (7)

[0066] <<White line area identification process>> Next, the process of identifying a white line area by the identifying unit 322A will be described.

[0067] Such a process of specifying the white line area is performed for the purpose of calibrating the pixel distance by the calibration unit 323A, which will be described later. The environment for calibrating the pixel distance (hereinafter referred to as the "calibration environment") is shown in Fig. 6. As shown in Fig. 6, the pixel distance is calibrated when a moving object MV, traveling at a constant speed in a straight line on a flat road surface, crosses a stop line SPL, which is a white line on the road surface with a length W (≈45 [cm]) in the traveling direction, at a right angle to the longitudinal direction of the stop line SPL.

[0068] In the following description, it is assumed that the stop line SPL used for calibrating the new pixel distance is a white line area extending from a driving direction position (that is, an X-direction position) XR to a driving direction position XP.

[0069] Now, upon receiving the captured image data sent from the acquisition unit 310A, the identification unit 322A identifies the white line area in the image obtained from the captured image data based on the brightness of each pixel in the image, and sends the identification result to the calibration unit 323A as white line area information (see FIG. 4). The identification result of such white line area includes two types of white line areas: a front partial white line shown in FIG. 7(A) and a rear partial white line shown in FIG. 7(B).

[0070] When the identification result indicates a partial white line in the front, the identification unit 322A identifies the number of pixels a in the X-direction length of the white line area in the image (hereinafter also referred to as the "front length a"). When the identification result indicates a partial white line in the rear, the identification unit 322A identifies the number of pixels b in the X-direction length of the non-white line area in the image (hereinafter also referred to as the "front length b").

[0071] In the case of a partial white line in the front, the specifying unit 322A sends the white line area information shown in Fig. 7(A) [white line flag: ON, front length: a] to the calibration unit 323A (see Fig. 4). In the case of a partial white line in the rear, the specifying unit 322A sends the white line area information shown in Fig. 7(B) [white line flag: ON, front length: b] to the calibration unit 323A (see Fig. 4).

[0072] Pixel Distance Calibration Process Next, the calibration process by calibration unit 323A will be described.

[0073] In the calibration process, the calibration unit 323A performs an intersection determination as to whether or not it has received information from the navigation device 220 that the moving object MV is within a predetermined distance from an intersection in the traveling direction. If the result of this intersection determination is positive, the calibration unit 323A performs a constant-speed straight-line determination as to whether or not it can be determined that the moving object MV is traveling straight at a constant speed on a flat road surface, based on the acceleration information, steering angle information, and tilt information sent from the ECU 230.

[0074] If the result of the constant speed straight-line determination is positive, the calibration unit 323A performs a white line start determination as to whether or not the white line area information sent from the determination unit 322A has become a "partial white line ahead." If the result of this white line start determination is positive, the calibration unit 323A collects pixel movement amounts sent from the movement amount detection unit 321A until the result of the white line end determination as to whether or not the white line area information sent from the determination unit 322A has become a "partial white line behind" becomes positive, on condition that the state in which the result of the constant speed straight-line determination is maintained as being positive.

[0075] Then, when the result of the white line end determination becomes positive, the calibration unit 323A calculates the forward length a included in the white line area information at the time when the "partial white line ahead" occurred, the collected pixel movement amounts ΔX1, ..., ΔX M Based on the forward length b contained in the white line area information at the time when the rear partial white line is detected, the provisional pixel distance PT [cm] is calculated using the following equation (8). PT = 45 / (a + ΔX1 + … + ΔX M -b) …(8)

[0076] Next, the calibration unit 323A calculates a weighted average of the calculated current provisional pixel distance and the internally stored provisional pixel distances calculated during a predetermined number of past calibrations to calculate a currently calibrated pixel distance. After this, the calibration unit 323A internally stores the current provisional pixel distance in place of the oldest of the provisional pixel distances that had been internally stored up until then.

[0077] In the weighted average, the longer the elapsed time, the less weight is given to the item.

[0078] The calibration unit 323A sends the calculated pixel distance to the second output unit 330. As a result, the second output unit 330 calculates the moving object velocity v using the new pixel distance.

[0079] 8 shows an example in which an image of a partial white line ahead is obtained at time T1, and an image of a partial white line behind is obtained at time T5. In the example of FIG. 8, the provisional pixel distance PT [cm] is calculated by the following formula (9). PT=45 / (a+ΔX1+ΔX2+ΔX3+ΔX4-b) …(9)

[0080] As described above, in the first embodiment, the movement amount detection unit 321A detects the pixel movement amount based on the captured image by the imaging unit 210 regarding the stop line, which is a road surface feature having a known length along the traveling direction of the moving object MV. The identification unit 322A also identifies the white line area in the captured image. Then, when it is determined that the moving object MV is traveling straight at a constant speed on a flat road surface based on the pixel movement amount, the identification result of the white line area in the captured image, as well as the known length of the stop line along the traveling direction of the moving object MV, the calibration unit 323A calibrates the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging unit 210 and the road surface.

[0081] Therefore, according to the first embodiment, even if the distance between the road surface and the imaging unit 210 changes due to a change in the weight of the moving object or a change in the air pressure of the wheels, the distance on the road surface corresponding to one pixel can be calibrated in accordance with the change.

[0082] In the first embodiment, the second output unit 330 calculates the moving object speed based on the calibration result and the pixel movement amount. Therefore, it is possible to output the moving object speed with high accuracy as the traveling information.

[0083] In the first embodiment, a stop line is used as a road feature, and the stop line area (i.e., a white line area) can be easily identified by checking the brightness of each pixel in the captured image.

[0084] In addition, in the first embodiment, information that the vehicle is approaching an intersection is acquired, and it is predicted in advance that the vehicle will pass the stop line. This allows efficient calibration.

[0085] Furthermore, in the first embodiment, a weighted average of the current provisional pixel distance and provisional pixel distances calculated during a predetermined number of past calibrations is calculated to obtain a final calibration result. This is because the expected length may deviate from 45 cm due to the stop line being chipped or the paint spilling over. This makes it possible to perform calibration that suppresses the effect of pixel distance on the calibration results, such as fluctuations in the length of the stop line in the direction of travel.

[0086] In the first embodiment, calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and when the road surface is not inclined. This is because when there is lateral movement, the stop line is passed at an angle, and calibration is performed based on a length of 45 cm or more, and when the acceleration or deceleration is large or the road surface is inclined, the suspension etc. expands and contracts, and the optical magnification at the time of imaging is changing. This allows for accurate calibration of pixel distance.

[0087] [Second embodiment] Next, a second embodiment of the present invention will be described with reference mainly to FIGS.

[0088] <Configuration> 9 is a block diagram showing a configuration of a detection device 300B according to the second embodiment. As shown in FIG. 9, the detection device 300B includes an imaging unit 210 F ,210 R and the ECU 230. The detection device 300B, the imaging unit 210 F ,210 R The ECU 230 is mounted on a moving object MV.

[0089] The imaging unit 210 F ,210 R Each of these is configured similarly to the imaging unit 210 described above. F is disposed in front of the moving body MV, and the imaging unit 210 R The imaging unit 210 F The imaging unit 210 is disposed at a distance D behind the moving body MV (see FIG. 10). F ,210 R are configured to capture images at the same time.

[0090] Imaging unit 210 F The data of the image captured by the imaging unit 210 (hereinafter referred to as the "front side captured image") is sent to the detection device 300B. R Data of an image captured by the rear side (hereinafter referred to as a "rear side captured image") is also sent to the detection device 300B.

[0091] As in the first embodiment, the ECU 230 transmits acceleration information, steering angle information, and inclination information to the detection device 300B. The ECU 230 further acquires the moving body speed transmitted from the detection device 300B, and uses the acquired moving body speed for driving control of the moving body MV.

[0092] The detection device 300B described above includes an acquisition section 310B and a first output section 320B as shown in FIG.

[0093] The acquisition unit 310B is the imaging unit 210 F The data of the front side captured image sent from the imaging unit 210 R The acquisition section 310B receives the data of the rear side captured image sent from the first output section 320B. Then, the acquisition section 310B sends the data of the front side captured image and the data of the rear side captured image to the first output section 320B.

[0094] The first output unit 320B receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. The first output unit 320B detects the pixel movement amount and calibrates the pixel distance when the same calibration conditions as in the first embodiment described above are satisfied. The detected pixel movement amount and the calibrated pixel distance are output to the second output unit 330. The configuration of the first output unit 320B having such a function will be described in detail later.

[0095] Each time the second output unit 330 receives the pixel movement amount sent from the first output unit 320B, it calculates the moving object speed as the traveling information of the moving object MV based on the latest calibrated pixel distance and the periodic time TP, in the same manner as in the first embodiment. Then, the second output unit 330 outputs the calculated moving object speed of the moving object MV to the ECU 230.

[0096] Configuration of first output section 320B Next, the configuration of the above-mentioned first output section 320B will be described.

[0097] 9, the first output section 320B includes a movement amount detection section 321B and an identification section 322B. In addition, the first output section 320B includes a calibration section 323B.

[0098] The movement amount detection unit 321B receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the movement amount detection unit 321B detects the pixel movement amount by a so-called displacement amount search method based on the data of the front side captured image and the data of the rear side captured image. The pixel movement amount detected in this way is sent to the calibration unit 323B and the second output unit 330.

[0099] The operation of the movement amount detection unit 321B will be described later in detail.

[0100] The above-mentioned determination unit 322B receives the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B. Then, the determination unit 322B determines the white line area in the image obtained from the data of the front side captured image and the data of the rear side captured image. Information on the white line area determined in this manner (hereinafter referred to as "white line area information") is sent to the calibration unit 323B.

[0101] The operation of the specification unit 322B will be described later in detail.

[0102] The calibration unit 323B receives the pixel movement amount sent from the movement amount detection unit 321B and the white line area information sent from the identification unit 322B. When it is determined that the moving object MV is traveling straight at a constant speed on a flat road surface based on the acceleration information, steering angle information, and inclination information sent from the ECU 230, the calibration unit 323B receives the pixel movement amount and the white line area information from the imaging unit 210 described above. F and the imaging unit 210 R The pixel distance is calibrated based on the distance D between the pixel 101 and the pixel 102. The pixel distance thus calibrated is sent to the second output section 330.

[0103] The operation of the calibration unit 323B will be described in detail later.

[0104] <Operation> Next, the operation of the detection device 300B calibrated as described above will be described, focusing on the processing performed by each element of the detection device 300B.

[0105] In addition, the imaging unit 210 F ,210 R It is assumed that the ECU 230 has already started operating and is sending the data of the front side captured image and the data of the rear side captured image of the road surface image captured at the periodic time TP to the detection device 300B in sequence. It is also assumed that the ECU 230 has already started operating and is sending the acceleration information, the steering angle information, and the inclination information to the detection device 300B (see FIG. 9).

[0106] Furthermore, in the detection device 300B, the pixel distance has already been calibrated a number of times, and the calibration unit 323B holds a calibration history including the calibration times of the most recent predetermined number of calibrations and a provisional pixel distance, which will be described later, and the second output unit 330 holds the most recently calibrated pixel distance. Here, during the period until the first calibration of the pixel distance is performed, the second output unit 330 holds an average pixel distance, as in the case of the first embodiment described above.

[0107] Image data acquisition process In the detection device 300B, the acquisition unit 310B is an imaging unit 210 F ,210 R The acquiring unit 310B receives the data of the front side captured image and the data of the rear side captured image sent from the acquiring unit 310B. Then, the acquiring unit 310B sends the data of the front side captured image and the data of the rear side captured image to the movement amount detecting unit 321B and the specifying unit 322B of the first output unit 320B (see FIG. 9).

[0108] Detecting pixel movement amount Next, a process of detecting the pixel movement amount by the movement amount detection unit 321B will be described.

[0109] Upon receiving the front side captured image data sent from the acquisition unit 310B, the movement amount detection unit 321B detects, as a front side pixel movement amount, the amount of displacement of the position of a common feature area between the current front side captured image obtained from the data of the current front side captured image and the previous front side captured image obtained from the data of the previous front side captured image, in the same manner as in the detection of the pixel movement amount in the above-mentioned first embodiment. Moreover, upon receiving the rear side captured image data sent from the acquisition unit 310B, the movement amount detection unit 321B detects, as a rear side pixel movement amount, the amount of displacement of the position of a common feature area between the current rear side captured image obtained from the data of the current rear side captured image and the previous rear side captured image obtained from the data of the previous rear side captured image, in the same manner as in the detection of the pixel movement amount in the first embodiment.

[0110] If there are multiple feature regions common to the current front-side captured image and the previous front-side captured image, the movement amount detection unit 321B uses the average of the displacement amounts of the multiple feature regions as the front pixel movement amount. If there are multiple feature regions common to the current rear-side captured image and the previous rear-side captured image, the movement amount detection unit 321B uses the average of the displacement amounts of the multiple feature regions as the rear pixel movement amount.

[0111] Next, the movement amount detection unit 321B calculates the average of the front pixel movement amount and the rear pixel movement amount as the pixel movement amount. Then, the movement amount detection unit 321B sends the calculated pixel movement amount to the calibration unit 323B and the second output unit 330 (see FIG. 9).

[0112] 《Output process of moving object speed v》 Next, the output process of the moving object velocity v by the second output unit 330 will be described.

[0113] Upon receiving the pixel movement amount sent from the movement amount detection unit 321B, the second output unit 330 calculates the moving object speed v based on the pixel movement amount, the held pixel distance (i.e., the most recently calibrated pixel distance) and the period time TP in the same manner as in the first embodiment. Then, the second output unit 330 outputs the calculated moving object speed v to the ECU 230.

[0114] <<White line area identification process>> Next, the process of identifying a white line area by the identification unit 322B will be described.

[0115] Such a process of identifying the white line area is performed for the purpose of calibrating the pixel distance by the calibration unit 323B, which will be described later. The calibration environment of the pixel distance is shown in Fig. 11. As shown in Fig. 11, the calibration of the pixel distance is performed when a moving object MV crosses a white line area LCP, such as a pedestrian crossing marking, on a flat road surface while traveling in a straight line at a constant speed.

[0116] In the following description, it is assumed that the white line area LCP used for calibrating the new pixel distance is the white line area from the driving direction position (that is, the X-direction position) XR to the driving direction position XP.

[0117] Now, upon receiving the data of the front side captured image and the data of the rear side captured image sent from the acquisition unit 310B, the identification unit 322B identifies the white line area in the front side captured image and the rear side captured image based on the brightness of each pixel in the images, and sends the identification result to the calibration unit 323B as white line area information (see FIG. 9). The identification result of such a white line area includes two types: a "first rear partial white line" shown in FIG. 12(A) and a "second rear partial white line" shown in FIG. 12(B). Here, the "first rear partial white line" is a white line detected by the imaging unit 210 installed at the front side. F The "second rear partial white line" is a state of the rear partial white line in the front side captured image obtained by the imaging unit 210 installed on the rear side. R This shows the state of the rear partial white line in a rear-side captured image obtained by imaging using the imaging device.

[0118] In the second embodiment, the identification unit 322B generates white line area information on the condition that the white line continues to be clearly identifiable in the front captured image until it becomes a "first rear partial white line" when passing the white line. Therefore, if the white line cannot be clearly identified due to tire marks or the like, the identification unit 322B does not generate white line area information. That is, in the second embodiment, if the white line cannot be clearly identified due to tire marks or the like, the pixel distance is not calibrated.

[0119] In the second embodiment, the reason why the white line area is specified by focusing on the first rear partial white line in the front captured image and the second rear partial white line in the rear captured image is that in the case of thin white lines, if there are multiple white lines in the captured image, measurement errors are likely to occur. By taking such measures, unnecessary calibration is not required, which eliminates waste.

[0120] When the identification result is a first rear partial white line in the front captured image, the identification unit 322B identifies the length a in the X direction of the non-white line region in the image (hereinafter also referred to as the "front length a"). When the identification result is a second rear partial white line in the rear captured image, the identification unit 322B identifies the length b in the X direction of the non-white line region in the image (hereinafter also referred to as the "front length b").

[0121] In the case of a first rear partial white line in the front captured image, the identification unit 322B sends the white line area information [first rear partial white line flag: ON, front length: a] shown in Fig. 12(A) to the calibration unit 323B (see Fig. 9). In the case of a second rear partial white line, the identification unit 322B sends the white line area information [second rear partial white line flag: ON, front length: b] shown in Fig. 12(B) to the calibration unit 323B (see Fig. 9).

[0122] Pixel Distance Calibration Process Next, the calibration process by the calibration unit 323B will be described.

[0123] In the calibration process, the calibration unit 323B performs a constant speed straight travel determination based on the acceleration information, steering angle information, and inclination information sent from the ECU 230 to determine whether or not it can be determined that the moving body MV is traveling straight at a constant speed on a flat road surface.

[0124] If the result of the constant speed straight-line determination is positive, the calibration unit 323B performs a front white line end determination as to whether or not the white line area information in the front captured image sent from the determination unit 322B has become a "first rear partial white line." If the result of this front white line end determination is positive, the calibration unit 323B performs a front white line end determination as to whether or not the white line area information in the rear captured image sent from the determination unit 322B has become a "second rear partial white line," on the condition that the result of the constant speed straight-line determination on a flat road surface is maintained as positive, and performs a front white line end determination as to whether or not the white line area information in the rear captured image sent from the determination unit 322B has become a "second rear partial white line." N (See Figure 13).

[0125] Then, when the result of the rear white line end determination becomes positive, the calibration unit 323B calculates the front length a included in the white line area information at the time when the white line area information in the front captured image becomes the "first rear partial white line", the collected pixel movement amounts ΔX1, ..., ΔX N , and the front length b included in the white line area information at the time when the white line area information in the rear captured image becomes "second rear partial white line", and the imaging unit 210 Fand the imaging unit 210 R Based on the distance D between the two points, the provisional pixel distance PT [cm] is calculated using the following formula (10). PT = D / (a + ΔX1 + … + ΔX N -b) …(10)

[0126] Next, the calibration unit 323B calculates a weighted average of the current calculated provisional pixel distance and the internally stored provisional pixel distances calculated during a predetermined number of past calibrations to calculate a currently calibrated pixel distance. After this, the calibration unit 323B internally stores the current provisional pixel distance in place of the oldest of the provisional pixel distances previously stored therein.

[0127] In the weighted average, as in the first embodiment, the weight is decreased as the elapsed time up to the present increases.

[0128] The calibration unit 323B sends the calculated pixel distance to the second output unit 330. As a result, the second output unit 330 calculates the moving object velocity v using the new pixel distance.

[0129] As described above, in the second embodiment, the imaging unit 210 disposed on the moving body MV at a known distance D along the traveling direction of the moving body MV is F ,210 R Based on the captured image of the white line, which is a characteristic part on the road surface, the movement amount detection unit 321B detects the pixel movement amount. Furthermore, the identification unit 322B identifies the white line area in the captured image. Then, when the calibration unit 323B can determine that the moving object MV is traveling straight at a constant speed on a flat road surface based on the pixel movement amount and the identification result of the white line area in the captured image, as well as the known distance D, the imaging unit 210 F ,210 R The distance on the road surface corresponding to one pixel is calibrated in relation to the distance between the camera and the road surface.

[0130] Therefore, according to the second embodiment, regardless of whether the length of the white line, which is a feature on the road surface, along the traveling direction of the moving body is known or unknown, even if the distance between the road surface and the imaging unit changes due to changes in the weight of the moving body or changes in the air pressure of the wheels, the distance on the road surface corresponding to one pixel can be calibrated in accordance with the change.

[0131] In the second embodiment, the second output unit calculates the moving object speed based on the calibration result and the pixel movement amount, so that the moving object speed can be output with high accuracy as the traveling information.

[0132] In the second embodiment, white lines are used as the road surface features, and therefore the white line area can be easily identified by checking the brightness of each pixel in the captured image.

[0133] In the second embodiment, similarly to the first embodiment, a weighted average of the current provisional pixel distance and provisional pixel distances calculated during a predetermined number of recent calibrations is calculated to obtain a final calibration result. This makes it possible to perform calibration that suppresses the influence of the pixel distance on the calibration result, such as the variation in the length of the white line in the traveling direction.

[0134] In the second embodiment, similarly to the first embodiment, calibration is performed when the vehicle is traveling straight ahead, traveling at a constant speed, and on a road surface with no inclination, so that accurate calibration of pixel distance can be performed.

[0135] [Modifications of the embodiment] The present invention is not limited to the above-described first and second embodiments, and various modifications are possible.

[0136] For example, in the above first embodiment, a stop line just before an intersection is used as a road feature, but other types of road markings may be used as road feature as long as they have a known length along the traveling direction of the moving object.

[0137] Furthermore, in the above second embodiment, white lines such as pedestrian crossings are used as the road surface features, but road markings other than white lines may also be used as the road surface features.

[0138] In the second embodiment, the pixel movement amount is detected as the average of the front pixel movement amount obtained from the time change of the front image and the rear pixel movement amount obtained from the time change of the rear image. However, either one of the front pixel movement amount or the rear pixel movement amount may be detected as the pixel movement amount.

[0139] In the second embodiment, the pixel distance is calibrated based on the change in the captured image from the time when the front captured image shows the first rear partial white line to the time when the rear captured image shows the second rear partial white line. However, the pixel distance may be calibrated based on the change in the captured image from the time when the front captured image shows the first front partial white line to the time when the rear captured image shows the second front partial white line.

[0140] Furthermore, in the above first and second embodiments, the pixel movement amount is detected by a so-called displacement amount search method, but the pixel movement amount may be detected by an image correlation method or a spatial filter method.

[0141] In the first and second embodiments, the pixel distance, which is the distance on the road surface corresponding to one pixel, is calibrated as the distance-related information related to the distance between the imaging unit and the road surface. Alternatively, the image element distance, which is the distance on the road surface corresponding to the length in the traveling direction of a specific image element, which is a set of multiple pixels, may be calibrated.

[0142] In addition, in the first and second embodiments described above, the second output unit outputs the moving body speed (traveling speed) as the traveling information, but the traveling distance may be output instead of or in addition to the moving body speed.

[0143] In the first and second embodiments, the pixel distance is calibrated by calculating a weighted average of the current provisional pixel distance and a predetermined number of past provisional pixel distances. However, the current provisional pixel distance may be used as the calibrated pixel distance as it is.

[0144] In addition, in the above first and second embodiments, it is assumed that the imaging unit is provided outside the detection device, but the detection device may include the imaging unit.

[0145] In the first and second embodiments, the detection device is mounted on the moving body. However, the detection device may be installed outside the moving body and connected to an imaging unit and an ECU mounted on the vehicle via a wireless communication network.

[0146] The detection device of the first and second embodiments may be configured as a computer as a calculation means having a central processing unit (CPU) or the like, and a prepared program may be executed by the computer to execute some or all of the functions of the detection device of the first and second embodiments. The program may be recorded on a computer-readable recording medium such as a hard disk, CD-ROM, or DVD, and the computer may load and execute the program from the recording medium. The program may be acquired in a form recorded on a portable recording medium such as a CD-ROM or DVD, or may be acquired in a form distributed via a network such as the Internet. EXAMPLES

[0147] Next, examples of the present invention will be described with reference to Fig. 14 to Fig. 23. In the following description, including the above-mentioned embodiment, the same or equivalent elements are denoted by the same reference numerals, and duplicated descriptions will be omitted as much as possible.

[0148] [First Example] First, a first embodiment of the present invention will be described with reference mainly to FIGS.

[0149] <Configuration> 14 shows a schematic configuration of a detection device 100A according to a first example. This detection device 100A is one aspect of the detection device 300A according to the first embodiment described above.

[0150] 14, the detection device 100A is connected to an imaging unit 210, a navigation device 220, and an ECU 230. The detection device 100A, the imaging unit 210, the navigation device 220, and the ECU 230 are mounted on a vehicle CR.

[0151] The detection device 100A includes a control unit 110A. The detection device 100A also includes a storage unit 120.

[0152] The control unit 110A includes a central processing unit (CPU) as a calculation means. The control unit 110A executes a program to function as the detection device 300A in the first embodiment, that is, to function as the acquisition unit 310A, the first output unit 320A, and the second output unit 330.

[0153] The program executed by the control unit 110A is stored in the storage unit 120 and is loaded from the recording unit and executed. This program may be acquired in a form recorded on a portable recording medium such as a CD-ROM or DVD, or may be acquired in a form distributed via a network such as the Internet.

[0154] The process executed by the control unit 110A will be described later.

[0155] The storage unit 120 stores various information data used by the control unit 110A. Such information data includes programs executed by the control unit 110A, the latest pixel movement amount, a calibration history including the latest pixel distance, etc. The storage unit 120 is accessible by the control unit 110A.

[0156] <Operation> Next, the detection operation by the detection device 100A configured as above will be described, focusing on the processing by the control unit 110A.

[0157] It is assumed that the imaging unit 210 has already started operation and is sequentially sending image data of road surface images captured at the periodic time TP to the detection device 100A. It is assumed that the navigation device 220 has already started operation and is configured to send a notification to the detection device 100A when the vehicle CR is within a predetermined distance from an intersection in the traveling direction. It is assumed that the ECU 230 has already started operation and is sending acceleration information, steering angle information, and inclination information to the detection device 100A (see FIG. 14).

[0158] Furthermore, in the detection device 100A, the pixel distance has already been calibrated a number of times, and the storage unit 120 stores the calibration history including the calibration times of the most recent predetermined number of times, the characteristics and in-image positions of the feature area extracted from the previous captured image, and a provisional pixel distance, which will be described later. Here, during the period until the first calibration of the pixel distance is performed, the storage unit 120 stores an average pixel distance.

[0159] Detecting pixel movement amount First, the process of detecting the pixel movement amount by the control unit 110A will be described.

[0160] In the pixel movement amount detection process, first, in step S11, the control unit 110A determines whether or not new captured image data has been received, as shown in Fig. 15. If the result of the determination in step S11 is negative (step S11: N), the process of step S11 is repeated.

[0161] When new captured image data is received and the result of the determination in step S11 becomes positive (step S11: Y), the process proceeds to step S12. In this step S12, the control unit 110A extracts a feature region in the captured image (current captured image) obtained from the new captured image data, and identifies the characteristics and the position in the image of the feature region.

[0162] Next, in step S13, the control unit 110A detects the pixel movement amount. When detecting the pixel movement amount, the control unit 110A refers to the characteristics and in-image positions of the feature areas extracted from the previous captured image in the storage unit 120, and detects the amount of displacement of the in-image positions of the common feature areas between the current captured image and the previous captured image as a new pixel movement amount.

[0163] When there are multiple common feature regions, the control unit 110A uses the average value of the respective displacement amounts of the multiple feature regions as the pixel movement amount.

[0164] Then, the control unit 110A registers the characteristics and the intra-image position of the feature area in the current captured image as the characteristics and the intra-image position of the feature area extracted from the previous captured image in the storage unit 120. The control unit 110A also registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

[0165] When the process of step S13 ends in this way, the process returns to step S11. Thereafter, steps S11 to S13 are repeated, and the pixel movement amount is detected every time new captured image data is received.

[0166] 《Output processing of vehicle speed v》 Next, the output process of the vehicle speed v by the control unit 110A will be described.

[0167] In the output process of the vehicle body speed v, first, in step S21, the control unit 110A determines whether or not a new pixel movement amount is detected, as shown in Fig. 16. If the result of the determination in step S21 is negative (step S21: N), the process of step S21 is repeated.

[0168] When a new pixel movement amount is detected and the result of the determination in step S21 becomes positive (step S21: Y), the process proceeds to step S22. In this step S22, the vehicle body speed v is calculated by the above-mentioned formula (7) based on the latest pixel movement amount (PN) and the latest pixel distance (PL) registered in the storage unit 120. Then, the control unit 110A outputs the calculated vehicle body speed v to the ECU 230.

[0169] When the process of step S22 ends in this way, the process returns to step S21. Thereafter, the processes of steps S21 and S22 are repeated, and a new vehicle body speed v is calculated every time a new pixel movement amount is detected, that is, every time new imaging data is received, and the calculated vehicle body speed v is output to the ECU 230.

[0170] <<White line area identification process>> Next, a process of specifying a white line area by the control unit 110A will be described. The process of specifying a white line area is executed in parallel with the process of detecting the pixel movement amount and the process of outputting the vehicle body speed v described above.

[0171] The process of identifying the white line area is performed for the purpose of calibrating the pixel distance, which will be described later. The calibration environment for calibrating the pixel distance is the same as that shown in FIG.

[0172] In the process of identifying the white line area, first, in step S31, the control unit 110A determines whether or not new captured image data has been received, as shown in Fig. 17. If the result of the determination in step S31 is negative (step S31: N), the process of step S31 is repeated.

[0173] When new captured image data is received and the result of the determination in step S31 becomes positive (step S31: Y), the process proceeds to step S32. In this step S32, the control unit 110A identifies the white line area in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

[0174] Next, in step S33, the control unit 110A judges whether the result of the determination of the white line area is "partial white line ahead (see FIG. 7(A))". If the result of the judgment in step S33 is negative (step S33: N), the process returns to step S31. Then, the processes in steps S31 to S33 are repeated until the result of the judgment in step S33 becomes positive.

[0175] If the result of identifying the white line area is "partial white line ahead" and the result of the determination in step S33 is affirmative (step S33: Y), the process proceeds to step S34. In step S34, the control unit 110A calculates the front length a (see FIG. 7(A)). Then, the process proceeds to step S35.

[0176] Next, in step S35, the control unit 110A determines whether or not new captured image data has been received If the result of the determination in step S35 is negative (step S35: N), the process of step S35 is repeated.

[0177] When new captured image data is received and the result of the determination in step S35 becomes positive (step S35: Y), the process proceeds to step S36. In this step S36, the control unit 110A identifies the white line area in the captured image obtained from the new captured image data based on the brightness of each pixel in the captured image.

[0178] Next, in step S37, the control unit 110A judges whether the result of the determination of the white line area is "rear partial white line (see FIG. 7(B))". If the result of the judgment in step S37 is negative (step S37: N), the process returns to step S35. Then, the processes in steps S35 to S37 are repeated until the result of the judgment in step S37 becomes positive.

[0179] When the result of identifying the white line area is "rear partial white line" and the result of the determination in step S37 is affirmative (step S37: Y), the process proceeds to step S38. In step S38, the control unit 110A calculates the front length b (see FIG. 7(B)). Then, the process returns to step S31.

[0180] Thereafter, steps S31 to S38 are repeated. As a result, the white line area is specified every time the vehicle CR crosses a white line.

[0181] <Calibration condition monitoring process> Next, the monitoring process of the calibration conditions by the control unit 110A will be described. The monitoring process of the calibration conditions is executed in parallel with the above-mentioned detection process of the pixel movement amount, the output process of the vehicle speed v, and the specification process of the white line area.

[0182] 18, in the monitoring process of the calibration conditions, first, in step S41, it is determined whether or not the vehicle CR is traveling straight based on the latest steering angle information sent from the ECU 230. If the result of the determination in step S41 is negative (step S41: N), the process proceeds to step S44 described later.

[0183] If the result of the determination in step S41 is positive (step S41: Y), the process proceeds to step S42. In this step S42, the control unit 110A determines whether the vehicle CR is accelerating or decelerating based on the latest acceleration information sent from the ECU 230. If the result of the determination in step S42 is positive (step S42: Y), the process proceeds to step S44.

[0184] If the result of the determination in step S42 is negative (step S42: N), the process proceeds to step S43. In this step S43, the control unit 110A determines whether or not there is a body inclination of the vehicle CR based on the latest inclination information sent from the ECU 230.

[0185] If the result of the determination in step S43 is affirmative (step S43: Y), the process proceeds to step S44. In step S44, the control unit 110A sets the calibration condition flag to "OFF." Then, the process returns to step S41.

[0186] On the other hand, if the result of the determination in step S43 is negative (step S43: N), the process proceeds to step S45. In this step S45, the control unit 110A sets the calibration condition flag to "ON." Then, the process returns to step S41.

[0187] Thereafter, the processes of steps S41 to S44 are repeated. As a result, if it is determined that the calibration condition that the vehicle CR is traveling straight ahead at a constant speed on a flat road surface is satisfied, the calibration condition flag is set to "ON". On the other hand, if it is determined that the calibration condition is not satisfied, the calibration condition flag is set to "OFF".

[0188] Pixel Distance Calibration Process Next, the pixel distance calibration process by the control unit 110A will be described. The pixel distance calibration process is executed in parallel with the pixel movement amount detection process, the vehicle speed v output process, the white line area specification process, and the calibration condition monitoring process.

[0189] In the pixel distance calibration process, as shown in Fig. 19, first, in step S50, the control unit 110A determines whether or not the control unit 110A has received a notification from the navigation device 220 that the vehicle is within a predetermined distance from an intersection in the traveling direction, thereby determining whether or not the vehicle has approached an intersection. If the result of the determination in step S50 is negative (step S50: N), the process of step S50 is repeated.

[0190] When the vehicle approaches an intersection and the result of the determination in step S50 becomes positive (step S50: Y), the process proceeds to step S51. In step S51, the control unit 110A determines whether the calibration condition flag is "ON" to determine whether the vehicle CR is traveling straight ahead at a constant speed on a flat road surface.

[0191] If the result of the determination in step S51 is negative (step S51: N), the process returns to step S50, and the processes in steps S50 and S51 are repeated.

[0192] If the result of the determination in step S51 is positive (step S51: Y), the process proceeds to step S52. In this step S52, the control unit 110A determines whether the "front partial white line" has been identified by the above-mentioned white line area identification process, and whether the front length a has been calculated.

[0193] If the result of the determination in step S51 is negative (step S52: N), the process returns to step S51. Then, on condition that the result of the determination in step S51 is not negative, the processes in steps S51 and S52 are repeated.

[0194] If the result of the determination in step S52 is affirmative (step S52: Y), the process proceeds to step S53. In this step S53, the distance parameter L is set to the forward length a.

[0195] Next, in step S54, the control unit 110A determines whether the calibration condition flag is "ON" to determine whether the vehicle CR is traveling straight ahead at a constant speed on a flat road surface. If the result of the determination in step S54 is negative (step S54: N), the process returns to step S50.

[0196] If the result of the determination in step S54 is positive (step S54: Y), the process proceeds to step S55. In this step S55, the control unit 110A detects a new pixel movement amount (ΔX j ) is detected.

[0197] If the result of the determination in step S55 is negative (step S55: N), the process returns to step S54. Then, on condition that the result of the determination in step S54 is not negative, the processes in steps S54 and S55 are repeated.

[0198] If the result of the determination in step S55 is positive (step S55: Y), the process proceeds to step S56. In this step S56, the control unit 110A adds a new pixel movement amount (ΔX j ) to update the value of the distance parameter L.

[0199] Next, in step S57, the control unit 110A determines whether the "rear partial white line" has been identified by the above-mentioned white line area identification process, and whether the front length b is calculated. If the result of the determination in step S57 is negative (step S57: N), the process returns to step S54. Then, the processes in steps S54 to S57 are repeated, provided that the result of the determination in step S54 is not negative.

[0200] If the result of the determination in step S57 is affirmative (step S57: Y), the process proceeds to step S58. In step S58, the control unit 110A subtracts the forward length b from the value of the distance parameter L up to that point, to update the value of the distance parameter L.

[0201] Next, in step S59, the control unit 110A calculates a tentative pixel distance PT [cm] based on the value of the distance parameter L at that time point, using the following equation (11). PT=45 / L …(11) Incidentally, formula (11) is equivalent to formula (8) described above.

[0202] Next, the control unit 110A calculates a weighted average of the calculated current tentative pixel distance and the tentative pixel distances calculated during a predetermined number of past calibrations stored in the storage unit 120 to calculate a new calibrated pixel distance. After this, the control unit 110A updates the latest pixel distance in the storage unit 120 to the newly calibrated pixel distance, and stores the current tentative pixel distance together with the current time in the storage unit 120, replacing the oldest of the tentative pixel distances stored in the storage unit 120 up to that point. As a result, when performing the output process of the vehicle body speed v described above, the control unit 110A calculates the vehicle body speed v using the new pixel distance.

[0203] When the process of step S59 ends in this way, the process returns to step S50. Then, the processes of steps S50 to S59 are repeated. As a result, the pixel distance is calibrated every time the vehicle CR travels straight ahead at a constant speed on a flat road and crosses the stop line (white line) before an intersection.

[0204] As described above, in the first embodiment, the control unit 110A detects the pixel movement amount based on the image captured by the imaging unit 210 of the vehicle CR. The control unit 110A also identifies the white line area in the captured image. Then, when the control unit 110A can determine that the vehicle CR is traveling straight at a constant speed on a flat road surface based on the pixel movement amount, the result of identifying the white line area in the captured image, and the known length of the stop line along the traveling direction of the vehicle CR, it calibrates the distance on the road surface corresponding to one pixel, which is related to the distance between the imaging unit 210 and the road surface.

[0205] Therefore, according to the first embodiment, even if the distance between the road surface and the imaging unit changes due to changes in vehicle weight or changes in wheel air pressure, the distance on the road surface corresponding to one pixel can be calibrated in accordance with the change.

[0206] In the first embodiment, the control unit 110A calculates the vehicle speed based on the calibration result and the pixel movement amount, and therefore, the vehicle speed can be output with high accuracy as the traveling information.

[0207] In the first embodiment, stop lines are used as the road surface features. Therefore, the stop line area (i.e., the white line area) can be easily identified by checking the brightness of each pixel in the captured image.

[0208] In addition, in the first embodiment, information indicating that the vehicle is approaching an intersection is acquired, and it is predicted in advance that the vehicle will pass the stop line. This allows efficient calibration.

[0209] In addition, in the first embodiment, a weighted average of the current provisional pixel distance and provisional pixel distances calculated during a predetermined number of past calibrations is calculated to obtain a final calibration result. This makes it possible to perform calibration that suppresses the influence of the pixel distance on the calibration result, such as the fluctuation in the length of the stop line in the traveling direction.

[0210] In the first embodiment, the calibration is performed when the vehicle is traveling straight ahead, at a constant speed, and on a road with no inclination, which allows accurate calibration of pixel distance.

[0211] [Second Example] Next, a second embodiment of the present invention will be described with reference mainly to FIGS.

[0212] <Configuration> 20 shows a schematic configuration of a detection device 100B according to a first example. This detection device 100B is one aspect of the detection device 300B according to the second embodiment described above.

[0213] As shown in FIG. 20, the detection device 100B includes an imaging unit 210 F ,210 R and the ECU 230. The detection device 100B, the imaging unit 210 F ,210 R and ECU 230 is mounted on a vehicle CR.

[0214] The detection device 100B includes a control unit 110B. The detection device 100B also includes a storage unit 120.

[0215] The control unit 110B includes a central processing unit (CPU) as a calculation means. The control unit 110B executes a program to function as the detection device 300B in the second embodiment, that is, to function as the acquisition unit 310B, the first output unit 320B, and the second output unit 330.

[0216] The program executed by the control unit 110B is stored in the storage unit 120 and is loaded from the recording unit and executed. This program may be acquired in a form recorded on a portable recording medium such as a CD-ROM or DVD, or may be acquired in a form distributed via a network such as the Internet.

[0217] The process executed by the control unit 110B will be described later.

[0218] The storage unit 120 stores various information data used by the control unit 110B. Such information data includes programs executed by the control unit 110B, the latest pixel movement amount, a calibration history including the latest pixel distance, etc. The storage unit 120 is accessible by the control unit 110B.

[0219] <Operation> Next, the detection operation by the detection device 100B configured as above will be described, focusing on the processing by the control unit 110B.

[0220] In addition, the imaging unit 210 F ,210 R It is assumed that the ECU 230 has already started operating and is sending the data of the front side captured image and the data of the rear side captured image of the road surface captured at the periodic time TP to the detection device 100B in sequence. It is also assumed that the ECU 230 has already started operating and is sending the acceleration information, the steering angle information, and the inclination information to the detection device 100B (see FIG. 20).

[0221] Furthermore, in the detection device 100B, the pixel distance has already been calibrated a number of times, and the storage unit 120 stores the calibration history including the calibration times of the most recent predetermined number of times, the characteristics and in-image positions of the feature area extracted from the previous captured image, and a provisional pixel distance, which will be described later. Here, during the period until the first calibration of the pixel distance is performed, the storage unit 120 stores an average pixel distance.

[0222] Detecting pixel movement amount First, the process of detecting the pixel movement amount by the control unit 110B will be described.

[0223] In the pixel movement amount detection process, as shown in Fig. 21, first, in step S61, the control unit 110B determines whether or not new data of a front side captured image and new data of a rear side captured image have been received. If the result of the determination in step S61 is negative (step S61: N), the process of step S61 is repeated.

[0224] When new front-side captured image data is received and the result of the determination in step S61 becomes positive (step S61: Y), the process proceeds to step S62. In step S62, the control unit 110B extracts a feature region in the front-side captured image obtained from the new front-side captured image data, and identifies the characteristics and the position in the image of the feature region.

[0225] Next, in step S63, the control unit 110B detects a front pixel movement amount. When detecting the front pixel movement amount, the control unit 110B refers to the characteristics and in-image positions of the feature areas extracted from the previous front-side captured image in the storage unit 120, and detects the amount of displacement of the in-image positions of the common feature areas between the current front-side captured image and the previous front-side captured image as a new front pixel movement amount.

[0226] When there are multiple common feature regions, the control unit 110B uses the average value of the displacement amounts of the multiple feature regions as the new front pixel movement amount.

[0227] Then, the control unit 110B registers the characteristics and the intra-image position of the feature area in the current captured image as the characteristics and the intra-image position of the feature area extracted from the previous captured image in the storage unit 120. In addition, the control unit 110B registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

[0228] Next, in step S64, the control unit 110B extracts a characteristic region in the rear-side captured image obtained from the data of the new rear-side captured image, and identifies the characteristics and the position in the image of the characteristic region.

[0229] Next, in step S65, the control unit 110B detects the rear pixel movement amount. When detecting the rear pixel movement amount, the control unit 110B refers to the characteristics and in-image positions of the feature areas extracted from the previous rear side captured image in the storage unit 120, and detects the amount of displacement of the in-image positions of the common feature areas between the current rear side captured image and the previous rear side captured image as a new rear pixel movement amount.

[0230] In addition, when there are multiple common feature regions, similarly to the case of detecting the front pixel movement amount, the control unit 110B adopts the average value of the respective displacement amounts of the multiple feature regions as the new rear pixel movement amount.

[0231] Next, in step S66, the control unit 110B detects a new pixel movement amount. In detecting the pixel movement amount, the control unit 110B detects the new pixel movement amount by calculating the average of the front pixel movement amount and the rear pixel movement amount.

[0232] Then, the control unit 110B registers the characteristics and in-image position of the characteristic area in the current front side captured image and the characteristics and in-image position of the characteristic area in the current rear side captured image as the characteristics and in-image position of the characteristic area in the previous front side captured image and the characteristics and in-image position of the characteristic area in the previous rear side captured image in the storage unit 120. In addition, the control unit 110B registers the new pixel movement amount in the storage unit 120 as the latest pixel movement amount.

[0233] When the process of step S66 ends in this way, the process returns to step S61. Thereafter, steps S61 to S66 are repeated, and the pixel movement amount is detected every time new data of a front side captured image and new data of a rear side captured image are received.

[0234] 《Output processing of vehicle speed v》 The output process of the vehicle body speed v by the control unit 110B is similar to the output process of the vehicle body speed v by the control unit 110A described above (see FIG. 16).

[0235] <<White line area identification process>> Next, a process of specifying a white line area by the control unit 110B will be described. The process of specifying a white line area is executed in parallel with the process of detecting the pixel movement amount and the process of outputting the vehicle body speed v described above.

[0236] The process of identifying the white line area is performed for the purpose of calibrating the pixel distance, which will be described later. The calibration environment for calibrating the pixel distance is the same as that shown in FIG.

[0237] In the process of identifying the white line area, first, in step S71, the control unit 110B determines whether or not new data of a front side captured image has been received, as shown in Fig. 22. If the result of the determination in step S71 is negative (step S71: N), the process of step S71 is repeated.

[0238] When new front-side captured image data is received and the result of the determination in step S71 becomes positive (step S71: Y), the process proceeds to step S72. In step S72, the control unit 110B identifies a white line area in the front-side captured image obtained from the new front-side captured image data based on the brightness of each pixel in the front-side captured image.

[0239] Next, in step S73, the control unit 110B judges whether the result of the determination of the white line area is the "first rear partial white line (see FIG. 12(A))". If the result of the judgment in step S73 is negative (step S73: N), the process returns to step S71. Then, the processes in steps S71 to S73 are repeated until the result of the judgment in step S73 becomes positive.

[0240] When the result of identifying the white line area in the front captured image is the "first rear partial white line" and the result of the determination in step S73 is affirmative (step S73: Y), the process proceeds to step S74. In this step S74, the control unit 110B calculates the front length a (see FIG. 12(A)).

[0241] In the second embodiment, the control unit 110B is configured to identify the "first rear partial white line" on the condition that the state in which the white line can be clearly identified in the front captured image continues until it becomes the "first rear partial white line."

[0242] Next, in step S75, the control unit 110B determines whether or not new rear side captured image data has been received. If the result of the determination in step S75 is negative (step S75: N), the process of step S75 is repeated.

[0243] When new rear-side captured image data is received and the result of the determination in step S75 becomes positive (step S75: Y), the process proceeds to step S76. In step S76, the control unit 110B identifies the white line area in the rear-side captured image obtained from the new rear-side captured image data based on the brightness of each pixel in the captured image.

[0244] Next, in step S77, the control unit 110B judges whether the result of the determination of the white line area is the "second rear partial white line (see FIG. 12(B))." If the result of the judgment in step S77 is negative (step S77: N), the process returns to step S75. Then, the processes in steps S75 to S77 are repeated until the result of the judgment in step S77 becomes positive.

[0245] When the result of identifying the white line area in the rear captured image is the "second rear partial white line" and the result of the determination in step S77 is affirmative (step S77: Y), the process proceeds to step S78. In this step S78, the control unit 110B calculates the front length b (see FIG. 12(B)). Then, the process returns to step S71.

[0246] Thereafter, steps S71 to S78 are repeated. As a result, the white line area is specified every time the vehicle CR crosses a white line.

[0247] <Calibration condition monitoring process> The monitoring process of the calibration conditions by the control unit 110B is similar to the monitoring process of the calibration conditions by the control unit 110A described above (see FIG. 18).

[0248] Pixel Distance Calibration Process Next, the pixel distance calibration process by the control unit 110B will be described. The pixel distance calibration process is executed in parallel with the pixel movement amount detection process, the vehicle speed v output process, the white line area specification process, and the calibration condition monitoring process.

[0249] In the pixel distance calibration process, as shown in Fig. 23, first, in step S81, the control unit 110B determines whether the calibration condition flag is "ON" to determine whether the vehicle CR is traveling straight ahead at a constant speed on a flat road surface. If the result of the determination in step S81 is negative (step S81: N), the process of step S81 is repeated.

[0250] If the result of the determination in step S81 is positive (step S81: Y), the process proceeds to step S82. In this step S82, the control unit 110B determines whether the front captured image is identified as a "first rear partial white line" by the above-mentioned white line area identification process, and whether the front length a is calculated.

[0251] If the result of the determination in step S82 is negative (step S82: N), the process returns to step S81, and the processes in steps S81 and S82 are repeated.

[0252] If the result of the determination in step S82 is affirmative (step S82: Y), the process proceeds to step S83. In this step S83, the distance parameter L is set to the forward length a.

[0253] Next, in step S84, the control unit 110B determines whether the calibration condition flag is "ON" to determine whether the vehicle CR is traveling straight ahead at a constant speed on a flat road surface. If the result of the determination in step S84 is negative (step S84: N), the process returns to step S81.

[0254] If the result of the determination in step S84 is positive (step S84: Y), the process proceeds to step S85. In this step S85, the control unit 110B detects a new pixel movement amount (ΔX j ) is detected.

[0255] If the result of the determination in step S85 is negative (step S85: N), the process returns to step S84. Then, on condition that the result of the determination in step S84 is not negative, the processes in steps S84 and S85 are repeated.

[0256] If the result of the determination in step S85 is affirmative (step S85: Y), the process proceeds to step S86. In this step S86, the control unit 110B adds a new pixel movement amount (ΔX j ) to update the value of the distance parameter L.

[0257] Next, in step S87, the control unit 110B determines whether the rear captured image is identified as a "second rear partial white line" by the above-mentioned white line area identification process, and the front length b is calculated. If the result of the determination in step S87 is negative (step S87: N), the process returns to step S84. Then, the processes of steps S84 to S87 are repeated, provided that the result of the determination in step S84 is not negative.

[0258] If the result of the determination in step S87 is affirmative (step S87: Y), the process proceeds to step S88. In step S88, the control unit 110B subtracts the forward length b from the value of the distance parameter L up to that point, to update the value of the distance parameter L.

[0259] Next, in step S89, the control unit 110B calculates the provisional pixel distance PT [cm] based on the value of the distance parameter L at that time point, using the following equation (12). PT=D / L …(12) Incidentally, formula (12) is equivalent to formula (10) described above.

[0260] Next, the control unit 110B calculates a weighted average of the calculated current tentative pixel distance and the tentative pixel distances calculated during a predetermined number of past calibrations stored in the storage unit 120 to calculate a new calibrated pixel distance. After this, the control unit 110B updates the latest pixel distance in the storage unit 120 to the newly calibrated pixel distance, and stores the current tentative pixel distance together with the current time in the storage unit 120, replacing the oldest of the tentative pixel distances stored in the storage unit 120 up to that point. As a result, when performing the output process of the vehicle body speed v described above, the control unit 110B calculates the vehicle body speed v using the new pixel distance.

[0261] When the process of step S89 ends in this way, the process returns to step S81. Then, the processes of steps S81 to S89 are repeated. As a result, the pixel distance is calibrated every time the vehicle CR travels straight ahead at a constant speed on a flat road and crosses a white line in front of an intersection.

[0262] As described above, in the second embodiment, the imaging unit 210 is disposed on the vehicle CR at a known distance D along the traveling direction of the vehicle CR. F ,210 R The control unit 110B detects the pixel movement amount based on the captured image by the imaging unit 210. The control unit 110B also identifies the white line area in the captured image. Then, when the control unit 110B can determine that the vehicle CR is traveling straight ahead at a constant speed on a flat road surface based on the pixel movement amount, the result of identifying the white line area in the captured image, and the known distance D, the control unit 110B F ,210 R The distance on the road surface corresponding to one pixel is calibrated in relation to the distance between the camera and the road surface.

[0263] Therefore, according to the second embodiment, regardless of whether the length of the white line, which is a feature on the road surface, along the traveling direction of the moving object is known or unknown, even if the distance between the road surface and the imaging unit changes due to changes in the weight of the moving object or changes in the air pressure of the wheels, the distance on the road surface corresponding to one pixel can be calibrated in accordance with the change.

[0264] In the second embodiment, the control unit 110B outputs the vehicle speed based on the calibration result and the pixel movement amount, so that the vehicle speed can be output with high accuracy as the traveling information.

[0265] In the second embodiment, white lines are used as road surface features, so that the white line area can be easily identified by checking the brightness of each pixel in the captured image.

[0266] In the second embodiment, similarly to the first embodiment, a weighted average of the current provisional pixel distance and provisional pixel distances calculated during a predetermined number of past calibrations is calculated to obtain a final calibration result. This makes it possible to perform calibration that suppresses the influence of the pixel distance on the calibration result, such as the variation in the length of the white line in the traveling direction.

[0267] In the second embodiment, similarly to the first embodiment, calibration is performed when the vehicle is traveling straight ahead, at a constant speed, and on a road with no inclination, so that accurate calibration of pixel distance can be performed.

[0268] [Modification of the embodiment] The present invention is not limited to the above-described first and second embodiments, and various modifications are possible.

[0269] For example, the first example can be modified in the same manner as the first embodiment described above, and the second example can be modified in the same manner as the second embodiment described above. [Explanation of symbols]

[0270] 100A, 100B … Detector 110A, 110B ... control unit (acquisition unit, first output unit, second output unit) 300A, 300B … Detector 310A,310B … Acquisition Department 320A,320B … 1st output section 321A, 321B … Movement detection section 322A,322B…Specific part 323A,323B … Calibration section 330 ... Second output section

Claims

[Claim 1] An acquisition unit that acquires an image of a road surface characteristic portion captured by an imaging unit mounted on the moving object; an output unit that outputs distance-related information related to a distance between a mounting position of the imaging unit and a road surface based on the image of the road-surface characteristic portion acquired by the acquisition unit; The imaging unit captures an image of a road surface directly below a fixed position at which the imaging unit is fixed to the moving body. A detection device comprising:

Images