Image processor, method for processing image, and image processing program

Abstract

To provide an image processor, a method for processing image, and an image processing program which can precisely estimate the posture of an in-vehicle camera.SOLUTION: The image processor according to the present application has a controller. The controller specifies the posture of an in-vehicle camera on the basis of an image taken by the in-vehicle camera. The controller extracts a feature point of which corner degree showing the similarity to a corner is at least a corner threshold value as a corner feature point from the image, estimates the posture on the basis of the extracted corner feature point, specifies the posture on the basis of the result of estimation in a case where the result of estimation satisfies a determination condition, and eases the corner threshold value in a case where the result of estimation does not satisfy a determination condition.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and an image processing program. [Background technology]

[0002] Conventionally, a technique has been proposed for extracting feature points from images captured by an in-vehicle camera such as a drive recorder, and estimating the attitude of the in-vehicle camera based on the feature points (see, for example, Patent Document 1). For example, in Patent Document 1, a corner degree indicating the likelihood of a corner such as a white line is calculated for each pixel based on the pixel values ​​of the camera image, and pixels with a corner degree equal to or greater than a threshold are extracted as corner feature points. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-191807 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has room for further improvement in terms of estimating the posture of the vehicle-mounted camera with high accuracy. In the conventional technology, for example, when the entire image is dark and the pixel value is low, such as at night, the corner degree is generally low. As a result, the number of corner feature points extracted from an image captured at night is small, and it cannot be said that the estimation accuracy of the posture of the vehicle-mounted camera is high.

[0005] The present application has been made in consideration of the above, and has an object to provide an image processing device, an image processing method, and an image processing program that can accurately estimate the attitude of an in-vehicle camera. [Means for solving the problem]

[0006] The image processing device according to the present application includes a controller that identifies an attitude of an on-board camera based on an image captured by the on-board camera, extracts feature points from the image that have a corner degree indicating a corner-likeness equal to or greater than a corner threshold as corner feature points, estimates the attitude based on the extracted corner feature points, identifies the attitude based on the estimation result if the estimation result satisfies a determination condition, and relaxes the corner threshold if the estimation result does not satisfy the determination condition. Effect of the Invention

[0007] According to one aspect of the embodiment, an effect is achieved in that the attitude of the vehicle-mounted camera can be estimated with high accuracy. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 1A is a diagram showing an overview of an image processing method according to an embodiment. [Figure 1B] FIG. 1B is a diagram showing an image captured by a camera mounted on a vehicle. [Diagram 2] FIG. 2 is a functional block diagram of the image processing device. [Diagram 3] FIG. 3 is a diagram for explaining the process of relaxing the corner threshold. [Figure 4] FIG. 4 is a diagram for explaining a process of relaxing the corner threshold during a predetermined period. [Diagram 5] FIG. 5 is a flowchart illustrating an example of a posture estimation process executed by the controller according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of an image processing device, an image processing method, and an image processing program will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiment.

[0010] First, an overview of an image processing method according to an embodiment will be described with reference to Fig. 1A and Fig. 1B. Fig. 1A is a diagram showing an overview of an image processing method according to an embodiment. Fig. 1B is a diagram showing an image (hereinafter, camera image) captured by a camera mounted on a vehicle (hereinafter, vehicle-mounted camera).

[0011] The image processing method according to the embodiment is executed by an image processing device mounted on a vehicle. The image processing device may be configured integrally with an on-board camera, or may be configured separately from the on-board camera. The on-board camera is a camera fixedly installed on a vehicle, and is configured as, for example, a camera mounted on a drive recorder, a camera that functions as a sensor for object detection, or the like.

[0012] As shown in FIG. 1A, in the image processing method, the following processes are executed in this order: preprocessing, feature point extraction process, optical flow process, noise removal (first noise removal) process, camera posture estimation process, noise removal (second noise removal) process, and angle estimation value determination process.

[0013] Specifically, the pre-processing is a process of setting a region of interest in the acquired camera image. The region of interest is, for example, a ROI (Region of Interest) region. For example, the image processing device 1 sets the central region of the camera image as the region of interest. More specifically, the image processing device 1 sets the region of interest while avoiding a region in which the body of the vehicle is reflected (lower end region) and a region in which the sky is reflected (upper end region). The example shown in FIG. 1B shows an example in which two regions of interest 50a, 50b are set by dividing the central region of the camera image in the horizontal direction (regions of interest 50a, 50b are arranged side by side in the left-right direction).

[0014] The feature point extraction process is a process of extracting corner feature points from the attention area set by the pre-processing. Specifically, the image processing device 1 extracts pixels having edge strength equal to or greater than a threshold as feature points based on the pixel values ​​of each pixel in the attention area. The image processing device 1 calculates a corner degree indicating the corner-likeness of the extracted feature points, and extracts feature points having a corner degree equal to or greater than the corner threshold as corner feature points that correspond to corners. In other words, the corner feature points are points (pixels) that indicate the corners of the white lines drawn on the road surface. In the example shown in FIG. 1B, five corner feature points 60a are extracted from the attention area 50a, and one corner feature point 60b is extracted from the attention area 50b.

[0015] In addition, as will be described in detail later, when there are few corner feature points extracted, such as at night, the image processing device 1 can increase the number of corner feature points extracted by relaxing the corner threshold value used in the feature point extraction process.

[0016] The optical flow process is a process of deriving an optical flow for each extracted corner feature point. Specifically, the image processing device 1 derives, as an optical flow, a motion vector indicating the movement (position change) of the corner feature point between two camera images captured at a predetermined time interval. In the example shown in Fig. 1B, an optical flow 70a of five corner feature points 60a extracted from a region of interest 50a and an optical flow 70b of one corner feature point 60b extracted from a region of interest 50b are shown.

[0017] The first noise removal process is a process of projecting the derived optical flow onto a predetermined plane and extracting a combination of two optical flows whose projected optical flows are parallel to each other. Specifically, the image processing device 1 extracts a combination of optical flows between the attention areas. Taking FIG. 1B as an example, the image processing device 1 extracts a combination of each optical flow 70a of the attention area 50a and an optical flow 70b of the attention area 50b. That is, in FIG. 1B, five combinations are extracted.

[0018] The camera attitude estimation process is a process of estimating the attitude of the vehicle-mounted camera based on a pair of extracted optical flows. Specifically, the image processing device 1 estimates the PAN, TILT, and ROL angles of the vehicle-mounted camera based on the pair of two optical flows.

[0019] The second noise removal process is a process for removing the estimated results of the posture when the vehicle speed is low and below a certain level, when the vehicle is turning, etc. In other words, the image processing device 1 keeps the estimated results when the vehicle speed is above a certain level and when the vehicle is traveling straight. The second noise removal process may be applied before the pre-processing.

[0020] The process of determining the angle estimate is a process of determining an estimate of the attitude of the vehicle-mounted camera based on the attitude estimation result. Specifically, first, the image processing device 1 generates a histogram in which the estimated values ​​of the attitude estimated for each optical flow pair are classified. More specifically, the image processing device 1 generates a histogram from the attitude estimation results for each optical flow pair extracted from each camera image captured over a predetermined period (e.g., 10 minutes). Then, the image processing device 1 determines the median value in the histogram as the estimated value of the attitude (specifies the attitude).

[0021] Here, as shown in FIG. 1A, in order to determine an estimated pose value with high accuracy from a histogram in the process of determining an estimated angle value, it is necessary to extract a sufficient number of corner feature points from camera images captured over a predetermined period of time to increase the frequency (the number of estimated values).

[0022] However, in a camera image captured at night, for example, the white lines also appear dark, making it difficult to extract corner feature points. Specifically, when the entire image is dark and the pixel values ​​are low, such as at night, the corner degree of each pixel is generally low. As a result, if each pixel is divided by the corner threshold, the number of corner feature points extracted will inevitably be reduced.

[0023] Therefore, in the present disclosure, when a situation in which the number of corner feature points is reduced, such as at night, is detected, the corner threshold is relaxed to increase the number of corner feature points to be extracted.

[0024] Specifically, in the process of determining an angle estimate, the image processing device 1 determines whether or not a histogram generated by accumulating the estimated values ​​of the orientation satisfies a judgment condition using a threshold. If the histogram satisfies the judgment condition, the image processing device 1 performs a process of determining an estimated value of the orientation. On the other hand, if the histogram does not satisfy the judgment condition, the image processing device 1 does not perform a process of determining an estimated value of the orientation, and performs a process of relaxing the corner threshold.

[0025] Specifically, the image processing device 1 determines that the determination condition is met when the total frequency of the histogram (the number of estimated values, the number of optical flow pairs) is equal to or greater than a threshold value and the standard deviation of the histogram is less than the threshold value.

[0026] On the other hand, when the total frequency of the histogram is less than the threshold or the standard deviation of the histogram is equal to or greater than the threshold, the image processing device 1 determines that the judgment condition is not satisfied and relaxes the corner threshold. That is, when the frequency of the histogram is low because few corner feature points are extracted at night or the like, the image processing device 1 relaxes the corner threshold to increase the number of corner feature points extracted.

[0027] This allows the image processing device 1 to ensure a high frequency of the histogram, and therefore allows the image processing device 1 to determine a highly accurate estimated value in the process of determining the angle estimated value. That is, the image processing device 1 according to the embodiment allows the attitude of the vehicle-mounted camera to be estimated with high accuracy.

[0028] Next, a configuration example of the image processing device 1 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a functional block diagram of the image processing device 1. As shown in Fig. 2, the image processing device 1 according to the embodiment includes a controller 2 and a storage unit 3. In addition, the image processing device 1 is connected to an in-vehicle camera 100, a vehicle speed sensor 200, a G sensor 300, and a steering angle sensor 400.

[0029] The vehicle-mounted camera 100 is a camera that is installed so as to be able to capture images of the outside of the vehicle. For example, the vehicle-mounted camera 100 is installed at a position that captures images of the front, sides, and rear of the vehicle. The vehicle-mounted camera 100 outputs the captured camera images to the image processing device 1.

[0030] The vehicle speed sensor 200 is a sensor that detects the traveling speed of the vehicle. The vehicle speed sensor 200 outputs the detected traveling speed of the vehicle (hereinafter, vehicle speed) to the image processing device 1.

[0031] The G sensor 300 is a sensor that detects the acceleration of the vehicle. The G sensor 300 outputs information on the detected acceleration to the image processing device 1.

[0032] The steering angle sensor 400 is a sensor that detects the steering angle of the vehicle. The steering angle sensor 400 outputs information on the detected steering angle to the image processing device 1.

[0033] The storage unit 3 includes a non-volatile storage medium such as a non-volatile memory, a flash memory, a hard disk drive, etc. As shown in FIG.

[0034] The threshold information 31 includes information on thresholds used in the processing of the controller 2. The threshold information 31 stores threshold parameters for calculating corner thresholds and information on cutoff thresholds.

[0035] Returning to the explanation of Fig. 2, the controller 2 will be explained. The controller 2 includes a computer and various circuits having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, an input / output port, etc. The CPU of the computer functions by, for example, reading and executing a program stored in the ROM.

[0036] The controller 2 performs various processes for estimating the attitude of the vehicle-mounted camera 100 .

[0037] The controller 2 first acquires a camera image from the vehicle-mounted camera 100. The controller 2 may acquire a camera image each time an image is captured by the vehicle-mounted camera 100, or may acquire the camera images collectively at a predetermined time interval (e.g., 10 minutes) at which the attitude determination process is performed.

[0038] Next, the controller 2 sets an attention region in the acquired camera image. For example, the controller 2 sets two attention regions by dividing the central region of the camera image in the horizontal direction, as shown in FIG. 1B.

[0039] Next, the controller 2 extracts corner feature points from the set region of interest. Specifically, the controller 2 extracts pixels having edge strength equal to or greater than a threshold based on pixel values ​​as feature points for each pixel in the region of interest, and further calculates a corner degree indicating the likelihood of the extracted feature points being corners (pixels where two edges intersect). The corner degree can be calculated using a known detection method such as the Harris operator or the KLT (Kanade-Lucas-Tomasi) tracker.

[0040] Subsequently, the controller 2 extracts pixels whose calculated corner degree satisfies the threshold value of the threshold information 31 as corner feature points. Specifically, the controller 2 first extracts pixels (feature points) having a corner degree equal to or greater than the truncation threshold value of the threshold information 31. Subsequently, the controller 2 extracts, as corner feature points, pixels having a corner degree equal to or greater than the corner threshold determined based on the threshold parameter of the threshold information 31 from among the pixels extracted by the truncation threshold. The corner threshold can be obtained by multiplying the corner degree of the maximum value among the corner degrees of each pixel by a threshold parameter that is a coefficient X (0 < X < 1).

[0041] In this way, the controller 2 can reduce the processing load of the extraction process of the corner feature points using the corner threshold by pre-screening the pixels by the truncation threshold.

[0042] Subsequently, the controller 2 derives the optical flow of the corner feature points extracted by the corner threshold. Subsequently, the controller 2 forms the optical flows of each of the two target regions into a pair and generates all combinations of the optical flows existing in the two target regions.

[0043] Subsequently, the controller 2 projects the derived optical flow onto a predetermined plane. Subsequently, the controller 2 extracts combinations of two optical flows that are parallel to each other in the plane onto which the two optical flows forming the pair are projected.

[0044] Subsequently, the controller 2 estimates the attitude of the in-vehicle camera 100 based on the two extracted optical flows forming a pair. Specifically, the controller 2 calculates PAN, TILT, and ROL, which are estimated values of the attitude of the in-vehicle camera 100. Note that, as a method for estimating the attitude of the in-vehicle camera 100 based on the optical flow, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2020-198074 can be adopted, and the description thereof is omitted here.

[0045] Next, the controller 2 judges whether the estimated posture estimation result satisfies the judgment condition. Specifically, the controller 2 generates a histogram for each of PAN, TILT, and ROL by accumulating the estimated values ​​of the posture estimated during a predetermined period (e.g., 10 minutes). If the sum of the frequencies in the generated histogram is equal to or greater than a threshold and the standard deviation of the histogram is less than a threshold, the controller 2 judges that the judgment condition is satisfied and determines the median value of the histogram as the estimated posture value.

[0046] That is, the controller 2 groups the corner feature points selected from each of the two regions of interest into pairs, estimates the orientation for each pair, and determines that the determination condition is satisfied when the number of pairs (the sum of the frequencies of the histograms) is equal to or greater than a threshold and the standard deviation of the estimated orientation is less than a threshold. This enables the controller 2 to increase the accuracy of the estimated orientation determined from the histograms.

[0047] Furthermore, if the sum of the frequencies in the generated histogram is less than the threshold value, or if the standard deviation of the histogram is equal to or greater than the threshold value, the controller 2 determines that the judgment condition is not satisfied. In this case, the controller 2 does not perform a process of determining an estimated value of the attitude from the histogram, but performs a process of relaxing the corner threshold value used in the next predetermined period (next 10 minutes). Note that the controller 2 may perform a process of relaxing the corner threshold value when a situation occurs multiple times in which the histogram generated every predetermined period does not satisfy the judgment condition.

[0048] Here, the process of relaxing the corner threshold will be described with reference to Fig. 3. Fig. 3 is a diagram for explaining the process of relaxing the corner threshold. Fig. 3 shows a graph with the Harris value (Harris operator) on the vertical axis and the extracted feature points on the horizontal axis. The graph in Fig. 3 also arranges the feature points in descending order of Harris value. Fig. 3 also shows a corner threshold TH1 and a cutoff threshold TH2 that is lower (relaxed) than the corner threshold TH1.

[0049] In Fig. 3, feature points with relatively large Harris values ​​mainly indicate corner feature points, and feature points with small Harris values ​​equal to or less than TH2 indicate feature points other than corner feature points (feature points detected in parts other than corners, such as the road surface or straight lines). As shown in Fig. 3, when the histogram does not satisfy the judgment condition, the controller 2 relaxes (lowers) the corner threshold value TH1 to be used in the next predetermined period. Specifically, the controller 2 lowers the value of the threshold parameter, which is the coefficient A used when calculating the corner threshold value. As a result, the corner threshold value is lowered, and the number of corner feature points to be extracted can be increased.

[0050] As described above, in the process of extracting corner feature points, feature points equal to or greater than the cutoff threshold are first extracted, and then corner feature points are extracted using the corner threshold, so that when the controller 2 lowers the corner threshold, it also relaxes (lowers) the cutoff threshold TH2, thereby making it possible to prevent the corner threshold TH1 from becoming lower than the cutoff threshold TH2.

[0051] The controller 2 is not limited to the configuration in which the cut-off threshold TH2 is lowered, and may be configured to limit the reduction of the corner threshold TH1 to a value higher than the cut-off threshold TH2.

[0052] Furthermore, if a histogram generated based on corner feature points extracted using the relaxed corner threshold TH1 satisfies the judgment condition in the next predetermined period, the controller 2 cancels (returns) the relaxation of the corner threshold TH1 in the next predetermined period and thereafter. In other words, if the posture estimation result based on the corner feature points extracted using the relaxed corner threshold satisfies the judgment condition, the controller 2 cancels the relaxation of the corner threshold. This makes it possible to prevent the number of corner feature points extracted from increasing too much, thereby reducing the processing load of the controller 2.

[0053] Note that the controller 2 may make the condition for releasing the relaxation of the corner threshold TH1 stricter than the determination condition. Thereby, after releasing the relaxation of the corner threshold TH1, it is possible to avoid the relaxation of the corner threshold TH1 being performed again immediately. That is, it is possible to avoid the frequent relaxation and release of the corner threshold TH1.

[0054] Further, the controller 2 may gradually release the relaxation of the corner threshold TH1, and may completely release the relaxation of the corner threshold TH1 when the situation where the histogram satisfies the determination condition continues a plurality of times in a row.

[0055] Note that in the above, the controller 2 relaxes the corner threshold in the next predetermined period based on the histogram obtained by accumulating the estimated values obtained during a predetermined period (for example, 10 minutes). However, for example, the relaxation or release of the corner threshold may be performed during the predetermined period. This point will be described with reference to FIG. 4.

[0056] FIG. 4 is a diagram for explaining the process of relaxing the corner threshold during a predetermined period. In FIG. 4, a graph is shown in which the vertical axis represents the estimated number of times (the total frequency of the histogram) and the horizontal axis represents time. Note that the time Tf on the horizontal axis indicates the end timing of the predetermined period (the determination timing of the estimated value of the posture).

[0057] The graph shown in FIG. 4 shows the transition of the target value and the actual value of the estimated number of times in the predetermined period Tf.

[0058] When the controller 2 anticipates from the estimation result (estimated number of times) at an arbitrary timing within the predetermined period Tf that the estimation result (estimated number of times) after the predetermined period Tf will not satisfy the determination condition, the controller 2 relaxes the corner threshold after that timing.

[0059] Specifically, if the real value a1 falls below the target value by a predetermined value or more during the period up to time t1 when the normal, unrelaxed corner threshold is used, the controller 2 predicts that the number of estimations will not reach the target value A after the predetermined period Tf. In other words, the controller 2 predicts that with the normal corner threshold, the estimation result after the predetermined period Tf will not satisfy the judgment condition.

[0060] In this case, the controller 2 increases the number of corner characteristic points to be extracted (which inevitably increases the number of estimations) by relaxing the corner threshold value after time t1.

[0061] Next, if the real value a2 exceeds the target value by a predetermined value or more between time t1 and time t2 when the relaxed corner threshold is used, the controller 2 predicts that the estimation count will reach the target value A after the predetermined period Tf. Specifically, the controller 2 predicts that the estimation count will reach the target value A after the predetermined period Tf, assuming that the real value increases at the same rate as the normal corner threshold up to time t1 after time t2. In other words, the controller 2 predicts that the estimation result after the predetermined period Tf will satisfy the judgment condition, even if the relaxation of the corner threshold is released.

[0062] In this case, the controller 2 cancels the relaxation of the corner threshold value after time t2, and extracts corner characteristic points using the normal corner threshold value.

[0063] In this way, the controller 2 relaxes and releases the corner threshold during the predetermined period Tf, thereby making it possible for the real value to exceed the target value as much as possible after the predetermined period Tf.

[0064] Next, a posture estimation process executed by the controller 2 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of a posture estimation process executed by the controller 2 according to the embodiment, and when the controller 2 is started, the process is repeated at regular intervals.

[0065] As shown in FIG. 5, first, the controller 2 acquires a camera image captured by the vehicle-mounted camera 100 (step S101).

[0066] Next, the controller 2 sets two regions of interest (ROI) in the acquired camera image (step S102).

[0067] Next, the controller 2 extracts corner feature points for each of the two regions of interest (step S103). Specifically, the controller 2 calculates a corner degree for each feature point of the regions of interest based on the pixel value, and extracts feature points whose corner degree is equal to or greater than a corner threshold as corner feature points.

[0068] Next, the controller 2 generates a combination of optical flows (corner feature points) between the two regions of interest for the corner feature points extracted from each of the two regions of interest (step S104).

[0069] Next, the controller 2 estimates the attitude of the vehicle-mounted camera 100 for each combination of optical flows (step S105).

[0070] Next, the controller 2 determines whether or not there remains any combination of optical flows whose postures have not been estimated (step S106), and if there remains any combination of optical flows whose postures have not been estimated (step S106: Yes), the process returns to step S105.

[0071] On the other hand, if there are no unestimated optical flow combinations remaining (step S106: No), that is, if posture estimation has been completed for all combinations, the controller 2 determines whether a predetermined period has elapsed (step S107).

[0072] If the predetermined period has elapsed (step S107: Yes), the controller 2 determines whether the posture estimation result satisfies a judgment condition (step S108). Specifically, the controller 2 determines whether the number of posture estimations (the number of combinations of corner feature points) is equal to or greater than a threshold, and whether the standard deviation calculated from the histogram of the posture estimation value is less than a threshold. If the predetermined period has not elapsed (step S107: No), the controller 2 returns to step S101.

[0073] If the estimation result satisfies the judgment condition (step S108: Yes), the controller 2 identifies the orientation of the vehicle-mounted camera 100 based on the estimation result (step S109) and ends the process. That is, the controller 2 identifies the orientation of the vehicle-mounted camera 100 if the number of orientation estimations (the number of combinations of corner feature points) is equal to or greater than a threshold and the standard deviation calculated from the histogram of the orientation estimates is less than a threshold.

[0074] If the estimation result does not satisfy the determination condition (step S108: No), the controller 2 relaxes (lowers) the corner threshold (step S110) and ends the process.

[0075] As described above, the image processing device 1 according to the embodiment includes the controller 2. The controller 2 identifies the attitude of the vehicle-mounted camera 100 based on an image captured by the vehicle-mounted camera 100. The controller 2 extracts, from the image, feature points whose corner degree indicating corner-likeness is equal to or greater than a corner threshold as corner feature points, estimates the attitude based on the extracted corner feature points, and if the estimation result satisfies the determination condition, identifies the attitude based on the estimation result, and if the estimation result does not satisfy the determination condition, relaxes the corner threshold.

[0076] As a result, when the reliability of the pose estimation of the vehicle-mounted camera 100 is low due to a small number of corner feature points extracted at night, etc., the image processing device 1 relaxes the corner threshold value to increase the number of corner feature points extracted. As a result, the image processing device 1 can accurately specify the estimated value of the pose because the reliability of the pose estimation of the vehicle-mounted camera 100 is high. That is, according to the image processing device 1 according to the embodiment, the pose of the vehicle-mounted camera can be accurately estimated.

[0077] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0078] 1 Image processing device 2. Controller 3 Storage section 31 Threshold Information 50a, 50b Areas of interest 60a, 60b Corner feature points 70a, 70b Optical Flow 100 In-vehicle Camera 200 Vehicle speed sensor 300 G Sensor 400 Steering Angle Sensor TH1 Corner Threshold TH2 cutoff threshold

Claims

1. a controller that identifies the attitude of the vehicle-mounted camera based on an image captured by the vehicle-mounted camera; The controller: From the image, feature points whose corner degree indicating a corner-likeness is equal to or greater than a corner threshold are extracted as corner feature points, the orientation is estimated based on the extracted corner feature points, and if the estimation result satisfies a judgment condition, the orientation is identified based on the estimation result, and if the estimation result does not satisfy the judgment condition, the corner threshold is relaxed. Image processing device.

2. The controller: Two regions of interest are set in the image, the corner feature points are extracted for each of the regions of interest, the corner feature points selected from each of the two regions of interest are grouped into pairs, the orientation is estimated for each pair, and it is determined that the determination condition is satisfied if the number of pairs is equal to or greater than a threshold and the standard deviation of the estimated values ​​of the orientation is less than a threshold. The image processing device according to claim 1 .

3. The controller: The feature points having the corner degree equal to or greater than a cutoff threshold that is less stringent than the corner threshold are extracted, and the corner feature points having the corner degree equal to or greater than the corner threshold are extracted from the extracted feature points. The image processing device according to claim 1 .

4. The controller: When the corner threshold is relaxed because the estimation result does not satisfy the judgment condition, the cutoff threshold is also relaxed. The image processing device according to claim 3 .

5. The controller: If the orientation estimation result based on the corner feature points extracted using the relaxed corner threshold satisfies the determination condition, the relaxation of the corner threshold is released. The image processing device according to claim 1 .

6. The controller: It is determined whether the estimation result obtained during a predetermined period satisfies the judgment condition, and when it is predicted from the estimation result at any timing within the predetermined period that the estimation result after the predetermined period does not satisfy the judgment condition, the corner threshold value after the timing is relaxed. The image processing device according to claim 2 .

7. An image processing method executed by an image processing device, comprising: A control step of determining the attitude of the vehicle-mounted camera based on an image captured by the vehicle-mounted camera, The control step includes: From the image, feature points whose corner degree indicating a corner-likeness is equal to or greater than a corner threshold are extracted as corner feature points, the orientation is estimated based on the extracted corner feature points, and if the estimation result satisfies a judgment condition, the orientation is identified based on the estimation result, and if the estimation result does not satisfy the judgment condition, the corner threshold is relaxed. Image processing methods.

8. causing a computer to execute a control procedure for identifying the attitude of the vehicle-mounted camera based on an image captured by the vehicle-mounted camera; The control procedure includes: From the image, feature points whose corner degree indicating a corner-likeness is equal to or greater than a corner threshold are extracted as corner feature points, the orientation is estimated based on the extracted corner feature points, and if the estimation result satisfies a judgment condition, the orientation is identified based on the estimation result, and if the estimation result does not satisfy the judgment condition, the corner threshold is relaxed. Image processing program.

Images