Information processing apparatus, information processing method, and computer-readable recording medium

Abstract

An information processing apparatus according to an embodiment includes a control unit (corresponding to an example of a "controller"). The control unit performs pose estimation processing to estimate a pose of a vehicle-mounted camera based on an optical flow of a feature point in a region of interest set in an image captured by the vehicle-mounted camera. The control unit performs first pose estimation processing using a first region of interest set to a rectangular shape when the vehicle-mounted camera is mounted in a first state; and performs second pose estimation processing using a second region of interest set according to a road surface shape when the vehicle-mounted camera is mounted in a second state.

Description

Technical Field

[0001] The embodiments discussed herein relate to information processing apparatus, information processing methods, and computer-readable recording media. Background Technology

[0002] The mounting position and orientation of a vehicle-mounted camera may change due to accidental contact or over time, leading to errors in the initial calibration of the installation. To detect this, conventional techniques are known to estimate the orientation of the vehicle-mounted camera based on images captured by the camera.

[0003] For example, the technique disclosed in Japanese Patent Application Publication No. 2021-086258 extracts feature points on the road surface from a rectangular region of interest (ROI) set in the captured image and estimates the pose of the vehicle camera based on the optical flow indicating the motion of the feature points between frames.

[0004] Based on this optical flow, pairs of parallel line segments in real space can be extracted using, for example, an online algorithm. (Keio University, [March 31, 2022 search], Internet)<URL:http: / / im-lab.net / artoolkit-overview / > To estimate the attitude of the vehicle-mounted camera (rotation angles of the pan, pitch, and roll axes).

[0005] However, there is still room for improvement in the aforementioned traditional techniques to enhance the accuracy of attitude estimation by vehicle-mounted cameras.

[0006] One aspect of this embodiment is proposed in view of the above circumstances, and its object is to provide an information processing apparatus, information processing method and computer program capable of improving the accuracy of attitude estimation of vehicle-mounted cameras. Summary of the Invention

[0007] An information processing apparatus according to one aspect of an embodiment includes a controller. The controller performs attitude estimation processing to estimate the attitude of the vehicle-mounted camera based on the optical flow of feature points in a region of interest set in an image captured by the vehicle-mounted camera. When the vehicle-mounted camera is mounted in a first state, the controller performs first attitude estimation processing using a first region of interest set to a rectangular shape, and when the vehicle-mounted camera is mounted in a second state, the controller performs second attitude estimation processing using a second region of interest set according to the road surface shape.

[0008] According to one aspect of the embodiments, the accuracy of attitude estimation of vehicle-mounted cameras can be improved. Attached Figure Description

[0009] Figure 1 This is a schematic diagram (1) of the attitude estimation method according to the embodiment;

[0010] Figure 2 This is a schematic diagram (2) of the attitude estimation method according to the embodiment;

[0011] Figure 3 This is a schematic diagram (3) of the attitude estimation method according to the embodiment;

[0012] Figure 4 This is a block diagram illustrating an example configuration of an in-vehicle device according to an embodiment;

[0013] Figure 5 This is a diagram showing the road surface ROI and the superimposed ROI (1);

[0014] Figure 6 This is a diagram showing the road surface ROI and the superimposed ROI (2);

[0015] Figure 7 This is a block diagram illustrating an example configuration of the attitude estimation unit; and

[0016] Figure 8 This is a flowchart illustrating the process performed by the vehicle-mounted device according to an embodiment. Detailed Implementation

[0017] Embodiments of the information processing apparatus, information processing method, and computer-readable recording medium disclosed in this application will be described in detail below with reference to the accompanying drawings. The present invention is not limited to the embodiments described below.

[0018] In the following text, it will be assumed that the information processing device according to the embodiment is an in-vehicle device 10 installed in a vehicle. The in-vehicle device 10 is, for example, a driving recorder. It will also be assumed that the information processing method according to the embodiment is a camera 11 installed on the in-vehicle device 10 (see...). Figure 4 A pose estimation method.

[0019] Figures 1 to 3 These are schematic diagrams (1) to (3) of the attitude estimation method according to the embodiments. First, before describing the attitude estimation method according to the embodiments, the problems of the prior art will be described in more detail. Figure 1 The content of the question is shown.

[0020] When estimating the pose of camera 11 based on optical flow of feature points on the road surface, the feature points to be extracted on the road surface include corners of road markings such as lane markings.

[0021] However, as Figure 1 As shown, for example, lane markings in the captured image appear to converge toward the vanishing point in the perspective view. Therefore, when using a rectangular ROI (hereinafter referred to as "rectangular ROI 30-1"), feature points of 3D objects outside the road surface are more easily extracted in the upper left and upper right of the rectangular ROI 30-1.

[0022] Figure 1 Examples are shown for extracting optical flow Op1 and Op2 based on feature points on the road surface and for extracting optical flow Op3 based on feature points of a 3D object outside the road surface.

[0023] Due to the [online] algorithm, Keio University, [searched on March 31, 2022], Internet<URL:http: / / im-lab.net / artoolkit-overview / > Assuming pairs of parallel line segments in real space, a pair of optical flows Op1 and Op2 is the correct combination in attitude estimation (hereinafter referred to as "correct flow"). Conversely, for example, a pair of optical flows Op1 and Op3 is an incorrect combination (hereinafter referred to as "incorrect flow").

[0024] Based on such an error stream, the attitude of camera 11 cannot be accurately estimated. For each pair of extracted optical flow pairs, the rotation angles of the pan, pitch, and roll axes are estimated, and the axis misalignment of the camera 11's attitude is determined based on the median value of the histogram. Therefore, the attitude estimation of camera 11 may become less accurate with more error streams.

[0025] To address this issue, instead of rectangular ROI 30-1, consider setting ROI 30 based on the road surface shape appearing in the captured image. However, in this case, if the calibration values ​​of camera 11 (mount position and pan, pitch, and roll) are not known in advance, it is impossible to set ROI 30 based on the road surface shape (hereinafter referred to as "road surface ROI 30-2").

[0026] Therefore, in the attitude estimation method according to the embodiment, the in-vehicle device 10 (see...) is included. Figure 4 The control unit 15 in the camera 11 performs a first attitude estimation process using a rectangular ROI 30-1 set to a rectangular shape when the camera 11 is in the early stage after installation, and performs a second attitude estimation process using a superimposed ROI 30-S set according to the road surface shape when the camera 11 is not in the early stage after installation.

[0027] Here, "early stage after installation" refers to the case where camera 11 is installed in a "first state." The "first state" is a state where camera 11 is assumed to be in the early stage after installation. For example, the first state is a state where the time elapsed since camera 11 was installed is less than a predetermined elapsed time. For example, the first state is a state where the number of calibrations since camera 11 was installed is less than a predetermined number. For example, the first state is a state where the misalignment of camera 11 since installation is less than a predetermined misalignment. Conversely, "not in the early stage after installation" refers to the case where camera 11 is installed in a "second state" different from the first state.

[0028] Specifically, such as Figure 2 As shown, in the attitude estimation method according to the embodiment, when the camera 11 is in the early stage after installation, the control unit 15 uses the optical flow of the rectangular ROI 30-1 to perform attitude estimation processing (step S1). When the camera 11 is not in the early stage after installation, the control unit 15 uses the optical flow of the road surface ROI 30-2 in the rectangular ROI 30-1 to perform attitude estimation processing (step S2). The road surface ROI 30-2 in the rectangular ROI 30-1 refers to the superimposed ROI 30-S, which is the superimposed portion where the rectangular ROI 30-1 and the road surface ROI 30-2 overlap.

[0029] like Figure 2 As shown, using superimposed optical flow from ROI 30-S results in fewer error flows. For example, optical flows Op4, Op5, and Op6, which are included in the processing target in step S1, are no longer included in step S2.

[0030] Figure 3 A comparison is shown between the case with rectangular ROI 30-1 and the case with superimposed ROI 30-S. When using superimposed ROI 30-S, there is less error flow, fewer estimations, and higher estimation accuracy compared to using rectangular ROI 30-1. However, the estimation time is slower and calibration values ​​are required.

[0031] However, these drawbacks in estimating time and calibration values ​​are compensated for by using the attitude estimation process of rectangular ROI 30-1 performed in step S1 when the camera 11 is in the early stage after installation.

[0032] In other words, by using the pose estimation method according to the embodiment, the accuracy of the pose estimation of camera 11 can be improved, while the advantages of the other can be used to compensate for the disadvantages of using rectangular ROI 30-1 and using superimposed ROI 30-S.

[0033] In this manner, in the attitude estimation method according to the embodiment, the control unit 15 performs a first attitude estimation process using a rectangular ROI 30-1 set to a rectangular shape when the camera 11 is in an early stage after installation, and performs a second attitude estimation process using a superimposed ROI 30-S set according to the road surface shape when the camera 11 is not in an early stage after installation.

[0034] Therefore, by using the attitude estimation method according to the embodiment, the accuracy of attitude estimation of camera 11 can be improved.

[0035] The following will describe in more detail an example configuration of the vehicle-mounted device 10 that applies the aforementioned attitude estimation method according to the embodiments.

[0036] Figure 4 This is a block diagram illustrating an example configuration of the vehicle-mounted device 10 according to an embodiment. (Further details will follow.) Figure 4 and Figure 7 In this document, only the components necessary to describe the features of this embodiment are shown, and descriptions of general components are omitted.

[0037] In other words, Figure 4 and Figure 7 Each component shown is a functional concept and does not necessarily have to be physically configured as shown. For example, the specific distribution and integration of the blocks are not limited to those shown in the figure, but can be configured, either functionally or physically, in any unit distribution and integration, depending on various loads and usage conditions.

[0038] In use Figure 4 and Figure 7 In the description, components that have already been described can be simplified or omitted.

[0039] like Figure 4 As shown, the vehicle-mounted device 10 according to the embodiment includes a camera 11, a sensor unit 12, a notification device 13, a memory unit 14, and a control unit 15.

[0040] Camera 11 includes, for example, an image sensor such as a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS), and uses such an image sensor to capture an image of a predetermined imaging area. Camera 11 is mounted, for example, at different locations on a vehicle, such as the windshield or dashboard, to capture a predetermined imaging area in front of the vehicle.

[0041] Sensor unit 12 comprises various sensors mounted on the vehicle, including, for example, a vehicle speed sensor and a G-sensor. Notification device 13 notifies the user of calibration information. Notification device 13 may be implemented, for example, by a display or a speaker.

[0042] Memory unit 14 is implemented using storage devices such as random access memory (RAM) and flash memory. Figure 4 In the example, memory unit 14 stores image information 14a and installation information 14b.

[0043] Image information 14a stores images captured by camera 11. Installation information 14b is information about the installation of camera 11. Installation information 14b includes design values ​​for the installation position and orientation of camera 11, as well as the aforementioned calibration values. Installation information 14b may also include various information that can be used to determine whether camera 11 is in an early stage after installation, such as the installation date and time, the time elapsed since camera 11 was installed, and the number of calibrations since camera 11 was installed.

[0044] The control unit 15 is a "controller" and is implemented, for example, by a central processing unit (CPU) or microprocessor unit (MPU) that executes a computer program (not shown) according to an embodiment, stored in a memory unit 14 with RAM as its working area. The control unit 15 may be implemented by an integrated circuit such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0045] The control unit 15 has a mode setting unit 15a, an attitude estimation unit 15b, and a calibration execution unit 15c, and implements or executes the following information processing functions and actions.

[0046] When the camera 11 is in the early stage after installation, the mode setting unit 15a sets the attitude estimation mode, which is the execution mode of the attitude estimation unit 15b, to the first mode. When the camera 11 is not in the early stage after installation, the mode setting unit 15a sets the attitude estimation mode of the attitude estimation unit 15b to the second mode.

[0047] When the execution mode is set to the first mode, the attitude estimation unit 15b uses the optical flow of the rectangular ROI 30-1 to perform the first attitude estimation process. When the execution mode is set to the second mode, the attitude estimation unit 15b uses the optical flow of the road surface ROI 30-2 (i.e., the superimposed ROI 30-S) in the rectangular ROI 30-1 to perform the second attitude estimation process.

[0048] Here, the pavement ROI 30-2 and the superimposed ROI 30-S will be described in detail. Figure 5 The diagram shows the road surface ROI 30-2 and the superimposed ROI 30-S (1). Figure 6 It is also a diagram showing the road surface ROI 30-2 and the superimposed ROI 30-S (2).

[0049] like Figure 5 As shown, the road surface ROI 30-2 is set as ROI 30 based on the road surface shape appearing in the captured image. The road surface ROI 30-2 is set as the following area based on known calibration values: approximately half a lane to the left and one lane to the right of the lane in which the vehicle is traveling, and approximately 20m deep.

[0050] like Figure 5 As shown, the superimposed ROI 30-S is the overlapping portion of rectangular ROI 30-1 and road surface ROI 30-2. More abstractly, the superimposed ROI 30-S can be considered a trapezoidal region, where the upper left region C-1 and upper right region C-2 have been removed from rectangular ROI 30-1, as shown below. Figure 6As shown, by removing the upper left region C-1 and the upper right region C-2 from the rectangular ROI 30-1 and using the resulting region as the region of interest for pose estimation, the frequency of error flow can be reduced, and the accuracy of pose estimation can be improved.

[0051] An example configuration of the attitude estimation unit 15b will be described in more detail. Figure 7 This is a block diagram illustrating an example configuration of the attitude estimation unit 15b. (See diagram for example.) Figure 7 As shown, the attitude estimation unit 15b includes an acquisition unit 15ba, a feature point extraction unit 15bb, a feature point following unit 15bc, a line segment extraction unit 15bd, a calculation unit 15be, a noise removal unit 15bf, and a decision unit 15bg.

[0052] The acquisition unit 15ba acquires the image captured by the camera 11 and stores the image in image information 14a. The feature point extraction unit 15bb sets a ROI 30 corresponding to the execution mode of the pose estimation unit 15b for each captured image stored in the image information 14a. The feature point extraction unit 15bb also extracts feature points included in the set ROI 30.

[0053] The feature point following unit 15bc follows each feature point extracted by the feature point extraction unit 15bb between frames and extracts optical flow for each feature point. The line segment extraction unit 15bd removes noise components from the optical flow extracted by the feature point following unit 15bc and extracts a set of line segment pairs based on the optical flow.

[0054] For each pair of line segments extracted by line segment extraction unit 15bd, calculation unit 15be uses an [online] algorithm. (Keio University, [searched March 31, 2022], Internet)<URL:http: / / im-lab.net / artoolkit-overview / > To calculate the rotation angles of the pan, pitch, and roll axes.

[0055] The noise removal unit 15bf removes the noise component caused by low speed and steering angle from the angle calculated by the calculation unit 15be based on the sensor values ​​of the sensor unit 12. The determination unit 15bg creates a histogram for each angle with the noise component removed and determines the estimated angle values ​​for pan, pitch, and roll based on the median values. The determination unit 15bg stores the determined angle estimates in the installation information 14b.

[0056] Now the description returns to Figure 4The calibration execution unit 15c performs calibration based on the estimation results of the attitude estimation unit 15b. Specifically, the calibration execution unit 15c compares the angle estimate value estimated by the attitude estimation unit 15b with the design value included in the installation information 14b and calculates the error.

[0057] If the calculated error is within tolerance, the calibration execution unit 15c notifies the external device 50 of the calibration value. The external device 50 is, for example, various devices that implement parking frame detection and automatic parking functions. The phrase "error within tolerance" means that the camera 11 is not misaligned.

[0058] If the calculated error exceeds the tolerance, the calibration execution unit 15c notifies the external device 50 of the calibration value and causes the external device 50 to stop the parking frame detection and automatic parking functions. The phrase "error exceeds tolerance" means that the camera 11 has axis misalignment.

[0059] The calibration execution unit 15c also notifies the notification device 13 of the calibration execution results. If necessary, based on the notification, the user will adjust the installation angle of the camera 11 at a dealer or other location.

[0060] The following will refer to Figure 8 Describe the process executed by the vehicle-mounted device 10. Figure 8 This is a flowchart illustrating the process performed by the vehicle-mounted device 10 according to an embodiment.

[0061] like Figure 8 As shown, the control unit 15 of the vehicle-mounted device 10 determines whether the camera 11 is in the early stage after installation (step S101). If the camera 11 is in the early stage after installation (step S101 is "yes"), the control unit 15 sets the attitude estimation mode to the first mode (step S102).

[0062] Then, the control unit 15 uses the optical flow of the rectangular ROI 30-1 to perform attitude estimation processing (step S103). If the camera 11 is not in the early stage after installation (step S101 is "No"), the control unit 15 sets the attitude estimation mode to the second mode (step S104).

[0063] Then, the control unit 15 uses the optical flow of the road surface ROI 30-2 in the rectangular ROI 30-1 to perform attitude estimation processing (step S103). The control unit 15 performs calibration based on the attitude estimation processing results of step S103 or step S105 (step S106).

[0064] Control unit 15 determines whether a processing end event exists (step S107). A processing end event may be, for example, a non-execution period of attitude estimation processing, engine shutdown, or power failure. If no processing end event occurs (step S107 is "No"), control unit 15 repeats the process starting from step S101. If a processing end event occurs (step S107 is "Yes"), control unit 15 terminates the process.

[0065] As described above, the vehicle-mounted device 10 according to the embodiment (corresponding to an example of an "information processing device") includes a control unit 15 (corresponding to an example of a "controller"). The control unit 15 performs attitude estimation processing to estimate the attitude of the camera 11 based on the optical flow of feature points in a ROI 30 (corresponding to an example of a "region of interest") set in an image captured by the camera 11 (corresponding to an example of a "vehicle-mounted camera"). When the camera 11 is mounted in a first state, the control unit 15 performs a first attitude estimation process using a rectangular ROI 30-1 (corresponding to an example of a "first region of interest") set to a rectangular shape, and when the camera 11 is mounted in a second state, the control unit 15 performs a second attitude estimation process using a superimposed ROI 30-S (corresponding to an example of a "second region of interest") set according to the road surface shape.

[0066] Therefore, by using the vehicle-mounted device 10 according to the embodiment, the accuracy of the attitude estimation of the camera 11 can be improved.

[0067] The control unit 15 uses superimposed ROIs 30-S set to a trapezoidal shape to perform the second attitude estimation process.

[0068] Therefore, by using the vehicle-mounted device 10 according to the embodiment, erroneous flow can be prevented, and on this basis, the accuracy of attitude estimation of the camera 11 can be improved.

[0069] The control unit 15 sets the superimposed ROI 30-S as a trapezoidal region obtained by removing the region other than the following region from the rectangular ROI 30-1: the region in the captured image that corresponds to the road surface shape that converges toward the vanishing point.

[0070] Therefore, using the vehicle-mounted device 10 according to the embodiment, the superimposed ROI 30-S can be set as a region of interest based on the road surface shape that converges toward the vanishing point.

[0071] Based on calibration values ​​related to the mounting of the camera 11, the control unit 15 sets the road surface ROI 30-2 (corresponding to the example of "third region of interest") according to the road surface shape in the captured image. The calibration values ​​related to the mounting of the camera 11 are obtained by performing a first pose estimation process, and the superimposed portion where the road surface ROI 30-2 and the rectangular ROI 30-1 overlap is set as the superimposed ROI 30-S.

[0072] Therefore, by using the vehicle-mounted device 10 according to the embodiment, the accuracy of the pose estimation of the camera 11 can be improved, while the advantages of the other can compensate for the corresponding disadvantages of using rectangular ROI 30-1 and using superimposed ROI 30-S.

[0073] The control unit 15 extracts a set of line segment pairs from the ROI 30 based on optical flow, and estimates the rotation angles of the pan, pitch, and roll axes of the camera 11 based on each of the line segment pairs.

[0074] Therefore, using the vehicle-mounted device 10 according to the embodiment, the rotation angles of the pan, pitch, and roll axes of the camera 11 can be estimated with high accuracy based on each pair of line segments with fewer error flows.

[0075] After generating a histogram of each of the estimated rotation angles, the control unit 15 determines the estimated angles for the pan, pitch, and roll axes based on the median values.

[0076] Therefore, using the vehicle-mounted device 10 according to the embodiment, the angle estimates of the pan axis, pitch axis and roll axis can be determined with high accuracy based on the intermediate value of the rotation angle estimated with high accuracy.

[0077] The control unit 15 determines the axis misalignment of the camera 11 based on the determined angle estimate.

[0078] Therefore, by using the vehicle-mounted device 10 according to the embodiment, the axis misalignment of the camera 11 can be determined with high accuracy based on the angle estimate.

[0079] When shaft misalignment is detected, control unit 15 stops at least one of the parking frame detection function or the automatic parking function.

[0080] Therefore, by using the vehicle-mounted device 10 according to the embodiment, operational errors in the parking frame detection function or automatic parking function can be prevented at least based on the axle misalignment determined with high accuracy.

[0081] The attitude estimation method according to an embodiment is an information processing method performed by an on-board device 10, and includes: performing attitude estimation processing to estimate the attitude of the camera 11 based on the optical flow of feature points in a region of interest (ROI) 30 set in an image captured by the camera 11. The attitude estimation method according to an embodiment further includes: performing a first attitude estimation process using a rectangular ROI 30-1 set to a rectangular shape when the camera 11 is mounted in a first state, and performing a second attitude estimation process using superimposed ROIs 30-S set according to the road surface shape when the camera 11 is mounted in a second state.

[0082] Therefore, by using the attitude estimation method according to the embodiment, the accuracy of attitude estimation of camera 11 can be improved.

[0083] The computer program according to an embodiment causes a computer to perform pose estimation processing to estimate the pose of the camera 11 based on the optical flow of feature points in a region of interest (ROI) 30 set in an image captured by the camera 11. The computer program according to an embodiment also causes the computer to: perform a first pose estimation process using a rectangular ROI 30-1 set to a rectangular shape when the camera 11 is mounted in a first state, and perform a second pose estimation process using superimposed ROIs 30-S set according to the road surface shape when the camera 11 is mounted in a second state.

[0084] Therefore, by utilizing the computer program according to the embodiment, the accuracy of attitude estimation of camera 11 can be improved. The computer program according to the embodiment can be recorded on a computer-readable recording medium, such as a hard disk, floppy disk (FD), CD-ROM, magneto-optical disk (MO), digital multifunction disc (DVD), and universal serial bus (USB) memory, and can be read from the recording medium by a computer for execution. A recording medium storing the program is also an embodiment of this disclosure.

Claims

1. An information processing apparatus, comprising: The controller is configured to estimate the pose of the vehicle-mounted camera based on images captured by the camera, wherein The controller is also configured to: When the installation state of the vehicle-mounted camera is in a first state where the calibration value is unknown, a first attitude estimation process is performed, including the following operations: Set a first region of interest in the shape of a rectangle in the captured image; The first calibration value is calculated based on the optical flow of feature points in the first region of interest. as well as Estimate the orientation of the vehicle-mounted camera; and When the installation state of the vehicle-mounted camera is a second state in which the calibration value is known, a second attitude estimation process is performed, comprising the following operations, wherein the calibration value is pre-calculated in the first attitude estimation process: By using known calibration values, a second region of interest corresponding to the road surface shape is set in the captured image, wherein the shape of the second region of interest is different from that of the first region of interest; The second calibration value is calculated based on the optical flow of feature points in the overlapping portion of the first and second regions of interest; and Estimate the orientation of the vehicle-mounted camera.

2. The information processing apparatus according to claim 1, wherein, The second region of interest is trapezoidal in shape.

3. The information processing apparatus according to claim 2, wherein, The second region of interest has the following shape: based on the shape of the road surface in the captured image that converges toward the vanishing point.

4. The information processing apparatus according to any one of claims 1 to 3, wherein, The controller is also configured to: Based on the optical flow, a pair of line segments is extracted from each of the first region of interest and the second region of interest; and The rotation angles of the panning axis, pitch axis, and roll axis of the vehicle-mounted camera are estimated based on each pair of the line segments as the first calibration value and the second calibration value.

5. The information processing apparatus according to claim 4, wherein, After generating a histogram of each of the estimated rotation angles, the controller determines the angle estimates for the pan, pitch, and roll axes based on the median values.

6. The information processing apparatus according to claim 5, wherein, The controller determines the axis misalignment of the vehicle-mounted camera based on the determined angle estimate.

7. The information processing apparatus according to claim 6, wherein, When the controller determines that the shaft is misaligned, it stops at least one of the parking frame detection function or the automatic parking function.

8. An information processing method executed by an information processing device, the information processing method comprising: Acquire images captured by an in-vehicle camera; When the installation state of the vehicle-mounted camera is in a first state where the calibration value is unknown, a first attitude estimation process is performed, including the following operations: Set a first region of interest in the shape of a rectangle in the captured image; The first calibration value is calculated based on the optical flow of feature points in the first region of interest. as well as Estimate the orientation of the vehicle-mounted camera; When the installation state of the vehicle-mounted camera is a second state in which the calibration value is known, a second attitude estimation process is performed, comprising the following operations, wherein the calibration value is pre-calculated in the first attitude estimation process: By using known calibration values, a second region of interest corresponding to the road surface shape is set in the captured image, wherein the shape of the second region of interest is different from that of the first region of interest; The second calibration value is calculated based on the optical flow of feature points in the overlapping portion of the first and second regions of interest; and Estimate the orientation of the vehicle-mounted camera.

9. A computer-readable recording medium having a program stored thereon, the program causing a computer to perform a process, the process comprising: Acquire images captured by an in-vehicle camera; When the installation state of the vehicle-mounted camera is in a first state where the calibration value is unknown, a first attitude estimation process is performed, including the following operations: Set a first region of interest in the shape of a rectangle in the captured image; The first calibration value is calculated based on the optical flow of feature points in the first region of interest. as well as Estimate the orientation of the vehicle-mounted camera; When the installation state of the vehicle-mounted camera is a second state in which the calibration value is known, a second attitude estimation process is performed, comprising the following operations, wherein the calibration value is pre-calculated in the first attitude estimation process: By using known calibration values, a second region of interest corresponding to the road surface shape is set in the captured image, wherein the shape of the second region of interest is different from that of the first region of interest; The second calibration value is calculated based on the optical flow of feature points in the overlapping portion of the first and second regions of interest; and Estimate the orientation of the vehicle-mounted camera.

Images