State estimation device and state estimation method
The state estimation device and method improve accuracy by incorporating a Coriolis term to align inertial and Earth-fixed coordinate systems, addressing deviations in existing technologies and enhancing position, velocity, and attitude estimation.
Patent Information
- Application Number
- JP2025078043
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2026-11-19
AI Technical Summary
Existing state estimation technologies for moving objects, such as vehicles, face challenges in improving the accuracy of position, velocity, and attitude estimation due to deviations caused by the Earth's rotation, particularly in integrating inertial measurement unit outputs with camera-based visual data.
A state estimation device and method that incorporates a Coriolis term in the optimization process to align inertial and Earth-fixed coordinate systems, utilizing image data from a camera and inertial data from an IMU to enhance accuracy by integrating a Coriolis force calculation, thereby improving the estimation of position, velocity, and attitude.
The integration of a Coriolis term in the optimization process enhances the accuracy of position, velocity, and attitude estimation by aligning coordinate systems, leading to improved estimation of vehicle state parameters.
Smart Images

Figure 2026182107000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a state estimation device that estimates a state including at least one of the position, velocity, and attitude of a moving object.
Background Art
[0002] Various techniques have been conventionally proposed for estimating a state including at least one of the position, velocity, and attitude of a moving object such as a vehicle equipped with a camera and an inertial measurement unit (see, for example, Patent Document 1). The inertial measurement unit is a device that detects three-dimensional inertial motion, and outputs translational motion and rotational motion in three orthogonal axis directions as inertial data.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of Invention
Problems to be Solved by the Invention
[0004] In this type of technology, further improvement of estimation accuracy is being pursued. The present disclosure has been made in view of the circumstances and the like exemplified above.
Means for Solving the Problems
[0005] According to one aspect of the present disclosure, a state estimation device (5) that estimates a state including at least one of the position, velocity, and attitude of a moving object (1) comprises: an image data acquisition unit (501) that acquires image data from an imaging device (3); a first estimation unit (503) that calculates a first estimated value corresponding to the estimated value of the state based on the acquired image data; an inertial data acquisition unit (502) that acquires inertial data from an inertial measurement unit (4); A second estimation unit (504) calculates a second estimated value corresponding to the estimated value of the state based on the acquired inertia data, A Coriolis force calculation unit (505) calculates a Coriolis term corresponding to the Coriolis force based on the acquired inertia data, A final estimation unit (506) calculates a final estimate, which is the final estimate of the state, based on the first estimate, the second estimate, and the Coriolis term. It is equipped with. According to another aspect of this disclosure, a state estimation method for estimating a state including at least one of the position, velocity, and orientation of a moving object (1) is: Image data is acquired from the imaging device (3), Based on the acquired image data, a first estimated value corresponding to the estimated value of the state is calculated. Inertial data is acquired from the inertial measurement device (4), Based on the acquired inertia data, a second estimated value corresponding to the estimated value of the state is calculated. Based on the acquired inertial data, the Coriolis term corresponding to the Coriolis force is calculated. Based on the first estimate, the second estimate, and the Coriolis term, a final estimate, which is the final estimate of the state, is calculated.
[0006] In addition, each element in the application documents may be denoted by a reference numeral in parentheses. However, such reference numerals merely indicate one example of the correspondence between the element and the specific means described in the embodiments below. Therefore, this disclosure is not limited in any way by the notation of the above reference numerals. [Brief explanation of the drawing]
[0007] [Figure 1] This is a plan view showing a schematic configuration of a vehicle as a mobile body to which one embodiment of the present disclosure is applied. [Figure 2] This block diagram shows the schematic functional configuration realized by the state estimation device shown in Figure 1. [Figure 3]Figure 2 is a conceptual diagram illustrating the general operation of the state estimation device. [Figure 4] Figure 2 is a conceptual diagram illustrating the general operation of the state estimation device. [Modes for carrying out the invention]
[0008] (Embodiment) Hereinafter, exemplary embodiments or specific examples of the present disclosure will be described with reference to the drawings as appropriate. Note that the following embodiments, their modifications, and the corresponding drawings are schematic or simplified for the purpose of concisely explaining the contents of the present disclosure, and are not intended to limit the contents of the present disclosure in any way. Therefore, it goes without saying that the descriptions in the drawings do not necessarily correspond to the specific device configurations actually manufactured and sold. In other words, it goes without saying that the present disclosure should not be interpreted restrictively based on the descriptions in the drawings and the corresponding descriptions of the device configurations, functions, or operations described below.
[0009] (Overall structure) First, referring to Figure 1, the vehicle 1 as a mobile body according to this embodiment has the basic configuration of a regular automobile that travels on roads, and is equipped with a box-shaped body 2 and a drive force generation mechanism, a drive force transmission mechanism, etc. inside it. In addition, the body 2 is equipped with an imaging device 3, an inertial measurement device 4, and a state estimation device 5.
[0010] The imaging device 3 generates image data by imaging the area around the vehicle 1 and is configured to output the generated image data to the state estimation device 5. In this embodiment, the imaging device 3 has a configuration as a so-called visible light camera equipped with an image sensor such as a CCD or CMOS. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary Metal Oxide Semiconductor.
[0011] The inertial measurement device 4 is provided to detect three-dimensional inertial motion of the vehicle 1, specifically translational motion and rotational motion in three orthogonal axis directions, and output inertial data as the detection result to the state estimation device 5. The inertial measurement device 4 may also be referred to as an IMU. IMU is an abbreviation for Inertial Measurement Unit.
[0012] More specifically, the inertial measurement device 4 includes a gyro sensor that detects angular velocities ωx, ωy, ωz of the vehicle 1 as the rotational motion, and an acceleration sensor that detects accelerations fx, fy, fz of the vehicle 1 as the translational motion. In the present embodiment, the inertial measurement device 4 has a configuration as a small MEMS-based IMU. MEMS is an abbreviation for Micro Electro Mechanical Systems.
[0013] The state estimation device 5 is configured to estimate a state including at least one of the position, velocity, and attitude of the vehicle 1 based on the image data acquired from the imaging device 3 and the inertial data acquired from the inertial measurement device 4. That is, in the present embodiment, the state estimation device 5 is configured to estimate the vehicle state using the technology of visual-inertial odometry (i.e., VIO). VIO is an abbreviation for Visual Inertial Odometry. Hereinafter, the state of the vehicle 1 that is the aforementioned estimation target by the state estimation device 5 is abbreviated as "vehicle state".
[0014] In the present embodiment, the state estimation device 5 has a configuration as a so-called ECU. ECU is an abbreviation for Electronic Control Unit. Specifically, the state estimation device 5 has a configuration as a so-called in-vehicle computer including at least an arithmetic device 51 and a storage device 52.
[0015] The arithmetic device 51 is configured by at least one or more CPUs, MPUs, or the like. CPU is an abbreviation for Central Processing Unit. MPU is an abbreviation for Micro Processor Unit.
[0016] The storage device 52 is configured by at least one or more non-transitory tangible storage media such as ROM and nonvolatile rewritable memory. ROM is an abbreviation for Read Only Memory. The nonvolatile rewritable memory is configured such that information can be rewritten while power is on, while information is retained in a non-rewritable state while power is off, and examples thereof include flash ROM and magnetic recording devices.
[0017] The storage device 52 stores a computer program, as well as various data such as initial values, maps, and lookup tables necessary for executing the program. That is, the state estimation device 5 is configured to perform the vehicle state estimation operation by the arithmetic device 51 reading and executing the state estimation program stored in the storage device 52.
[0018] (State Estimation Device) Figure 2 shows an example of a functional block configuration realized by the arithmetic device 51 provided in the state estimation device 5 according to the present disclosure reading a vehicle state estimation program, which is a computer program, from the storage device 52 and executing the program. Hereinafter, an outline of the functional configuration of the state estimation device 5 according to the present embodiment will be described with reference to Figure 2.
[0019] As shown in Figure 2, the state estimation device 5 includes, as functional configurations realized by executing a computer program, an image data acquisition unit 501, an inertial data acquisition unit 502, a first estimation unit 503, a second estimation unit 504, a Coriolis force calculation unit 505, a final estimation unit 506, and an absolute azimuth calculation unit 507. Note that these functional configurations are provided for convenience to describe the functions performed by the state estimation device 5, and do not imply that the corresponding software routines or hardware units must exist separately and independently of each other.
[0020] The image data acquisition unit 501 acquires, or receives, image data from the imaging device 3 and temporarily stores it. The inertial data acquisition unit 502 acquires, or receives, inertial data from the inertial measurement device 4 and temporarily stores it.
[0021] The first estimation unit 503 calculates a first estimated value corresponding to the estimated value of the vehicle state based on the image data acquired by the image data acquisition unit 501. Specifically, the first estimation unit 503 has the function of an image processing unit that extracts feature points from the image data and tracks those feature points. That is, the first estimated value is the time trajectory of at least one of the position, speed, and attitude angle of the vehicle 1, calculated by image processing.
[0022] Such an image processing unit extracts feature points based on local feature quantities, for example, using SIFT or SURF, and then associates the feature points extracted in the current image frame with the feature points extracted in the previous image frame using nearest neighbor search or similar methods. SIFT is an abbreviation for Scale Invariant Feature Transform. SURF is an abbreviation for Speeded-Up Robust Features. Note that the extraction and matching of feature points in the image processing unit may be achieved by means other than those described above.
[0023] The second estimation unit 504 calculates a second estimated value corresponding to the estimated value of the vehicle state based on the inertial data acquired by the inertial data acquisition unit 502. Specifically, the second estimation unit 504 calculates the time trajectory of at least one of the position, velocity, and attitude angle of the vehicle 1 as a second estimated value by integrating the acceleration and angular velocity included in the inertial data. The second estimation unit 504 may also be referred to as the inertial processing unit.
[0024] More specifically, the second estimation unit 504 integrates the angular velocity, which is the sensor output of the gyro sensor, to determine the attitude and rotation matrix of the vehicle 1. The second estimation unit 504 also integrates the product of the acceleration, which is the sensor output of the acceleration sensor, and the rotation matrix to determine the velocity of the vehicle 1, and then integrates the determined velocity of the vehicle 1 to determine the position of the vehicle 1. The second estimation unit 504 calculates three attitude angles: roll angle, pitch angle, and yaw angle.
[0025] The functional configurations of the image data acquisition unit 501, the inertial data acquisition unit 502, the first estimation unit 503, and the second estimation unit 504 are basically the same as those of the prior art, including Patent Document 1, and are already known or well-known at the time of filing this application. Therefore, further details regarding these functional configurations will be omitted. For an overview of visual inertial odometry, please also refer to Non-Patent Document 1 below.
[0026] [Non-patent document 1] T.Qin, P.Li and S.Shen, "VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator," in IEEE Transactions on Robotics,vol.34,no.4,pp.1004-1020,Aug.2018,doi:10.1109 / TRO.2018.2853729.
[0027] The Coriolis force calculation unit 505 calculates a Coriolis term, which corresponds to the Coriolis force, based on the inertial data acquired by the inertial data acquisition unit 502. The Coriolis term is a term used to suppress the decrease in estimation accuracy due to the deviation of the estimated vehicle state between the inertial coordinate system and the Earth-fixed coordinate system caused by the Earth's rotation. Details of the Coriolis term and the background for its use will be explained in detail later in the description of the operation overview.
[0028] The final estimation unit 506 calculates a final estimate, which is an estimate of the final vehicle state, based on the first estimate, the second estimate, and the Coriolis term. Specifically, the final estimation unit 506 calculates the final estimate using visual inertial odometry based on the tracking results of feature points extracted from the camera image and the measurement results of inertial data by the inertial measurement device 4. The final estimate includes at least one of the following: position, velocity, attitude angle, bias error, depth to feature points, etc.
[0029] The final estimation unit 506 performs an optimization calculation to minimize the bias error between the visual odometry calculation result by the first estimation unit 503 and the inertial odometry calculation result by the second estimation unit 504, and is therefore related to the functional configuration of the optimization calculation in the prior art, such as in Patent Document 1. However, the final estimation unit 506 in this embodiment differs significantly from the prior art in that it uses a Coriolis term in such an optimization calculation. Details of this optimization calculation will be explained later when describing the overview of operation.
[0030] The absolute bearing calculation unit 507 estimates the absolute bearing angle as a vehicle state based on the final estimated value calculated by the final estimation unit 506. In other words, the absolute bearing calculation unit 507 calculates the absolute bearing angle from the gravitational acceleration term and the Earth's rotational angular velocity term calculated by the final estimation unit 506. Details of this calculation will be explained later in the description of the operation overview.
[0031] (Operation overview) The following describes the operation of the apparatus configuration according to this embodiment, as well as the execution of the methods and programs according to this embodiment, along with the problems of the prior art and the effects achieved by this embodiment. In the following description, the state estimation apparatus 5 according to this embodiment, and the state estimation method and state estimation program executed thereunder, may be collectively referred to as "this embodiment."
[0032] As is well known, in conventional technology, the approximate vehicle state is calculated by integrating the IMU output, and then this is combined with the result of calculating the vehicle state by image processing of the camera output and optimized to calculate a relatively accurate vehicle state. In this case, the integration of the IMU output is performed in an inertial frame that is different from the coordinate system of the camera output, and deviates from the integral value in the Earth-fixed coordinate system, which is the coordinate system of the camera output.
[0033] Specifically, Figure 3 shows the movement trajectory of vehicle 1 obtained from the IMU output. On the other hand, Figure 4 shows the movement trajectory of vehicle 1 obtained from the camera output. In Figures 3 and 4, t1, t2, and t3 represent time, respectively. Linear arrows indicate acceleration at each time, and arc arrows indicate angular velocity at each time.
[0034] Due to the Earth's rotation, the inertial coordinate system rotates relative to the Earth-fixed coordinate system. Therefore, the trajectory of movement in inertial space differs from the trajectory of movement in the Earth-fixed coordinate system. That is, as shown in Figure 4, due to the rotation effect, the dashed trajectory obtained from the camera output appears to be twisted by the Coriolis force, resulting in less rotation than the original dashed trajectory. This trajectory deviation tends to increase with increasing speed and integration time.
[0035] Thus, conventional visual inertia odometry still had room for improvement in terms of the accuracy of optimization calculations. Here, in order to unify the coordinates, it is necessary to consider Coriolis acceleration or construct an attitude system using absolute orientation. Therefore, this embodiment estimates absolute orientation by using a Coriolis term corresponding to the Coriolis force in the optimization calculation, and by considering that absolute orientation, the accuracy of position / velocity / attitude angle estimation is improved.
[0036] Specifically, the operation of each functional component in the state estimation device 5 will be explained below. First, the image data acquisition unit 501 acquires image data from the imaging device 3. In addition, the inertial data acquisition unit 502 acquires inertial data from the inertial measurement device 4.
[0037] The first estimation unit 503 calculates a first estimated value corresponding to the estimated value of the vehicle state based on the image data acquired by the image data acquisition unit 501. Specifically, the first estimation unit 503 detects feature points and assigns the time of detection to the feature points. Furthermore, the first estimation unit 503 performs feature point tracking processing using the feature points detected in the previous sampling and the image obtained in the current sampling, and links the previous sampling and the current sampling (i.e., assigns the same feature point ID). The above is repeated for a predetermined time. In this way, the first estimation unit 503 has the functions of a feature point detection unit, a time tag assignment unit, a feature point tracking unit, and a feature point information buffering unit.
[0038] The second estimation unit 504 calculates a second estimated value corresponding to the estimated value of the vehicle state based on the inertial data acquired by the inertial data acquisition unit 502. Specifically, the second estimation unit 504 integrates the inertial data and generates a time trajectory of the vehicle state over the same predetermined period of time as described above. The Coriolis force calculation unit 505 calculates a Coriolis term corresponding to the Coriolis force based on the inertial data acquired by the inertial data acquisition unit 502.
[0039] The final estimation unit 506 calculates a final estimated value, which is the final estimated value of the vehicle state, based on the first estimated value calculated by the first estimation unit 503, the second estimated value calculated by the second estimation unit 504, and the Coriolis term calculated by the Coriolis force calculation unit 505. That is, the final estimation unit 506 performs an optimization calculation using feature point information, the time trajectory of the vehicle state based on inertial data, and the Coriolis force to obtain the vehicle state, such as the position, velocity, attitude angle, bias error, and depth to the feature points of vehicle 1. At the same time, it also obtains the gravitational acceleration and the angular velocity of the Earth's rotation in the inertial frame. The absolute direction calculation unit 507 calculates the absolute direction from the gravitational acceleration and the angular velocity of the Earth's rotation in the inertial frame.
[0040] The vehicle state estimation in this embodiment, particularly the details of the Coriolis term, will be explained below using mathematical formulas.
[0041] In this embodiment, when integrating the IMU acceleration output, instead of simply integrating the acceleration, the calculation is performed using the following equation (1), which includes a term due to the Coriolis force. The coordinate transformation matrix in equation (1) corresponds to the Coriolis term, and determining this is equivalent to determining the absolute direction.
number
[0042] In this embodiment, in addition to the conventional VIO output, i.e., position, velocity, and attitude angle, this coordinate transformation matrix is estimated. For estimation, the following equation derivation uses the expressions of equations (2) and (3) below.
number
number
[0043] Integrating and rearranging equation (1) above, the positional displacement T can be accurately approximated by equation (4) below. Here, the first term on the right-hand side is the IMU output f b The first term is the position obtained by integrating only the first term, the second term is a linear polynomial relating to the acceleration bias, the third term is a linear polynomial relating to the acceleration due to gravity, and the fourth term is a linear polynomial relating to the angular velocity of the Earth's rotation.
number
[0044] In conventional VIOs, as described in Non-Patent Document 1 above, it was common to formulate and solve the optimization problem using position, velocity, attitude angle, acceleration bias, angular velocity bias, and depth to feature points as target variables. In this embodiment, however, the gravitational acceleration and Earth's rotational angular velocity are further added as target variables for optimization, and the gravitational acceleration g in the third term of the right-hand side of equation (4) and the Earth's rotational angular velocity ω in the fourth term of the same equation are determined by solving the optimization problem.
[0045] The coordinate transformation matrix can be derived from the obtained gravitational acceleration term g and Earth rotation angular velocity term ω. That is, the gravitational acceleration g in the Earth-fixed coordinate system is known, and similarly, the Earth rotation angular velocity ω is also known, and the gravitational acceleration term g and the Earth rotation angular velocity term ω are calculated by optimization. Therefore, the only unknown variable is the term in the coordinate transformation matrix, which can be determined using methods such as the Davenport Q-method. The derived coordinate transformation matrix becomes a variable representing absolute direction.
[0046] In the optimization calculation described above, the accuracy of position, velocity, and attitude angle is already improved because the optimization is performed while taking into account the shift in the coordinate system. Therefore, according to this embodiment, compared to the conventional technology that does not take the Coriolis force into consideration, the accuracy of estimating the depth to the position, velocity, and attitude angle feature points is improved, and it is also possible to estimate the absolute azimuth angle at the same time. Thus, according to this embodiment, it is possible to further improve the accuracy of estimating the vehicle state compared to the conventional technology.
[0047] (modified version) This disclosure is not limited to the embodiments and specific examples described above. Therefore, the embodiments, etc. can be modified as appropriate. Representative modifications are described below. In the description of the modifications below, the differences from the embodiments, etc. will be mainly described. In addition, parts that are the same or equivalent to each other in the embodiments, etc. and the modifications below are denoted by the same reference numerals. Therefore, in the description of the modifications below, with respect to components that have the same reference numerals as in the embodiments, etc., the descriptions in the embodiments, etc. can be appropriately referenced unless there is a technical inconsistency or special additional explanation.
[0048] This disclosure is not limited to the specific uses or device configurations shown in the embodiments described above. For example, Vehicle 1 may be a so-called ordinary automobile, a heavy automobile, a two-wheeled vehicle, or a three-wheeled vehicle. Furthermore, Vehicle 1 may be used not on public roads but within a specific site such as a factory. In other words, there are no particular limitations on the type of Vehicle 1, its use, the shape and size of the vehicle body 2, etc. Also, the scope of application of this disclosure is not limited to Vehicle 1, but may include flying objects such as drones.
[0049] The imaging device 3 may be a so-called monocular camera or a compound camera. Furthermore, the imaging device 3 is not limited to a so-called visible light camera, but may also have sensitivity in the infrared region.
[0050] The scope of application of this disclosure is not limited to VIO, but may also be LIO or VLIO. That is, the imaging device 3 is not limited to a camera, but may also be a LiDAR. LIO is an abbreviation for LiDAR-Inertial Odometry. VLIO is an abbreviation for Visual-LiDAR Inertial Odometry. LiDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging.
[0051] The absolute bearing calculated and output by the absolute bearing calculation unit 507 can be referred to as the "final estimated value," similar to the final estimated value (e.g., position, velocity, attitude angle, etc.) calculated and output by the final estimation unit 506. Therefore, the final estimation unit 506 and the absolute bearing calculation unit 507 can be evaluated as a single unit. Accordingly, this disclosure is not limited to a configuration in which the final estimation unit 506 and the absolute bearing calculation unit 507 are provided as separate and independent functional configurations or functional blocks.
[0052] The state estimation device 5 may be an ECU for controlling the operation of the vehicle. Such an ECU may also be called an operation control ECU, but it may be an operation assistance ECU or an automated driving ECU. An operation assistance ECU is an ECU configured to achieve Level 1 or 2 of the operation automation level specified in the standard "SAE J3016" published by SAE International. An automated driving ECU is an ECU configured to achieve any of the operation automation levels 3 to 5 specified in the same standard.
[0053] The state estimation device 5, in whole or in part, may be configured to include a digital circuit, such as an ASIC or FPGA, that is capable of realizing the functions or operations described above. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array. In other words, the in-vehicle microcomputer part and the digital circuit part can coexist in the state estimation device 5.
[0054] The computer program according to this disclosure, which enables the execution of various operations, procedures, or processes described in the above embodiments, can be downloaded or upgraded to a non-volatile rewritable memory via V2X communication using a communication device. V2X is an abbreviation for Vehicle to X. Alternatively, such a computer program can be downloaded or upgraded via terminal equipment installed at the vehicle's manufacturing plant, repair shop, dealership, etc. The non-volatile rewritable memory where such a computer program is stored is not limited to semiconductor memory such as flash memory, but may also be an optical or magnetic recording medium, etc.
[0055] Thus, each of the above functional configurations and processes may be realized by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. Alternatively, each of the above functional configurations and processes may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, each of the above functional configurations and processes may be realized by one or more dedicated computers configured by a combination of one or more processors programmed to perform one or more functions, one or more memories, and one or more other processors configured with one or more hardware logic circuits.
[0056] Furthermore, the computer program may be stored in a computer-readable, non-transitional, tangible storage medium as instructions executed by the computer. In other words, each of the above functional configurations and processes can be represented as a computer program including the procedures for realizing them, or as a non-transitional, tangible storage medium storing said computer program.
[0057] It goes without saying that the elements constituting the above embodiments are not necessarily essential unless explicitly stated to be particularly essential or considered to be fundamentally essential. Furthermore, when numerical values such as the number, numerical values, quantities, or ranges of components are mentioned, this disclosure is not limited to those specific numbers unless explicitly stated to be particularly essential or considered to be fundamentally limited to those specific numbers. Similarly, when the shape, orientation, positional relationship, etc., of components are mentioned, this disclosure is not limited to those shapes, orientations, positional relationships, etc., unless explicitly stated to be particularly essential or considered to be fundamentally limited to those specific shapes, orientations, positional relationships, etc.
[0058] Similar expressions such as "acquisition," "calculation," "estimation," "detection," and "detection" can be appropriately substituted for each other within the limits of what is technically consistent. Similarly, "exceeding the threshold" and "above the threshold" can be appropriately substituted for each other within the limits of what is technically consistent. The same applies to "below the threshold" and "below the threshold."
[0059] Modifications are not limited to the examples given above. For example, all or part of one of the modifications may be combined with all or part of another, provided that it does not conflict with the technical specifications. Furthermore, all or part of the specific examples may be combined with all or part of the modifications, provided that it does not conflict with the technical specifications.
[0060] (Disclosure perspective) As is evident from the above-described descriptions of the configuration and operation relating to the embodiments and modifications, this disclosure includes at least the following aspects:
[0061] [Perspective 1-1] A state estimation device (5) for estimating a state of a moving object (1) that includes at least one of its position, velocity, and attitude, Image data acquisition unit (501) acquires image data from imaging device (3), A first estimation unit (503) calculates a first estimated value corresponding to the estimated value of the state based on the acquired image data, An inertial data acquisition unit (502) acquires inertial data from the inertial measuring device (4), A second estimation unit (504) calculates a second estimated value corresponding to the estimated value of the state based on the acquired inertia data, A Coriolis force calculation unit (505) calculates a Coriolis term corresponding to the Coriolis force based on the acquired inertia data, A final estimation unit (506) calculates a final estimate, which is the final estimate of the state, based on the first estimate, the second estimate, and the Coriolis term. A state estimation device equipped with the following features. [Perspective 1-2] The system further includes an absolute azimuth calculation unit (507) that estimates the absolute azimuth angle as the state based on the aforementioned final estimate. The state estimation device described in perspective 1-1. [Perspectives 1-3] The final estimation unit calculates the final estimated value using visual inertia odometry based on the tracking results of feature points extracted from the camera image. A state estimation device as described in perspective 1-1 or perspective 1-2. [Perspectives 1-4] Mounted on the aforementioned vehicle as a mobile unit, A state estimation device described in any one of the viewpoints 1-1 to 1-3.
[0062] [Perspective 2-1] A state estimation method for estimating a state of a moving object (1) that includes at least one of its position, velocity, and attitude, Image data is acquired from the imaging device (3), Based on the acquired image data, a first estimated value corresponding to the estimated value of the state is calculated. Inertial data is acquired from the inertial measurement device (4), Based on the acquired inertia data, a second estimated value corresponding to the estimated value of the state is calculated. Based on the acquired inertial data, the Coriolis term corresponding to the Coriolis force is calculated. Based on the first estimate, the second estimate, and the Coriolis term, a final estimate, which is the final estimate of the state, is calculated. State estimation method. [Perspective 2-2] Based on the aforementioned final estimate, the absolute azimuth angle as the state is estimated. The state estimation method described in perspective 2-1. [Perspective 2-3] The final estimated value is calculated using visual inertia odometry based on the tracking results of feature points extracted from the camera image. The state estimation method described in perspective 2-1 or perspective 2-2. [Perspective 2-4] The aforementioned moving object is a vehicle. A state estimation method described in one of the following perspectives 2-1 to 2-3. [Explanation of symbols]
[0063] 1. Vehicle (mobile object) 3. Imaging device 4. Inertial Measurement Device 5. State Estimation Device 501 Image Data Acquisition Unit 502 Inertial Data Acquisition Unit 503 First state estimation unit 504 Second state estimation unit 505 Coriolis Force Calculation Unit 506 Final State Estimation Unit
Claims
1. A state estimation device (5) for estimating the state of a moving object (1), which includes at least one of the position, velocity, and attitude, Image data acquisition unit (501) acquires image data from imaging device (3), A first estimation unit (503) calculates a first estimated value corresponding to the estimated value of the state based on the acquired image data, An inertial data acquisition unit (502) acquires inertial data from the inertial measurement device (4), A second estimation unit (504) calculates a second estimated value corresponding to the estimated value of the state based on the acquired inertia data, A Coriolis force calculation unit (505) calculates a Coriolis term corresponding to the Coriolis force based on the acquired inertia data, A final estimation unit (506) calculates a final estimate, which is the final estimate of the state, based on the first estimate, the second estimate, and the Coriolis term. A state estimation device equipped with the following features.
2. The system further includes an absolute azimuth calculation unit (507) that estimates the absolute azimuth angle as the state based on the aforementioned final estimate. The state estimation device according to claim 1.
3. The final estimation unit calculates the final estimated value using visual inertia odometry based on the tracking results of feature points extracted from the camera image. The state estimation device according to claim 1.
4. Mounted on the aforementioned vehicle as a mobile unit, The state estimation device according to claim 1.
5. A state estimation method for estimating the state of a moving object (1), which includes at least one of its position, velocity, and attitude, Image data is acquired from the imaging device (3), Based on the acquired image data, a first estimated value corresponding to the estimated value of the state is calculated. Inertial data is acquired from the inertial measurement device (4), Based on the acquired inertia data, a second estimated value corresponding to the estimated value of the state is calculated. Based on the acquired inertial data, the Coriolis term corresponding to the Coriolis force is calculated. Based on the first estimate, the second estimate, and the Coriolis term, a final estimate, which is the final estimate of the state, is calculated. State estimation method.
6. Based on the aforementioned final estimate, the absolute azimuth angle as the state is estimated. The state estimation method according to claim 5.
7. The final estimated value is calculated using visual inertia odometry based on the tracking results of feature points extracted from the camera image. The state estimation method according to claim 5.
8. The aforementioned moving object is a vehicle. The state estimation method according to claim 5.
Citation Information
Patent Citations
State estimation device and state estimation method
JP2023183773A
