Orientation estimation device and orientation estimation method
The orientation estimation device uses a Kalman filter and smoothing techniques to enhance the accuracy of estimating surrounding moving object directions, addressing low accuracy issues in existing methods and improving systems like collision mitigation braking and automatic following.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- MITSUBISHI ELECTRIC CORP
- Filing Date
- 2024-08-27
- Publication Date
- 2026-07-03
AI Technical Summary
Existing methods for estimating the direction of surrounding moving objects, such as vehicles, suffer from low accuracy, particularly in distinguishing between forward and backward directions, leading to unreliable orientation estimation.
The orientation estimation device employs a Kalman filter to estimate the state of surrounding moving objects, a direction estimation unit to determine the orientation using observed and calculated velocities, and a smoothing unit to refine the estimated directions, incorporating features to handle low-speed and reversed orientations.
This approach significantly improves the accuracy of estimating the direction of surrounding moving objects, enhancing systems like collision mitigation braking and automatic following by providing precise orientation data.
Smart Images
Figure 0007884705000005 
Figure 0007884705000006 
Figure 0007884705000007
Abstract
Description
Technical Field
[0001] The disclosed technology relates to a direction estimation technology for estimating the direction of a surrounding moving object.
Background Art
[0002] For example, there are driving support technologies for moving objects such as a collision damage mitigation braking system and an automatic following system in a vehicle. In the driving support technology, the state of a surrounding moving object existing around the moving object may be estimated and used. Patent Document 1 discloses a method for estimating the direction of a moving object using image processing (image processing apparatus and image processing method). The image processing apparatus and image processing method of Patent Document 1 extract a target vehicle from an image by edge detection and estimate the direction of the vehicle from its symmetry. Specifically, when it has symmetry, it is estimated to be in the forward or backward direction, and when it has no symmetry, it is estimated to be in the lateral or diagonal direction. That is, the method shown in Patent Document 1 is a method for estimating the direction of a vehicle based on the symmetry of the shape of the vehicle in the image.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, the method of Patent Document 1 has a problem that it cannot specify whether the actual direction of the vehicle is either the forward or backward direction, and the accuracy of the direction as an estimation result is low.
[0005] The present disclosure aims to solve the above problems and improve the accuracy of estimating the direction of a surrounding moving object.
Means for Solving the Problems
[0006] The orientation estimation device disclosed herein is A Kalman filter unit that estimates the state of surrounding moving objects observed by a camera sensor in a moving object using a Kalman filter, A direction estimation unit estimates the direction of a moving object by taking as input the direction of the moving object observed by the camera sensor and the direction of the moving object calculated using the velocity estimated by the Kalman filter unit, A direction smoothing unit that smooths the direction of the surrounding moving object estimated by the direction estimation unit described above, It is equipped with these features. [Effects of the Invention]
[0007] This disclosure has the effect of improving the accuracy of estimating the orientation of surrounding moving objects. [Brief explanation of the drawing]
[0008] [Figure 1] Figure 1 is a diagram showing an example configuration of the orientation estimation device 100 according to Embodiment 1 of the present disclosure. [Figure 2] Figure 2 is a flowchart showing an example of the processing of the orientation estimation device 100 according to Embodiment 1 of this disclosure. [Figure 3] Figure 3 is a diagram showing an example of the detailed configuration of the orientation estimation device 100 (100A) according to Embodiment 1 of the present disclosure, and is a diagram showing an example of the configuration of an orientation estimation system including the orientation estimation device 100 (100A). [Figure 4] Figure 4 is a flowchart showing an example of processing in the orientation estimation device 100 (100A) according to Embodiment 1 of this disclosure. [Figure 5] Figure 5 is a flowchart showing an example of processing in the orientation estimation unit 120 according to Embodiment 1 of this disclosure (first example). [Figure 6] Figure 6 is a flowchart showing an example of processing in the orientation smoothing section 130 according to Embodiment 1 of this disclosure (second example). [Figure 7]Figure 7 shows a first example of a hardware configuration for realizing the functions of the configuration described herein. [Figure 8] Figure 8 shows a second example of a hardware configuration for realizing the functionality of the configuration described herein. [Modes for carrying out the invention]
[0009] To further illustrate this disclosure, embodiments of this disclosure will be described below with reference to the accompanying drawings.
[0010] Embodiment 1. Embodiment 1 describes a basic configuration example of the present disclosure and a more detailed configuration example.
[0011] An example of the configuration of the apparatus according to Embodiment 1 of this disclosure will be described. Figure 1 is a diagram showing an example configuration of the orientation estimation device 100 according to Embodiment 1 of the present disclosure. The orientation estimation device 100 estimates the orientation of surrounding moving objects that are present around the moving object. The orientation estimation device 100 shown in Figure 1 includes a Kalman filter unit 110, an orientation estimation unit 120, and an orientation smoothing unit 130.
[0012] The Kalman filter unit 110 has the function of estimating the state of the surrounding moving object. Specifically, the Kalman filter unit 110 estimates the state of the surrounding moving object observed by the camera sensor on the moving object using a Kalman filter.
[0013] The orientation estimation unit 120 has the function of estimating the orientation of the surrounding moving object using the estimation results from the Kalman filter unit 110. Specifically, the orientation estimation unit 120 estimates the orientation of the surrounding moving object by taking as input the orientation of the surrounding moving object observed by the camera sensor unit 200 and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter unit.
[0014] The orientation smoothing unit 130 has the function of smoothing the orientation of the surrounding moving object estimated by the orientation estimation unit. Specifically, the orientation smoothing unit 130 performs processing using the orientation of the surrounding moving object output by the orientation estimation unit 120 and outputs the orientation of the surrounding moving object output by the orientation estimation device 100.
[0015] An example of processing by the orientation estimation device according to Embodiment 1 of this disclosure will be described. Figure 2 is a flowchart showing an example of the processing of the orientation estimation device 100 according to Embodiment 1 of this disclosure. The process shown in Figure 2 is the method using the orientation estimation device 100. For example, the orientation estimation device 100 shown in Figure 1 starts the processing shown in Figure 2 when it becomes possible to process, such as when it receives a command from an external source or a processing signal from an external source. ("Start")
[0016] The orientation estimation device 100 then performs a state estimation process (step ST1000). In the state estimation process, the Kalman filter unit 110 of the orientation estimation device 100 estimates the state of the surrounding moving objects observed by the camera sensor unit 200 on the moving object using a Kalman filter.
[0017] The orientation estimation device 100 then performs orientation estimation processing (step ST2000). In the orientation estimation processing, the orientation estimation unit 120 of the orientation estimation device 100 estimates the orientation of the surrounding moving object using the orientation of the surrounding moving object observed by the camera sensor unit 200 and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter unit 110 as input.
[0018] The orientation estimation device 100 then performs orientation smoothing (step ST3000). In the orientation estimation process, the orientation smoothing unit 130 of the orientation estimation device 100 smooths the orientation of the surrounding moving object estimated by the orientation estimation unit 120.
[0019] The orientation estimation device 100 performs orientation smoothing and then outputs the orientation of the moving object after smoothing (moving object orientation information), after which it terminates the process ("terminate").
[0020] Next, an example of the configuration of the orientation estimation device 100(100A) and the mobile body mounting system 1 including the orientation estimation device 100(100A) when the orientation estimation device 100 is applied to a mobile body mounting device will be described. Figure 3 is a diagram showing an example of the detailed configuration of the orientation estimation device 100 (100A) according to Embodiment 1 of the present disclosure, and is a diagram showing an example of the configuration of an orientation estimation system including the orientation estimation device 100 (100A). The mobile vehicle-mounted system 1 assists the movement of the mobile vehicle by using the orientation of surrounding mobile vehicles. For example, the mobile vehicle-mounted system 1 functions as a system such as a collision damage mitigation braking system or an automatic following system. The mobile body mounting system 1 shown in Figure 3 is composed of a direction estimation device 100 (100A), a camera sensor unit 200, and an output destination device 300.
[0021] The camera sensor unit 200 is, for example, a camera mounted on a moving object that captures images in the direction of the moving object's movement. The camera sensor unit 200 is a camera sensor that observes surrounding moving objects (or, if the moving object is a vehicle, surrounding vehicles). Specifically, the camera sensor unit 200 observes surrounding moving objects that are present around the moving object within its shooting range as observation targets and outputs an image that includes the observed surrounding moving objects.
[0022] The output destination device 300 is a device that processes information using the orientation of the surrounding moving object, and in particular generates and outputs information that supports movement by the moving object. The output destination device 300 receives and processes the orientation of the surrounding moving object estimated by the orientation estimation device 100A.
[0023] The orientation estimation device 100A, with an improved configuration, estimates the orientation of surrounding moving objects located around the moving object. The orientation estimation device 100A shown in Figure 3 comprises a Kalman filter unit 110, an orientation estimation unit 120, and an orientation smoothing unit 130.
[0024] The Kalman filter unit 110, similar to the Kalman filter unit 110 already described, has the function of estimating the state of the surrounding moving objects observed by the camera sensor unit 200 in the moving object using a Kalman filter. The Kalman filter unit 110 shown in Figure 3 is composed of an initial value calculation unit 111, a smoothing unit 112, a prediction unit 113, and a delay unit 114.
[0025] The initial value calculation unit 111 has the function of calculating initial values from the observed values of the camera sensor unit 200. The smoothing unit 112 has the function of smoothing the state vector. The prediction unit 113 has the function of predicting the state vector. The delay unit 114 has the function of delaying the output value of the smoothing unit 112 by one cycle.
[0026] The orientation estimation unit 120, similar to the orientation estimation unit 120 already described, has the function of estimating the orientation of the surrounding moving object using the orientation of the surrounding moving object obtained using the orientation of the surrounding moving object observed by the camera sensor unit 200 and the velocity estimated by the Kalman filter unit 110.
[0027] The orientation estimation unit 120 shown in Figure 3 further outputs the orientation of the surrounding moving object observed by the camera sensor unit 200 if the surrounding moving object is moving at a low speed below a predetermined speed.
[0028] The orientation estimation unit 120 shown in Figure 3 further has a function to output the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter unit 110 if the surrounding moving object is not moving at a low speed below a predetermined speed, and the orientation of the surrounding moving object observed by the camera sensor unit 200 and the orientation of the surrounding moving object calculated from the velocity estimated by the Kalman filter unit 110 are reversed front to back.
[0029] The orientation estimation unit 120 shown in Figure 3 further has the function of outputting both the orientation of the surrounding moving object observed by the camera sensor unit 200 and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter unit 110, provided that the surrounding moving object is not moving at a low speed below a predetermined speed and the orientation of the surrounding moving object calculated from the orientation of the surrounding moving object observed by the camera sensor unit 200 and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter unit 110.
[0030] The orientation smoothing unit 130 has the function of smoothing the orientation of the surrounding moving object estimated by the orientation estimation unit 120.
[0031] The orientation smoothing unit 130 shown in Figure 3 further has the function of outputting the orientation of the peripheral moving body from the previous cycle when there is only one orientation of the peripheral moving body input from the orientation estimation unit 120, and the difference between the orientation of the peripheral moving body input from the orientation estimation unit 120 and the orientation of the peripheral moving body from the previous cycle is greater than a predetermined difference.
[0032] The orientation smoothing unit 130 shown in Figure 3 further has the function of calculating the orientation of the peripheral moving object using the orientation input from the orientation estimation unit 120 and the orientation of the peripheral moving object in the previous cycle, if there is only one orientation of the peripheral moving object input from the orientation estimation unit 120, and the difference between the orientation of the peripheral moving object input from the orientation estimation unit 120 and the orientation of the peripheral moving object in the previous cycle is smaller than a predetermined difference.
[0033] The orientation smoothing unit 130 shown in FIG. 3 further has a function of, when there are a plurality of orientations of the surrounding moving objects input from the orientation estimation unit 120, for each orientation of the surrounding moving objects input from the orientation estimation unit 120, outputting the orientation of the surrounding moving objects in the previous cycle when the difference between the orientation of the surrounding moving objects input from the orientation estimation unit 120 and the orientation of the surrounding moving objects in the previous cycle is greater than a predetermined difference, and calculating the orientation of the surrounding moving objects using the orientation of the surrounding moving objects input from the orientation estimation unit 120 and the orientation of the surrounding moving objects in the previous cycle when the difference between the orientation of the surrounding moving objects input from the orientation estimation unit 120 and the orientation of the surrounding moving objects in the previous cycle is less than the predetermined difference.
[0034] Next, an operation example (processing example) in the mobile body-mounted system 1 and the orientation estimation device 100A according to Embodiment 1 of the present disclosure will be described. First, the camera sensor unit 200 observes information such as an observation value z represented by the following formula (1). z=(x wp ,y wp ,v x ,v y ,a x ,θ o ,L,W) ···(1) Here, in formula (1), "x wp " is the longitudinal position of the target vehicle, "y wp " is the lateral position of the target vehicle, "v x " is the longitudinal ground speed of the target vehicle, "v y " is the lateral ground speed of the target vehicle, "a x " is the longitudinal ground acceleration of the target vehicle, "θ o " is the orientation obtained by image processing, "L" is the depth of the target vehicle, and "W" is the width of the target vehicle. Next, the Kalman filter unit will be described. First, the initial value calculation unit will be described. The initial value of the state vector and the initial value of the error covariance matrix are shown below. TIFF0007884705000001.tif75166 TIFF0007884705000002.tif77166
[0035] Next, the prediction unit 113 will be described. The prediction unit 113 uses the state vector xs obtained from the smoothing unit 112. k , error covariance matrix ps k from state vector xp k , error covariance matrix pp k We calculate this. Here, we use a linear motion model to make predictions. XP k =Fmat×xs k ...(8) pp k =Fmat×ps k ×Fmat t +Q ···(9) Here, in equations (8) and (9), “Fmat” is the state transition matrix, and “Q” is the error covariance matrix of process noise. The state transition matrix Fmat can be defined from the motion model as follows: TIFF0007884705000003.tif46166 Here, in equation (10), "dt" is the sampling time.
[0036] The delay unit 114 uses the state vector xp generated by the prediction unit 113. k , error covariance matrix pp k It has the function of delaying the signal by one cycle and inputting it to the smoothing unit 112.
[0037] Next, an example of the processing in the direction estimation unit 120 will be explained. Figure 4 is a flowchart showing an example of processing in the orientation estimation unit 120 according to Embodiment 1 of this disclosure. The direction estimation unit 120 starts processing, for example, when it receives the processing result from the Kalman filter unit 110. (Start) Direction “θ” v " is the vertical ground velocity vx generated by the Kalman filter section 110 k , lateral ground speed vy k It is generated from the following equation (11). TIFF0007884705000004.tif15166 The direction estimation unit 120 executes a process to determine whether the target vehicle is traveling at a certain speed or higher. (Step ST2010 "Is the target vehicle traveling at a certain speed or higher?") Step ST2010 determines whether the target vehicle is traveling at a certain speed or higher. When the vehicle is moving at a low speed, the velocity vector becomes more susceptible to noise. v The estimation accuracy may deteriorate. Therefore, if the vehicle in question is moving at a low speed, θ o This will be the output value. (Output θ0 (Step ST2020)) Furthermore, one example of a determination method in step ST2010 is to use the following formula (12). (vx k 2 +vy k 2 ) 1 / 2 >v move° ...(12) Here “v move° Let "" be an arbitrary threshold. Other methods include accumulating the above judgments at arbitrary intervals and adopting the one with the most occurrences, or updating the judgment when it occurs consecutively for an arbitrary period of time.
[0038] The orientation estimation unit 120 then performs a process to determine whether the target vehicle is reversed front to back (step ST2030 "Is the target vehicle reversed front to back?"). Step ST2030 has a function to determine whether the target vehicle is reversed front to back. Image processing can determine the orientation of the target vehicle, for example, whether it is facing forward or backward, or diagonally or sideways, but it cannot determine whether it is facing forward or backward. In other words, the orientation of the target vehicle obtained through image processing may be in the opposite direction to the actual orientation of the vehicle, i.e., it may differ by π rad. Therefore, in step ST2030, θ o and θ vIf the front-to-back direction is reversed, θ o Assuming the direction is reversed, θ v The output value is set to (if the target vehicle is reversed front to back (step ST2030 "YES"), θ v Output (Step ST2040)) Also, θ o and θ v The following is an example of how to determine if the front-to-back direction is reversed. |θ o -(θ v +π)|>reverse th ...(13) Here, in equation (13), “reverse th " is an arbitrary threshold. Other methods include accumulating the above judgments at arbitrary intervals and adopting the one with the most occurrences, and updating the judgment when it occurs consecutively for an arbitrary number of periods. Also, if it is determined that the front-to-back direction has not been reversed, then θ o and θ v Both will be used as output values. (If the target vehicle is not reversed front to back (step ST2030 "NO"), "θ0" or "θ v Output " (Step ST2050)) The orientation estimation unit 120 outputs the orientation of the surrounding moving object and then terminates processing and waits. (Termination)
[0039] Next, the orientation smoothing section 130 will be explained. The orientation smoothing unit 130 has the function of smoothing the orientation of the target vehicle obtained from the orientation estimation unit 120. It also includes a function to remove outliers. By incorporating the above features, the system can provide a smooth response to temporal changes in the orientation of the target vehicle, and outliers can be removed to suppress misestimation of orientation. Figure 5 is a flowchart showing an example of processing in the orientation smoothing unit 130 according to Embodiment 1 of this disclosure (first example). In Figure 5, the flowchart of the orientation smoothing unit 130 is shown when there is only one output value from the orientation estimation unit 120. The orientation smoothing unit 130 starts processing, for example, when it receives the estimation result from the orientation estimation unit 120. (Start)
[0040] Step ST3010 is a flow for removing outliers. The orientation smoothing unit 130 performs a process to determine whether it is an outlier. (Step ST3010 "Is it an outlier?")
[0041] If the orientation smoothing unit 130 does not contain an outlier (step ST3010 "NO"), it then performs the processing in step ST3020. If the orientation smoothing unit 130 does not contain an outlier, it performs the calculation processing in step ST3040 (θ=θ) based on the calculation processing in step ST3020 (K=1-exp(-dt / τ)). t-1 +K(θ t -θ t-1 The gain K of )) is calculated.
[0042] If the orientation smoothing unit 130 finds an outlier (step ST3010 "YES"), it then executes the process in step ST3030. If the orientation smoothing unit 130 finds an outlier, the process in step ST3030 executes the calculation process in step ST3040 (θ=θ t-1 +K(θ t -θ t-1 The gain K of )) is set to 0 (K=0), and it functions to retain the previous value. Here, "dt" is the sampling time and "τ" is the time constant. Furthermore, in some cases, outliers are determined by calculating the difference from the previous value. The determination formula is shown below. |θ t -θ t-1 |>θ th ...(14) Here, in equation (14), “θ t " is the direction of the target vehicle for this cycle, "θ t-1 " is the direction of the target vehicle in the previous cycle, "θ th “ is an arbitrary threshold.” In the calculation process of step ST3150, described later, smoothing is performed using the gain K obtained in the calculation process of step ST3020 (K=1-exp(-dt / τ)), the orientation of the target vehicle in the current and previous periods. The smoothed θ here is output as an estimated value of the direction. The orientation smoothing unit 130 outputs the estimated orientation θ, then terminates processing and waits ("terminate").
[0043] Next, a second example of the processing of the orientation smoothing section 130 will be described. Figure 6 is a flowchart showing an example of processing in the orientation smoothing unit 130 according to Embodiment 1 of this disclosure (second example). Figure 6 shows a flowchart of the orientation smoothing unit 130 when there are two types of output values from the orientation estimation unit 120. In Figure 6, a function that takes two orientation data as input and smooths each of them is described. In other words, smoothing is performed twice in one cycle, which further improves the accuracy of estimating the orientation of the target vehicle.
[0044] In the processing of step ST3110, the orientation smoothing unit 130 initially sets the orientation of the target vehicle to be input. In step ST3110, the direction "θ" is used. o The input is "θ", but the direction is "θ". v You may input "θ". However, in that case, the direction "θ" will be used in the calculation process of step ST3170. o Enter ". Steps ST3120 to ST3150 perform the same processing as steps ST3010 to ST3040 in Figure 5. Step ST3160 determines whether to perform smoothing again based on the number of loop iterations. If the loop is completed for the first time, in step ST3170, the orientation of the other target vehicle (θ in Figure 6) is determined. v ) is used as the input value, and smoothing is performed again. If the loop count reaches day 2, the process ends, and the θ calculated in step ST3150 is output as an estimated value of the vehicle's orientation. Although Figure 6 shows two input values for the direction of the target vehicle, the same process can be used to estimate the direction of the target vehicle even in systems where three or more input values may be generated. The orientation smoothing unit 130 outputs the estimated orientation θ, then terminates processing and waits ("terminate").
[0045] In this embodiment, the orientation of a moving object (target vehicle) can be calculated using either the orientation calculated by image processing, the orientation calculated from the velocity vector using a Kalman filter, or both, and the orientation can be accurately estimated in order to accurately grasp the situation of the target vehicle in the tracking filter of the camera for in-vehicle sensor fusion. In contrast, processing using only a Kalman filter has problems, such as low accuracy at low vehicle speeds when calculating direction using the ratio of absolute velocities. This is because low speeds are highly susceptible to noise. This disclosure addresses this problem by enabling accurate operation of the collision mitigation braking system and the automatic following system by accurately estimating the orientation of the target vehicle and accurately understanding the vehicle's condition.
[0046] This embodiment shows a configuration that includes the following: A Kalman filter unit that estimates the state of surrounding moving objects observed by a camera sensor in a moving object using a Kalman filter, A direction estimation unit estimates the direction of a moving object by taking as input the direction of the moving object observed by the camera sensor and the direction of the moving object calculated using the velocity estimated by the Kalman filter unit, A direction smoothing unit that smooths the direction of the surrounding moving object estimated by the direction estimation unit described above, A direction estimation device equipped with the following features. This disclosure thus has the effect of providing a direction estimation device that can improve the accuracy of estimating the direction of a moving object in the vicinity.
[0047] This embodiment shows a configuration that includes the following: The orientation estimation unit described above The system is characterized by outputting the orientation of the surrounding moving object observed by the camera sensor when the surrounding moving object is moving at a low speed below a predetermined speed. The orientation estimation device described in claim 1. This disclosure has the effect of providing a direction estimation method that enables further improvement in the accuracy of estimating the direction of a surrounding moving object when it is moving at a low speed.
[0048] This embodiment further describes a configuration that includes the following: The orientation estimation unit described above The system is characterized in that, if the surrounding moving object is not moving at a low speed below a predetermined speed, and the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated from the speed estimated by the Kalman filter unit are reversed front to back, the system outputs the orientation of the surrounding moving object calculated using the speed estimated by the Kalman filter unit. The orientation estimation device according to claim 1. As a result, this disclosure has the effect of providing a direction estimation device that can further improve the accuracy of estimating the direction of a surrounding moving object when it is moving at a speed above a predetermined speed. Furthermore, by applying the above configuration to a system including a direction estimation device, the above direction estimation method, or a program, the same effects as described above can be achieved.
[0049] This embodiment further describes a configuration that includes the following: The orientation estimation unit described above If the surrounding moving object is not moving at a low speed below a predetermined speed, and the orientation of the surrounding moving object calculated from the orientation of the surrounding moving object observed by the camera sensor and the speed estimated by the Kalman filter unit is not reversed front to back, This system is characterized by outputting both the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter. The orientation estimation device according to claim 1. As a result, this disclosure has the effect of providing a direction estimation device that can further improve the accuracy of estimating the direction of a surrounding moving object when it is moving at a speed above a predetermined speed. Furthermore, by applying the above configuration to a system including a direction estimation device, the above direction estimation method, or a program, the same effects as described above can be achieved.
[0050] This embodiment further describes a configuration that includes the following: The above-mentioned oriented smooth portion is, When there is only one orientation for the surrounding moving object input from the orientation estimation unit, The device is characterized by outputting the direction of the peripheral moving object from the previous cycle if the difference between the direction of the peripheral moving object input from the direction estimation unit and the direction of the peripheral moving object in the previous cycle is greater than a predetermined difference. Direction estimation device. This disclosure also has the effect of providing a direction estimation device that enables further improvement in the accuracy of estimating the direction of a moving object in the vicinity. Furthermore, by applying the above configuration to a system including a direction estimation device, the above measurement method, or the above program, the same effects as described above can be achieved.
[0051] This embodiment further describes a configuration that includes the following: The above-mentioned oriented smooth portion is, If there is only one orientation for the surrounding moving object input from the orientation estimation unit, If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is smaller than a predetermined difference, the orientation of the peripheral moving object is calculated using the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle. Direction estimation device. This disclosure also has the effect of providing a direction estimation device that enables further improvement in the accuracy of estimating the direction of a moving object in the vicinity. Furthermore, by applying the above configuration to a system including a direction estimation device, the above direction estimation method, or a program, the same effects as described above can be achieved.
[0052] This embodiment further describes a configuration that includes the following: The above-mentioned oriented smooth portion is, If there are multiple orientations of the surrounding moving objects input from the orientation estimation unit, For each input orientation of the surrounding moving object, If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is greater than a predetermined difference, the orientation of the peripheral moving object in the previous cycle is output. If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is smaller than a predetermined difference, the orientation of the peripheral moving object is calculated using the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle. Characterized by, The orientation estimation device described in claim 1. This disclosure also has the effect of providing a direction estimation device that enables further improvement in the accuracy of estimating the direction of a moving object in the vicinity. Furthermore, by applying the above configuration to a system including a direction estimation device, the above direction estimation method, or a program, the same effects as described above can be achieved.
[0053] This embodiment further describes a configuration that includes the following: A method for estimating orientation using an orientation estimation device, The Kalman filter section of the orientation estimation device described above performs a Kalman filter process that estimates the state of surrounding moving objects observed by a camera sensor on a moving object using a Kalman filter, The orientation estimation unit of the orientation estimation device performs an orientation estimation process that estimates the orientation of a moving object by taking as input the orientation of the moving object observed by the camera sensor and the orientation of the moving object calculated using the velocity estimated by the Kalman filter unit, The orientation smoothing unit of the orientation estimation device performs orientation smoothing processing to smooth the orientation of the surrounding moving object estimated by the orientation estimation unit, A method for estimating orientation, comprising the following features. This disclosure can further provide a direction estimation method that enables improved accuracy in estimating the direction of a moving object in the vicinity.
[0054] Here, we will describe the hardware configuration required to realize the functions of this disclosure. Figure 7 shows a first example of a hardware configuration for realizing the functions of the configuration described herein. Figure 8 shows a second example of a hardware configuration for realizing the functionality of the configuration described herein. The orientation estimation device 100 (100A) of this disclosure is implemented by hardware as shown in Figure 7 or Figure 8.
[0055] As shown in Figure 7, the orientation estimation device 100 (100A) is composed of, for example, a processor 10001, a memory 10002, an input / output interface 10003, and a communication circuit 10004. The processor 10001 and memory 10002 are, for example, components installed in a computer. Memory 10002 stores a program that causes the computer to function as a Kalman filter unit 110, an initial value calculation unit 111, a smoothing unit 112, a prediction unit 113, a delay unit 114, a direction estimation unit 120, a direction smoothing unit 130, and a control unit (not shown). By the processor 10001 reading and executing the program stored in memory 10002, the functions of the Kalman filter unit 110, the initial value calculation unit 111, the smoothing unit 112, the prediction unit 113, the delay unit 114, the direction estimation unit 120, the direction smoothing unit 130, and the control unit (not shown) are realized. Furthermore, a storage unit (not shown) is realized by memory 10002 or other memory (not shown). Furthermore, a communication unit (not shown) is realized by the communication circuit 10004.
[0056] Processor 10001 uses, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, or a DSP (Digital Signal Processor). Memory 10002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory; it may be a magnetic disk such as a hard disk or flexible disk; it may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc); or it may be a magneto-optical disk. The processor 10001 and the memory 10002 or the communication circuit 10004 are connected in a way that allows them to transmit data to each other. Furthermore, the processor 10001, the memory 10002, and the communication circuit 10004 are connected to other hardware via the input / output interface 10003 in a way that allows them to transmit data to each other.
[0057] Alternatively, the functions of the Kalman filter unit 110, initial value calculation unit 111, smoothing unit 112, prediction unit 113, delay unit 114, direction estimation unit 120, direction smoothing unit 130, and control unit (not shown) in the direction estimation device 100 (100A) may be realized by a dedicated processing circuit 20001, as shown in Figure 8.
[0058] The processing circuit 20001 may utilize, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration). Furthermore, a storage unit (not shown) is realized by memory 20002 or other memory (not shown). Memory 20002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory; it may be a magnetic disk such as a hard disk or flexible disk; it may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc); or it may be a magneto-optical disk. Furthermore, a communication unit (not shown) is realized by the communication circuit 20004. The processing circuit 20001 and the memory 20002 or the communication circuit 20004 are connected in a way that allows them to transmit data to each other. Furthermore, the processing circuit 20001, the memory 20002, and the communication circuit 20004 are connected in a way that allows them to transmit data to other hardware via the input / output interface 20003. In addition, the functions of the Kalman filter unit 110, initial value calculation unit 111, smoothing unit 112, prediction unit 113, delay unit 114, direction estimation unit 120, direction smoothing unit 130, and control unit (not shown) in the direction estimation device 100 (100A) may be implemented by separate processing circuits, or they may be implemented together in a single processing circuit.
[0059] Alternatively, some functions of the direction estimation device 100 (100A), including the Kalman filter unit 110, initial value calculation unit 111, smoothing unit 112, prediction unit 113, delay unit 114, direction estimation unit 120, direction smoothing unit 130, and a control unit (not shown), may be implemented by the processor 10001 and memory 10002, while the remaining functions are implemented by the processing circuit 20001.
[0060] Within the scope of this disclosure, it is possible to freely combine the embodiments, modify any component of each embodiment, or omit any component of each embodiment. [Industrial applicability]
[0061] This disclosure can improve the accuracy of estimating the orientation of surrounding moving objects present around a moving object, and is therefore suitable for use in orientation estimation devices, or, for example, moving object support devices that assist the movement of a moving object, or moving object-mounted systems including these. [Explanation of Symbols]
[0062] 1 Mobile-mounted system, 100 (100A) Direction estimation device, 110 Kalman filter section, 111 Initial value calculation section, 112 Smoothing section, 113 Prediction section, 114 Delay section, 120 Direction estimation section, 130 Direction smoothing section, 200 Camera sensor section, 300 Output destination device, 10001 Processor, 10002 Memory, 10003 Input / output interface, 10004 Communication circuit, 20001 Processing circuit, 20002 Memory, 20003 Input / output interface, 20004 Communication circuit.
Claims
1. A Kalman filter unit that estimates the state of surrounding moving objects observed by a camera sensor in a moving object using a Kalman filter, A direction estimation unit estimates the direction of a moving object by taking as input the direction of the moving object observed by the camera sensor and the direction of the moving object calculated using the velocity estimated by the Kalman filter unit, A direction smoothing unit that smooths the direction of the surrounding moving object estimated by the direction estimation unit described above, A direction estimation device equipped with the following features.
2. The orientation estimation unit described above The system is characterized by outputting the orientation of the surrounding moving object observed by the camera sensor when the surrounding moving object is moving at a low speed below a predetermined speed. The orientation estimation device according to claim 1.
3. The orientation estimation unit described above is: The system is characterized in that, if the surrounding moving object is not moving at a low speed below a predetermined speed, and the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated from the speed estimated by the Kalman filter unit are reversed front to back, the system outputs the orientation of the surrounding moving object calculated using the speed estimated by the Kalman filter unit. The orientation estimation device according to claim 1 or claim 2.
4. The orientation estimation unit described above is: If the surrounding moving object is not moving at a low speed below a predetermined speed, and the orientation of the surrounding moving object calculated from the orientation of the surrounding moving object observed by the camera sensor and the speed estimated by the Kalman filter unit is not reversed front to back, This system is characterized by outputting both the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter. The orientation estimation device according to claim 1 or claim 2.
5. The above-mentioned oriented smooth portion is, When there is only one orientation for the surrounding moving object input from the orientation estimation unit, The device is characterized by outputting the direction of the peripheral moving object from the previous cycle if the difference between the direction of the peripheral moving object input from the direction estimation unit and the direction of the peripheral moving object in the previous cycle is greater than a predetermined difference. The orientation estimation device according to claim 1 or claim 2.
6. The above-mentioned oriented smooth portion is, If there is only one orientation for the surrounding moving object input from the orientation estimation unit, If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is smaller than a predetermined difference, the orientation of the peripheral moving object is calculated using the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle. The orientation estimation device according to claim 1 or claim 2.
7. The above-mentioned oriented smooth portion is, If there are multiple orientations of the surrounding moving objects input from the orientation estimation unit, For each input orientation of the surrounding moving object, If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is greater than a predetermined difference, the orientation of the peripheral moving object in the previous cycle is output. If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is smaller than a predetermined difference, the orientation of the peripheral moving object is calculated using the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle. Characterized by, The orientation estimation device according to claim 1 or claim 2.
8. A method for estimating orientation using an orientation estimation device, The Kalman filter section of the orientation estimation device described above performs a Kalman filter process that estimates the state of surrounding moving objects observed by a camera sensor on a moving object using a Kalman filter, The orientation estimation unit of the orientation estimation device performs an orientation estimation process that estimates the orientation of a surrounding moving object by taking as input the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter unit, The orientation smoothing unit of the orientation estimation device performs orientation smoothing processing to smooth the orientation of the surrounding moving object estimated by the orientation estimation unit, A method for estimating orientation, comprising the following features.