Direction estimation device and direction estimation method

Abstract

A direction estimation device (100) is provided with: a Kalman filter unit (110) that estimates, using a Kalman filter, the state of a surrounding mobile body observed by a camera sensor in a mobile body; a direction estimation unit (120) that estimates the direction of the surrounding mobile body by taking, as input, the direction of the surrounding mobile body observed by the camera sensor and the direction of the surrounding mobile body calculated using the speed estimated by the Kalman filter unit; and a direction smoothing unit (130) that smooths the direction of the surrounding mobile body estimated by the direction estimation unit.

Description

Orientation estimation device and orientation estimation method

[0001] The disclosed technology relates to a direction estimation technology for estimating the direction of a nearby moving object.

[0002] For example, there are driving support technologies for moving objects, such as collision damage mitigation braking systems and automatic following systems for vehicles. Driving support technologies may estimate and use the state of surrounding moving objects around a moving object. Patent Document 1 discloses a method for estimating the orientation of a moving object using image processing (image processing device and image processing method). The image processing device and image processing method of Patent Document 1 extract a target vehicle from an image by edge detection and estimate the orientation of the vehicle based on its symmetry. Specifically, if there is symmetry, the orientation is estimated to be forward or backward, and if there is no symmetry, the orientation is estimated to be sideways or diagonally. In other words, the method disclosed in Patent Document 1 is a method for estimating the orientation of a vehicle based on the symmetry of the vehicle's shape in an image.

[0003] Japanese Patent Application Laid-Open No. 2004-126947

[0004] However, the method of Patent Document 1 has a problem in that it is not possible to identify whether the actual direction of the vehicle is forward or backward, and the accuracy of the direction estimation result is low.

[0005] The present disclosure is intended to solve the above-mentioned problems, and aims to improve the accuracy of estimating the orientation of a nearby moving object.

[0006] The orientation estimation device of the present disclosure includes a Kalman filter unit that estimates the state of a surrounding moving body observed by a camera sensor in a moving body using a Kalman filter; an orientation estimation unit that estimates the orientation of a surrounding moving body using as input the orientation of the surrounding moving body observed by the camera sensor and the orientation of the surrounding moving body calculated using the speed estimated by the Kalman filter unit; and an orientation smoothing unit that smooths the orientation of the surrounding moving body estimated by the orientation estimation unit.

[0007] According to the present disclosure, it is possible to improve the accuracy of estimating the orientation of a nearby moving object.

[0008] FIG. 1 is a diagram illustrating an example configuration of an orientation estimation device 100 according to a first embodiment of the present disclosure. FIG. 2 is a flowchart illustrating an example of processing by the orientation estimation device 100 according to the first embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of a detailed configuration of the orientation estimation device 100 (100A) according to the first embodiment of the present disclosure, and is a diagram illustrating an example of a configuration of an orientation estimation system including the orientation estimation device 100 (100A). FIG. 4 is a flowchart illustrating an example of processing by the orientation estimation device 100 (100A) according to the first embodiment of the present disclosure. FIG. 5 is a flowchart illustrating an example (first example) of processing by the orientation estimation unit 120 according to the first embodiment of the present disclosure. FIG. 6 is a flowchart illustrating an example (second example) of processing by the orientation smoothing unit 130 according to the first embodiment of the present disclosure. FIG. 7 is a diagram illustrating a first example of a hardware configuration for realizing functions according to the configuration of the present disclosure. FIG. 8 is a diagram illustrating a second example of a hardware configuration for realizing functions according to the configuration of the present disclosure.

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1. In embodiment 1, a basic configuration example of the present disclosure and a more detailed configuration example thereof will be described.

[0011] An example configuration of an apparatus according to a first embodiment of the present disclosure will be described. FIG. 1 is a diagram illustrating an example configuration of an orientation estimation apparatus 100 according to the first embodiment of the present disclosure. The orientation estimation apparatus 100 estimates the orientations of surrounding moving objects present around a moving object. The orientation estimation apparatus 100 illustrated in FIG. 1 is configured to include a Kalman filter unit 110, an orientation estimation unit 120, and an orientation smoothing unit 130.

[0012] The Kalman filter unit 110 has a function of estimating the state of a nearby moving object. Specifically, the Kalman filter unit 110 estimates the state of a nearby moving object observed by a camera sensor in the moving object using a Kalman filter.

[0013] The orientation estimation unit 120 has a function of estimating the orientation of a surrounding moving object using the estimation result by the Kalman filter unit 110. Specifically, the orientation estimation unit 120 estimates the orientation of a surrounding moving object using as input the orientation of the surrounding moving object observed by the camera sensor unit 200 and the orientation of the surrounding moving object calculated using the speed estimated by the Kalman filter unit.

[0014] The orientation smoothing unit 130 has a function of smoothing the orientations of the peripheral moving objects estimated by the orientation estimation unit 120. Specifically, the orientation smoothing unit 130 performs processing using the orientations of the peripheral moving objects output by the orientation estimation unit 120, and outputs the orientations of the peripheral moving objects output by the orientation estimation device 100.

[0015] An example of processing by the orientation estimation device according to the first embodiment of the present disclosure will be described. FIG. 2 is a flowchart illustrating an example of processing by the orientation estimation device 100 according to the first embodiment of the present disclosure. The processing illustrated in FIG. 2 is a method performed by the orientation estimation device 100. For example, the orientation estimation device 100 illustrated in FIG. 1 starts the processing illustrated in FIG. 2 when it is ready to perform processing, such as when it receives an external command or a processing signal from the outside. ("Start")

[0016] Orientation estimation device 100 then executes a state estimation process (step ST1000). In the state estimation process, Kalman filter unit 110 of orientation estimation device 100 estimates the states of surrounding moving objects observed by camera sensor unit 200 in the moving object using a Kalman filter.

[0017] Next, the orientation estimation device 100 executes an orientation estimation process (step ST2000). In the orientation estimation process, the orientation estimation unit 120 of the orientation estimation device 100 estimates the orientation of the surrounding moving object using the orientation of the surrounding moving object observed by the camera sensor unit 200 and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter unit 110 as inputs.

[0018] Orientation estimation apparatus 100 then performs orientation smoothing processing (step ST3000). In the orientation estimation processing, orientation smoothing section 130 of orientation estimation apparatus 100 smoothes the orientations of the peripheral moving objects estimated by orientation estimation section 120.

[0019] After performing the orientation smoothing process, the orientation estimation device 100 outputs the smoothed orientation of the moving body (moving body orientation information), and then ends the process ("End").

[0020] Next, an example of the configuration of the orientation estimation device 100 (100A) and a mobile-mounted system 1 including the orientation estimation device 100 (100A) when the orientation estimation device 100 is applied to a mobile-mounted device will be described. FIG. 3 is a diagram showing an example of the detailed configuration of the orientation estimation device 100 (100A) according to the first embodiment of the present disclosure, and is a diagram showing an example of the configuration of an orientation estimation system including the orientation estimation device 100 (100A). The mobile-mounted system 1 provides assistance regarding the movement of a mobile object using the orientations of surrounding mobile objects. For example, the mobile-mounted system 1 functions as a system such as a collision damage mitigation braking system or an automatic following system. The mobile-mounted system 1 shown in FIG. 3 is configured to include the orientation estimation device 100 (100A), a camera sensor unit 200, and an output destination device 300.

[0021] The camera sensor unit 200 is, for example, an imaging device mounted on a moving body and capturing an image in the moving direction of the moving body. The camera sensor unit 200 is a camera sensor that observes surrounding moving bodies (target vehicles such as surrounding vehicles if the moving body is a vehicle). Specifically, the camera sensor unit 200 observes surrounding moving bodies present around the moving body within the imaging range as observation targets, and outputs a captured image including the observed surrounding moving bodies.

[0022] The destination device 300 is a device that performs processing using the orientation of a nearby moving object, and in particular generates and outputs information that supports movement by the moving object. The destination device 300 receives the orientation of a nearby moving object estimated by the orientation estimation device 100A and performs processing.

[0023] The orientation estimation device 100A estimates the orientation of a nearby moving object present around the moving object with a configuration with further improved accuracy. The orientation estimation device 100A shown in FIG. 3 includes a Kalman filter unit 110, an orientation estimation unit 120, and an orientation smoothing unit 130.

[0024] Similar to the Kalman filter unit 110 already described, the Kalman filter unit 110 has a function of estimating, by a Kalman filter, the state of a surrounding moving object observed by the camera sensor unit 200 in the moving object. The Kalman filter unit 110 shown in FIG. 3 is configured to include an initial value calculation unit 111, a smoothing unit 112, a prediction unit 113, and a delay unit 114.

[0025] The initial value calculation unit 111 has a function of calculating an initial value from the observed value of the camera sensor unit 200. The smoothing unit 112 has a function of smoothing the state vector. The prediction unit 113 has a function of predicting the state vector. The delay unit 114 has a function of delaying the output value of the smoothing unit 112 by one period.

[0026] Like the orientation estimation unit 120 already described, the orientation estimation unit 120 has the function of estimating the orientation of a surrounding moving body using the orientation of the surrounding moving body observed by the camera sensor unit 200 and the orientation of the surrounding moving body obtained using the velocity estimated by the Kalman filter unit 110.

[0027] The orientation estimation unit 120 shown in FIG. 3 further outputs the orientation of the surrounding moving object observed by the camera sensor unit 200 when the surrounding moving object is moving at a low speed that is less than a predetermined speed.

[0028] The orientation estimation unit 120 shown in Figure 3 further has the function of outputting the orientation of the surrounding moving body calculated using the speed estimated by the Kalman filter unit 110 when the surrounding moving body is not moving at a low speed that is less than a predetermined speed and the orientation of the surrounding moving body observed by the camera sensor unit 200 and the orientation of the surrounding moving body calculated from the speed estimated by the Kalman filter unit 110 are reversed front to back.

[0029] The orientation estimation unit 120 shown in Figure 3 further has the function of outputting both the orientation of the surrounding moving body observed by the camera sensor unit 200 and the orientation of the surrounding moving body calculated using the speed estimated by the Kalman filter unit 110 when the surrounding moving body is not moving at a low speed that is less than a predetermined speed and the orientation of the surrounding moving body observed by the camera sensor unit 200 and the orientation of the surrounding moving body calculated from the speed estimated by the Kalman filter unit 110 are not reversed front to back.

[0030] The direction smoothing unit 130 has a function of smoothing the direction of the surrounding moving object estimated by the direction estimation unit 120 .

[0031] The orientation smoothing unit 130 shown in Figure 3 further has the function of outputting the orientation of the surrounding moving body in the previous cycle when there is only one orientation of the surrounding moving body input from the orientation estimation unit 120 and the difference between the orientation of the surrounding moving body input from the orientation estimation unit 120 and the orientation of the surrounding moving body in the previous cycle is greater than a predetermined difference.

[0032] The orientation smoothing unit 130 shown in Figure 3 further has the function of calculating the orientation of the surrounding moving body using the orientation of the surrounding moving body input from the orientation estimation unit 120 and the orientation of the surrounding moving body in the previous cycle when there is only one orientation of the surrounding moving body input from the orientation estimation unit 120 and the difference between the orientation of the surrounding moving body input from the orientation estimation unit 120 and the orientation of the surrounding moving body in the previous cycle is smaller than a predetermined difference.

[0033] The orientation smoothing unit 130 shown in Figure 3 further has the function of, when there are multiple orientations of surrounding moving bodies input from the orientation estimation unit 120, outputting the orientation of the surrounding moving body in the previous cycle if the difference between the orientation of the surrounding moving body input from the orientation estimation unit 120 and the orientation of the surrounding moving body in the previous cycle is greater than a predetermined difference, and calculating the orientation of the surrounding moving body using the orientation of the surrounding moving body input from the orientation estimation unit 120 and the orientation of the surrounding moving body in the previous cycle if the difference between the orientation of the surrounding moving body input from the orientation estimation unit 120 and the orientation of the surrounding moving body in the previous cycle is smaller than the predetermined difference.

[0034] Next, an operation example (processing example) of the moving body mounted system 1 and the orientation estimation device 100A according to the first embodiment of the present disclosure will be described. First, the camera sensor unit 200 observes information such as the observation value z shown in the following equation (1). z=(x ｗｐ , y ｗｐ , v ｘ , v ｙ , a ｘ , θ ｏ , L, W) ... (1) Here, in the formula (1), "x ｗｐ " is the vertical position of the target vehicle, "y ｗｐ " is the horizontal position of the target vehicle, "v ｘ " is the ground longitudinal speed of the target vehicle, "v ｙ " is the lateral ground speed of the target vehicle, "a ｘ " is the longitudinal acceleration of the target vehicle, "θ ｏ " is the orientation obtained by image processing, "L" is the depth of the target vehicle, and "W" is the width of the target vehicle. Next, the Kalman filter unit 110 will be explained. First, the initial value calculation unit 21 will be explained. The initial values ​​of the state vector and the error covariance matrix are shown below.

[0035] Next, the prediction unit 113 will be described. The prediction unit 113 predicts the state vector xs obtained from the smoothing unit 112. ｋ , error covariance matrix ps ｋ from the state vector xp ｋ , error covariance matrix pp ｋ Here, a linear motion model is used to make the prediction. ｋ = Fmat × xs ｋ ... (8) pp ｋ = Fmat × ps ｋ ×Fmat ｔ +Q (9) Here, in equations (8) and (9), "Fmat" is a state transition matrix. Also, "Q" is an error covariance matrix of process noise. The state transition matrix Fmat can be defined as follows from the motion model: Here, in equation (10), "dt" is the sampling time.

[0036] The delay unit 114 receives the state vector xp generated by the prediction unit 113. ｋ , error covariance matrix pp ｋ is delayed by one period and input to the smoothing unit 112.

[0037] Next, an example of processing in the orientation estimation unit 120 will be described. FIG. 4 is a flowchart showing an example of processing in the orientation estimation unit 120 according to the first embodiment of the present disclosure. The orientation estimation unit 120 starts processing when it receives, for example, a processing result from the Kalman filter unit 110. ("Start") The orientation "θ ｖ " is the vertical ground speed vx generated by the Kalman filter unit 110. ｋ , lateral ground speed vy ｋ is generated by the following equation (11). The direction estimation unit 120 executes a process to determine whether the target vehicle is traveling at a certain speed or above. (Step ST2010 "Is the target vehicle traveling at a certain speed or above?") In step ST2010, it is determined whether the target vehicle is traveling at a certain speed or above. If the target vehicle is traveling at a low speed, the velocity vector is easily affected by noise, and θ ｖ Therefore, when the target vehicle is moving at a low speed, the estimation accuracy of θ ｏ is the output value. (θ ０ (Step ST2020)) Furthermore, one example of the determination method in step ST2010 is a method using the following equation (12): (vx ｋ ２ +vy ｋ ２ ) １／２ >v ｍｏｖｅ° ... (12) Here, "v ｍｏｖｅ° " is an arbitrary threshold value. Other methods include accumulating the above judgments for an arbitrary period and adopting the one with the highest number of judgments, or updating the judgment if an arbitrary period continues in succession.

[0038] The direction estimation unit 120 then executes a process to determine whether the target vehicle is inverted (step ST2030 "Is the target vehicle inverted?"). Step ST2030 has the function of determining whether the target vehicle is inverted. The direction of the target vehicle obtained by image processing can be determined to be, for example, forward or backward, diagonal or sideways, but it cannot be determined whether it is forward or backward. In other words, the direction of the target vehicle obtained by image processing may be opposite to the actual vehicle direction, that is, it may differ by π rad. Therefore, in step ST2030, θ ｏ and θ ｖ If the front-to-back direction of θ ｏ is considered to be in the opposite direction, and θ ｖ (If the target vehicle is inverted ("YES" in step ST2030), θ ｖ (Step ST2040)) Also, θ ｏ and θ ｖ The following is an example of determining whether the front-to-back direction of |θ is reversed. ｏ -(θ ｖ +π)|>reverse ｔｈ ...(13) Here, in the formula (13), "reverse ｔｈ " is an arbitrary threshold value. In addition to this, there is a method in which the above judgment is accumulated for an arbitrary period and the one with the largest number of times is adopted, or a method in which the judgment is updated when an arbitrary period continues. Also, if it is determined that the front-rear direction is not reversed, θ ｏ and θ ｖ Both are output values. (If the target vehicle is not reversed ("NO" in step ST2030), "θ ０ "or"θ ｖ " (step ST2050)) After outputting the orientation of the nearby moving object, orientation estimation unit 120 ends the process and waits ("END").

[0039] Next, the direction smoothing unit 130 will be described. The direction smoothing unit 130 has the function of re-smoothing the direction of the target vehicle obtained by the direction estimation unit 120. It also has the function of removing outliers. By providing the above functions, a smooth response is achieved to temporal changes in the direction of the target vehicle, and removing outliers can suppress erroneous direction estimation. FIG. 5 is a flowchart showing an example (first example) of processing in the direction smoothing unit 130 according to the first embodiment of the present disclosure. FIG. 5 shows a flowchart of the direction smoothing unit 130 when the direction estimation unit 120 has one type of output value. The direction smoothing unit 130 starts processing, for example, when it receives an estimation result from the direction estimation unit 120. ("Start")

[0040] Step ST3010 is a flow for removing outliers. The direction smoothing unit 130 executes a process for determining whether an outlier exists (Step ST3010 "Is it an outlier?").

[0041] If the direction smoothing unit 130 determines that the value is not an outlier ("NO" in step ST3010), it then executes the process of step ST3020. If the value is not an outlier, the direction smoothing unit 130 executes the process of step ST3040 (θ=θ) by the calculation process of step ST3020 (K=1-exp(-dt / τ)). ｔ－１ +K(θ ｔ -θ ｔ－１ )) is calculated.

[0042] If the direction smoothing unit 130 determines that the value is an outlier ("YES" in step ST3010), it then executes the process of step ST3030. If the value is an outlier, the direction smoothing unit 130 performs the calculation process of step ST3040 (θ=θ ｔ－１ +K(θ ｔ -θ ｔ－１ )) is set to 0 (K=0) and functions to hold the previous value. Here, "dt" is the sampling time and "τ" is the time constant. In addition, there is an example where the determination of whether or not it is an outlier is calculated from the difference with the previous value. The determination formula is shown below. |θ ｔ -θ ｔ－１ |>θ ｔｈ...(14) Here, in equation (14), "θ ｔ " is the direction of the target vehicle in this cycle, "θ ｔ－１ " is the direction of the target vehicle in the previous period, "θ ｔｈ " is an arbitrary threshold value. In the calculation process of step ST3150 described later, smoothing is performed using the gain K obtained in the calculation process of step ST3020 (K=1-exp(-dt / τ)) and the orientations of the target vehicle in the current cycle and the previous cycle. The smoothed θ is output as an estimated orientation value. After outputting the estimated orientation θ, the orientation smoothing unit 130 ends the processing and waits ("end").

[0043] Next, a second example of the processing of the direction smoothing unit 130 will be described. FIG. 6 is a flowchart showing an example (second example) of the processing in the direction smoothing unit 130 according to the first embodiment of the present disclosure. FIG. 6 shows a flowchart of the direction smoothing unit 130 when there are two types of output values ​​from the direction estimation unit 120. FIG. 6 describes a unit that has the function of inputting two types of direction data and smoothing each of them. In other words, smoothing is performed twice during one cycle, thereby further improving the estimation accuracy of the direction of the target vehicle.

[0044] In the process of step ST3110, the direction smoothing unit 130 sets the direction of the target vehicle to be input first. ｏ " is input, but the direction "θ ｖ However, in this case, the direction "θ ｏ 6. Steps ST3120 to ST3150 are the same as steps ST3010 to ST3040 in FIG. 5. In step ST3160, it is determined whether smoothing should be performed again depending on the number of loops. If the number of loops is the first, the direction of another target vehicle (θ ｖ) as an input value and perform smoothing again. If the loop count is the second day, the process ends, and the θ calculated in step ST3150 is output as an estimated value of the orientation of the target vehicle. Note that although two input values ​​of the orientation of the target vehicle are listed in the example of FIG. 6, the orientation of the target vehicle can be estimated by similar processing even in a system in which three or more input values ​​can be generated. After outputting the estimated value of orientation θ, the orientation smoothing unit 130 ends the processing and waits ("End").

[0045] In this embodiment, the orientation of a nearby moving object (target vehicle) can be calculated using either or both of the orientation calculated by image processing and the orientation calculated from a velocity vector using a Kalman filter, and the orientation can be accurately estimated to accurately grasp the situation of the target vehicle in the tracking filter of the on-board sensor fusion camera. In contrast, processing using only a Kalman filter has a problem in that, for example, the accuracy of orientation calculation using the ratio of absolute velocities is low when the target vehicle is traveling at low speeds. This is because speeds at low speeds are highly affected by noise. The present disclosure addresses this problem by accurately estimating the orientation of the target vehicle and accurately grasping the situation of the target vehicle, thereby enabling accurate operation of a collision damage mitigation braking system and an automatic tracking system.

[0046] This embodiment has shown an aspect including the following configuration: an orientation estimation device including: a Kalman filter unit that estimates the state of a peripheral moving object observed by a camera sensor in a moving object using a Kalman filter; an orientation estimation unit that estimates the orientation of a peripheral moving object using as input the orientation of the peripheral moving object observed by the camera sensor and the orientation of the peripheral moving object calculated using the velocity estimated by the Kalman filter unit; and an orientation smoothing unit that smoothes the orientation of the peripheral moving object estimated by the orientation estimation unit. This makes it possible to provide an orientation estimation device that makes it possible to improve the accuracy of estimating the orientation of a peripheral moving object.

[0047] This embodiment has shown an aspect including the following configuration: The orientation estimation device according to claim 1, wherein the orientation estimation unit outputs the orientation of the surrounding moving object observed by the camera sensor when the surrounding moving object is moving at a low speed that is less than a predetermined speed. This makes it possible to provide an orientation estimation method that can further improve the accuracy of estimating the orientation of a surrounding moving object when the surrounding moving object is moving at a low speed.

[0048] This embodiment further illustrates a configuration including the following: The orientation estimation device according to claim 1, wherein the orientation estimation unit outputs the orientation of the surrounding moving object calculated using the speed estimated by the Kalman filter unit when the surrounding moving object is not moving at a low speed below a predetermined speed and the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated from the speed estimated by the Kalman filter unit are reversed. This provides an effect of providing an orientation estimation device that can further improve the accuracy of estimating the orientation of a surrounding moving object moving at a predetermined speed or higher. Furthermore, the present disclosure provides the same effect as the above by applying the above configuration to a system, etc. including the orientation estimation device, the orientation estimation method, or a program.

[0049] This embodiment further illustrates a configuration including the following: The orientation estimation device according to claim 1 , wherein the orientation estimation unit outputs both the orientation of the peripheral moving object observed by the camera sensor and the orientation of the peripheral moving object calculated using the speed estimated by the Kalman filter unit when the peripheral moving object is not moving at a low speed below a predetermined speed and the orientation of the peripheral moving object observed by the camera sensor and the orientation of the peripheral moving object calculated using the speed estimated by the Kalman filter unit are not reversed front to back. This provides an advantage of providing an orientation estimation device that can further improve the accuracy of estimating the orientation of a peripheral moving object moving at a predetermined speed or higher. Furthermore, the present disclosure provides the same advantage as the above by applying the above configuration to a system, etc. including the orientation estimation device, the orientation estimation method, or a program.

[0050] This embodiment further illustrates a form including the following configuration: The orientation estimation device is characterized in that, when the orientation of the peripheral moving body input from the orientation estimation unit is one, and when the difference between the orientation of the peripheral moving body input from the orientation estimation unit and the orientation of the peripheral moving body in the previous cycle is greater than a predetermined difference, the orientation smoothing unit outputs the orientation of the peripheral moving body in the previous cycle. This provides an effect of providing an orientation estimation device that makes it possible to further improve the accuracy of estimating the orientation of the peripheral moving body. Furthermore, the present disclosure provides the same effect as the above by applying the above configuration to a system, etc. including the orientation estimation device, the measurement method, or the program.

[0051] This embodiment further illustrates an aspect including the following configuration: An orientation estimation device, characterized in that the orientation smoothing unit calculates the orientation of a peripheral moving body using the orientation of the peripheral moving body input from the orientation estimation unit and the orientation of the peripheral moving body in the previous cycle when the orientation of the peripheral moving body input from the orientation estimation unit is one and the difference between the orientation of the peripheral moving body input from the orientation estimation unit and the orientation of the peripheral moving body in the previous cycle is smaller than a predetermined difference. This provides an effect of providing an orientation estimation device that makes it possible to further improve the accuracy of estimating the orientation of a peripheral moving body. Furthermore, the present disclosure provides the same effect as the above by applying the above configuration to a system, etc. including an orientation estimation device, the orientation estimation method, or a program.

[0052] This embodiment further illustrates a configuration including the following: The orientation estimation device according to claim 1 , wherein, when multiple orientations of peripheral moving objects are input from the orientation estimation unit, the orientation smoothing unit, for each input orientation of the peripheral moving object, outputs the orientation of the peripheral moving object in the previous cycle if a difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is greater than a predetermined difference, and calculates the orientation of the peripheral moving object using the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle if the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is smaller than the predetermined difference. This provides an effect of providing an orientation estimation device that can further improve the accuracy of estimating the orientation of a peripheral moving object. Furthermore, the present disclosure provides an effect similar to the effect described above by applying the above configuration to a system, etc. including the orientation estimation device, the orientation estimation method, or a program.

[0053] This embodiment further illustrates an aspect including the following configuration: an orientation estimation method using an orientation estimation device, comprising: a Kalman filter process in which a Kalman filter unit of the orientation estimation device estimates a state of a peripheral moving body observed by a camera sensor in a moving body using a Kalman filter; an orientation estimation process in which the orientation estimation unit of the orientation estimation device estimates the orientation of a peripheral moving body using as input the orientation of the peripheral moving body observed by the camera sensor and the orientation of the peripheral moving body calculated using the velocity estimated by the Kalman filter unit; and an orientation smoothing process in which an orientation smoother of the orientation estimation device smooths the orientation of the peripheral moving body estimated by the orientation estimation unit. This enables the present disclosure to further provide an orientation estimation method that makes it possible to improve the accuracy of estimating the orientation of a peripheral moving body.

[0054] Here, a hardware configuration for realizing the functions of the present disclosure will be described. Fig. 7 is a diagram showing a first example of a hardware configuration for realizing the functions of the configuration of the present disclosure. Fig. 8 is a diagram showing a second example of a hardware configuration for realizing the functions of the configuration of the present disclosure. The orientation estimation device 100 (100A) of the present disclosure is realized by hardware such as that shown in Fig. 7 or Fig. 8.

[0055] As shown in FIG. 7 , the orientation estimation device 100 (100A) is configured with, for example, a processor 10001, a memory 10002, an input / output interface 10003, and a communication circuit 10004. The processor 10001 and the memory 10002 are mounted on, for example, a computer. The memory 10002 stores a program for causing the computer to function as a Kalman filter unit 110, an initial value calculation unit 111, a smoothing unit 112, a prediction unit 113, a delay unit 114, an orientation estimation unit 120, an orientation smoothing unit 130, and a control unit (not shown). The processor 10001 reads and executes the program stored in the memory 10002, thereby realizing the functions of the Kalman filter unit 110, the initial value calculation unit 111, the smoothing unit 112, the prediction unit 113, the delay unit 114, the orientation estimation unit 120, the orientation smoothing unit 130, and a control unit (not shown). Furthermore, a storage unit (not shown) is realized by the memory 10002 or another memory (not shown). Furthermore, a communication unit (not shown) is realized by the communication circuit 10004.

[0056] The processor 10001 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, or a DSP (Digital Signal Processor). The memory 10002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory, or a magnetic disk such as a hard disk or flexible disk, or an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk. The processor 10001 and the memory 10002 or the communication circuit 10004 are connected in a state capable of transmitting data to each other. The processor 10001, memory 10002, and communication circuit 10004 are connected via an input / output interface 10003 so as to be capable of transmitting data to and from other hardware.

[0057] Alternatively, the functions of the Kalman filter unit 110, initial value calculation unit 111, smoothing unit 112, prediction unit 113, delay unit 114, orientation estimation unit 120, orientation smoothing unit 130, and a control unit (not shown) in the orientation estimation device 100 (100A) may be realized by a dedicated processing circuit 20001, as shown in FIG. 8.

[0058] The processing circuit 20001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration), etc. In addition, a storage unit (not shown) is realized by the memory 20002 or another memory (not shown). The memory 20002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory, or a magnetic disk such as a hard disk or flexible disk, or an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk. The communication circuit 20004 implements a communication unit (not shown). The processing circuit 20001 and the memory 20002 or the communication circuit 20004 are connected in a state where they can transmit data to each other. Furthermore, the processing circuit 20001, the memory 20002, and the communication circuit 20004 are connected in a state where they can transmit data to other hardware via the input / output interface 20003. Note that the functions of the Kalman filter unit 110, the initial value calculation unit 111, the smoothing unit 112, the prediction unit 113, the delay unit 114, the direction estimation unit 120, the direction smoothing unit 130, and a control unit (not shown) in the direction estimation device 100 (100A) may be realized by separate processing circuits, or may be realized collectively by a processing circuit.

[0059] Alternatively, in the orientation estimation device 100 (100A), some of the functions of the Kalman filter unit 110, initial value calculation unit 111, smoothing unit 112, prediction unit 113, delay unit 114, orientation estimation unit 120, orientation smoothing unit 130, and a control unit not shown may be realized by the processor 10001 and memory 10002, and the remaining functions may be realized by the processing circuit 20001.

[0060] It should be noted that, within the scope of this disclosure, the embodiments may be freely combined, any component of each embodiment may be modified, or any component of each embodiment may be omitted.

[0061] The present disclosure can improve the accuracy of estimating the orientation of surrounding moving bodies present around a moving body, and is therefore suitable for use in orientation estimation devices, or, for example, moving body support devices that support the movement of moving bodies, moving body mounted systems that include these, etc.

[0062] 1 Mobile body mounted system, 100 (100A) Orientation estimation device, 110 Kalman filter unit, 111 Initial value calculation unit, 112 Smoothing unit, 113 Prediction unit, 114 Delay unit, 120 Orientation estimation unit, 130 Orientation smoothing unit, 200 Camera sensor unit, 300 Output destination device, 10001 Processor, 10002 Memory, 10003 Input / output interface, 10004 Communication circuit, 20001 Processing circuit, 20002 Memory, 20003 Input / output interface, 20004 Communication circuit.

Claims

1. A Kalman filter unit that estimates the state of surrounding moving objects observed by a camera sensor in a moving object using a Kalman filter, A direction estimation unit estimates the direction of a moving object by taking as input the direction of the moving object observed by the camera sensor and the direction of the moving object calculated using the velocity estimated by the Kalman filter unit, A direction smoothing unit that smooths the direction of the surrounding moving object estimated by the direction estimation unit described above, A direction estimation device equipped with the following features.

2. The orientation estimation unit described above is: The system is characterized by outputting the orientation of the surrounding moving object observed by the camera sensor when the surrounding moving object is moving at a low speed below a predetermined speed. The orientation estimation device according to claim 1.

3. The orientation estimation unit described above The system is characterized in that, if the surrounding moving object is not moving at a low speed below a predetermined speed, and the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated from the speed estimated by the Kalman filter unit are reversed front to back, the system outputs the orientation of the surrounding moving object calculated using the speed estimated by the Kalman filter unit. The orientation estimation device according to claim 1 or claim 2.

4. The orientation estimation unit described above is: If the surrounding moving object is not moving at a low speed below a predetermined speed, and the orientation of the surrounding moving object calculated from the orientation of the surrounding moving object observed by the camera sensor and the speed estimated by the Kalman filter unit is not reversed front to back, This system is characterized by outputting both the orientation of the surrounding moving object observed by the camera sensor and the orientation of the surrounding moving object calculated using the velocity estimated by the Kalman filter. The orientation estimation device according to claim 1 or claim 2.

5. The above-mentioned oriented smooth portion is, When there is only one orientation for the surrounding moving object input from the orientation estimation unit, The device is characterized by outputting the direction of the peripheral moving object from the previous cycle if the difference between the direction of the peripheral moving object input from the direction estimation unit and the direction of the peripheral moving object in the previous cycle is greater than a predetermined difference. The orientation estimation device according to claim 1 or claim 2.

6. The above-mentioned oriented smooth portion is, If there is only one orientation for the surrounding moving object input from the orientation estimation unit, If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is smaller than a predetermined difference, the orientation of the peripheral moving object is calculated using the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle. The orientation estimation device according to claim 1 or claim 2.

7. The above-mentioned oriented smooth portion is, If there are multiple orientations of the surrounding moving objects input from the orientation estimation unit, For each input orientation of the surrounding moving object, If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is greater than a predetermined difference, the orientation of the peripheral moving object in the previous cycle is output. If the difference between the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle is smaller than a predetermined difference, the orientation of the peripheral moving object is calculated using the orientation of the peripheral moving object input from the orientation estimation unit and the orientation of the peripheral moving object in the previous cycle. Characterized by, The orientation estimation device according to claim 1 or claim 2.

8. A method for estimating orientation using an orientation estimation device, The Kalman filter section of the orientation estimation device described above performs a Kalman filter process that estimates the state of surrounding moving objects observed by a camera sensor on a moving object using a Kalman filter, The orientation estimation unit of the orientation estimation device performs an orientation estimation process that estimates the orientation of a moving object by taking as input the orientation of the moving object observed by the camera sensor and the orientation of the moving object calculated using the velocity estimated by the Kalman filter unit, The orientation smoothing unit of the orientation estimation device performs orientation smoothing processing to smooth the orientation of the surrounding moving object estimated by the orientation estimation unit, A method for estimating orientation, comprising the following features.