Apparatus and method for estimating motion state information of a moving object based on an artificial neural network in the absence of absolute position information

Abstract

The present invention discloses an apparatus and method for estimating position and motion state information of a moving body. An apparatus for estimating position and motion state information of a moving body according to one embodiment of the present invention may include: a sensor data acquisition unit configured to acquire sensor data related to the behavior of the moving body from one or more sensors provided on the moving body; a motion state calculation unit configured to receive sensor data acquired from the sensor data acquisition unit and calculate motion state information of the moving body using an artificial neural network; a position estimation unit configured to estimate the position of the moving body by applying the motion state information calculated by the motion state calculation unit to a kinematic model of the moving body in a section where absolute position information is not provided; and a correction parameter update unit configured to update a correction parameter to compensate for an error in the motion state information calculated by the artificial neural network based on absolute position information when absolute position information is acquired from the outside.

Description

Technology Field

[0001] The present invention relates to a navigation technology for a moving object in an environment where absolute position information is absent. Specifically, it relates to an apparatus and method for estimating the velocity and sideslip angle of a moving object using a time-series artificial neural network that receives data from an internal sensor and an inertial sensor (IMU) of the moving object, and estimating the position through learning using multi-horizon kinematic integral loss and online correction based on a Kalman filter. Background Technology

[0002] The following descriptions are intended to aid in understanding the technical significance of the invention and are not based on the premise that they were publicly known prior to the filing. Therefore, the fact that a description is included in the background technology of the invention should not be regarded as publicly known technology in itself.

[0003] With the proliferation of autonomous driving technology, the demand for operating unmanned vehicles in various irregular environments, such as smart agriculture, logistics, and forest monitoring, is surging. To ensure these vehicles safely reach their destinations, navigation technology capable of precisely estimating their position and attitude in real time is essential. In typical outdoor environments, Global Navigation Satellite Systems (GNSS) serve as the primary means of position recognition. While GNSS demonstrates high reliability in open spaces, the visibility of satellite signals is significantly degraded in densely wooded areas like orchards or forests, or in urban canyons lined with high-rise buildings. In these shadowed areas, multipath phenomena occur where satellite signals are blocked or reflected off surrounding obstacles, leading to increased errors in position information; in severe cases, situations frequently arise where the position solution itself cannot be calculated. Consequently, navigation systems relying entirely on GNSS cannot guarantee continuous mission execution in environments with unstable signal reception.

[0004] Dead-reckoning technology, utilizing internal sensors of the vehicle, is used complementarily for position estimation in the event of poor GNSS signals. Wheel odometry, primarily used in vehicle-type vehicles, calculates travel distance based on the number of wheel rotations. However, on rough terrain such as unpaved roads or slippery surfaces, wheel slip is inevitable, causing a discrepancy between the actual travel distance and the calculated distance. Since wheel odometry cannot physically distinguish errors caused by such slip and simply accumulates them, the accuracy of position estimation drops sharply as the driving distance increases. Furthermore, inertial navigation using an Inertial Measurement Unit (IMU) to measure acceleration and angular velocity also contains inherent scale factor errors and bias instability in the case of low-cost MEMS sensors. Since these sensor errors cause a drift phenomenon that exponentially amplifies position errors through the integration process, there are clear limitations to using them as a means of independent navigation for extended periods unless expensive tactical-grade sensors are used.

[0005] Sensor fusion technologies utilizing LiDAR or cameras are being attempted to overcome the physical limitations of sensors. While this method corrects position by extracting feature points from the surrounding environment, it reduces the stability of navigation solutions due to matching failures or mismatches occurring in environments with repetitive tree patterns (such as orchards), significant light intensity variations, or open areas lacking feature points. Recently, research has been conducted to estimate the speed or slip of moving objects using artificial intelligence; however, existing learning-based methodologies suffer from poor estimation performance in low-speed driving sections where it is difficult to secure accurate ground truth data. Furthermore, trained models have the disadvantage of failing to reflect system biases in real-time, such as hardware aging or changes in tire pressure that occur during actual operation. Some commercial systems adopt a method of burying guide lines in the ground, but this entails high infrastructure construction costs and limitations on the operational radius.

[0006] Accordingly, there is an urgent need for technology that can robustly estimate the position of a moving object without expensive sensors or separate infrastructure, even in environments where GNSS signals are blocked or unreliable. In particular, a new type of state estimation and position correction method is required that can effectively suppress wheel slip or sensor cumulative errors, and maintain continuous navigation performance by adapting to various environmental factors and system biases that change during driving. The problem to be solved

[0007] The present invention aims to accurately estimate the ground velocity (SOG) and sideslip angle of a moving body using an internal sensor and an IMU in an irregular environment where it is difficult to receive absolute position information such as GNSS, and thereby perform stable position estimation.

[0008] In addition, the present invention aims to secure robust performance even in situations involving low-speed driving or label noise by learning trajectory-level consistency through multi-horizon kinematic integral loss.

[0009] In addition, the present invention aims to provide the ability to maintain navigation performance even during long-term operation by estimating parameters that correct the prediction bias of a neural network online when receiving absolute position information intermittently.

[0010] However, the technical problem that this embodiment aims to solve is not limited to the technical problem described above, and other technical problems may exist. means of solving the problem

[0011] A method for estimating position and motion state information of a moving body according to one embodiment of the invention may include: acquiring sensor data related to the behavior of the moving body from one or more sensors provided in the moving body; inputting the sensor data into an artificial neural network to estimate motion state information of the moving body; applying the estimated motion state information to a kinematic model of the moving body in a section where absolute position information is not provided to estimate the position of the moving body; and updating a correction parameter to compensate for an error in the motion state information estimated by the artificial neural network based on absolute position information when absolute position information is acquired from the outside.

[0012] According to one embodiment, after the step of updating the correction parameter, the step of calculating the position of the moving body by reflecting the updated correction parameter may be repeated.

[0013] According to one embodiment, the artificial neural network can learn the sideslip angle based on the error between the predicted displacement and the actual displacement calculated by integrating motion state information calculated by the artificial neural network according to a kinematic model, without using a reference label for the sideslip angle.

[0014] According to one embodiment, the artificial neural network may have a time-series neural network structure comprising at least one of a recurrent neural network, a long-term and short-term memory neural network, a gate recurrent unit, or a transformer.

[0015] According to one embodiment, the motion state information may include information related to the ground speed, sideslip angle, or direction of movement of the moving body.

[0016] According to one embodiment, sensor data related to the movement of a moving body may include acceleration data and angular velocity data obtained from an inertial sensor.

[0017] According to one embodiment, sensor data related to the movement of a moving body may further include speed information, attitude information, or internal sensor data obtained from a driving device or direction control device of the moving body.

[0018] According to one embodiment, the kinematic model of a moving body may include a model that calculates a change in the position of the moving body by integrating motion state information.

[0019] According to one embodiment, the correction parameter may include a velocity scale correction parameter, a directional angle offset correction parameter, or a combination thereof, applied to motion state information calculated by an artificial neural network.

[0020] According to one embodiment, the mobile body may be any one of a ground mobile body, a sea mobile body, an underwater mobile body, or an aerial mobile body.

[0021] An apparatus for estimating the position and motion state information of a moving body according to another embodiment of the present invention may include: a sensor data acquisition unit configured to acquire sensor data related to the behavior of the moving body from one or more sensors provided on the moving body; a motion state calculation unit configured to receive sensor data acquired from the sensor data acquisition unit and calculate motion state information of the moving body using an artificial neural network; a position estimation unit configured to estimate the position of the moving body by applying the motion state information calculated by the motion state calculation unit to a kinematic model of the moving body in a section where absolute position information is not provided; and a correction parameter update unit configured to update a correction parameter to compensate for an error in the motion state information calculated by the artificial neural network based on absolute position information when absolute position information is acquired from the outside.

[0022] According to one embodiment, after the correction parameter is updated by the correction parameter update unit, the position estimation unit may be configured to repeatedly perform the operation of calculating the position of the moving body by reflecting the updated correction parameter.

[0023] According to one embodiment, the motion state calculation unit may be configured to learn the side slip angle based on the error between the predicted displacement and the actual displacement calculated by integrating the motion state information calculated by the motion state calculation unit according to a kinematic model, without using a reference label for the side slip angle.

[0024] According to one embodiment, the motion state output unit may have a time series neural network structure comprising at least one of a recurrent neural network, a long short-term memory neural network, a gate recurrent unit, or a transformer.

[0025] According to one embodiment, the motion state information may include information related to the ground speed, sideslip angle, or direction of movement of the moving body.

[0026] According to one embodiment, the sensor data acquisition unit may be configured to acquire sensor data including acceleration data and angular velocity data acquired from an inertial sensor.

[0027] According to one embodiment, the sensor data acquisition unit may be configured to acquire sensor data including further internal sensor data acquired from the speed information, attitude information, or driving device or direction control device of the moving body.

[0028] According to one embodiment, the position estimation unit may include a kinematic model that calculates a change in the position of a moving body by integrating motion state information.

[0029] According to one embodiment, the correction parameter may include a velocity scale correction parameter, a directional angle offset correction parameter, or a combination thereof, applied to motion state information calculated by an artificial neural network. Effects of the invention

[0030] According to the present invention, position estimation accuracy can be improved and cumulative error reduced in sections where absolute position information cannot be obtained.

[0031] In addition, according to the present invention, costs can be reduced by lowering dependence on expensive GNSS / INS equipment, and the speed and side slip of a moving object can be estimated in real time even in low-spec embedded systems.

[0032] In addition, the present invention can provide generalized navigation performance in various driving environments by eliminating bias caused by sensor or environmental changes through online parameter correction performed upon receiving absolute position information.

[0033] However, the effects obtainable through the present invention are not limited to those described above, and other unmentioned technical effects will be clearly understood by a person skilled in the art from the description of the invention below. Brief explanation of the drawing

[0034] FIG. 1 is a diagram showing the overall configuration block of a device for estimating motion state information of a moving body according to one embodiment of the present invention. FIG. 2 is a block diagram that more specifically illustrates the data flow and coupling relationships between the internal components of the device shown in FIG. 1. Figure 3 is a conceptual diagram illustrating the relationship between the heading angle, sideslip angle, and direction of movement of a moving body. Figure 4 is a conceptual diagram illustrating the relationship in which the position is updated kinematically using information on the motion state of a moving body. Figure 5 is a diagram illustrating the concept of kinematic integral loss that considers multiple integral intervals during artificial neural network training. FIG. 6 is a flowchart showing the overall processing procedure of a method for estimating motion state information of a moving body according to one embodiment of the present invention. Figure 7 is a diagram showing an operation flow in which motion state calculation, kinematics prediction, and state updating based on an artificial neural network are repeatedly performed depending on whether absolute position information is provided. Specific details for implementing the invention

[0035] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0036] To clearly explain the present invention, parts unrelated to the description have been omitted from the drawings, and similar parts throughout the specification have been given similar reference numerals. Furthermore, while describing with reference to the drawings, even components indicated by the same name may have different drawing numbers depending on the drawing, and drawing numbers are indicated merely for the convenience of explanation; the concept, feature, function, or effect of each component is not to be interpreted restrictively by the corresponding drawing number.

[0037] Similar reference numerals are used for similar components when describing each drawing. Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains.

[0038] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0039] Furthermore, the expression "and / or" as used in this specification is used to include not only cases of "inclusion alone" but also cases of "inclusion together." That is, the expression "A and / or B" is a concept that encompasses "cases where only A is included," "cases where only B is included," and "cases where both A and B are included." Accordingly, when components, functions, elements, or modules, etc., described in this specification are listed using the expression "and / or," such components may be implemented individually or in any combination, and the technical interpretation thereof should be understood by reflecting this comprehensive meaning.

[0040] In this specification, "part" or "module" means a software or hardware component. However, "part" or "module" is not limited to hardware and software. "Part" or "module" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, by example, "part" or "module" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within a component and a "part" or "module" may be combined into a smaller number of components and "parts" or "modules," or separated into additional components and "parts" or "modules."

[0041] As used in this specification, the term "mobile body" may be defined as a concept encompassing all objects capable of moving and changing their position in space, and may be understood as a concept not limited by a specific structure, a specific propulsion method, or a specific operating environment. For example, a mobile body may include ground mobile bodies moving along the ground, marine and underwater mobile bodies moving on or under a water surface, and aerial mobile bodies flying in the air, and may be defined regardless of whether it is manned or unmanned, or autonomous or remotely controlled. Furthermore, a mobile body may move using wheels, tracks, propellers, thrusters, rotors, or other driving means, and the shape or size of the mobile body may not act as a limiting factor in this invention.

[0042] In this specification, "moving body" may be defined as an object for which motion state information and position need to be estimated over time, and the same conceptual processing structure may be applied even if sensor configurations, sensor placements, and movement characteristics differ depending on the type of moving body. In particular, the moving body is not defined based on a specific vehicle model, a specific axle structure, specific tire characteristics, or specific road surface conditions, and the motion characteristics of the moving body can be expressed in a generalized form through artificial neural networks and kinematic models. As such, the present invention is not limited to vehicle-centric navigation technology but can be extended to a generalized technical structure for estimating motion state information and position for various moving bodies.

[0043] Furthermore, in this specification, the term "moving body" may be defined to include cases where side slip angles, slip ratios, or similar relative motion state variables cannot be directly measured, and may be considered to enable estimation based on sensor data even in environments where such state variables are not directly measured. Accordingly, the definition of a moving body may not necessarily include slip ratio-based weights, empirical parameters dependent on specific driving states, or correction coefficients specific to a particular moving body, and these elements may not serve as prerequisites for constituting the concept of a moving body of the present invention.

[0044] An apparatus and method for estimating motion state information of a moving body according to embodiments of the present invention will be described below with reference to the attached drawings.

[0045] The device for estimating motion state information of a moving body (100) illustrated in FIG. 1 is a diagram schematically showing the overall system structure configured to stably estimate the position of a moving body even in an environment where absolute position information is not continuously provided, by taking various sensor information obtainable from the moving body as input.

[0046] Referring to FIG. 1, a device (100) for estimating motion state information of a moving body according to an embodiment of the present invention may include a sensor data acquisition unit (110), a motion state calculation unit (120), a position estimation unit (130), and a correction parameter update unit (140). Here, each component is externally represented as a single block, but this does not mean a physical whole but rather a set of logical components that are functionally combined, and each component may be implemented independently or may be implemented in software on one or more processors.

[0047] The sensor data acquisition unit (110) can acquire data generated from a sensor placed inside or coupled to the moving body, and can widely handle sensor data including physical quantities reflecting the translational motion, rotational motion, attitude change, acceleration, and deceleration state of the moving body. Such sensor data may vary depending on the shape of the moving body, driving method, propulsion means, and type of moving medium, but the sensor data acquisition unit (110) is not limited to a specific moving body structure.

[0048] Additionally, the sensor data acquisition unit (110) can be configured in a form that does not assume as input reference labels or external measurement values ​​regarding relative motion states that are difficult to measure directly, such as side slip angles or slip ratios. As a result, applicability can be maintained even in moving bodies or operating environments where it is difficult to directly measure the corresponding physical quantities.

[0049] The sensor data acquisition unit (110) can operate based on sensor data that can be acquired even in sections where the movement speed of the moving body is low, sections where the direction of movement changes rapidly, or situations where the provision of absolute position information is limited due to external environmental factors. This sensor data can be converted into motion state information that reflects temporal continuity in a subsequent step.

[0050] In some embodiments, the sensor data acquisition unit (110) is not limited to a method of transmitting the acquired sensor data as is, but may be provided to subsequent configurations in a form to which time synchronization, normalization, scaling, noise reduction processing, etc. are applied. The specific implementation of such preprocessing methods may vary depending on the type of mobile body, sensor configuration, and operating environment, and the specific processing method of sensor data and the relationship of utilization in subsequent stages will be described later in the section describing other components. Such a sensor data acquisition unit (110) can form an input layer so that subsequent stages are not limited to a specific mobile body or a specific operating environment.

[0051] The motion state calculation unit (120) can perform the role of calculating motion state information of a moving body using sensor data transmitted from the sensor data acquisition unit (110). The motion state calculation unit (120) can be configured to express dynamic characteristics that appear during the movement of the moving body in the form of state variables, and these state variables can be combined with a kinematic model and utilized in the subsequent position calculation process.

[0052] The motion state information calculated by the motion state calculation unit (120) may include the ground-based velocity of the moving body, an angle component related to the direction of movement, and a state variable that reflects the difference between the direction of movement of the moving body and the actual direction of movement. The composition or definition method of these state variables may vary depending on the type of moving body, sensor configuration, and operating environment, and the motion state calculation unit (120) can calculate state variables, such as the sideslip angle, where direct measurement or securing reference labels is limited, through estimation based on sensor data. This reflects a learning and estimation structure that does not rely on external measurement values ​​for specific state variables.

[0053] The motion state calculation unit (120) can be implemented as a structure including an artificial neural network, and the artificial neural network may have a time series processing structure capable of reflecting the temporal continuity of sensor data. This time series processing structure can be implemented in various forms, such as a neural network having a recurrent structure, a neural network including a long short-term memory structure, a gate-based recurrent structure, or a structure including an attention mechanism, and can be configured to be replaceable with various learning structures rather than being limited to a specific neural network structure. Through this, the motion state calculation unit (120) can effectively express the non-linear relationship between sensor data and motion state information.

[0054] In addition, the motion state calculation unit (120) can utilize a data-based learned representation rather than relying solely on a physical model in the process of calculating motion state information. Accordingly, applicability can be maintained even when the motion characteristics of the moving body are difficult to explain by simple kinematic relationships, and can be extended to provide generalized estimation performance even in environments where the driving conditions of the moving body, the moving medium, or disturbance factors change.

[0055] The position estimation unit (130) can estimate the change in position of the moving body by receiving motion state information calculated by the motion state calculation unit (120) as input. The position estimation unit (130) can be configured to continuously calculate the change in position that occurs as the moving body moves over time, and the kinematic relationship of the moving body can be considered together during this process. This position calculation operation can be performed based on a generalized kinematic relationship that is applicable regardless of the shape or driving method of the moving body.

[0056] The position estimation unit (130) may have a structure that allows it to continue performing position estimation even when absolute position information is not continuously provided from the outside. Accordingly, the position estimation unit (130) can maintain position calculation based on motion state information regardless of whether absolute position information is received, and if absolute position information is provided, it can be utilized to adjust subsequent position calculation results based on said information.

[0057] In the position calculation process, the position estimation unit (130) can derive the direction of movement of the moving body using state variables representing the ground-based speed and direction of movement of the moving body. At this time, the direction of movement is not limited to simple heading values, but can be determined by considering state variables that reflect the difference between the actual movement path and the direction of movement of the moving body. For example, if a state variable representing the relative deviation between the direction of movement of the moving body and the actual direction of movement is included, the position estimation unit (130) can use this information to express the direction of movement more realistically.

[0058] Additionally, the position estimation unit (130) can calculate the change in position by integrating the movement direction and velocity information over time, and this integration process can be implemented in various ways depending on the movement characteristics of the moving body. A structure that updates the position based on the movement direction defined through a kinematic model can operate in a way that maintains relative consistency between motion state information even in an environment where sensor errors accumulate over time. As a result, the position calculation result can be continuously updated without abrupt fluctuations.

[0059] Furthermore, the position estimation unit (130) can repeatedly perform position calculation operations even after the correction parameters calculated by the correction parameter update unit are reflected. Accordingly, the position of the moving body can be updated in a form that reflects both the state before and after correction, and through this repeated execution structure, the stability of the position estimation result can be maintained even during long-term operation. More detailed information regarding the configuration of the kinematic model of the position estimation unit (130) will be described later with reference to FIGS. 3 and 4.

[0060] When absolute position information is obtained from an external source, the correction parameter update unit (140) can adjust the error included in the motion state information of the moving body by using the difference between the absolute position information and the position calculated by the position estimation unit (130). The correction parameter update unit (140) does not presuppose an environment where absolute position information is always provided, and operates only at the time when absolute position information is provided to update the correction parameter to be reflected in the subsequent position calculation process.

[0061] The correction parameter update unit (140) may not be based on a structure in which the artificial neural network included in the motion state calculation unit (120) directly outputs the correction parameter. Instead, it may operate by estimating the correction parameter using the difference between the motion state information calculated by the artificial neural network and the actual position. According to this configuration, the correction parameter update unit (140) can mitigate the accumulation of errors during the operation phase while being separated from the learning process of the artificial neural network.

[0062] The correction parameters updated by the correction parameter update unit (140) may include forms such as speed scale correction values ​​or directional angle offset correction values ​​applied to motion state information, and may be extended to correction parameters corresponding to multiple state variables depending on the type of moving body or sensor configuration. These correction parameters are not limited to a single value and may be managed as a set of state variables that are updated over time.

[0063] The correction parameter update unit (140) may be implemented using a filter structure that performs state estimation and error correction, which may be one example of implementation utilizing the relationship between the artificial neural network output and absolute position information. However, such a filter structure is merely one of several methods for implementing the function of the correction parameter update unit (140) and may not be limited to a specific algorithm or a specific mathematical model. Accordingly, the correction parameter update unit (140) may be implemented by applying various state estimation techniques or error correction techniques.

[0064] The correction parameter update unit (140) can transmit the updated correction parameter to the position estimation unit (130) so that it is reflected in the subsequent position calculation process. In this case, even in sections where absolute position information is not provided again, the position calculation operation can be repeated while maintaining the correction parameter updated immediately prior. Through this iterative structure, the position of the moving body can be continuously updated even during long-term operation.

[0065] FIG. 2 illustrates the overall data flow in which sensor data generated inside a moving body is input into an artificial neural network (220), and the output of the artificial neural network (220) is calculated as a system position through a state estimation structure.

[0066] In FIG. 2, sensor measurement data can be input into an artificial neural network (220) from an internal sensor (211) of a moving body, an inertial measurement unit (212), and moving body attitude information (213). The artificial neural network (220) can calculate motion state information of the moving body by reflecting the temporal changes and interrelationships of these sensor data, and in FIG. 2, the motion state information is exemplified as the moving body velocity and sideslip angle. The output of the artificial neural network (220) is transmitted to the prediction stage (231) of a Kalman filter (230) and can be used to calculate a predicted position based on a kinematic model.

[0067] The internal sensor (211) of the moving body may include sensor data generated by functionally linking with the drive system, direction control system, or controller inside the moving body. The sensor data generated by the internal sensor (211) of the moving body may consist of physical quantities related to the propulsion state, direction control state, or internal control commands of the moving body. For example, in the case of a ground moving body, it may be extended to information such as wheel rotation speed, propulsion output, and steering input; in the case of a sea or underwater moving body, it may be substituted with thruster rotation speed, thrust command, and direction control input; and in the case of an aerial moving body, it may be extended to rotor rotation speed, thrust distribution information, and steering input. Such internal sensor (211) of the moving body may function as a source of internal state information that is not dependent on a specific moving body shape or a specific drive method.

[0068] The inertial measurement device (212) may be composed of a group of inertial sensors that measure the linear acceleration and angular velocity of a moving body, and may provide an input signal that can continuously observe the translational and rotational motion of the moving body over time. Since the measurement values ​​of the inertial measurement device (212) may have the characteristic of accumulating bias and noise during the integration process, there may be limitations in providing a stable absolute position over a long period of time on its own. In the structure of FIG. 2, the inertial measurement device (212) is used as an input to an artificial neural network (220), and by the artificial neural network (220) converting the temporal pattern of the inertial data into a learned representation, the cumulative error characteristics of the inertial data can be mitigated during the operation phase.

[0069] The attitude information (213) of the moving body may consist of information indicating attitude states such as roll, pitch, and heading of the moving body, and may be generated as a result of combined estimation with an attitude sensor, an attitude estimator, or an inertial measurement device (212). As shown in FIG. 3, the heading angle θ represents the direction in which the body coordinate system of the moving body faces the reference coordinate system and may not coincide with the actual direction of movement of the moving body. The actual direction of movement of the moving body may be defined as a combination of the heading angle θ and the sideslip angle β, and this combined relationship may be expressed as an angle component representing the direction of movement relative to the ground.

[0070] Meanwhile, the mobile body in FIG. 3 and FIG. 4, which will be described later, is depicted in the form of a vehicle for convenience of explanation, but this is merely one embodiment, and the mobile body defined in the present invention can be extended to include a ground mobile body, a sea mobile body, an underwater mobile body, or an aerial mobile body.

[0071] The artificial neural network (220) can calculate motion state information of the moving body by receiving as input the time-series characteristics of sensor data provided from the moving body internal sensor (211), the inertial measurement unit (212), and the moving body attitude information (213). In FIG. 3, the output of the artificial neural network (220) is shown as the moving body velocity and the sideslip angle β, and the moving body velocity can be interpreted as a translational velocity component with respect to the reference coordinate system. The sideslip angle β represents the angle difference between the heading direction in which the moving body is facing and the actual direction of movement, and may correspond to a state variable that is difficult to measure directly with an external sensor. Accordingly, the artificial neural network (220) can be configured as a learning structure that does not rely on a reference label for the sideslip angle β.

[0072] The motion state information output by the artificial neural network (220) can be converted into a predicted position change over time through the kinematic integral relationship shown in FIG. 4, and the difference between the predicted displacement and the actual displacement can be calculated as a loss. In FIG. 4, the heading angle θ at time point i. i and sideslip angle β i Given , the direction of motion of the moving body can be expressed as a combination of two angles, and the angular velocity ω i With the time interval dt applied, the direction θ of time point i+1 i+1 This can be calculated. This kinematic integral structure is the position coordinate x i , y i a x i+1 , y i+1 It can be clearly explained through the process of updating.

[0073] In FIG. 5, the multi-horizon integral loss can simultaneously consider the error between the predicted displacement and the actual displacement for multiple horizons H having different integration ranges. Short horizons can reflect short-term consistency, and long horizons can reflect long-term accumulated error; furthermore, considering that the error accumulation characteristic increases as the integration range lengthens, this can be extended by setting different weights for each horizon. This loss design can maintain a structure that does not rely on slip rate-based weights or specific moving object-dependent parameters.

[0074] A Kalman filter (230) may be illustrated as an embodiment of a state estimation structure that calculates system position and correction parameters by combining the output of an artificial neural network (220), a kinematic model, and an absolute position (240). In FIG. 2, the Kalman filter (230) is divided into a prediction step (231) and an update step (232), and the prediction step (231) can generate a predicted state by time-updating the previous state according to the kinematic model. The update step (232) operates only when an absolute position (240) is provided and can correct the state using the difference between the predicted state and the absolute position measurement. In this process, correction parameters corresponding to the bias or scale error of the output of the artificial neural network (220) can be estimated.

[0075] The prediction step (231) can be performed continuously even in sections where absolute position information is not provided, and the position prediction can be maintained by repeatedly applying the kinematic relationships illustrated in FIGS. 3 and FIGS. 4. The update step (232) is performed only when the absolute position (240) is provided, and the updated correction parameters are reflected in subsequent prediction steps so that position calculation can be performed repeatedly. This structure can form an operational structure in which position estimation is not interrupted even in an environment where absolute position information is provided intermittently.

[0076] FIG. 6 is a flowchart illustrating a procedure in which a method for estimating the motion state and position of a moving body according to an embodiment of the present invention is performed in chronological order.

[0077] Step S610 is a step of generating internal sensors, inertial sensors, and attitude information of the moving body, wherein basic sensor data reflecting the behavioral state of the moving body may be generated from sensors equipped inside the moving body or functionally coupled with the moving body. In Step S610, the internal sensors of the moving body may generate data including physical quantities associated with the propulsion state, direction control state, or internal control commands of the moving body, and the inertial sensors may measure the linear acceleration and angular velocity of the moving body to continuously express the translational and rotational motion of the moving body over time, and the attitude information of the moving body may be generated as state quantities representing the spatial attitude of the moving body, such as roll, pitch, and heading.

[0078] The sensor data generated in step S610 can consist of data that can be acquired regardless of whether absolute position information is provided, and since it can be continuously generated even in sections where the moving speed of the moving body is low or where the direction of movement changes rapidly, it can be used as input for artificial neural network-based inference and kinematics-based position estimation performed in subsequent steps.

[0079] Step S620 is a step of inputting sensor data generated in Step S610 into an artificial neural network to calculate motion state information of the moving body, and the ground-referenced velocity and sideslip angle of the moving body can be predicted using the artificial neural network. Here, the ground-referenced velocity can be defined as the translational velocity component of the moving body moving relative to the reference coordinate system, and the sideslip angle can be defined as a state variable representing the angular difference between the heading direction in which the body of the moving body is facing and the direction in which the moving body is actually moving. In Step S620, state variables such as the sideslip angle, which are difficult to measure directly with external sensors or for which the acquisition of reference labels is limited, can be calculated through the learned representation of the artificial neural network, and this is based on an inference structure utilizing the temporal continuity and cross-correlation of the sensor data.

[0080] Additionally, in step S630, the direction of movement of the moving body relative to the surface can be calculated through the combination of the heading angle of the moving body and the predicted sideslip angle, and this direction of movement information can be used as a direct input for kinematic integration performed in a subsequent step.

[0081] Step S630 is a position estimation step performed in a section where absolute position information is not provided, and the position of the moving body can be estimated using the motion state information and the kinematic model of the moving body calculated in Step S620. In Step S630, the change in position can be calculated by integrating the ground-referenced velocity and the surface-referenced direction of movement of the moving body over time, and a kinematic integration relationship that considers both angular velocity and time interval can be applied to perform a position update from time point i to time point i+1. Step S630 can be performed repeatedly while absolute position information is not provided, thereby allowing the position estimation of the moving body to be continuously maintained even in the section where absolute position information is absent.

[0082] Step S640 is a correction parameter estimation and update step performed when absolute position information is secured, in which correction parameters corresponding to the ground-based velocity and surface-based movement direction of the moving object can be estimated and updated. In Step S640, the difference between the absolute position information and the position predicted in Step S630 can be calculated, and scale correction parameters for velocity or angle offset correction parameters for movement direction can be calculated in a direction that reduces the difference. The correction parameters estimated in Step S640 are not limited to values ​​directly output by the artificial neural network, but can be treated as parameters estimated in the operation phase based on the discrepancy between the output of the artificial neural network and the actual position.

[0083] Step S650 is a step of correcting the position of a moving object using the correction parameters updated in Step S640, and the position of the moving object can be recalculated based on the corrected velocity value and the corrected direction of movement. In Step S650, the position estimation result can be corrected at the point where absolute position information is provided, and in the section where absolute position information is not provided again thereafter, Step S630 can be repeated based on the corrected state. In this way, Steps S630, S640, and S650 can be performed cyclically depending on whether absolute position information is provided, and the repetition arrow structure illustrated in FIG. 6 represents this procedural cyclic relationship.

[0084] As described above, the steps illustrated in FIG. 6 form a procedural structure that enables the motion state and position of a moving body to be continuously estimated even in an environment where absolute position information is provided intermittently, and the overall flow of the present invention, which estimates and corrects state variables that are difficult to measure directly, such as sideslip angles, through artificial neural networks and kinematic integral relationships without reference labels, is explained step by step.

[0085] FIG. 7 is a flowchart illustrating an overall operation algorithm in which the motion state estimation and position calculation of a moving body are performed repeatedly over time according to an embodiment of the present invention, showing a process in which a motion state calculation step using an artificial neural network, a kinematics-based prediction step, and a state update and online correction step performed when absolute position information is provided are combined into a single closed-loop structure.

[0086] Step S710 corresponds to a step of calculating motion state information of a moving body using an artificial neural network, and motion state information corresponding to the ground-referenced velocity and sideslip angle of the moving body can be predicted by using time-series sensor data transmitted from a sensor data acquisition unit as input. An artificial neural network that performs motion state calculation according to an embodiment of the present invention can be implemented with a structure capable of reflecting the temporal continuity of sensor data collected in a time-series manner and the dependency relationship between states, and can perform state estimation that reflects change patterns over time and cumulative effects, rather than static estimation based on sensor data at a single point in time.

[0087] Examples of such time-series neural network structures may include recurrent neural network structures, long short-term memory structures, gate-based recurrent structures, and structures including self-attention mechanisms; however, they are not limited to a specific structure and can be replaced with various neural network structures capable of mapping time-series inputs to motion state outputs.

[0088] The input to the artificial neural network may consist of multidimensional time-series data including internal sensor data, inertial sensor data, and attitude information of the moving body, and the output of the artificial neural network may consist of the moving body's ground-relative velocity, sideslip angle, or motion state variables directly related thereto. In this case, the artificial neural network does not require direct reference labels for specific motion state variables as input, but can produce output values ​​by converting the inherent relationship between the sensor data and the changes in the moving body's motion into a learned representation.

[0089] Step S720 corresponds to a step where the output of the artificial neural network is input into the prediction stage of the navigation filter to perform kinematics-based state prediction. In Step S720, the velocity and sideslip angle of the moving body output by the artificial neural network, along with heading and angular velocity information that may be provided from the attitude information of the moving body, are combined to calculate the direction of movement of the moving body, and a change in position can be predicted using the corresponding direction of movement and velocity. This prediction stage may correspond to the prediction process of the filter, and the output of the artificial neural network may be directly used in the dead reckoning operation.

[0090] In cases where absolute position information is not provided or reliability is reduced, in step S730, only the prediction step is repeated so that the system position of the moving object can be continuously calculated using a dead reckoning method. According to this structure, position estimation can be maintained without interruption even in environments where GNSS signals are blocked or inaccurate.

[0091] In step S740, the availability of absolute position information may be determined, and a subsequent update step may be performed only if the absolute position information is provided with an accuracy exceeding a certain standard. If absolute position information is provided, the navigation filter update process may be performed in steps S750 and S760, and during this process, the accumulated position error may be corrected using the absolute position information. At this time, the update process may not merely correct the current position but may form a structure that also estimates correction parameters corresponding to low-frequency bias components or scale errors that may be included in the artificial neural network output.

[0092] Specifically, during the update process of the navigation filter, scale correction parameters for the moving object's velocity and offset correction parameters for the angular components related to the direction of movement can be estimated. The scale correction parameter for the moving object's velocity can be used to compensate for velocity scale errors that may occur due to changes in the effective tire radius, alignment errors between sensors, changes in platform characteristics, etc., while the offset correction parameter for the angular components related to the direction of movement can be used to compensate for directional errors that occur in environments where it is difficult to secure an accurate heading.

[0093] The original kinematic relationship to which these online correction parameters are not applied can be expressed as Equations 1 and 2 below.

[0094] (Mathematical Formula 1)

[0095]

[0096] (Mathematical Formula 2)

[0097]

[0098] Here is the point in time i The position coordinates of the moving object at, is the ground velocity (SOG) of a moving object inferred by an artificial neural network, is the sideslip angle inferred by the artificial neural network, is the heading angle of the moving object measured by the sensor, represents the rotational angular velocity of the moving body measured by the inertial sensor.

[0099] In contrast, the velocity scale correction parameter estimated during the navigation filter update process and movement direction offset correction parameter When applied, the kinematic relationship can be defined as Equations 3 and 4 below.

[0100] (Mathematical Formula 3)

[0101]

[0102] (Mathematical Formula 4)

[0103]

[0104] The correction parameters estimated in this way can be applied even in sections where absolute position information is not provided again to improve the accuracy of dead reckoning, and by using absolute position information during the navigation filter update process to correct accumulated position errors, it enables the artificial neural network output used in the subsequent prediction step to be utilized in a more accurate state.

[0105] Meanwhile, the operational structure illustrated in FIG. 7 can be directly linked to the learning method of an artificial neural network. Artificial neural network learning according to one embodiment of the present invention is not limited to a method using the correct label of the motion state itself, but can be performed based on the consistency of the position displacement calculated by integrating the motion state information output by the artificial neural network into the kinematic model of the moving body. Since navigation performance can be evaluated by position accuracy calculated using ground-based velocity and direction of movement, a kinematic loss function based on position displacement can be used in the present invention.

[0106] The present invention is not limited to a method of using only the displacement error for a single integration interval as the loss, but may use a multi-horizon kinematic integral loss function that simultaneously considers the lengths of multiple integration intervals shown in FIG. 5. Such a loss function can be expressed in the form of Equation 5 below.

[0107] (Mathematical Formula 5)

[0108]

[0109] Here, the predicted displacement calculated by integrating from time t to t+H using the artificial neural network output and moving body kinematics and the actual displacement calculated from a reference sensor or external reference means The error between them can be reflected as loss. Here, H represents the length of the time interval during which kinematic integration is performed, and S represents the total length of the time series data used for training, can represent the number of different integration interval lengths.

[0110] In the multi-horizon kinematic integral loss function, short and long integral intervals can be considered simultaneously, allowing short-term state consistency and long-term cumulative error characteristics to be learned together. Additionally, to mitigate the characteristic where the range of displacement error increases as the integral interval length increases, a normalization factor may be applied according to the integral interval length, and this normalization factor may be set in the form of 1 / H, 1 / √H, or an equivalent function.

[0111] According to this learning structure, instead of directly overfitting to the velocity or angle values ​​at individual time points, the artificial neural network can be learned based on the consistency of the movement trajectory formed by integration over time, and relatively stable motion state estimation becomes possible even in low-speed sections or sections with large disturbances. In addition, when combined with the navigation filter-based online correction structure shown in Fig. 7, low-frequency bias components that are difficult to remove during the learning phase can be gradually corrected during the operation phase.

[0112] At least some of the configurations of the embodiments described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an Arithmetic Logic Unit (ALU), a Digital Signal Processor, a microcomputer, a Field Programmable Gate Array (FPGA), a Programmable Logic Unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions.

[0113] The processing unit may execute an operating system and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, the processing unit may be described as being used as a single unit, but a person of ordinary skill in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements.

[0114] For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible. Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure the processing unit to operate as desired or command the processing unit independently or collectively.

[0115] Software and / or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by a processing device or to provide instructions or data to a processing device. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0116] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software.

[0117] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiments, and vice versa.

[0118] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art to which the present invention pertains will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive, and the scope of the present invention is defined by the claims set forth below, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0119] 100: Moving object motion state information estimation device 110: Sensor data acquisition unit 120: Motion state output unit 130: Position calculation unit 140: Correction parameter update section 211, 212, 213: Sensor modules 220: Artificial Neural Network 230: Kalman filter 231: Prediction step 232: Update phase 240: Absolute position

Claims

Claim 1 A step of acquiring sensor data related to the behavior of the moving body based on internal sensors of the moving body, an inertial measurement unit, and attitude information of the moving body; a step of inputting the sensor data into a time-series artificial neural network to calculate motion state information including information related to the ground velocity, sideslip angle, and direction of movement of the moving body; a step of calculating a reference direction of movement of the moving body by combining the heading angle and the sideslip angle of the moving body in a section where absolute position information is not provided, calculating a change in position by time integrating the ground velocity and the reference direction of movement according to the kinematic model of the moving body, and estimating the position of the moving body by accumulating the change in position and updating the predicted position of the moving body. A method for estimating motion state information of a moving body, comprising the step of updating a correction parameter to compensate for an error in the motion state information calculated by the time series artificial neural network based on the absolute position information when absolute position information is obtained from an external source, wherein the absolute position information is used to update the correction parameter but is not used as an input to the time series artificial neural network, and the time series artificial neural network is characterized by not using a reference label for the sideslip angle, calculating a predicted displacement by integrating the ground velocity and the sideslip angle according to the kinematic model for each of a plurality of integration intervals of different lengths, and being trained using a multi-horizon kinematic integration loss that includes the error between the predicted displacement and the actual displacement in each integration interval. Claim 2 A method for estimating motion state information of a moving body according to claim 1, wherein, after the step of updating the correction parameter, the step of calculating the position of the moving body by reflecting the updated correction parameter is repeatedly performed. Claim 3 delete Claim 4 A method for estimating motion state information of a moving body according to claim 1, wherein the time series artificial neural network has a time series neural network structure comprising at least one of a recurrent neural network, a long short-term memory neural network, a gate recurrent unit, or a transformer. Claim 5 delete Claim 6 A method for estimating motion state information of a moving body according to claim 1, wherein the sensor data related to the behavior of the moving body includes acceleration data and angular velocity data obtained from an inertial sensor. Claim 7 A method for estimating motion state information of a moving body according to claim 6, wherein the sensor data related to the behavior of the moving body further includes speed information, attitude information, or internal sensor data obtained from a driving device or direction control device of the moving body. Claim 8 delete Claim 9 A method for estimating motion state information of a moving body according to claim 1, wherein the correction parameter comprises a velocity scale correction parameter, a directional angle offset correction parameter, or a combination thereof, applied to the motion state information calculated by the time series artificial neural network. Claim 10 A method for estimating motion state information of a moving body according to claim 1, wherein the moving body is any one of a ground moving body, a sea moving body, an underwater moving body, or an aerial moving body. Claim 11 A device for estimating motion state information of a moving body, comprising: a sensor data acquisition unit configured to acquire sensor data related to the behavior of the moving body based on an internal sensor of the moving body, an inertial measurement unit, and attitude information of the moving body; a motion state calculation unit configured to receive the sensor data acquired from the sensor data acquisition unit and to calculate motion state information including information related to the ground speed, sideslip angle, and direction of movement of the moving body using a time-series artificial neural network; and a position estimation unit configured to estimate the position of the moving body by combining the motion state information calculated by the motion state calculation unit and the kinematic model of the moving body in a section where absolute position information is not provided, wherein the reference direction of movement of the moving body is calculated by combining the heading angle of the moving body and the sideslip angle, the change in position is calculated by time integrating the ground speed and the reference direction of movement according to the kinematic model of the moving body, and the predicted position of the moving body is updated by accumulating the change in position. A device for estimating motion state information of a moving body, comprising a correction parameter update unit configured to update a correction parameter to compensate for an error in the motion state information calculated by the time series artificial neural network based on the absolute position information when absolute position information is obtained from the outside, wherein the absolute position information is used to update the correction parameter but is not used as an input to the time series artificial neural network, and the time series artificial neural network is characterized by not using a reference label for the sideslip angle, and calculating a predicted displacement by integrating the ground velocity and the sideslip angle according to the kinematic model for each of a plurality of integration intervals of different lengths, and learning using a multi-horizon kinematic integration loss that includes the error between the predicted displacement and the actual displacement in each integration interval. Claim 12 An apparatus for estimating motion state information of a moving body, wherein, in claim 11, after the correction parameter is updated by the correction parameter update unit, the position estimation unit is configured to repeatedly perform the operation of calculating the position of the moving body by reflecting the updated correction parameter. Claim 13 delete Claim 14 An apparatus for estimating motion state information of a moving body, wherein the time series artificial neural network comprises a time series neural network structure including at least one of a recurrent neural network, a long short-term memory neural network, a gate recurrent unit, or a transformer, according to claim 11. Claim 15 delete Claim 16 The apparatus for estimating motion state information of a moving body according to claim 11, wherein the sensor data acquisition unit is configured to acquire sensor data including acceleration data and angular velocity data acquired from an inertial sensor. Claim 17 A device for estimating motion state information of a moving body, wherein the sensor data acquisition unit is configured to acquire sensor data that further includes speed information, attitude information, or internal sensor data acquired from a driving device or direction control device of the moving body according to claim 16. Claim 18 delete Claim 19 An apparatus for estimating motion state information of a moving body, wherein the correction parameter comprises a velocity scale correction parameter, a direction angle offset correction parameter, or a combination thereof, applied to the motion state information calculated by the time series artificial neural network according to claim 11. Claim 20 A device for estimating motion state information of a mobile body, wherein the mobile body is any one of a ground mobile body, a sea mobile body, an underwater mobile body, or an aerial mobile body, in claim 11.

Images