A combined tracking method and apparatus

Abstract

The application provides a combination tracking method and device for selecting a suitable tracking mode in automatic driving, comprising: judging the available state of a GNSS combination inertial navigation tracking mode and the available state of a visual tracking mode; in response to the GNSS combination inertial navigation tracking mode being available and the visual tracking mode being unavailable, using the GNSS combination inertial navigation tracking mode; in response to the GNSS combination inertial navigation tracking mode being unavailable and the visual tracking mode being available, using the visual tracking mode; in response to the GNSS combination inertial navigation tracking mode and the visual tracking mode both being unavailable, exiting automatic tracking and issuing an alarm to remind a user to take over; and in response to the GNSS combination inertial navigation tracking mode and the visual tracking mode both being available, selecting one of the GNSS combination inertial navigation tracking mode and the visual tracking mode based on the state of the vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, and more particularly to a combined tracking method and apparatus in the field of autonomous driving. Background Technology

[0002] In recent years, intelligent driving vehicles have experienced unprecedented development. Autonomous tracking is one of the core technologies of intelligent driving vehicles. Currently, cameras are commonly used in intelligent driving vehicles to recognize lane lines and achieve automatic tracking.

[0003] Vision-based tracking involves mounting image recognition sensors on intelligent vehicles to extract, classify, and track lane line features. Feedback control then adjusts the vehicle's heading in real time to follow the desired path, achieving trajectory following. Its main advantage is that it can perform digital track recognition and tracking offline, but it is significantly affected by the external environment. However, vision-based tracking methods are inevitably affected by external factors such as lighting and shadow interference, insufficient light at night, and glare from rain-soaked roads. Furthermore, high traffic volume on certain road sections exacerbates lane line wear. These factors can easily cause visual recognition failure, leading to tracking takeover and the loss of autonomous tracking capabilities. In addition, due to the limited field of view of camera lenses, visual tracking struggles to track sharp curves and has low cornering speeds, all of which contribute to the low availability of vision-based tracking systems.

[0004] Tracking based on GNSS-inertial navigation systems primarily relies on real-time indexing of pre-aiming points within a smoothed path trajectory using vehicle status and GNSS positioning signals. The steering command for the lead vehicle's front axle is calculated based on the lateral and heading deviations between the pre-aiming points and the vehicle's position, and then output to the steering actuator to achieve trajectory following. Currently, the positioning accuracy of civilian GNSS is around 10 meters. Using real-time kinematic (RTK) technology, the accuracy can be improved to 1-2 cm, meeting the tracking and positioning accuracy requirements of intelligent driving vehicles. The advantage of GNSS-inertial navigation-based tracking is its all-weather capability, unaffected by ambient light or weather conditions. However, its signal is blocked by buildings, and tracking fails in areas where GNSS signals are unavailable, such as tunnels.

[0005] To overcome the aforementioned shortcomings of existing technologies, there is an urgent need in this field for a combined tracking method that integrates visual tracking and GNSS-inertial navigation tracking modes. By using an adaptive tracking mode switching method, this method can compensate for the poor curve-passing performance and susceptibility to external environmental influences of vision-based automatic tracking systems. It can also complement the advantages of visual recognition and GNSS-inertial navigation technologies, thereby reducing the frequency of tracking takeover by intelligent driving vehicles, improving autonomous tracking performance, and enhancing the availability of automatic tracking systems. Summary of the Invention

[0006] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0007] To overcome the aforementioned deficiencies in the prior art, this invention provides a combined tracking method for selecting an applicable tracking mode in autonomous driving, comprising: determining the availability of a GNSS combined inertial navigation tracking mode and a visual tracking mode; in response to the availability of the GNSS combined inertial navigation tracking mode and the unavailability of the visual tracking mode, using the GNSS combined inertial navigation tracking mode; in response to the unavailability of the GNSS combined inertial navigation tracking mode and the availability of the visual tracking mode, using the visual tracking mode; in response to the unavailability of both the GNSS combined inertial navigation tracking mode and the visual tracking mode, exiting automatic tracking and issuing an alarm to remind the user to take over; and in response to the availability of both the GNSS combined inertial navigation tracking mode and the visual tracking mode, selecting one of the two tracking modes based on the vehicle status.

[0008] In one embodiment, preferably, determining the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode includes: acquiring road environment information, which includes mandatory road environment information; and determining the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on the mandatory road environment information.

[0009] In one embodiment, preferably, the mandatory road environment information includes lane line occlusion information and GNSS signal occlusion information; determining the availability status of the GNSS integrated inertial navigation tracking mode and the availability status of the visual tracking mode based on the mandatory road environment information includes: the visual tracking mode being unavailable in response to lane line occlusion; and the GNSS integrated inertial navigation tracking mode being unavailable in response to GNSS signal occlusion.

[0010] In one embodiment, preferably, the available states of the GNSS integrated inertial navigation tracking mode include the working state of the GNSS and the life signal state of the GNSS integrated inertial navigation device, and the available states of the visual tracking mode include the visual recognition function state and the life signal state of the tracking camera device.

[0011] In one embodiment, preferably, the GNSS operating states include point positioning, point orientation, RTK floating-point delocalization, RTK floating-point delocalization, RTK fixed delocalization, and RTK fixed delocalization; determining the availability of the GNSS integrated inertial navigation tracking mode and the availability of the visual tracking mode includes: when the GNSS operating state is point positioning, point orientation, RTK floating-point delocalization, or RTK floating-point delocalization, the GNSS integrated inertial navigation tracking mode is unavailable; and when the GNSS operating state is RTK fixed delocalization or RTK fixed delocalization, the GNSS integrated inertial navigation tracking mode is available.

[0012] In one embodiment, preferably, the vehicle state includes vehicle speed; selecting one of a GNSS integrated inertial navigation tracking mode and a visual tracking mode based on the vehicle state includes: using the GNSS integrated inertial navigation tracking mode when the vehicle speed exceeds a preset speed threshold; and using the visual tracking mode when the vehicle speed does not exceed the preset speed threshold.

[0013] In one embodiment, preferably, selecting either the GNSS integrated inertial navigation tracking mode or the visual tracking mode based on the vehicle state further includes: acquiring the curvature of the road segment; when the condition is met... When using the GNSS integrated inertial navigation tracking mode; and when the following conditions are met... When using the visual tracking mode, k1 is the curvature scaling factor.

[0014] In one embodiment, preferably, selecting one of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on the vehicle status further includes: obtaining the visual tracking road segment failure rate and the GNSS integrated inertial navigation tracking road segment failure rate.

[0015] When satisfied At that time, the GNSS integrated inertial navigation tracking mode is used; and

[0016] When satisfied When using visual tracking mode, k2 is the failure rate scaling factor.

[0017] In one embodiment, preferably, the preset speed threshold is a threshold range to avoid frequent switching of the tracking mode.

[0018] In one embodiment, preferably, an angle increment limit is introduced during the switching process between the GNSS combined inertial navigation tracking mode and the visual tracking mode to ensure a smooth transition of the tracking mode.

[0019] Another aspect of the present invention provides a combined tracking device, comprising: a memory; and a processor coupled to the memory, the processor being configured to: determine the availability of a GNSS combined inertial navigation tracking mode and a visual tracking mode; in response to the availability of the GNSS combined inertial navigation tracking mode and the unavailability of the visual tracking mode, use the GNSS combined inertial navigation tracking mode; in response to the unavailability of the GNSS combined inertial navigation tracking mode and the availability of the visual tracking mode, use the visual tracking mode; in response to the unavailability of both the GNSS combined inertial navigation tracking mode and the visual tracking mode, exit automatic tracking and issue an alarm to remind the user to take over; and in response to the availability of both the GNSS combined inertial navigation tracking mode and the visual tracking mode, select one of the GNSS combined inertial navigation tracking mode and the visual tracking mode based on the vehicle status.

[0020] In one embodiment, preferably, the processor is further configured to: acquire road environment information, including mandatory road environment information; and determine the availability status of the GNSS integrated inertial navigation tracking mode and the availability status of the visual tracking mode based on the mandatory road environment information.

[0021] In one embodiment, preferably, the mandatory road environment information includes lane line occlusion information and GNSS signal occlusion information, and the processor is further configured to: disable the visual tracking mode in response to lane line occlusion; and disable the GNSS combined inertial navigation tracking mode in response to GNSS signal occlusion.

[0022] In one embodiment, preferably, the available states of the GNSS integrated inertial navigation tracking mode include the working state of the GNSS and the life signal state of the GNSS integrated inertial navigation device, and the available states of the visual tracking mode include the visual recognition function state and the life signal state of the tracking camera device.

[0023] In one embodiment, preferably, the GNSS operating states include point positioning, point orientation, RTK floating-point delocalization, RTK floating-point deorientation, RTK fixed delocalization, and RTK fixed deorientation. The processor is further configured to: when the GNSS operating state is point positioning, point orientation, RTK floating-point delocalization, or RTK floating-point deorientation, the GNSS integrated inertial navigation tracking mode is unavailable; and when the GNSS operating state is RTK fixed delocalization or RTK fixed deorientation, the GNSS integrated inertial navigation tracking mode is available.

[0024] In one embodiment, preferably, the vehicle state includes vehicle speed; the processor is further configured to: use a GNSS combined inertial navigation tracking mode when the vehicle speed exceeds a preset speed threshold; and use a visual tracking mode when the vehicle speed does not exceed the preset speed threshold.

[0025] In one embodiment, preferably, the processor is further configured to: acquire the curvature of the road segment; when the condition is met... When using the GNSS integrated inertial navigation tracking mode; and when the following conditions are met... When using the visual tracking mode, k1 is the curvature scaling factor.

[0026] In one embodiment, preferably, the processor is further configured to: acquire the visual tracking segment failure rate and the GNSS combined inertial navigation tracking segment failure rate;

[0027] When satisfied At that time, the GNSS integrated inertial navigation tracking mode is used; and

[0028] When satisfied When using visual tracking mode, k2 is the failure rate scaling factor.

[0029] In one embodiment, preferably, the preset speed threshold is a threshold range to avoid frequent switching of the tracking mode.

[0030] In one embodiment, preferably, the processor is further configured to: introduce an angle increment limiter during the switching between the GNSS combined inertial navigation tracking mode and the visual tracking mode to ensure a smooth transition of the tracking mode.

[0031] The present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described combined tracking methods. Attached Figure Description

[0032] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0033] Figure 1 This is a schematic flowchart illustrating a combined tracking method according to an embodiment of one aspect of the present invention;

[0034] Figure 2 This is a schematic diagram illustrating the judgment logic and tracking mode switching in a combined tracking method according to an embodiment of the present invention; and

[0035] Figure 3 This is a schematic diagram of the combined tracking device according to another embodiment of the present invention. Detailed Implementation

[0036] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a thorough understanding of the invention, many specific details will be included in the following description. The invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0037] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0038] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0039] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0040] A GNSS (Global Navigation Satellite System) is a space-based radio navigation and positioning system that uses observations such as pseudorange, ephemeris, and satellite launch time from a group of satellites, as well as user clock bias, for navigation and positioning. It provides users with all-weather, three-dimensional coordinates, velocity, and time information from any location on the Earth's surface or in near-Earth space.

[0041] These mainly include the US GPS system, Russia's GLONASS navigation system, the EU's GALILEO system, and my country's BeiDou satellite navigation system.

[0042] An inertial navigation system (INS), also known as an inertial reference system, is an autonomous navigation system that does not rely on external information or radiate energy externally (unlike radio navigation). Its operating environment includes not only the air and ground but also underwater. The basic working principle of inertial navigation is based on Newton's laws of motion. By measuring the acceleration of the vehicle in the inertial reference frame, integrating it over time, and transforming it into the navigation coordinate system, information such as velocity, yaw angle, and position in the navigation coordinate system can be obtained.

[0043] The navigation and positioning functions provided by the GNSS (Global Navigation Satellite System) integrated inertial navigation system are well applied to autonomous tracking technology in automatic driving, providing accurate tracking services unaffected by ambient light, rain, snow, or other weather conditions. However, when the tracking signal is blocked by buildings, the GNSS integrated inertial navigation system cannot function.

[0044] Visual tracking involves installing image recognition sensors on intelligent vehicles to extract, classify, and track lane line features. Feedback control then adjusts the vehicle's heading in real time to follow the desired path, achieving trajectory following. It can perform digital track recognition and tracking even in offline standalone mode, enabling autonomous tracking. However, visual tracking is susceptible to factors such as ambient light and weather conditions like rain and snow.

[0045] Based on the fact that both tracking methods have their own advantages and disadvantages, this invention provides a combined tracking method to integrate visual tracking and GNSS-inertial navigation tracking modes. By using an adaptive tracking mode switching method, it compensates for the shortcomings of visual-based automatic tracking systems, such as poor curve passing performance and susceptibility to external environmental influences. It complements the advantages of visual recognition and GNSS-inertial navigation technologies, thereby reducing the tracking takeover frequency of intelligent driving vehicles, improving autonomous tracking performance, and enhancing the availability of automatic tracking systems.

[0046] Figure 1 This is a schematic flowchart illustrating a combined tracking method according to an embodiment of one aspect of the present invention.

[0047] Please refer to Figure 1 The combined tracking method 100 provided by the present invention includes:

[0048] Step 101: Determine the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode;

[0049] Step 102: In response to the availability of GNSS integrated inertial navigation tracking mode and the unavailability of visual tracking mode, use GNSS integrated inertial navigation tracking mode; and

[0050] Step 103: In response to the GNSS integrated inertial navigation tracking mode being unavailable and the visual tracking mode being available, use the visual tracking mode.

[0051] First, it is necessary to determine the availability of the two tracking modes, and then select the available tracking mode for automatic tracking based on the availability.

[0052] In one embodiment, the available states of the GNSS integrated inertial navigation tracking mode include the working state of the GNSS and the life signal state of the GNSS integrated inertial navigation device, and the available states of the visual tracking mode include the visual recognition function state and the life signal state of the tracking camera device.

[0053] Understandably, the life support signal of the GNSS integrated inertial navigation system is set to valid and usable when it is normal, and invalid and unusable when the life support signal is abnormal. The visual recognition function status refers to, for example, being set to valid when the camera module outputs correct trajectory curve parameters, and invalid otherwise. Similarly, the life support signal of the tracking camera is set to valid when it is normal, and invalid when the life support signal is abnormal.

[0054] In one embodiment, the GNSS operating states include point positioning, point orientation, RTK floating-point delocalization, RTK floating-point delocalization, RTK fixed delocalization, and RTK fixed delocalization. Determining the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode includes: when the GNSS operating state is point positioning, point orientation, RTK floating-point delocalization, or RTK floating-point delocalization, the GNSS integrated inertial navigation tracking mode is unavailable; and when the GNSS operating state is RTK fixed delocalization or RTK fixed delocalization, the GNSS integrated inertial navigation tracking mode is available. The above-mentioned GNSS operating states are only used for judgment, the purpose of which is to determine the availability of the GNSS integrated inertial navigation tracking mode based on different GNSS operating states. The specific meaning represented by each operating state should be known to those skilled in the art, and will not be elaborated further herein.

[0055] In one embodiment, determining the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode includes: acquiring road environment information, which includes mandatory road environment information; and determining the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on the mandatory road environment information.

[0056] Since certain environmental factors directly affect the availability of tracking modes, we refer to these factors as mandatory environmental information. By obtaining and judging this mandatory environmental information, we can determine whether the corresponding tracking mode is available.

[0057] In one embodiment, the mandatory road environment information includes lane line occlusion information and GNSS signal occlusion information; determining the availability status of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on the mandatory road environment information includes: the visual tracking mode being unavailable in response to lane line occlusion; and the GNSS integrated inertial navigation tracking mode being unavailable in response to GNSS signal occlusion.

[0058] Understandably, the obstruction of lane lines prevents the camera from accurately capturing their image information, thus rendering visual tracking ineffective. Similarly, when GNSS signals are obstructed, such as in tunnels or under viaducts, the GNSS tracking mode also becomes unusable.

[0059] The above criteria for determining whether the GNSS combined inertial navigation tracking mode and visual tracking mode are available can be regarded as high-level criteria for selecting the appropriate tracking mode. The conditions of these criteria directly determine the availability of the two tracking modes respectively, so that the available automatic tracking mode can be selected according to the judgment results to reduce the tracking takeover rate.

[0060] Please return Figure 1 The combined tracking method 100 provided by the present invention further includes:

[0061] Step 105: In response to the availability of both GNSS integrated inertial navigation tracking mode and visual tracking mode, select one of the two modes based on the vehicle status.

[0062] When the high-level criteria indicate that both the GNSS combined inertial navigation tracking mode and the visual tracking mode are available, it is necessary to further determine which tracking mode is more suitable for the current vehicle or road conditions.

[0063] In one embodiment, the vehicle state includes vehicle speed; selecting one of a GNSS integrated inertial navigation tracking mode and a visual tracking mode based on the vehicle state includes: using the GNSS integrated inertial navigation tracking mode when the vehicle speed exceeds a preset speed threshold; and using the visual tracking mode when the vehicle speed does not exceed the preset speed threshold.

[0064] Because vehicle speed significantly affects the effectiveness of tracking control during the tracking process, visual tracking is generally considered to be more sensitive to vehicle speed. When the speed is too high, the control effect of visual tracking will decrease significantly, which is especially noticeable on curved roads. Therefore, it is necessary to select a more appropriate tracking mode based on the current vehicle speed.

[0065] In one embodiment, the preset speed threshold is a threshold range, which provides a buffer for switching between the two tracking modes, thereby avoiding frequent switching between the two tracking modes.

[0066] In one embodiment, an incremental angle limiting is introduced during the switching between GNSS integrated inertial navigation tracking mode and visual tracking mode to ensure a smooth transition between tracking modes. The introduction of incremental angle limiting makes the tracking switching process smoother, without causing abrupt changes in vehicle direction due to changes in tracking methods, thus improving the user's driving experience.

[0067] In addition to selecting a more suitable tracking mode based on vehicle speed, in one embodiment, selecting either the GNSS combined inertial navigation tracking mode or the visual tracking mode based on vehicle status also includes: obtaining the curvature of the road segment.

[0068] When satisfied At that time, the GNSS integrated inertial navigation tracking mode is used; and

[0069] When satisfied When using the visual tracking mode, k1 is the curvature scaling factor.

[0070] When the road section has a large curvature and the vehicle speed is high, it is not suitable to use the visual tracking mode. Therefore, the road curvature is also one of the factors to consider when selecting a suitable tracking mode.

[0071] In addition, the speed threshold can be set according to the curvature of the line. When the curvature is large, the speed threshold can be lowered accordingly, and when the curvature is small, the speed threshold can be increased accordingly to adapt to the characteristics of straight lines and curves.

[0072] In one embodiment, selecting either the GNSS integrated inertial navigation tracking mode or the visual tracking mode based on the vehicle state further includes: obtaining the visual tracking segment failure rate and the GNSS integrated inertial navigation tracking segment failure rate.

[0073] When satisfied At that time, the GNSS integrated inertial navigation tracking mode is used; and

[0074] When satisfied When using visual tracking mode, k2 is the failure rate scaling factor.

[0075] On roads, there are often sections where lane markings are worn down. In these sections, visual tracking is prone to loss of recognition, leading to tracking takeover. On the other hand, in cities, the influence of uniform buildings creates canyon-like airflow in narrow areas with dense building spacing, a phenomenon known as the "urban canyon effect." In densely built-up areas, the urban canyon effect can cause GNSS signal instability, leading to positioning signal drift or ineffectiveness, which in turn causes tracking takeover.

[0076] The failure rate of visual tracking and GNSS-inertial navigation tracking on a road segment are quantitative indicators of the number of failures that occur when a vehicle uses each of the two tracking modes on a given road segment. A lower failure rate indicates a more ideal performance of the tracking mode on that segment. This indicator can quantify the degree of lane wear and urban canyon effect on the route. Therefore, by further combining the road segment failure rate indicator with the actual failure rate, the control effects of the two tracking control methods on that road segment can be considered more comprehensively, leading to more precise optimization of the tracking mode.

[0077] The above factors can be regarded as low-level criteria for selecting the applicable tracking mode. Under the premise that the high-level criteria ensure the reliability of tracking, the tracking mode is adaptively optimized based on the vehicle status and road environment of the low-level criteria. By integrating visual tracking and GNSS combined inertial navigation tracking, the advantages are complemented, thereby improving the availability of the automatic tracking system.

[0078] Figure 2 This is a schematic diagram illustrating the judgment logic and tracking mode switching in a combined tracking method according to an embodiment of the present invention.

[0079] like Figure 2 As shown, the combined tracking method provided by this invention first determines the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on high-level criteria. These high-level criteria include failure assessment and mandatory road environment conditions. Failure assessment includes the operating status of the GNSS and the life signal status of the GNSS integrated inertial navigation device, as well as the status of the visual recognition function and the life signal status of the tracking camera device, etc. These failure assessment factors directly determine whether the two tracking modes are available. Mandatory road environment conditions include whether lane lines are obstructed and whether GNSS signals are obstructed, which also directly determine the availability of the two tracking methods.

[0080] If both tracking modes are usable after all higher-level criteria are met, further optimization of the tracking method is needed based on lower-level criteria. For example... Figure 2 As shown, low-level criteria include vehicle status and non-mandatory road environment. Vehicle status includes, for example, vehicle speed as described above, while non-mandatory road environment includes the curvature of the road segment and the failure rate of the two tracking modes on that segment. These factors help the vehicle's infotainment system select a more suitable and preferred tracking method for autonomous tracking.

[0081] When making judgments, the dividing line between the two modes, such as the speed threshold, is set to a threshold range, thereby avoiding frequent switching between modes. Furthermore, when switching tracking modes, a limit on the angle increment is introduced, making the transition between modes smooth and stable, thus improving the user experience.

[0082] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0083] Another aspect of the present invention provides a combined tracking device, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the steps of the combined tracking method described in any of the preceding claims.

[0084] Figure 3 This is a schematic diagram of the combined tracking device according to another embodiment of the present invention.

[0085] like Figure 3 As shown, the computer system / server 300 of the combined tracking device is presented in the form of a general-purpose computer device. The components of the computer system / server 300 may include one or more processors 302, memory 301, and bus 303 connecting different system components (including memory 301 and processor 302).

[0086] Bus 303 includes a data bus, an address bus, and a control bus. The number of bits in the data bus is proportional to the data transfer rate (product of the operating frequency). The number of bits in the address bus determines the maximum addressable memory space. The control bus (read / write) indicates the type of bus cycle and the time when the current input / output operation is completed. Processor 302 is connected to memory 301 via bus 303 and configured to implement the vehicle control method provided in any of the above embodiments.

[0087] The processor 302, as the computing and control core of the computer system / server 300 of the vehicle control device, is the final execution unit for information processing and program execution. All software layer operations in the computer system are ultimately mapped to operations of the processor 302 through the instruction set. The processor 302's main functions are processing instructions, executing operations, controlling timing, and processing data.

[0088] Memory 301 refers to various storage devices in a computer that store programs and data. Memory 301 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 304 and / or cache memory 305.

[0089] Random Access Memory (RAM) 304 is an internal memory that directly exchanges data with the processor 302. It can be read and written at any time (except during refresh) and is very fast. It is typically used as a temporary data storage medium for the operating system or other running programs. The data stored in it will be lost if power is lost. Cache Memory 305 is a level-one memory located between main memory and the processor 302. It has a smaller capacity but a much higher speed than main memory, approaching the speed of the processor 302.

[0090] The computer system / server 300 may further include other removable / non-removable, volatile / non-volatile computer system storage media. In this embodiment, the storage system 306 can be used to read and write non-removable, non-volatile magnetic media.

[0091] The memory 301 may also include at least one set of program modules 307. Program modules 307 may be stored in the memory 301. Program modules 307 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data; each or some combination of these examples may include an implementation of a network environment. Program modules 307 typically perform the functions and / or methods described in the embodiments of the present invention.

[0092] The computer system / server 300 can also communicate with one or more external devices 308 (e.g., keyboard, pointing device, display 309, etc.), one or more devices that enable a user to interact with the computer system / server 300, and / or any device that enables the computer system / server 300 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication can be performed through the input / output (I / O) interface 310.

[0093] The computer system / server 300 can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 311. Figure 3 As shown, network adapter 311 communicates with other modules of computer system / server 300 via bus 303. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with computer system / server 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0094] The present invention also provides an embodiment of a computer storage medium.

[0095] The computer storage medium contains a computer program. When executed by a processor, this computer program can implement the steps of any of the above-described combined line-following methods.

[0096] Those skilled in the art will understand that information, signals, and data can be represented using any of a variety of different techniques and arts. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0097] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0098] The processors described herein can be implemented using electronic hardware, computer software, or any combination thereof. Whether such processors are implemented as hardware or software will depend on the specific application and the overall design constraints imposed on the system. As an example, the processors, any portion thereof, or any combination thereof presented in this disclosure can be implemented using microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuitry, and other suitable processing components configured to perform the various functions described throughout this disclosure. The functionality of the processors, any portion thereof, or any combination thereof presented in this disclosure can be implemented using software executed by a microprocessor, microcontroller, DSP, or other suitable platform.

[0099] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0100] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0101] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A combined tracking method for selecting an applicable tracking mode in autonomous driving, comprising: Determine the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode; In response to the availability of the GNSS integrated inertial navigation tracking mode and the unavailability of the visual tracking mode, the GNSS integrated inertial navigation tracking mode is used. In response to the GNSS integrated inertial navigation tracking mode being unavailable, the visual tracking mode is available, and the visual tracking mode is used. In response to the fact that both the GNSS combined inertial navigation tracking mode and the visual tracking mode are unavailable, automatic tracking is exited and an alarm is issued to remind the user to take over. as well as In response to the availability of both the GNSS integrated inertial navigation tracking mode and the visual tracking mode, one of the two modes is selected based on the vehicle status. Wherein, the vehicle state includes vehicle speed, and the step of selecting one of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on the vehicle state includes: Obtain the curvature of the road segment; When the vehicle speed is greater than At that time, the GNSS integrated inertial navigation tracking mode is used; and When the vehicle speed is ≤ When using the visual tracking mode, k1 is the curvature scaling factor.

2. The combined tracking method as described in claim 1, characterized in that, The determination of the availability of the GNSS integrated inertial navigation tracking mode and the availability of the visual tracking mode includes: Obtain road environment information, including mandatory road environment information; and The availability status of the GNSS integrated inertial navigation tracking mode and the availability status of the visual tracking mode are determined based on the mandatory road environment information.

3. The combined tracking method as described in claim 2, characterized in that, The mandatory road environment information includes lane line obstruction information and GNSS signal obstruction information; The determination of the availability status of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on the mandatory road environment information includes: The visual tracking mode is unavailable in response to lane line obstruction; and In response to GNSS signal obstruction, the GNSS combined inertial navigation tracking mode is unavailable.

4. The combined tracking method as described in claim 1, characterized in that, The available states of the GNSS integrated inertial navigation tracking mode include the working state of the GNSS and the life signal state of the GNSS integrated inertial navigation device. The available states of the visual tracking mode include the visual recognition function state and the life signal state of the tracking camera device.

5. The combined tracking method as described in claim 4, characterized in that, The GNSS operating states include single-point positioning, single-point orientation, RTK floating-point delocalization, RTK floating-point delocalization, RTK fixed delocalization, and RTK fixed delocalization. The determination of the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode includes: When the GNSS is in single-point positioning, single-point orientation, RTK floating-point de-positioning, or RTK floating-point de-orientation mode, the GNSS combined inertial navigation tracking mode is unavailable. as well as When the GNSS is in RTK fixed localization or RTK fixed orientation mode, the GNSS combined inertial navigation tracking mode is available.

6. The combined tracking method as described in claim 1, characterized in that, The step of selecting one of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on vehicle state also includes: When the vehicle speed exceeds a preset speed threshold, the GNSS combined inertial navigation tracking mode is used; and When the vehicle speed does not exceed a preset speed threshold, the visual tracking mode is used.

7. The combined tracking method as described in claim 1, characterized in that, The step of selecting one of the GNSS integrated inertial navigation tracking mode and the visual tracking mode based on vehicle state also includes: Obtain the failure rate of visual tracking segments and the failure rate of GNSS-inertial navigation integrated tracking segments; When the vehicle speed is greater than At that time, the GNSS integrated inertial navigation tracking mode is used; and When the vehicle speed is ≤ When using visual tracking mode, k2 is the failure rate scaling factor.

8. The combined tracking method as described in claim 6, characterized in that, The preset speed threshold is a range to avoid frequent switching of the tracking mode.

9. The combined tracking method as described in claim 1, characterized in that, An angle increment limiter is introduced during the switching process between the GNSS combined inertial navigation tracking mode and the visual tracking mode to ensure a smooth transition of the tracking mode.

10. A combined tracking device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to: Determine the availability of the GNSS integrated inertial navigation tracking mode and the visual tracking mode; In response to the availability of the GNSS integrated inertial navigation tracking mode and the unavailability of the visual tracking mode, the GNSS integrated inertial navigation tracking mode is used. In response to the GNSS integrated inertial navigation tracking mode being unavailable, the visual tracking mode is available, and the visual tracking mode is used. In response to the fact that both the GNSS combined inertial navigation tracking mode and the visual tracking mode are unavailable, automatic tracking is exited and an alarm is issued to remind the user to take over. as well as In response to the availability of both the GNSS integrated inertial navigation tracking mode and the visual tracking mode, one of the two modes is selected based on the vehicle status. The vehicle status includes vehicle speed, and the processor is further configured to: Obtain the curvature of the road segment; When the vehicle speed is greater than At that time, the GNSS integrated inertial navigation tracking mode is used; and When the vehicle speed is ≤ When using the visual tracking mode, k1 is the curvature scaling factor.

11. The combined tracking device as claimed in claim 10, characterized in that, The processor is further configured to: Obtain road environment information, including mandatory road environment information; and The availability status of the GNSS integrated inertial navigation tracking mode and the availability status of the visual tracking mode are determined based on the mandatory road environment information.

12. The combined tracking device as claimed in claim 11, characterized in that, The mandatory road environment information includes lane line obstruction information and GNSS signal obstruction information, and the processor is further configured to: The visual tracking mode is unavailable in response to lane line obstruction; and In response to GNSS signal obstruction, the GNSS combined inertial navigation tracking mode is unavailable.

13. The combined tracking device as described in claim 10, characterized in that, The available states of the GNSS integrated inertial navigation tracking mode include the working state of the GNSS and the life signal state of the GNSS integrated inertial navigation device. The available states of the visual tracking mode include the visual recognition function state and the life signal state of the tracking camera device.

14. The combined tracking device as described in claim 13, characterized in that, The GNSS operating states include single-point positioning, single-point orientation, RTK floating-point de-positioning, RTK floating-point de-orientation, RTK fixed de-positioning, and RTK fixed de-orientation. The processor is further configured to: When the GNSS is in single-point positioning, single-point orientation, RTK floating-point de-positioning, or RTK floating-point de-orientation mode, the GNSS combined inertial navigation tracking mode is unavailable. as well as When the GNSS is in RTK fixed localization or RTK fixed orientation mode, the GNSS combined inertial navigation tracking mode is available.

15. The combined tracking device as described in claim 10, characterized in that, The processor is further configured to: When the vehicle speed exceeds a preset speed threshold, the GNSS combined inertial navigation tracking mode is used; and When the vehicle speed does not exceed a preset speed threshold, the visual tracking mode is used.

16. The combined tracking device as claimed in claim 10, characterized in that, The processor is further configured to: Obtain the failure rate of visual tracking segments and the failure rate of GNSS-inertial navigation integrated tracking segments; When the vehicle speed is greater than At that time, the GNSS integrated inertial navigation tracking mode is used; as well as When the vehicle speed is ≤ When using visual tracking mode, k2 is the failure rate scaling factor.

17. The combined tracking device as claimed in claim 15, characterized in that, The preset speed threshold is a range to avoid frequent switching of the tracking mode.

18. The combined tracking device as claimed in claim 10, characterized in that, The processor is further configured to: An angle increment limiter is introduced during the switching process between the GNSS combined inertial navigation tracking mode and the visual tracking mode to ensure a smooth transition of the tracking mode.

19. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of any one of the combined tracking methods as described in claims 1 to 9.

Images