Vehicle course angle calibration method and device and electronic equipment
Through the data fusion of the vehicle camera and the roadside unit RSU, Kalman filtering processing is used to realize the online heading angle calibration of the inertial navigation system, solving the INS error and drift problems, and improving the continuity and stability of the navigation system.
Patent Information
- Application Number
- CN202410890477.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-03
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing inertial navigation system (INS) under long-term use or environmental factors leads to heading angle measurement errors and drifts, affecting navigation accuracy and stability. Traditional calibration methods require vehicle parking, resulting in interruption of navigation system and discontinuity of data.
The vehicle camera collects the surrounding environment images, detects the target environmental elements, and acquires positioning information in combination with the roadside unit RSU, uses Kalman filtering to estimate the heading angle error, and compensates the heading angle measured by the INS to achieve online calibration.
Without interrupting vehicle navigation, the continuity and stability of the navigation system are improved, the measurement accuracy and adaptability of heading angles are enhanced, and the dependence on the environment is reduced.
Smart Images

Figure CN118936460A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to vehicle technology, and in particular to a vehicle heading angle calibration method, device, electronic device, computer-readable storage medium and computer program product. Background Art
[0002] In modern automotive technology, heading angle stability is crucial to vehicle navigation, control and safety. Currently, the Inertial Navigation System (INS) is widely used to measure the heading angle. By using sensors such as accelerometers and gyroscopes, INS can obtain the vehicle's motion status in real time, including position, speed, attitude and other information, providing important data support for vehicle navigation and control.
[0003] Sensors such as accelerometers and gyroscopes in INS have certain errors and drift problems. Long-term use or environmental factors will cause these errors and drifts to accumulate, affecting the measurement accuracy and stability of the heading angle. However, the traditional INS heading angle calibration solution requires the vehicle to stop and calibrate regularly, which will cause interruptions in the vehicle navigation system and discontinuous data. Summary of the invention
[0004] The present application provides a vehicle heading angle calibration method, device, electronic device, computer-readable storage medium and computer program product, which can perform online calibration of the heading angle without interrupting vehicle navigation, thereby improving the continuity and stability of the navigation system.
[0005] The technical solution of this application is implemented as follows:
[0006] The present application provides a vehicle heading angle calibration method, comprising:
[0007] Performing image acquisition and processing of the vehicle's surrounding environment through the vehicle's camera, and performing target detection processing on the acquired image to obtain image positioning information of target environmental elements in the vehicle's surrounding environment;
[0008] Acquire RSU positioning information of the target environmental element from a roadside unit RSU;
[0009] Determine the positioning information error of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental element and the RSU positioning information
[0010] Performing Kalman filtering on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error;
[0011] The heading angle measured by the inertial navigation system INS of the vehicle is compensated according to the estimated heading angle error.
[0012] The present application provides a vehicle heading angle calibration device, comprising:
[0013] A first perception module is used to collect and process images of the vehicle's surrounding environment through the vehicle's camera, and to perform target detection processing on the collected images to obtain image positioning information of target environmental elements in the vehicle's surrounding environment;
[0014] The second perception module is used to obtain the RSU positioning information of the target environmental element from the roadside unit RSU;
[0015] An error calculation module, used to determine the positioning information error of the vehicle in the navigation coordinate system according to the image positioning information of the target environmental element and the RSU positioning information;
[0016] A Kalman filter module is used to perform Kalman filter processing on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error;
[0017] The compensation module is used to compensate the heading angle measured by the inertial navigation system INS of the vehicle according to the estimated heading angle error.
[0018] The present application provides an electronic device, including:
[0019] A memory for storing executable instructions;
[0020] The processor is used to implement the vehicle heading angle calibration method provided in the present application when executing the executable instructions stored in the memory.
[0021] The present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the vehicle heading angle calibration method provided in the present application.
[0022] The present application provides a computer program product, which includes executable instructions for causing a processor to execute and implement the vehicle heading angle calibration method provided by the present application.
[0023] This application has the following beneficial effects:
[0024] This application uses the vehicle's camera to collect and process images of the vehicle's surroundings, and performs target detection processing on the collected images to obtain image positioning information of target environmental elements in the vehicle's surroundings; obtains RSU positioning information of target environmental elements from the roadside unit RSU; determines the positioning information error of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental elements and the RSU positioning information; performs Kalman filtering on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error; and compensates for the heading angle measured by the vehicle's inertial navigation system INS based on the estimated heading angle error. This application combines the data collected by the camera and INS respectively, and by fusing the data from these two different sources, it achieves accurate calibration of the vehicle's heading angle; at the same time, it is able to perform online calibration of the heading angle without interrupting vehicle navigation, thereby improving the continuity and stability of the navigation system. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0026] Figure 1 is a schematic diagram of the architecture of a vehicle heading angle calibration system provided in an embodiment of the present application;
[0027] Figure 2 is a structural schematic diagram of a vehicle-mounted device provided in an embodiment of the present application;
[0028] Figure 3A is a schematic diagram of a first process flow of a vehicle heading angle calibration method provided in an embodiment of the present application;
[0029] Figure 3B is a second flow chart of the vehicle heading angle calibration method provided in an embodiment of the present application;
[0030] Figure 4 is a third flow chart of the vehicle heading angle calibration method provided in an embodiment of the present application;
[0031] Figure 5 It is a schematic diagram of the coordinate system provided in the embodiment of the present application. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.
[0033] In the following description, reference is made to "some embodiments", which describes a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. In the following description, reference is made to the term "multiple" to mean at least two.
[0034] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0036] The embodiments of the present application provide a vehicle heading angle calibration method, device, electronic device, computer-readable storage medium, and computer program product, which can perform online calibration of the heading angle without interrupting vehicle navigation, thereby improving the continuity and stability of the navigation system. The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as a vehicle-mounted device or as a server, wherein the vehicle-mounted device is a general term for electronic devices inside the vehicle.
[0037] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the vehicle heading angle calibration system 100 provided in an embodiment of the present application, wherein the vehicle-mounted device 400 is connected to the server 200 via the network 300, wherein the network 300 may be a wide area network or a local area network, or a combination of the two.
[0038] In some embodiments, taking the electronic device as an on-board device as an example, the vehicle heading angle calibration method provided in the embodiments of the present application can be implemented by the on-board device. For example, the on-board device 400 performs image acquisition and processing of the vehicle's surrounding environment through the vehicle's camera, and performs target detection processing on the acquired image to obtain image positioning information of the target environmental elements in the vehicle's surrounding environment; obtains the RSU positioning information of the target environmental elements from the roadside unit RSU; determines the positioning information error of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental elements and the RSU positioning information; performs Kalman filtering on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error; and compensates for the heading angle measured by the vehicle's inertial navigation system INS based on the estimated heading angle error. In the above manner, the vehicle heading angle calibration is independently implemented by the on-board device 400, which can be applicable to various driving scenarios, such as scenarios without network or with poor network conditions.
[0039] In some embodiments, taking the electronic device as a server as an example, the vehicle heading angle calibration method provided in the embodiments of the present application can be implemented by the server. For example, the vehicle-mounted device 400 collects and processes images of the vehicle's surroundings through the vehicle's camera, and sends the collected images to the server 200, so that the server 200 performs target detection processing on the received images to obtain image positioning information of the target environmental elements in the vehicle's surroundings (or, the vehicle-mounted device 400 can also send the image positioning information of the target environmental elements to the server 200); the vehicle-mounted device 400 obtains the RSU positioning information of the target environmental elements from the roadside unit RSU, and sends the RSU positioning information of the target environmental elements to the server 200; the server 200 determines the positioning information of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental elements and the RSU positioning information. Information error; the server 200 performs Kalman filtering on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error; the server 200 compensates the heading angle measured by the inertial navigation system INS of the vehicle according to the estimated heading angle error, wherein the server 200 can obtain the heading angle measured by INS from the vehicle-mounted device 400, and send the compensated heading angle to the vehicle-mounted device 400 so that the vehicle-mounted device 400 can further apply the compensated heading angle, or the server 200 can also send the estimated heading angle error to the vehicle-mounted device 400 so that the vehicle-mounted device 400 can compensate the heading angle measured by the inertial navigation system INS of the vehicle according to the received estimated heading angle error. In the above method, the server 200 undertakes most of the computing tasks, which can improve computing efficiency and reduce the computing pressure of the vehicle-mounted device 400.
[0040] In some embodiments, the vehicle-mounted device 400 or the server 200 can implement the vehicle heading angle calibration method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP, and the small program can be controlled to run or closed by the user. In short, the above-mentioned computer program can be an application, module or plug-in in any form.
[0041] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The vehicle-mounted device 400 and the server 200 may be connected via wireless communication.
[0042] Taking the electronic device provided in the embodiment of the present application as an example, which is a vehicle-mounted device, it can be understood that in the case where the electronic device is a server, Figure 2 Parts of the structure shown in FIG. 1 (such as a user interface, a presentation module, and an input processing module) may be omitted. Figure 2 , Figure 2 is a structural diagram of a vehicle-mounted device 400 provided in an embodiment of the present application, Figure 2 The vehicle-mounted device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the vehicle-mounted device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .
[0043] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.
[0044] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as microphones, touch screen displays, cameras, other input buttons and controls.
[0045] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.
[0046] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.
[0047] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.
[0048] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;
[0049] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;
[0050] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);
[0051] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.
[0052] In some embodiments, the vehicle heading angle calibration device provided in the embodiments of the present application can be implemented in a software manner. Figure 2 The vehicle heading angle calibration device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, including the following software modules: a first perception module 4551, a second perception module 4552, an error calculation module 4553, a Kalman filter module 4554 and a compensation module 4555. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.
[0053] The vehicle heading angle calibration method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the electronic device provided in the embodiment of the present application.
[0054] See also Figure 3A , Figure 3A is a flow chart of a vehicle heading angle calibration method provided in an embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.
[0055] In step 101, the vehicle's surrounding environment is imaged and processed by the vehicle's camera, and the captured image is subjected to target detection processing to obtain image positioning information of target environmental elements in the vehicle's surrounding environment.
[0056] For example, when a vehicle passes by a target environmental element (i.e., the target environmental element is within the field of view of the vehicle's camera), the vehicle's camera performs image acquisition processing on the vehicle's surroundings, and performs target detection processing on the acquired image to obtain image positioning information of the target environmental element in the vehicle's surroundings. The image positioning information of the target environmental element is described in the image coordinate system.
[0057] It is worth mentioning that environmental elements, or traffic elements, are the basic components of the road traffic system, such as traffic lights and road signs (such as speed limit signs, prohibition signs, etc.). The target environmental elements refer to the environmental elements for which the corresponding positioning information (referred to as RSU positioning information for easy distinction) is stored in the roadside unit RSU. The type of the target environmental element can be set according to the actual application scenario. For example, the target environmental element can be set to the traffic lights that are widely distributed in the road traffic system. This ensures that the heading angle calibration can be performed most of the time during the vehicle's driving process, fully meeting the heading angle calibration requirements. Among them, the RSU is an important component of the vehicle-road cooperative system. Its main function is to collect and process real-time data from vehicles and provide traffic information to vehicles.
[0058] It is worth noting that the positioning information involved in the embodiments of the present application includes two types of information: coordinates and heading angles.
[0059] In some embodiments, the number of cameras used for image acquisition processing may be one or more.
[0060] In some embodiments, the vehicle has multiple cameras; the above-mentioned target detection processing of the captured images can be achieved in the following way to obtain image positioning information of target environmental elements in the vehicle's surrounding environment: the images captured by multiple cameras of the vehicle are fused in the BEV image coordinate system to obtain a BEV image; the BEV image is subjected to target detection processing to obtain image positioning information of target environmental elements in the vehicle's surrounding environment.
[0061] Here, the vehicle's surrounding environment can be imaged and processed by multiple cameras of the vehicle, wherein different cameras have different fields of view. Then, the images collected by the multiple cameras of the vehicle are fused in a bird's eye view (BEV) image coordinate system to obtain a BEV image, and the BEV image is subjected to target detection processing to obtain image positioning information of target environmental elements in the vehicle's surrounding environment. The image positioning information of the target environmental elements obtained here is described in the BEV image coordinate system, or the BEV image coordinate system. The BEV space. Through the above method, the problem of the limitation of the field of view of a single camera can be effectively overcome, the accuracy and reliability of environmental perception can be improved, and the image positioning information of the target environmental elements obtained can be more accurate and reliable.
[0062] In some embodiments, before the vehicle's surrounding environment is imaged and processed by the vehicle's camera, the vehicle heading angle calibration method further includes: calibrating the camera and INS for external parameters; and calibrating the camera and INS for time.
[0063] In order to ensure that the data collected by the camera and INS can be effectively integrated, the camera and INS can be calibrated externally (or spatially) before they work. The camera and INS are physically independent and may be installed in different parts of the vehicle. Therefore, the data they observe are relative to their respective local coordinate systems. The role of extrinsic calibration is to unify the data collected by the camera and INS into a global coordinate system so that they can work together to provide consistent environmental perception results. At the same time, the camera and INS are also time-calibrated to ensure that the data collected by the camera and INS correspond to the same time, avoiding poor heading angle calibration due to time asynchrony.
[0064] In step 102, the RSU positioning information of the target environmental element is obtained from the roadside unit RSU.
[0065] Here, the RSU positioning information of the target environmental element is obtained from the roadside unit RSU by wireless communication.
[0066] It is worth noting that the target environment element in step 102 is the same as the target environment element in step 101 .
[0067] In step 103, the positioning information error of the vehicle in the navigation coordinate system is determined based on the image positioning information of the target environmental element and the RSU positioning information.
[0068] Here, since the heading angle of the vehicle is described in the navigation coordinate system, the image positioning information of the target environmental elements and the RSU positioning information can be transformed into the navigation coordinate system, and then the positioning information described from the perspective of the target environmental elements can be transformed to that described from the perspective of the vehicle. Finally, the error between the two positioning information of the vehicle in the navigation coordinate system is calculated, that is, the positioning information error of the vehicle in the navigation coordinate system.
[0069] In step 104, Kalman filtering is performed on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error.
[0070] Here, the calculated positioning information error of the vehicle in the navigation coordinate system may contain noise and deviation. Therefore, in the embodiment of the present application, the positioning information error of the vehicle in the navigation coordinate system is further optimized by Kalman filtering to obtain an estimated heading angle error. Among them, the key idea of Kalman filtering is data fusion and iterative update. By combining prior information and actual measurements, the system can better estimate the current state and gradually improve the accuracy of the estimate through iterative updates, that is, use the Kalman gain to correct the state prediction value (estimated heading angle error) to make it close to the true value.
[0071] In some embodiments, the above-mentioned Kalman filtering processing of the positioning information error of the vehicle in the navigation coordinate system to obtain the estimated heading angle error can be achieved in the following manner: a recursive difference equation of the Kalman filter is established; wherein the recursive difference equation takes the measurement vector as input; the positioning information error of the vehicle in the navigation coordinate system is used as the measurement vector and input into the recursive difference equation to obtain the estimated heading angle error output by the recursive difference equation.
[0072] Here, firstly, a recursive difference equation of a Kalman filter is established based on the Kalman filter principle, wherein the recursive difference equation takes a measurement vector as input. For example, the Kalman filter may be a linear discrete Kalman filter, which only needs an estimated value at a previous moment and an observed value (i.e., a measurement vector) at a current moment to perform state estimation, and the recursive difference equation of the linear discrete Kalman filter includes a prediction equation group and a correction equation group, wherein the prediction equation group includes an equation for state prediction and an equation for predicting error covariance, and the correction equation group includes an equation for calculating a Kalman gain, an equation for state update, and an equation for updating the estimated error covariance.
[0073] After the recursive difference equation is established, the positioning information error of the vehicle in the navigation coordinate system is used as the measurement vector and input into the recursive difference equation to obtain the estimated heading angle error output by the recursive difference equation. In this way, the optimal estimation is achieved through data iteration, which improves the accuracy of the estimated heading angle error.
[0074] In step 105, the heading angle measured by the inertial navigation system INS of the vehicle is compensated according to the estimated heading angle error.
[0075] The estimated heading angle error reflects the measurement error of the heading angle. Therefore, the heading angle measured by the vehicle's inertial navigation system INS is compensated according to the estimated heading angle error to correct the INS heading angle deviation problem. For example, the heading angle measured by INS can be subtracted from the estimated heading angle error to obtain the target heading angle, and the obtained target heading angle can be further applied, such as for vehicle navigation, vehicle control, automatic driving, etc.
[0076] It is worth noting that the above steps 101 to 105 can be implemented during the driving of the vehicle without stopping the vehicle to achieve heading angle calibration, thereby avoiding interruption of the vehicle navigation system and data discontinuity.
[0077] like Figure 3AAs shown, the embodiment of the present application performs image acquisition and processing of the vehicle's surroundings through the vehicle's camera, and performs target detection processing on the acquired images to obtain image positioning information of target environmental elements in the vehicle's surroundings; obtains RSU positioning information of target environmental elements from the roadside unit RSU; determines the positioning information error of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental elements and the RSU positioning information; performs Kalman filtering on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error; and compensates for the heading angle measured by the vehicle's inertial navigation system INS based on the estimated heading angle error. The present application combines the data collected by the camera and INS respectively, and by fusing the data from these two different sources, it realizes the precise calibration of the vehicle's heading angle, thereby improving the stability and accuracy of the vehicle's heading angle; at the same time, it is able to perform online calibration of the heading angle without interrupting the vehicle's navigation, thereby improving the continuity and stability of the navigation system.
[0078] In some embodiments, see Figure 3B , Figure 3B is a flow chart of a vehicle heading angle calibration method provided in an embodiment of the present application, Figure 3A The illustrated step 103 can be implemented through steps 201 to 203 , which will be described in conjunction with each step.
[0079] In step 201, first positioning information of a vehicle in a navigation coordinate system is determined according to image positioning information of a target environmental element.
[0080] Here, a series of transformations are performed on the image positioning information of the target environmental elements to obtain the first positioning information of the vehicle in the navigation coordinate system.
[0081] In some embodiments, the above-mentioned determination of the first positioning information of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental element can be achieved in the following manner: performing coordinate system transformation processing on the image positioning information of the target environmental element to obtain the first positioning information of the target environmental element in the navigation coordinate system; performing posture transformation processing on the first positioning information of the target environmental element in the navigation coordinate system according to the relative posture to obtain the first positioning information of the vehicle in the navigation coordinate system; wherein the relative posture represents the posture between the vehicle and the target environmental element.
[0082] In a series of transformation processes, the image positioning information of the target environmental element is first subjected to coordinate system transformation processing, that is, it is transformed from the image coordinate system to the navigation coordinate system, and the first positioning information of the target environmental element in the navigation coordinate system is obtained; then, the first positioning information of the target environmental element in the navigation coordinate system is subjected to posture transformation processing according to the relative posture, that is, the positioning information described from the perspective of the target environmental element is transformed to that described from the perspective of the vehicle, and the first positioning information of the vehicle in the navigation coordinate system is obtained, wherein the relative posture represents the posture between the vehicle and the target environmental element.
[0083] In some embodiments, the above-mentioned coordinate system transformation processing of the image positioning information of the target environmental element can be achieved in the following manner to obtain the first positioning information of the target environmental element in the navigation coordinate system: transform the image positioning information of the target environmental element to the vehicle coordinate system to obtain the first positioning information of the target environmental element in the vehicle coordinate system; transform the first positioning information of the target environmental element in the vehicle coordinate system to the navigation coordinate system to obtain the first positioning information of the target environmental element in the navigation coordinate system.
[0084] In the process of transforming the image positioning information of the target environmental element from the image coordinate system to the navigation coordinate system, first, the image positioning information of the target environmental element is transformed to the vehicle coordinate system (from the image coordinate system to the vehicle coordinate system) to obtain the first positioning information of the target environmental element in the vehicle coordinate system, wherein the external parameters of the camera can be used to realize the transformation from the image coordinate system to the vehicle coordinate system; then, the first positioning information of the target environmental element in the vehicle coordinate system is transformed to the navigation coordinate system (from the vehicle coordinate system to the navigation coordinate system) to obtain the first positioning information of the target environmental element in the navigation coordinate system.
[0085] In some embodiments, before transforming the first positioning information of the target environmental element in the vehicle coordinate system to the navigation coordinate system, the vehicle heading angle calibration method also includes: measuring the roll angle, pitch angle and heading angle of the vehicle through INS; determining a transformation matrix based on the roll angle, pitch angle and heading angle measured by INS; wherein the transformation matrix is used to transform the first positioning information of the target environmental element in the vehicle coordinate system to the navigation coordinate system.
[0086] Here, the transformation from the vehicle coordinate system to the navigation coordinate system can be achieved using a transformation matrix. For example, the three-axis attitude angle of the vehicle is measured by INS. The three-axis attitude angle is used to describe the rotation state of the vehicle relative to the navigation coordinate system in three-dimensional space. The three-axis attitude angle includes roll angle, pitch angle and heading angle, which correspond to rotations around the X-axis, Y-axis and Z-axis respectively. Then, the transformation matrix is determined based on the roll angle, pitch angle and heading angle measured by INS. The transformation matrix is an effective tool for describing the above-mentioned rotation state. Therefore, the transformation from the vehicle coordinate system to the navigation coordinate system can be achieved based on the transformation matrix.
[0087] In step 202, second positioning information of the vehicle in the navigation coordinate system is determined according to the RSU positioning information of the target environmental element.
[0088] Similar to step 201, a series of transformations can be performed on the RSU positioning information of the target environmental element to obtain the second positioning information of the vehicle in the navigation coordinate system.
[0089] In step 203, a positioning information error between the first positioning information and the second positioning information of the vehicle in the navigation coordinate system is determined.
[0090] Here, an error between the first positioning information and the second positioning information of the vehicle in the navigation coordinate system is calculated as the positioning information error of the vehicle in the navigation coordinate system.
[0091] like Figure 3B As shown, the embodiment of the present application determines the first positioning information of the vehicle in the navigation coordinate system according to the image positioning information of the target environmental element; determines the second positioning information of the vehicle in the navigation coordinate system according to the RSU positioning information of the target environmental element; and determines the positioning information error between the first positioning information and the second positioning information of the vehicle in the navigation coordinate system. The above method measures the positioning information error in the same navigation coordinate system, which can improve the accuracy and interpretability of the positioning information error and provide a reliable basis for subsequent compensation.
[0092] Below, an exemplary application of the embodiment of the present application in a practical application scenario will be described. For ease of understanding, taking a vehicle equipped with INS and 6 cameras as an example, the embodiment of the present application provides the following Figure 4 A flow chart of the vehicle heading angle calibration method shown in FIG. Figure 4 Provide explanation.
[0093] First, calibrate multiple sensors in the vehicle. For example, perform internal calibration on the six cameras in the vehicle to obtain the projection matrix R of the six cameras. ic ; Calibrate the extrinsic parameters of the six cameras and INS to obtain R mc; At the same time, the discrete time calibration method is used to calibrate the time of the six cameras and INS to ensure that the data collected by different sensors correspond to the same time. Through the above calibration process, the data collected by different sensors can be synchronized in space and time, improving the accuracy of subsequent processing.
[0094] When a vehicle passes a traffic light, the vehicle controller (such as a domain controller) receives six images collected by six cameras at the same time, processes these six images through the BEV visual perception model, and obtains the image positioning information of the traffic light in the BEV space. In the processing of the BEV visual perception model, the six images are first fused in the BEV space to obtain the BEV image, and then the BEV image is used to locate the traffic light. The image positioning information of the traffic light in the BEV space is transformed by the camera external parameters to obtain the first positioning information (x, y, σ) of the traffic light in the vehicle coordinate system. bc , where x and y represent coordinates, and σ represents the heading angle. Combining the Gaussian projection transformation principle, the vehicle's Global Navigation Satellite System (GNSS) coordinates, and the relative posture (referring to the posture between the vehicle and the traffic light), the first positioning information of the traffic light in the vehicle coordinate system is transformed to obtain the first positioning information of the vehicle in the navigation coordinate system.
[0095] At the same time, the vehicle controller obtains the RSU positioning information (B, L, H) of the traffic light from the RSU, where B represents latitude, L represents longitude, and H represents elevation. Similarly, a series of transformations are performed on the RSU positioning information of the traffic light to obtain the second positioning information of the vehicle in the navigation coordinate system.
[0096] Then, the positioning information error (x) between the first positioning information and the second positioning information of the vehicle in the navigation coordinate system is determined. ck ,y ck ,ε ψk The vehicle's positioning information error in the navigation coordinate system is processed by Kalman filtering to obtain an estimated heading angle error, and the heading angle measured by the INS (the measurement time is the same as the time when the camera collects the image) is compensated according to the estimated heading angle error. In this way, the heading drift of the INS can be corrected in real time during the vehicle's driving process to avoid parking calibration.
[0097] Next, the vehicle heading angle calibration method provided in the embodiment of the present application will be further explained in the form of steps.
[0098] Step S1: Considering the GNSS positioning system and the INS reference coordinate system, any of the following coordinate systems can be set as the navigation coordinate system: WGS-84 geodetic coordinate system (e system), Gaussian plane rectangular coordinate system (n system), navigation coordinate system (t system), geographic coordinate system (g system), carrier coordinate system (b system). Figure 5 As shown, the coordinate system with the vehicle as the origin is the vehicle coordinate system, and the other coordinate system is the navigation coordinate system.
[0099] Assuming that the roll angle of the vehicle measured by INS is φ, the pitch angle is θ, and the heading angle is ψ, the transformation matrix R from the vehicle coordinate system to the navigation coordinate system is b2n for:
[0100]
[0101] Step S2: Set the actual heading angle of the vehicle at time k to ψ k ', the actual forward speed of the vehicle in the longitudinal direction is v k ', the true acceleration in the longitudinal direction of the vehicle body is a k ',but
[0102] ψ k '=ψ k +ε ψk +ξ ψk
[0103] v k '=v k +ε vk +ξ vk
[0104] a k '=a k +ε ak +ξ ak
[0105] Among them, ψ k Represents the measured value of the heading angle, v k represents the measurement of forward speed, a k Indicates the measured value of acceleration; ε ψk Represents the measurement error of the heading angle, ε vk represents the measurement error of the forward speed, ε ak represents the measurement error of acceleration; ξ ψk represents the random measurement error of the heading angle, ξ vk represents the random measurement error of the forward speed, ξ ak Represents the random measurement error of acceleration, where the random measurement error refers to the random noise in the measurement process.
[0106] Based on Newton's second law, the motion equation of the vehicle in the navigation coordinate system is established:
[0107] x ck =x ck-1 +v k 'cosψ k 'dt
[0108] y ck =y ck-1 +v k 'sinψ k 'dt
[0109] Step S3: Set ψ k '=ψ k +ε ψk +ξ ψk Substituting into the vehicle's motion equation in the navigation coordinate system, we get:
[0110] x ck =x ck-1 +v k cosψ k dt-v k ε ψk sinψ k dt
[0111] y ck =y ck-1 +v k sinψ k dt+v k ε ψk cosψ k dt
[0112] The above equation is expressed in the form of the state transition equation of the Kalman filter as:
[0113] X k =A k X k-1 +b k +u k
[0114]
[0115] Among them, X k =[x ck ,y ck ,ε ψk ] represents the state space vector (or measurement vector, observation vector); A k Represents the state transfer matrix, which is updated in real time by the measurements of the INS.
[0116] Step S4: Calculate the positioning information error (x ck ,yck ,ε ψk ).
[0117] Step S5: The positioning information error (x ck ,y ck ,ε ψk ) as the measurement vector, combined with the coordinate transformation formula, the measurement equation of the Kalman filter is as follows:
[0118] Z gk =H k X k +d k +ν k
[0119] in, ξ gxk , gyk represents the random positioning error of the traffic light target detected by the BEV visual perception model in the navigation coordinate system; the noise variance matrix of the measurement vector is Among them, r xk 2 For gxk The variance statistic, r yk 2 For gyk The variance statistic, r Ψk 2 For gΨk The variance statistic of .
[0120] Based on the above derivation, the recursive difference equation of the linear discrete Kalman filter is used to predict the state vector and correct the measurement vector.
[0121] Among them, the prediction equation group is:
[0122]
[0123] The correction equations are:
[0124] K k =P k - H k T (H k P k - H k T +R k ) -1
[0125]
[0126] P k =(IK kH k ) k -
[0127] After being processed by the recursive difference equation of the linear discrete Kalman filter, the estimated heading angle error can be obtained. The heading angle measured by the INS is subtracted from the estimated heading angle error to obtain the target heading angle, thereby achieving heading angle compensation. The obtained target heading angle can be further applied, for example, it can be used in vehicle navigation systems to improve the accuracy and reliability of vehicle positioning and navigation; it can be used in vehicle control systems to improve the direction control ability of the vehicle during driving and enhance the driving stability and comfort of the vehicle; it can be used in autonomous driving technology to provide important support for autonomous driving technology, thereby improving the vehicle's perception of the surrounding environment and enhancing the stability and safety of the autonomous driving system.
[0128] The embodiments of the present application can at least achieve the following technical effects:
[0129] 1) Traditional methods usually rely on single sensor data to control the heading angle stability, which often leads to insufficient accuracy or poor environmental adaptability. The embodiment of the present application combines the data of BEV and INS, and through the fusion of data from these two different sources, it realizes the accurate measurement and stable control of the vehicle heading angle; at the same time, it can comprehensively utilize the advantages of the two sensors, thereby reducing the complexity of sensor data fusion and improving the stability and accuracy of the overall navigation system.
[0130] 2) The performance of the visual perception system may be limited in complex environments, for example, its performance may decrease in severe weather conditions or low light environments, resulting in reduced accuracy in target detection and tracking. The embodiments of the present application can compensate for the limitations of the visual perception system in specific environments by combining INS data, thereby improving the stability and reliability of the overall system.
[0131] 3) Traditional methods are highly dependent on the surrounding environment. For example, INS calibration requires an open environment, and the visual perception system requires good lighting conditions and clear images. The embodiment of the present application can achieve accurate estimation of the heading angle in different environments by comprehensively utilizing the data of the two sensors, thereby reducing the dependence on the surrounding environment and improving the applicability and stability of the system.
[0132] 4) By adopting the vehicle heading angle calibration method provided in the embodiment of the present application, the vehicle can perceive the surrounding environment more accurately and realize accurate measurement and control of the heading angle through data fusion, which helps to reduce the driving instability caused by heading deviation and improve the driving stability of the car, thereby providing a smoother and more comfortable driving experience; at the same time, the INS can be calibrated online without interrupting the vehicle navigation, thereby improving the continuity and stability of the navigation system.
[0133] 5) By accurately controlling the heading angle of the vehicle, it helps to avoid potential safety hazards, such as avoiding collisions with other vehicles or obstacles, and reducing the risk of traffic accidents caused by unstable heading. Therefore, the embodiments of the present application can significantly improve the safety of automobile driving.
[0134] 6) The embodiments of the present application utilize the global vision and unified coordinate framework of the BEV to achieve comprehensive perception of the vehicle's surrounding environment, effectively overcoming the problem of the limited vision of a single sensor, which enables the vehicle to adapt to different road conditions and driving environments, including complex traffic scenes and changeable weather conditions. Therefore, the use of the embodiments of the present application can improve the environmental adaptability of the vehicle, allowing it to maintain stable driving performance in various scenarios.
[0135] The following continues to describe the exemplary structure of the vehicle heading angle calibration device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the vehicle heading angle calibration device 455 of the memory 450 may include: a first perception module 4551, which is used to collect and process images of the vehicle's surrounding environment through the vehicle's camera, and perform target detection processing on the collected images to obtain image positioning information of target environmental elements in the vehicle's surrounding environment; a second perception module 4552, which is used to obtain RSU positioning information of the target environmental elements from the roadside unit RSU; an error calculation module 4553, which is used to determine the positioning information error of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental elements and the RSU positioning information; a Kalman filter module 4554, which is used to perform Kalman filter processing on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error; and a compensation module 4555, which is used to compensate for the heading angle measured by the vehicle's inertial navigation system INS according to the estimated heading angle error.
[0136] In some embodiments, the error calculation module 4553 is also used to: determine the first positioning information of the vehicle in the navigation coordinate system based on the image positioning information of the target environmental element; determine the second positioning information of the vehicle in the navigation coordinate system based on the RSU positioning information of the target environmental element; and determine the positioning information error between the first positioning information and the second positioning information of the vehicle in the navigation coordinate system.
[0137] In some embodiments, the error calculation module 4553 is also used to: perform coordinate system transformation processing on the image positioning information of the target environmental element to obtain the first positioning information of the target environmental element in the navigation coordinate system; perform posture transformation processing on the first positioning information of the target environmental element in the navigation coordinate system according to the relative posture to obtain the first positioning information of the vehicle in the navigation coordinate system; wherein the relative posture represents the posture between the vehicle and the target environmental element.
[0138] In some embodiments, the error calculation module 4553 is also used to: transform the image positioning information of the target environmental element to the vehicle coordinate system to obtain the first positioning information of the target environmental element in the vehicle coordinate system; transform the first positioning information of the target environmental element in the vehicle coordinate system to the navigation coordinate system to obtain the first positioning information of the target environmental element in the navigation coordinate system.
[0139] In some embodiments, the error calculation module 4553 is also used to: measure the roll angle, pitch angle and heading angle of the vehicle through INS; determine the transformation matrix based on the roll angle, pitch angle and heading angle measured by INS; wherein the transformation matrix is used to transform the first positioning information of the target environmental element in the vehicle coordinate system to the navigation coordinate system.
[0140] In some embodiments, the vehicle has multiple cameras; the first perception module 4551 is also used to: fuse the images captured by the multiple cameras of the vehicle in a bird's-eye view BEV image coordinate system to obtain a BEV image; perform target detection processing on the BEV image to obtain image positioning information of target environmental elements in the vehicle's surrounding environment.
[0141] In some embodiments, the Kalman filter module 4554 is also used to: establish a recursive difference equation of the Kalman filter; wherein the recursive difference equation takes a measurement vector as input; the positioning information error of the vehicle in the navigation coordinate system is used as a measurement vector and input into the recursive difference equation to obtain an estimated heading angle error output by the recursive difference equation.
[0142] In some embodiments, the vehicle heading angle calibration device 455 further includes a calibration module for: performing external parameter calibration on the camera and INS; and performing time calibration on the camera and INS.
[0143] The embodiment of the present application provides a computer program product or a computer program, which includes executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the executable instructions from the computer-readable storage medium, and the processor executes the executable instructions, so that the electronic device executes the vehicle heading angle calibration method described in the embodiment of the present application.
[0144] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the vehicle heading angle calibration method provided by the embodiment of the present application.
[0145] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.
[0146] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.
[0147] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).
[0148] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.
[0149] The above are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.
Claims
1. A vehicle heading angle calibration method, characterized in that: include: Performing image acquisition and processing of the vehicle's surrounding environment through the vehicle's camera, and performing target detection processing on the acquired image to obtain image positioning information of target environmental elements in the vehicle's surrounding environment; Acquire RSU positioning information of the target environmental element from a roadside unit RSU; Determining the positioning information error of the vehicle in the navigation coordinate system according to the image positioning information of the target environmental element and the RSU positioning information; Performing Kalman filtering on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error; The heading angle measured by the inertial navigation system INS of the vehicle is compensated according to the estimated heading angle error.
2. The method according to claim 1, characterized in that The determining, according to the image positioning information of the target environmental element and the RSU positioning information, the positioning information error of the vehicle in the navigation coordinate system comprises: Determining first positioning information of the vehicle in a navigation coordinate system according to the image positioning information of the target environmental element; Determine second positioning information of the vehicle in the navigation coordinate system according to the RSU positioning information of the target environmental element; A positioning information error between first positioning information and second positioning information of the vehicle in a navigation coordinate system is determined.
3. The method according to claim 2, characterized in that The determining the first positioning information of the vehicle in the navigation coordinate system according to the image positioning information of the target environmental element includes: Performing coordinate system transformation processing on the image positioning information of the target environmental element to obtain first positioning information of the target environmental element in the navigation coordinate system; The first positioning information of the target environmental element in the navigation coordinate system is subjected to a posture transformation process according to the relative posture to obtain the first positioning information of the vehicle in the navigation coordinate system; wherein the relative posture represents the posture between the vehicle and the target environmental element.
4. The method according to claim 3, characterized in that The performing coordinate system transformation processing on the image positioning information of the target environmental element to obtain first positioning information of the target environmental element in the navigation coordinate system includes: Transforming the image positioning information of the target environmental element into a vehicle coordinate system to obtain first positioning information of the target environmental element in the vehicle coordinate system; The first positioning information of the target environmental element in the vehicle coordinate system is transformed into the navigation coordinate system to obtain the first positioning information of the target environmental element in the navigation coordinate system.
5. The method according to claim 4, characterized in that Before transforming the first positioning information of the target environmental element in the vehicle coordinate system to the navigation coordinate system, the method further includes: Measuring the roll angle, pitch angle and heading angle of the vehicle by the INS; A transformation matrix is determined according to the roll angle, pitch angle and heading angle measured by the INS; wherein the transformation matrix is used to transform the first positioning information of the target environmental element in the vehicle coordinate system to the navigation coordinate system.
6. The method according to claim 1, characterized in that The number of cameras of the vehicle includes a plurality; the target detection processing is performed on the collected images to obtain the image positioning information of the target environmental elements in the environment around the vehicle, including: fusing the images respectively collected by the multiple cameras of the vehicle in a bird's-eye view BEV image coordinate system to obtain a BEV image; The BEV image is subjected to target detection processing to obtain image positioning information of target environmental elements in the vehicle surroundings.
7. The method according to claim 1, characterized in that The performing Kalman filtering on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error includes: Establishing a recursive difference equation of a Kalman filter; wherein the recursive difference equation takes a measurement vector as input; The positioning information error of the vehicle in the navigation coordinate system is used as a measurement vector and input into the recursive difference equation to obtain an estimated heading angle error output by the recursive difference equation.
8. The method according to claim 1, characterized in that The method further comprises: Performing external parameter calibration on the camera and the INS; The camera and the INS are time-calibrated.
9. A vehicle heading angle calibration device, characterized in that: include: A first perception module is used to collect and process images of the vehicle's surrounding environment through the vehicle's camera, and to perform target detection processing on the collected images to obtain image positioning information of target environmental elements in the vehicle's surrounding environment; The second perception module is used to obtain the RSU positioning information of the target environmental element from the roadside unit RSU; An error calculation module, used to determine the positioning information error of the vehicle in the navigation coordinate system according to the image positioning information of the target environmental element and the RSU positioning information; A Kalman filter module is used to perform Kalman filter processing on the positioning information error of the vehicle in the navigation coordinate system to obtain an estimated heading angle error; The compensation module is used to compensate the heading angle measured by the inertial navigation system INS of the vehicle according to the estimated heading angle error.
10. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the vehicle heading angle calibration method described in any one of claims 1 to 8 when executing the executable instructions stored in the memory.
11. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to implement the vehicle heading angle calibration method described in any one of claims 1 to 8 when executed by a processor.
12. A computer program product, characterized in that The method comprises executable instructions for implementing the vehicle heading angle calibration method according to any one of claims 1 to 8 when executed by a processor.
Citation Information
Patent Citations
Agricultural machinery roll angle test method based on double antenna GNSS (global navigation satellite system) receiver / gyroscope information fusion
CN106772517A
Obstacle positioning method and device for automatic driving system
CN111443704A
Fusion sensing positioning system based on vehicle infrastructure cooperation
CN116170749A
Own position locating device, own position locating method, and own position locating program
JP2008249639A
Using object to align and calibrate inertial navigation system
US20110178708A1
Cited By
Indoor vehicle positioning method and device and storage medium
CN120107357A
Vehicle-road cooperation calibration system and method, electronic equipment and vehicle
CN120412284A
Vehicle positioning anomaly detection method, device, equipment, medium and program
CN120800433A
Vehicle positioning anomaly detection method, device, equipment, medium and program
CN120800433B
