A method and device for processing observation noise

Abstract

An embodiment of the present invention relates to a method and apparatus for processing observation noise. The method includes: obtaining a two-dimensional image captured by an in-vehicle camera at time t as a first image; using a target detection model to perform target detection on the first image to obtain a first target; determining the radial distance and the lateral X-axis angle between the host vehicle and the first target in the host vehicle coordinate system according to the first target coordinates (x t , y t ); determining the radial sample variance #imgabs0# and the tangential sample variance #imgabs1# of the first target at time t according to the radial distance d t and the angle α t ; constructing an observation noise covariance matrix R t of the first target at time t according to the radial sample variance #imgabs2#, the tangential sample variance #imgabs3# and the angle α t . Through the present invention, the stability and robustness of the target motion state prediction of the perception module can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a method and device for processing observation noise. Background Art

[0002] During the driving process of a vehicle, the perception module in an autonomous driving system or a driverless system of the vehicle continuously tracks the motion states of surrounding environment targets. The implementation process of the perception module for processing the target motion state tracking is briefly as follows: receiving an image generated by an in-vehicle camera shooting the traffic environment around the vehicle, and performing target detection on the image based on a preset target detection model to obtain the current observation position of the target in the vehicle coordinate system, and then using a Kalman filter to predict the motion state of the target at the current moment based on the current observation position of the target and the predicted value of the motion state at the previous moment. It is not difficult to see from the above description that the position detection accuracy of the target detection model used by the perception module will directly affect the prediction accuracy of the target motion state of the perception module. In principle, to ensure the stability and robustness of the target motion state prediction of the perception module, it is necessary to uniformly plan the output stability and robustness of all target detection models used by it, but this is very difficult to achieve in the actual implementation process. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, device, electronic device and computer-readable storage medium for processing observation noise in view of the defects of the prior art, statistically calculate the radial and tangential sample variances of the target coordinates output by the target detection model in the radial and tangential directions of the camera, and construct an observation noise covariance matrix of the Kalman filter based on the obtained radial and tangential sample variances. Through the present invention, it is neither necessary to modify any target detection model nor to modify the Kalman filter to adapt to each target detection model. Only by sending the obtained observation noise covariance matrix into the Kalman filter as the observation noise of the current observation quantity can the different output errors of different target detection models be adaptively corrected, so as to achieve the purpose of improving the prediction stability and robustness of the target motion state of the perception module.

[0004] To achieve the above purpose, a first aspect of an embodiment of the present invention provides a method for processing observation noise, and the method includes:

[0005] Obtain a two-dimensional image captured by an in-vehicle camera at time t as a first image;

[0006] Use a target detection model to perform target detection on the first image to obtain a first target; the first target corresponds to a first target coordinate (x t , y t ); the first target coordinate (x t , y t) The coordinate system is the vehicle's own coordinate system;

[0007] According to the first target coordinate (x t , y t ), determine the radial distance and the lateral X-axis angle between the vehicle itself and the first target in the vehicle's own coordinate system to obtain the corresponding radial distance d t and the angle α t ;

[0008] According to the radial distance d t and the angle α t determine the radial sample variance of the first target at time t and the tangential sample variance

[0009] According to the radial sample variance the tangential sample variance and the angle α t construct the observation noise covariance matrix R of the first target at time t t .

[0010] Preferably, the step of determining the radial distance and the lateral X-axis angle between the vehicle itself and the first target in the vehicle's own coordinate system according to the first target coordinate (x t , y t ) to obtain the corresponding radial distance d t and the angle α t specifically includes:

[0011] When the vehicle's own coordinate system is the right-front-up coordinate system, with the vehicle's driving direction as the positive direction of the Y-axis and the direction perpendicular to the Y-axis on the right side of the vehicle as the positive direction of the X-axis, and according to the first target coordinate (x t , y t ) calculate the radial distance d t and the angle α t ,

[0012]

[0013] Preferably, the step of determining the radial sample variance and the tangential sample variance of the first target at time t according to the radial distance d t and the angle α t specifically includes:

[0014] ​​Obtain the radial distance sequences of the n first targets from time t-(n - 1) to time t, and obtain the lateral X-axis angle sequences of the n first targets from time t-(n - 1) to time t; the radial distance sequences include multiple radial distances d i ; the angle sequences include multiple angles α i ; t-(n - 1)≤i≤t, and n is a positive integer;

[0015] Determine the average radial distance d av and the average angle α av ;

[0016] According to the radial distance d t and the average radial distance d av determine the radial sample variance of the first target at time t

[0017] According to the radial distance d t , the angle α t and the average angle α av determine the tangential sample variance of the first target at time t

[0018] Preferably, constructing the observation noise covariance matrix R of the tangential sample variance and the angle α t of the first target at time t, specifically including: t According to the radial sample variance

[0019] the tangential sample variance and the angle α construct the observation noise covariance matrix R t of the first target at time t in the vehicle coordinate system t ,

[0020]

[0021] Preferably, the method further includes determining the predicted state quantity of the first target at time t according to the first target coordinates (x t , y t ) and the observation noise covariance matrix R t , specifically:

[0022] Use the Kalman filter to perform state prediction on the first target;

[0023] Substitute the observation noise covariance matrix R t into the Kalman filter gain equation K of the Kalman filter t = P t,t-1 H T (HP t,t-1 H T + R t ) to calculate the Kalman filter gain K at time t t ; where H is the state-observation transformation matrix, and P t,t-1 is the state noise covariance matrix from time t-1 to time t, and H and P t,t-1 are known quantities;

[0024] Use the first target coordinates (x t , y t ) as the observed quantity B of the Kalman filter at time t t , B t = [x t y t T ;

[0025] Substitute the state-observation transformation matrix H, the Kalman filter gain K t and the observed quantity B t into the Kalman filter state prediction equation A of the Kalman filter t = FA t-1 + K t [B t - H(FA t-1 )] to calculate the predicted state quantity A of the first target at time t t ; where F is the state transition matrix, and A t-1 is the predicted state quantity at time t-1, and F and A t-1 are known quantities.

[0026] A second aspect of the embodiments of the present invention provides a device for implementing the method for processing observation noise described in the first aspect above. The device includes: an acquisition module, a target detection processing module, and an observation noise processing module;

[0027] The acquisition module is used to acquire a two-dimensional image captured by the vehicle-mounted camera at time t as the first image;

[0028] The target detection processing module is used to perform target detection on the first image using a target detection model to obtain a first target; the first target corresponds to a first target coordinate (x t , y t ); the first target coordinate (x t , y t ​) The coordinate system is the ego-vehicle coordinate system;

[0029] The observation noise processing module is configured to determine the radial distance and the lateral X-axis angle between the ego-vehicle and the first target in the ego-vehicle coordinate system according to the first target coordinates (x t , y t ) to obtain the corresponding radial distance d t and the angle α t ; and determine the radial sample variance of the first target at time t according to the radial distance d t and the angle α t and determine the tangential sample variance and the tangential sample variance And according to the radial sample variance The tangential sample variance and the angle α t Construct the observation noise covariance matrix R of the first target at time t t .

[0030] A third aspect of the embodiments of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0031] The processor is used to be coupled with the memory, read and execute the instructions in the memory to implement the method steps described in the first aspect above;

[0032] The transceiver is coupled with the processor, and the processor controls the transceiver to perform message sending and receiving.

[0033] A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is caused to execute the instructions of the method described in the first aspect above.

[0034] The embodiments of the present invention provide a method, apparatus, electronic device, and computer-readable storage medium for processing observation noise, which statistically calculate the radial and tangential sample variances of the target coordinates output by the target detection model in the radial and tangential directions of the camera, and construct the observation noise covariance matrix of the Kalman filter based on the obtained radial and tangential sample variances. Through the present invention, neither the target detection model needs to be modified, nor the Kalman filter needs to be modified to adapt to each target detection model. Only by sending the obtained observation noise covariance matrix into the Kalman filter as the observation noise of the current observation quantity can the different output errors of different target detection models be adaptively corrected. In this way, not only the compatibility of the perception module with different target detection models is improved, but also the stability and robustness of the target motion state prediction of the perception module are improved. Description of the Drawings

[0035] Figure 1 Schematic diagram of a method for processing observation noise provided in the first embodiment of the present invention;

[0036] Figure 2 Radial distance d in the right-front-sky coordinate system provided in the first embodiment of the present invention t and included angle α t schematic diagram;

[0037] Figure 3 Module structure diagram of a device for processing observation noise provided in the second embodiment of the present invention;

[0038] Figure 4 Schematic diagram of the structure of an electronic device provided in the third embodiment of the present invention. Detailed implementation manners

[0039] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] Before elaborating on the embodiments of the present invention, a brief introduction to the Kalman filter is first given. The equations of the Kalman filter are generally composed of a motion state prediction equation and an observation-state conversion equation:

[0041] The motion state prediction equation is A t = FA t-1 + w t-1 ,

[0042] The observation-state conversion equation is B t = HA t + v t ;

[0043] Among them, A t is the predicted state quantity at time t, A t-1 is the predicted state quantity at time t-1, F is the state transition matrix, w t-1 is the system noise of the target at time t-1, B t is the observed quantity at time t, H is the state-observation conversion matrix, v t is the observation noise at time t. Further, both the state transition matrix F and the state-observation conversion matrix H are determined by a pre-set target motion model (such as a uniform motion model, a uniformly accelerated motion model, etc.). Specifically, reference can be made to the setting implementation of the Kalman filter equations for different motion models disclosed, which will not be elaborated one by one here.

[0044] Given the known predicted state variable A t-1 , observed variable B t , system noise w t-1 , state transition matrix F, and state-observation conversion matrix H, solving the above Kalman filter equations yields the predicted state variable A at time t t There are multiple implementation methods, and a common one is to solve based on a linear relationship. The detailed derivation process can refer to the published calculation implementation of linear / offline Kalman filtering and will not be elaborated here one by one; the derivation result of this method is to introduce the Kalman filter gain K and combine it with the observation-state conversion equation to transform the motion state prediction equation into:

[0045] A t = FA t-1 + K t [B t - H(FA t-1 )],

[0046] K t = P t,t-1 H T (HP t,t-1 H T + R t );

[0047] Among them, K t is the Kalman filter gain at time t, P t,t-1 is the state noise covariance matrix from time t-1 to time t, R t is the observation noise covariance matrix at time t. Given the system noise w t-1 , the state noise covariance matrix P t,t-1 is also known.

[0048] It can be easily seen from the transformed motion state prediction equation that given the predicted state variable A t-1 , observed variable B t , system noise w t-1 , state transition matrix F, state-observation conversion matrix H, and state noise covariance matrix P t,t-1 , only by confirming the observation noise covariance matrix R t can the Kalman filter gain K t be obtained, and thus the predicted state variable A t can be obtained.

[0049] Referring to the above principle, Embodiment 1 of the present invention provides a method for processing observation noise, which statistically calculates the radial and tangential sample variances of the target coordinates output by the target detection model in the radial and tangential directions of the camera, and constructs the observation noise covariance matrix of the Kalman filter based on the obtained radial and tangential sample variances; then substituting the obtained observation noise covariance matrix into the above-converted motion state prediction equation to obtain the predicted state quantity A t ; In this way, neither the target detection model needs to be modified, nor the Kalman filter needs to be modified to adapt to each target detection model. Only by sending the obtained observation noise covariance matrix into the Kalman filter as the observation noise of the current observation quantity can the different output errors of different target detection models be adaptively corrected, thereby not only improving the compatibility of the perception module with different target detection models, but also improving the stability and robustness of the target motion state prediction of the perception module.

[0050] Figure 1 The following is a schematic diagram of a method for processing observation noise provided by Embodiment 1 of the present invention. As Figure 1 shown, the method mainly includes the following steps:

[0051] Step 1: Obtain a two-dimensional image captured by the vehicle-mounted camera at time t as the first image.

[0052] Here, in the vehicle autonomous driving system or the driverless system, the perception module obtains a two-dimensional image generated by the vehicle-mounted camera taking pictures of the traffic environment around the vehicle at time t, and uses it as the first image to be detected.

[0053] Step 2: Use the target detection model to perform target detection on the first image to obtain the first target;

[0054] Among them, the first target corresponds to a first target coordinate (x t , y t ); the coordinate system of the first target coordinate (x t , y t ) is the vehicle coordinate system.

[0055] Here, the perception module uses a target detection model pre - installed locally that supports target depth estimation and the conversion from pixel coordinate system to vehicle coordinate system to perform target detection on the first image row. This detection model can identify one or more entity targets (such as vehicles, traffic signs, obstacles, pedestrians, animals, plants, buildings, etc.), and can perform depth estimation and pixel coordinate positioning for each identified entity target. Furthermore, based on the well - known conversion method of pixel coordinate system - image coordinate system - camera coordinate system - world coordinate system - vehicle coordinate system, it can locate the position coordinates of each entity target in the vehicle coordinate system according to the depth estimation information and pixel coordinate information of each entity target, thereby outputting the corresponding vehicle coordinate system coordinates, that is, the first target coordinates (x t , y t ). The target detection model in the embodiments of this method can be implemented using a variety of model structures. Among them, the commonly used one is the 3D monocular (Mono3D) target detection model. For the specific implementation of the Mono3D target detection model, reference can be made to relevant implementation documents, and no further elaboration will be made here. From the well - known vehicle coordinate system structure, there are two types of vehicle coordinate systems: the right - front - top vehicle coordinate system and the front - left - top vehicle coordinate system. In the right - front - top vehicle coordinate system, the X - axis is the right - hand direction facing the front of the vehicle, the Y - axis is the vehicle driving direction, and the Z - axis is the direction perpendicular to the ground and pointing to the roof of the vehicle. In the front - left - top vehicle coordinate system, the X - axis is the vehicle driving direction, the Y - axis is the left - hand direction facing the front of the vehicle, and the Z - axis is the direction perpendicular to the ground and pointing to the roof of the vehicle. In the embodiments of the present invention, the right - front - top vehicle coordinate system is defaultly adopted.

[0056] Step 3, determine the radial distance and the lateral X - axis angle between the vehicle and the first target in the vehicle coordinate system according to the first target coordinates (x t , y t ) to obtain the corresponding radial distance d t and the angle α t ;

[0057] Specifically, when the vehicle coordinate system is the right - front - top coordinate system, with the vehicle driving direction as the positive direction of the Y - axis and the direction perpendicular to the Y - axis on the right side of the vehicle as the positive direction of the X - axis, and according to the first target coordinates (x t , y t ) calculate the radial distance d t and the angle α t ,

[0058]

[0059] Here, Figure 2 is the schematic diagram of the radial distance d t and the angle α t in the right - front - top coordinate system provided by Embodiment 1 of the present invention. From Figure 2It can be seen that the radial distance d t from the first target coordinate (x t , y t ) to the origin O and the included angle α t in the right-front-day vehicle coordinate system are defined as follows. That is, the radial distance d t is the straight-line distance from the origin to the first target coordinate (x t , y t ) along the radial direction of the camera. The included angle α t is the included angle between the radial straight line from the first target coordinate (x t , y t ) to the origin and the horizontal X-axis. In fact, the included angle α t is also the included angle between the tangential straight line at the first target coordinate (x t , y t ) and the Y-axis. Based on the definitions of the radial distance d t and the included angle α t , using the Pythagorean theorem and trigonometric functions, the corresponding values of the radial distance d t , y t ) can be calculated. It should be noted that the current step is based on the calculation method of the radial distance d t and the included angle α t in the right-front-day vehicle coordinate system. If the current vehicle coordinate system is the front-left-day vehicle coordinate system, the horizontal axis becomes the Y-axis, and the radial distance d t and the included angle α t are still t t but α t , y t ) becomes the included angle between the radial straight line from the origin to the first target coordinate (x

[0060] Step 4. Determine the radial sample variance t and the tangential sample variance t of the first target at time t according to the radial distance d and the included angle α

[0061] Specifically, it includes: Step 41. Obtain the radial distance sequences of n first targets from time t-(n-1) to time t; and obtain the included angle sequences of the horizontal X-axis of n first targets from time t-(n-1) to time t;

[0062] Among them, the radial distance sequence includes multiple radial distances d i ; the included angle sequence includes multiple included angles α i ; t-(n-1)≤i≤t, and n is a positive integer;

[0063] Here, since the sensing module calculates the radial distance d t and the included angle α t at each moment, the radial distance and included angle at each previous historical moment can be obtained at any moment t;

[0064] Step 42: Determine the average radial distance d av and the average included angle α av of the first target at moment t according to the radial distance sequence and the included angle sequence;

[0065] Among them,

[0066] Step 43: Determine the radial sample variance t of the first target at moment t according to the radial distance d av and the average radial distance d

[0067] Among them,

[0068] Step 44: Determine the tangential sample variance t of the first target at moment t according to the radial distance d t the included angle α av and the average included angle α

[0069] Among them,

[0070] For example, if n = 100, the radial distance sequence consists of 100 radial distances d i and the included angle sequence consists of 100 included angles α i ; then, the average radial distance d av and the average included angle α av at moment t are: The radial sample variance of the first target at moment t is The tangential sample variance is

[0071] Step 5: Construct the observation noise covariance matrix R of the first target at moment t according to the radial sample variance the tangential sample variance t and the included angle α t ;

[0072] Specifically, it includes: according to the radial sample variance the tangential sample variance and the included angle α t , construct the observation noise covariance matrix R of the first target in the ego-vehicle coordinate system at time t t ,

[0073]

[0074] Here, in fact, the tangential-radial covariance matrix composed of the radial sample variance and the tangential sample variance is rotated once according to the corresponding relationship between the radial-tangential and the X-Y axes of the ego-vehicle coordinate system which is specifically the right-front-day coordinate system

[0075] The observation noise covariance matrix R obtained by the method of the embodiment of the present invention through the above steps 1-5 t is actually the observation noise corresponding to the current target detection model

[0076] After the method of the embodiment of the present invention obtains the observation noise covariance matrix R t , according to the first target coordinates (x t , y t ) and the observation noise covariance matrix R t determine the predicted state quantity of the first target at time t, specifically including

[0077] Step A1, use the Kalman filter to perform state prediction on the first target

[0078] Step A2, substitute the observation noise covariance matrix R t into the Kalman filter gain equation K t =P t,t-1 H T (HP t,t-1 H T +R t ) of the Kalman filter, and calculate the Kalman filter gain K at time t t ;

[0079] wherein, H is the state-observation transformation matrix, P t,t-1 is the state noise covariance matrix from time t-1 to time t, and H, P t,t-1 are known quantities

[0080] Step A3, use the first target coordinates (x t , y t ) as the observed quantity B t of the Kalman filter at time t, B t =[x t y t T ; ​

[0081] Step A4, substitute the state-observation conversion matrix H, the Kalman filter gain K t , and the observed quantity B t into the Kalman filter state prediction equation A of the Kalman filter t = FA t-1 + K t [B t - H(FA t-1 )], and calculate the predicted state quantity A of the first target at time t t ;

[0082] where F is the state transition matrix, and A t-1 is the predicted state quantity at time t-1, and F and A t-1 are known quantities.

[0083] Here, the perception module uses the first target coordinates (x t , y t ) as the observed quantity B t , B t = [x t y t T ; and together with the observation noise covariance matrix R t , send them into the above-mentioned converted motion state prediction equation to calculate the predicted state quantity A at time t t . In specific calculations, the perception module first substitutes the observation noise covariance matrix R t into K t = P t,t-1 H T (HP t,t-1 H T + R t ) to calculate the Kalman filter gain K t ; here, as known from the previous text, P t,t-1 is the state noise covariance matrix from time t-1 to time t, and the state noise covariance matrix P t-1 is known and can be looked up when the system noise w t,t-1 is known; H is the state-observation conversion matrix, and the state-observation conversion matrix H is fixed and known after the target motion model is set. After obtaining the Kalman filter gain K t , the perception module then substitutes the observed quantity B t , the Kalman filter gain K t into A t = FA t-1 + K t [B t - H(FA t-1 )] to calculate the predicted state quantity A t ; here, as known from the previous text, A t-1 ​The predicted state quantity at time t-1 is known and can be queried; F is the state transition matrix, which is fixed and known after the target motion model is set.

[0084] Figure 3 As shown in the module structure diagram of a device for processing observation noise provided in the second embodiment of the present invention, the device is a terminal device or a server for implementing the foregoing method embodiment, or may be a device that enables the foregoing terminal device or server to implement the foregoing method embodiment. For example, the device may be a device or a chip system of the foregoing terminal device or server. As Figure 3 shown, the device includes: an acquisition module 201, a target detection processing module 202, and an observation noise processing module 203.

[0085] The acquisition module 201 is configured to acquire a two-dimensional image captured by a vehicle-mounted camera at time t as a first image.

[0086] The target detection processing module 202 is configured to perform target detection on the first image using a target detection model to obtain a first target; the first target corresponds to a first target coordinate (x t , y t ); the coordinate system of the first target coordinate (x t , y t ) is the ego-vehicle coordinate system.

[0087] The observation noise processing module 203 is configured to determine the radial distance and the lateral X-axis angle between the ego-vehicle and the first target in the ego-vehicle coordinate system according to the first target coordinate (x t , y t ) to obtain the corresponding radial distance d t and the angle α t ; and determine the radial sample variance t and the tangential sample variance t of the first target at time t according to the radial distance d and the angle α ; and construct the observation noise covariance matrix R of the tangential sample variance and the angle α t of the first target at time t. t .

[0088] A device for processing observation noise provided in an embodiment of the present invention can execute the method steps in the foregoing method embodiment, and its implementation principle and technical effect are similar, and will not be described herein again.

[0089] It should be noted that it should be understood that the division of each module of the above device is only a division of logical functions. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the acquisition module can be a separately established processing element, or can be integrated in a certain chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and the function of the above determination module can be called and executed by a certain processing element of the above device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or can be independently implemented. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the integrated logic circuit in the processor element or the instruction in the form of software.

[0090] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as: one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. Again, when a certain module above is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. Again, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0091] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the foregoing method embodiments are generated in whole or in part. The foregoing computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The foregoing computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the foregoing computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.). The foregoing computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0092] Figure 4 FIG. 4 is a schematic structural diagram of an electronic device provided in Embodiment 3 of the present invention. The electronic device can be the foregoing terminal device or server, or a terminal device or server that is connected to the foregoing terminal device or server and implements the method of the embodiments of the present invention. As Figure 4 shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver operations of the transceiver 303. Various instructions can be stored in the memory 302 to complete various processing functions and implement the processing steps described in the foregoing method embodiments. Preferably, the electronic device related to the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to implement communication connections between components. The foregoing communication port 306 is used for the electronic device to connect and communicate with other peripherals.

[0093] In Figure 4The system bus 305 mentioned above can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. This system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 it is only represented by a thick line in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to implement communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include Random Access Memory (RAM), and may also include non-volatile memory, such as at least one disk memory.

[0094] The above-mentioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), a Graphics Processing Unit (GPU), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0095] It should be noted that the embodiment of the present invention also provides a computer-readable storage medium, in which instructions are stored. When it runs on a computer, it enables the computer to execute the methods and processing procedures provided in the above embodiments.

[0096] The embodiment of the present invention also provides a chip for running instructions. This chip is used to execute the processing steps described in the foregoing method embodiments.

[0097] The embodiment of the present invention provides a method, device, electronic device, and computer-readable storage medium for processing observation noise. The radial and tangential sample variances of the target coordinates output by the target detection model are statistically analyzed in the radial and tangential directions of the camera, and an observation noise covariance matrix of the Kalman filter is constructed based on the obtained radial and tangential sample variances. Through the present invention, neither the target detection model needs to be modified, nor the Kalman filter needs to be modified to adapt to each target detection model. Only by sending the obtained observation noise covariance matrix into the Kalman filter as the observation noise of the current observed quantity can the different output errors of different target detection models be adaptively corrected. In this way, not only the compatibility of the perception module with different target detection models is improved, but also the stability and robustness of the target motion state prediction of the perception module are improved.

[0098] Those skilled in the art should further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0099] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0100] The specific embodiments described above have further elaborated on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing observation noise, characterized in that The method includes: Obtaining a two-dimensional image captured by an in-vehicle camera at time t as a first image; Performing object detection on the first image using an object detection model to obtain a first object; the first object corresponds to a first object coordinate (x t , y t ); the coordinate system of the first object coordinate (x t , y t ) is the ego-vehicle coordinate system; According to the first target coordinate (x t , y t ), the radial distance and the lateral X-axis angle between the host vehicle and the first target in the host vehicle coordinate system are determined to obtain the corresponding radial distance d t and the angle α t ; According to the radial distance d t and the included angle α t determine the radial sample variance of the first target at time t and the tangential sample variance According to the radial sample variance the tangential sample variance and the included angle α t construct the observation noise covariance matrix R of the first target at time t t ; wherein, determining the radial sample variance and the tangential sample variance of the first target at time t according to the radial distance d t and the included angle α t specifically includes: and the tangential sample variance Specifically, it includes: Obtain the radial distance sequences corresponding to the n first targets from time t-(n-1) to time t; and obtain the transverse X-axis angle sequences corresponding to the n first targets from time t-(n-1) to time t; the radial distance sequences include a plurality of radial distances d i ; the angle sequences include a plurality of angles α i ; t-(n-1)≤i≤t, where n is a positive integer; Determine the average radial distance d of the first target at time t according to the radial distance sequence and the included angle sequence av and the average included angle α av ; Based on the radial distance d t and the average radial distance d av determine the radial sample variance of the first target at time t According to the radial distance d t , the included angle α t and the average included angle α av to determine the tangential sample variance of the first target at time t The said according to the said radial sample variance The said tangential sample variance and the said included angle α t Construct the observation noise covariance matrix R of the said first target at time t t , specifically including: According to the radial sample variance the tangential sample variance and the included angle α t , construct the observation noise covariance matrix R of the first target in the ego-vehicle coordinate system at time t t , 2. The processing method of the observation noise according to claim 1, wherein Said determining the radial distance and the lateral X-axis angle between the host vehicle and the first target in the host vehicle coordinate system according to the first target coordinates (x t , y t ) to obtain the corresponding radial distance d t and the angle α t , specifically including: When the vehicle's own coordinate system is the right-front-top coordinate system, with the vehicle's driving direction as the positive direction of the Y-axis and the direction perpendicular to the Y-axis on the right side of the vehicle as the positive direction of the X-axis, and according to the first target coordinate (x t , y t ), calculate the radial distance d t and the included angle α t , 3. The method for processing observation noise according to claim 1, wherein The method further includes determining a predicted state quantity of the first target at time t according to the first target coordinate (x t , y t ) and the observation noise covariance matrix R t , specifically: Using a Kalman filter to perform state prediction on the first target; Substitute the observation noise covariance matrix R t into the Kalman filter gain equation K of the Kalman filter t = P t,t-1 H T (HP t,t-1 H T + R t ), and calculate the Kalman filter gain K at time t t ; where H is the state-observation transition matrix, and P t,t-1 is the state noise covariance matrix from time t - 1 to time t, and H and P t,t-1 are known quantities; Take the first target coordinate (x t , y t ) as the observation B of the Kalman filter at time t t , B t = [x t y t T ;​ Substitute the state-observation conversion matrix H, the Kalman filter gain K t and the observed quantity B t into the Kalman filter state prediction equation A of the Kalman filter t = FA t-1 + K t [B t - H(FA t-1 )], and calculate the predicted state quantity A of the first target at time t t ; where F is the state transition matrix, A t-1 is the predicted state quantity at time t-1, and F, A t-1 are known quantities.

4. An apparatus for implementing the method for processing the observation noise according to any one of claims 1-3, characterized in that, The device includes: an acquisition module, a target detection and processing module, and an observation noise processing module; The acquisition module is configured to obtain a two-dimensional image captured by an in-vehicle camera at time t as a first image; The target detection processing module is used to perform target detection on the first image using a target detection model to obtain a first target; the first target corresponds to a first target coordinate (x t , y t ); the coordinate system of the first target coordinate (x t , y t ) is the ego-vehicle coordinate system; The observation noise processing module is used to determine the radial distance and the lateral X-axis angle between the host vehicle and the first target in the host vehicle coordinate system according to the first target coordinates (x t , y t ) to obtain the corresponding radial distance d t and the angle α t ; and determine the radial sample variance t and the tangential sample variance t of the first target at time t according to the radial distance d and the angle α ; and construct the observation noise covariance matrix R of the first target at time t according to the radial sample variance , the tangential sample variance t and the angle α t .

5. An electronic device, characterized in that, It includes: A memory, a processor, and a transceiver; The processor is used to be coupled with the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-3; The transceiver is coupled with the processor, and the processor controls the transceiver to perform message sending and receiving.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1-3.

Images