Target trajectory prediction method, device and computer readable storage medium

Abstract

The application relates to a trajectory prediction method, device and computer readable storage medium of a target. The method comprises the following steps: acquiring the position of the target and the pose of a self vehicle in a global coordinate system in N historical frames of images by a radar and vision device; converting the position of the target in the global coordinate system in each frame of image of the N historical frames of images into N positions of the target in a vehicle coordinate system in a current frame of image according to the position of the target in the global coordinate system and the pose of the self vehicle in the N historical frames of images, wherein the vehicle coordinate system is a vehicle coordinate system in which the self vehicle is located; acquiring the credibility of the target being perceived by the radar and vision device in each frame of image; and obtaining the motion trajectory of the target by screening a fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame of image and the credibility of the target being perceived by the radar and vision device in each frame of image. The technical scheme provided by the application can efficiently and accurately predict the trajectory of the target without improving the hardware performance of the sensor.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, and in particular to a target trajectory prediction method, device and computer readable storage medium. BACKGROUND

[0002] In the field of automatic driving, effective prediction of the trajectory of a target such as a pedestrian or a vehicle is of great significance to the safe operation of an autonomous vehicle. Currently, there are mainly two methods for predicting the trajectory of a target, namely, a Kalman filter-based method and a deep learning-based method. Specifically, the Kalman filter-based trajectory prediction method mainly switches different Kalman motion models to predict the trajectory of a target according to the motion state of the target, such as accelerating straight, decelerating straight, constant speed, or decelerating lane change. The deep learning-based trajectory prediction method trains a trajectory prediction model by inputting the historical trajectory of a target into the trajectory prediction model, and obtains the trajectory prediction model based on mutual supervision training of a to-be-determined trajectory prediction model and a to-be-determined trajectory backtracking model, thereby predicting the trajectory of the target.

[0003] However, the above-mentioned Kalman filter-based trajectory prediction method has a high requirement for sensor hardware, and requires that the error of the observation quantity (such as speed and / or acceleration) of the target be within the corresponding error range of the model, otherwise the error will be large. The deep learning-based trajectory prediction method needs a large number of training sample sets, and has a large workload in extracting the features of the target, and the efficiency of the algorithm is low, which is not necessarily suitable for commercial environment. SUMMARY

[0004] To solve or partially solve the problems in the related art, the present application provides a target trajectory prediction method, device and computer readable storage medium, which can accurately predict the trajectory of a target efficiently without improving the performance of sensor hardware.

[0005] The first aspect of the present application provides a target trajectory prediction method, comprising:

[0006] obtaining the position of the target and the pose of the ego vehicle in the historical N frames of images in a global coordinate system by using a radar and vision device;

[0007] converting the position of the target in the global coordinate system in each frame of image of the historical N frames of images into N positions of a vehicle coordinate system in a current frame of image according to the position of the target in the global coordinate system and the pose of the ego vehicle in the global coordinate system in the historical N frames of images, the vehicle coordinate system being a vehicle coordinate system in which the ego vehicle is located;

[0008] obtaining the credibility of the target being perceived by the radar and vision device in each frame of image;

[0009] According to the N positions of the target in the vehicle coordinate system in the current frame image and the credibility of the target being perceived by the radar vision device in each frame image, the motion trajectory of the target is obtained by screening the fitted spline curve.

[0010] The second aspect of the present application provides a trajectory prediction device of a target, comprising:

[0011] The first obtaining module is configured to obtain the position of the target and the pose of the ego vehicle in the global coordinate system in the historical N frame images by the radar vision device.

[0012] The conversion module is configured to convert the position of the target in the global coordinate system in each frame image of the historical N frame images into N positions of the target in the vehicle coordinate system in the current frame image according to the position of the target in the global coordinate system and the pose of the ego vehicle in the historical N frame images.

[0013] The second obtaining module is configured to obtain the credibility of the target being perceived by the radar vision device in each frame image.

[0014] The fitting module is configured to obtain the motion trajectory of the target by screening the fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame image and the credibility of the target being perceived by the radar vision device in each frame image.

[0015] The third aspect of the present application provides an electronic device, comprising:

[0016] A processor; and

[0017] A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described above.

[0018] The fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon, which, when executed by the processor of an electronic device, causes the processor to perform the method as described above.

[0019] The technical scheme provided in the application can be known, compared with the track prediction method based on Kalman filtering provided by the related art, the technical scheme of the application has higher requirements for sensor hardware, the technical scheme of the application does not have special changes on the hardware such as a radar and vision device, only the position of a target in a global coordinate system and the pose of a vehicle in historical N frames of images and the reliability of the target perceived by the radar and vision device in each frame of image are obtained through the radar and vision device, and subsequently, according to the algorithm, the N positions of the target in the vehicle coordinate system in the current frame of image and the reliability of the target perceived by the radar and vision device in each frame of image, the motion track of the target is obtained by screening the fitted spline curve. Therefore, compared with the track prediction method based on deep learning which needs a large number of training sample sets and has low algorithm efficiency, the technical scheme of the application can accurately predict the track of the target efficiently without improving the performance of the sensor hardware.

[0020] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory and are not restrictive of the application. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout and in which:

[0022] Figure 1 is a flowchart of a track prediction method of a target provided by an embodiment of the application;

[0023] Figure 2 is a structural diagram of a track prediction device of a target provided by an embodiment of the application;

[0024] Figure 3 is a structural diagram of an electronic device provided by an embodiment of the application. DETAILED DESCRIPTION

[0025] Embodiments of the present application will be described in more detail by referring to the attached drawings. Although embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0026] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in this application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms "first," "second," "third," etc. can be employed in this application to describe various information, but these information should not be limited to these terms. These terms are only used to distinguish one piece of information from another piece of information of the same type. For example, the first information can also be called the second information without departing from the scope of the application, and similarly, the second information can also be called the first information. Therefore, the features defined with "first," "second" can explicitly or implicitly include one or more of the features. In the description of the application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0028] In the field of automatic driving, the effective prediction of the trajectory of a target such as a pedestrian or a vehicle is of great significance to the safe operation of an automatic driving vehicle. At present, the trajectory of a target is mainly predicted by two methods, namely, a Kalman filter-based method and a deep learning-based method. Specifically, the Kalman filter-based trajectory prediction method mainly switches different Kalman motion models to predict the trajectory of a target according to the motion state of the target, such as accelerating straight, decelerating straight, constant speed, or decelerating lane change. The deep learning-based trajectory prediction method is to input the historical trajectory of a target into a corresponding trajectory prediction model, and to obtain a trajectory prediction model based on mutual supervision training of a to-be-determined trajectory prediction model and a to-be-determined trajectory backtracking model, so as to predict the trajectory of the target. However, the above-mentioned Kalman filter-based trajectory prediction method has a high requirement for sensor hardware, and requires that the error of the observation quantity (such as speed and / or acceleration) of the target is within the corresponding error range of the model, otherwise the error is large. The deep learning-based trajectory prediction method needs a large number of training sample sets, and has a large workload in extracting the features of the target, and the efficiency of the algorithm is low, which is not necessarily suitable for commercial environment.

[0029] To solve the above problems, the embodiments of the present application provide a trajectory prediction method of a target, which can accurately predict the trajectory of the target efficiently without improving the performance of the sensor hardware.

[0030] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0031] Referring to Figure 1This is a flowchart illustrating a target trajectory prediction method according to an embodiment of this application, mainly including steps S101 to S104, as described below:

[0032] Step S101: Obtain the position of the target and the pose of the vehicle in the global coordinate system from the historical N frames of images using the radar vision device.

[0033] In this application embodiment, "radar-visual device" is a general term for various radars (e.g., millimeter-wave radar) and cameras. In principle, both radar and cameras can obtain the target's position through scanning or imaging. However, this application leverages the advantages of both radar and camera, using a fusion algorithm to obtain the target's position and the vehicle's pose in a global coordinate system. Here, the target's position in the global coordinate system is denoted as O. i (x i ,y i ,δ i The pose of the vehicle in the global coordinate system is denoted as... In the above expression, x i and y i Let δ represent the longitudinal and lateral displacements of the target in the current vehicle coordinate system in the i-th frame image, respectively. i Let x be the longitudinal and lateral displacements of the target in the current vehicle coordinate system. i and y i confidence level and Let t represent the longitudinal displacement, lateral displacement, and rotation angle of the vehicle's coordinate center in the global coordinate system in the i-th frame image, respectively. i This represents the timestamp of the i-th frame, with index i = 0, 1, ..., N-1. The O value output by the radar-guided device... i (x i ,y i ,δ i )and It is possible to obtain the position of the target and the pose of the vehicle in the global coordinate system from N historical frames of images. It should be noted that the N historical frames of images in this embodiment refer to images acquired by the radar vision device from a period of time before the current moment, and N here is a natural number.

[0034] Step S102: Based on the position of the target in the global coordinate system and the pose of the vehicle in the historical N frames, convert the position of the target in the global coordinate system in each frame of the historical N frames into N positions in the vehicle coordinate system in the current frame, where the vehicle coordinate system is the vehicle coordinate system in which the vehicle is located.

[0035] Since the vehicle coordinate system is a coordinate system established with respect to a certain center (for example, a geometric center) of the ego vehicle, and the state (position, speed, and orientation, etc.) of the ego vehicle is always changing, in order to facilitate the research or calculation, it is necessary to unify the positions of the target in the global coordinate system and the pose of the ego vehicle in the historical N frames of images to a certain frame of image. In the embodiments of the present application, the positions of the target in the global coordinate system in each frame of image of the historical N frames of images are converted into N positions of the target in the vehicle coordinate system in the current frame of image according to the positions of the target in the global coordinate system and the pose of the ego vehicle in the historical N frames of images. As an embodiment of the present application, the positions of the target in the global coordinate system in each frame of image of the historical N frames of images are converted into N positions of the target in the vehicle coordinate system in the current frame of image according to the positions of the target in the global coordinate system and the pose of the ego vehicle in the historical N frames of images can be: calculating the longitudinal displacement difference, the lateral displacement difference, and the rotation angle difference of the ego vehicle from the i-th frame of image to the current frame of image according to the longitudinal displacement, the lateral displacement, and the rotation angle of the coordinate center of the ego vehicle in the global coordinate system in the global coordinate system in the historical N frames of images; calculating the lateral displacement and the longitudinal displacement of the ego vehicle coordinate system from the i-th frame of image to the current frame of image ego vehicle coordinate system according to the longitudinal displacement difference, the lateral displacement difference, and the rotation angle of the ego vehicle from the i-th frame of image to the current frame of image; and calculating the position of the target in the global coordinate system in the current frame of image in the vehicle coordinate system according to the rotation angle difference of the ego vehicle from the i-th frame of image to the current frame of image, the lateral displacement and the longitudinal displacement of the ego vehicle coordinate system from the i-th frame of image to the current frame of image ego vehicle coordinate system, wherein i=0, 1, 2, …, N-1. Here, any one of the N positions of the target in the vehicle coordinate system in the current frame of image is denoted as M(x ` i ,y i ` ), then according to the above technical solution, there are:

[0036]

[0037]

[0038] wherein, and are the longitudinal displacement and the lateral displacement of the coordinate center of the ego vehicle in the global coordinate system in the global coordinate system in the historical N frames of images, respectively, i , Y i , are the longitudinal displacement difference, the lateral displacement difference, and the rotation angle difference of the ego vehicle from the i-th frame of image to the current frame of image, wherein:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045] Step S103: obtaining the credibility of the target being perceived by the radar vision device in each frame image.

[0046] Not every target perceived by the radar vision device is a credible target, for example, some objects only accidentally enter the field of view of the radar vision device, and are not targets that the radar vision device needs to track and locate. In theory, under the premise that the ego vehicle speed is constant, the longer the target perceived by the radar vision device exists, the higher the credibility of the target being perceived by the radar vision device. As an embodiment of the present application, the credibility of the target being perceived by the radar vision device in each frame image can be obtained by: obtaining the survival time of the target being perceived by the radar vision device when the ego vehicle speed is a preset value; and calculating the credibility of the target being perceived by the radar vision device in each frame image by interpolation according to the survival time of the target being perceived by the radar vision device when the ego vehicle speed is the preset value. In the above embodiment, the survival time of the target being perceived by the radar vision device when the ego vehicle speed is the preset value is actually obtained by calibration, which can be obtained by collecting test data. Different vehicle speeds correspond to different target existence times, and a credibility two-dimensional table is obtained. Then, the survival time of the target being perceived by the radar vision device when the ego vehicle speed (ego_spd) is the preset value (exist_time_) is obtained by querying the credibility two-dimensional table. Since the credibility two-dimensional table cannot exhaustively list the survival time of the target being perceived by the radar vision device when the ego vehicle is at each speed, the credibility of the target being perceived by the radar vision device in each frame image can be calculated by interpolation according to the survival time of the target being perceived by the radar vision device when the ego vehicle speed is the preset value. Here, the credibility of the target being perceived by the radar vision device in the i-th frame image is denoted as i = 0, 1,..., N-1.

[0047] Step S104: obtaining the motion trajectory of the target by screening the fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame image and the credibility of the target being perceived by the radar vision device in each frame image.

[0048] As an embodiment of the present application, the step S104 of obtaining the motion trajectory of the target by screening the fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame image and the credibility of the target being perceived by the radar vision device in each frame image can be implemented by steps S1041 to S1043, which are described in detail as follows:

[0049] Step S1041: Calculate the center of N positions of the target in the vehicle coordinate system in the current frame image or the clustering center of each historical trajectory.

[0050] In the embodiments of the present application, the motion trajectory of the target is obtained by spline curve fitting, wherein the spline curve can be a straight line or a curve. Different selection of the spline curve can result in different calculation of the center of N positions of the target in the vehicle coordinate system in the current frame image or the clustering center of each historical trajectory. If the spline curve is a straight line, the center of N positions of the target in the vehicle coordinate system in the current frame image is obtained by clustering algorithm. Conversely, if the spline curve is a curve, N positions of the target in the vehicle coordinate system in the current frame image are divided into several historical trajectories according to the time stamp, and the clustering center of each historical trajectory is calculated by clustering algorithm.

[0051] Step S1042: Fit m spline curves according to the center of N positions of the target in the vehicle coordinate system in the current frame image or the clustering center of each historical trajectory.

[0052] Specifically, if the spline curve is a straight line, the turning angles corresponding to the N positions of the target in the vehicle coordinate system in the current frame image can be sampled, i.e. 360° is divided into m equal parts, and m straight lines passing through the center of N positions of the target in the vehicle coordinate system in the current frame image are drawn. If the spline curve is a curve, for the case that the spline curve is an n-th order polynomial (n<N): N positions of the target in the vehicle coordinate system in the current frame image are divided into n+1 segments according to the time stamp, the first segment is ((x`0,y`0),...,((x`p-1,y`p-1)), the i-th segment is [((x`p,y`p),...,((x`p+n,y`p+n)], and p=[N / (n+1)]+1, where [] is the rounding operation. A clustering center Qi is obtained for each historical trajectory, and the n-th order polynomial curve parameters of the spline curve are obtained by n+1 clustering centers Qi: p p ip ip ip+p ip+p

[0053] a=[a0a1...a n ] T

[0054] Step S1043: Obtain the cost function of N positions of the target in the vehicle coordinate system in the current frame image corresponding to the m spline curves, and select the spline curve corresponding to the minimum cost value from the m spline curves as the motion trajectory of the target.

[0055] ​​​​​​Specifically, the implementation of step S1043 can be: obtaining N normal vectors on the spline curve according to the N positions of the target in the vehicle coordinate system in the current frame image; projecting the N normal vectors to the horizontal axis and the vertical axis of the vehicle coordinate system respectively to obtain N first projection quantities and N second projection quantities respectively; calculating the sum of the product of the reliability of the target being perceived by the radar in each frame image and the N normal vectors and the first projection quantities and the second projection quantities to obtain N corresponding cost values; comparing the N cost values, and taking the spline curve corresponding to the minimum cost value in the N cost values as the motion trajectory of the target. Here, any one of the N normal vectors on the spline curve is denoted as (X, Y), the first projection quantity obtained by projecting the normal vector (X, Y) corresponding to the i-th position to the horizontal axis of the vehicle coordinate system is denoted as the second projection quantity obtained by projecting the normal vector (X, Y) corresponding to the i-th position to the vertical axis of the vehicle coordinate system is denoted as the reliability of the target being perceived by the radar in the i-th frame image the product of the normal vector (X, Y) and the first projection quantity and the second projection quantity is denoted as Line_cost_i

[0056] After obtaining the motion trajectory of the target through the above steps S101 to S104, the state parameters such as the velocity and acceleration of the target in the horizontal axis direction and the vertical axis direction in the global coordinate system can also be obtained by performing linear fitting on the motion equations of the target in the horizontal axis direction and the vertical axis direction, that is, the method of the above embodiment further includes: fitting the motion trajectory of the target in displacement and time to obtain the velocity of the target in the horizontal axis direction and the vertical axis direction in the global coordinate system; and fitting the motion trajectory of the target in velocity and time to obtain the acceleration of the target in the horizontal axis direction and the vertical axis direction in the global coordinate system. Assuming that the target trajectory is represented as

[0057] T i (x ` i ,y i ` ,t i ), i = 0, 1,..., N-1

[0058] then the data (t i , x i ) can be used for fitting to obtain the velocity of the target in the horizontal axis direction in the global coordinate system the data A straight line fitting is performed, that is, the parameters corresponding to the foot of the perpendicular of the trajectory points are calculated in time sequence as t, A least square straight line fitting is performed on the parameters t, and the fitting slope is taken as the rate of change of t to obtain the acceleration of the target in the horizontal axis direction of the global coordinate system Similarly, the velocity of the target in the vertical axis direction of the global coordinate system can be fitted and the acceleration

[0059] From the above Figure 1 It can be known from the trajectory prediction method of the target in the example that, compared with the trajectory prediction method based on Kalman filtering provided in the related art, the technical solution has higher requirements on sensor hardware, and the technical solution of the present application does not need to be specially changed on the hardware such as a radar vision device, and only needs to obtain the position of the target in the global coordinate system and the pose of the ego vehicle in the historical N frames of images through the radar vision device and obtain the credibility of the target being perceived by the radar vision device in each frame of image, and subsequently, according to the algorithm, the N positions of the target in the vehicle coordinate system in the current frame of image and the credibility of the target being perceived by the radar vision device in each frame of image, the motion trajectory of the target is obtained by screening the fitted spline curve. Therefore, compared with the trajectory prediction method based on deep learning which needs a large number of training sample sets and has low algorithm efficiency, the technical solution of the present application can accurately predict the trajectory of the target efficiently without improving the performance of the sensor hardware.

[0060] Referring to Figure 2 is a structural schematic diagram of a trajectory prediction device for a target according to an embodiment of the present application. Only parts related to the embodiments of the present application are shown for ease of description. Figure 2 The trajectory prediction device for a target in the example mainly comprises a first obtaining module 201, a conversion module 202, a second obtaining module 203 and a fitting module 204, wherein:

[0061] The first obtaining module 201 is configured to obtain the position of the target in the global coordinate system and the pose of the ego vehicle in the historical N frames of images through the radar vision device;

[0062] The conversion module 202 is configured to convert the position of the target in the global coordinate system in each frame of image of the historical N frames of images into N positions in the vehicle coordinate system in the current frame of image according to the position of the target in the global coordinate system and the pose of the ego vehicle in the historical N frames of images, wherein the vehicle coordinate system is the vehicle coordinate system in which the ego vehicle is located;

[0063] The second obtaining module 203 is configured to obtain the credibility of the target being perceived by the radar vision device in each frame of image;

[0064] The fitting module 204 is configured to obtain the motion trajectory of the target by screening the fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame image and the credibility of the target being perceived by the radar and vision device in each frame image.

[0065] As to the apparatus in the above-mentioned embodiments, the specific manners in which the respective modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0066] From the above Figure 2 It can be known from the trajectory prediction apparatus of the example target that, compared with the Kalman filtering-based trajectory prediction method provided by the related art, the technical solution has higher requirements on sensor hardware, and the technical solution of the present application does not need to be specially changed on the radar and vision device and the like, and only needs to obtain the position of the target in a global coordinate system and the pose of the ego vehicle in historical N frame images and the credibility of the target being perceived by the radar and vision device in each frame image, and subsequently obtains the motion trajectory of the target by screening the fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame image and the credibility of the target being perceived by the radar and vision device in each frame image. Therefore, compared with the deep learning-based trajectory prediction method which needs a large number of training sample sets and has low algorithm efficiency, the technical solution of the present application can accurately predict the trajectory of the target efficiently without improving the performance of the sensor hardware.

[0067] Figure 3 FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application.

[0068] Referring to Figure 3 The electronic device 300 includes a memory 310 and a processor 320.

[0069] The processor 320 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0070] The memory 310 can include various types of storage units, such as a system memory, a read-only memory (ROM), and a permanent storage device. Among them, the ROM can store static data or instructions required by the processor 320 or other modules of the computer. The permanent storage device can be a read-write storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, a flash memory) as a permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (such as a floppy disk, an optical drive). The system memory can be a read-write storage device or a volatile read-write storage device, such as a dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during runtime. In addition, the memory 310 can include a combination of any computer readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 310 can include a read and / or write removable storage device, such as a compact disc (CD), a read-only digital versatile disc (such as DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a min SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer readable storage medium does not include a carrier wave and an instantaneous electronic signal transmitted by wireless or wired transmission.

[0071] The memory 310 stores executable code, which, when processed by the processor 320, can cause the processor 320 to perform part or all of the above-mentioned methods.

[0072] In addition, the method according to the present application can also be implemented as a computer program or computer program product, which includes computer program code instructions for executing part or all of the steps of the above-mentioned methods of the present application.

[0073] Alternatively, the present application can also be implemented as a computer readable storage medium (or non-transitory machine readable storage medium or machine readable storage medium) having executable code (or computer program or computer instruction code) stored thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to execute part or all of the steps of the above-mentioned methods according to the present application.

[0074] Having described various embodiments of the application, it is to be understood that the above description is meant to be illustrative only, and that many modifications and variations of the embodiments described herein are possible. It is therefore to be understood that within the scope of the appended claims, and their equivalents, many alternatives to the embodiments described herein are possible. The selection of terms to be used in the description is not intended to limit the scope of the embodiments described herein, but rather to best explain the principles of the embodiments, practical application, or improvement over the technology in the art, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A trajectory prediction method of a target, characterized by, The method comprises: acquiring the position of the target and the pose of the ego vehicle in the historical N frames of images in a global coordinate system through a radar and vision device; converting the position of the target in each frame of image of the historical N frames of images in the global coordinate system into N positions of the target in the vehicle coordinate system in the current frame of image according to the position of the target in the global coordinate system and the pose of the ego vehicle in the global coordinate system in the historical N frames of images, wherein the vehicle coordinate system is the vehicle coordinate system in which the ego vehicle is located; acquiring the credibility of the target being perceived by the radar and vision device in each frame of image; obtaining the motion trajectory of the target by screening the fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame of image and the credibility of the target being perceived by the radar and vision device in each frame of image.

2. The trajectory prediction method of an object according to claim 1, wherein, The conversion of the position of the target in each frame of image of the historical N frames of images in the global coordinate system into N positions of the target in the vehicle coordinate system in the current frame of image according to the position of the target in the global coordinate system and the pose of the ego vehicle in the global coordinate system in the historical N frames of images comprises: calculating the longitudinal displacement difference, the lateral displacement difference and the rotation angle difference of the ego vehicle from the i-th frame of image to the current frame of image according to the longitudinal displacement, the lateral displacement and the rotation angle of the coordinate center of the ego vehicle in the global coordinate system in the global coordinate system in the historical N frames of images, wherein i=0, 1, 2, …, N-1; calculating the lateral displacement and the longitudinal displacement of the ego vehicle coordinate system of the i-th frame of image to the ego vehicle coordinate system of the current frame of image according to the longitudinal displacement difference, the lateral displacement difference and the rotation angle of the ego vehicle from the i-th frame of image to the current frame of image; calculating the position of the target in the vehicle coordinate system in the current frame of image in the global coordinate system according to the rotation angle difference of the ego vehicle from the i-th frame of image to the current frame of image, the lateral displacement and the longitudinal displacement of the ego vehicle coordinate system of the i-th frame of image to the ego vehicle coordinate system of the current frame of image.

3. The trajectory prediction method of an object according to claim 1, wherein, The acquisition of the credibility of the target being perceived by the radar and vision device in each frame of image comprises: acquiring the retention time of the target being perceived by the radar and vision device when the speed of the ego vehicle is a preset value; calculating the credibility of the target being perceived by the radar and vision device in each frame of image by interpolation according to the retention time of the target being perceived by the radar and vision device when the speed of the ego vehicle is the preset value.

4. The trajectory prediction method of an object according to claim 1, wherein, The obtaining of the motion trajectory of the target by screening the fitted spline curve according to the position of the target in the vehicle coordinate system in the current frame of image and the credibility of the target being perceived by the radar and vision device in each frame of image comprises: calculating the center of the N positions of the target in the vehicle coordinate system in the current frame of image or the clustering center of each historical trajectory; fitting to obtain m spline curves according to the center of the N positions of the target in the vehicle coordinate system in the current frame of image or the clustering center of each historical trajectory; calculating the cost function of the N positions of the target in the vehicle coordinate system in the current frame of image corresponding to the m spline curves, and screening the spline curve corresponding to the minimum cost from the m spline curves as the motion trajectory of the target.

5. The trajectory prediction method of an object according to claim 4, wherein, The calculation of the center of the N positions of the target in the vehicle coordinate system in the current frame of image or the clustering center of each historical trajectory comprises: If the spline curve is a straight line, a clustering algorithm is used to obtain the center of the N positions of the target in the vehicle coordinate system in the current frame image; If the spline curve is a curve, the N positions of the target in the vehicle coordinate system in the current frame image are divided into several historical trajectories according to the time stamp, and a clustering algorithm is used to calculate the clustering center of each historical trajectory.

6. The trajectory prediction method of an object according to claim 4, wherein, The method for obtaining the cost function of the N positions of the target in the vehicle coordinate system in the current frame image corresponding to the m spline curves, and screening out the spline curve corresponding to the minimum cost value from the m spline curves as the motion trajectory of the target, comprises: According to the N positions of the target in the vehicle coordinate system in the current frame image, N normal vectors on the spline curve are obtained; The N normal vectors are projected onto the horizontal and vertical axes of the vehicle coordinate system respectively to obtain N first projection quantities and second projection quantities respectively; The sum of the product of the reliability of the target being perceived by the radar vision device in each frame image and the N normal vectors and the first projection quantities and the second projection quantities is calculated to obtain N corresponding cost values; The N cost values are compared, and the spline curve corresponding to the minimum cost value in the N cost values is taken as the motion trajectory of the target.

7. The trajectory prediction method of an object according to claim 1, wherein, The method further comprises: The motion trajectory of the target is fitted in displacement and time to obtain the velocity of the target in the horizontal and vertical axes of the global coordinate system; and The motion trajectory of the target is fitted in velocity and time to obtain the acceleration of the target in the horizontal and vertical axes of the global coordinate system.

8. A trajectory prediction device of a target, characterized by, The device comprises: A first obtaining module configured to obtain the position of the target and the pose of the ego vehicle in the global coordinate system in N historical frames of images by a radar vision device; A conversion module configured to convert the position of the target in the global coordinate system in each frame of image of the N historical frames of images into N positions in the vehicle coordinate system in the current frame of image according to the position of the target in the global coordinate system and the pose of the ego vehicle in the N historical frames of images, the vehicle coordinate system being the vehicle coordinate system in which the ego vehicle is located; A second obtaining module configured to obtain the reliability of the target being perceived by the radar vision device in each frame of image; A fitting module configured to obtain the motion trajectory of the target by screening the fitted spline curve according to the N positions of the target in the vehicle coordinate system in the current frame of image and the reliability of the target being perceived by the radar vision device in each frame of image.

9. An electronic device, comprising: comprise: a processor; and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method of any one of claims 1 to 7. ​

Images