A vehicle pose estimation method and system based on monocular high-position camera picture acquisition

Abstract

The application discloses a vehicle posture estimation method and system based on monocular high-position camera collected pictures, relates to the field of intelligent traffic management, and comprises the following steps: generating a to-be-trained sample picture dataset and label information corresponding to the to-be-trained sample picture dataset through video pictures collected by fixed-position cameras arranged at various positions in a city, and further generating a single target prediction model; in combination with an existing mature target detection model, posture information of a target vehicle in a picture collected by a monocular high-position camera can be estimated, and two-dimensional projection coordinates of the posture information and a directed bounding box of the target are further analyzed, so that the estimation of the vehicle posture in a high-position fixed camera scene is realized, richer vehicle structure information can be provided for reference information of subsequent complex tasks, and the manual labeling cost and complexity in a special application scene are greatly reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent traffic management, in particular to a vehicle pose estimation method and system based on monocular high-position camera collected pictures. BACKGROUND

[0002] Vehicle pose estimation in the field of intelligent traffic is a basic and important visual task, which plays a crucial role in judging the potential behavior intention of vehicles. With the development of deep learning technology, it is possible to quickly and accurately judge the vehicle pose according to the target picture features, but the difficulty in obtaining three-dimensional labels greatly limits the further improvement of algorithm performance.

[0003] Currently, the algorithms for vehicle pose estimation mainly focus on the heading angle estimation of target vehicles in the vehicle-mounted scene. After years of technical accumulation, there are various data sets with three-dimensional labels in the autonomous driving scene that can be used for training, which also provides favorable data guarantee for actual model training. However, in the high-position fixed camera scene, there are many problems such as camera pose diversity, lack of open-source labeled data sets with similar perspectives, and difficulty in obtaining three-dimensional labels in actual scenes, which greatly limits the possibility of using deep learning technology in this field. Therefore, how to create a sample data set for training, establish a pose estimation model training method in a general high-position video scene, and thus analyze the pose structure information of the target vehicle is one of the problems to be solved in commercial application scenarios. SUMMARY

[0004] To solve the above technical problems, the present application provides a vehicle pose estimation method and system based on monocular high-position camera collected pictures, which can solve the problem of the absence of vehicle pose estimation means in the high-position fixed camera scene.

[0005] To achieve the above purpose, on the one hand, the present application provides a vehicle pose estimation method based on monocular high-position camera collected pictures, which comprises:

[0006] obtaining a training sample picture data set and label information corresponding to the training sample picture data set;

[0007] generating a single target prediction model according to the generated data feature information of the training sample picture data set;

[0008] cropping the target detection frame in the picture collected by the monocular high-position camera according to the single target prediction model, then scaling the cropped picture to the same size as the training sample picture data, and inputting the scaled picture into the single target prediction model to obtain a prediction result;

[0009] According to the annotation information, the prediction result is analyzed to obtain the attitude information of the target and the two-dimensional projection coordinates of the directed bounding box.

[0010] Further, the step of obtaining the training sample picture dataset and the annotation information corresponding to the training sample picture dataset comprises:

[0011] According to a preset vehicle three-dimensional simulation model, simulation picture data similar to the monocular high-position camera perspective, size information of a stereoscopic directed bounding box corresponding to each simulation target in the simulation picture data, and two-dimensional plane projection vertex coordinate information thereof are generated.

[0012] According to the simulation picture data, the size information of the stereoscopic directed bounding box corresponding to each simulation target in the simulation picture data, and the two-dimensional plane projection vertex coordinate information thereof, the training sample picture dataset and the annotation information corresponding to the training sample picture dataset are generated.

[0013] Further, the step of generating a single target prediction model according to the generated data feature information of the training sample picture dataset comprises:

[0014] According to the generated data feature information of the training sample picture dataset, a robust loss function and a relative error loss function are selected.

[0015] According to the robust loss function and the relative error loss function, a single target prediction model is generated.

[0016] Further, the step of analyzing the prediction result according to the annotation information to obtain the attitude information of the target and the two-dimensional projection coordinates of the directed bounding box comprises:

[0017] According to the correlation transformation relationship between the relative size information, the local offset information, the transformation rotation vector in the annotation information and the attitude information of the target in the target detection box, the prediction result is analyzed to obtain the attitude information of the target and the two-dimensional projection coordinates of the directed bounding box.

[0018] Further, the method further comprises: updating the training sample picture dataset from the analysis result by screening the target whose directed bounding box projection meets the preset condition.

[0019] On the other hand, the application provides a vehicle attitude estimation system based on monocular high-position camera collected pictures, the system comprising: an obtaining unit for obtaining a training sample picture dataset and annotation information corresponding to the training sample picture dataset;

[0020] The generating unit is configured to generate a single target prediction model according to the generated data feature information of the to-be-trained sample picture data set.

[0021] The obtaining unit is further configured to crop a target detection frame in a picture collected by a monocular high-position camera, perform scaling on the cropped target detection frame to make the size of the target detection frame consistent with the size of the to-be-trained sample picture data, and input the scaled picture into the single target prediction model to obtain a prediction result.

[0022] The analyzing unit is configured to analyze the prediction result according to the annotation information, and obtain the pose information of the target and the two-dimensional projection coordinates of the directed bounding box.

[0023] Further, the obtaining unit is specifically configured to generate simulated picture data similar to the view angle of the monocular high-position camera according to a preset vehicle three-dimensional simulation model, size information of a stereoscopic directed bounding box corresponding to each simulated target in the simulated picture data, and two-dimensional plane projection vertex coordinate information of the stereoscopic directed bounding box, generate the to-be-trained sample picture data set and the annotation information corresponding to the to-be-trained sample picture data set according to the simulated picture data, the size information of the stereoscopic directed bounding box corresponding to each simulated target in the simulated picture data, and the two-dimensional plane projection vertex coordinate information of the stereoscopic directed bounding box.

[0024] Further, the generating unit is specifically configured to select a robust loss function and a relative error loss function according to the generated data feature information of the to-be-trained sample picture data set, and generate a single target prediction model according to the robust loss function and the relative error loss function.

[0025] Further, the analyzing unit is specifically configured to analyze the prediction result according to a correlation transformation relationship between the relative size information, the local offset information, the transformation rotation vector in the annotation information, and the pose information of the target in the target detection frame, and obtain the pose information of the target and the two-dimensional projection coordinates of the directed bounding box.

[0026] Further, the system further comprises:

[0027] The updating unit is configured to update the to-be-trained sample picture data set by screening targets whose projection of the directed bounding box meets a preset condition from the analysis result.

[0028] The application provides a vehicle posture estimation method and system based on monocular high-position camera collected pictures. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of a vehicle posture estimation method based on monocular high-position camera collected pictures provided by the application;

[0030] Figure 2 is a structural schematic diagram of a vehicle posture estimation system based on monocular high-position camera collected pictures provided by the application. DETAILED DESCRIPTION

[0031] The technical solutions of the application will be further described in detail below with reference to the drawings and examples.

[0032] A vehicle posture estimation method based on monocular high-position camera collected pictures provided by the embodiment of the application, as shown in Figure 1 includes the following steps:

[0033] 101. Obtain a to-be-trained sample picture dataset and annotation information corresponding to the to-be-trained sample picture dataset.

[0034] For the embodiment of the application, step 101 can specifically include: generating simulated picture data similar to the monocular high-position camera perspective, size information of a stereoscopic directional bounding box corresponding to each simulated target in the simulated picture data, and two-dimensional plane projection vertex coordinate information of the simulated picture data according to a preset vehicle three-dimensional simulation model; generating the to-be-trained sample picture dataset and the annotation information corresponding to the to-be-trained sample picture dataset according to the simulated picture data, the size information of the stereoscopic directional bounding box corresponding to each simulated target in the simulated picture data, and the two-dimensional plane projection vertex coordinate information.

[0035] Specifically, for example, step 101.1: using self-developed vehicle three-dimensional simulation software, generate simulated picture data with similar view angle to the actual scene, size information size0=(wo,lo,ho) of the three-dimensional oriented bounding box corresponding to each simulated target in the scene, and two-dimensional plane projection vertex coordinate information Kp80, which is used as the initial basic scene simulation data set D1;

[0036] Step 101.2: for data set D1, calculate the minimum circumscribed rectangle according to Kp80, and take it as the approximate target detection frame (x1, y1, w, h), and the region center point is c, carry out region picture cutting to obtain the patch data set of the target, and according to the maximum side size s0=max(h, w) of the patch picture, rescale it to a fixed size s, the rescaling ratio is rsa=s / s0, obtain the size-regularized picture data set D2, wherein the part of the original picture beyond the boundary has no effective area and is directly filled with (0, 0, 0). At the same time, the corresponding target oriented bounding box projection coordinate point group Kp80 is also subjected to corresponding translation and scaling: Kp8=(Kp80-(x1, y1))*rsa; the target size information is rescaled and converted using the long side l: size=(wr,1,hr)=(wo / lo,1,ho / lo). Finally, the conversion picture data set D2 corresponding to the label information projection coordinates Kp8 and the target relative size size are obtained, and the bounding box coordinates in the world coordinate system of the virtual parallel horizontal plane can be calculated using size:

[0037] Step 101.3: using PnP algorithm, calculating the pose information of the target through two-dimensional Kp8-three-dimensional P wrd point pair, including rotation vector rot and local translation vector trans0, further converting the rotation vector rot=(r1, r2, r3) into complex spherical coordinate svar=(s1, s2, s3) representation method for subsequent calculation and analysis, the specific calculation method is as follows:

[0038]

[0039] Step 101.4: Use the transformed attitude angle data svar to screen for structurally diverse samples in dataset D2. The specific steps are as follows: Take the svar data as the feature vector corresponding to the picture, use the clustering algorithm to obtain the clustering cluster label labeli, (i = 1,.., cn) of the picture, and take the sample with the highest cosine similarity to the in-cluster average feature vector svar_even_labeli as the in-cluster sample example picture img_labeli, and save the in-cluster average feature vector svar_even_labeli and the number of in-cluster samples ni together with the clustering result data and the sample example picture; According to the clustering result and the preset total number of training sets N, calculate the expected number mi = N / cn in each clustering cluster: If ni >> mi, resample the existing data to mi; If ni << mi, according to the in-cluster sample example picture img_labeli and the in-cluster average attitude angle information svar_even_labeli, regenerate the simulated data according to the above steps until the deviation of the data volume ni in each generated clustering cluster does not exceed the threshold (such as 10% mi).

[0040] Step 101.5: Convert the local translation vector trans0 into the form of the offset and depth combination of the physical center point plane. The specific calculation method is as follows: Calculate the corresponding rotation matrix R according to the already solved rotation vector rot, calculate the representation trans = R * trans0 = (t1, t2, d) of trans0 in the camera coordinate system, and obtain the projection coordinate tra = (t1 / d, t2 / d) on the two-dimensional plane; Use the center point coordinate c of the acquisition area to calculate the center offset coordinate of the projection coordinate point as dtr = tra - c = (dtr1, dtr2); Combine the final local physical center point offset vector as tr = (dtr1, dtr2, d).

[0041] Step 101.6: Due to the particularity of the scene used by this model, the targets are all vehicles. Therefore, according to the statistical information, there is a certain value range for the relative width / height in size = (wr, 1, hr); In actual calculation, the relative size anchor vector size_anchor (such as [0.4, 0.4]) can be set, and then calculate the relative deviation value Ddim of wr\hr, and the estimation deviation range [-s, s] of Ddim can be limited by function transformation, such as f = s * tanh.

[0042] 102. Generate a single target prediction model according to the generated data feature information of the待训练样本图片数据集.

[0043] For the embodiment of the present application, step 102 can specifically include: selecting a robust loss function and a relative error loss function according to the generated data feature information of the to-be-trained sample picture data set; and generating a single target prediction model according to the robust loss function and the relative error loss function.

[0044] For example, based on the generated data features described above, a deep learning model for predicting a single target relative size svar, a local physical center point offset tr, and a target pose svar is designed, which specifically includes the following three aspects: first, pose estimation deep learning model structure design: the model result mainly includes a picture feature extraction backbone, a local center point displacement prediction head, and a pose size prediction head, and the direct prediction of the three parts is given as Ddim, tr, and svar;

[0045] Second, multi-target learning loss design: because the regression difficulty and prediction value range of the three groups of variables are inconsistent, different loss function forms are adopted, for example, the following forms can be used: the relative width and height size deviation loss adopts Huber loss, loss_dim = Huber_loss; the relative center point two-dimensional plane offset (dtr1, dtr2) adopts Huber loss, loss_dxy = Huber_loss; the depth estimation loss adopts a relative error L2-log loss, the rotation vector svar adopts a relative error L2-log loss, In summary, the final target loss function can be expressed in the following form, and the specific weighting parameters w i The weighting parameters w can be pre-set in an empirical manner or dynamically adjusted in a multi-task learning manner:

[0046] loss = w1loss_dim + w2loss_dxy + w3loss_d + w4loss_r, training data enhancement scheme: in order to improve the stability, accuracy and generalization of the final acquired training model for predicting the actual input data, reasonable data enhancement operation needs to be performed on the input data in actual training, which is mainly divided into three categories: the first category is the data enhancement of the color space of the picture, which is suitable for the color diversity and random interference of the actual data, which can include: random brightness / contrast / saturation transformation, random picture graying, random increase of Gaussian / pepper noise, random Gaussian blur, random local small area erasing, etc. The second category is background replacement data enhancement, which is suitable for the possibility of abnormal texture interference in the surrounding area in actual situation: screening color pictures with large scene difference, randomly cutting off the edge local area of the input picture patch, pasting to the randomly extracted background picture, and performing original input picture size area cutting to obtain the enhanced picture. The third category is target geometric structure transformation enhancement, which is suitable for the diversity of actual data structure and the randomness of detection result, which includes: random two-dimensional plane picture rotation, random horizontal flip, random picture center point translation cutting, random picture size scaling transformation, etc. It should be noted that the corresponding rotation variable svar and translation variable tr need to be adjusted after the geometric structure transformation, such as picture random scaling size s2, the depth d needs to be adjusted to d'=d / s2; random picture center point offset, dtr1 and dtr2 need to be offset by the same amount.

[0047] 10 3、According to the single target prediction model, the target detection frame in the picture collected by the monocular high camera is cut and scaled to the same size as the training sample picture data, and the scaled picture is input into the single target prediction model to obtain the prediction result.

[0048] 104、According to the annotation information, the prediction result is analyzed to obtain the pose information of the target and the two-dimensional projection coordinates of the directed bounding box.

[0049] For the embodiment of the application, step 104 can specifically include: according to the correlation transformation relationship between the relative size information, local offset information, transformation rotation vector in the annotation information and the pose information of the target in the target detection frame, the prediction result is analyzed to obtain the pose information of the target and the two-dimensional projection coordinates of the directed bounding box.

[0050] Specifically, for example, using the trained prediction model, the target detection frame obtained by the pre-target detection model for the actual online data is cropped to the target area, and then scaled to the same size as the training data, with the center point being c and the scaling ratio being s3; the scaled picture is input into the trained prediction model to obtain a prediction result, and the correlation conversion relationship between the direct prediction quantity Ddim, tr, svar designed in step 101 and the actual pose information size, trans, rot is used to analyze the model result, to obtain the final required pose information and two-dimensional projection coordinates of the directed bounding box, and the specific calculation method is as follows: (1)

[0051] size = [size_anchor[1] + Ddim[1], 1, size_anchor[2] + Ddim[2]]; (2)

[0052] trans = [(tr[1] + c) * tr[3], tr[2] + c) * tr[3], tr[3]]; (3) rot: calculated according to eq2 in step 1.3; (4) kp8: calculate the rotation matrix R through rot, and then further obtain the vertex kp8_cam of the directed bounding box in the camera coordinate system = R * Pwrd + trans, and project it to the two-dimensional plane kp8_img = [kp8_cam[1] / kp8_cam[3], kp8_cam[2] / kp8_cam[3]]; (5) vehicle heading angle yaw estimation: assuming that the rotation matrix composed of the known camera pose angle is R cam , the rotation matrix calculated through the heading angle is R yaw , and the relationship between the estimated rotation matrix R and the two can be represented as R = R cam R yaw , and the corresponding heading angle rotation matrix can be represented as R yaw = R c T am , and finally the heading angle yaw is calculated through the optimal matrix approximation method; (6) according to the scaling ratio s3 of the original picture and the coordinate position of the cropped region, the estimated depth d and the projection point kp8 can be scaled and restored to the original picture for representation.

[0053] Further, in order to further improve the fine degree of the to-be-trained sample picture data set, the embodiment of the present application can further include: updating the to-be-trained sample picture data set from the analysis result of the projection of the directed bounding box meeting the preset condition.

[0054] The application provides a vehicle posture estimation method based on monocular high-position camera collected pictures, generates a to-be-trained sample picture dataset and label information corresponding to the to-be-trained sample picture dataset through video pictures collected by fixed-position cameras arranged at various positions in a city, and further generates a single target prediction model, so that the posture information of a target vehicle in a picture collected by a monocular high-position camera can be estimated in combination with an existing mature target detection model, and the two-dimensional projection coordinates of the target posture information and a directed bounding box are further analyzed, so that the vehicle posture estimation in a high-position fixed camera scene is realized, richer vehicle structure information can be provided for reference information of subsequent complex tasks, and the manual labeling cost and complexity in a special application scene are greatly reduced.

[0055] To implement the method provided by the embodiment of the application, the embodiment of the application provides a vehicle posture estimation system based on monocular high-position camera collected pictures, as shown in the figure, the system comprises an acquisition unit 21, a generation unit 22, an analysis unit 23 and an update unit 24. Figure 2

[0056] The acquisition unit 21 is configured to acquire a to-be-trained sample picture dataset and label information corresponding to the to-be-trained sample picture dataset.

[0057] The generation unit 22 is configured to generate a single target prediction model according to the generated data feature information of the to-be-trained sample picture dataset.

[0058] The acquisition unit 21 is further configured to perform scaling on a target detection frame in a picture collected by a monocular high-position camera after the target detection frame is cropped according to the single target prediction model, so that the size of the picture after scaling is consistent with the size of the to-be-trained sample picture dataset, and input the picture after scaling into the single target prediction model to acquire a prediction result.

[0059] The analysis unit 23 is configured to analyze the prediction result according to the label information to acquire the posture information of the target and the two-dimensional projection coordinates of a directed bounding box.

[0060] Further, the acquisition unit 21 is specifically configured to generate simulated picture data similar to the perspective of a monocular high-position camera, size information of a three-dimensional directed bounding box corresponding to each simulated target in the simulated picture data and two-dimensional plane projection vertex coordinate information of the three-dimensional directed bounding box according to a preset vehicle three-dimensional simulation model, and generate the to-be-trained sample picture dataset and the label information corresponding to the to-be-trained sample picture dataset according to the simulated picture data, the size information of the three-dimensional directed bounding box corresponding to each simulated target in the simulated picture data and the two-dimensional plane projection vertex coordinate information of the three-dimensional directed bounding box.

[0061] ​Further, the generating unit 22 is specifically configured to select a robust loss function and a relative error loss function according to the generated data feature information of the to-be-trained sample picture data set; and generate a single target prediction model according to the robust loss function and the relative error loss function.

[0062] Further, the analyzing unit 23 is specifically configured to analyze the prediction result according to a correlation conversion relationship between relative size information, local offset information and a conversion rotation vector in the labeling information and the pose information of the target in the target detection frame, and obtain the pose information of the target and the two-dimensional projection coordinates of the directed bounding box.

[0063] Further, the system further comprises an updating unit 24 configured to update the to-be-trained sample picture data set from the target whose projection of the directed bounding box meets a preset condition.

[0064] The vehicle pose estimation system based on monocular high-position camera collected pictures provided by the application generates a to-be-trained sample picture data set and labeling information corresponding to the to-be-trained sample picture data set through videos collected by fixed-position cameras arranged at various positions in a city, and further generates a single target prediction model. In combination with an existing mature target detection model, the pose information of a target vehicle in a picture collected by a monocular high-position camera can be estimated, and the pose information of the target and the two-dimensional projection coordinates of the directed bounding box can be further analyzed. The vehicle pose estimation in a high-position fixed camera scene is realized. The vehicle structure information is more abundant, which can be used as reference information for subsequent complex tasks, and the manual labeling cost and complexity in a special application scene are greatly reduced.

[0065] It should be understood that the particular order or hierarchy of steps in processes disclosed is an example. Based upon design preferences, it should be understood that specific order or hierarchy of steps in a process can be re-arranged while remaining within the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.

[0066] In the above detailed description, various features are grouped together in single embodiments for the purpose of streamlining the disclosure. Such disclosed approaches should not be interpreted as reflecting an intention that the claimed subject matter requires more features than are explicitly recited in each claim. On the contrary, as indicated previously, the inventiveness of the claimed subject matter lies in less than all features of the disclosed subject matter. Accordingly, the claims are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate preferred embodiment of the present application.

[0067] The foregoing description of the exemplary embodiment ( s ) of the application will be better understood when read in conjunction with the appended drawings. It should be understood that the Figures are not drawn to scale. Included in the drawings are the following Figures, presented to provide a further understanding of the application and to portray the principles and features of the application in a forewarning manner:

[0068] The foregoing description of the exemplary embodiment ( s ) of the application will be better understood when read in conjunction with the appended drawings. It should be understood that the Figures are not drawn to scale. Included in the drawings are the following Figures, presented to provide a further understanding of the application and to portray the principles and features of the application in a forewarning manner:

[0069] Those of skill would further appreciate that the various illustrative logical blocks, modules, and steps described in connection with the embodiment disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0070] The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. The various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality, without limitation. Depending upon the implementation, the various illustrative blocks, modules, circuits, and steps could be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of them. The software implemented as a computer program, software, or firmware instructions can be stored in any suitable

[0071] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium can be coupled to the processor, such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as distinct components in a user terminal. Additionally, in some embodiments, the medium can comprise computer-readable storage media or communications media.

[0072] In one or more exemplary designs, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media can be any available media that can be accessed by a computer. By way of example, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or other wire-based, fiber- optic based, or wireless based communications, then the coaxial cable, fiber optic cable, twisted pair, DSL, or other wire-based, fiber-optic based, or wireless based communications are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, DVD, floppy disk, and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0073] The specific implementation described above is illustrative for purposes of teaching the present application. The application should not be limited, however, to these specific implementation. Any modifications, equivalents, or improvements, as long as they are within the spirit and scope of the present application are included in the scope of the present application.

Claims

1. A vehicle pose estimation method based on monocular high camera for collecting pictures, characterized in that, The method is applied to a high-position fixed camera scene, and the method comprises the following steps: Obtaining a training sample picture data set and annotation information corresponding to the training sample picture data set; Generating a single target prediction model according to generated data feature information of the training sample picture data set; After the single target prediction model is used for cutting a target detection frame in a picture collected by a monocular high-position camera, the picture is scaled to have a same size as the training sample picture data, and the scaled picture is input into the single target prediction model to obtain a prediction result; According to the annotation information, the prediction result is analyzed to obtain attitude information and two-dimensional projection coordinates of a directed bounding box of the target; The step of obtaining the training sample picture data set and the annotation information corresponding to the training sample picture data set comprises the following steps: A three-dimensional simulation software of a preset vehicle is used to generate simulation picture data similar to an actual scene view angle, size information size0=(wo,lo,ho) of a three-dimensional directed bounding box corresponding to each simulation target in the scene, and two-dimensional plane projection vertex coordinate information Kp80, which are used as initial basic scene simulation data set D1; For the data set D1, a minimum circumscribed rectangle is calculated according to Kp80, and is used as an approximate target detection frame (x1,y1,w,h). The region center point is c. The patch data set of the target is obtained by cutting a region picture, and the patch picture is rescaled to a fixed size s according to the maximum side size s0=max(h,w). The rescaling ratio is rsa=s / s0. The size-regularized picture data set D2 is obtained, wherein the part of the original picture beyond the boundary without an effective region is directly filled with (0,0,0). At the same time, the corresponding target directed bounding box projection coordinate point group Kp80 is also subjected to corresponding translation and scaling change: Kp8=(Kp80-(x1,y1))*rsa; the target size information is rescaled and converted by using the long side l: size=(wr,1,hr)=(wo / lo,1,ho / lo); finally, the converted picture data set D2 corresponding annotation information projection coordinates Kp8 and target relative size size are obtained, and the size can be used to calculate the bounding box coordinates in the world coordinate system of a virtual parallel horizon: By two-dimensional Kp8-three-dimensional P wrd The pose information of the point to the target includes a rotation vector rot and a local translation vector trans0, and the rotation vector rot=(r1, r2, r3) can be further converted into a complex spherical coordinate svar=(s1, s2, s3). The structure diversity sample screening of the data set D2 is performed by using the converted pose angle data svar, and the specific steps are as follows: the svar data is used as the feature vector of the corresponding picture, the clustering algorithm is used to obtain the clustering cluster label labeli, i = 1,.., cn of the picture, and the sample example picture img_labeli with the highest cosine similarity with the average feature vector svar_even_labeli in the cluster is taken as the sample example picture img_labeli in the cluster, and the average feature vector svar_even_labeli in the cluster and the number of samples ni in the cluster are taken as the clustering result data and the sample example picture are saved together; according to the clustering result and the total amount of the preset training set N, the expected number of each clustering cluster is calculated as mi = N / cn: if ni >> mi, the existing data is resampled to mi; if ni << mi, according to the sample example picture img_labeli in the cluster and the average pose angle information svar_even_labeli in the cluster, the simulation data is regenerated according to the above steps until the data amount ni in each clustering cluster deviates by not more than a threshold value; According to the rotation vector rot that has been solved, the rotation matrix R corresponding to the rotation vector rot is converted, the trans0 is calculated in the camera coordinate system, the trans = R*trans0 = (t1, t2, d) is calculated, and the projection coordinates tra = (t1 / d, t2 / d) on the two-dimensional plane are obtained; the center offset coordinates dtr of the projection coordinate point are calculated by using the center point coordinates c of the collection area, and the center offset coordinates dtr = tra-c = (dtr 1, dtr2) are calculated; and the final local physical center point offset vector tr = (dtr 1, dtr2, d) is combined.

2. The vehicle pose estimation method based on monocular high camera picture acquisition according to claim 1, characterized in that, The step of generating a single target prediction model according to the generated data feature information of the to-be-trained sample picture data set comprises: Selecting a robust loss function and a relative error loss function according to the generated data feature information of the to-be-trained sample picture data set; Generating a single target prediction model according to the robust loss function and the relative error loss function.

3. The vehicle pose estimation method based on monocular high camera picture acquisition according to claim 1, characterized in that, The step of analyzing the prediction result according to the label information to obtain the pose information of the target and the two-dimensional projection coordinates of the directed bounding box comprises: According to the correlation transformation relationship between the relative size information, the local offset information, the transformed rotation vector and the pose information of the target in the target detection box, the prediction result is analyzed to obtain the pose information of the target and the two-dimensional projection coordinates of the directed bounding box.

4. The vehicle pose estimation method based on monocular high camera picture acquisition according to claim 1, characterized in that, The method further comprises: Updating the to-be-trained sample picture data set by screening the target whose projection of the directed bounding box meets the preset condition from the analysis result.

5. A vehicle pose estimation system based on monocular high camera to capture pictures, characterized in that, Applied to a high fixed camera scene, the system comprises: An acquisition unit is configured to acquire a to-be-trained sample picture data set and label information corresponding to the to-be-trained sample picture data set; A generation unit is configured to generate a single target prediction model according to generated data feature information of the to-be-trained sample picture data set. The acquisition unit is further configured to crop a target detection frame in a picture collected by a monocular high camera according to the single-target prediction model, perform scaling on the cropped target detection frame to make the size of the target detection frame consistent with the size of the to-be-trained sample picture data, and input the scaled picture into the single-target prediction model to obtain a prediction result; The analysis unit is configured to analyze the prediction result according to the labeling information, and obtain pose information and two-dimensional projection coordinates of a directed bounding box of the target; The acquisition unit is specifically configured to generate, by using a preset vehicle three-dimensional simulation software, simulation picture data having a similar view angle to an actual scene, size information size0=(wo, lo, ho) of a three-dimensional directed bounding box corresponding to each simulation target in the scene, and two-dimensional plane projection vertex coordinate information Kp80 of the three-dimensional directed bounding box, and use the simulation picture data, the size information size0=(wo, lo, ho), and the two-dimensional plane projection vertex coordinate information Kp80 as an initial basic scene simulation dataset D1; For the dataset D1, a minimum circumscribed rectangle is calculated according to the Kp80, and is used as an approximate target detection frame (x1, y1, w, h). A region picture is cropped to obtain a patch dataset of the target, and the patch picture is rescaled to a fixed size s according to a maximum side size s0=max(h, w) and a rescaling ratio rsa=s / s0. An effective region part of the original picture beyond the boundary is directly filled with (0, 0, 0) to obtain a size-regularized picture dataset D2. At the same time, the corresponding target directed bounding box projection coordinate point group Kp80 is also subjected to corresponding translation and scaling: Kp8=(Kp80-(x1, y1))*rsa. The target size information is rescaled and converted using a long side l: size=(wr, 1, hr)=(wo / lo, 1, ho / lo). Finally, the converted picture dataset D2 is obtained, and labeling information projection coordinates Kp8 and target relative size size of the converted picture dataset D2 are obtained. The size can be used to calculate the bounding box coordinates in a world coordinate system of a virtual parallel horizon: By two-dimensional Kp8-three-dimensional P wrd The pose information of the point to the target includes a rotation vector rot and a local translation vector trans0. Further, the rotation vector rot=(r1, r2, r3) can be converted into a complex spherical coordinate svar=(s1, s2, s3) representation method. The transformed pose angle data svar is used for structure diversity sample screening of the data set D2, and the specific steps are as follows: the svar data is used as the feature vector of the corresponding picture, the clustering algorithm is used to obtain the clustering cluster label labeli, i=1,..,cn corresponding to the picture, and the sample example picture img_labeli with the highest cosine similarity with the average feature vector svar_even_labeli in the cluster is taken as the sample example picture img_labeli in the cluster, and the average feature vector svar_even_labeli in the cluster and the number of samples ni in the cluster are taken as the clustering result data and the sample example picture are saved together; according to the clustering result and the preset total amount of training set N, the expected number of each clustering cluster is calculated as mi=N / cn: if ni>>mi, the existing data is resampled to mi; if ni<<mi, according to the cluster sample example picture img_labeli and the cluster average pose angle information svar_even_labeli, the simulation data is regenerated according to the above steps until the data amount ni in each cluster deviates by not more than a threshold value; According to the rotation vector rot that has been solved, the rotation matrix R corresponding to the rotation vector rot is converted, the trans0 is calculated in the camera coordinate system, and the trans=R*trans0=(t1,t2,d) is calculated. The projection coordinates tra=(t1 / d,t2 / d) in the two-dimensional plane are obtained. The center offset coordinates dtr of the projection coordinate point are calculated by using the center point coordinates c of the collection area, and the center offset coordinates dtr=(dtr 1,dtr2) are calculated. The final local physical center point offset vector tr=(dtr 1,dtr2,d) is combined.

6. The vehicle pose estimation system based on monocular high-position camera picture acquisition according to claim 5, characterized in that, The generation unit is specifically configured to select a robust loss function and a relative error loss function according to the generated data feature information of the to-be-trained sample picture data set; and generate a single target prediction model according to the robust loss function and the relative error loss function.

7. The vehicle pose estimation system based on monocular high-position camera picture acquisition according to claim 5, characterized in that, The analysis unit is specifically configured to analyze the prediction result according to the correlation transformation relationship between the relative size information, the local offset information, the transformed rotation vector and the pose information of the target in the target detection frame, and obtain the pose information of the target and the two-dimensional projection coordinates of the oriented bounding box. 8.The vehicle pose estimation system based on monocular high camera according to claim 5, wherein, The system further comprises: An updating unit configured to update the to-be-trained sample picture data set from the targets whose projection of the oriented bounding box meets a preset condition according to the analysis result.

Images