Method for marking invisible wheel grounding points and vehicle pose prediction method

Abstract

The present application relates to the technical field of automatic driving, and particularly provides a labeling method of invisible wheel grounding point, a vehicle posture prediction method, a device, a medium and a vehicle, aiming to solve the problem of how to effectively label the invisible wheel grounding point and effectively predict the vehicle posture in the case that only one wheel grounding point is visible. To this end, the present application can label the invisible wheel grounding point according to the principle of perspective projection, and the labeling process is more accurate, easy to operate and low in cost. Moreover, the present application can predict the wheel grounding point connecting angle of the to-be-predicted image with only one visible wheel grounding point according to the trained wheel grounding point connecting angle prediction model, and perform wheel grounding point detection, so as to obtain the posture angle of the vehicle according to the obtained final wheel grounding point connecting angle and final wheel grounding point, which can expand the use scene of the wheel grounding point and effectively improve the accuracy and safety of the automatic driving process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and particularly provides a method for labeling a non-visible wheel contact point, a vehicle pose prediction method and device, a medium and a vehicle. BACKGROUND

[0002] The wheel contact point generally refers to a contact point between a tire of a vehicle and a ground. The wheel contact point, in combination with intrinsic and extrinsic parameters of a camera, can be used to estimate a position and an angle of the vehicle. In the application of the wheel contact point to learning and prediction of a vehicle pose, at least two wheel contact points on the same side of the vehicle are required. However, in the case of a vehicle image truncation scene or a situation where one wheel contact point of the vehicle is blocked, it is difficult to obtain at least two wheel contact points on the same side. In this case, another wheel contact point needs to be labeled and predicted.

[0003] There are mainly two methods for labeling a non-visible wheel contact point in the prior art: one is to label a 3D box, and this labeling method has a high labeling cost and limits the number of labeled samples. The other is to label directly on a 2D image. When a labeler encounters a situation where only one wheel contact point is visible, the labeler has two choices: one is to give up labeling in this case, which greatly limits the use scenarios of the wheel contact point, such as a vehicle image truncation scene which is very concerned in actual application; the other is that the labeler labels the non-visible wheel contact point according to personal experience, which has a large human factor interference and makes it difficult to unify the labeling rules, resulting in a large deviation of the labeling result.

[0004] Meanwhile, in the prediction of a vehicle pose, there is also a situation where a vehicle image is truncated (part of the vehicle is outside the image) and at least two wheel contact points on the same side of the vehicle cannot be effectively obtained, so that the wheel contact point cannot be used for prediction of the vehicle pose. The prediction of the vehicle pose is very important for automatic driving. If the prediction of the vehicle pose is inaccurate, it may lead to misrecognition of other vehicles invading the lane of the vehicle, resulting in the occurrence of situations such as no braking, and affecting the accuracy and safety of automatic driving.

[0005] Correspondingly, there is a need in the field for a new labeling method for a non-visible wheel contact point and a vehicle pose prediction method to solve the above problems. SUMMARY

[0006] In order to overcome the above defects, the present application is proposed to solve or at least partially solve the problem of how to effectively label a non-visible wheel contact point and effectively predict a vehicle pose in the case where only one wheel contact point is visible.

[0007] In a first aspect, the present application provides a method for labeling a non-visible wheel contact point, the method comprising:

[0008] annotating a visible wheel contact point in a to-be-annotated image;

[0009] obtaining an intersection of extension lines of two parallel lines existing in the vehicle itself on the same side of the visible wheel contact point in the to-be-annotated image according to the principle of perspective projection;

[0010] annotating the invisible wheel contact point according to the intersection and the visible wheel contact point.

[0011] In one technical solution of the above-mentioned method for annotating an invisible wheel contact point, the step of "annotating the invisible wheel contact point according to the intersection and the visible wheel contact point" comprises:

[0012] establishing a ray with the intersection as a starting point and pointing to the visible wheel contact point;

[0013] annotating the invisible wheel contact point on the ray according to the length of the vehicle.

[0014] In a second aspect, the present application provides a vehicle attitude prediction method, which comprises:

[0015] performing wheel contact point detection on the to-be-predicted image to obtain a final wheel contact point of the to-be-predicted image;

[0016] applying a trained wheel contact point connecting line angle prediction model to predict the wheel contact point connecting line angle of the to-be-predicted image to obtain a final wheel contact point connecting line angle in the to-be-predicted image;

[0017] obtaining an attitude angle of the vehicle according to the final wheel contact point connecting line angle and the final wheel contact point, wherein the wheel contact point connecting line angle prediction model is trained based on a labeled data set, the labeled data set contains a first labeled image, the first labeled image contains a labeled visible wheel contact point and a labeled invisible wheel contact point, and the invisible wheel contact point is annotated according to the method in the technical solution of the above-mentioned method for annotating an invisible wheel contact point.

[0018] In one technical solution of the above-mentioned vehicle attitude prediction method, the step of "performing wheel contact point detection on the to-be-predicted image to obtain a final wheel contact point of the to-be-predicted image" comprises:

[0019] performing wheel contact point detection on the to-be-predicted image to obtain a wheel contact point in the to-be-predicted image and a prediction probability of the wheel contact point;

[0020] selecting a wheel contact point with the maximum prediction probability as the final wheel contact point.

[0021] In one of the technical solutions of the vehicle posture prediction method, the wheel grounding point connecting line angle includes a vehicle left side connecting line angle and a vehicle right side connecting line angle, and the step of "applying the trained wheel grounding point connecting line angle prediction model to predict the wheel grounding point connecting line angle of the to-be-predicted image to obtain the final wheel grounding point connecting line angle in the to-be-predicted image" includes:

[0022] applying the trained wheel grounding point connecting line angle prediction model to obtain the vehicle left side connecting line angle and the vehicle right side connecting line angle of the to-be-predicted image;

[0023] determining the final wheel grounding point connecting line angle according to the vehicle left side connecting line angle, the vehicle right side connecting line angle, and the position of the final wheel grounding point.

[0024] In one of the technical solutions of the vehicle posture prediction method, the step of "determining the final wheel grounding point connecting line angle according to the vehicle left side connecting line angle, the vehicle right side connecting line angle, and the position of the final wheel grounding point" includes:

[0025] if the final wheel grounding point is located on the left side of the vehicle, the vehicle left side connecting line angle is taken as the final wheel grounding point connecting line angle;

[0026] if the final wheel grounding point is located on the right side of the vehicle, the vehicle right side connecting line angle is taken as the final wheel grounding point connecting line angle.

[0027] In one of the technical solutions of the vehicle posture prediction method, the step of "applying the trained wheel grounding point connecting line angle prediction model to obtain the vehicle left side connecting line angle and the vehicle right side connecting line angle of the to-be-predicted image" includes:

[0028] applying the trained wheel grounding point connecting line angle prediction model to obtain the sine value and the cosine value of the vehicle left side connecting line angle of the to-be-predicted image; and obtaining the vehicle left side connecting line angle according to the sine value and the cosine value;

[0029] applying the trained wheel grounding point connecting line angle prediction model to obtain the sine value and the cosine value of the right side connecting line angle of the to-be-predicted image; and obtaining the vehicle right side connecting line angle according to the sine value and the cosine value.

[0030] In one of the technical solutions of the vehicle posture prediction method, the step of "obtaining the posture angle of the vehicle according to the final wheel grounding point connecting line angle and the final wheel grounding point" includes:

[0031] obtaining the position of a second point on the wheel grounding point connecting line where the final wheel grounding point is located in the to-be-predicted image according to the final wheel grounding point connecting line angle and the final wheel grounding point.

[0032] According to the position of the final wheel contact point and the position of the second point, an attitude angle of the vehicle is obtained.

[0033] In one of the technical solutions of the vehicle attitude prediction method, it is judged whether there is another wheel contact point except the final wheel contact point on the same side of the final wheel contact point as a visible wheel contact point.

[0034] If yes, the visible wheel contact point is taken as the second point.

[0035] If no, the position of the second point is obtained according to the following formula:

[0036]

[0037]

[0038] wherein p x and p y are x-axis and y-axis coordinate values of the second point respectively, are x-axis and y-axis coordinate values of the final wheel contact point respectively, Δ is an arbitrary positive integer, and θ is a connecting angle of the final wheel contact point.

[0039] In one of the technical solutions of the vehicle attitude prediction method, the method further comprises judging whether another wheel contact point is a visible wheel contact point according to the following steps:

[0040] A prediction probability of the other wheel contact point is compared with a preset prediction probability threshold value, and when the prediction probability is greater than the prediction probability threshold value, the other wheel contact point is determined as a visible wheel contact point.

[0041] In a third aspect, a control device is provided, which comprises a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to execute the vehicle attitude prediction method in any one of the technical solutions of the vehicle attitude prediction method.

[0042] In a fourth aspect, a computer readable storage medium is provided, which has a plurality of program codes stored therein, the program codes being adapted to be loaded and run by a processor to execute the vehicle attitude prediction method in any one of the technical solutions of the vehicle attitude prediction method.

[0043] In a fifth aspect, a vehicle is provided, which comprises the control device in the technical solution of the control device.

[0044] The one or more technical solutions of the present application have at least one or more of the following beneficial effects.

[0045] In the implementation of the technical solutions of the present application, the present application can mark the invisible wheel contact point based on the intersection of the extensions of the two parallel lines existing in the vehicle itself in the to-be-labeled image and the visible wheel contact point according to the principle of perspective projection, can accurately label the invisible wheel contact point when there is only one visible wheel contact point in the to-be-labeled image, and the labeling process is easier to operate and has low cost. Further, the present application can predict the wheel contact point connection angle of the to-be-predicted image with only one visible wheel contact point according to the trained wheel contact point connection angle prediction model, and perform wheel contact point detection, and obtain the attitude angle of the vehicle according to the obtained final wheel contact point connection angle and the final wheel contact point. Through the above configuration mode, the present application can effectively predict the attitude angle of the vehicle according to the final wheel contact point connection angle and the final wheel contact point in the vehicle truncation scene in the to-be-predicted image, greatly expands the use scene of the wheel contact point, and can effectively improve the accuracy and safety of the automatic driving process.

[0046] Scheme 1. A method for labeling an invisible wheel contact point, characterized in that the method comprises:

[0047] labeling a visible wheel contact point in a to-be-labeled image;

[0048] obtaining an intersection of extensions of two parallel lines existing in the vehicle itself on the same side of the visible wheel contact point in the to-be-labeled image according to the principle of perspective projection;

[0049] labeling the invisible wheel contact point according to the intersection and the visible wheel contact point.

[0050] Scheme 2. The method for labeling an invisible wheel contact point according to scheme 1, characterized in that the step of "labeling the invisible wheel contact point according to the intersection and the visible wheel contact point" comprises:

[0051] establishing a ray pointing to the visible wheel contact point with the intersection as the starting point;

[0052] labeling the invisible wheel contact point on the ray according to the length of the vehicle.

[0053] Scheme 3. A vehicle attitude prediction method, characterized in that the method comprises:

[0054] performing wheel contact point detection on the to-be-predicted image to obtain the final wheel contact point of the to-be-predicted image;

[0055] applying the trained wheel contact point line angle prediction model to predict the wheel contact point line angle of the to-be-predicted image to obtain a final wheel contact point line angle in the to-be-predicted image;

[0056] obtaining the pose angle of the vehicle according to the final wheel contact point line angle and the final wheel contact point, wherein the wheel contact point line angle prediction model is trained based on a labeled data set, the labeled data set containing a first labeled image, the first labeled image containing a labeled visible wheel contact point and a labeled invisible wheel contact point, the invisible wheel contact point being labeled according to any one of the methods in schemes 1 to 2.

[0057] Scheme 4. The vehicle pose prediction method according to scheme 3, characterized in that the step of "performing wheel contact point detection on the to-be-predicted image to obtain a final wheel contact point of the to-be-predicted image" comprises:

[0058] performing wheel contact point detection on the to-be-predicted image to obtain a wheel contact point in the to-be-predicted image and a prediction probability of the wheel contact point;

[0059] selecting the wheel contact point with the maximum prediction probability as the final wheel contact point.

[0060] Scheme 5. The vehicle pose prediction method according to scheme 4, characterized in that the wheel contact point line angle comprises a vehicle left side line angle and a vehicle right side line angle, and the step of "applying the trained wheel contact point line angle prediction model to predict the wheel contact point line angle of the to-be-predicted image to obtain a final wheel contact point line angle in the to-be-predicted image" comprises:

[0061] applying the trained wheel contact point line angle prediction model to obtain a vehicle left side line angle and a vehicle right side line angle of the to-be-predicted image;

[0062] determining the final wheel contact point line angle according to the vehicle left side line angle, the vehicle right side line angle and the position of the final wheel contact point.

[0063] Scheme 6. The vehicle pose prediction method according to scheme 5, characterized in that the step of "determining the final wheel contact point line angle according to the vehicle left side line angle, the vehicle right side line angle and the position of the final wheel contact point" comprises:

[0064] if the final wheel contact point is located on the left side of the vehicle, taking the vehicle left side line angle as the final wheel contact point line angle;

[0065] If the final wheel grounding point is located on the right side of the vehicle, the right side of the vehicle is connected to the angle as the final wheel grounding point connecting angle.

[0066] Scheme 7. The vehicle posture prediction method according to scheme 5, characterized in that the step of "applying the trained wheel grounding point connecting angle prediction model to obtain the vehicle left side connecting angle and the vehicle right side connecting angle of the image to be predicted" comprises:

[0067] applying the trained wheel grounding point connecting angle prediction model to obtain the sine value and the cosine value of the vehicle left side connecting angle of the image to be predicted; and obtaining the vehicle left side connecting angle according to the sine value and the cosine value;

[0068] applying the trained wheel grounding point connecting angle prediction model to obtain the sine value and the cosine value of the right side connecting angle of the image to be predicted; and obtaining the vehicle right side connecting angle according to the sine value and the cosine value.

[0069] Scheme 8. The vehicle posture prediction method according to scheme 5, characterized in that the step of "obtaining the posture angle of the vehicle according to the final wheel grounding point connecting angle and the final wheel grounding point" comprises:

[0070] obtaining the position of a second point on the wheel grounding point connecting line where the final wheel grounding point is located in the image to be predicted according to the final wheel grounding point connecting angle and the final wheel grounding point;

[0071] obtaining the posture angle of the vehicle according to the position of the final wheel grounding point and the position of the second point.

[0072] Scheme 9. The vehicle posture prediction method according to scheme 8, characterized in that the step of "obtaining the position of a second point on the wheel grounding point connecting line where the final wheel grounding point is located in the image to be predicted according to the final wheel grounding point connecting angle and the final wheel grounding point" comprises:

[0073] determining whether there is another wheel grounding point on the same side of the final wheel grounding point as the visible wheel grounding point except the final wheel grounding point;

[0074] If yes, the visible wheel grounding point is taken as the second point;

[0075] If no, the position of the second point is obtained according to the following formula:

[0076]

[0077]

[0078] wherein, px , p y are the x-axis and y-axis coordinate values of the second point, respectively, are the x-axis and y-axis coordinate values of the final wheel contact point, respectively, Δ is any positive integer, and θ is the connecting angle of the final wheel contact point.

[0079] Scheme 10. The vehicle pose prediction method according to scheme 9, characterized in that the method further comprises judging whether the other wheel contact point is a visible wheel contact point according to the following steps:

[0080] comparing the prediction probability of the other wheel contact point with a preset prediction probability threshold value, and determining that the other wheel contact point is a visible wheel contact point when the prediction probability is greater than the prediction probability threshold value.

[0081] Scheme 11. A control device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by the processor to execute the vehicle pose prediction method according to any one of schemes 3 to 10.

[0082] Scheme 12. A computer readable storage medium, wherein a plurality of program codes are stored, characterized in that the program codes are adapted to be loaded and run by a processor to execute the vehicle pose prediction method according to any one of schemes 3 to 10.

[0083] Scheme 13. A vehicle, characterized in that the vehicle comprises the control device according to scheme 11. BRIEF DESCRIPTION OF DRAWINGS

[0084] The disclosure of the present application will become more apparent from the following description in conjunction with the accompanying drawings. It is readily understood by those skilled in the art that the drawings are merely intended to illustrate the present application and are not intended to limit the scope of protection of the present application. Among them:

[0085] Figure 1 is a schematic diagram of the main steps of the labeling method of the invisible wheel contact point according to an embodiment of the present application;

[0086] Figure 2 is a schematic diagram of the main steps of the labeling method of the invisible wheel contact point according to an embodiment of the present application;

[0087] Figure 3 is a schematic diagram of the intersection of the extension lines of the two parallel lines existing in the vehicle itself on the same side of the visible wheel contact point in the image to be labeled according to an example of an embodiment of the present application;

[0088] Figure 4 is a schematic diagram of the intersection of the extension lines of the two parallel lines existing in the vehicle itself on the same side of the visible wheel contact point in the image to be labeled according to an example of an embodiment of the present application; Figure 3the intersection in the figure as the starting point, and the result of marking the invisible wheel contact point on the ray pointing to the visible wheel contact point;

[0089] Figure 5 the result of marking the invisible wheel contact point according to another example of an embodiment of the present application;

[0090] Figure 6 the main step flow chart of the vehicle posture prediction method according to an embodiment of the present application;

[0091] Figure 7 the definition of the final wheel contact point connecting angle according to an embodiment of the present application;

[0092] List of reference signs :

[0093] 1: visible wheel contact point; 2: intersection; 3: invisible wheel contact point. DETAILED DESCRIPTION

[0094] Some embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art will understand that these embodiments are only used to explain the technical principles of the present application, and are not intended to limit the protection scope of the present application.

[0095] In the description of the present application, "module" and "processor" can include hardware, software or a combination of both. A module can include hardware circuit, various suitable sensors, communication port, memory, and can also include software part such as program code, and can be a combination of software and hardware. The processor can be a central processor, microprocessor, image processor, digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor can be implemented in software, hardware or a combination of both. The non-transitory computer readable storage medium includes any suitable medium that can store program code, such as magnetic disk, hard disk, optical disk, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or both A and B. The term "at least one of A or B" or "at least one of A and B" has similar meaning as "A and / or B", and can include only A, only B or both A and B. The singular form of the term "one", "this" can also include plural forms.

[0096] Referring to the accompanying drawings Figure 1 , Figure 1 the main step flow chart of the marking method of the invisible wheel contact point according to an embodiment of the present application. As shown in the figure, the marking method of the invisible wheel contact point according to an embodiment of the present application includes the following steps: Figure 1As shown, the marking method of the invisible wheel grounding point in the embodiment of the present application mainly includes the following steps S101-S103.

[0097] Step S101: Mark the visible wheel grounding point in the image to be marked.

[0098] In this embodiment, the visible wheel grounding point in the image to be marked can be marked first.

[0099] Step S102: According to the principle of perspective projection, obtain the intersection point of the extension lines of the two parallel lines existing in the vehicle itself on the same side of the visible wheel grounding point in the image to be marked.

[0100] In this embodiment, according to the principle of perspective projection, the process of image acquisition by the image acquisition device is the process of mapping the information of a three-dimensional object to a two-dimensional plane. In this mapping process, the extension lines of parallel lines of a three-dimensional object in the physical world will intersect at a point when mapped to a two-dimensional plane. For example, when the vehicle is imaged, the parallel lines existing in the vehicle itself include the lower edge line of the vehicle window, the bottom edge line of the vehicle, the upper edge line of the vehicle body, and the connecting line of the two wheel grounding points. Therefore, the intersection point of the extension lines of the two parallel lines existing in the vehicle itself on the same side of the visible wheel grounding point can be obtained first. Please refer to FIG. 2 for details. Figure 3 , Figure 3 FIG. 2 is a schematic diagram of the intersection point of the extension lines of the two parallel lines existing in the vehicle itself on the same side of the visible wheel grounding point according to an example of the embodiment of the present application. As shown in FIG. 2, the intersection point 2 of the extension lines of the two parallel lines existing in the vehicle itself can be obtained by intersecting the lower edge line of the vehicle window and the bottom edge line of the vehicle. Figure 3

[0101] Step S103: Mark the invisible wheel grounding point according to the intersection point and the visible wheel grounding point.

[0102] In this embodiment, the invisible wheel grounding point can be marked according to the intersection point obtained in step S102 and the visible wheel grounding point. Specifically, after the intersection point of the extension lines of the two parallel lines existing in the vehicle itself on the same side of the visible wheel grounding point is determined, it is determined that this intersection point is also on the connecting line of the two wheel grounding points, and since the marking of the invisible wheel grounding point is generally used to predict the attitude angle of the vehicle, as long as the angle of the connecting line of the two wheel grounding points is correct, the position deviation of the invisible wheel grounding point will not affect the actual application. Therefore, the invisible wheel grounding point can be marked according to the intersection point and the visible wheel grounding point.

[0103] ​Based on the above steps S101-S103, the embodiments of the present invention can mark the invisible wheel grounding point based on the principle of perspective projection, the intersection of the extensions of two parallel lines existing on the vehicle itself in the image to be marked, and the visible wheel grounding point. This can achieve accurate marking of the invisible wheel grounding point when there is only one visible wheel grounding point in the image to be marked, and the marking process is easier to operate and has a lower cost.

[0104] In one embodiment of the present invention, step S103 may further include the following steps S1031 and S1032:

[0105] Step S1031: Establish a ray starting from the intersection point and pointing towards the visible wheel contact point.

[0106] Step S1032: Mark the invisible wheel contact points on the ray according to the length of the vehicle.

[0107] In this embodiment, the intersection point obtained in step S102 can be used as the starting point to establish a ray pointing to the visible wheel contact point, and the invisible wheel contact point can be marked on the established ray according to the length of the vehicle. See Appendix. Figure 4 and attached Figure 5 , Figure 4 This is an example of an embodiment of the present invention. Figure 3 The diagram shows the result of marking the invisible wheel contact points on the ray pointing from the intersection of the two points to the visible wheel contact points. Figure 5 This is a schematic diagram showing the annotation results of the invisible wheel contact point according to another embodiment of the present invention. Figure 4 and Figure 5 As shown, the invisible wheel grounding point 3 can be marked on the ray pointing from intersection point 2 to the visible wheel grounding point 1.

[0108] In one implementation, see 2. Figure 2 This is a schematic flowchart illustrating the main steps of a method for marking invisible wheel contact points according to an embodiment of the present invention. Figure 2 As shown, the method for marking the wheel contact point may include the following steps S201 to S205:

[0109] Step S201: Mark the visible wheel contact points.

[0110] In this embodiment, the method described in step S201 is similar to the method described in step S101 above, and will not be repeated here for the sake of simplicity.

[0111] Step S202: Determine whether parallel lines existing in the vehicle itself can be found in the image to be annotated; if yes, proceed to step S203; if no, end the annotation process.

[0112] In the embodiment, it can be determined whether the parallel lines can be found in the image to be labeled.

[0113] Step S203: draw two parallel lines in the image to be labeled and appropriately extend them to intersect.

[0114] In the embodiment, the method described in step S203 is similar to the method described in the foregoing step S102, and thus will not be described here again for simplicity.

[0115] Step S204: emit a ray from the intersection point to the visible wheel grounding point on the same side.

[0116] In the embodiment, the method described in step S204 is similar to the method described in the foregoing step S1031, and thus will not be described here again for simplicity.

[0117] Step S205: label the invisible wheel grounding point on the ray according to the length of the vehicle.

[0118] In the embodiment, the method described in step S205 is similar to the method described in the foregoing step S1032, and thus will not be described here again for simplicity.

[0119] Further, the present application also provides a vehicle posture prediction method.

[0120] Referring to the accompanying drawings Figure 6 , Figure 6 is a schematic diagram of the main steps of the vehicle posture prediction method according to an embodiment of the present application. As shown in Figure 6 , the vehicle posture prediction method of the embodiment of the present application can include the following steps S301 to S303:

[0121] Step S301: performing wheel grounding point detection on the image to be predicted to obtain the final wheel grounding point of the image to be predicted.

[0122] In the embodiment, the wheel grounding point detection can be performed on the image to be predicted to obtain the final wheel grounding point in the image to be detected.

[0123] In one embodiment, the wheel grounding point detection can be performed on the image to be predicted according to the method disclosed in the patent application with the publication number CN110246183A and the patent name of wheel grounding point detection method, device and storage medium.

[0124] Step S302: applying the trained wheel grounding point connecting line angle prediction model to predict the wheel grounding point connecting line angle of the image to be predicted to obtain the final wheel grounding point connecting line angle in the image to be predicted.

[0125] In the embodiment, the trained wheel contact point connection angle prediction model can be applied to predict the wheel contact point connection angle of the to-be-predicted image. The wheel contact point connection angle prediction model is trained based on a labeled data set. The labeled data set includes a first labeled image. The first labeled image includes one labeled visible wheel contact point and one labeled invisible contact point. The invisible contact point can be labeled according to the method described in the above embodiment of the labeling method of the invisible wheel contact point.

[0126] In one embodiment, the labeled data set can further include a second labeled image. The second labeled image includes two labeled visible wheel contact points.

[0127] In one embodiment, reference can be made to the accompanying drawings Figure 7 , Figure 7 is a schematic diagram of the definition of the final wheel contact point connection angle according to one embodiment of the embodiment of the present application. As shown in Figure 7 , a two-dimensional coordinate system can be established based on the plane where the to-be-predicted image is located. A direction of the plane where the to-be-predicted image is located is selected as the x-axis of the two-dimensional coordinate system, and a direction perpendicular to the x-axis on the plane where the to-be-predicted image is located is selected as the y-axis of the two-dimensional coordinate system. For example, the right horizontal direction of the plane where the to-be-predicted image is located can be selected as the x-axis of the two-dimensional coordinate system, and the direction perpendicular to the x-axis on the plane where the to-be-predicted image is located can be selected as the y-axis of the two-dimensional coordinate system. The wheel contact point connection angle can be defined as the included angle between the ray of the same side rear wheel contact point and the front wheel contact point and the x-axis, which is the wheel contact point connection angle (theta2d). For example, if the ray is the same as the positive direction of the x-axis, then theta2d=0; if the ray is the same as the positive direction of the y-axis, then theta2d=pi / 2; if the ray is the same as the negative direction of the x-axis, then theta2d=pi; and if the ray is the same as the negative direction of the y-axis, then theta2d=-pi / 2.

[0128] In one embodiment, since only two points are needed to determine a straight line, the labeling of the wheel contact points can be simplified to at most two wheel contact points on one side. If there are more than two wheel contact points in actual labeling, two wheel contact points can be selected at will. Thus, for one vehicle, there are a total of four wheel contact points. The four wheel contact points can be defined as the left front wheel, the left rear wheel, the right rear wheel, and the right front wheel, which can be represented as left front wheel p lf =[x lf ,y lf ,v lf ], left rear wheel p lb =[x lb ,y lb ,v lb ], right rear wheel p rb= [x rb , y rb , v rb ], p rf = [x rf , y rf , v rf ], where x and y represent the x-axis coordinate and y-axis coordinate of the corresponding point respectively, and v represents the visibility label of the corresponding point, v = 0 means that the point is invisible and not labeled, v = 1 means that the point is invisible but the position is labeled, and v = 2 means that the point is visible and the position is labeled. The wheel contact point connecting line angle prediction model can be constructed according to the above definition, and the wheel contact point connecting line angle prediction model is trained by applying the labeled training set:

[0129] 1) If there are two labeled wheel contact points on the left side of the wheel and at least one wheel contact point is visible, the wheel contact point connecting line angle on the left side of the vehicle can be obtained according to the following formula (1)

[0130]

[0131] 2) Similarly, the wheel contact point connecting line angle on the right side of the vehicle can be obtained using the same method

[0132] In one embodiment, considering that the regression range of the wheel contact point connecting line angle (-90° to 180°) is large, the sine value and the cosine value of the wheel contact point connecting line angle (the regression range is between -1 and 1) can be predicted by using the wheel contact point connecting line angle prediction model. The wheel contact point connecting line angle prediction model can include two fully connected layers, and the sine value and the cosine value of the wheel contact point connecting line angle on the left side of the vehicle and the sine value and the cosine value of the wheel contact point connecting line angle on the right side of the vehicle are learned through the two fully connected layers, and the corresponding outputs are respectively, which are the sine value of the left wheel contact point connecting line angle, the cosine value of the left wheel contact point connecting line angle, the sine value of the right wheel contact point connecting line angle, and the cosine value of the right wheel contact point connecting line angle. That is, the wheel contact point connecting line angle prediction model can obtain the sine value and the cosine value of the left wheel contact point connecting line angle, and the sine value and the cosine value of the right wheel contact point connecting line angle.

[0133] The loss function in the training process of the wheel contact point connecting line angle prediction model can be obtained according to the following formulas (2)-(6):

[0134]

[0135] m l = ((v lf > 0) && (v lb > 0) && ((vlf >1|v lb |>1)) (3)

[0136]

[0137] m r =((v rf >0)&&(v rb >0)&&(v rf >1|v rb |>1)) (5)

[0138] L1(a,b)=|a-b| (6)

[0139] wherein, and are the loss function of the left wheel grounding point connecting line angle and the loss function of the right wheel grounding point connecting line angle, respectively, m l and m r are the limit function of the left wheel grounding point connecting line angle and the limit function of the right wheel grounding point connecting line angle, respectively, the purpose of the limit function is to shield some invalid cases, and L1(a,b) is the overall loss function of the wheel grounding point connecting line angle prediction model, a is a predicted value, and b is a target value.

[0140] Step S303: obtaining the attitude angle of the vehicle according to the final wheel grounding point connecting line angle and the final wheel grounding point.

[0141] In the embodiment, the attitude angle of the vehicle can be obtained according to the final wheel grounding point connecting line angle and the final wheel grounding point.

[0142] Based on the above steps S301-S303, the embodiment of the present application can predict the wheel grounding point connecting line angle of the to-be-predicted image with only one visible wheel grounding point according to the trained wheel grounding point connecting line angle prediction model, and perform wheel grounding point detection, and obtain the attitude angle of the vehicle according to the obtained final wheel grounding point connecting line angle and final wheel grounding point. Through the above configuration mode, the embodiment of the present application can effectively predict the attitude angle of the vehicle according to the final wheel grounding point connecting line angle and the final wheel grounding point when only one wheel grounding point is visible in the to-be-predicted image under the vehicle truncation scene, greatly expanding the use scene of the wheel grounding point, and effectively improving the accuracy and safety of the automatic driving process.

[0143] The steps S301-S303 are further described as follows:

[0144] In one embodiment of the embodiment of the present application, step S301 can further include the following steps S3011 and S3012:

[0145] Step S3011: performing wheel grounding point detection on the image to be predicted to obtain the wheel grounding points in the image to be predicted and the prediction probabilities of the wheel grounding points.

[0146] Step S3012: selecting the wheel grounding point with the maximum prediction probability as the final wheel grounding point.

[0147] In the embodiment, the wheel grounding point detection can be performed on the image to be predicted to obtain the position of each wheel grounding point and the prediction probability of the corresponding wheel grounding point, and the wheel grounding point with the maximum prediction probability is selected as the final wheel grounding point. The higher the prediction probability is, the higher the confidence of the wheel grounding point is.

[0148] In one embodiment, the preset wheel grounding point detection model can be trained by labeling the training set to obtain the trained wheel grounding point detection model, and the wheel grounding point detection model is applied to detect the wheel grounding points in the image to be predicted to obtain all the wheel grounding points of the vehicle and the prediction probabilities of the wheel grounding points.

[0149] In one embodiment of the embodiment of the present application, step S302 can further include the following step S3021 and step S3022:

[0150] Step S3021: applying the trained wheel grounding point connecting angle prediction model to obtain the left connecting angle of the vehicle and the right connecting angle of the vehicle of the image to be predicted.

[0151] In the embodiment, step S3021 can further include the following step S30211 and step S30212:

[0152] Step S30211: applying the trained wheel grounding point connecting angle prediction model to obtain the sine value and the cosine value of the left connecting angle of the vehicle of the image to be predicted; and obtaining the left connecting angle of the vehicle according to the sine value and the cosine value.

[0153] Step S30212: applying the trained wheel grounding point connecting angle prediction model to obtain the sine value and the cosine value of the right connecting angle of the image to be predicted; and obtaining the right connecting angle of the vehicle according to the sine value and the cosine value.

[0154] Step S3022: determining the final wheel grounding point connecting angle according to the left connecting angle of the vehicle, the right connecting angle of the vehicle and the position of the final wheel grounding point.

[0155] In the embodiment, the wheel grounding point connecting angle can include a vehicle left side connecting angle and a vehicle right side connecting angle, and the wheel grounding point connecting angle prediction model can obtain the vehicle left side connecting angle and the vehicle right side connecting angle of the to-be-predicted image. The final wheel grounding point connecting angle can be determined according to the position of the final wheel grounding point.

[0156] In one embodiment, the step S3022 can further include a step S30221 and a step S30222.

[0157] The step S30221: If the final wheel grounding point is located on the left side of the vehicle, the vehicle left side connecting angle is taken as the final wheel grounding point connecting angle.

[0158] The step S30222: If the final wheel grounding point is located on the right side of the vehicle, the vehicle right side connecting angle is taken as the final wheel grounding point connecting angle.

[0159] In the embodiment, since the final wheel grounding point is the wheel grounding point with the highest prediction probability, that is, the final wheel grounding point has the highest confidence, the wheel grounding point connecting angle on the same side of the final wheel grounding point can be selected as the final wheel grounding point connecting angle.

[0160] In one embodiment of the embodiment of the application, the step S303 can further include a step S3031 and a step S3032.

[0161] The step S3031: According to the final wheel grounding point connecting angle and the final wheel grounding point, the position of a second point on the wheel grounding point connecting line where the final wheel grounding point is located in the to-be-predicted image is obtained.

[0162] In the embodiment, the step S3031 can further include a step S30311 to a step S30313.

[0163] The step S30311: It is judged whether there is another wheel grounding point on the same side of the final wheel grounding point as a visible wheel grounding point except the final wheel grounding point; if yes, the step S30312 is jumped to; if no, the step S30313 is jumped to.

[0164] In the embodiment, it can be judged whether another wheel grounding point is visible on the same side of the final wheel grounding point, and specifically, the prediction probability of the wheel grounding point obtained in the step S3011 can be used to judge whether the wheel grounding point is a visible wheel grounding point. If the prediction probability of the wheel grounding point is greater than a preset prediction probability threshold, it can be determined that the wheel grounding point is a visible wheel grounding point.

[0165] The step S30312: The visible wheel grounding point is taken as the second point.

[0166] In the embodiment, the visible wheel grounding point can be taken as the second point.

[0167] In one embodiment, if multiple visible wheel grounding points are obtained, the visible wheel grounding point with the maximum prediction probability can be selected as the second point, or a visible wheel grounding point can be randomly selected as the second point.

[0168] Step S30313: Obtain the position of the second point according to the following formula (7) and formula (8):

[0169]

[0170]

[0171] wherein p x , p y are the x-axis and y-axis coordinate values of the second point, respectively, are the x-axis and y-axis coordinate values of the final wheel grounding point, respectively, and Δ is any positive integer and θ is the angle of the final wheel grounding point. That is, any point on the straight line determined by the final wheel grounding point and the angle of the final wheel grounding point is taken as the second point.

[0172] Step S3032: Obtain the attitude angle of the vehicle according to the position of the final wheel grounding point and the position of the second point.

[0173] In the embodiment, the attitude angle of the vehicle can be calculated according to the position of the final wheel grounding point and the position of the second point, in combination with the intrinsic and extrinsic parameters of the vehicle-mounted camera.

[0174] In one embodiment, the attitude angle of the vehicle can be obtained based on the two wheel grounding points on the same side according to the method disclosed in the patent application with the publication number CN111967360A and the patent name of target vehicle attitude detection method based on wheels.

[0175] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art can understand that, in order to achieve the effect of the present application, the different steps do not necessarily have to be executed in such an order, they can be executed simultaneously (in parallel) or in other orders, and these changes are within the protection scope of the present application.

[0176] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable storage medium can include any entity or device, medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, according to the legislation and patent practice in some jurisdictions, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0177] Further, the present application also provides a control device. In an embodiment of the control device according to the present application, the control device comprises a processor and a storage device, the storage device can be configured to store a program for executing the vehicle pose prediction method of the above-mentioned method embodiments, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program for executing the vehicle pose prediction method of the above-mentioned method embodiments. For the convenience of description, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The control device can be a control device device formed by various electronic devices.

[0178] Further, the present application also provides a computer readable storage medium. In an embodiment of the computer readable storage medium according to the present application, the computer readable storage medium can be configured to store a program for executing the vehicle pose prediction method of the above-mentioned method embodiments, which can be loaded and run by a processor to implement the above-mentioned vehicle pose prediction method. For the convenience of description, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The computer readable storage medium can be a storage device device formed by various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present application is a non-transitory computer readable storage medium.

[0179] Further, it should be understood that, since the setting of each module is only for illustrating the functional units of the device of the present application, the physical device corresponding to the module can be the processor itself, or a part of software, hardware or the combination of software and hardware in the processor. Therefore, the number of each module in the figure is only illustrative.

[0180] Those skilled in the art can understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solution to deviate from the principles of the present application, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present application.

[0181] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without deviating from the principles of the present application, and the technical solutions after such changes or replacements will fall within the protection scope of the present application.

Claims

1. A vehicle pose prediction method, characterized by, The method comprises: performing wheel grounding point detection on the image to be predicted to obtain a final wheel grounding point of the image to be predicted; applying the trained wheel grounding point connecting line angle prediction model to predict the wheel grounding point connecting line angle of the image to be predicted to obtain a final wheel grounding point connecting line angle of the image to be predicted; obtaining the attitude angle of the vehicle according to the final wheel grounding point connecting line angle and the final wheel grounding point, wherein the wheel grounding point connecting line angle prediction model is trained based on a labeled data set, the labeled data set contains a first labeled image, the first labeled image contains a labeled visible wheel grounding point and a labeled invisible wheel grounding point, and the invisible wheel grounding point is labeled according to the following steps: labeling the visible wheel grounding point in the image to be labeled; obtaining the intersection of the extensions of two parallel lines existing in the vehicle itself on the same side of the visible wheel grounding point in the image to be labeled according to the principle of perspective projection; establishing a ray pointing to the visible wheel grounding point with the intersection as the starting point; labeling the invisible wheel grounding point on the ray according to the length of the vehicle; the wheel grounding point connecting line angle includes a left side connecting line angle of the vehicle and a right side connecting line angle of the vehicle, and the step of "applying the trained wheel grounding point connecting line angle prediction model to predict the wheel grounding point connecting line angle of the image to be predicted to obtain a final wheel grounding point connecting line angle of the image to be predicted" comprises: applying the trained wheel grounding point connecting line angle prediction model to obtain the left side connecting line angle of the vehicle and the right side connecting line angle of the vehicle of the image to be predicted; determining the final wheel grounding point connecting line angle according to the position of the left side connecting line angle of the vehicle, the right side connecting line angle of the vehicle and the final wheel grounding point.

2. The vehicle attitude prediction method according to claim 1, characterized by, the step of "performing wheel grounding point detection on the image to be predicted to obtain a final wheel grounding point of the image to be predicted" comprises: performing wheel grounding point detection on the image to be predicted to obtain the wheel grounding point in the image to be predicted and the prediction probability of the wheel grounding point; selecting the wheel grounding point with the maximum prediction probability as the final wheel grounding point.

3. The vehicle attitude prediction method according to claim 1, characterized by, the step of "determining the final wheel grounding point connecting line angle according to the position of the left side connecting line angle of the vehicle, the right side connecting line angle of the vehicle and the final wheel grounding point" comprises: if the final wheel grounding point is located on the left side of the vehicle, taking the left side connecting line angle of the vehicle as the final wheel grounding point connecting line angle; if the final wheel grounding point is located on the right side of the vehicle, taking the right side connecting line angle of the vehicle as the final wheel grounding point connecting line angle.

4. The vehicle attitude prediction method according to claim 1, characterized by, the step of "applying the trained wheel grounding point connecting line angle prediction model to obtain the left side connecting line angle of the vehicle and the right side connecting line angle of the vehicle of the image to be predicted" comprises: applying the trained wheel grounding point connecting line angle prediction model to obtain the sine value and the cosine value of the left side connecting line angle of the vehicle of the image to be predicted; and obtaining the left side connecting line angle of the vehicle according to the sine value and the cosine value. the step of "applying the trained wheel grounding point connecting line angle prediction model to obtain the left side connecting line angle of the vehicle and the right side connecting line angle of the vehicle of the image to be predicted" comprises: applying the trained wheel grounding point connecting line angle prediction model to obtain the sine value and the cosine value of the left side connecting line angle of the vehicle of the image to be predicted; and obtaining the left side connecting line angle of the vehicle according to the sine value and the cosine value. The trained wheel contact point connecting line angle prediction model is applied to obtain a sine value and a cosine value of an angle of a right connecting line of the to-be-predicted image; and the right connecting line angle of the vehicle is obtained according to the sine value and the cosine value.

5. The vehicle attitude prediction method according to claim 1, characterized by, The step of "obtaining the attitude angle of the vehicle according to the final wheel contact point connecting line angle and the final wheel contact point" comprises: obtaining a position of a second point on a wheel contact point connecting line where the final wheel contact point is located in the to-be-predicted image according to the final wheel contact point connecting line angle and the final wheel contact point; obtaining the attitude angle of the vehicle according to the position of the final wheel contact point and the position of the second point.

6. The vehicle attitude prediction method according to claim 5, characterized by, The step of "obtaining a position of a second point on a wheel contact point connecting line where the final wheel contact point is located in the to-be-predicted image according to the final wheel contact point connecting line angle and the final wheel contact point" comprises: determining whether there is another wheel contact point on the same side of the final wheel contact point as the visible wheel contact point; if yes, taking the visible wheel contact point as the second point; if no, obtaining the position of the second point according to the following formula: wherein, , are the x-axis and y-axis coordinate values of the second point, respectively, , are the x-axis and y-axis coordinate values of the final wheel contact point, respectively, is any positive integer, is the angle of the final wheel contact point.

7. The vehicle attitude prediction method according to claim 6, characterized by, The method further comprises determining whether another wheel contact point is a visible wheel contact point according to the following steps: comparing a prediction probability of the other wheel contact point with a preset prediction probability threshold value, and determining that the other wheel contact point is a visible wheel contact point when the prediction probability is greater than the prediction probability threshold value.

8. A control device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the vehicle attitude prediction method in any one of claims 1 to 7.

9. A computer readable storage medium having stored therein a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the vehicle attitude prediction method in any one of claims 1 to 7.

10. A vehicle characterized by comprising: The vehicle comprises the control device of claim 8.

Images