Detection Method, Device, Electronic Device and Storage Medium for Wheel Ground Contact Point

Abstract

The present invention provides a detection method, device, electronic device and storage medium for the wheel contact point. The method includes: obtaining an image containing a target vehicle; inputting the image containing the target vehicle into a pre-trained wheel contact point prediction model to obtain the contact points of each wheel of the target vehicle; wherein, the wheel contact point prediction model is trained based on the images of sample vehicles. Through the wheel contact point prediction model of the present invention, the detection accuracy of the wheel contact point is more accurate, providing a relatively robust result for both the subsequent prediction of the vehicle's own attitude and subsequent tasks.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method, device, electronic device and storage medium for detecting a wheel contact point. Background Art

[0002] Accurate positioning is a very important part of various technologies such as autonomous driving and obstacle avoidance navigation. For example, when a vehicle is driving in the current lane, it needs to determine the position relationship with other vehicles and make corresponding control actions to prevent collision accidents. The current mainstream vehicle positioning methods are three-dimensional (3D) detection and positioning methods, and detection and positioning methods based on wheel contact points.

[0003] The premise of realizing the detection and positioning method based on the wheel contact point is to realize the detection of the wheel contact point. The wheel contact point detection method in the prior art is to identify the outer contour of at least one obstacle vehicle from the collected image to be detected, and then input the outer contour of the obstacle vehicle into the neural network model, and use the output information of the neural network model to determine the wheel contact point of the obstacle vehicle.

[0004] The defects of the wheel contact point detection method in the prior art are: the processing method for the collected information is single, and the judgment accuracy of the wheel contact point position is still not ideal, which in turn has an adverse effect on the prediction of the vehicle's own posture and other subsequent tasks. Summary of the invention

[0005] The present invention provides a wheel contact point detection method, device, electronic device and storage medium, which are used to solve the problem of inaccurate prediction of vehicle orientation and vehicle distance in the prior art, and achieve more accurate prediction of target vehicle orientation and target vehicle distance by accurately measuring the target wheel contact point.

[0006] The present invention provides a method for detecting a wheel contact point, comprising:

[0007] Acquire an image containing a target vehicle;

[0008] Inputting the image containing the target vehicle into a pre-trained wheel contact point prediction model to obtain the contact point of each wheel of the target vehicle;

[0009] The wheel contact point prediction model is obtained by training based on images of sample vehicles.

[0010] According to a wheel contact point detection method provided by the present invention, the wheel contact point prediction model includes an image preprocessing layer, an approximate contact point acquisition layer, a candidate contact point set acquisition layer and a contact point determination layer;

[0011] Among them, the image preprocessing layer is used to obtain the heat map of the grounding points of each wheel of the target vehicle, the position coordinates of the center point of the target vehicle, and the deviation values between the center point of the target vehicle and the grounding points of each wheel of the target vehicle according to the image containing the target vehicle;

[0012] The approximate grounding point acquisition layer is used to obtain the approximate grounding points of each wheel of the target vehicle according to the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the grounding points of each wheel of the target vehicle;

[0013] The candidate grounding point set acquisition layer is used to obtain the candidate grounding point sets of each wheel of the target vehicle according to the heat map of the grounding points of each wheel of the target vehicle;

[0014] The grounding point determination layer is used to determine the grounding points of each wheel of the target vehicle according to the distances between the approximate grounding points of each wheel of the target vehicle and the respective candidate grounding points in the candidate grounding point sets of each wheel of the target vehicle.

[0015] According to a method for detecting the grounding points of wheels provided by the present invention, the step of inputting the image containing the target vehicle into a pre-trained wheel grounding point prediction model to obtain the grounding points of each wheel of the target vehicle includes:

[0016] Inputting the image containing the target vehicle into the image preprocessing layer to obtain the heat map of the grounding points of each wheel of the target vehicle, the position coordinates of the center point of the target vehicle, and the deviation values between the center point of the target vehicle and the grounding points of each wheel of the target vehicle;

[0017] Inputting the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the grounding points of each wheel of the target vehicle into the approximate grounding point acquisition layer to obtain the approximate grounding points of each wheel of the target vehicle;

[0018] Inputting the heat map of the grounding points of each wheel of the target vehicle into the candidate grounding point set acquisition layer to obtain the candidate grounding point sets of each wheel of the target vehicle;

[0019] Inputting the approximate grounding points of each wheel of the target vehicle and the candidate grounding point sets of each wheel of the target vehicle into the grounding point determination layer to obtain the grounding points of each wheel of the target vehicle.

[0020] According to a method for detecting the grounding points of wheels provided by the present invention, the step of inputting the heat map of the grounding points of each wheel of the target vehicle into the candidate grounding point set acquisition layer to obtain the candidate grounding point sets of each wheel of the target vehicle includes:

[0021] Inputting the heat map of the grounding points of each wheel of the target vehicle into the first processing layer of the candidate grounding point set acquisition layer to obtain the initial values of the candidate grounding points of each wheel of the target vehicle output by the first processing layer;

[0022] The initial values of the candidate grounding points of each wheel of the target vehicle and the quantization errors corresponding to each wheel of the target vehicle are input into the second processing layer of the candidate grounding point set acquisition layer to obtain the candidate grounding point sets of each wheel of the target vehicle.

[0023] According to a method for detecting the grounding point of a wheel provided by the present invention, the step of inputting the approximate grounding points of each wheel of the target vehicle and the candidate grounding point sets of each wheel of the target vehicle into the grounding point determination layer to obtain the grounding points of each wheel of the target vehicle includes:

[0024] Input the approximate grounding points of each wheel of the target vehicle and the candidate grounding point sets of each wheel of the target vehicle into the distance calculation layer of the grounding point determination layer to obtain the distances between the approximate grounding points of each wheel of the target vehicle output by the distance calculation layer and each candidate grounding point in the candidate grounding point sets of each wheel of the target vehicle;

[0025] Input the distances between the approximate grounding points of each wheel of the target vehicle output by the distance calculation layer and each candidate grounding point in the candidate grounding point sets of each wheel of the target vehicle into the judgment layer of the grounding point determination layer to obtain the grounding points of each wheel of the target vehicle output by the judgment layer; wherein, the judgment layer is pre-set with constraint conditions.

[0026] According to a method for detecting the grounding point of a wheel provided by the present invention, the method further includes:

[0027] Obtain the labeled map of the sample vehicle and obtain the heat map of the grounding points of each wheel of the sample vehicle; wherein, the labeled map of the sample vehicle includes the labeled information of the center point of the sample vehicle and the labeled information of the grounding points of the wheels of the sample vehicle;

[0028] Input the image of the sample vehicle into the wheel grounding point prediction model to be trained to obtain the grounding points of each wheel of the sample vehicle and the deviation values between the center point of the sample vehicle and the grounding points of each wheel of the sample vehicle;

[0029] Calculate the position coordinates of the predicted center point of the sample vehicle according to the grounding points of each wheel of the sample vehicle and the deviation values between the center point of the sample vehicle and the grounding points of each wheel of the sample vehicle;

[0030] Calculate the loss function value of this round of training according to the position coordinates of the center point of the sample vehicle and the position coordinates of the predicted center point of the sample vehicle; wherein, the position coordinates of the center point of the sample vehicle are obtained from the labeled information of the center point of the sample vehicle in the labeled map of the sample vehicle;

[0031] If the change value between the loss function value of the current round of training and that of the previous round of training is greater than a preset threshold, continue the training process; or if the change value between the loss function value of the current round of training and that of the previous round of training is less than or equal to the preset threshold and the loss function value of the current round of training tends to be stable, end the training.

[0032] The present invention also provides a detection device for the wheel contact point, including:

[0033] A target vehicle information acquisition module, configured to acquire an image including the target vehicle;

[0034] A wheel contact point prediction module, configured to input the image including the target vehicle into a pre-trained wheel contact point prediction model to obtain the contact points of each wheel of the target vehicle;

[0035] Wherein, the wheel contact point prediction model is trained based on the images of sample vehicles.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the detection method for the wheel contact point as described are implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the detection method for the wheel contact point as described are implemented.

[0038] The present invention also provides a computer program product, which includes computer-executable instructions that are used to implement the steps of the detection method for the wheel contact point as described when executed.

[0039] The detection method, device, electronic device, and storage medium for the wheel contact point provided by the present invention obtain an image including the target vehicle; input the image including the target vehicle into a pre-trained wheel contact point prediction model to obtain the contact points of each wheel of the target vehicle. Through the wheel contact point prediction model of the present invention, the detection accuracy of the wheel contact point is more accurate, providing a relatively robust result for both the prediction of the vehicle's own attitude and subsequent tasks, and achieving significant progress. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 is a schematic flowchart of the method for detecting the wheel contact point provided by the present invention;

[0042] Figure 2 is a schematic structural diagram of the device for detecting the wheel contact point provided by the present invention;

[0043] Figure 3 is a schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0044] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0045] The following combines Figures 1 - 3 , and uses embodiments to specifically describe the method, device, electronic device and storage medium for detecting the wheel contact point of the present invention.

[0046] Figure 1 is a flowchart of the method for detecting the wheel contact point provided by the present invention. As Figure 1 shown, the method for detecting the wheel contact point provided by the present invention includes:

[0047] Step S110, acquire an image including a target vehicle.

[0048] In this embodiment, the target vehicle refers to a vehicle in front of or behind the current road on which the vehicle is traveling or a vehicle in a neighboring lane on the left or right that may cut into the lane on which the vehicle is traveling. That is to say, vehicles within a certain distance that may affect the safe driving of the vehicle will be regarded as target vehicles. The present invention adopts a strong assumption that a general vehicle has four wheels. Based on the strong assumption that the target vehicle has four wheels, the contact points of the four wheels will be obtained finally.

[0049] In this embodiment, assume that there is a four-wheel motor vehicle in front of the lane on which the vehicle is traveling and within the capture range of the camera of the vehicle. Then this four-wheel motor vehicle will be automatically defined as the target vehicle, and image information will be collected. The image information including the target vehicle includes the vehicle center point and the wheel contact point.

[0050] It should be emphasized that the image including the target vehicle captured by the vehicle camera is essentially a picture on the road. That is to say, the picture captured by the vehicle camera within the capture range may include many vehicles, that is, the target vehicle is not unique.

[0051] Step S120: Input the image including the target vehicle into a pre-trained wheel ground contact point prediction model to obtain the ground contact points of each wheel of the target vehicle; wherein, the wheel ground contact point prediction model is trained based on the images of sample vehicles.

[0052] In this embodiment, the wheel ground contact point prediction model is a network model with deep learning capabilities. During training, various possible application scenarios during the driving of the vehicle itself will be simulated, such as rainy or snowy weather, sudden deceleration of the vehicle directly in front of the vehicle itself, the vehicle itself passing through traffic lights, vehicles in the adjacent lane of the vehicle itself merging into the lane of the vehicle itself, and so on, various normal or abnormal application scenarios. Based on the labeled maps of a large number of sample vehicles and the heat maps of the ground contact points of each wheel of the sample vehicles obtained under these scenarios, input them into the wheel ground contact point prediction model and train repeatedly until the output value of the wheel ground contact point prediction model meets the accuracy requirements for the wheel ground contact points preset in the present invention.

[0053] The wheel ground contact point prediction model includes an image preprocessing layer, an approximate ground contact point acquisition layer, a candidate ground contact point set acquisition layer, and a ground contact point determination layer. Among them, the image preprocessing layer is used to obtain the heat map of the ground contact points of each wheel of the target vehicle, the position coordinates of the center point of the target vehicle, and the deviation values between the center point of the target vehicle and the ground contact points of each wheel of the target vehicle according to the image including the target vehicle; the approximate ground contact point acquisition layer is used to obtain the approximate ground contact points of each wheel of the target vehicle according to the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the ground contact points of each wheel of the target vehicle; the candidate ground contact point set acquisition layer is used to obtain the candidate ground contact point sets of each wheel of the target vehicle according to the heat map of the ground contact points of each wheel of the target vehicle; the ground contact point determination layer is used to determine the ground contact points of each wheel of the target vehicle according to the distances between the approximate ground contact points of each wheel of the target vehicle and each candidate ground contact point in the candidate ground contact point sets of each wheel of the target vehicle. After the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the ground contact points of each wheel of the target vehicle are input into the approximate ground contact point acquisition layer, the approximate ground contact point acquisition layer of the wheel ground contact point prediction model will execute a top-down method, that is, the top-down method.

[0054] In this embodiment, the number of channels of the deviation values between the center point of the target vehicle and the ground contact points of each wheel of the target vehicle is 8. Every 2 channels out of the 8 channels correspond to one wheel. One of the 2 channels represents the deviation value of the ground contact point of the corresponding wheel in the x direction from the center point of the target vehicle, and the other represents the deviation value of the ground contact point of the corresponding wheel in the y direction from the center point of the target vehicle. For example, the 1st and 2nd channels among the 8 channels describe the deviation values of the left rear wheel, the 3rd and 4th channels describe the deviation values of the left front wheel, the 5th and 6th channels describe the deviation values of the right rear wheel, and the 7th and 8th channels describe the deviation values of the right front wheel.

[0055] Assume that the center point position coordinates of the target vehicle are (x 0 , y 0 ). The deviation values of the left rear wheel are (δx 1 , δy 1 ), the deviation values of the left front wheel are (δx 2 , δy 2 ), the deviation values of the right rear wheel are (δx 3 , δy 3 ), and the deviation values of the right front wheel are (δx 4 , δy 4 ). The coordinate values of the approximate grounding points of the final left rear wheel are obtained as (x 1 = x 0 + δx 1 , y 1 = y 0 + δy 1 ), the coordinate values of the approximate grounding points of the left front wheel are (x 2 = x 0 + δx 2 , y 2 = y 0 + δy 2 ), the coordinate values of the approximate grounding points of the right rear wheel are (x 3 = x 0 + δx 3 , y 3 = y 0 + δy 3 ), and the coordinate values of the approximate grounding points of the right front wheel are (x 4 = x 0 + δx 4 , y 4 = y 0 + δy 4 ).

[0056] The approximate grounding points calculated here are not the real wheel grounding points. In the subsequent steps, the real wheel grounding points will be determined based on the approximate grounding points.

[0057] After inputting the heat maps of the grounding points of each wheel of the target vehicle into the candidate grounding point set acquisition layer, the candidate grounding point set acquisition layer of the wheel grounding point prediction model will execute a bottom-up method, i.e., the bottom-up method.

[0058] In this embodiment, the heat maps of the grounding points of each wheel of the target vehicle are input into the first processing layer of the candidate grounding point set acquisition layer to obtain the initial values of the candidate grounding points of each wheel of the target vehicle output by the first processing layer; the initial values of the candidate grounding points of each wheel of the target vehicle and the quantization errors corresponding to each wheel of the target vehicle are input into the second processing layer of the candidate grounding point set acquisition layer to obtain the candidate grounding point sets of each wheel of the target vehicle.

[0059] In this embodiment, the heatmap of the grounding points of each wheel of the target vehicle is a 4-channel heatmap, and the 4 channels respectively represent the heatmap of the grounding point of the left rear wheel, the heatmap of the grounding point of the left front wheel, the heatmap of the grounding point of the right rear wheel, and the heatmap of the grounding point of the right front wheel. Performing maxpooling on the heatmaps of these channels respectively can obtain one or more extreme points, and the values of these extreme points are the initial values of the candidate grounding points. The quantization error refers to the error generated in the present invention due to the different resolutions of the input and output images. For example, the resolution of the input image is 512x512, and the resolution of the output image is 128x128, that is, it is downsampled by 4 times. Then, on the 128x128 feature map, one point represents a range of 4x4 on the original 512x512 image. That is to say, if the points on the 128x128 image are mapped back to the 512x512 image, some places will not be represented. Therefore, a quantization error is introduced to find the actual position of the points on the 128x128 image on the 512x512 image. For example, there is a point with the position coordinates (23, 20) on the 128x128 image, and when mapped back to the original image, it is at the point (132, 80). However, if the corresponding real point is at the position (133, 80), the point at this position after downsampling and rounding also falls on the point (23, 20). Therefore, such a quantization error is needed to move the point (132, 80) to the point (133, 80).

[0060] Adding the initial values of the candidate grounding points to the quantization errors corresponding to each wheel of the target vehicle to obtain the set of candidate grounding points for each wheel of the target vehicle.

[0061] For example, the set of candidate grounding points for the left rear wheel is {(X1, Y1), (X2, Y2), (X3, Y3),...}, the set of approximate grounding points for the left front wheel is {(M1, N1), (M2, N2), (M3, N3),...}, the set of approximate grounding points for the right rear wheel is {(H1, G1), (H2, G2), (H3, G3),...}, and the set of approximate grounding points for the right front wheel is {(L1, I1), (L2, I2), (L3, I3),...}. Among them, each wheel has at least one candidate wheel grounding point, so each wheel corresponds to a set of candidate wheel grounding points. That is to say, after processing the heatmap of the grounding points of each wheel, the number of candidate wheel grounding points obtained for each wheel is not unique.

[0062] After inputting the approximate ground contact points of each wheel of the target vehicle and the set of candidate ground contact points of each wheel of the target vehicle into the ground contact point determination layer, the ground contact point determination layer of the wheel ground contact point prediction model will determine the final ground contact points of the wheels of the target vehicle through calculation.

[0063] In this embodiment, the distances between the approximate ground contact points of each wheel of the target vehicle and each candidate ground contact point in the set of candidate ground contact points of each wheel of the target vehicle are obtained by inputting the approximate ground contact points of each wheel of the target vehicle and the set of candidate ground contact points of each wheel of the target vehicle into the distance calculation layer of the ground contact point determination layer; the distances between the approximate ground contact points of each wheel of the target vehicle output by the distance calculation layer and each candidate ground contact point in the set of candidate ground contact points of each wheel of the target vehicle are input into the judgment layer of the ground contact point determination layer, and the ground contact points of each wheel of the target vehicle output by the judgment layer are obtained; wherein, the judgment layer is preset with constraint conditions.

[0064] Specifically, the approximate wheel ground contact points obtained by the top-down method, including the approximate ground contact point of the left rear wheel (x 1 =x 0 +δx 1 , y 1 =y 0 +δy 1 ), the approximate ground contact point of the left front wheel (x 2 =x 0 +δx 2 , y 2 =y 0 +δy 2 ), the approximate ground contact point of the right rear wheel (x 3 =x 0 +δx 3 , y 3 =y 0 +δy 3 ) and the approximate ground contact point of the right front wheel (x 4 =x 0 +δx 4 , y 4 =y 0 +δy 4)。Combined with the candidate wheel grounding points obtained by the bottom-up method, including the left rear wheel candidate grounding points {(X1, Y1), (X2, Y2), (X3, Y3),...}, the left front wheel approximate grounding points {(M1, N1), (M2, N2), (M3, N3),...}, the right rear wheel approximate grounding points {(H1, G1), (H2, G2), (H3, G3),...} and the right front wheel approximate grounding points {(L1, I1), (L2, I2), (L3, I3),...}, through comprehensive analysis, the finally determined wheel grounding points are obtained. For the left rear wheel, taking the approximate grounding point (x1, y1) as the center, calculate the differences between the approximate grounding point (x1, y1) and all the left rear wheel candidate grounding points {(X1, Y1), (X2, Y2), (X3, Y3),...}, obtaining a set of differences, and then find the candidate grounding point corresponding to the smallest difference, which is used to represent the final wheel grounding point jointly selected by the top-down method and the bottom-up method. By analogy, the final wheel grounding points of the other three wheels can be obtained. It should be noted that the finally selected wheel grounding points should also meet the constraint conditions, that is, the grounding points of this wheel need to be within a certain range of the 2d bbox of the vehicle. For example, a threshold is set in advance, which can be a value range or a numerical value. When obtaining the smallest difference between the approximate wheel grounding point and the candidate wheel grounding point, the smallest difference also needs to meet the preset threshold to finally determine the candidate wheel grounding point corresponding to the smallest difference as the real wheel grounding point.

[0065] The wheel grounding point detection method provided by the present invention detects the grounding points of each wheel of the target vehicle through a wheel grounding point prediction model based on the collected image information including the target vehicle and the heat map of the grounding points of each wheel of the target vehicle, making the detection accuracy of the wheel grounding points more accurate and providing a relatively robust result for both the prediction of the vehicle's own attitude and subsequent tasks.

[0066] Based on any of the above embodiments, in this embodiment, according to a wheel grounding point detection method provided by the present invention, the method further includes:

[0067] Obtain the annotation map of the sample vehicle and the heat map of the grounding points of each wheel of the sample vehicle; wherein, the annotation map of the sample vehicle includes the annotation information of the center point of the sample vehicle and the annotation information of the grounding points of the wheels of the sample vehicle;

[0068] Input the image of the sample vehicle into the wheel grounding point prediction model to be trained, obtain the grounding points of each wheel of the sample vehicle, and the deviation values between the center point of the sample vehicle and the grounding points of each wheel of the sample vehicle;

[0069] Calculate the position coordinates of the predicted center point of the sample vehicle based on the grounding points of each wheel of the sample vehicle and the deviation values between the center point of the sample vehicle and the grounding points of each wheel of the sample vehicle;

[0070] Calculate the loss function value of the current round of training based on the position coordinates of the center point of the sample vehicle and the position coordinates of the predicted center point of the sample vehicle; wherein, the position coordinates of the center point of the sample vehicle are obtained from the annotation information of the center point of the sample vehicle in the annotation map of the sample vehicle;

[0071] In the case where the change value between the loss function value of the current round of training and the loss function value of the previous round of training is greater than a preset threshold, continue the training process, or in the case where the change value between the loss function value of the current round of training and the loss function value of the previous round of training is less than or equal to the preset threshold and the loss function value of the current round of training tends to be stable, end the training.

[0072] In this embodiment, the corner matching method is used to judge the training effect of the model. The corner matching originally refers to finding the corresponding relationship of the feature pixel points between two images to determine the position relationship between the two images.

[0073] In this embodiment, corner matching is applied to the training process of the wheel grounding point detection model. Specifically, a deviation value between the predicted wheel grounding point calculated according to the wheel grounding point detection model to be trained and the center point of the vehicle (this deviation value is data with 8 channels) is calculated, and the position coordinates of the predicted center point of the sample vehicle are calculated based on this deviation value; the loss function value is calculated according to the position coordinates of the predicted center point of the sample vehicle and the actual center point position coordinates of the sample vehicle; if the loss function value is large (indicating that the deviation value between the position coordinates of the predicted center point of the sample vehicle and the actual center point position coordinates of the sample vehicle is large), it means that the current wheel grounding point detection model is not perfect enough and the parameters in the model need to be adjusted. If the loss function value is within the preset threshold range, it means that the termination condition of the model training is reached, and the model obtained at this time is the wheel grounding point detection model that can be put into practical application.

[0074] The wheel grounding point detection method provided by the present invention, through the wheel grounding point prediction model, based on the collected image information containing the target vehicle, introduces an optimized corner matching method, making the detection accuracy of the wheel grounding point more accurate, and providing a more robust result for both the subsequent prediction of the vehicle's own posture and subsequent tasks.

[0075] The detection device for the wheel grounding point provided by the present invention will be described below. The detection device for the wheel grounding point described below can be correspondingly referred to the detection method for the wheel grounding point described above.

[0076] Figure 2 It is a structural diagram of the detection device for the wheel grounding point provided by the present invention. As Figure 2 shown, the detection device for the wheel grounding point provided by the present invention includes:

[0077] A target vehicle information acquisition module 210, configured to acquire the original image of the target vehicle;

[0078] A wheel grounding point prediction module 220, configured to input the image including the target vehicle into a pre-trained wheel grounding point prediction model to obtain the grounding points of each wheel of the target vehicle; wherein, the wheel grounding point prediction model is trained based on the images of sample vehicles.

[0079] The detection device for the wheel grounding point provided by the present invention, through the target vehicle information acquisition module 210 and the wheel grounding point prediction module 220, makes the detection accuracy of the wheel grounding point more accurate, and provides a relatively robust result for both the subsequent prediction of the vehicle's own posture and subsequent tasks.

[0080] Based on any of the above embodiments, in this embodiment, according to a detection device for a wheel grounding point provided by the present invention, the device further includes: an image preprocessing layer unit, an approximate grounding point acquisition layer unit, a candidate grounding point set acquisition layer unit, and a grounding point determination layer unit.

[0081] Among them, the image preprocessing layer unit is configured to, according to the image including the target vehicle, acquire the heat map of the grounding points of each wheel of the target vehicle, the position coordinates of the center point of the target vehicle, and the deviation values between the center point of the target vehicle and the grounding points of each wheel of the target vehicle; the approximate grounding point acquisition layer unit is configured to, according to the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the grounding points of each wheel of the target vehicle, acquire the approximate grounding points of each wheel of the target vehicle; the candidate grounding point set acquisition layer unit is configured to, according to the heat map of the grounding points of each wheel of the target vehicle, acquire the candidate grounding point set of each wheel of the target vehicle; the grounding point determination layer unit is configured to, according to the distances between the approximate grounding points of each wheel of the target vehicle and each candidate grounding point in the candidate grounding point set of each wheel of the target vehicle, determine the grounding points of each wheel of the target vehicle.

[0082] Based on any of the above embodiments, in this embodiment, according to a detection device for a wheel grounding point provided by the present invention, the candidate grounding point set acquisition layer unit further includes:

[0083] The first processing layer subunit of the candidate ground contact point set acquisition layer is configured to input the heat maps of the ground contact points of each wheel of the target vehicle into the first processing layer of the candidate ground contact point set acquisition layer, and obtain the initial values of the candidate ground contact points of each wheel of the target vehicle output by the first processing layer;

[0084] The second processing layer subunit of the candidate ground contact point set acquisition layer is configured to input the initial values of the candidate ground contact points of each wheel of the target vehicle and the quantization errors corresponding to each wheel of the target vehicle into the second processing layer of the candidate ground contact point set acquisition layer, and obtain the candidate ground contact point sets of each wheel of the target vehicle.

[0085] Based on any of the above embodiments, in this embodiment, according to a wheel ground contact point detection device provided by the present invention, the ground contact point determination layer unit further includes:

[0086] The distance calculation subunit of the ground contact point determination layer is configured to input the approximate ground contact points of each wheel of the target vehicle and the candidate ground contact point sets of each wheel of the target vehicle into the distance calculation layer of the ground contact point determination layer, and obtain the distances between the approximate ground contact points of each wheel of the target vehicle and each candidate ground contact point in the candidate ground contact point sets of each wheel of the target vehicle output by the distance calculation layer;

[0087] The judgment subunit of the ground contact point determination layer is configured to input the distances between the approximate ground contact points of each wheel of the target vehicle and each candidate ground contact point in the candidate ground contact point sets of each wheel of the target vehicle output by the distance calculation layer into the judgment layer of the ground contact point determination layer, and obtain the ground contact points of each wheel of the target vehicle output by the judgment layer; wherein, constraint conditions are preset in the judgment layer.

[0088] Based on any of the above embodiments, in this embodiment, according to a wheel ground contact point detection device provided by the present invention, it further includes:

[0089] The information acquisition unit of the sample vehicle is configured to acquire the labeled map of the sample vehicle and acquire the heat maps of the ground contact points of each wheel of the sample vehicle; wherein, the labeled map of the sample vehicle includes the labeled information of the center point of the sample vehicle and the labeled information of the ground contact points of the wheels of the sample vehicle;

[0090] The wheel ground contact point prediction model training unit is configured to input the image of the sample vehicle into the wheel ground contact point prediction model to be trained, and obtain the ground contact points of each wheel of the sample vehicle and the deviation values between the center point of the sample vehicle and the ground contact points of each wheel of the sample vehicle;

[0091] The wheel ground contact point prediction model training result judgment unit is used to calculate the position coordinates of the predicted center point of the sample vehicle according to the ground contact points of each wheel of the sample vehicle and the deviation values between the center point of the sample vehicle and the ground contact points of each wheel of the sample vehicle; calculate the loss function value of this round of training according to the position coordinates of the center point of the sample vehicle and the position coordinates of the predicted center point of the sample vehicle; among them, the position coordinates of the center point of the sample vehicle are obtained from the annotation information of the center point of the sample vehicle in the annotation map of the sample vehicle; when the change value between the loss function value of this round of training and the loss function value of the previous round of training is greater than the preset threshold, continue the training process, or when the change value between the loss function value of this round of training and the loss function value of the previous round of training is less than or equal to the preset threshold, and the loss function value of this round of training tends to be stable, end the training.

[0092] On the other hand, the present invention also provides an electronic device, Figure 3 which exemplifies a schematic physical structure diagram of an electronic device, as Figure 3 shown. The electronic device may include a processor 310, a communication bus 340, a memory 330, a communication interface 320, and a computer program stored on the memory 330 and executable on the processor 310. Among them, the processor 310, the communication interface 320, and the memory 330 complete mutual communication through the communication bus 340. The processor 310 can call the logical instructions in the memory 330 to execute the method for detecting the wheel ground contact point, and this method includes:

[0093] Obtain an image containing the target vehicle;

[0094] Input the image containing the target vehicle into a pre-trained wheel ground contact point prediction model to obtain the ground contact points of each wheel of the target vehicle; among them, the wheel ground contact point prediction model is trained based on the images of the sample vehicle.

[0095] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting the wheel ground contact point can be realized, and this method includes:

[0096] Obtain an image containing the target vehicle;

[0097] Input the image containing the target vehicle into a pre-trained wheel ground contact point prediction model to obtain the ground contact points of each wheel of the target vehicle; among them, the wheel ground contact point prediction model is trained based on the images of the sample vehicle.

[0098] Finally, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can implement the method for detecting the wheel contact point, and the method includes:

[0099] Obtain an image including the target vehicle;

[0100] Input the image including the target vehicle into a pre-trained wheel contact point prediction model to obtain the contact points of each wheel of the target vehicle; wherein, the wheel contact point prediction model is trained based on the images of sample vehicles.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the wheel contact points, characterized in that, it includes: obtaining an image containing the target vehicle; inputting the image containing the target vehicle into a pre-trained wheel contact point prediction model to obtain the contact points of each wheel of the target vehicle; wherein, the wheel contact point prediction model is trained based on the images of sample vehicles; the wheel contact point prediction model includes an image preprocessing layer, an approximate contact point acquisition layer, a candidate contact point set acquisition layer, and a contact point determination layer; wherein, the image preprocessing layer is used to obtain the heat map of the contact points of each wheel of the target vehicle, the position coordinates of the center point of the target vehicle, and the deviation values between the center point of the target vehicle and the contact points of each wheel of the target vehicle according to the image containing the target vehicle; the approximate contact point acquisition layer is used to obtain the approximate contact points of each wheel of the target vehicle according to the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the contact points of each wheel of the target vehicle; the candidate contact point set acquisition layer is used to obtain the candidate contact point sets of each wheel of the target vehicle according to the heat map of the contact points of each wheel of the target vehicle; the contact point determination layer is used to determine the contact points of each wheel of the target vehicle according to the distances between the approximate contact points of each wheel of the target vehicle and each candidate contact point in the candidate contact point sets of each wheel of the target vehicle.

2. The method for detecting the wheel contact points according to claim 1, characterized in that, the step of inputting the image containing the target vehicle into a pre-trained wheel contact point prediction model to obtain the contact points of each wheel of the target vehicle includes: inputting the image containing the target vehicle into the image preprocessing layer to obtain the heat map of the contact points of each wheel of the target vehicle, the position coordinates of the center point of the target vehicle, and the deviation values between the center point of the target vehicle and the contact points of each wheel of the target vehicle; inputting the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the contact points of each wheel of the target vehicle into the approximate contact point acquisition layer to obtain the approximate contact points of each wheel of the target vehicle; inputting the heat map of the contact points of each wheel of the target vehicle into the candidate contact point set acquisition layer to obtain the candidate contact point sets of each wheel of the target vehicle; inputting the approximate contact points of each wheel of the target vehicle and the candidate contact point sets of each wheel of the target vehicle into the contact point determination layer to obtain the contact points of each wheel of the target vehicle.

3. The method for detecting the wheel contact points according to claim 2, characterized in that, the step of inputting the heat map of the contact points of each wheel of the target vehicle into the candidate contact point set acquisition layer to obtain the candidate contact point sets of each wheel of the target vehicle includes: inputting the heat map of the contact points of each wheel of the target vehicle into the first processing layer of the candidate contact point set acquisition layer to obtain the initial values of the candidate contact points of each wheel of the target vehicle output by the first processing layer; inputting the initial values of the candidate contact points of each wheel of the target vehicle and the quantization errors corresponding to each wheel of the target vehicle into the second processing layer of the candidate contact point set acquisition layer to obtain the candidate contact point sets of each wheel of the target vehicle.

4. The method for detecting the wheel contact points according to claim 2, It is characterized in that inputting the approximate ground contact points of each wheel of the target vehicle and the set of candidate ground contact points of each wheel of the target vehicle into the ground contact point determination layer to obtain the ground contact points of each wheel of the target vehicle, including: inputting the approximate ground contact points of each wheel of the target vehicle and the set of candidate ground contact points of each wheel of the target vehicle into the distance calculation layer of the ground contact point determination layer to obtain the distances between the approximate ground contact points of each wheel of the target vehicle output by the distance calculation layer and each candidate ground contact point in the set of candidate ground contact points of each wheel of the target vehicle; inputting the distances between the approximate ground contact points of each wheel of the target vehicle output by the distance calculation layer and each candidate ground contact point in the set of candidate ground contact points of each wheel of the target vehicle into the judgment layer of the ground contact point determination layer to obtain the ground contact points of each wheel of the target vehicle output by the judgment layer; wherein, the judgment layer is preset with constraint conditions.

5. The method for detecting the ground contact point of a wheel according to claim 1, It is characterized in that the method further includes: obtaining an annotated map of a sample vehicle and obtaining a heat map of the ground contact points of each wheel of the sample vehicle; wherein, the annotated map of the sample vehicle includes the annotated information of the center point of the sample vehicle and the annotated information of the ground contact points of the wheels of the sample vehicle; inputting the image of the sample vehicle into a wheel ground contact point prediction model to be trained to obtain the ground contact points of each wheel of the sample vehicle and the deviation values between the center point of the sample vehicle and the ground contact points of each wheel of the sample vehicle; calculating the position coordinates of the predicted center point of the sample vehicle according to the ground contact points of each wheel of the sample vehicle and the deviation values between the center point of the sample vehicle and the ground contact points of each wheel of the sample vehicle; calculating the loss function value of this round of training according to the position coordinates of the center point of the sample vehicle and the position coordinates of the predicted center point of the sample vehicle; wherein, the position coordinates of the center point of the sample vehicle are obtained from the annotated information of the center point of the sample vehicle in the annotated map of the sample vehicle; in the case that the change value between the loss function value of this round of training and the loss function value of the previous round of training is greater than a preset threshold, continuing the training process, or in the case that the change value between the loss function value of this round of training and the loss function value of the previous round of training is less than or equal to the preset threshold and the loss function value of this round of training tends to be stable, ending the training.

6. A device for detecting the ground contact point of a wheel, It is characterized in that including: a target vehicle information acquisition module for acquiring an image containing the target vehicle; a wheel ground contact point prediction module for inputting the image containing the target vehicle into a pre-trained wheel ground contact point prediction model to obtain the ground contact points of each wheel of the target vehicle, and the wheel ground contact point prediction model includes an image preprocessing layer, an approximate ground contact point acquisition layer, a candidate ground contact point set acquisition layer, and a ground contact point determination layer; wherein, the image preprocessing layer is used to obtain a heat map of the ground contact points of each wheel of the target vehicle, the position coordinates of the center point of the target vehicle, and the deviation values between the center point of the target vehicle and the ground contact points of each wheel of the target vehicle according to the image containing the target vehicle. The approximate ground contact point acquisition layer is configured to acquire the approximate ground contact points of each wheel of the target vehicle according to the position coordinates of the center point of the target vehicle and the deviation values between the center point of the target vehicle and the ground contact points of each wheel of the target vehicle; The candidate ground contact point set acquisition layer is configured to acquire the candidate ground contact point sets of each wheel of the target vehicle according to the heat maps of the ground contact points of each wheel of the target vehicle; The ground contact point determination layer is configured to determine the ground contact points of each wheel of the target vehicle according to the distances between the approximate ground contact points of each wheel of the target vehicle and each candidate ground contact point in the candidate ground contact point sets of each wheel of the target vehicle; Wherein, the ground contact point prediction model of the wheel is trained based on the images of the sample vehicle.

7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the program, the steps of the method for detecting the ground contact point of the wheel according to any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by the processor, the steps of the method for detecting the ground contact point of the wheel according to any one of claims 1 to 5 are implemented.

9. A computer program product, the computer program product comprising computer-executable instructions, wherein, when the instructions are executed, the steps of the method for detecting the ground contact point of the wheel according to any one of claims 1 to 5 are implemented.

Images