Wheel position determination method and device, electronic equipment and vehicle

Abstract

The invention discloses a wheel position determination method and device, electronic equipment and a vehicle. The method comprises the following steps: acquiring a vehicle image; inputting the vehicle image into a pre-trained orientation recognition network, wherein the pre-trained orientation recognition network is obtained by training by using a preset mixed loss function; the orientation recognition network is used for recognizing the types and grounding point coordinates of two wheels belonging to the same vehicle in the vehicle image, the types comprise a front wheel and a rear wheel, and the mixed loss function is used for constraining the angle between a connecting line between the grounding point coordinates of the front wheel and the grounding point coordinates of the rear wheel and a preset coordinate axis during training; and constraining the grounding point coordinate of each wheel, and constraining the category of each wheel. Therefore, according to the embodiment of the invention, the orientation recognition network obtained by training the angle loss function can more accurately recognize the grounding point coordinate of the wheel.

Description

Technical Field

[0001] The present application belongs to the technical field of vehicle identification, and in particular, relates to a method, device, electronic equipment and vehicle for determining wheel positions. Background Art

[0002] In the relevant wheel recognition technology, especially when identifying the coordinates of the wheel's contact point, the neural network used to identify the wheel often only uses the position loss function to constrain the coordinates of the wheel's contact point during training. In other words, only the position loss function is used to train the recognition of the predicted wheel's contact point coordinates.

[0003] However, this method still makes it difficult to accurately train the neural network used to identify the ground contact point coordinates, which results in the ground contact point coordinates of the wheel being identified using the neural network. The identified ground contact point coordinates are often not accurate enough and have a large gap with the actual ground contact point coordinates. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and vehicle for determining wheel position, which can identify more accurate wheel contact point coordinates by using an orientation recognition network trained with an angle loss function.

[0005] In a first aspect, an embodiment of the present application provides a method for determining a wheel position, the method comprising:

[0006] Acquire vehicle images;

[0007] Inputting the vehicle image into a pre-trained orientation recognition network, where the pre-trained orientation recognition network is trained using a preset hybrid loss function;

[0008] An orientation recognition network is used to identify the categories and ground contact point coordinates of two wheels belonging to the same vehicle in a vehicle image, where the categories include front wheels and rear wheels. A mixed loss function is used to constrain the angle between the line between the ground contact point coordinates of the front wheel and the ground contact point coordinates of the rear wheel and the preset coordinate axis, constrain the ground contact point coordinates of each wheel, and constrain the category of each wheel during training.

[0009] Furthermore, after using the orientation recognition network to identify the categories and ground contact point coordinates of two wheels belonging to the same vehicle in the vehicle image, the method further includes:

[0010] Determine whether the two wheels are of the same category;

[0011] When both wheels are of the same category, determining the direction of the vehicle to be a direction away from or toward a preset target position corresponding to each category;

[0012] When the two wheels are not of the same type, the direction from the ground contact point coordinates of the rear wheel to the ground contact point coordinates of the front wheel is regarded as the vehicle orientation.

[0013] Furthermore, before inputting the vehicle image into the pre-trained orientation recognition network, the method further includes:

[0014] Inputting the preset training image, the corresponding actual category, the actual ground contact point coordinates, and the actual angle between the line connecting the two actual ground contact point coordinates and the preset coordinate axis when the two wheels are not of the same actual category into the orientation recognition network to be trained;

[0015] Using the orientation recognition network to be trained to identify two wheels of the same vehicle in the training image, and output the predicted category and predicted grounding point coordinates of each wheel;

[0016] When the two wheels are not of the same category, calculating the predicted angle between the straight line between the predicted contact point coordinates of the two wheels and the coordinate axis;

[0017] The predicted category, predicted grounding point coordinates, predicted angle, corresponding actual category, corresponding actual grounding point coordinates and corresponding actual angle are input into the mixed loss function, and the mixed loss is calculated using the mixed loss function;

[0018] When the mixed loss reaches a convergence threshold of the mixed loss function, it is determined that the orientation recognition network to be trained completes training.

[0019] Furthermore, the mixed loss is calculated using the mixed loss function, including:

[0020] The mixed loss function is used to calculate the category difference loss between the predicted category and the actual category;

[0021] The hybrid loss function is used to calculate the position difference loss between the predicted ground point coordinates and the actual ground point coordinates;

[0022] The angle difference loss between the predicted angle and the actual angle is calculated using a mixed loss function;

[0023] The category difference loss, position difference loss and angle difference loss are weighted to obtain a mixed loss.

[0024] The mixed loss function includes a cross entropy loss function, and the predicted category is the category corresponding to the maximum probability among the probabilities of the orientation recognition network to be trained identifying the category of the wheel as each category;

[0025] Furthermore, the mixed loss function is used to calculate the category difference loss between the predicted category and the actual category, including:

[0026] The predicted probability that the orientation recognition network to be trained recognizes the wheel as the predicted category and the actual probability that the wheel is the corresponding actual category are both input into the cross entropy loss function;

[0027] The cross entropy loss function is used to calculate the difference between the predicted probability and the actual probability and use it as the category difference loss.

[0028] Wherein, the hybrid loss function includes a position loss function, and the position loss function includes a first position loss function and a second position loss function;

[0029] Furthermore, a hybrid loss function is used to calculate the position difference loss between the predicted ground point coordinates and the actual ground point coordinates, including:

[0030] Calculate the difference between the predicted grounding point coordinates and the actual grounding point coordinates;

[0031] When the difference is less than a preset difference limit, the predicted grounding point coordinates and the actual grounding point coordinates are input into a first position loss function, where the first position loss function is used to represent the degree of difference of the difference;

[0032] The first position loss function is used to calculate the difference between the predicted grounding point coordinates and the actual grounding point coordinates, and used as the position difference loss;

[0033] When the difference is greater than or equal to the difference limit, the predicted grounding point coordinates and the actual grounding point coordinates are input into a second position loss function, and the second position loss function is used to represent the degree of difference of the difference;

[0034] The second position loss function is used to calculate the difference between the predicted ground point coordinates and the actual ground point coordinates, and used as the position difference loss.

[0035] Among them, the hybrid loss function includes an angle loss function;

[0036] Furthermore, the angle difference loss between the predicted angle and the actual angle is calculated using a hybrid loss function, including:

[0037] Input the predicted angle and the actual angle into the angle loss function;

[0038] The angle loss function is used to calculate the difference between the predicted angle and the actual angle and use it as the angle difference loss.

[0039] In a second aspect, an embodiment of the present application provides a device for determining a wheel position, the device comprising:

[0040] An acquisition module, used for acquiring vehicle images;

[0041] An input module, used for inputting the vehicle image into a pre-trained orientation recognition network, wherein the pre-trained orientation recognition network is trained using a preset hybrid loss function;

[0042] The recognition module is used to use the orientation recognition network to identify the categories and ground contact point coordinates of two wheels belonging to the same vehicle in the vehicle image, wherein the categories include front wheels and rear wheels, and the mixed loss function is used to constrain the angle between the line between the ground contact point coordinates of the front wheel and the ground contact point coordinates of the rear wheel and the preset coordinate axis, constrain the ground contact point coordinates of each wheel, and constrain the category of each wheel during training.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising:

[0044] a processor and a memory storing computer program instructions;

[0045] When the processor executes the computer program instructions, the method for determining the wheel position as described in any of the above items is implemented.

[0046] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, a method for determining a wheel position as described in any of the above items is implemented.

[0047] In a fifth aspect, an embodiment of the present application provides a method for determining the wheel position as described in any of the above items, whereby when instructions in a computer program product are executed by a processor of an electronic device.

[0048] In a sixth aspect, an embodiment of the present application further provides a vehicle, the vehicle comprising a wheel position determination device or electronic device, the electronic device executing any one of the wheel position determination methods as described above.

[0049] The wheel position determination method, device, electronic device and vehicle of the embodiments of the present application identify the category of the input vehicle image and the coordinates of the wheel's contact point based on a pre-trained orientation recognition network, wherein the pre-trained orientation recognition network is trained using an angle loss function, and the angle loss function is specifically used to constrain the angle between the line between the contact point coordinates of the front wheel and the contact point coordinates of the rear wheel and the preset coordinate axis when the two wheels of any vehicle are of different categories. Therefore, the angle loss function can be considered to constrain the contact point coordinates from the angle dimension. Accordingly, the orientation recognition network trained using the angle loss function can identify more accurate contact point coordinates. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 is a flowchart of a method for determining a wheel position provided in an embodiment of the present application;

[0052] Figure 2 is a schematic diagram of a process for identifying a vehicle orientation provided in an embodiment of the present application;

[0053] Figure 3 is a schematic diagram of a vehicle heading towards a target location provided in an embodiment of the present application;

[0054] Figure 4 is a schematic diagram of a vehicle heading away from a target position provided in an embodiment of the present application;

[0055] Figure 5 is a schematic diagram of a vehicle oriented sideways provided in an embodiment of the present application;

[0056] Figure 6 It is a flowchart of training a direction recognition network provided in an embodiment of the present application;

[0057] Figure 7 is a structural schematic diagram of a device for determining a wheel position provided in an embodiment of the present application;

[0058] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0060] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0061] As described in the background technology section, the related wheel position determination technology is still difficult to meet the needs in actual use.

[0062] In the process of implementing the present application, the applicant discovered that in the relevant wheel recognition technology, especially when identifying the coordinates of the wheel's contact point, when training the neural network used to identify the wheel, it is often only the position loss function that is used to constrain the coordinates of the wheel's contact point. In other words, only the position loss function is used to train the recognition of the predicted wheel's contact point coordinates.

[0063] However, this method still makes it difficult to accurately train the neural network used to identify the ground contact point coordinates, which results in the ground contact point coordinates of the wheel being identified using the neural network. The identified ground contact point coordinates are often not accurate enough and have a large gap with the actual ground contact point coordinates.

[0064] In order to solve the problems of the prior art, the embodiments of the present application provide a method, device, electronic device and vehicle for determining the wheel position. Based on the acquired vehicle image, the vehicle image can be input into a pre-trained orientation recognition network, and the pre-trained orientation recognition network can be used to identify the position of each wheel of the vehicle.

[0065] The method for determining the wheel position provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0066] refer to Figure 1 A method for determining a wheel position according to an embodiment of the present application comprises the following steps:

[0067] Step S101: Acquire a vehicle image.

[0068] In this step, an image acquisition device pre-installed in the vehicle may be used to acquire the vehicle image in the driving environment.

[0069] The vehicle image may specifically be, for example, an image of the driving environment ahead taken by the autonomous driving vehicle, and the image may include at least one other vehicle.

[0070] In some embodiments, the vehicle image to be identified may be captured at a predetermined target location, or may be a vehicle image input from the cloud or cloud platform, or other remote control platforms.

[0071] Step S102: input the vehicle image into a pre-trained orientation recognition network, wherein the pre-trained orientation recognition network is trained using a preset hybrid loss function.

[0072] In this step, the vehicle image obtained in the previous step is input into a pre-trained orientation recognition network.

[0073] The pre-trained orientation recognition network may be specifically trained using an angle loss function.

[0074] Specifically, the pre-trained orientation recognition network can be trained using a hybrid loss function alone.

[0075] The hybrid loss function specifically includes multiple angle loss functions, position loss functions and category loss functions.

[0076] Based on this, the orientation recognition network trained using the mixed loss function can obtain more accurate ground contact point coordinates when identifying the ground contact point coordinates of the wheel in the following steps.

[0077] Step S103, using the orientation recognition network to identify the categories and ground contact point coordinates of two wheels belonging to the same vehicle in the vehicle image, the categories including front wheels and rear wheels, and the mixed loss function is used to constrain the angle between the line between the ground contact point coordinates of the front wheel and the ground contact point coordinates of the rear wheel and the preset coordinate axis, constrain the ground contact point coordinates of each wheel, and constrain the category of each wheel during training.

[0078] In this step, after the vehicle image is input into the pre-trained orientation recognition network, the orientation recognition network can be used to recognize two wheels belonging to the same vehicle in the vehicle image.

[0079] When identifying the wheels of each vehicle in the vehicle image, for each vehicle, only the two wheels closest to a preset target position are identified. The target position may be, for example, the position where the vehicle image is taken.

[0080] In this embodiment, based on the above-identified wheels of each vehicle, a pre-trained orientation recognition network may be further used to identify the category and contact point coordinates of each wheel.

[0081] Among them, the position where the vehicle's wheel contacts the road surface is taken as the grounding point, and the coordinates of the grounding point in the pixel coordinate system of the vehicle image or in other coordinate systems are taken as the grounding point coordinates; the wheel categories may specifically include front wheels and rear wheels; the preset coordinate axis may be, for example, any coordinate axis in the pixel coordinate system of the vehicle image, or other preset coordinate axes.

[0082] In this embodiment, the orientation recognition network for identifying the coordinates of the touchdown point is trained using a mixed loss function. When the mixed loss function includes an angle loss function, since the angle loss function is specifically used to constrain the angle between the line connecting the coordinates of the two touchdown points and the preset coordinate axis when the two wheels do not belong to the same category, the mixed loss function including the angle loss function can constrain the coordinates of the touchdown point.

[0083] Furthermore, when the above-mentioned hybrid loss function also includes a position loss function that directly constrains the touchdown point coordinates, it can be considered that when training the orientation recognition network, in addition to using the position loss function to constrain the coordinates of each identified touchdown point, the touchdown point coordinates are also constrained from the angle dimension, so that the trained orientation recognition network can more accurately identify the touchdown point coordinates of each wheel.

[0084] Furthermore, when the above-mentioned mixed loss function also includes a category loss function for constraining the wheel category, it can be considered that when training the orientation recognition network, for each identified wheel, in addition to using the position loss function and the angle loss function to constrain the docking point coordinates, the wheel category is also constrained from the category dimension, so that the trained orientation recognition network can more accurately identify the wheel category corresponding to each grounding point coordinate.

[0085] In another embodiment of the present application, based on the above-identified coordinates of the contact points of the two wheels of the same vehicle, the orientation of the vehicle can be further identified.

[0086] refer to Figure 2 In this embodiment, when identifying the direction of the vehicle, the following steps are specifically included:

[0087] Step S201: Determine whether the two wheels are of the same category.

[0088] In this step, for the two wheels of any vehicle, based on the categories of each wheel determined in the above-mentioned embodiment, it can be determined whether the categories of the two wheels belong to the same category.

[0089] Step S202: When both wheels are of the same category, determine that the direction of the vehicle is a direction away from or towards a preset target position corresponding to each category.

[0090] In this step, based on the judgment in the previous step, when the judgment result is the same category, the corresponding direction is determined according to the category of the wheel.

[0091] Specifically, for any vehicle, when the two identified wheels are of the same category, the vehicle is considered to be facing the target position or close to facing the target position, and the specific direction of the vehicle can be determined based on the category of the wheels.

[0092] Specifically, Figure 3 A schematic diagram showing a vehicle heading toward a target location is shown.

[0093] like Figure 3 As shown, when the categories of the two wheels of any vehicle are both front wheels, it is considered that the front of the vehicle is facing the target position, or is close to facing the target position, so that the direction of the vehicle is determined to be the direction towards the target position.

[0094] Specifically, Figure 4 A schematic diagram showing a vehicle heading away from a target position is shown.

[0095] like Figure 4 As shown, when the categories of both wheels of any vehicle are rear wheels, it is considered that the front and rear of the vehicle are facing the target position, or are close to facing the target position, so that the direction of the vehicle is determined to be heading towards the direction away from the target position.

[0096] Step S203: When the two wheels are not of the same type, the direction from the contact point coordinates of the rear wheel to the contact point coordinates of the front wheel is used as the orientation of the vehicle.

[0097] In this step, based on the judgment in the previous step, for any vehicle, when the two identified wheels are not of the same category, the orientation of the vehicle can be determined based on the contact point coordinates of each wheel determined above.

[0098] Specifically, Figure 5 A schematic diagram showing a vehicle oriented sideways is shown.

[0099] like Figure 5 As shown, for any vehicle with two wheels of different categories, after the categories of each wheel are determined, the contact point coordinates of the front wheel and the contact point coordinates of the rear wheel are further determined.

[0100] Furthermore, the contact point coordinates of the front wheel and the contact point coordinates of the rear wheel are connected by a straight line, and the orientation of the vehicle is determined as a direction from the contact point coordinates of the rear wheel to the contact point coordinates of the front wheel along the straight line.

[0101] It can be seen that the orientation recognition network trained based on the angle loss function can be more accurate in identifying the coordinates of the wheel's contact point. Furthermore, the straight line direction obtained using the coordinates of the front wheel's contact point and the rear wheel's contact point is also more accurate, so that a more accurate vehicle orientation can be predicted.

[0102] In another embodiment of the present application, before inputting the vehicle image into the pre-trained orientation recognition network, the orientation recognition network to be trained needs to be trained to obtain the pre-trained orientation recognition network.

[0103] refer to Figure 6 In this embodiment, when training the orientation recognition network, the following steps are specifically included:

[0104] Step S601: input a preset training image, a corresponding actual category, actual contact point coordinates, and an actual angle between a line connecting two actual contact point coordinates and a preset coordinate axis when two wheels are not of the same actual category into a direction recognition network to be trained.

[0105] In this embodiment, in order to train the orientation recognition network to be trained, it is necessary to first collect a plurality of training images for training, and obtain an actual label corresponding to each training image.

[0106] A plurality of training images containing vehicles are collected, and for each training image, the actual category and actual contact point coordinates of the wheels of each vehicle therein are annotated, and the actual angles of the two actual contact points of each vehicle are also annotated.

[0107] Furthermore, in each training image, when labeling, for any vehicle, only two wheels closest to the location where the training image was taken are labeled.

[0108] Furthermore, when marking the actual angle for each vehicle, the actual contact point coordinates of the two wheels of the vehicle can be connected with a straight line only when the two wheels of the vehicle do not belong to the same actual category, and the angle between the preset coordinate axis and the straight line can be used as the actual angle.

[0109] In some embodiments, in addition to marking the actual angle of the vehicle when the two wheels of the vehicle do not belong to the same actual category, when the two wheels of the vehicle belong to the same actual category, the actual angle of the vehicle may be regarded as 0 or other preset values.

[0110] Based on this, for each training image, an actual label of each vehicle in the training image is obtained, and the actual label includes the actual category, actual contact point and actual angle of the two wheels mentioned above.

[0111] Furthermore, each training image and its corresponding actual label are input into the orientation recognition network to be trained, so as to perform multiple rounds of training on it in the following steps.

[0112] Step S602: using the orientation recognition network to be trained to identify two wheels of the same vehicle in the training image, and outputting the predicted category and predicted ground contact point coordinates of each wheel.

[0113] In this embodiment, in each round of training, after each training image is input into the orientation recognition network to be trained, the orientation of each vehicle in each training image is predicted respectively, and the corresponding actual label is used to evaluate the prediction result of this round, so as to adjust the parameters of the orientation recognition network to be trained according to the evaluation result for the next round of training.

[0114] In this embodiment, in each round of training, after each training image is input into the orientation recognition network to be trained, for each vehicle in each training image, the orientation recognition network to be trained is used to identify two wheels of the vehicle.

[0115] Furthermore, the predicted category and predicted ground contact point coordinates of each wheel are identified for the two obtained wheels.

[0116] When the orientation recognition network to be trained identifies the predicted category of the wheel, based on the two pre-set categories of the front wheel and the rear wheel, the orientation recognition network to be trained can obtain the predicted probabilities of the two categories after identification.

[0117] That is, the probability of identifying the wheel as the front wheel and the probability of identifying the vehicle as the rear wheel are obtained.

[0118] Based on this, the orientation recognition network to be trained can use the category corresponding to the highest probability among the various probabilities as the output prediction category, and use the predicted highest probability as the actual probability.

[0119] Furthermore, based on the above-obtained probabilities, the probability corresponding to the actual category is determined and used as the actual probability.

[0120] Step S603: When the two wheels are not of the same category, calculate the predicted angle between the straight line between the predicted contact point coordinates of the two wheels and the coordinate axis.

[0121] Based on this, it can be determined whether the predicted categories of the two wheels are the same category.

[0122] Further, when it is determined that the predicted categories of the two wheels are not the same category, the angle between the line connecting the two wheels and the preset coordinate axis is calculated.

[0123] Specifically, the predicted contact point coordinates of the two wheels are connected by a straight line, and the angle between the straight line and the preset coordinate axis is determined.

[0124] Among them, the preset coordinate axis can be, for example, any coordinate axis in the pixel coordinate system of the training image, for example, the x-axis (horizontal axis) in the pixel coordinate system, and based on this, the angle between the straight line between the two predicted ground point coordinates and the x-axis is determined and used as the predicted angle.

[0125] In some embodiments, when marking the actual type of the wheel and the actual contact point coordinates, the actual angle may not be marked first, and the actual angle is calculated using the actual contact point coordinates in each round of training.

[0126] Specifically, in each round of training, before the following step S604, for each vehicle whose two wheels do not belong to the same actual category in the training image, a line connecting the coordinates of the two actual ground contact points is determined, and the angle between the line and the x-axis is calculated and used as the actual angle corresponding to the vehicle.

[0127] In some embodiments, when the two wheels of the vehicle belong to the same actual category, the actual angle of the vehicle is regarded as 0 or other preset values.

[0128] Based on this, for each vehicle in each training image, the actual category, actual contact point and actual angle of the two wheels are determined as the actual label of the vehicle.

[0129] Step S604: input the predicted category, the predicted grounding point coordinates, the predicted angle, the corresponding actual category, the corresponding actual grounding point coordinates and the corresponding actual angle into the mixed loss function, and use the mixed loss function to calculate the mixed loss.

[0130] In this embodiment, a mixed loss function is set in the orientation recognition network to be trained. According to this, in each round of training, when evaluating the output prediction results, the mixed loss function can be used to calculate the mixed loss of the output prediction results, and the orientation recognition network to be trained can be adjusted according to the calculated mixed loss.

[0131] Specifically, based on the identification of the wheel category in the aforementioned step, when the predicted categories of the two wheels are not the same category, the mixed loss can be calculated using a preset mixed loss function based on the above prediction results.

[0132] When calculating the mixed loss, the predicted category, predicted grounding point coordinates and predicted angle obtained above, as well as the corresponding actual category, actual grounding point coordinates and actual angle can be input into a preset mixed loss function, and the mixed loss can be calculated using the mixed loss function.

[0133] In a specific example, the hybrid loss function specifically includes: a category loss function, a position loss function and an angle loss function.

[0134] Based on this, the category loss function can be used to calculate the category difference loss between the predicted category and the corresponding actual category; the position loss function can be used to calculate the position difference loss between the predicted grounding point coordinates and the corresponding actual grounding point coordinates; the angle loss function can be used to calculate the angle difference loss between the predicted angle and the corresponding actual angle.

[0135] Furthermore, the hybrid loss function can be used to weight the above-mentioned category difference loss, position difference loss and angle difference loss to obtain a hybrid loss.

[0136] In some specific examples, when calculating the category difference loss using the category loss function, based on the probability of identifying the wheel as each category, the category corresponding to the predicted probability can be output as the predicted category, and the highest probability can be used as the predicted probability, and the probability corresponding to the actual category can be used as the actual probability.

[0137] Based on this, we can input both the predicted probability and the actual probability into the cross entropy loss function, and then use the cross entropy loss function to calculate the cross entropy loss, that is, the degree of difference between the predicted category and the actual category, and use the cross entropy loss as the category difference loss.

[0138] Specifically, the category difference loss can be calculated according to the cross entropy loss function formula (1) shown below:

[0139]

[0140] Among them, L cls represents the calculated category difference loss, y represents the predicted probability, represents the actual probability.

[0141] It can be seen that the cross entropy loss function specifically characterizes the degree of difference between the predicted category and the actual category. Based on this, the category difference loss of the orientation recognition model to be trained when predicting each wheel category in this round of training can be obtained.

[0142] In some specific examples, when calculating the category difference loss using the position loss function, the difference between the predicted ground point coordinates and the actual ground point coordinates can be calculated first, and the position difference loss can be calculated based on the difference.

[0143] Specifically, the position difference loss can be calculated according to the position loss function formula (2) shown below:

[0144]

[0145] Among them, L loc Represents category difference loss, smooth L1 represents a regression loss function, s represents the predicted ground point coordinates, Indicates the actual grounding point coordinates.

[0146] Among them, based on the x-axis (horizontal axis) and y-axis (vertical axis) in the pixel coordinate system of the training image, the difference between the predicted grounding point coordinates and the actual grounding point coordinates on the x-axis, that is, the difference in the horizontal axis, can be used as the difference between the predicted grounding point coordinates and the actual grounding point coordinates; the difference between the predicted grounding point coordinates and the actual grounding point coordinates on the y-axis, that is, the difference in the vertical axis, can be used as the difference between the predicted grounding point coordinates and the actual grounding point coordinates; the predicted grounding point coordinates and the actual grounding point coordinates can also be used as two-dimensional vectors respectively, and the difference between the two two-dimensional vectors can be used as the difference between the predicted grounding point coordinates and the actual grounding point coordinates; the difference between the predicted grounding point coordinates and the actual grounding point coordinates can also be calculated according to other calculation methods.

[0147] In this embodiment, the position loss function includes a first position loss function and a second position loss function, both of which characterize the degree of difference between the actual grounding point coordinates and the predicted grounding point coordinates. Therefore, the first position loss function or the second position loss function can be selected according to the difference.

[0148] Specifically, when the difference between the actual grounding point coordinates and the predicted grounding point coordinates is small, it is necessary to use the first position loss function to exponentially reduce the difference. When the difference between the actual grounding point coordinates and the predicted grounding point coordinates is average or large, the difference does not need to be exponentially adjusted.

[0149] In this embodiment, 1 can be used as the difference limit. When the absolute value of the difference between the actual grounding point coordinates and the predicted grounding point coordinates is less than 1, the difference is considered to be small; when the absolute value of the difference between the actual grounding point coordinates and the predicted grounding point coordinates is greater than or equal to 1, the difference is considered to be average or large.

[0150] In some specific examples, when the difference between the actual grounding point coordinates and the predicted grounding point coordinates is less than 1, the difference between the actual grounding point coordinates and the predicted grounding point coordinates, that is, the position difference loss, can be calculated according to the first position loss function formula (3) shown below:

[0151]

[0152] When the difference between the actual grounding point coordinates and the predicted grounding point coordinates is greater than or equal to 1, the difference between the actual grounding point coordinates and the predicted grounding point coordinates, i.e., the position difference loss, can be calculated according to the second position loss function formula (4) shown below:

[0153]

[0154] in, It represents the difference between the predicted grounding point coordinates and the actual grounding point coordinates.

[0155] It can be seen that the position loss function specifically characterizes the degree of difference between the predicted grounding point coordinates and the actual grounding point coordinates. Based on this, the position difference loss of the orientation recognition model to be trained when predicting the grounding point coordinates of each wheel in this round of training can be obtained.

[0156] In some specific examples, when calculating the category difference loss using the angle loss function, for each vehicle, based on the obtained predicted angle and the actual angle acquired in advance, the tangent trigonometric function can be used to quantify the difference between the predicted angle and the actual angle.

[0157] The actual angle specifically represents the angle between the straight line and the preset coordinate axis x-axis after the actual contact point coordinates of the front wheel of the vehicle and the actual contact point coordinates of the rear wheel are connected by a straight line.

[0158] Specifically, the angle difference between the predicted angle and the actual angle may be calculated, and the tangent trigonometric function of the angle difference may be calculated, and the calculation result may be used as the angle difference loss representing the difference between the predicted angle and the actual angle.

[0159] In a specific example, the angle difference loss can be calculated according to the angle loss function formula (4) shown below:

[0160]

[0161] Among them, L angle represents the calculated angle difference loss, θ represents the predicted angle, Indicates the actual angle.

[0162] It can be seen that the angle loss function specifically characterizes the degree of difference between the predicted angle and the actual class angle. Based on this, the angle difference loss of the orientation recognition model to be trained when predicting each vehicle in this round of training can be obtained.

[0163] Based on this, the difference weights can be set for the category difference loss, position difference loss and angle difference loss determined above, and used as hyperparameters in the orientation recognition network to be trained. The category difference loss, position difference loss and angle difference loss can be weighted using their respective difference weights, and the weighted result can be used as a mixed loss.

[0164] In a specific example, the category difference loss, position difference loss and angle difference loss can be weighted according to the following formula (5):

[0165] L=w1×L cls +w2×L loc +w3×L angle (5)

[0166] Among them, L represents the mixed loss, w1 represents the category difference weight corresponding to the category difference loss, w2 represents the position difference weight corresponding to the position difference loss, and w3 represents the angle difference weight corresponding to the angle difference loss.

[0167] In some embodiments, when it is determined that the predicted categories of the two wheels are the same category, the angle difference loss can be set to 0 when performing the mixed loss calculation, or the predicted angle can be set to 0 or other preset values ​​so that the difference between the predicted angle and the actual angle is 0.

[0168] Based on this, when the predicted categories of the two wheels are the same, since the angle difference loss is 0, the mixed loss can be calculated using the predicted predicted category and predicted ground contact point coordinates according to the above formula (4).

[0169] Step S605: When the mixed loss reaches a convergence threshold of the mixed loss function, it is determined that the orientation recognition network to be trained has completed training.

[0170] In this embodiment, based on the mixed loss obtained in the aforementioned steps, a preset convergence threshold can be used to judge the mixed loss obtained, thereby determining whether the orientation recognition network to be trained in each round of training is trained.

[0171] Specifically, when the obtained mixed loss is less than or equal to the convergence threshold, it is considered that the orientation recognition network to be trained has completed training, and when the obtained mixed loss is greater than the convergence threshold, it is considered that the orientation recognition network to be trained has not completed training.

[0172] Furthermore, when the obtained mixed loss is greater than the convergence threshold, the parameters in the orientation recognition network to be trained are adjusted according to the obtained mixed loss, thereby obtaining an updated orientation recognition network.

[0173] Furthermore, each training image and the corresponding actual label are input into the updated orientation recognition network, and the next round of training is performed until the mixed loss obtained by training is less than or equal to the above-mentioned convergence threshold, thereby obtaining a trained orientation recognition network.

[0174] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, an embodiment of the present application further provides a device for determining the position of a wheel.

[0175] refer to Figure 7 , the wheel position determination device comprises:

[0176] An acquisition module 701 is used to acquire a vehicle image;

[0177] An input module 702 is used to input the vehicle image into a pre-trained orientation recognition network, where the pre-trained orientation recognition network is trained using a preset hybrid loss function;

[0178] The identification module 703 is used to use the orientation recognition network to identify the categories and ground contact point coordinates of two wheels belonging to the same vehicle in the vehicle image, where the categories include front wheels and rear wheels. The mixed loss function is used to constrain the angle between the line between the ground contact point coordinates of the front wheel and the ground contact point coordinates of the rear wheel and the preset coordinate axis, constrain the ground contact point coordinates of each wheel, and constrain the category of each wheel during training.

[0179] In one embodiment, the input module 702 is specifically used for:

[0180] Before feeding the vehicle image into the pre-trained orientation recognition network, execute:

[0181] Inputting the preset training image, the corresponding actual category, the actual ground contact point coordinates, and the actual angle between the line connecting the two actual ground contact point coordinates and the preset coordinate axis when the two wheels are not of the same actual category into the orientation recognition network to be trained;

[0182] Using the orientation recognition network to be trained to identify two wheels of the same vehicle in the training image, and output the predicted category and predicted grounding point coordinates of each wheel;

[0183] When the two wheels are not of the same category, calculating the predicted angle between the straight line between the predicted contact point coordinates of the two wheels and the coordinate axis;

[0184] The predicted category, predicted grounding point coordinates, predicted angle, corresponding actual category, corresponding actual grounding point coordinates and corresponding actual angle are input into the mixed loss function, and the mixed loss is calculated using the mixed loss function;

[0185] When the mixed loss reaches a convergence threshold of the mixed loss function, it is determined that the orientation recognition network to be trained completes training.

[0186] Among them, the mixed loss is calculated using the mixed loss function, including:

[0187] The mixed loss function is used to calculate the category difference loss between the predicted category and the actual category;

[0188] The hybrid loss function is used to calculate the position difference loss between the predicted ground point coordinates and the actual ground point coordinates;

[0189] The angle difference loss between the predicted angle and the actual angle is calculated using a mixed loss function;

[0190] The category difference loss, position difference loss and angle difference loss are weighted to obtain a mixed loss.

[0191] The mixed loss function includes a cross entropy loss function, and the predicted category is the category corresponding to the maximum probability among the probabilities of the orientation recognition network to be trained identifying the category of the wheel as various categories.

[0192] Furthermore, the mixed loss function is used to calculate the category difference loss between the predicted category and the actual category, including:

[0193] The predicted probability that the orientation recognition network to be trained recognizes the wheel as the predicted category and the actual probability that the wheel is the corresponding actual category are both input into the cross entropy loss function;

[0194] The cross entropy loss function is used to calculate the difference between the predicted probability and the actual probability and use it as the category difference loss.

[0195] The mixed loss function includes a position loss function, and the position loss function includes a first position loss function and a second position loss function.

[0196] Furthermore, a hybrid loss function is used to calculate the position difference loss between the predicted ground point coordinates and the actual ground point coordinates, including:

[0197] Calculate the difference between the predicted grounding point coordinates and the actual grounding point coordinates;

[0198] When the difference is less than a preset difference limit, the predicted grounding point coordinates and the actual grounding point coordinates are input into a first position loss function, where the first position loss function is used to represent the degree of difference of the difference;

[0199] The first position loss function is used to calculate the difference between the predicted grounding point coordinates and the actual grounding point coordinates, and used as the position difference loss;

[0200] When the difference is greater than or equal to the difference limit, the predicted grounding point coordinates and the actual grounding point coordinates are input into a second position loss function, and the second position loss function is used to represent the degree of difference of the difference;

[0201] The second position loss function is used to calculate the difference between the predicted ground point coordinates and the actual ground point coordinates, and used as the position difference loss.

[0202] Among them, the mixed loss function includes the angle loss function.

[0203] Furthermore, the angle difference loss between the predicted angle and the actual angle is calculated using a hybrid loss function, including:

[0204] Input the predicted angle and the actual angle into the angle loss function;

[0205] The angle loss function is used to calculate the difference between the predicted angle and the actual angle and use it as the angle difference loss.

[0206] In one embodiment, the identification module 703 is specifically used for:

[0207] After using the orientation recognition network to identify the categories and ground contact point coordinates of the two wheels belonging to the same vehicle in the vehicle image, perform:

[0208] Determine whether the two wheels are of the same category;

[0209] When both wheels are of the same category, determining the direction of the vehicle to be a direction away from or toward a preset target position corresponding to each category;

[0210] When the two wheels are not of the same type, the direction from the ground contact point coordinates of the rear wheel to the ground contact point coordinates of the front wheel is regarded as the vehicle orientation.

[0211] For the convenience of description, the above devices are described in terms of functions and are divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0212] The device of the above embodiment is used to implement the corresponding method for determining the wheel position in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0213] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for determining the wheel position of any of the above-mentioned embodiments is implemented.

[0214] Figure 8A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0215] The electronic device may include a processor 801 and a memory 802 storing computer program instructions.

[0216] Specifically, the processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0217] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 802 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory.

[0218] The memory 802 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0219] The processor 801 implements any one of the wheel position determination methods in the above embodiments by reading and executing computer program instructions stored in the memory 802 .

[0220] In one example, the electronic device may further include a communication interface 803 and a bus 810. Figure 8 As shown, the processor 801, the memory 802, and the communication interface 803 are connected via a bus 810 and communicate with each other.

[0221] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0222] Bus 810 includes hardware, software or both, and the components of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (Accelerated Graphics Port, AGP) or other graphics bus, enhanced industry standard architecture (Extended Industry Standard Architecture, EISA) bus, front-end bus (FSB), hypertransmission (Hyper Transport, HT) interconnection, industry standard architecture (IndustryStandard Architecture, ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 810 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0223] The electronic device can execute the method for determining the wheel position in the embodiment of the present application based on the orientation recognition network trained by the angle loss function.

[0224] In addition, in combination with the wheel position determination method in the above embodiment, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the wheel position determination methods in the above embodiment is implemented.

[0225] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, implements any one of the wheel position determination methods in the above embodiments.

[0226] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0227] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0228] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a vehicle, which includes a wheel position determination device and / or electronic device of any of the above-mentioned embodiments, and the electronic device executes any of the above-mentioned wheel position determination methods.

[0229] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0230] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0231] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for determining a wheel position, characterized in that: include: Acquire vehicle images; Inputting the vehicle image into a pre-trained orientation recognition network, wherein the pre-trained orientation recognition network is trained using a preset hybrid loss function; The orientation recognition network is used to identify the categories and contact point coordinates of two wheels belonging to the same vehicle in the vehicle image, wherein the categories include front wheels and rear wheels. The mixed loss function is used to constrain the angle between the line between the contact point coordinates of the front wheel and the contact point coordinates of the rear wheel and the preset coordinate axis, constrain the contact point coordinates of each wheel, and constrain the category of each wheel during training.

2. The method for determining the wheel position according to claim 1, characterized in that: After using the orientation recognition network to identify the categories and ground contact point coordinates of two wheels belonging to the same vehicle in the vehicle image, the method further includes: Determine whether the two wheels are of the same category; When both wheels are of the same category, determining the direction of the vehicle to be a direction away from or towards a preset target position corresponding to each category; When the two wheels are not of the same type, the direction from the ground contact point coordinates of the rear wheel to the ground contact point coordinates of the front wheel is taken as the orientation of the vehicle.

3. The method for determining the wheel position according to claim 1, characterized in that: Before inputting the vehicle image into a pre-trained orientation recognition network, the method further includes: Inputting the preset training image, the corresponding actual category, the actual ground contact point coordinates, and the actual angle between the line connecting the two actual ground contact point coordinates and the preset coordinate axis when the two wheels are not of the same actual category into the orientation recognition network to be trained; Using the orientation recognition network to be trained to identify two wheels of the same vehicle in the training image, and outputting the predicted category and predicted ground contact point coordinates of each wheel; When the two wheels are not of the same category, calculating a predicted angle between a straight line between the predicted contact point coordinates of the two wheels and the coordinate axis; The predicted category, the predicted grounding point coordinates, the predicted angle, the corresponding actual category, the corresponding actual grounding point coordinates and the corresponding actual angle are input into the mixed loss function, and the mixed loss is calculated using the mixed loss function; When the mixed loss reaches a convergence threshold of the mixed loss function, it is determined that the orientation recognition network to be trained completes training.

4. The method for determining the wheel position according to claim 3, characterized in that: The step of calculating the mixed loss using the mixed loss function includes: Calculating the category difference loss between the predicted category and the actual category using the mixed loss function; Calculating the position difference loss between the predicted ground point coordinates and the actual ground point coordinates using the hybrid loss function; Calculating the angle difference loss between the predicted angle and the actual angle using the hybrid loss function; The category difference loss, the position difference loss and the angle difference loss are weighted to obtain the mixed loss.

5. The method for determining the wheel position according to claim 4, characterized in that: The hybrid loss function includes a cross entropy loss function, the predicted category is a category corresponding to the maximum probability among the probabilities of the orientation recognition network to be trained identifying the category of the wheel as various categories, and the use of the hybrid loss function to calculate the category difference loss between the predicted category and the actual category includes: Inputting the predicted probability of the orientation recognition network to be trained recognizing the wheel as the predicted category and the actual probability of the orientation recognition network to be trained recognizing the wheel as the corresponding actual category into the cross entropy loss function; The cross entropy loss function is used to calculate the degree of difference between the predicted probability and the actual probability, and used as the category difference loss.

6. The method for determining the wheel position according to claim 4, characterized in that: The hybrid loss function includes a position loss function, the position loss function includes a first position loss function and a second position loss function, and the use of the hybrid loss function to calculate the position difference loss between the predicted ground point coordinates and the actual ground point coordinates includes: Calculating the difference between the predicted grounding point coordinates and the actual grounding point coordinates; When the difference is less than a preset difference limit, inputting the predicted grounding point coordinates and the actual grounding point coordinates into the first position loss function, where the first position loss function is used to represent the degree of difference of the difference; Calculating the difference between the predicted grounding point coordinates and the actual grounding point coordinates using the first position loss function, and using the difference as the position difference loss; When the difference is greater than or equal to the difference limit, inputting the predicted grounding point coordinates and the actual grounding point coordinates into the second position loss function, where the second position loss function is used to represent the degree of difference of the difference; The second position loss function is used to calculate the difference between the predicted grounding point coordinates and the actual grounding point coordinates, and the difference is used as the position difference loss.

7. The method for determining the wheel position according to claim 4, characterized in that: The hybrid loss function includes an angle loss function, and the use of the hybrid loss function to calculate the angle difference loss between the predicted angle and the actual angle includes: Inputting the predicted angle and the actual angle into the angle loss function; The angle loss function is used to calculate the difference between the predicted angle and the actual angle, and the difference is used as the angle difference loss.

8. A device for determining a wheel position, characterized in that: The device comprises: An acquisition module, used for acquiring vehicle images; An input module, used for inputting the vehicle image into a pre-trained orientation recognition network, wherein the pre-trained orientation recognition network is trained using a preset hybrid loss function; The recognition module is used to use the orientation recognition network to identify the categories and contact point coordinates of two wheels belonging to the same vehicle in the vehicle image, wherein the categories include front wheels and rear wheels, and the mixed loss function is used to constrain the angle between the line between the contact point coordinates of the front wheel and the contact point coordinates of the rear wheel and the preset coordinate axis, constrain the contact point coordinates of each wheel, and constrain the category of each wheel.

9. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for determining the wheel position according to any one of claims 1 to 7 is implemented.

10. A vehicle, characterized in that: The device comprises the wheel position determination device as claimed in claim 8 or the electronic device as claimed in claim 9.

Images