Vehicle wheel point detection method, device, equipment and storage medium

Abstract

The present application discloses a vehicle wheel point detection method, device, equipment and storage medium, the method comprising: obtaining a to-be-detected image; inputting the to-be-detected image into a trained feature extraction model to determine a first feature and a second feature; the first feature comprises a vehicle center point feature map; the vehicle center point feature map comprises at least one vehicle center point; the second feature comprises a wheel key point feature map and an offset distance; the wheel key point feature map comprises at least one wheel key point, and the wheel key point corresponds to the offset distance one by one; determining an offset position corresponding to the wheel key point according to the offset distance; if the offset position matches the vehicle center point, determining the wheel key point as the wheel point corresponding to the vehicle center point, in this way, the accuracy and efficiency of matching the vehicle and the wheel point can be guaranteed, and the hardware resources are less occupied and the hardware requirements are low.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a vehicle wheel point detection method, device, equipment and storage medium. Background Art

[0002] The wheel point refers to the point of intersection between the wheel and the ground. Its position can be obtained through a specific algorithm, and the obtained position can be used for positioning, posture detection, etc. However, there is a key problem in practical applications, that is, the matching problem of wheel points and vehicles. If the detected wheel point cannot be matched with the vehicle, then this wheel point is meaningless and has no application value. At present, most of the matching logic between vehicles and wheels is to perform vehicle detection first, identify the vehicle detection frame of the vehicle, and then perform a wheel point detection in the vehicle detection frame. The wheel point is matched with the vehicle through the two detection matching logics. Although the matching of wheel points and vehicles can be achieved, it will increase the time of recognition reasoning and the scheduling complexity of the model, occupy more software and hardware resources, have high hardware requirements, and have poor real-time performance. Summary of the invention

[0003] The embodiments of the present application provide a vehicle wheel point detection method, device, equipment and storage medium, which can ensure the accuracy and efficiency of matching the vehicle and wheel points, and occupy less hardware resources and have low hardware requirements.

[0004] On the one hand, an embodiment of the present application provides a vehicle wheel point detection method, the method comprising:

[0005] Get the image to be detected;

[0006] Input the image to be detected into the trained feature extraction model to determine the first feature and the second feature; the first feature includes a vehicle center point feature map; the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance; the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one;

[0007] Determine the offset position corresponding to the wheel key point according to the offset distance;

[0008] If the offset position matches the vehicle center point, the wheel key point is determined to be the wheel point corresponding to the vehicle center point.

[0009] Furthermore, after the image to be detected is input into the trained feature extraction model and the first feature and the second feature are determined, the method further includes:

[0010] A vehicle detection frame corresponding to the vehicle center point is determined according to the vehicle position information corresponding to the vehicle center point in the first feature.

[0011] Further, after determining that the wheel key point is the wheel point corresponding to the vehicle center point, the method further includes:

[0012] Determine the matching of the vehicle detection frame corresponding to the wheel key point and the vehicle center point.

[0013] Further, determining that the offset position matches the center point of the vehicle includes:

[0014] Determine a preset value; the preset value is a ratio of the length of the offset area to the length of the vehicle detection frame; the preset value is less than 1;

[0015] Determine the offset area corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the preset value;

[0016] If the offset position is within the offset region, it is determined that the offset position matches the center point of the vehicle.

[0017] Further, determining that the offset position matches the center point of the vehicle also includes:

[0018] If the offset region includes multiple offset positions, the multiple offset positions are sorted according to the confidence levels corresponding to the wheel key points in the second feature;

[0019] Selecting an offset position whose confidence meets a preset requirement from among the multiple offset positions to obtain an offset position set; the offset position set includes at least one offset position;

[0020] Determining that the offset position set matches the offset region;

[0021] If there is no overlapping area in the offset area, or there is an overlapping area in the offset area, and the offset position set matching the offset area is outside the overlapping area, it is determined that the offset position set matches the vehicle center point; the overlapping area is an area where multiple offset areas overlap.

[0022] Further, after determining that the offset position set matches the offset region, the method further includes:

[0023] If there is an overlapping area in the offset area, and there is an offset position in the offset position set matching the vehicle center point in the overlapping area, the preset value is reduced to obtain a reduced preset value;

[0024] Re-determine the offset region corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the reduced preset value; repeat the steps: if the offset region includes multiple offset positions, sort the multiple offset positions according to the size of the confidence corresponding to the wheel key point; select the offset position whose confidence meets the preset requirements among the multiple offset positions to obtain an offset position set; determine that the offset position set matches the offset region;

[0025] Until the offset position set matching the offset area is no longer within the overlapping area, it is determined that the offset position set matches the vehicle center point.

[0026] Furthermore, the second feature also includes classification information corresponding to the key points of the wheel;

[0027] Furthermore, if the offset position matches the center point of the vehicle, determining the wheel key point as the wheel point corresponding to the center point of the vehicle also includes:

[0028] If the offset region includes a plurality of offset positions, the plurality of offset positions are classified according to the classification information corresponding to the wheel key points to obtain a plurality of first-category offset positions and a plurality of second-category offset positions;

[0029] sorting the plurality of first-category offset positions and the plurality of second-category offset positions respectively according to the magnitude of the confidence corresponding to the wheel key point in the second feature;

[0030] Selecting first-category offset positions whose confidences meet preset requirements from among the plurality of first-category offset positions and second-category offset positions whose confidences meet preset requirements from among the plurality of second-category offset positions, respectively, to obtain a first-category offset position set and a second-category offset position set;

[0031] Determine that the first type offset position set and the second type offset position set match the vehicle center point;

[0032] Determine a first type of offset position set as a first type of wheel point corresponding to the center point of the vehicle;

[0033] The second type of offset position set is determined to be the second type of wheel point corresponding to the vehicle center point.

[0034] On the other hand, an embodiment of the present application provides a vehicle wheel point detection device, the device comprising:

[0035] Image acquisition module, used to acquire the image to be detected;

[0036] The feature extraction module is used to input the image to be detected into the trained feature extraction model to determine the first feature and the second feature; the first feature includes a vehicle center point feature map; the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance; the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one;

[0037] An offset position determination module determines an offset position corresponding to a key point of a wheel according to an offset distance;

[0038] The wheel point determination module is used to determine the wheel key point as the wheel point corresponding to the vehicle center point if the offset position matches the vehicle center point.

[0039] Furthermore, the device also includes:

[0040] The vehicle detection frame determination module is used to determine the vehicle detection frame corresponding to the vehicle center point according to the vehicle position information corresponding to the vehicle center point in the first feature.

[0041] Furthermore, the device also includes:

[0042] The wheel point determination module determines the matching of the vehicle detection frame corresponding to the wheel key point and the vehicle center point.

[0043] Furthermore, the device also includes:

[0044] The wheel point determination module is used to determine a preset value; the preset value is the ratio of the length of the offset area to the length of the vehicle detection frame; the preset value is less than 1;

[0045] Determine the offset area corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the preset value;

[0046] If the offset position is within the offset region, it is determined that the offset position matches the vehicle center point.

[0047] Furthermore, the device also includes:

[0048] A wheel point determination module, for sorting the multiple offset positions according to the confidence levels corresponding to the wheel key points in the second feature if the offset region includes multiple offset positions;

[0049] Selecting an offset position whose confidence meets a preset requirement from among the multiple offset positions to obtain an offset position set; the offset position set includes at least one offset position;

[0050] Determining that the offset position set matches the offset region;

[0051] If there is no overlapping area in the offset area, or there is an overlapping area in the offset area, and the offset position set matching the offset area is outside the overlapping area, it is determined that the offset position set matches the vehicle center point; the overlapping area is an area where multiple offset areas overlap.

[0052] Furthermore, the device also includes:

[0053] A wheel point determination module, configured to reduce a preset value to obtain a reduced preset value if there is an overlapping area in the offset area and an offset position in the offset position set matching the vehicle center point is within the overlapping area;

[0054] Re-determine the offset region corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the reduced preset value; repeat the steps: if the offset region includes multiple offset positions, sort the multiple offset positions according to the size of the confidence corresponding to the wheel key point; select the offset position whose confidence meets the preset requirements among the multiple offset positions to obtain an offset position set; determine that the offset position set matches the offset region;

[0055] Until the offset position set matching the offset area is no longer within the overlapping area, it is determined that the offset position set matches the vehicle center point.

[0056] Furthermore, the device also includes:

[0057] A wheel point determination module is used for classifying the multiple offset positions according to the classification information corresponding to the wheel key points if the offset area includes multiple offset positions, so as to obtain multiple first-category offset positions and multiple second-category offset positions;

[0058] sorting the plurality of first-category offset positions and the plurality of second-category offset positions respectively according to the magnitude of the confidence corresponding to the wheel key point in the second feature;

[0059] Selecting first-category offset positions whose confidences meet preset requirements from among the plurality of first-category offset positions and second-category offset positions whose confidences meet preset requirements from among the plurality of second-category offset positions, respectively, to obtain a first-category offset position set and a second-category offset position set;

[0060] Determine that the first type offset position set and the second type offset position set match the vehicle center point;

[0061] Determine a first type of offset position set as a first type of wheel point corresponding to the center point of the vehicle;

[0062] The second type of offset position set is determined to be the second type of wheel point corresponding to the vehicle center point.

[0063] On the other hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded by the processor and executes the vehicle wheel point detection method as described above.

[0064] On the other hand, an embodiment of the present application provides a computer storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the vehicle wheel point detection method as described above.

[0065] The vehicle wheel point detection method, device, equipment and storage medium provided in the embodiments of the present application have the following technical effects:

[0066] Acquire a picture to be detected; input the picture to be detected into a trained feature extraction model to determine a first feature and a second feature; the first feature includes a vehicle center point feature map; the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance; the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one; determine the offset position corresponding to the wheel key point according to the offset distance; if the offset position matches the vehicle center point, determine the wheel key point as the wheel point corresponding to the vehicle center point, so that the accuracy and efficiency of the matching between the vehicle and the wheel point can be guaranteed, and the hardware resources are less occupied and the hardware requirements are low. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0069] Figure 2 It is a flow chart of a vehicle wheel point detection method provided in an embodiment of the present application;

[0070] Figure 3 It is a flow chart of a vehicle wheel point detection method provided in an embodiment of the present application;

[0071] Figure 4 It is a flow chart of a vehicle wheel point detection method provided in an embodiment of the present application;

[0072] Figure 5 It is a flow chart of a vehicle wheel point detection method provided in an embodiment of the present application;

[0073] Figure 6 It is a flow chart of a vehicle wheel point detection method provided in an embodiment of the present application;

[0074] Figure 7 It is a structural schematic diagram of a vehicle wheel point detection device provided in an embodiment of the present application;

[0075] Figure 8 This is a hardware structure block diagram of a server of a vehicle wheel point detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0077] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0078] See also Figure 1 , Figure 1 It is a schematic diagram of an application environment provided in an embodiment of the present application, including a camera 101, a server 102 and a client 103, wherein the camera 101 is used to take photos of the surrounding area of ​​the vehicle, the server 102 receives the photos taken by the camera 101 and processes them to obtain wheel points corresponding to the vehicle in the photos, and the client 103 receives the wheel points corresponding to the vehicle in the photos obtained by the server 102 and performs corresponding operations according to the wheel points.

[0079] Specifically, the server 102 obtains the image to be detected sent by the camera 101, inputs the image to be detected into the trained feature extraction model, and determines the first feature and the second feature, wherein the first feature includes a vehicle center point feature map, and the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance, and the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one. The offset position corresponding to the wheel key point is determined according to the offset distance. If the offset position matches the vehicle center point, the wheel key point is determined to be the wheel point corresponding to the vehicle center point.

[0080] In the embodiment of the present application, the camera 101 can be set around the vehicle to take real-time photos of the road conditions around the vehicle to determine the vehicle status around the vehicle and assist the control system in adjusting the driving status of the vehicle. The camera 101 can also be set on a third-party device other than the vehicle, such as a road or traffic inspection device to determine the driving status of the vehicle and assist in determining whether the corresponding vehicle is driving illegally, or on a vehicle inspection device to determine the status of vehicle components, such as tire pressure, etc., to assist in determining whether the vehicle is in good condition.

[0081] In the embodiment of the present application, the server 102 may also be other computer terminals having the same functions as the server, or similar computing devices. Further, the server 102 may be replaced by a server system, computing platform, or a server cluster including multiple servers.

[0082] In the embodiment of the present application, according to the different locations where the camera 101 is set, the client 103 also has a variety of modules, systems or devices. If the camera 101 is set on a car, the client 103 can be the control system of the car. The client 103 receives the wheel points sent by the server 102, determines the driving status of the surrounding vehicles, and judges and adjusts the driving status of the vehicle based on this, such as acceleration, deceleration, lane change and overtaking. If the camera 101 is set on a third-party device other than the car, the client 103 can be a corresponding third-party device. The client 103 uses the wheel points sent by the server 102 to determine the information it needs. For example, if the client 103 is a vehicle inspection device, the corresponding client 103 can determine information such as tire pressure based on the wheel points sent by the server 102 to determine whether the vehicle is in good condition.

[0083] The following describes a specific embodiment of a vehicle wheel point detection method of the present application. Figure 2 It is a flowchart of a vehicle wheel point detection method provided in an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in the order shown in the embodiment or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include:

[0084] S201: Obtain a picture to be detected.

[0085] In the embodiment of the present application, after obtaining the image to be detected, the image to be detected can be preprocessed.

[0086] As an optional implementation, the preprocessing may include filtering, enhancing color contrast, adjusting size, etc. Optionally, the color contrast of the image to be detected may be enhanced by using operations such as histogram processing.

[0087] S203: Input the image to be detected into a trained feature extraction model to determine the first feature and the second feature.

[0088] In the embodiment of the present application, the first feature includes a vehicle center point feature map and vehicle position information, wherein the vehicle center point feature map includes at least one vehicle center point, and each vehicle center point has corresponding vehicle position information. The vehicle center point is the center point of the vehicle detection frame corresponding to the vehicle identified by it. The vehicle position information is the position information of the four corners of the vehicle detection frame corresponding to the vehicle identified by it. Optionally, the vehicle position information may also be the width information and height information of the vehicle detection frame corresponding to the vehicle identified by it.

[0089] As an optional implementation, the center point of the vehicle and the vehicle position information are represented by coordinates. Correspondingly, the origin of the coordinates and the settings of the x-axis and y-axis can be adjusted according to actual needs, and there is no restriction or requirement for this in this application. Optionally, the lower left corner of the image to be detected can be set as the origin of the coordinates, the width direction of the image to be detected is set to the x-axis, and the height direction is set to the y-axis.

[0090] In the embodiment of the present application, the second feature includes a wheel key point feature map and an offset distance, wherein the wheel key point feature map includes at least one wheel key point, and each wheel key point has an offset distance corresponding thereto. The offset distance is the centripetal offset between the wheel key point output by the feature extraction model according to the training result and the corresponding vehicle center point.

[0091] As an optional implementation, the wheel key point and the offset distance are represented by coordinates. A positive value of the offset distance coordinate value represents a positive offset relative to the wheel key point along the coordinate axis, and a negative value of the offset distance coordinate value represents a negative offset relative to the wheel key point along the coordinate axis.

[0092] As an optional implementation, the second feature further includes a confidence level corresponding to the wheel key point. The confidence level corresponding to the wheel key point indicates the confidence level that the wheel key point is a wheel point.

[0093] As an optional implementation, the feature extraction model can be a DLA-34 network model. Through iterative deep aggregation (IDA), the backbone network is divided into three stages, and tree nodes between different stages of the backbone network are constructed. Hierarchical deep aggregation (HDA) is used to construct tree nodes between different blocks in the same stage and between different stages in the backbone network, thereby realizing multi-scale and multi-depth feature aggregation, thereby obtaining a classification and recognition structure with higher accuracy using fewer parameters and high computational efficiency.

[0094] S205: Determine the offset position corresponding to the wheel key point according to the offset distance.

[0095] In the embodiment of the present application, the coordinates of the wheel key point and the coordinates of the corresponding offset distance are added to obtain the coordinates of the offset position corresponding to the wheel key point. The offset position is the vehicle center point corresponding to the wheel key point identified by the feature extraction model according to the training results. By comparing the offset position with the vehicle center point, the vehicle center point matching the wheel key point can be confirmed.

[0096] As an optional implementation, step S203: inputting the image to be detected into the trained feature extraction model, and after determining the first feature and the second feature, further includes the step of determining a vehicle detection frame. Specifically, according to the vehicle position information corresponding to the center point of the vehicle, the positions of the four corners of the corresponding vehicle detection frame are obtained, and the vehicle detection frame is obtained by sequentially connecting the positions of the four corners.

[0097] S207: If the offset position matches the center point of the vehicle, determine that the wheel key point is the wheel point corresponding to the center point of the vehicle.

[0098] In an embodiment of the present application, if the offset position corresponding to the wheel key point matches the vehicle center point, it means that the vehicle key point matches the vehicle center point, then the wheel key point can be determined as the wheel point corresponding to the vehicle center point, and further the correspondence between the vehicle and the wheel point can be determined.

[0099] As an optional implementation, after determining that the wheel key point is the wheel point corresponding to the vehicle center point, the wheel key point is further matched with the vehicle detection frame corresponding to the vehicle center point. The wheel key point is the wheel point corresponding to the vehicle detection frame, thereby establishing a matching relationship between the vehicle detection frame and the wheel point. In the subsequent application of the wheel point and the vehicle detection frame, the determined matching relationship can always be maintained.

[0100] As an optional implementation, Figure 3 A schematic diagram of a vehicle wheel point detection method provided in an embodiment of the present application is shown. Specifically, Figure 3 As shown, determining the offset position matches the vehicle center point includes the following steps:

[0101] S301: Determine a preset value.

[0102] In the embodiment of the present application, the preset value is the ratio of the length of the offset region to the length of the vehicle detection frame. The offset region is the offset range of the acceptable offset position of the vehicle center point. Optionally, the preset value is less than 1, and the offset region is smaller than the size of the vehicle detection frame. The size of the offset region can affect the matching accuracy between the wheel key point and the vehicle center point. The smaller the offset region, the higher the matching accuracy between the wheel key point and the vehicle center point.

[0103] As an optional implementation, the preset value is set to 0.5.

[0104] S303: Determine an offset region corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and a preset value.

[0105] In an embodiment of the present application, the width and length of the offset area are determined based on the vehicle detection box and preset value corresponding to the center point of the vehicle, and / or the coordinates of the four corners of the offset area are determined, and the range size of the offset area is determined based on the width and length of the offset area and the coordinates of the four corners.

[0106] S305: If the offset position is within the offset region, determine that the offset position matches the center point of the vehicle.

[0107] In an embodiment of the present application, if the offset position corresponding to a wheel key point is located in an offset region of a vehicle center point, and there is no other offset position in the offset region, it is determined that the offset position corresponding to the wheel key point matches the vehicle center point.

[0108] The above method is applicable to the case where there is only one offset position in the offset area corresponding to the vehicle center point. When there are multiple offset positions in an offset area, it is necessary to further confirm the multiple offset positions to determine the offset position and wheel key points that match the offset area and the vehicle center point corresponding to the offset area. As an optional implementation, Figure 4 A schematic diagram of a vehicle wheel point detection method provided in an embodiment of the present application is shown. Figure 4 As shown, in the case where the offset region includes multiple offset positions, determining that the offset position matches the vehicle center point includes the following steps:

[0109] S401: Determine a preset value.

[0110] In the embodiment of the present application, the preset value is the ratio of the length of the offset region to the length of the vehicle detection frame. The offset region is the offset range of the acceptable offset position of the vehicle center point. Optionally, the preset value is less than 1, and the offset region is smaller than the size of the vehicle detection frame. The size of the offset region can affect the matching accuracy between the wheel key point and the vehicle center point. The smaller the offset region, the higher the matching accuracy between the wheel key point and the vehicle center point.

[0111] As an optional implementation, the preset value is set to 0.5.

[0112] S403: Determine an offset region corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and a preset value.

[0113] S405: If the offset region includes multiple offset positions, the multiple offset positions are sorted according to the confidence levels corresponding to the wheel key points in the second feature.

[0114] In the embodiment of the present application, the confidence level corresponding to the wheel key point is the confidence level that the wheel key point is a wheel point. The greater the confidence level, the greater the probability that the wheel key point is a wheel point, and the more reliable the wheel key point is. Therefore, when there are multiple offset positions, the wheel key point with a greater corresponding confidence level is more likely to match the center point of the vehicle, and the offset position corresponding to the wheel key point with a greater confidence level is more reliable and more accurate.

[0115] S407: Selecting offset positions whose confidences meet preset requirements from among the multiple offset positions to obtain an offset position set.

[0116] In the embodiment of the present application, the offset position set includes at least one offset position.

[0117] As an optional implementation, the preset requirement for selecting the confidence level can be adjusted according to actual needs. Optionally, if the most reliable offset position is to be selected to match the offset region, the preset requirement can be set to the maximum value among multiple confidence levels, and the corresponding offset position set only includes the corresponding offset position with the maximum confidence level.

[0118] S409: Determine whether the offset position set matches the offset region.

[0119] S411: If the offset regions do not have overlapping regions, or if the offset regions have overlapping regions, and the offset position set matching the offset regions is outside the overlapping region, determine that the offset position set matches the center point of the vehicle.

[0120] In the embodiment of the present application, the overlapping area is an area where multiple offset areas overlap. If there is no overlapping area in the offset areas, or there is an overlapping area in the offset areas, and the offset position set matching the offset area is outside the overlapping area, it means that the offset position set is uniquely corresponding to the offset area, and the offset position set can only match the vehicle center point corresponding to the offset area.

[0121] Figure 5 A schematic diagram of a vehicle wheel point detection method provided in an embodiment of the present application is shown. Figure 5 As shown, after step S409: determining that the offset position set matches the offset region, the following further includes:

[0122] S413: If there is an overlapping area in the offset area, and there is an offset position in the offset position set matching the vehicle center point in the overlapping area, the preset value is reduced to obtain a reduced preset value.

[0123] In an embodiment of the present application, if there is an offset position in the offset position set matching the vehicle center point within the overlapping area, it means that the range of the offset area is too large and the selection of the offset position set is not accurate enough. The preset value needs to be reduced to reduce the range of the offset area.

[0124] S415: Re-determine the offset area corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the reduced preset value; repeat the steps: if the offset area includes multiple offset positions, sort the multiple offset positions according to the size of the confidence corresponding to the wheel key point; select the offset position whose confidence meets the preset requirements among the multiple offset positions to obtain an offset position set; determine that the offset position set matches the offset area.

[0125] If there are still offset positions in the offset position set obtained again within the overlapping region, the above steps S413-S415 are repeatedly performed until the offset position set matching the offset region is no longer within the overlapping region, and it is determined that the offset position set matches the vehicle center point.

[0126] As an optional implementation, the second feature also includes classification information corresponding to wheel key points. The classification information of wheel key points includes first-category wheel key points and second-category wheel key points. Optionally, the first-category wheel key points are front wheels, and the second-category wheel key points are rear wheels. On this basis, each vehicle center point can correspond to two types of wheel key points. If you want to obtain the first-category wheel points and the second-category wheel points corresponding to the vehicle, that is, the front wheel points and the rear wheel points, the wheel key points are classified when selecting the wheel key points. Specifically, Figure 6 A schematic diagram of a vehicle wheel point detection method provided in an embodiment of the present application is shown. Figure 6As shown, step S207: if the offset position matches the center point of the vehicle, determining the wheel key point as the wheel point corresponding to the center point of the vehicle may also include:

[0127] S601: Determine a preset value.

[0128] In the embodiment of the present application, the preset value is the ratio of the length of the offset region to the length of the vehicle detection frame. The offset region is the offset range of the acceptable offset position of the vehicle center point. Optionally, the preset value is less than 1, and the offset region is smaller than the size of the vehicle detection frame. The size of the offset region can affect the matching accuracy between the wheel key point and the vehicle center point. The smaller the offset region, the higher the matching accuracy between the wheel key point and the vehicle center point.

[0129] As an optional implementation, the preset value is set to 0.5.

[0130] S603: Determine an offset region corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and a preset value.

[0131] S605: If the offset region includes a plurality of offset positions, classify the plurality of offset positions according to classification information corresponding to the wheel key points to obtain a plurality of first-category offset positions and a plurality of second-category offset positions.

[0132] S607: Sort the plurality of first-category offset positions and the plurality of second-category offset positions respectively according to the magnitude of the confidence corresponding to the wheel key point in the second feature.

[0133] S609: Select first-category offset positions whose confidences meet preset requirements from among multiple first-category offset positions, and second-category offset positions whose confidences meet preset requirements from among multiple second-category offset positions, respectively, to obtain a first-category offset position set and a second-category offset position set.

[0134] As an optional implementation, the preset requirement for selecting the confidence level can be adjusted according to actual needs. Optionally, if the most reliable first-class offset position and / or second-class offset position is to be selected to match the offset region, the preset requirement can be set to the maximum value among multiple confidence levels, and the corresponding first-class offset position set and / or second-class offset position set only includes the corresponding first-class offset position and / or second-class offset position with the highest confidence level.

[0135] S611: Determine whether the first type offset position set and the second type offset position set match the vehicle center point.

[0136] In an embodiment of the present application, if there is an offset position in the first type of offset position set that is within the overlapping area, and / or there is an offset position in the second type of offset position set that is within the overlapping area, as in the above-mentioned steps S413-S415, the preset value is reset, and the offset area is re-determined after the preset value is reduced. According to the re-determined offset area, the first type of offset position set and / or the second type of offset position set are correspondingly re-determined until there is no offset position in the first type of offset position set that is within the overlapping area, and there is no offset position in the second type of offset position set that is within the overlapping area, and it is determined that the first type of offset position set and the second type of offset position set match the vehicle center point.

[0137] S613: Determine that the first type of offset position set is the first type of wheel point corresponding to the vehicle center point.

[0138] S615: Determine that the second type offset position set is the second type wheel point corresponding to the vehicle center point.

[0139] In an embodiment of the present application, the first-category wheel key points and the second-category wheel key points are distinguished, and the first-category wheel key points and the second-category wheel key points that match the vehicle center point are respectively determined. The first-category wheel points and the second-category wheel points corresponding to the vehicle center point can be obtained at the same time, which is convenient for determining the corresponding vehicle attitude angle and other information in subsequent applications.

[0140] The present application also provides a vehicle wheel point detection device, Figure 7 is a schematic diagram of the structure of a vehicle wheel point detection device provided in an embodiment of the present application, such as Figure 7 As shown, the device comprises:

[0141] The picture acquisition module 701 is used to acquire the picture to be detected.

[0142] The feature extraction module 702 is used to input the image to be detected into a trained feature extraction model to determine the first feature and the second feature; the first feature includes a vehicle center point feature map; the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance; the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one.

[0143] The offset position determination module 703 determines the offset position corresponding to the wheel key point according to the offset distance.

[0144] The wheel point determination module 704 is used to determine the wheel key point as the wheel point corresponding to the vehicle center point if the offset position matches the vehicle center point.

[0145] As an optional implementation, the device further includes:

[0146] The vehicle detection frame determination module 705 is used to determine the vehicle detection frame corresponding to the vehicle center point according to the vehicle position information corresponding to the vehicle center point in the first feature.

[0147] As an optional implementation, the device further includes:

[0148] The wheel point determination module 704 determines the matching of the wheel key point with the vehicle detection frame of the vehicle center point.

[0149] As an optional implementation, the device further includes:

[0150] The wheel point determination module 704 is used to determine a preset value; the preset value is the ratio of the length of the offset area to the length of the vehicle detection frame; the preset value is less than 1;

[0151] Determine the offset area corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the preset value;

[0152] If the offset position is within the offset region, it is determined that the offset position matches the center point of the vehicle.

[0153] As an optional implementation, the device further includes:

[0154] The wheel point determination module 704 is used to sort the multiple offset positions according to the confidence level corresponding to the wheel key point in the second feature if the offset area includes multiple offset positions;

[0155] Selecting an offset position whose confidence meets a preset requirement from among the multiple offset positions to obtain an offset position set; the offset position set includes at least one offset position;

[0156] Determining that the offset position set matches the offset region;

[0157] If there is no overlapping area in the offset area, or there is an overlapping area in the offset area, and the offset position set matching the offset area is outside the overlapping area, it is determined that the offset position set matches the vehicle center point; the overlapping area is an area where multiple offset areas overlap.

[0158] As an optional implementation, the device further includes:

[0159] The wheel point determination module 704 is configured to reduce the preset value to obtain the reduced preset value if there is an overlapping area in the offset area and there is an offset position in the offset position set matching the vehicle center point within the overlapping area;

[0160] Re-determine the offset region corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the reduced preset value; repeat the steps: if the offset region includes multiple offset positions, sort the multiple offset positions according to the size of the confidence corresponding to the wheel key point; select the offset position whose confidence meets the preset requirements among the multiple offset positions to obtain an offset position set; determine that the offset position set matches the offset region;

[0161] Until the offset position set matching the offset area is no longer within the overlapping area, it is determined that the offset position set matches the vehicle center point.

[0162] As an optional implementation, the device further includes:

[0163] The wheel point determination module 704 is used for classifying the multiple offset positions according to the classification information corresponding to the wheel key points if the offset region includes multiple offset positions, so as to obtain multiple first-category offset positions and multiple second-category offset positions;

[0164] sorting the plurality of first-category offset positions and the plurality of second-category offset positions respectively according to the magnitude of the confidence corresponding to the wheel key point in the second feature;

[0165] Selecting first-category offset positions whose confidences meet preset requirements from among the plurality of first-category offset positions and second-category offset positions whose confidences meet preset requirements from among the plurality of second-category offset positions, respectively, to obtain a first-category offset position set and a second-category offset position set;

[0166] Determine that the first type offset position set and the second type offset position set match the vehicle center point;

[0167] Determine a first type of offset position set as a first type of wheel point corresponding to the center point of the vehicle;

[0168] The second type of offset position set is determined to be the second type of wheel point corresponding to the vehicle center point.

[0169] The device and method embodiments in the device embodiment are based on the same application concept.

[0170] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 8 1 is a hardware structure block diagram of a server of a vehicle wheel point detection method provided in an embodiment of the present application. Figure 8As shown, the server 800 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 810 (the processor 810 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 830 for storing data, and one or more storage media 820 (such as one or more mass storage devices) for storing application programs 823 or data 822. Among them, the memory 830 and the storage medium 820 can be short-term storage or permanent storage. The program stored in the storage medium 820 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processing unit 810 can be configured to communicate with the storage medium 820 and execute a series of instruction operations in the storage medium 820 on the server 800. The server 800 may also include one or more power supplies 860, one or more wired or wireless network interfaces 850, one or more input and output interfaces 840, and / or one or more operating systems 821, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0171] The input / output interface 840 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the server 800. In one example, the input / output interface 840 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 840 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0172] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only and does not limit the structure of the above electronic device. Figure 8 More or fewer components as shown, or with Figure 8 Different configurations are shown.

[0173] An embodiment of the present application also provides a vehicle wheel point detection device, the device includes a processor and a memory, the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement a vehicle wheel point detection method.

[0174] An embodiment of the present application also provides a storage medium, which can be set in a server to store at least one instruction, at least one program, a code set or an instruction set related to a vehicle wheel point detection method in a method embodiment. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the vehicle wheel point detection method provided in the above method embodiment.

[0175] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0176] It can be seen from the embodiments of the vehicle wheel point detection method, device, equipment and storage medium provided by the above-mentioned application that in the present application, a picture to be detected is obtained; the picture to be detected is input into a trained feature extraction model to determine a first feature and a second feature; the first feature includes a vehicle center point feature map; the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance; the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one; the offset position corresponding to the wheel key point is determined according to the offset distance; if the offset position matches the vehicle center point, the wheel key point is determined to be the wheel point corresponding to the vehicle center point. In this way, the accuracy and efficiency of the matching between the vehicle and the wheel point can be guaranteed, and the hardware resources are less occupied and the hardware requirements are low.

[0177] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0178] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0179] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0180] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A vehicle wheel point detection method, characterized in that: The method comprises: Get the image to be detected; Input the image to be detected into a trained feature extraction model to determine a first feature and a second feature; the first feature includes a vehicle center point feature map; the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance; the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one; Determine a vehicle detection frame corresponding to the vehicle center point according to the vehicle position information corresponding to the vehicle center point in the first feature; Determine the offset position corresponding to the wheel key point according to the offset distance; Determine a preset value; the preset value is a ratio of the length of the offset area to the length of the vehicle detection frame; the preset value is less than 1; Determine an offset area corresponding to the center point of the vehicle according to a vehicle detection frame corresponding to the center point of the vehicle and a preset value; If the offset position is within the offset region, determining that the offset position matches the center point of the vehicle; If the offset position matches the vehicle center point, the wheel key point is determined to be the wheel point corresponding to the vehicle center point.

2. A vehicle wheel point detection method according to claim 1, characterized in that: After determining that the wheel key point is the wheel point corresponding to the vehicle center point, the method further includes: Determine that the wheel key point matches a vehicle detection frame corresponding to the vehicle center point.

3. A vehicle wheel point detection method according to claim 2, characterized in that: Determining that the offset position matches the vehicle center point also includes: If the offset region includes a plurality of offset positions, sorting the plurality of offset positions according to the magnitude of the confidence corresponding to the wheel key point in the second feature; Selecting an offset position whose confidence meets a preset requirement from among the multiple offset positions to obtain an offset position set; the offset position set includes at least one offset position; Determining that the offset position set matches the offset region; If there is no overlapping area among the offset areas, or there is an overlapping area among the offset areas, and the offset position set matching the offset area is outside the overlapping area, it is determined that the offset position set matches the center point of the vehicle; the overlapping area is an area where multiple offset areas overlap.

4. A vehicle wheel point detection method according to claim 3, characterized in that: After determining that the offset position set matches the offset area, the method further includes: If there is an overlapping area among the offset areas, and there is an offset position in the offset position set matching the center point of the vehicle within the overlapping area, the preset value is reduced to obtain a reduced preset value; Re-determine the offset region corresponding to the center point of the vehicle according to the vehicle detection frame corresponding to the center point of the vehicle and the reduced preset value; repeat the steps: if the offset region includes multiple offset positions, sort the multiple offset positions according to the magnitude of the confidence corresponding to the wheel key point; select the offset position whose confidence meets the preset requirement among the multiple offset positions to obtain the offset position set; determine that the offset position set matches the offset region; Until the offset position set matching the offset area is no longer within the overlapping area, it is determined that the offset position set matches the vehicle center point.

5. A vehicle wheel point detection method according to claim 2, characterized in that: The second feature also includes classification information corresponding to the wheel key point; If the offset position matches the vehicle center point, determining the wheel key point as the wheel point corresponding to the vehicle center point also includes: If the offset region includes a plurality of offset positions, classifying the plurality of offset positions according to the classification information corresponding to the wheel key points to obtain a plurality of first-category offset positions and a plurality of second-category offset positions; sorting the plurality of first-category offset positions and the plurality of second-category offset positions respectively according to the magnitude of the confidence corresponding to the wheel key point in the second feature; Respectively selecting the first type of offset positions whose confidences meet the preset requirements from the multiple first type of offset positions and the second type of offset positions whose confidences meet the preset requirements from the multiple second type of offset positions to obtain a first type of offset position set and a second type of offset position set; Determine that the first type of offset position set and the second type of offset position set match the vehicle center point; Determine the first type of offset position set as the first type of wheel points corresponding to the center point of the vehicle; The second-type offset position set is determined to be the second-type wheel point corresponding to the vehicle center point.

6. A vehicle wheel point detection device, characterized in that: The device comprises: Image acquisition module, used to acquire the image to be detected; A feature extraction module is used to input the image to be detected into a trained feature extraction model to determine a first feature and a second feature; the first feature includes a vehicle center point feature map; the vehicle center point feature map includes at least one vehicle center point; the second feature includes a wheel key point feature map and an offset distance; the wheel key point feature map includes at least one wheel key point, and the wheel key point corresponds to the offset distance one by one; A vehicle detection frame determination module, used to determine a vehicle detection frame corresponding to the vehicle center point according to the vehicle position information corresponding to the vehicle center point in the first feature; An offset position determination module, used to determine the offset position corresponding to the wheel key point according to the offset distance; A wheel point determination module, used to determine a preset value; the preset value is a ratio of the length of the offset area to the length of the vehicle detection frame; the preset value is less than 1; according to the vehicle detection frame corresponding to the vehicle center point and the preset value, determine the offset area corresponding to the vehicle center point; if the offset position is within the offset area, determine that the offset position matches the vehicle center point; The wheel point determination module is used to determine that the wheel key point is the wheel point corresponding to the vehicle center point if the offset position matches the vehicle center point.

7. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded by the processor and executes the vehicle wheel point detection method as described in any one of claims 1-5.

8. A computer storage medium, characterized in that: The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the vehicle wheel point detection method as described in any one of claims 1-5.

Images