Vehicle profile parameter inversion and three-dimensional bounding box reconstruction method based on external profile

Abstract

The invention provides a vehicle profile parameter inversion and three-dimensional bounding box reconstruction method based on an external profile. According to the method, a morphological operation method is utilized, the appearance feature of the outer contour of a running vehicle is continuously extracted and processed according to a time sequence, vehicle space geometric parameter key points which completely describe the appearance of the vehicle are collected, inversion of vehicle appearance parameters is achieved according to the relative positions of the vehicle space geometric parameter key points, and the appearance of the vehicle is obtained. And reconstructing a three-dimensional bounding box of the vehicle model, so that the problems that the vehicle load position and size information are difficult to directly recognize and accurate geometric reconstruction cannot be performed on the vehicle in the image in the existing recognition method are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure health monitoring, and in particular to a vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on external contour. Background Art

[0002] Vehicle load is one of the key factors affecting the life of a bridge and is also the basis for evaluating the ultimate bearing capacity of a bridge. Bridge design needs to take into account various extreme effects, among which vehicle overload is an important factor. Only by having an accurate understanding of the vehicle load level can we scientifically predict the ultimate bearing level of the bridge within its design service life, thereby ensuring the structural safety reserve of the bridge. With the rapid increase in traffic flow, the vehicle load borne by highway bridges has also continued to grow, exacerbating structural fatigue damage. In order to ensure the knowability and measurability of bridge loads and improve disaster emergency response capabilities, it is necessary to find a way to quickly determine the distribution of traffic flow on the bridge deck.

[0003] The key to identifying the traffic load distribution on the bridge deck is to accurately locate the load position of each vehicle. For the longitudinal position, if the vehicle is close to the mid-span, the beam will be subjected to greater bending moment and deflection, while the support reaction will not be affected much; on the contrary, if it is close to the support, the structural response is relatively small, but the support reaction will increase significantly. Generally speaking, when the longitudinal deviation is 5% of the span, the maximum internal force and deformation of the simply supported beam will increase by about 10%. For the lateral position, when the vehicle is too close to the edge of the bridge deck, it will cause obvious torque and stress concentration, resulting in increased support reaction. If the lateral offset reaches 1 / 4 of the bridge deck width, the most unfavorable support reaction will increase by more than 15%, seriously threatening the safety of the bridge structure. Therefore, the estimated error of the load position has a very large impact on the safety assessment results of small-span bridges.

[0004] In order to determine the distribution of vehicle loads, most existing practices simplify the axle load into a single concentrated load applied to the bridge deck. The vehicle bounding box in the image is identified through a convolutional neural network, and the center of the bounding box is used as the approximate location of the concentrated load. For mixed traffic, a single load model is used. Because it overestimates the local effect of the load and underestimates the overall stress level, it is difficult to accurately capture the internal force distribution of bridge components under the combined action of multiple vehicle loads, which may ultimately lead to deviations in the safety assessment results and fail to reflect the actual stress level of the bridge. In order to ensure the authenticity of the bridge safety assessment, it is necessary to simulate the actual action form of the vehicle axle load as much as possible, requiring accurate identification of the wheel position.

[0005] In addition, the geometric dimension data of vehicles provide valuable traffic behavior information for bridge health monitoring, which is of great significance for assessing structural and operational risks, making management decisions, and guiding maintenance. For example, the dimensional data of large vehicles is closely related to their aerodynamic effects, which can be used to evaluate the wind vibration and turbulence that may be caused by the passage of vehicles on the bridge; the continuously acquired vehicle geometric information is also helpful in analyzing the composition of traffic flow and providing a decision-making basis for the reasonable design of traffic organization and transformation plans; during traffic congestion, if the distance between vehicles is too small and there is a risk of chain rear-end collision, traffic restrictions can be implemented to evacuate vehicles as soon as possible to ensure the safety of the bridge structure; identifying super-high and super-wide models can estimate their collision and wear risks on bridge components, and take corresponding preventive measures to avoid potential structural damage. Therefore, accurately obtaining vehicle geometric dimensions is also an important part of bridge risk management.

[0006] In order to reconstruct the 3D bounding box and obtain the geometric parameters of the vehicle, some methods try to create a data set of vehicle parts and use convolutional neural networks to identify vehicle parts such as wheels. Vehicle parts (such as wheels, rear ends, and body, etc.) usually only occupy a few dozen pixels in the image. After several convolutions, the original features are completely aliased, and the vehicle can only be detected when it is very close to the camera. In other words, for the detection of small targets such as wheels and rear ends, the recognition effect of convolutional neural networks is poor, and even if the corresponding data set is established, it is difficult to perform fine segmentation.

[0007] From the above analysis, we can see that the existing method identifies the vehicle as a single rectangular frame, simplifies the vehicle load into a single concentrated load, and the actual vehicle load acts on the ground through the center point of the bounding box. The distribution of the concentrated load at the axle is not considered, and the correct load position cannot be located. The convolutional neural network cannot perform fine-grained segmentation of vehicle components, geometrically reconstruct the vehicle model, and obtain the vehicle's geometric size information. Therefore, it is of great practical engineering significance to accurately identify vehicle components from bridge videos and determine the load position and geometric size. Summary of the invention

[0008] The purpose of the present invention is to solve the problem that existing recognition methods are difficult to directly identify the vehicle load position and size information and cannot accurately reconstruct the vehicle geometry in the image. A vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on external contour is proposed.

[0009] The present invention is implemented by the following technical scheme. The present invention proposes a vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on external contour, and the method comprises the following steps:

[0010] Step 1: Constructing contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an open operation on the pixel connected domain, smooth the boundary of the pixel connected domain, obtain the continuous and clear outer contour of the vehicle, and form the contour point set S1;

[0011] Step 2: Use the line L1 connecting the two frames of the same vehicle to divide the lower half of the point set S2, fit S2 into the function curve C1, search for the minimum point of C1 to collect the key points of the wheel position P1, P2, P3...P m , determine the number of axles to be n and the number of wheels to be 2n;

[0012] Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 , S 2-2 , S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. 2-1 and S 2-(n+1) The shortest point P 21 , P 23 As a vehicle commander key point;

[0013] Step 4: The vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x , T y , keep sliding in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ;

[0014] Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1, Q2, Q3, and Q2 on the other side of the vehicle based on symmetry. m ;

[0015] Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3;

[0016] Step 7: Based on the relative positions of the key points determined by the above process, the vehicle load position and geometric dimensions are determined, and the vehicle 3D bounding box is reconstructed.

[0017] Furthermore, the step 2 specifically includes:

[0018] Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form a new point set S2;

[0019] Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis;

[0020] Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve;

[0021] Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key point of the wheel position P1, P2, P3...P m .

[0022] Furthermore, the vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. The wheel position and number of wheels on one side of the vehicle can be confirmed by searching the peaks; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base is linearly independent and has the maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1):

[0023]

[0024] In the formula, X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

[0025] Furthermore, the step three specifically includes:

[0026] Step 3-1: According to the position of the wheel position key points in S2, cut the point set S2 into several new sets S 2-1 , S 2-2 , S 2-3 ......S 2-(n+1) ;

[0027] Step 3-2: Take the set S 2-2 , S 2-3 ......S 2-n , use the K-Means algorithm for each of the above sets to divide the set into three clusters, and all the points in the middle cluster constitute a new set S 2-2-1 , S 2-3-1 ......S2-n-1 ;

[0028] Step 3-3: S 2-2-1 , S 2-3-1 ......S 2-n-1 Gather into a new set S3, and fit S3 into a straight line L2;

[0029] Step 3-4: Calculate L2 and S 2-1 and S 2(n+1) The distance between all points in the L2 point set S is the point with the smallest distance. 2-1 , S 2-3 The intersection point of the curves represented by 21 and P 23 Determined as the key point of vehicle length.

[0030] Furthermore, in the K-Means algorithm, multiple iterations are performed to ensure that the distance from all points in each cluster to the nearest center point is minimized. The number of clusters is manually set initially, and the cluster centers are randomly generated. The coordinates of the cluster centers are updated in each iteration. The update process is expressed as formula (2):

[0031]

[0032] In the formula, c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

[0033] Furthermore, the step 4 specifically includes:

[0034] Step 4-1: Filter out the P in S2 21 The points on the right form a new geometry S4;

[0035] Step 4-2: Create a sliding box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points in n, record T x The x coordinate p at this time x , set the step size to 1, slide T along the x direction x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until;

[0036] Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the slide is continued in the y direction in the same way until it reaches P 21 Until then, record the y coordinate p when it stops y and P20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set;

[0037] Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 It is the key point of vehicle width.

[0038] Furthermore, the step six specifically includes:

[0039] Step 6-1: Filter the points located above L1 in the vehicle outer contour S1 to form a set S5;

[0040] Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

[0041] Furthermore, the step seven specifically includes:

[0042] Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1P2 intersects with the extension line of P1P2 at point B;

[0043] Step 7-2: Pass P 23 The vertical line L4 intersects with the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect with the extension line of P4P5 at point C.

[0044] Step 7-3: ABCD is a quadrilateral describing the lower edge of the vehicle. Slide quadrilateral ABCD along the height direction to form quadrilateral EFGH passing through P3, and connect the two quadrilaterals in the height direction; finally, a rectangular bounding box ABCD-EFGH describing the three-dimensional shape of the vehicle can be obtained.

[0045] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour are implemented.

[0046] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The present invention proposes a vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on external contour, which can effectively collect wheel position key points and vehicle geometry key points in the vehicle contour curve, directly identify the actual load position and geometric size information of the vehicle, and reconstruct the three-dimensional bounding box of the vehicle.

[0049] 2. The present invention takes the invariance of the vehicle's external contour as the starting point, identifies the key points of the vehicle's spatial geometric parameters, and reconstructs the vehicle's three-dimensional bounding box model. It has good stability and high accuracy, solves the problem of poor recognition of vehicle components such as wheels by convolutional neural networks, and provides reliable traffic information for bridge structure health monitoring.

[0050] 3. Since the present invention does not use convolutional neural networks to directly identify vehicle parts, it avoids the labeling work of a large number of data sets. The vehicle contour can be identified by an instance segmentation model trained with an open source data set, which greatly reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 is a simplified flow chart of a method according to an embodiment of the present invention;

[0053] Figure 2 is an external outline diagram of a vehicle according to an embodiment of the present invention;

[0054] Figure 3 Schematic diagram of the wheel position key point search method of the present invention;

[0055] Figure 4 It is a schematic diagram of the vehicle length key point search method of the present invention;

[0056] Figure 5 Schematic diagram of the vehicle width key point search method of the present invention;

[0057] Figure 6 It is a schematic diagram of the vehicle height key point search method of the present invention;

[0058] Figure 7 It is a schematic diagram of the geometric reconstruction result of the present invention. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The present invention proposes a vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on external contour. The appearance feature of the external contour of a moving vehicle is continuously extracted and processed in time sequence by using a morphological operation method, and the key points of the vehicle space geometric parameters that fully describe the vehicle shape are collected. The vehicle shape parameters are inverted according to the relative positions of the key points of the vehicle space geometric parameters, and the three-dimensional bounding box of the vehicle model is reconstructed. The vehicle external contour is a pixel connected domain that completely covers the vehicle shape. The vehicle external contour can be identified by manual picking or using instance segmentation models such as Mask R-CNN, YOLCAT and BlendMask. The vehicle shape parameters are a data description of the vehicle geometric size information and wheel position in the image; the three-dimensional bounding box is a rectangular bounding box that is established by using the relative positions of the key points of the vehicle space geometric parameters in the image and can completely surround the area where the vehicle is located. The key points of the vehicle space geometric parameters include two types, one is a control point in the image that can locate the position of the vehicle components, including the wheel position key point, and the other is a control point in the image that can describe the vehicle size information, including the vehicle length key point, the vehicle width key point and the vehicle height key point.

[0061] Specifically, combined Figure 1-Figure 7 The present invention proposes a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on an external contour, the method comprising the following steps:

[0062] Step 1: Constructing contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an open operation on the pixel connected domain, smooth the boundary of the pixel connected domain, obtain the continuous and clear outer contour of the vehicle, and form the contour point set S1;

[0063] Step 2: Use the line L1 connecting the two frames of the same vehicle to divide the lower half of the point set S2, fit S2 into the function curve C1, search for the minimum point of C1 to collect the key points of the wheel position P1, P2, P3...P m , determine the number of axles to be n and the number of wheels to be 2n;

[0064] The step 2 specifically includes:

[0065] Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form a new point set S2;

[0066] Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis;

[0067] Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve;

[0068] Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key point of the wheel position P1, P2, P3...P m .

[0069] The vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. The wheel position and number of wheels on one side of the vehicle can be confirmed by searching the peaks; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base is linearly independent and has the maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1):

[0070]

[0071] In the formula, X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

[0072] Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 , S 2-2 , S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. 2-1 and S 2-(n+1) The shortest point P 21 , P 23 As a vehicle commander key point;

[0073] The step three specifically includes:

[0074] Step 3-1: According to the position of the wheel position key points in S2, cut the point set S2 into several new sets S 2-1 , S 2-2 , S 2-3 ......S 2-(n+1) ;

[0075] Step 3-2: Take the set S 2-2 , S 2-3 ......S 2-n , use the K-Means algorithm for each of the above sets to divide the set into three clusters, and all the points in the middle cluster constitute a new set S 2-2-1 , S 2-3-1 ......S 2-n-1 ;

[0076] Step 3-3: S 2-2-1 , S 2-3-1 ......S 2-n-1 Gather into a new set S3, and fit S3 into a straight line L2;

[0077] Step 3-4: Calculate L2 and S 2-1 and S 2(n+1) The distance between all points in the L2 point set S is the point with the smallest distance. 2-1 , S 2-3 The intersection point of the curves represented by 21 and P 23 Determined as the key point of vehicle length.

[0078] The K-Means algorithm is a clustering algorithm. In the K-Means algorithm, multiple iterations are performed to ensure that the distance from all points in each cluster to the nearest center point is minimized. The number of clusters is manually set initially, and the cluster centers are randomly generated. The coordinates of the cluster centers are updated in each iteration. The update process is expressed as formula (2):

[0079]

[0080] In the formula, c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

[0081] Step 4: The vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x , T y , keep sliding in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ;

[0082] The step 4 specifically includes:

[0083] Step 4-1: Filter out the P in S2 21 The points on the right form a new geometry S4;

[0084] Step 4-2: Create a sliding box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points in n, record T x The x coordinate p at this time x , set the step size to 1, slide T along the x direction x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until;

[0085] Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the slide is continued in the y direction in the same way until it reaches P 21 Until then, record the y coordinate p when it stops y and P 20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set;

[0086] Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 It is the key point of vehicle width.

[0087] Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1, Q2, Q3, and Q2 on the other side of the vehicle based on symmetry. m ;

[0088] Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3;

[0089] The step six specifically includes:

[0090] Step 6-1: Filter the points located above L1 in the vehicle outer contour S1 to form a set S5;

[0091] Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

[0092] Step 7: Based on the relative positions of the key points determined by the above process, the vehicle load position and geometric dimensions are determined, and the vehicle 3D bounding box is reconstructed.

[0093] The step seven specifically includes:

[0094] Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1P2 intersects with the extension line of P1P2 at point B;

[0095] Step 7-2: Pass P 23 The vertical line L4 intersects with the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect with the extension line of P4P5 at point C.

[0096] Step 7-3: ABCD is a quadrilateral describing the lower edge of the vehicle. Slide quadrilateral ABCD along the height direction to form quadrilateral EFGH passing through P3, and connect the two quadrilaterals in the height direction; finally, a rectangular bounding box ABCD-EFGH describing the three-dimensional shape of the vehicle can be obtained.

[0097] Example

[0098] The present invention is applicable to the inversion of shape parameters and the reconstruction of three-dimensional bounding boxes of vehicles with any number of axles. In the following, a two-axle vehicle is taken as an example to describe the implementation of the present invention in detail. The implementation described in the accompanying drawings is exemplary and is only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0099] Figure 1 The present invention is a flow chart of a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on an external contour, the method comprising the following steps:

[0100] Step 1: Construct contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an open operation on the pixel connected domain, smooth the boundary of the pixel connected domain, and obtain a continuous and clear outer contour of the vehicle, such as Figure 2 As shown, a contour point set S1 is formed;

[0101] Step 2: Use the line L1 connecting the two frames of the same vehicle to divide the lower half of the point set S2, fit S2 into the function curve C1, search for the minimum point of C1 to collect the key points P1 and P2 of the wheel position, and determine that the number of axles is 2 and the number of wheels is 4;

[0102] Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 , S 2-2 , S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. 2-1 and S 2-(n+1) The shortest point P 21 , P 23 As a vehicle commander key point;

[0103] Step 4: The vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x , T y , keep sliding in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ;

[0104] Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1 and Q2 of the wheel position on the other side of the vehicle based on symmetry;

[0105] Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3;

[0106] Step 7: Based on the relative positions of the key points determined by the above process, the vehicle load position and geometric dimensions are determined, and the vehicle 3D bounding box is reconstructed.

[0107] like Figure 3 , step 2 specifically includes:

[0108] Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form a new point set S2;

[0109] Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis;

[0110] Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve;

[0111] Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key points P1 and P2 of the wheel position;

[0112] The vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. The wheel position and number of wheels on one side of the vehicle can be confirmed by searching the peaks; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base is linearly independent and has the maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1):

[0113]

[0114] In the formula, X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

[0115] like Figure 4 , step three specifically includes:

[0116] Step 3-1: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 , S 2-2 , S 2-3 ;

[0117] Step 3-2: Take the set S 2-2 , use the K-Means algorithm to divide the set into three clusters, all in S 2-2 The points form a new set S 2-2-1 ;

[0118] Step 3-3: S 2-2-1 Record it as the new set S3, and fit S3 into the straight line L2;

[0119] Step 3-4: Calculate L2 and S 2-1 and S 2-3 The distance between all points in the L2 point set S is the point with the smallest distance. 2-1 , S 2-3 The intersection point of the curves represented by 21 and P 23 Determined as the key point of vehicle length;

[0120] The K-Means algorithm is a clustering algorithm. Through multiple iterations, it can ensure that the distance from all points in each cluster to the nearest center point is minimized. The number of clusters is manually set initially, and the cluster centers are randomly generated. The cluster center coordinates are updated in each iteration. The update process is expressed as formula (2):

[0121]

[0122] In the formula, c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

[0123] like Figure 5 , step four specifically includes:

[0124] Step 4-1: Filter out the P in S2 21 The points on the right form a new geometry S4;

[0125] Step 4-2: Create a sliding box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points in n, record T x The x coordinate p at this time x , set the step size to 1, slide T along the x direction x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until;

[0126] Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the slide is continued in the y direction in the same way until it reaches P 21 Until then, record the y coordinate p when it stops y0 and P 20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set;

[0127] Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 It is the vehicle width control point.

[0128] like Figure 6 , step six specifically includes:

[0129] Step 6-1: Filter the points located above L1 in the vehicle outer contour S1 to form a set S5;

[0130] Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

[0131] Step 7 specifically includes:

[0132] Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1P2 intersects with the extension line of P1P2 at point B;

[0133] Step 7-2: Pass P 23 The vertical line L4 intersects with the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect with the extension line of P4P5 at point C.

[0134] Step 7-3: ABCD is a quadrilateral describing the lower edge of the vehicle. Slide the quadrilateral ABCD along the height direction to form a quadrilateral EFGH passing through P3. Connect the two quadrilaterals in the height direction. Finally, you can get the rectangular bounding box ABCD-EFGH that describes the three-dimensional shape of the vehicle.

[0135] The geometric reconstruction results are as follows Figure 7 As shown in the figure, the vehicle's outer contour can be identified by manual picking or using instance segmentation models such as Mask R-CNN, YOLCAT, and BlendMask.

[0136] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour are implemented.

[0137] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour.

[0138] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0139] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0140] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0141] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0142] The above is a detailed introduction to the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour proposed in the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on external contours, characterized in that: The method comprises the following steps: Step 1: Constructing contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an open operation on the pixel connected domain, smooth the boundary of the pixel connected domain, obtain the continuous and clear outer contour of the vehicle, and form the contour point set S1; Step 2: Use the line L1 connecting the two frames of the same vehicle to divide the lower half of the point set S2, fit S2 into the function curve C1, search for the minimum point of C1 to collect the key points of the wheel position P1, P2, P3...P m , determine the number of axles to be n and the number of wheels to be 2n; Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 , S 2-2 , S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. 2-1 and S 2-(n+1) The shortest point P 21 , P 23 As a vehicle commander key point; Step 4: The vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x , T y , keep sliding in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ; Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1, Q2, Q3, and Q2 on the other side of the vehicle based on symmetry. m ; Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3; Step 7: Based on the relative positions of the key points determined by the above process, the vehicle load position and geometric dimensions are determined, and the vehicle 3D bounding box is reconstructed.

2. The method according to claim 1, characterized in that The step 2 specifically includes: Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form a new point set S2; Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis; Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve; Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key point of the wheel position P1, P2, P3...P m .

3. The method according to claim 2, characterized in that The vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. The wheel position and number of wheels on one side of the vehicle can be confirmed by searching the peaks; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base are linearly independent and have the maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1): In the formula, X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

4. The method according to claim 3, characterized in that The step three specifically includes: Step 3-1: According to the position of the wheel position key points in S2, cut the point set S2 into several new sets S 2-1 , S 2-2 , S 2-3 ......S 2-(n+1) ; Step 3-2: Take the set S 2-2 , S 2-3 ......S 2-n , use the K-Means algorithm for each of the above sets to divide the set into three clusters, and all the points in the middle cluster constitute a new set S 2-2-1 , S 2-3-1 ......S 2-n-1 ; Step 3-3: S 2-2-1 , S 2-3-1 ......S 2-n-1 Gather into a new set S3, and fit S3 into a straight line L2; Step 3-4: Calculate L2 and S 2-1 and S 2(n+1) The distance between all points in the L2 point set S is the point with the smallest distance. 2-1 , S 2-3 The intersection point of the curves represented by 21 and P 23 Determined as the key point of vehicle length.

5. The method according to claim 4, characterized in that In the K-Means algorithm, multiple iterations are performed to ensure that the distance from all points in each cluster to the nearest center point is minimized. The number of clusters is manually set initially, and the cluster centers are randomly generated. The cluster center coordinates are updated in each iteration. The update process is expressed as formula (2): In the formula, c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

6. The method according to claim 5, characterized in that The step 4 specifically includes: Step 4-1: Filter out the P in S2 21 The points on the right form a new geometry S4; Step 4-2: Create a sliding box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points in n, record T x The x coordinate p at this time x , set the step size to 1, slide along the x direction T x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until; Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the slide is continued in the y direction in the same way until it reaches P 21 Until then, record the y coordinate p when it stops y and P 20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set; Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 It is the key point of vehicle width.

7. The method according to claim 6, characterized in that The step six specifically includes: Step 6-1: Filter the points located above L1 in the vehicle outer contour S1 to form a set S5; Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

8. The method according to claim 7, characterized in that The step seven specifically includes: Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1P2 intersects with the extension line of P1P2 at point B; Step 7-2: Pass P 23 The vertical line L4 intersects with the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect with the extension line of P4P5 at point C. Step 7-3: ABCD is a quadrilateral describing the lower edge of the vehicle. Slide quadrilateral ABCD along the height direction to form quadrilateral EFGH passing through P3, and connect the two quadrilaterals in the height direction; finally, a rectangular bounding box ABCD-EFGH describing the three-dimensional shape of the vehicle can be obtained.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images