A method for vehicle shape parameter inversion and 3D bounding box reconstruction based on external contour

Abstract

This paper proposes a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on the vehicle's external contour. This method utilizes morphological operations to continuously extract and process the appearance features of a moving vehicle's external contour in chronological order. Key points of the vehicle's spatial geometric parameters are collected to fully describe the vehicle's shape. Based on the relative positions of these key points, the vehicle's shape parameters are inverted and the three-dimensional bounding box of the vehicle model is reconstructed. This method addresses the difficulty of existing recognition methods in directly identifying vehicle load position and size information, and thus incapable of accurately reconstructing the vehicle's geometry in an image.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure health monitoring, and in particular to a vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on external contour. Background Art

[0002] Vehicle load is a key factor affecting bridge lifespan and forms the basis for assessing a bridge's ultimate bearing capacity. Bridge design must consider a variety of extreme effects, with vehicle overload being a key factor. Only with an accurate understanding of vehicle load levels can we scientifically predict the ultimate bearing capacity of a bridge within its design lifespan, thereby ensuring the bridge's structural safety reserve. With the rapid increase in traffic volume, the vehicle loads borne by highway bridges have also continued to grow, exacerbating structural fatigue damage. To ensure the knowledge and measurability of bridge loads and enhance disaster preparedness and emergency response capabilities, it is necessary to find a method to quickly determine the distribution of traffic flow on the bridge deck.

[0003] The key to identifying the distribution of traffic loads on a bridge deck is to accurately locate the load position of each vehicle. For the longitudinal position, if a vehicle approaches the mid-span, the beam will experience greater bending moments and deflections, while the support reactions will be less affected. Conversely, if it approaches the supports, the structural response will be relatively small, but the support reactions will increase significantly. Generally speaking, when the longitudinal deviation is 5% of the span, the maximum internal forces and deformations of a simply supported beam will increase by approximately 10%. For the lateral position, if a vehicle approaches the edge of the bridge deck too closely, significant torque and stress concentration will be introduced, leading to increased support reactions. If the lateral offset reaches 1 / 4 of the bridge deck width, the most unfavorable support reactions will increase by more than 15%, seriously threatening the safety of the bridge structure. Therefore, errors in the estimated load position have a significant impact on the safety assessment results of small-span bridges.

[0004] To determine the distribution of vehicle loads, most existing approaches simplify the axle load into a single concentrated load applied to the bridge deck. A convolutional neural network is used to identify the vehicle bounding box in the image, and the center of the bounding box is used as the approximate location of the concentrated load. For mixed traffic, a single load model is used. This overestimates the local effects of the load and underestimates the overall stress level, making it difficult to accurately capture the internal force distribution of bridge components under the combined action of multiple vehicle loads. This can ultimately lead to deviations in safety assessment results and a failure to reflect the actual stress level of the bridge. To ensure the authenticity of bridge safety assessments, the actual action of vehicle axle loads must be simulated as much as possible, requiring accurate identification of wheel positions.

[0005] Furthermore, vehicle geometry data provides valuable traffic behavior information for bridge health monitoring, which is crucial for assessing structural and operational risks, making management decisions, and guiding maintenance. For example, the dimensional data of large vehicles is closely related to their aerodynamic effects and can be used to evaluate the potential wind vibration and turbulence effects on bridges caused by vehicles passing through. Continuously acquired vehicle geometry information also helps analyze traffic flow composition, providing a basis for decision-making in the rational design of traffic organization and renovation plans. During periods of traffic congestion, if the distance between vehicles is too close, posing a risk of rear-end collisions, traffic restrictions can be implemented to quickly evacuate vehicles and ensure the safety of bridge structures. Identifying overly high and overwide vehicles can predict their impact and wear risks on bridge components, allowing for the implementation of appropriate preventative measures to avoid potential structural damage. Therefore, accurately acquiring vehicle geometry is also a crucial component of bridge risk management.

[0006] To reconstruct the 3D bounding box and obtain the vehicle's geometric parameters, some approaches attempt to create a dataset of vehicle components and use convolutional neural networks to identify vehicle parts such as wheels. Vehicle components (such as wheels, rear ends, and body parts) typically only occupy a few dozen pixels in an image. After several convolutions, the original features are completely blurred, and the vehicle can only be detected when it is extremely close to the camera. This means that convolutional neural networks are less effective at detecting small objects such as wheels and rear ends, and even if a corresponding dataset is established, fine segmentation is difficult.

[0007] The above analysis shows that existing methods identify vehicles as single rectangular boxes and simplify the vehicle load to a single concentrated load. The actual vehicle load acts on the ground through the center of the bounding box, but the distribution of the concentrated load at the axles is not considered. This makes it impossible to accurately locate the load position. Convolutional neural networks also fail to perform fine-grained segmentation of vehicle components, geometrically reconstruct the vehicle model, and obtain vehicle geometry information. Therefore, accurately identifying vehicle components and determining load location and geometry from bridge footage is of great practical engineering significance. Summary of the Invention

[0008] The purpose of the present invention is to solve the problem that existing recognition methods are difficult to directly identify the vehicle load position and size information and cannot accurately reconstruct the vehicle geometry in the image. A vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour is proposed.

[0009] The present invention is implemented by the following technical solution. The present invention proposes a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on the external contour, the method comprising the following steps:

[0010] Step 1: Constructing the contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an opening operation on the connected pixel domain, smooth the boundaries of the connected pixel domain, and obtain a continuous and clear outer contour of the vehicle to form the contour point set S1;

[0011] Step 2: Use the line L1 connecting the two frames of the same vehicle to split the lower half of the point set S2, fit S2 into the function curve C1, search for the minimum point of C1 to collect the key points of the wheel position P1, P2, P3...P m , determine the number of axles to be n and the number of wheels to be 2n;

[0012] Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 、S 2-2 、S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. The L2 and S 2-1 and S 2-(n+1) The shortest point P 21 、P 23 As a key point for the vehicle commander;

[0013] Step 4: Vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x , T y , continuously slide in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ;

[0014] Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1, Q2, Q3...Q on the other side of the vehicle based on symmetry. m ;

[0015] Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3;

[0016] Step 7: Based on the relative positions of the key points determined in the previous process, determine the vehicle load position and geometric dimensions, and reconstruct the vehicle's 3D bounding box.

[0017] Furthermore, the step 2 specifically includes:

[0018] Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form the new point set S2;

[0019] Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis;

[0020] Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve;

[0021] Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key point of the wheel position P1, P2, P3...P m .

[0022] Furthermore, the vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. By searching for peaks, the position and number of wheels on one side of the vehicle can be confirmed; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base are linearly independent and have maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1):

[0023]

[0024] Where X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

[0025] Furthermore, the step three specifically includes:

[0026] Step 3-1: According to the position of the wheel position key point in S2, cut the point set S2 into several new sets S 2-1 、S 2-2 、S 2-3 ......S 2-(n+1) ;

[0027] Step 3-2: Take the set S 2-2 、S 2-3 ......S 2-n , use the K-Means algorithm to divide each of the above sets into three clusters, and all the points in the middle cluster constitute a new set S 2-2-1 、S 2-3-1 ......S2-n-1 ;

[0028] Step 3-3: S 2-2-1 、S 2-3-1 ......S 2-n-1 Gather into a new set S3, and fit S3 into a straight line L2;

[0029] Step 3-4: Calculate L2 and S 2-1 and S 2(n+1) The distance between all points in the set is the point with the smallest distance, which is the line L2. 2-1 、S 2-3 The intersection of the curves represented by 21 and P 23 Determined as the key point of vehicle length.

[0030] Furthermore, in the K-Means algorithm, multiple iterations are performed to ensure that the distance from all points in each cluster to the nearest center point is minimized. The number of clusters is manually set initially, and the cluster centers are randomly generated. The cluster center coordinates are updated in each iteration. The update process is expressed as formula (2):

[0031]

[0032] Where c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

[0033] Furthermore, the step 4 specifically includes:

[0034] Step 4-1: Filter out the S2 in P 21 The points on the right form a new geometry S4;

[0035] Step 4-2: Create a sliding box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points n in T x The x coordinate p at this time x , set the step size to 1, slide T along the x direction x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until;

[0036] Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the same method is used to slide in the y direction until it reaches P 21 Until now, record the y coordinate p when it stops y and P20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set;

[0037] Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 is the key point of vehicle width.

[0038] Furthermore, the step six specifically includes:

[0039] Step 6-1: Filter the points located above L1 in the vehicle outer contour S1 to form a set S5;

[0040] Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

[0041] Furthermore, the step seven specifically includes:

[0042] Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1 and P2 intersects at point B;

[0043] Step 7-2: Pass P 23 The vertical line L4 intersects the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect it with the extension line of P4P5 at point C.

[0044] Step 7-3: ABCD is the quadrilateral that describes the lower edge of the vehicle. Slide quadrilateral ABCD along the height direction to form quadrilateral EFGH passing through P3. Connect the two quadrilaterals in the height direction. Finally, you can get the rectangular bounding box ABCD-EFGH that describes the three-dimensional shape of the vehicle.

[0045] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour are implemented.

[0046] The present invention also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The present invention proposes a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on the external contour. This method can effectively collect key wheel position points and vehicle geometric key points in the vehicle contour curve, directly identify the actual load position and geometric size information of the vehicle, and reconstruct the three-dimensional bounding box of the vehicle.

[0049] 2. The present invention takes the invariance of the vehicle's external contour as its starting point, identifies the key points of the vehicle's spatial geometric parameters, and reconstructs the vehicle's three-dimensional bounding box model. It has good stability and high accuracy, and solves the problem of poor recognition of vehicle components such as wheels by convolutional neural networks, providing reliable traffic information for bridge structure health monitoring.

[0050] 3. Since the present invention does not use convolutional neural networks to directly identify vehicle parts, it avoids the labeling work of large data sets. The vehicle outline can be identified by the instance segmentation model trained with the open source data set, which greatly reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0052] Figure 1 is a simplified flow chart of a method according to an embodiment of the present invention;

[0053] Figure 2 is an external outline diagram of a vehicle according to an embodiment of the present invention;

[0054] Figure 3 Schematic diagram of the wheel position key point search method of the present invention;

[0055] Figure 4 Schematic diagram of the vehicle length key point search method of the present invention;

[0056] Figure 5 Schematic diagram of the vehicle width key point search method of the present invention;

[0057] Figure 6 Schematic diagram of the vehicle height key point search method of the present invention;

[0058] Figure 7 It is a schematic diagram of the geometric reconstruction result of the present invention. DETAILED DESCRIPTION

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0060] This invention proposes a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on the vehicle's external contour. Using morphological operations, the method continuously extracts and processes the appearance features of a moving vehicle's external contour in chronological order, collecting key points of vehicle spatial geometric parameters that fully describe the vehicle's shape. Based on the relative positions of these key points, the vehicle's shape parameters are inverted, and a three-dimensional bounding box of the vehicle model is reconstructed. The vehicle's external contour is a connected domain of pixels that completely covers the vehicle's shape. The vehicle's external contour can be identified manually or using instance-based segmentation models such as Mask R-CNN, YOLCAT, and BlendMask. The vehicle's shape parameters are a data description of the vehicle's geometric dimensions and wheel positions in an image. The three-dimensional bounding box is a rectangular parallelepiped bounding box that completely encloses the vehicle area, constructed using the relative positions of the key points of the vehicle's spatial geometric parameters in the image. These key points include two types: control points in the image that locate vehicle components, including wheel position key points; and control points in the image that describe vehicle dimensions, including vehicle length key points, vehicle width key points, and vehicle height key points.

[0061] Specifically, combined Figure 1-Figure 7 The present invention proposes a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on an external contour, the method comprising the following steps:

[0062] Step 1: Constructing the contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an opening operation on the connected pixel domain, smooth the boundaries of the connected pixel domain, and obtain a continuous and clear outer contour of the vehicle to form the contour point set S1;

[0063] Step 2: Use the line L1 connecting the two frames of the same vehicle to split the lower half of the point set S2, fit S2 into the function curve C1, search for the minimum point of C1 to collect the key points of the wheel position P1, P2, P3...P m , determine the number of axles to be n and the number of wheels to be 2n;

[0064] The second step specifically includes:

[0065] Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form the new point set S2;

[0066] Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis;

[0067] Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve;

[0068] Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key point of the wheel position P1, P2, P3...P m .

[0069] The vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. By searching for peaks, the wheel position and number of wheels on one side of the vehicle can be confirmed; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base are linearly independent and have maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1):

[0070]

[0071] Where X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

[0072] Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 、S 2-2 、S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. The L2 and S 2-1 and S 2-(n+1) The shortest point P 21 、P 23 As a key point for the vehicle commander;

[0073] The step three specifically includes:

[0074] Step 3-1: According to the position of the wheel position key point in S2, cut the point set S2 into several new sets S 2-1 、S 2-2 、S 2-3 ......S 2-(n+1) ;

[0075] Step 3-2: Take the set S 2-2 、S 2-3 ......S 2-n , use the K-Means algorithm to divide each of the above sets into three clusters, and all the points in the middle cluster constitute a new set S 2-2-1 、S 2-3-1 ......S 2-n-1 ;

[0076] Step 3-3: S 2-2-1 、S 2-3-1 ......S 2-n-1 Gather into a new set S3, and fit S3 into a straight line L2;

[0077] Step 3-4: Calculate L2 and S 2-1 and S 2(n+1) The distance between all points in the set is the point with the smallest distance, which is the line L2. 2-1 、S 2-3 The intersection of the curves represented by 21 and P 23 Determined as the key point of vehicle length.

[0078] The K-Means algorithm is a clustering algorithm. In the K-Means algorithm, multiple iterations are performed to ensure that the distance from all points in each cluster to the nearest center point is minimized. The number of clusters is manually set initially, and the cluster centers are randomly generated. The cluster center coordinates are updated in each iteration. The update process is expressed as formula (2):

[0079]

[0080] Where c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

[0081] Step 4: Vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x , T y , continuously slide in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ;

[0082] The step 4 specifically includes:

[0083] Step 4-1: Filter out the S2 in P 21 The points on the right form a new geometry S4;

[0084] Step 4-2: Create Sliding Box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points n in T x The x coordinate p at this time x , set the step size to 1, slide T along the x direction x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until;

[0085] Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the same method is used to slide in the y direction until it reaches P 21 Until now, record the y coordinate p when it stops y and P 20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set;

[0086] Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 is the key point of vehicle width.

[0087] Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1, Q2, Q3...Q on the other side of the vehicle based on symmetry. m ;

[0088] Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3;

[0089] The step six specifically includes:

[0090] Step 6-1: Filter the points above L1 in the vehicle outer contour S1 to form a set S5;

[0091] Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

[0092] Step 7: Based on the relative positions of the key points determined in the previous process, determine the vehicle load position and geometric dimensions, and reconstruct the vehicle's 3D bounding box.

[0093] The step seven specifically includes:

[0094] Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1 and P2 intersects at point B;

[0095] Step 7-2: Pass P 23 The vertical line L4 intersects the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect it with the extension line of P4P5 at point C.

[0096] Step 7-3: ABCD is the quadrilateral that describes the lower edge of the vehicle. Slide quadrilateral ABCD along the height direction to form quadrilateral EFGH passing through P3. Connect the two quadrilaterals in the height direction. Finally, you can get the rectangular bounding box ABCD-EFGH that describes the three-dimensional shape of the vehicle.

[0097] Example

[0098] The present invention is applicable to the inversion of shape parameters and 3D bounding box reconstruction of vehicles with any number of axles. Below, a two-axle vehicle is used as an example to describe the present invention in detail. The embodiments depicted in the accompanying drawings are exemplary and intended only to explain the present invention, and are not to be construed as limiting the present invention.

[0099] Figure 1 The present invention is a simplified flow chart of a method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on an external contour, the method comprising the following steps:

[0100] Step 1: Construct the contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an open operation on the pixel connected domain, smooth the boundary of the pixel connected domain, and obtain a continuous and clear outer contour of the vehicle, such as Figure 2 As shown, a contour point set S1 is formed;

[0101] Step 2: Use the line L1 connecting the two mass points of the same vehicle to split the lower half of the point set S2, fit S2 to the function curve C1, search for the minimum point of C1 to collect the key points P1 and P2 of the wheel position, and determine the number of axles to be 2 and the number of wheels to be 4;

[0102] Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 、S 2-2 、S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. The L2 and S 2-1 and S 2-(n+1) The shortest point P 21 、P 23 As a key point for the vehicle commander;

[0103] Step 4: Vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x 、T y , continuously slide in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ;

[0104] Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1 and Q2 of the wheel position on the other side of the vehicle based on symmetry;

[0105] Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3;

[0106] Step 7: Based on the relative positions of the key points determined in the previous process, determine the vehicle load position and geometric dimensions, and reconstruct the vehicle's 3D bounding box.

[0107] like Figure 3 , step 2 specifically includes:

[0108] Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form the new point set S2;

[0109] Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis;

[0110] Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve;

[0111] Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key points P1 and P2 of the wheel position;

[0112] The vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. By searching for peaks, the wheel position and number of wheels on one side of the vehicle can be confirmed; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base are linearly independent and have maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1):

[0113]

[0114] Where X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

[0115] like Figure 4 , step three specifically includes:

[0116] Step 3-1: Cut the point set S2 into a new set S according to the position of the wheel position key point in S2 2-1 、S 2-2 、S 2-3 ;

[0117] Step 3-2: Take the set S 2-2 , the K-Means algorithm is used to divide the set into three clusters, all of which are located in S 2-2 The points form a new set S 2-2-1 ;

[0118] Step 3-3: S 2-2-1 Record it as the new set S3, and fit S3 into the straight line L2;

[0119] Step 3-4: Calculate L2 and S 2-1 and S 2-3 The distance between all points in the set is the point with the smallest distance, which is the line L2. 2-1 、S 2-3 The intersection of the curves represented by 21 and P 23 Determined as the key point of vehicle length;

[0120] The K-Means algorithm is a clustering algorithm that can ensure that the distance from all points in each cluster to the nearest center point is minimized through multiple iterations. The number of clusters is manually set initially, and the cluster centers are randomly generated. The cluster center coordinates are updated in each iteration. The update process is expressed as formula (2):

[0121]

[0122] Where c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

[0123] like Figure 5 , step four specifically includes:

[0124] Step 4-1: Filter out the S2 in P 21 The points on the right form a new geometry S4;

[0125] Step 4-2: Create a sliding box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points n in T x The x coordinate p at this time x , set the step size to 1, slide T along the x direction x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until;

[0126] Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the same method is used to slide in the y direction until it reaches P 21 Until now, record the y coordinate p when it stops y0 and P 20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set;

[0127] Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 is the vehicle width control point.

[0128] like Figure 6 , step six specifically includes:

[0129] Step 6-1: Filter the points located above L1 in the vehicle outer contour S1 to form a set S5;

[0130] Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

[0131] Step seven specifically includes:

[0132] Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1 and P2 intersects at point B;

[0133] Step 7-2: Pass P 23 The vertical line L4 intersects the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect it with the extension line of P4P5 at point C.

[0134] Step 7-3: ABCD is the quadrilateral that describes the bottom edge of the vehicle. Slide quadrilateral ABCD vertically to form quadrilateral EFGH, which passes through P3. Connect the two quadrilaterals vertically. Ultimately, you get the rectangular bounding box ABCD-EFGH that describes the vehicle's 3D shape.

[0135] The geometric reconstruction results are as follows Figure 7 As shown in the figure, the vehicle's outer contour can be identified by manual picking or using instance segmentation models such as Mask R-CNN, YOLCAT, and BlendMask.

[0136] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour are implemented.

[0137] The present invention also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour.

[0138] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0139] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0140] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0141] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0142] The above is a detailed introduction to the vehicle shape parameter inversion and three-dimensional bounding box reconstruction method based on the external contour proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for inverting vehicle shape parameters and reconstructing a three-dimensional bounding box based on external contours, characterized in that: The method comprises the following steps: Step 1: Constructing the contour point set S1: Extract the outer contour of the same vehicle from the vehicle driving video in chronological order, perform an opening operation on the connected pixel domain, smooth the boundaries of the connected pixel domain, and obtain a continuous and clear outer contour of the vehicle to form the contour point set S1; Step 2: Use the line L1 connecting the two frames of the same vehicle to split the lower half of the point set S2, fit S2 into the function curve C1, search for the minimum point of C1 to collect the key points of the wheel position P1, P2, P3...P m , determine the number of axles to be n and the number of wheels to be 2n; Step 3: According to the position of the wheel position key point in S2, cut the point set S2 into a new set S 2-1 、S 2-2 、S 2-3 , remove S 2-2 The remaining points are used to generate a set S3, and the straight line L2 describing the vehicle body is fitted. The L2 and S 2-1 and S 2-(n+1) The shortest point P 21 、P 23 As a key point for the vehicle commander; Step 4: Vehicle width and length share the key point P 21 , filter the vehicle tail contour point set S4, and create a sliding box T x , T y , continuously slide in the corresponding direction, count the number of landing points in the box, search for the mutation position of the number of landing points, and collect the key points P of the vehicle width 21 and P 22 ; Step 5: Connect P 21 and P 22 To calculate the vehicle width, restore the key points Q1, Q2, Q3...Q on the other side of the vehicle based on symmetry. m ; Step 6: Use the line L1 connecting the two frames of the same vehicle to split the upper half of the point set S3, and collect the endpoint vehicle height key point P3 farthest from L1 in the point set S3; Step 7: Based on the relative positions of the key points determined in the previous process, determine the vehicle load position and geometric dimensions, and reconstruct the vehicle's 3D bounding box.

2. The method according to claim 1, characterized in that The second step specifically includes: Step 2-1: Use the line connecting the vehicle mass points between the two frames to construct the segmentation line L1, divide the contour point set S1 into two parts, and take the lower half to form the new point set S2; Step 2-2: Use the PCA algorithm to decorrelate S2, construct a new orthogonal basis, and rotate S2 onto the orthogonal basis; Step 2-3: Fit the point set S2 on the new orthogonal basis into a polynomial function f(x), and search for the minimum point of the polynomial function f(x) curve; Step 2-4: Use inverse transformation to restore the minimum point back to the original image coordinate system, and determine the obtained point as the key point of the wheel position P1, P2, P3...P m .

3. The method according to claim 2, characterized in that The vehicle mass point is the center of the vehicle contour, and the center of gravity of the contour point set S1 is calculated; the polynomial function f(x) is composed of multiple quadratic functions, and the number of peaks of its function curve C1 corresponds to the number of axles, that is, one peak corresponds to two wheels. By searching for peaks, the wheel position and number of wheels on one side of the vehicle can be confirmed; the PCA algorithm is a principal component analysis method, which can rotate the lower contour S2 to a set of new orthogonal bases. The data in each direction of the orthogonal base are linearly independent and have maximum difference, which is convenient for highlighting the extreme point position and fitting the polynomial function f(x). The PCA algorithm is expressed as formula (1): Where X is the original observation data; Y is the new data after rotation to the main direction; is the mean of the original observation data X; S is the standard deviation of the original observation data X.

4. The method according to claim 3, characterized in that The step three specifically includes: Step 3-1: According to the position of the wheel position key point in S2, cut the point set S2 into several new sets S 2-1 、S 2-2 、S 2-3 ......S 2-(n+1) ; Step 3-2: Take the set S 2-2 、S 2-3 ......S 2-n , use the K-Means algorithm to divide each of the above sets into three clusters, and all the points in the middle cluster constitute a new set S 2-2-1 、S 2-3-1 ......S 2-n-1 ; Step 3-3: S 2-2-1 、S 2-3-1 ......S 2-n-1 Gather into a new set S3, and fit S3 into a straight line L2; Step 3-4: Calculate L2 and S 2-1 and S 2(n+1) The distance between all points in the set is the point with the smallest distance, which is the line L2. 2-1 、S 2-3 The intersection of the curves represented by 21 and P 23 Determined as the key point of vehicle length.

5. The method according to claim 4, characterized in that In the K-Means algorithm, multiple iterations are performed to ensure that the distance from all points in each cluster to the nearest center point is minimized. The number of clusters is manually set initially, and the cluster centers are randomly generated. The coordinates of the cluster centers are updated in each iteration. The update process is expressed as formula (2): Where c is the new cluster center of cluster j; n is the number of points in cluster j; a j is the midpoint position of cluster j.

6. The method according to claim 5, characterized in that The step 4 specifically includes: Step 4-1: Filter out the S2 in P 21 The points on the right form a new geometry S4; Step 4-2: Create Sliding Box T x , the length in the x direction is 1 / 50 of the interval length, the initial position of the sliding box is consistent with the left side of the target bounding box, and the statistics S4 falls on T x The number of points n in T x The x coordinate p at this time x , set the step size to 1, slide T along the x direction x , if the number of points counted next time is greater than n, update p x , until the sliding frame reaches P 21 until; Step 4-3: Create Sliding Box T y , the length in the y direction is 1 / 20 of the interval length, and the same method is used to slide in the y direction until it reaches P 21 Until now, record the y coordinate p when it stops y and P 20 (p x , p y0 ), y0 is the lower limit of the y-direction range of the point set; Step 4-4: Calculate all horizontal coordinates in S4 as P x The point and P 20 The distance to the nearest point P 22 The coordinates of (p x , p y ), determine P 21 and P 22 is the key point of vehicle width.

7. The method according to claim 6, characterized in that The step six specifically includes: Step 6-1: Filter the points located above L1 in the vehicle outer contour S1 to form a set S5; Step 6-2: Calculate the distance between all points in S5 and L1. The endpoint with the farthest distance is the vehicle height key point P3.

8. The method according to claim 7, characterized in that The step seven specifically includes: Step 7-1: Connect P1P2, P4P5 and P 21 P 22 , over P 22 The vertical line L3 and the extension line of P4P5 intersect at point A. 21 P 22 The parallel line of P1 and P2 intersects at point B; Step 7-2: Pass P 23 The vertical line L4 intersects the extension line of P1P2 at point D. Draw a line parallel to P1P2 through D and intersect it with the extension line of P4P5 at point C. Step 7-3: ABCD is the quadrilateral that describes the lower edge of the vehicle. Slide quadrilateral ABCD along the height direction to form quadrilateral EFGH passing through P3. Connect the two quadrilaterals in the height direction. Finally, you can get the rectangular bounding box ABCD-EFGH that describes the three-dimensional shape of the vehicle.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images