Visual measurement method for vehicle model parameters of intelligent parking system

Abstract

The invention relates to a vehicle model parameter visual measurement method for an intelligent parking system. The method comprises the following steps: acquiring an image of a to-be-parked vehicle; the vehicle image is preprocessed, the vehicle type is identified, and a three-dimensional vehicle model is constructed; on the basis of a three-dimensional vehicle model, boundary dimension parameters of a to-be-parked vehicle are determined through visual measurement and are transmitted by a system, a parking space most suitable for storing the to-be-parked vehicle is matched by combining the spatial dimension of a stereo garage, and then the to-be-parked vehicle is automatically conveyed to a designated parking space through a consignment robot with the corresponding dimension. Aiming at the characteristics of the overall dimension of the passenger vehicle, the invention proposes a vision method combining single vision and binocular vision, designs a visual measurement method for the overall dimension of the passenger vehicle based on an improved genetic algorithm, realizes the type recognition of the vehicle and the visualization of the overall dimension parameters of the vehicle, improves the detection efficiency and the accuracy of a measurement result, and improves the accuracy of the overall dimension of the passenger vehicle. Reliable data parameters are provided for automatic model selection of a carrying robot and garage size model selection in an intelligent parking system.

Description

Technical Field

[0001] The present invention relates to the field of visual measurement technology, and in particular to a method for visually measuring vehicle model parameters of an intelligent parking system. Background Art

[0002] Existing urban parking lots do not have much intelligent control and require personnel supervision, which cannot solve the problem of inefficient parking lot management. Many large parking lot intelligent systems are imperfect. They only display the number of empty parking spaces, but cannot tell the specific location of the empty spaces and the best way to reach the empty parking spaces. It is not easy for car owners to find empty parking spaces when they park and pick up their vehicles autonomously, which is time-consuming and labor-intensive. At present, intelligent parking lots are constantly improving, but there are still various problems that do not meet user needs. This requires the new intelligent parking system to be more intelligent, automated, and humanized; the intelligent parking system is part of the intelligent parking system, and the research on the intelligent parking system is a supplement to the intelligent parking system. In the intelligent parking system, the vehicle is parked in the designated parking space by a transport robot. In order to better select the appropriate transport robot for different models, accurate vehicle model and vehicle outline size information is required, which requires the real-time measurement of the vehicle size. The traditional measurement method relies on manual labor and is inefficient. Human factors will have a significant impact on the measurement results, and it is no longer suitable for the needs of automated measurement; machine vision measurement has the advantages of non-contact measurement, good reproducibility, high precision, and high reliability, and is widely used in vehicle outline size measurement.

[0003] In the context of the rapid development of machine vision technology for measuring vehicle contour dimensions, many different vision-based vehicle dimension measurement methods have been proposed. Li S et al. used a monocular camera to combine the data identified by the car and the license plate recognition system to achieve the measurement of the three-dimensional parameters of the vehicle. It does not require any training, and can be achieved by relying on basic computer vision technologies such as ROI extraction, edge detection, and spatial coordinate transformation. It has good detection stability, but the algorithm uses the information of the license plate as the measurement standard. In practical applications, it is difficult to accurately extract the image size of the area where the license plate is located, which affects the scope of use of this method; Feng Fan et al. studied and designed an algorithm for the numerical measurement of the length dimension of a vehicle body image based on the positioning of the vehicle with a monocular camera. The algorithm of this dimension measurement method first performs secondary denoising and fusion on the pixels of each point in the vehicle image, and then uses a grayscale image projection method to accurately locate the image of the head or tail point of the entire vehicle in turn, and finally combines the focal length and angle of the monocular camera to calculate the body size of the entire vehicle in the detection image. This measurement method makes the measured data processing speed fast and high-precision, but there is a problem that the measured parameters are relatively small, so it has not been widely adopted. Yuan Han and other researchers proposed a method for automatic measurement of vehicle outline dimensions based on large binocular automatic vision. The hardware structure of this measurement method is mainly composed of two pairs of large binocular vision cameras installed on the top and side of the vehicle. First, the collected image is segmented by combining the single Gaussian optical model and the background optical addition and subtraction algorithm to calculate the entire vehicle body, and then stereo matching based on Harris corner point features is performed on it. The length and width of the vehicle body are accurately calculated by combining the optical principle of binocular vision. This measurement system can measure a lot of parameter data and has a good measurement effect, but it is not able to perform targeted parameter measurements according to the actual conditions of various models; Matthew et al. first proposed a super-large truck height ranging technology based on computer vision. This detection method is to automatically collect the information of the vehicle in motion through a monitoring image camera connected to an overpass or tunnel, so as to transmit the video stream, and then use straight line detection and spot tracing to judge the upper and lower points on each truck in the corresponding pixel position coordinates, and convert them into coordinates in the 3D world, and finally accurately measure the height of each truck. Since this method only needs to calculate a small amount of vehicle information, the work is relatively time-consuming, but the measured vehicle size parameters are very limited.

[0004] From the research results obtained, it can be seen that the current vehicle size measurement system has potential problems such as low measurement accuracy, incomplete measurement parameters, long measurement time, etc., which leads to some results not being widely used. Summary of the invention

[0005] The purpose of the present invention is to propose a visual measurement method for vehicle model parameters of an intelligent parking system to solve the problems existing in the above-mentioned prior art. According to the characteristics of the outline size of a passenger car, a visual method combining monocular and binocular vision is proposed, and a passenger car outline size visual measurement system based on an improved genetic algorithm is designed, which realizes the type recognition of the vehicle and the visualization of the vehicle outline size parameters, improves the efficiency of detection and the accuracy of the measurement results, and provides reliable data parameters for the automatic selection of the carrier robot and the garage size selection in the intelligent parking system.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] The visual measurement method of vehicle model parameters of the intelligent parking system includes:

[0008] Collecting images of vehicles to be parked;

[0009] Preprocessing the vehicle image to identify the vehicle type;

[0010] Based on the identified vehicle type, stereo matching is performed on the vehicle to be parked to construct a three-dimensional vehicle model;

[0011] Based on the three-dimensional vehicle model, measurement parameters are determined and visual measurement is performed.

[0012] Optionally, preprocessing the vehicle image includes:

[0013] Performing edge detection on the vehicle image;

[0014] Extract feature points and process feature lines on the vehicle image after edge detection;

[0015] Based on the extracted feature points and feature lines, the vehicle type is identified.

[0016] Optionally, the method for edge detection of the vehicle image is: using a genetic algorithm combined with improved beetle whisker search, including:

[0017] S1.1. grayscale processing of the vehicle image, setting the initial value of the algorithm;

[0018] S1.2. Perform chromosome encoding on the grayscale values ​​of pixels and randomly select the primary population;

[0019] S1.3. Calculate the gradient change rate of each individual in the population based on a preset formula and calculate the fitness value of the individual;

[0020] S1.4. Producing offspring through crossover and mutation;

[0021] S1.5. Use the improved beetle whisker search method to guide the movement of the parent crossover point position involved in the crossover operation in the new generation population and update the offspring;

[0022] S1.6 If i < T, then i = i + 1 and go to S1.3; if i = T, then go to S1.7; where i is the number of iterations that have been performed, and T is the upper limit of the number of iterations.

[0023] S1.7 Output the optimal solution in the population, that is, the detected image edge.

[0024] Wherein, the preset formula is:

[0025]

[0026] Where is the gradient direction, is the partial derivative of the function f with respect to the x coordinate, is the partial derivative of the function f with respect to the y coordinate, and θ is the angle between the gradient direction and the x-axis.

[0027] The improved beetle antennae search method is:

[0028] Change the search step length by changing the step length parameter of the original beetle antennae search. First, determine the range of the search direction with the first preset step length, and then perform a fine search with the second preset step length.

[0029] Optionally, the extraction of feature points and feature lines from the vehicle image after edge detection includes:

[0030] Use the Harris algorithm to extract feature points in the vehicle image after edge detection;

[0031] Convert the pixels of the feature points to the sub-pixel level;

[0032] Use the Hough transform to extract feature lines in the vehicle image after edge detection.

[0033] Optionally, based on the recognized vehicle type, perform stereo matching on the vehicle to be parked and construct a three-dimensional vehicle model, including:

[0034] S2.1. Use the left and right cameras to collect the left vehicle image and the right vehicle image of the vehicle to be parked;

[0035] S2.2. Use the Sobel operator to perform edge detection on the left vehicle image and the right vehicle image to obtain edge points;

[0036] S2.3. Based on the camera calibration parameters, obtain the fundamental matrix of the left and right cameras;

[0037] S2.4. Select an arbitrary edge point in the image captured by the left camera, and calculate the corresponding epipolar line in the image captured by the right camera by using the basic matrix, which is recorded as l2; select an arbitrary point on l2, and calculate the corresponding epipolar line in the image captured by the left camera by using the basic matrix, which is recorded as l1; wherein all edge points on l1 and l2 correspond one to one;

[0038] S2.5. Based on the preset similarity evaluation function, calculate the evaluation value of the similarity between the edge points on l1 and l2 and establish a two-dimensional similarity evaluation function list M, in which the units M(i,j) are the similarity evaluation values ​​of the i-th point on l1 and the j-th point on l2 respectively; if the threshold of a certain unit M(i,j) in the list is the smallest in its row and column, and the threshold level is less than the threshold of a certain unit, then the i-th point on l1 and the j-th point on l2 are preliminarily determined to correspond to one point;

[0039] S2.6. Repeat S2.5 until all edge feature points are matched and calculated. After each overall matching of the two epipolar lines, the points that have been calculated will no longer appear in subsequent calculations, that is, the uniqueness constraint is satisfied.

[0040] S2.7. According to the limited gradient of the screening disparity value, a corresponding screening is performed among the candidate related pixel points obtained under the preset constraint conditions to obtain the optimized matching points;

[0041] S2.8. Based on the optimized matching points, find the ideal matching point pair closest to the epipolar line, obtain the final precise matching point pair, calculate the three-dimensional coordinates of the vehicle based on the final precise matching point pair, and construct a three-dimensional vehicle model.

[0042] Optionally, the preset similarity evaluation function is:

[0043]

[0044] Where f(x,y) is the matching function used to evaluate the matching degree between point (u,v) and point (u+x,v+y) in image 1 and image 2, M is the normalization factor to ensure that the value of the matching function is within a reasonable range, ∑ (u，υ)∈W is the sum of all points (u, v) in window W, I1(u, v) is the gray value of point (u, v) in image 1, is the average gray value of image 1, I2(u+x,v+y) is the gray value of the midpoint (u+x,v+y) of image 2, is the average gray value of image 2, S(x,y) is the final matching score function, which combines gray matching and gradient matching, f(x,y) is the gray matching score, and k is the proportional probability parameter used to balance the weights of gray matching and gradient matching. is the gradient difference between image 1 and image 2 in the x direction, is the gradient difference between image 1 and image 2 in the y direction, u0 is the horizontal coordinate of the reference point in image 1, and v0 is the vertical coordinate of the reference point in image 1.

[0045] The preset constraint conditions include: epipolar constraint, uniqueness constraint, mutual correspondence constraint, compatibility constraint and similarity constraint, and parallax continuity constraint.

[0046] Optionally, determining measurement parameters based on the three-dimensional vehicle model and performing visual measurement includes:

[0047] Determine measurement parameters; wherein the measurement parameters include: vehicle length, vehicle width, vehicle height, wheelbase, front overhang, and rear overhang;

[0048] Based on the three-dimensional vehicle model, the measurement points of the vehicle length, vehicle height, wheelbase, front overhang and rear overhang are positioned by using a grayscale projection method, and based on the positioned measurement points, the vehicle length, vehicle height, wheelbase, front overhang and rear overhang are measured.

[0049] Optionally, measuring the vehicle length, width and height includes:

[0050] Using sensors to measure the width of the vehicle;

[0051] According to the center of gravity principle, the center of gravity of the 3D vehicle model is located at the geometric center of the 3D vehicle model, the position of the rotation axis is determined, each row is scanned and the center of gravity coordinates (x, y) of each row are calculated;

[0052] The center of gravity coordinates (x j ,y j )(j=1,2,...,M) is a set of data, and the least square method is used to solve the linear equation y of the initial position of the vertical principal axis j =k1x j +b1;

[0053] Similarly, scan each column in turn and calculate the centroid coordinates (x i ,y i )(i=1,2,...,N), use the least squares method to get the linear equation y of the initial horizontal position i =k2x i +b2;

[0054] After solving the two linear equations, the intersection point (x0, y0) is found; a line passing through the intersection point rotates around the intersection point, and the initial position of the line starts from being parallel to the horizontal axis and gradually changes to a position perpendicular to the horizontal axis. The direction of rotation gradually increases in the direction of the slope of the horizontal axis; when the surface area of ​​a rectangle that can contain the entire body around the body is the smallest, the resulting rectangle is the smallest circumscribed rectangle around the body;

[0055] The length of the minimum circumscribed rectangle is the length of the vehicle, and the row coordinates of the upper side corresponding to the width of the minimum circumscribed rectangle minus the row coordinates of the lowest point of the wheel obtained by Hough transform is the distance of the vehicle height in the image, which is then calculated by the coordinate system transformation matrix to obtain the actual vehicle height.

[0056] Optionally, measuring the wheelbase includes:

[0057] The wheel position is detected by adding the preset condition of Hough transform, and the center points of the two circles detected are the wheelbase measurement points;

[0058] Wherein, the preset condition of the Hough transform is:

[0059]

[0060] Where F(x,y) is the pixel, G(x,y) is the region of interest required in the transformation, w and h represent the length and width of the image, abs(y1-y2)≤3 means that the two centers are basically on the same horizontal line, and abs(x1-x2)≥w / 2 represents the horizontal distance constraint between the two centers.

[0061] The beneficial effects of the present invention are:

[0062] Aiming at the characteristics of passenger car outline dimensions, the present invention proposes a visual method combining monocular and binocular vision, designs a passenger car outline dimension visual measurement system based on an improved genetic algorithm, realizes vehicle type recognition and visualization of vehicle outline dimension parameters, improves detection efficiency and accuracy of measurement results, and provides reliable data parameters for automatic selection of carrier robots and garage size selection in intelligent parking systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0064] Figure 1 A flowchart of an algorithm for applying a genetic algorithm combined with beetle whisker search to image edge detection according to an embodiment of the present invention;

[0065] Figure 2 : This is an edge image comparison of an embodiment of the present invention; wherein, (a) is an original pixel point, (b) is a pixel point based on a genetic algorithm, (c) is a pixel point using an improved genetic algorithm, (d) is an original image, (e) is a detected edge based on a genetic algorithm, (f) is a detected edge after the improved algorithm, (g) is an original image, (h) is a detected edge based on a genetic algorithm, and (i) is a detected edge after the improved algorithm;

[0066] Figure 3 Schematic diagram of calculating corner points with sub-pixel accuracy according to an embodiment of the present invention; wherein (a) is point p within the region, and (b) is point p at the edge of the region;

[0067] Figure 4 is a schematic diagram of a classification system according to an embodiment of the present invention;

[0068] Figure 5 A schematic diagram of determining theoretical matching points of an epipolar bundle according to an embodiment of the present invention;

[0069] Figure 6 A schematic diagram of a measurement process according to an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of vehicle size measurement parameters according to an embodiment of the present invention;

[0071] Figure 8 Schematic diagram of horizontal projection of a vehicle image according to an embodiment of the present invention; wherein (a) is a vehicle image, and (b) is a pixel horizontal projection;

[0072] Fig. 9 Schematic diagram of the process of finding the best measurement point according to an embodiment of the present invention; wherein (a) is for finding the convex hull point, and (b) is for finding the measurement point using the rotating caliper algorithm;

[0073] Fig.10 Schematic diagram of the minimum circumscribed rectangle of a vehicle body according to an embodiment of the present invention;

[0074] Fig.11 Schematic diagram of Hough transform circle detection in an embodiment of the present invention; wherein (a) is unconstrained Hough circle detection, and (b) is constrained Hough circle detection;

[0075] Fig.12 This is a diagram of the vehicle size measurement interface according to an embodiment of the present invention;

[0076] Fig.13 This is a vehicle model data transmission diagram according to an embodiment of the present invention;

[0077] Fig.14 The figure is a flow chart of a method for visually measuring vehicle model parameters of an intelligent parking system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0080] like Fig.14 As shown, this embodiment proposes a method for visually measuring vehicle model parameters of an intelligent parking system, including:

[0081] Collecting images of vehicles to be parked;

[0082] Preprocessing the vehicle image to identify the vehicle type;

[0083] Based on the identified vehicle type, stereo matching is performed on the vehicle to be parked to construct a three-dimensional vehicle model;

[0084] Based on the three-dimensional vehicle model, measurement parameters are determined and visual measurement is performed.

[0085] Further, preprocessing the vehicle image includes:

[0086] Performing edge detection on the vehicle image;

[0087] Extract feature points and process feature lines on the vehicle image after edge detection;

[0088] Based on the extracted feature points and feature lines, the vehicle type is identified.

[0089] Furthermore, the method for edge detection of the vehicle image is: using a genetic algorithm combined with improved beetle whisker search, including:

[0090] S1.1. grayscale processing of the vehicle image, setting the initial value of the algorithm;

[0091] S1.2. Perform chromosome encoding on the grayscale values ​​of pixels and randomly select the primary population;

[0092] S1.3. Calculate the gradient change rate of each individual in the population based on a preset formula and calculate the fitness value of the individual;

[0093] S1.4. Producing offspring through crossover and mutation;

[0094] S1.5. Use the improved beetle antennae search method to guide the movement of the crossover points of the parents participating in the crossover operation in the new generation of population, and update the offspring.

[0095] S1.6 If i < T, then i = i + 1, go to S1.3; otherwise, stop the iteration.

[0096] S1.7. The algorithm ends, and the optimal solution in the population is output, that is, the detected image edge.

[0097] The improved beetle antennae search method is as follows:

[0098] Change the search step size by changing the step size parameter of the original beetle antennae search. First, use the first preset step size to determine the range of the search direction, and then use the second preset step size for fine search.

[0099] Furthermore, the extraction of feature points and feature lines from the vehicle image after edge detection includes:

[0100] Use the Harris algorithm to extract feature points in the vehicle image after edge detection.

[0101] Convert the pixels of the feature points to the sub-pixel level.

[0102] Use the Hough transform to extract feature lines in the vehicle image after edge detection.

[0103] Specifically, in this embodiment, the improved beetle antennae search algorithm

[0104] The beetle antennae search optimization algorithm is an intelligent algorithm based on bionic principles. Inspired by observing the foraging behavior of beetles, the search direction is determined by the different signal strengths received by the left and right antennae of the beetle. After determining the direction, it jumps to the next position to continue the search until the food position, that is, the optimal solution of the objective function, is found. The BAS search algorithm only needs one individual to achieve omnidirectional search during the optimization process, is not restricted by the function form and gradient information, and in multi-objective optimization, the dimensions do not interfere with each other, greatly reducing the complexity of the optimization algorithm. Compared with other optimization algorithms, this algorithm can well avoid falling into the local optimal solution, has strong applicability, a simple process, a small amount of computation, and can find the global optimal solution of the objective function faster.

[0105] Considering that the BAS algorithm with a fixed step size has the same search distance each time, while the gradient amplitude in the image is constantly changing, important pixel points may be missed during the search, resulting in the continuous amplification of the search error in the later stage. In particular, weak edges and complex edges will be ignored, affecting the accuracy of edge detection.

[0106] To solve the above problems, this embodiment combines an improved BAS algorithm, in which the search step length is changed by changing the step length parameter, first using a large step length to determine the approximate range of the search direction, then narrowing the parameter for a fine search, and finally finding the optimal solution. This variable step length method not only improves the accuracy of edge information but also increases the speed of calculation.

[0107] The main formula parameters involved in the algorithm include: random initial solution x = rands (S, 1), x is a random initial value in (S, 1), problem dimension S, the step size l of the longhorn beetle's movement, variable step size parameter E, parameter decay factor t that decreases linearly with the increase of the number of iterations t∈(0,1), the distance between the two whiskers of the longhorn beetle d0, and the number of iterations n.

[0108] Random variable of beetle whisker orientation

[0109]

[0110] The coordinate expression of the left whisker of the longicorn is:

[0111]

[0112] The coordinate expression of the right whisker of the longicorn is:

[0113]

[0114] Calculate the function fitness value:

[0115] F L =f(X L ) (4)

[0116] F R =f(X R ) (5)

[0117] The next position of the longicorn is calculated as:

[0118]

[0119] The termination condition is to determine whether the difference between the direction of the gradient change rate and the coordinates of the longhorn beetle position and the initial position is within the set range. The coordinate value of the pixel point is the fitness value after the termination iteration. If the solution of formula (7) is satisfied, it is the edge point of the image.

[0120]

[0121] 3GA-IBAS edge detection algorithm:

[0122] The premature convergence problem of the genetic algorithm can just be solved by the beetle antennae search algorithm. On this basis, a genetic algorithm combined with the beetle antennae search is proposed and applied to image edge detection. In order to meet the convergence conditions, the local optimal solution of the genetic algorithm is transformed into the starting position of the beetle search for beetle antennae search operation, and finally the edge information of the image is displayed.

[0123] The basic steps of the algorithm are as follows (i represents the number of iterations that have been carried out, T represents the upper limit of the number of iterations, set to 100):

[0124] (1) Simply process the image and set the initial values for each parameter in the algorithm; the initial value is a threshold, which has different values in different scenario applications and needs to be combined with the current scenario during the debugging process to find the value with the best effect (the vehicle contour is clear and there are few interfering lines);

[0125] (2) Perform chromosome coding on the gray values of pixel points and randomly select the initial population; among them, the initial population is the encoded gray contour, which is not directly determined, that is, the coding will be different, and through genetic iteration, the optimized coding, that is, the most definite image edge, is found.

[0126] (3) Calculate the gradient change rate of each individual in the population according to Equation (8) and calculate the fitness value of the individual;

[0127] (4) Generate offspring through crossover and mutation;

[0128] (5) Use the beetle antennae search to guide the movement of the parent crossover points participating in the crossover operation in the new generation population, update the offspring, and improve the population diversity;

[0129] (6) If i < T, then i = i + 1 and go to step (3), otherwise stop the iteration;

[0130] (7) The algorithm ends, and the optimal solution in the population is output, that is, the detected image edge.

[0131]

[0132] The algorithm flowchart is as Figure 1 shown.

[0133] In order to more intuitively reflect the improvement in the performance of the improved algorithm, two different groups of images are respectively processed by edge detection based on the genetic algorithm and the improved edge detection. The following figure is the result graph after detection, and the comparison is made from pixels and the overall edge respectively.

[0134] From Figure 2 it can be clearly seen that the improved algorithm has a very good improvement effect on the edge detection results of images in various forms, achieving the purpose of improving the algorithm.

[0135] The improvement of the algorithm in convergence time is also very obvious, as shown in Table 1:

[0136] Table 1 Edge detection algorithm time consumption table

[0137]

[0138] This embodiment utilizes the advantages of the beetle whisker search algorithm to improve the genetic algorithm, improves the operation speed, and solves the problem that the result of the genetic algorithm is a local optimal solution. When the improved algorithm is used to detect an image, a more continuous, clear, and complete edge can be obtained. Compared with the existing image edge detection algorithm, the improved algorithm not only reduces the convergence time, but also makes the detected image edge more complete, verifying the feasibility of the algorithm.

[0139] Feature extraction and vehicle type classification:

[0140] Feature extraction is used to obtain feature points for binocular positioning, and the horizontal distance between the vehicle and the camera is obtained, providing accurate external parameters for monocular measurement. Therefore, the quality of feature extraction in two-dimensional images will have a direct impact on the subsequent measurement accuracy, and it is one of the important areas of machine vision research.

[0141] Feature point extraction:

[0142] A feature point (also called a corner point) is a point in an image where the brightness changes sharply and the curvature of the image boundary curve is sufficiently large. Compared with other features in an image, a corner point is the most important feature.

[0143] Corner point extraction algorithms are generally divided into two categories: methods based on contour curves and methods based on grayscale images. The former extracts corner points from images based on data based on changes in image contour curvature. Since the amount of data contained in changes in image contour curvature is huge, the amount of data for different types of images may also change accordingly. The calculation process is relatively complicated, and the effect of image preprocessing will also have a great impact on the accuracy and quality of corner point detection results. To obtain an ideal corner point, many factors need to be controlled; the latter detects feature points by judging the severity of grayscale changes in the neighborhood of pixels, such as Moravec algorithm, Forstner algorithm, SUSAN algorithm, Harris algorithm, etc. The Harris algorithm only requires the first-order derivative of the grayscale value of the pixel, has strong stability, and has strong anti-interference ability. Therefore, this embodiment uses the Harris algorithm to extract feature points from vehicle images. The specific process is as follows:

[0144] Harris corner operator:

[0145] The grayscale gradient value at the corner point of the image is not unique and discontinuous. Based on this phenomenon, the following matrix can be obtained to detect the corner point.

[0146]

[0147] Where: I x and I y is the grayscale gradient of the image along the x and y directions; in order to remove the influence of noise on the corner point extraction results, consider using a Gaussian filter Smoothing filtering is performed to eliminate noise points, where the mean of G(σ) is zero and the variance is σ.

[0148] Harris corner points can be defined as feature points that satisfy the following conditions:

[0149] R = detM-k(traceM) 2 >T (10)

[0150] in: k is a constant and T is the threshold.

[0151] Pixels whose R value exceeds the threshold T are detected as corner points. Since the Harris operator is used to obtain corner points in a simple way, it has high stability and can obtain accurate corner points even for large image interference factors.

[0152] Sub-pixel corner detection

[0153] Compared with target recognition, camera calibration and geometric measurement require higher-precision feature points. Most corner point detection methods only obtain integer coordinate values ​​in the pixel coordinate system. To obtain higher-precision coordinate values, the corner points need to be improved to the sub-pixel level.

[0154] In the process of camera calibration, sub-pixel corner point detection method is used to obtain more accurate calibration points, which greatly improves the accuracy of the entire measurement system. The following is the process of accurately determining the corner point coordinates to sub-pixel accuracy:

[0155] like Figure 3 As shown, Figure 3 (a) is point p in the region, Figure 3 (b) is when point p is at the edge of the region; in these two forms, the dot product of the gradient of pixel point p and its corresponding vector qp is 0. According to the above conditions, find a set of equations that meet the requirements around point p and solve the set of equations. The solution of the set of equations is the sub-pixel-precision coordinate of the corner point q.

[0156] Feature line extraction

[0157] Line detection methods can generally be roughly divided into three types: edge connection detection method based on gradient information, chain code detection method and Hough transform. Hough transform finds the curve by finding the peak value of the coordinate space, and has a strong resistance to noise. The application scenario of this embodiment meets the prerequisite for using Hough transform, so Hough transform is used for line detection.

[0158] Feature point optimization

[0159] Sub-pixel corner detection and Hough transform are combined to optimize corner points. The specific steps are as follows.

[0160] 1. Use the algorithm introduced in this embodiment to extract the corner points on the calibration plate and preliminarily obtain the coordinates of the corner points;

[0161] 2. Use the existing Hough transform edge detection function in halcon to select the area of ​​the calibration plate and obtain the edge information of the area of ​​interest;

[0162] 3. Extend the edge to obtain the coordinates of the corner points, and use the dichotomy method to solve the coordinates of the existing points and the coordinates of the corresponding points multiple times to obtain a more accurate point position.

[0163] By matching the image coordinates calculated in this way with the world coordinates and obtaining a calibration transformation matrix, which replaces the original calibration matrix, the accuracy of the measurement results can be further improved.

[0164] Vehicle type recognition based on pattern classification

[0165] The process of pattern division relies on describing the attributes of the target to express the category symbol assigned to the target. A classifier is an algorithm. The information it inputs into the database is the expression of the detected target, and the output information is the label of the category. One of its basic methods is to represent the detected sample code to be identified as a vector. The classifier classifies all targets by extracting the specified features, and classifies the target into one of the given m specified feature categories. The m specified categories are assigned to the representation features C1, C2, C3, ..., C m , where C m =C r is called a rejected class. When the system cannot assign an object to any of the approved classes, it assigns it to C r kind.

[0166] Figure 4The block diagram shown is a general schematic diagram of the classification system structure design. The input is an n-dimensional feature vector x, which represents the target to be classified. For a possible type classification, the system can be represented by a simple function box form (a column of blocks in the middle of the figure). It needs to contain a relevant basic knowledge K of the system type and the ability to process information. f(x,K) is a function of a judgment value. The calculation results of m different categories of data are directly passed into the system decision stage, and finally the category of the area where the target is located is determined by calculation.

[0167] This embodiment uses the above method to identify the vehicle type. The input is the vehicle feature vector after image background difference processing, which removes the interference features in the image and can reduce the misjudgment rate to a greater extent. The output is the level to which the vehicle belongs. Table 2 is the parameters for judging the vehicle level, which are the basis for classification.

[0168] Table 2 Vehicle Classification Parameters

[0169]

[0170] According to Table 2 above, common passenger cars can be divided into four categories. By classifying the vehicles in the image, the size range of some vehicles can be preliminarily obtained, which reduces the location area for finding the best measurement point and makes the entire measurement process faster and more accurate.

[0171] Research on Stereo Vision Matching

[0172] The accuracy of stereo vision measurement results is determined by the quality of stereo vision matching. The matching process is the most critical and difficult step in the entire stereo vision. Factors such as shooting time, shooting angle, environmental changes, camera differences, grayscale distortion and geometric distortion can all lead to differences between image pairs. Constructing a matching algorithm with the smallest matching error possible has become the focus of research on stereo vision.

[0173] The monocular camera can quickly identify and classify vehicles through a single size, and quickly match them with parking spaces and shipping robots. It also provides a basis for later improvement of calibration matrix parameters and calibration of binocular vision.

[0174] Furthermore, based on the identified vehicle type, stereo matching is performed on the vehicle to be parked, and a three-dimensional vehicle model is constructed, including:

[0175] S2.1. Use the left and right cameras to capture the left and right vehicle images of the vehicle to be parked;

[0176] S2.2. Use the Sobel operator to perform edge detection on the left vehicle image and the right vehicle image to obtain edge points;

[0177] S2.3. Based on the camera calibration parameters, obtain the basic matrix of the left and right cameras;

[0178] S2.4. Select an arbitrary edge point in the image captured by the left camera, and calculate the corresponding epipolar line in the image captured by the right camera by using the basic matrix, which is recorded as l2; select an arbitrary point on l2, and calculate the corresponding epipolar line in the image captured by the left camera by using the basic matrix, which is recorded as l1; wherein all edge points on l1 and l2 correspond one to one;

[0179] S2.5. Based on the preset similarity evaluation function, calculate the evaluation value of the similarity between the edge points on l1 and l2 and establish a two-dimensional similarity evaluation function list M, in which the units M(i,j) are the similarity evaluation values ​​of the i-th point on l1 and the j-th point on l2 respectively; if the threshold of a certain unit M(i,j) in the list is the smallest in its row and column, and the threshold level is less than the threshold of a certain unit, then the i-th point on l1 and the j-th point on l2 are preliminarily determined to correspond to one point;

[0180] S2.6. Repeat S2.5 until all edge feature points are matched and calculated. After each overall matching of the two epipolar lines, the points that have been calculated will no longer appear in subsequent calculations, that is, the uniqueness constraint is satisfied.

[0181] S2.7. According to the limited gradient of the screening disparity value, a corresponding screening is performed among the candidate related pixel points obtained under the preset constraint conditions to obtain the optimized matching points;

[0182] S2.8. Based on the optimized matching points, find the ideal matching point pair closest to the epipolar line, obtain the final precise matching point pair, calculate the three-dimensional coordinates of the vehicle based on the final precise matching point pair, and construct a three-dimensional vehicle model.

[0183] Specifically, in this embodiment, for the vehicle profile measurement system designed in this embodiment, in order to achieve high-precision detection, the corresponding matching calculation method that needs to be used has very high accuracy. In order to facilitate the selection of a reasonable matching algorithm during research, the main features of the measurement are analyzed and summarized:

[0184] (1) The object being measured is large;

[0185] (2) The vehicle has no obvious texture features;

[0186] (3) The lighting in the working environment changes dramatically.

[0187] (4) There are many types of vehicles, and the dimensions of different models vary greatly;

[0188] (5) The binocular sensor model was built by ourselves. To ensure that the entire vehicle is within the field of view when the distance between the camera and the vehicle is minimal, the model has a large parallax.

[0189] In combination with the particularity of the size measurement system and the essential properties of the object to be measured, this embodiment selects a method that combines regional grayscale matching with feature matching, and makes good use of the advantages and disadvantages of various matching technologies to achieve the best match. First, by extracting some geometric physical features (such as edge contours or points of interest) of the detected vehicle in the two photos, an initial candidate matching set is established mainly based on the constraint relationship of the epipolar line. Then, the regional grayscale matching algorithm is used to compare the grayscale information or other edge contour information near the feature point with the grayscale value for similarity, remove some incorrectly matched points, and complete the entire stereo matching process.

[0190] Constraints can be used to reduce the problem of unclear corresponding points in stereo matching. The algorithm designed in this embodiment mainly uses the following constraints:

[0191] (1) Exopolar line constraints

[0192] In a set of image pairs, take a point on the epipolar line of one image, and the corresponding point is also on the epipolar line of the other image. After the camera is distorted and corrected to meet the above constraints, the speed and accuracy of stereo matching can be greatly improved.

[0193] (2) Uniqueness Constraint

[0194] Each pixel in a set of image pairs corresponds uniquely.

[0195] (3) Mutual correspondence constraints

[0196] Suppose there is a point p1 in the left image, and the corresponding point p2 is searched in the right image based on p1; if the corresponding point p1 cannot be found based on p2 in the reverse search, it means that the matching effect of this set of corresponding points is not very good and should be excluded. When the corresponding points cannot be determined due to occlusion, high light or noise environment, these defective points can be directly excluded by using mutual correspondence constraints.

[0197] (4) Compatibility constraints and similarity constraints

[0198] Photometric compatibility refers to the similarity of the grayscale correspondence between the two stereo images, which is measured by using the cross-correlation function described in the previous section. Similarity refers to the fact that the corresponding edge feature points in the two images have basically the same geometric characteristics. For these edge features, the edge direction is a relatively stable match. Since the relative displacement and rotation between the two cameras in the binocular configuration used in the measurement system are not large, we can use the difference in gradient values ​​between the edge points in the two photos to measure the geometric similarity.

[0199] Based on the above two constraints, the following evaluation function is proposed:

[0200]

[0201] The first term in the above formula 11 mainly describes in detail the spatial grayscale directional correlation of the corresponding template neighborhood, and the second term mainly describes in detail the grayscale correlation between the gradient direction and the corresponding amplitude between the grayscale center point and the pixel of each corresponding template. The optimal match is obtained when S(x, y) is minimized. Among them, the value of the proportional probability parameter k is related to the value of the probability gradient function calculation formula used in the actual design. In the actual experiment process, it can be found that when the sobel operator is selected to calculate the image gradient, k is 0.2, which can achieve a better display effect for general gradient images.

[0202] (5) Parallax continuity constraint

[0203] Generally speaking, the points on the surface of an object are distributed continuously, and the distance difference between two very close points on the same continuous surface of the same object is also very small, and the image parallax of the object is continuous. This constraint condition is for points within the surface of the object. If the point is located on the boundary of the object, this constraint no longer holds. For example, two points on different surfaces on both sides of the boundary have a large difference in depth, so the parallax is also large.

[0204] The consistency of disparity can be measured by a disparity gradient. Assume that there are two points p1 and q1 in one image of a set of image pairs corresponding to two points p2 and q2 in the other image, and the points in these two spaces are considered as adjacent points, then the disparity gradient formula is defined as follows:

[0205]

[0206] Among them, p1, p2, q1, q2 are vectors of image coordinates, and ||·|| is the vector modulus. The size of the disparity can directly reflect the depth information of the spatial position point from the camera imaging plane. Therefore, the depth difference between adjacent points in space can be reflected by the disparity gradient. The depth between two matching points on the spatial surface has strong continuity. If p1, p2 and q1, q2 are the projections of two corresponding points on the same surface in space on two images, the human eye can only correctly match two corresponding points with a disparity gradient less than 2. This has been proven by many representative experiments. Therefore, it can be assumed that if the matches of p1, p2 and q1, q2 are completely compatible, then G d The result should be less than or equal to 2; if it is greater than 2, the matching error is large, indicating that the distance between the two spatial points in the depth direction is greatly different and they are not suitable as matching point pairs.

[0207] Based on the above constraints, this embodiment designs a stereo matching algorithm, and the steps are as follows:

[0208] 1. Use the Sobel operator to extract edge feature points to be matched.

[0209] 2. The basic matrix is ​​calculated from the camera calibration parameters.

[0210] 3. Select an arbitrary edge point in the image captured by the left camera. The corresponding epipolar line in the image captured by the right camera can be calculated by the basic matrix, which is recorded as l2; select any point on l2, and obtain the corresponding epipolar line in the left image through the above calculation, which is recorded as l1. It can be seen from the epipolar line constraint that all points on l1 and l2 correspond one to one. Using formula (11) as the similarity evaluation function, calculate the evaluation value of the similarity between the edge points on l1 and l2 and establish a two-dimensional similarity evaluation function list M. The unit M(i,j) in the table is the similarity evaluation value of the i-th point on l1 and the j-th point on l2. If the threshold of a unit M(i,j) in the list is the smallest in its row and column, and the threshold level is less than the threshold of a certain unit, then the i-th point on l1 and the j-th point on l2 are preliminarily determined to correspond to one point. The process of this method uses a mutual correspondence constraint, that is, according to the i-th point on l1, the most similar j-th point is found on l2, and then the most similar i-th point is found on l1 according to the point. If the i-th point can still be found, the two points correspond. In this step, due to the possible errors in the calibration of the camera, the corresponding point may not appear exactly on the epipolar line, so a smaller neighborhood of pixels near the epipolar line can also be included in the search area.

[0211] 4. Repeat step 3 until all edge feature points are matched. After each overall match of the two epipolar lines, the points that have been calculated will no longer appear in subsequent calculations, which means that the uniqueness constraint is satisfied.

[0212] 5. According to the limited gradient of the screening disparity value, the candidate related pixel points obtained under the above-mentioned various constraints are screened accordingly. In an adjacent domain of a candidate point in a visible graph I1, a candidate point closest to the point can be found. If the candidate point can be found and obtained, it can be combined with its corresponding point in a visible graph I2 into formula (12) to calculate the disparity gradient of a graph. If the Gd calculated between two points in the graph is less than or equal to 2, the two pairs of corresponding points are confirmed and regarded as a correct match; if the candidate point is not found or the Gd calculated between the two points is greater than 2, it may be necessary to leave it for the next subsequent analysis. The above verification process is performed on each candidate point in I1 and I2 in the prescribed order. Since each candidate point that has been fully confirmed can provide more relevant information to the surrounding points, it is also included in the process of calculating the disparity gradient. This step well implements the constraint conditions for the matching points and retains the situation where the matching points are on both sides of the discontinuous boundary as much as possible.

[0213] The algorithm designed above is simple in principle, highly practical, and has a relatively fast operation speed. When matching stereo images with good edge extraction effects, it can achieve a high matching rate and obtain dense corresponding points on the edges. The matching results of multiple times are relatively ideal. This algorithm optimizes the corresponding points, making the stereo matching effect better and greatly improving the accuracy of the data obtained from the subsequent vehicle profile measurement.

[0214] In this embodiment, the epipolar geometric constraint optimization matching points include:

[0215] Due to the influence of image distortion, noise, etc., the matching point pair (m1, m2) calculated by the above method cannot appear accurately on the epipolar line, and the ideal matching point pair Only in this way can the geometric relationship of the epipolar lines be accurately satisfied.

[0216] If the ideal matching point pair closest to the stereo matching point pair (m1, m2) is found The stereo matching result is the most accurate, that is, by finding a matching point pair Make min Minimum:

[0217]

[0218] Where d represents the offset distance between two ideal matching points and two actual matching points. Based on the geometric relationship between the ideal points on the epipolar line, the distance between two points can be converted to the distance from the point to the epipolar line. Formula (13) can be replaced by formula (14):

[0219] d min=d(m1,l m1 )+d(m2,l m2 ) (14)

[0220] Under the condition that the basic matrix is ​​known, a point in the left image I1 and its epipolar line can determine a point in the right image I2 and its epipolar line. Therefore, we can establish a corresponding epipolar line pair on images I1 and I2, and further construct the same epipolar line bundle to obtain the ideal matching point pair between the two.

[0221] Due to the direct geometric linear constraints on the epipolar lines and the direct definition of the equation, it is clear that all epipolar lines will pass through two fixed points on the line connecting the optical centers of the two cameras, which are the poles e1 and e2. Define a point on the upper left of image I1 as the origin of the image, with the y-axis direction as shown in the figure, and define a point g=(0,t) moving on the y-axis of the image T , the line between this point and the pole is represented as an epipolar line in the image, such as Figure 5 shown.

[0222] Using formula (14), among all the epipolar line pairs, we can find a set of epipolar line pairs that makes the sum of the distances from the matching point pair to the corresponding epipolar line the shortest. The matching point deviation caused by noise and other factors determines the only epipolar line. A perpendicular line is drawn through points m1 and m2 to the corresponding epipolar line. The foot of the perpendicular line is the ideal matching point. The actual position problem is converted into a problem of finding the minimum value of the equation about t. The value of t can be quickly obtained by taking the derivative to obtain the maximum value. This can obtain a precisely designed and optimized spatial matching point pair. When the internal and external parameters of the camera have been calibrated, the depth information of the spatial point can be accurately obtained by using the binocular stereo vision measurement platform.

[0223] In the case where the hardware equipment is built, the software system suitable for measuring the size of the whole vehicle is edited using the improved algorithm. The main flow chart is as follows: Figure 6 shown.

[0224] The three main aspects of the analysis and research are image processing algorithms, vehicle type recognition and stereo vision matching. By improving the existing edge detection algorithm, an algorithm that is more suitable for detecting vehicle edges is proposed. The improved algorithm extracts clearer vehicle edge information, which provides a favorable premise for determining the best measurement point. The input of the vehicle type recognition method based on pattern recognition selected in this embodiment is the vehicle outline after the image is processed by foreground difference. The clear edges are conducive to the accurate classification of the vehicle. Finally, the stereo matching algorithm in binocular vision is discussed and analyzed, and a method based on epipolar constraints is proposed to optimize the matching points. This method not only improves the accuracy of stereo matching, but also further improves the basic matrix determined in the binocular camera calibration process. The research of this embodiment improves the system measurement speed and vehicle size accuracy, and achieves the purpose of improvement.

[0225] Further, based on the three-dimensional vehicle model, determining measurement parameters, and performing visual measurement includes:

[0226] Determine measurement parameters; wherein the measurement parameters include: vehicle length, vehicle width, vehicle height, wheelbase, front overhang, and rear overhang;

[0227] Based on the three-dimensional vehicle model, the measurement points of length, vehicle width, vehicle height, wheelbase, front overhang and rear overhang are positioned by using a grayscale projection method, and based on the positioned measurement points, the vehicle length, vehicle width, vehicle height, wheelbase, front overhang and rear overhang are measured.

[0228] Furthermore, measuring the vehicle length, width and height includes:

[0229] Step 1. For the actual position coordinate set of the vehicle found by segmentation in the two-dimensional image coordinate system, use the convex hull algorithm to find a convex polygon p = (p1, p2, ..., p n );

[0230] Step 2. Based on the convex polygon, use the rotating caliper algorithm to find the farthest anti-podal point;

[0231] Step 3. Take the anti-podal point with the longest distance in the region as the optimal size measurement point, rotate the parallel tangent corresponding to the anti-podal point with the longest distance, and then perform the operation in step 2 to find another longest distance. In this step, add a preset range constraint to the rotation angle of the two tangents;

[0232] Step 4. Use the two maximum distances found above as the length and width of the vehicle's circumscribed rectangle respectively.

[0233] Specifically, in this embodiment, the whole vehicle size parameter measurement includes:

[0234] Measurement size parameters determination:

[0235] This embodiment is aimed at passenger vehicles, and measures several important vehicle parameters such as length, width, height, wheelbase, front overhang, and rear overhang. Figure 7 As shown. By comparing with the existing vehicle size data in the database, the system can more accurately classify vehicle grades, thereby providing more reliable real-time data for the intelligent parking system, which facilitates the system to more reasonably allocate the transport robot and empty parking spaces corresponding to the vehicle model. An important reference for the selection of the transport robot is the minimum ground clearance of the vehicle, which is obtained by calling the database parameters after determining the vehicle model. Among the types of passenger cars, the appearance of the vehicle hardly changes too much, so the method of obtaining their size parameters is consistent. The following introduces the positioning methods of these types of parameter measurement points.

[0236] Vehicle length and height measurement point positioning:

[0237] The vehicle length refers to the maximum distance between the front and rear ends of the vehicle. To measure the length of the vehicle, the dimension parameters need to be accurately located at the front and rear end of the vehicle. Due to the presence of noise in the image, the method of simply locating the left and right extreme points is not accurate enough. Therefore, in this embodiment, the grayscale projection method is used to locate the measurement point. The result of projecting the vehicle image is as follows: Figure 8 As shown: Among them, Figure 8 (a) is a vehicle picture;

[0238] Figure 8 (b) shows the relationship between the pixel value and the coordinate in the horizontal direction of the car image. Figure 8 The projection curve of (b) sets the judgment conditions to obtain more accurate measurement points at the front and rear ends of the vehicle:

[0239]

[0240] Where S(x) represents the pixel value of the xth column, and dS(x) / dx represents the horizontal difference of S(x). The first points on the left and right sides of the vehicle image that meet the above inequality conditions are the two measurement points of the vehicle length.

[0241] When measuring the vehicle's contour dimensions, the measurement points located by the above method have large errors because the longitudinal symmetry plane of the vehicle is not directly opposite to the camera imaging plane. In this case, it is obviously unreasonable to use the above method to determine the measurement points, so a new method needs to be selected to determine the measurement points.

[0242] The difference method is used to separate the vehicle and the background in the image, and the convex hull and caliper algorithm are used to find the best size measurement point on the segmented vehicle target. The best measurement point on the collected vehicle image is matched through the definition of each size parameter of the vehicle to achieve the measurement of the vehicle length and width. Fig. 9 As shown, among which, Fig. 9 (a) is to find the convex hull points. Fig. 9 (b) is the rotating caliper algorithm to find the measurement point; the specific steps are as follows:

[0243] Step 1: For the actual position coordinate set of the vehicle found by segmentation in the two-dimensional image coordinate system, use the convex hull algorithm to find a convex polygon p = (p1, p2, ..., p n ),like Fig. 9 As shown in (a);

[0244] Step 2: Use the rotating caliper algorithm to find the farthest anti-podal point. Fig. 9 As shown in (b), find the two endpoints on the convex polygon in the Y-axis direction, assuming that p i 、p j , the distance between the two points is recorded as s1, the two points are taken as the first pair of anti-podal points, and two parallel tangents are constructed at the two anti-podal points. Rotate the two tangents, if any tangent coincides with one side of the convex polygon, a new pair of anti-podal points will be obtained, and the distance between the new anti-podal points is recorded as s i , until the tangent returns to the original position and the rotation ends. Get the anti-podal point distance set (s1, s2, …, s n ), take the longest distance in the anti-podal point distance set:

[0245] S max =Max{s1,s2,...,s n} (16)

[0246] The farthest antipodal point in the region is taken as the best size measurement point, such as Fig. 9 As shown in points M and N in (b).

[0247] Step 3: Rotate the parallel tangent corresponding to the farthest anti-podal point by 90°, and then perform the operation in step 2 to find another farthest distance. In this step, a range constraint is added to the rotation angle of the two tangents, otherwise the anti-podal point distance found will be the same as the distance found the first time.

[0248] Step 4: Use the two maximum distances found above as the length and width of the vehicle's circumscribed rectangle.

[0249] This method solves the problem that the deflection angle of the collected vehicle images affects the selection of measurement points, but the process of finding the next distance within a certain range according to relevant conditions is itself a process that needs to be optimized. Therefore, it is necessary to optimize the antipodal point in combination with the minimum circumscribed rectangle to obtain the best measurement point.

[0250] The following is a brief introduction to the process of obtaining the minimum circumscribed rectangle of a vehicle.

[0251] 1. According to the principle of center of gravity, the center of gravity of a single object is located at the geometric center of the object, the position of the rotation axis is determined, each row is scanned and the center of gravity coordinates (x, y) of each row are calculated.

[0252] 2. Take the centroid coordinates (x j ,y j )(j=1,2,...,M) is a set of data, and the least square method is used to solve the linear equation y of the initial position of the vertical principal axis j =k1x j +b1.

[0253] 3. Similarly, scan each column in turn and calculate the centroid coordinates (x i ,y i )(i=1,2,...,N), use the least squares method to get the linear equation y of the initial horizontal position i =k2x i +b2.

[0254] 4. After solving the two linear equations, the intersection point (x0, y0) can be found. A line passing through the intersection point rotates around the intersection point. The initial position of the line starts from the same direction as the horizontal axis and gradually changes to a position perpendicular to the horizontal axis. The direction of rotation gradually increases in the direction of the slope of the horizontal axis. When the surface area of ​​a rectangle that can contain the entire body around the body is the smallest, the resulting rectangle is the smallest circumscribed rectangle around the body, such as Fig.10 shown.

[0255] The length of the minimum circumscribed rectangle is the length of the vehicle, and the row coordinates of the upper side corresponding to the width of the minimum circumscribed rectangle minus the row coordinates of the lowest point of the wheel obtained by Hough transform is the distance of the vehicle height in the image. The actual vehicle height can be calculated by the coordinate system transformation matrix.

[0256] The length of the minimum circumscribed rectangle is the length of the vehicle, and the row coordinates of the upper side corresponding to the width of the minimum circumscribed rectangle minus the row coordinates of the lowest point of the wheel obtained by Hough transform is the distance of the vehicle height in the image. The actual vehicle height can be calculated by the coordinate system transformation matrix.

[0257] Wheelbase and front and rear overhang measurement point positioning

[0258] The distance between the centers of the two wheels on the same side of the car is the wheelbase of the car. This embodiment uses Hough transform to detect the wheel target circle, obtain the coordinates of the center of the circle, and thus determine the measurement point of the wheelbase, and finally calculate the wheelbase. The principle of Hough transform has been explained in Chapter 4 and will not be introduced here.

[0259] The wheel has a circular feature, and the wheel position is detected through Hough transform. The center points of the two circles detected are the wheelbase measurement points. However, since there are many curves in the edge image of the vehicle, the algorithm will mistakenly detect these curves as circles when performing Hough transform. Even if the radius of the circle to be detected is set in advance, it can only reduce the number of detected circles, and this false detection phenomenon still exists. This embodiment constrains the Hough transform by the region of interest, the radius of the detection circle, and the relative position of the wheel. Under the constrained conditions, the Hough transform obviously only detects the target circle of the wheel. The detection results are as follows: Fig.11 As shown: Among them, Fig.11 (a) is the unconstrained Hough circle detection, Fig.11 (b) is the constrained Hough circle detection;

[0260] When the Hough transform detects the target circle at the wheel location, the constraints used are displayed as the red box area in the figure above. The constraints are as follows:

[0261]

[0262] Where F(x,y) is the pixel, G(x,y) is the region of interest required in the transformation, w and h represent the length and width of the image, abs(y1-y2)≤3 means that the two centers are basically on the same horizontal line, and abs(x1-x2)≥w / 2 represents the horizontal distance constraint between the two centers.

[0263] After the above operations, Hough transform easily detects the target circle at the wheel, and obtains the exact position of the axis measurement point by querying the centers of the two target circles, thereby calculating the wheelbase. The front and rear overhang measurement points of the vehicle can be found through the vehicle length measurement points mentioned above, so I will not introduce them in detail.

[0264] Measurement experiment results and analysis:

[0265] The "Vehicle Dimension Measurement" software is written based on C# and serial communication. It sends and receives data from the sensor through the serial port to visualize the data and facilitate recording of measurement results. The upper computer UI interface is as follows Fig.12 shown.

[0266] After the design and debugging of the vehicle size measurement system, three different models of cars were measured. Each sample vehicle was measured multiple times, and three groups of data with smaller measurement errors were selected. The three types of vehicles measured were Toyota Corolla, Toyota Camry and Jeep Compass, and the measurement data are shown in Table 3.

[0267] Table 3 Comparison and error between visual size measurement and manual measurement results

[0268]

[0269]

[0270] By statistically analyzing the data collected in Table 3 above, it is found that the maximum absolute error of the dimensional measurement system is 99 mm, and the average maximum absolute error is 34.67 mm. According to the analysis of the measurement data, it can be seen that the system has high stability during the measurement process, and it takes about 15 seconds to measure the size of the whole vehicle. By querying the relevant vehicle detection indicators, it can be seen that the accuracy of the dimensional results measured by the measurement system meets the relevant requirements. So far, this embodiment has achieved the goal of accurately measuring the vehicle contour size in an automatic, fast, and non-contact manner.

[0271] The measurement system measures the dimensions of vehicles entering the parking lot, obtains the vehicle model (A-class, B-class, C-class, D-class, SUV) through the pattern recognition module, and transmits the vehicle model data to the empty parking space matching module and the carrier robot model selection module in the intelligent parking system. The empty parking space matching module matches empty parking spaces and vehicles in real time, improving the utilization rate of parking space; the carrier robot model selection module selects the corresponding model of carrier robot according to the vehicle model, improving the safety of the intelligent parking process. The measurement system realizes the interaction of real-time data between modules in the intelligent parking system, making the parking system more intelligent. Fig.13 It is a schematic diagram of data transmission in the system.

[0272] This embodiment mainly studies how to determine the position of the best measurement point in the process of measuring vehicle dimensions to ensure that the dimension data obtained by the visual measurement system is more accurate. By comparing and improving the method of obtaining the position of each dimension measurement point, the best measurement point position information suitable for the measurement system of this embodiment is obtained, and then combined with the coordinate transformation matrix in the measurement system, the actual dimension parameters of the vehicle are finally obtained. The dimension error measured by this embodiment meets the design requirements and vehicle parameter detection standards, proving the effectiveness of the system. It also provides real-time and accurate vehicle model data for the intelligent parking system, improves the accuracy of carrier robot selection and empty parking space matching, reduces the safety issues that may occur during the operation of the entire parking system, and achieves the design purpose of the measurement system.

[0273] This embodiment studies the demand for automated measurement of vehicle outline dimensions in intelligent parking application scenarios, and the method of obtaining the outline dimensions of large-sized measurement objects. Finally, the visual measurement method is selected to measure the vehicle outline dimensions. In order to ensure the effectiveness and accuracy of the actual vehicle outline dimension measurement results, the research process combines the environmental characteristics of the actual measurement site, the appearance characteristics of the measured vehicle, and the technical level that can be achieved by visual research at this stage, and designs a whole vehicle outline dimension measurement system for the intelligent parking system. This measurement method overcomes the problems of large manual measurement workload, low measurement efficiency, and large deviation in measurement results. A method of joint measurement combining monocular vision and binocular vision is proposed to suppress the influence of multi-point stereo matching errors on vehicle dimensions, and realize vehicle classification and automated, digital measurement of whole vehicle dimensions. The main research content of this embodiment is divided into four parts: camera and hand-eye calibration, image edge detection, vehicle type recognition, and stereo vision matching. The vehicle target is extracted by background difference method, put into the classifier, and different models are quickly identified. The size parameter range of different models is preliminarily determined. Then, binocular vision technology is combined with key vehicle size measurement to obtain more accurate real-time vehicle outline size parameters, providing effective data support for the intelligent parking system; the existing algorithm is analyzed and improved to make the algorithm more suitable for the research of this topic, and some creative research results are obtained. The system is based on industrial computers and industrial cameras as hardware to complete the development of machine vision measurement system. The whole measurement system can realize the rapid identification of vehicle types and accurate measurement of vehicle outline size. It improves the automation and safety of parking systems in intelligent parking lots.

[0274] When the vehicle outline dimension measuring device based on machine vision measurement method is placed at the entrance of the parking lot, it can provide real-time vehicle dimension data for the intelligent parking lot system, making the parking process safer and the parking space utilization more reasonable. This embodiment studies and designs a passenger car outline dimension measuring system based on machine vision for use in the intelligent parking system by referring to a large number of related contents, which effectively improves the intelligence and automation of the parking process in the intelligent parking lot.

[0275] This embodiment mainly uses machine vision measurement technology to measure the vehicle's contour dimensions and classify the vehicle models when entering the parking lot, providing good data support for the intelligent parking system. According to the actual measurement environment, a single-eye and binocular combined measurement method is selected to complete the acquisition of vehicle images, perform image preprocessing, extract vehicle edge information, and camera calibration on the PC side, and finally complete the calculation of the actual size of the vehicle. The traditional manual measurement method takes about 20 minutes to measure the entire vehicle. The time required for the machine vision-based vehicle contour dimension measurement method varies with the complexity of the image processing algorithm, and generally completes the measurement of the entire vehicle within 1 minute.

[0276] The main research results of this embodiment in terms of rapid measurement of key vehicle dimensions are as follows:

[0277] 1. Through the analysis of existing methods for measuring vehicle dimensions, research on related contents and design a more efficient vehicle outline dimension measurement solution.

[0278] 2. Research the calibration technology in visual measurement, build a whole-vehicle measurement model based on the combination of monocular and binocular, improve the method of extracting corner point positions in calibration, use Halcon's existing assistant to calibrate camera parameters, obtain a more accurate basic matrix, and improve measurement accuracy; combine the camera and manipulator with the nine-point calibration method, use the transformation matrix to unify the coordinates, realize automatic adjustment of the binocular platform, and provide more reliable real-time vehicle model parameters for the intelligent parking system.

[0279] 3. A vehicle image edge detection algorithm based on an improved genetic algorithm is proposed. By using the beetle whisker search algorithm to improve the premature convergence of the genetic algorithm, the vehicle edge can be detected more accurately. The system uses the background difference method to extract the edge features of the vehicle image data collected by a single camera, reduce interference factors, accurately extract the required vehicle edge features, calculate the size of the extracted vehicle edge, and use pattern recognition to obtain the vehicle type. The stereo matching algorithm is studied. By analyzing and comparing the advantages and disadvantages of each stereo matching algorithm, feature-based stereo matching and regional grayscale-based stereo matching are combined, and the epipolar line method is used to optimize the matching points, remove points with large errors, and achieve accurate image matching. It provides real-time parameters for the selection of carrier robots and empty parking space matching in the parking system.

[0280] 4. By combining multiple methods to find the measurement points, the best measurement points for measuring the dimensions of the vehicle are determined to improve the measurement accuracy. The host computer UI operation interface is designed to summarize and compare a large amount of data, realize visualization, and facilitate users to analyze the accuracy of the measurement results. Through the transmission of vehicle parameters, an effective basis is provided for the application of the intelligent parking system.

[0281] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for visually measuring vehicle model parameters of an intelligent parking system, characterized in that: Including: Collecting vehicle images of the vehicle to be parked; Preprocessing the vehicle images to identify the vehicle type; Based on the identified vehicle type, performing stereo matching on the vehicle to be parked to construct a three-dimensional vehicle model; Based on the three-dimensional vehicle model, determining measurement parameters and performing visual measurement.

2. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 1, characterized in that: Preprocessing the vehicle images includes: Performing edge detection on the vehicle images; Performing feature point extraction and feature line processing on the vehicle images after edge detection; Identifying the vehicle type based on the extracted feature points and feature lines.

3. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 2, characterized in that: The method for performing edge detection on the vehicle images is: using a genetic algorithm combined with improved beetle antennae search, including: S1.

1. Performing grayscale processing on the vehicle images and setting the initial values of the algorithm; S1.

2. Encoding the grayscale values of pixel points into chromosomes and randomly selecting the initial population; S1.

3. Calculating the gradient change rate of each individual in the population based on a preset formula and calculating the fitness value of the individual; S1.

4. Generating offspring through crossover and mutation; S1.

5. Using the improved beetle antennae search method to guide the movement of the parent crossover points participating in the crossover operation in the new generation population and updating the offspring; S1.6 If i < T, then i = i + 1 and go to S1.3, if i = T, then go to S1.7; where, i is the number of iterations that have been performed, and T is the upper limit of the number of iterations; S1.

7. Outputting the optimal solution in the population, that is, the detected image edge; Wherein, the preset formula is: Among them, ▽θ is the gradient direction, is the partial derivative of the function f with respect to the x-coordinate, is the partial derivative of function f with respect to the y-coordinate, and θ is the angle between the gradient direction and the x-axis; The method for improving beetle antennae search is: Changing the search step length by changing the step length parameter of the original beetle antennae search, first determining the range of the search direction with the first preset step length, and then performing fine search with the second preset step length.

4. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 2, characterized in that: Performing feature point extraction and feature line extraction on the vehicle images after edge detection includes: Using the Harris algorithm to extract feature points in the vehicle images after edge detection; Converting the pixels of the feature points to the sub-pixel level; Using the Hough transform to extract feature lines in the vehicle images after edge detection.

5. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 1, characterized in that: Based on the identified vehicle type, performing stereo matching on the vehicle to be parked to construct a three-dimensional vehicle model includes: S2.

1. Using left and right cameras to collect the left vehicle image and the right vehicle image of the vehicle to be parked; S2.

2. Using the Sobel operator to perform edge detection on the left vehicle image and the right vehicle image to obtain edge points; S2.

3. Based on the camera calibration parameters, obtaining the fundamental matrix of the left and right cameras; S2.

4. Selecting an arbitrary edge point in the image collected by the left camera, calculating the corresponding epipolar line in the image collected by the right camera by the fundamental matrix, denoted as l2; taking an arbitrary point on l2 and calculating the corresponding epipolar line in the image collected by the left camera by the fundamental matrix, denoted as l1; where, all the edge points on l1 and l2 correspond one by one; S2.

5. Based on the preset similarity evaluation function, calculate the evaluation value of the similarity between the edge points on l1 and l2 and establish a two-dimensional similarity evaluation function list M, in which the units M(i,j) are the similarity evaluation values ​​of the i-th point on l1 and the j-th point on l2 respectively; if the threshold of a certain unit M(i,j) in the list is the smallest in its row and column, and the threshold level is less than the threshold of a certain unit, then the i-th point on l1 and the j-th point on l2 are preliminarily determined to correspond to one point; S2.

6. Repeat S2.5 until all edge feature points are matched and calculated. After each overall matching of the two epipolar lines, the points that have been calculated will no longer appear in subsequent calculations, that is, the uniqueness constraint is satisfied. S2.

7. According to the limited gradient of the screening disparity value, a corresponding screening is performed among the candidate related pixel points obtained under the preset constraint conditions to obtain the optimized matching points; S2.

8. Based on the optimized matching points, find the ideal matching point pair closest to the epipolar line, obtain the final precise matching point pair, calculate the three-dimensional coordinates of the vehicle based on the final precise matching point pair, and construct a three-dimensional vehicle model.

6. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 5, characterized in that: The preset similarity evaluation function is: Where f(x,y) is the matching function used to evaluate the matching degree between point (u,v) and point (u+x,v+y) in image 1 and image 2, M is the normalization factor to ensure that the value of the matching function is within a reasonable range, ∑ (u，v)∈W is the sum of all points (u, v) in window W, I1(u, v) is the gray value of point (u, v) in image 1, is the average gray value of image 1, I2(u+x,v+y) is the gray value of the midpoint (u+x,v+y) of image 2, is the average gray value of image 2, S(x,y) is the final matching score function, which combines gray matching and gradient matching, f(x,y) is the gray matching score, and k is the proportional probability parameter used to balance the weights of gray matching and gradient matching. is the gradient difference between image 1 and image 2 in the x direction, is the gradient difference between image 1 and image 2 in the y direction, u0 is the horizontal coordinate of the reference point in image 1, and v0 is the vertical coordinate of the reference point in image 1; The preset constraint conditions include: epipolar constraint, uniqueness constraint, mutual correspondence constraint, compatibility constraint and similarity constraint, and parallax continuity constraint.

7. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 1, characterized in that: Determining measurement parameters based on the three-dimensional vehicle model and performing visual measurement includes: Determine measurement parameters; wherein the measurement parameters include: vehicle length, vehicle width, vehicle height, wheelbase, front overhang, and rear overhang; Based on the three-dimensional vehicle model, the measurement points of the vehicle length, vehicle height, wheelbase, front overhang and rear overhang are positioned by using a grayscale projection method, and based on the positioned measurement points, the vehicle length, vehicle height, wheelbase, front overhang and rear overhang are measured.

8. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 7, characterized in that: Measuring vehicle length, width and height includes: Using sensors to measure the width of the vehicle; According to the center of gravity principle, the center of gravity of the 3D vehicle model is located at the geometric center of the 3D vehicle model, the position of the rotation axis is determined, each row is scanned and the center of gravity coordinates (x, y) of each row are calculated; The center of gravity coordinates (x j ,y j )(j=1,2,…,M) is a set of data, and the least square method is used to solve the linear equation y of the initial position of the vertical principal axis j =k1x j +b1; Similarly, scan each column in turn and calculate the centroid coordinates (x i ,y i )(i=1,2,...,N), use the least squares method to get the linear equation y of the initial horizontal position i =k2x i +b2; After solving the two linear equations, the intersection point (x0, y0) is found; a line passing through the intersection point rotates around the intersection point, and the initial position of the line starts from being parallel to the horizontal axis and gradually changes to a position perpendicular to the horizontal axis. The direction of rotation gradually increases in the direction of the slope of the horizontal axis; when the surface area of ​​a rectangle that can contain the entire body around the body is the smallest, the resulting rectangle is the smallest circumscribed rectangle around the body; The length of the minimum circumscribed rectangle is the length of the vehicle, and the row coordinates of the upper side corresponding to the width of the minimum circumscribed rectangle minus the row coordinates of the lowest point of the wheel obtained by Hough transform is the distance of the vehicle height in the image, which is then calculated by the coordinate system transformation matrix to obtain the actual vehicle height.

9. The method for visually measuring vehicle model parameters of an intelligent parking system according to claim 7, characterized in that: Measuring wheelbase includes: The wheel position is detected by adding the preset condition of Hough transform, and the center points of the two circles detected are the wheelbase measurement points; Wherein, the preset condition of the Hough transform is: Where F(x,y) is the pixel, G(x,y) is the region of interest required in the transformation, w and h represent the length and width of the image, abs(y1-y2)≤3 means that the two centers are basically on the same horizontal line, and abs(x1-x2)≥w / 2 represents the horizontal distance constraint between the two centers.

Images