Vehicle autonomous parking navigation method based on panoramic perception

Abstract

The invention relates to the technical field of vehicle traffic control, in particular to a vehicle autonomous parking navigation method based on panoramic perception. According to the method, information significance is obtained by using gray value distribution in a first suspected license plate area, and screening is carried out to obtain a real license plate area and a second suspected license plate area. And according to the character position rationality and character shape rationality of the suspected character region in the second suspected license plate region, screening out a real license plate in the second suspected license plate region. And predicting moving tracks of other vehicles in the target vehicle environment according to the real license plate so as to adjust a parking strategy of the target vehicle. According to the invention, the accurate license plate information is obtained by accurately identifying the license plates of other vehicles in the environment to assist the parking strategy of the target vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle traffic control, and in particular to a vehicle autonomous parking navigation method based on panoramic perception. Background Art

[0002] In large-scale intelligent scenarios such as smart factories, automatic parking of vehicles is an important technology of intelligent transportation systems. In actual scenarios, the target vehicle can determine the planned route of each vehicle through license plate recognition and vehicle networking technology, thereby assisting the target vehicle in making safe and reasonable route planning. Existing technologies can use panoramic perception to obtain videos of the surrounding environment using the vehicle's own 360-degree camera, and then determine the vehicles in the environment. However, when determining the license plate information of the environment vehicle, due to camera distortion and light interference, areas that do not belong to the license plate characters will be identified as license plate areas, and then the license plate will be misidentified, resulting in errors in the automatic parking route planning of the target vehicle, creating safety hazards. Summary of the invention

[0003] In order to solve the technical problem in the prior art that the wrong license plate area is identified as the license plate area during license plate recognition due to factors such as environmental interference, resulting in the inability to accurately determine the trajectory of other vehicles in the environment during automatic parking, and further resulting in the inability of the target vehicle to effectively plan the automatic parking route, the purpose of the present invention is to provide a vehicle autonomous parking navigation method based on panoramic perception, and the technical solution adopted is as follows: The present invention proposes a vehicle autonomous parking navigation method based on panoramic perception, the method comprising: Obtain an environment video frame captured by a panoramic image acquisition device of the target vehicle, and obtain a first suspected license plate area of ​​other vehicles in the environment video frame; Obtaining a grayscale histogram of the first suspected license plate area, and obtaining information significance according to the position distribution of peaks in the grayscale histogram; dividing the first suspected license plate area into a real license plate area and a second suspected license plate area according to the information significance; Obtain a suspected character region composed of closed edges in the second suspected license plate region; obtain the rationality of the character position of the second suspected license plate region based on the uniformity of the position distribution of the suspected character region in the second suspected license plate region; perform linear fitting and circular fitting on the closed edge of the suspected character region, and obtain the rationality of the character shape of the second suspected license plate region based on the fitting deviation; screen out the real license plate region in the second suspected license plate region based on the rationality of the character position and the rationality of the character shape; The moving trajectories of other vehicles are predicted according to the real license plate area, and the parking strategy of the target vehicle is adjusted according to the moving trajectories of other vehicles.

[0004] Furthermore, the method for obtaining the information saliency includes: Select the first two largest peaks in the grayscale histogram as representative peaks according to the peak value; obtain the peak point grayscale difference between the two representative peaks; Using the Otsu threshold algorithm to obtain a grayscale segmentation threshold of the first suspected license plate area, the grayscale segmentation threshold divides the grayscale histogram into two parts, and uses the grayscale standard deviation of the two parts as a representative standard deviation; according to the size of the representative standard deviation, the grayscale distribution concentration is obtained; The information significance is obtained according to the peak point grayscale difference and the grayscale distribution concentration.

[0005] Furthermore, if the information significance of the first suspected license plate region is greater than a preset significance threshold, it is classified as a real license plate region, otherwise it is classified as a second suspected license plate region.

[0006] Furthermore, the closed edge acquisition method includes: Obtain an LBP image of the second suspected license plate area, perform edge detection on the LBP image, and obtain the closed edge.

[0007] Furthermore, the method for obtaining the rationality of the character position includes: The vector between the center point of the suspected character area and the center point of the second suspected license plate area is used as the position vector of each suspected character area; the difference between the position vector of each suspected character area and the position vector of the corresponding character of the standard license plate is counted to obtain the representative difference, the representative difference is negatively correlated mapped and normalized to obtain the rationality of the character position.

[0008] Furthermore, the method for obtaining the rationality of the character shape includes: For any suspected character area, the closed edge is divided into multiple independent edges, and the independent edges are fitted with straight lines and circles to obtain the fitting determination coefficients under the two fitting processes, as well as the mean square error between the fitting value and the true value; for each fitting process, the fitting effect is obtained according to the fitting determination coefficient and the mean square error; the maximum fitting effect of the two fitting processes is selected as the edge shape rationality of the independent edge; the average value of the edge shape rationality of all independent edges is taken as the initial character shape rationality of the suspected character area; The average value of the initial character shape rationality of all the suspected character regions of the second suspected license plate region is normalized to obtain the character shape rationality.

[0009] Furthermore, the overall character rationality of the second suspected license plate area is obtained based on the character position rationality and the character shape rationality. If the overall character rationality is greater than a preset rationality threshold, the second suspected license plate area is taken as the real license plate area.

[0010] Furthermore, the overall character rationality is an average value of the character position rationality and the character shape rationality.

[0011] Furthermore, obtaining the information significance according to the peak point grayscale difference and the grayscale distribution concentration includes: The peak point grayscale difference is the absolute value of the difference between the peak points of two representative peaks; the product of the peak point grayscale difference and the grayscale distribution concentration is used as the information significance.

[0012] Furthermore, the method for obtaining the grayscale distribution centrality includes: The average values ​​between the representative standard deviations are negatively correlated and normalized to obtain the grayscale distribution centrality.

[0013] The present invention has the following beneficial effects: The present invention takes into account that the first suspected license plate area initially obtained has a non-license plate area, because the characters in the license plate area have obvious visual differences from the background, so the first suspected license plate area is first screened using information prominence, that is, the greater the information prominence, the more obvious the character information in the first suspected license plate area, so a portion of the real license plate area can be directly screened out. For the second suspected license plate area, it may lack detailed information due to environmental factors such as lighting, and the prominence is low. The edges of the characters distributed in the area may also be the edges of the real license plate characters. Therefore, the second suspected license plate area is further analyzed to determine the suspected character area. Because the character area in the license plate has a fixed position distribution law and font style, the real license plate area can be further screened out by the rationality of the character position and the rationality of the character shape, and finally the movement trajectory of other vehicles is obtained to assist the target vehicle in adjusting its own parking strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1A flow chart of a vehicle autonomous parking navigation method based on panoramic perception provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a vehicle autonomous parking navigation method based on panoramic perception proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0017] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0018] A specific scheme of a vehicle autonomous parking navigation method based on panoramic perception provided by the present invention is described in detail below with reference to the accompanying drawings.

[0019] See also Figure 1 , which shows a flow chart of a vehicle autonomous parking navigation method based on panoramic perception provided by an embodiment of the present invention, the method comprising: Step S1: obtaining an environment video frame captured by a panoramic image acquisition device of a target vehicle, and obtaining a first suspected license plate area of ​​other vehicles in the environment video frame.

[0020] In an embodiment of the present invention, the target vehicle can collect a panoramic video of the surrounding environment through a surround view camera, i.e., a 360-degree imaging camera, wherein the video frame in the panoramic video is the environmental video frame. The first suspected license plate area of ​​other vehicles in the environmental video frame can be obtained by using the existing license plate area recognition algorithm. It should be noted that in order to improve the image quality and facilitate subsequent algorithm processing, the first suspected license plate area obtained can also be grayed and filtered for denoising, which are technical means well known to those skilled in the art and will not be elaborated here.

[0021] In the embodiment of the present invention, the vehicle image in the environment video frame can be identified by the vehicle recognition algorithm in OpenCV, and then the license plate area can be identified by the existing semantic segmentation network. Both OpenCV and the semantic segmentation network are well-known technical means to those skilled in the art, and are not limited or elaborated here.

[0022] Step S2: Obtain a grayscale histogram of the first suspected license plate area, obtain information significance according to the position distribution of the peaks in the grayscale histogram; and divide the first suspected license plate area into a real license plate area and a second suspected license plate area according to the information significance.

[0023] For license plates, the characters on the license plates have significant color differences from the background, so the license plate area can be regarded as composed of two pixels with large grayscale differences. Therefore, in order to analyze the grayscale distribution of the first suspected license plate area, the embodiment of the present invention obtains a grayscale histogram. The peaks in the grayscale histogram can represent the distribution characteristics of the main grayscale value. The farther the position distance between the peaks, the more consistent with the grayscale distribution relationship between the characters and the background; and the more concentrated the peaks are, the purer the pixel information in the area is, and the area may only contain characters and background information. Therefore, the information significance can be obtained based on the position distribution of the peaks in the grayscale histogram. The greater the information significance, the more consistent the information in the first suspected license plate area is with the characters plus background information of the real license plate; otherwise, it means that the first suspected license plate area is not a real license plate, or the information is not significant due to factors such as lighting, and further analysis is required. Therefore, the embodiment of the present invention first divides the first suspected license plate area into a real license plate area and a second suspected license plate area according to the information significance, and further analyzes the second suspected license plate area.

[0024] Preferably, in the embodiment of the present invention, considering that the background in the license plate area is low grayscale information and the characters are high grayscale information, the grayscale histogram can be analyzed in two parts, so the method for obtaining the information saliency includes: For the grayscale histogram, the horizontal axis is the grayscale value and the vertical axis is the frequency of pixel occurrence. Therefore, for the license plate area, its grayscale histogram should have two relatively obvious peaks, both of which are peaks with larger peak values, and there is an obvious grayscale difference between the two peaks, that is, the distance between the two peaks is far. Therefore, according to the peak size, the first two largest peaks in the grayscale histogram are selected as representative peaks, and the peak point grayscale difference between the two representative peaks is obtained. The greater the peak point grayscale difference, the more the position distribution between the two representative peaks conforms to the distribution characteristics of the real license plate.

[0025] Further considering that the two peaks in the grayscale histogram of the real license plate area should be relatively smooth, because the license plate area only includes characters and background information, there will be no interference from other information, so the grayscale segmentation threshold of the first suspected license plate area is obtained by using the Otsu threshold algorithm. The grayscale segmentation threshold divides the grayscale histogram into two parts, and the grayscale standard deviation of the two parts is used as the representative standard deviation. For the grayscale standard deviation of these two parts, the grayscale standard deviation between the two parts is small, which means that the information of the low grayscale area and the high grayscale area of ​​the grayscale histogram is relatively simple, which is consistent with the information distribution of the real license plate area. Therefore, the concentration of grayscale distribution can be obtained according to the representative standard deviation, that is, the smaller the two representative standard deviations, the more concentrated the grayscale distribution and the simpler the information.

[0026] The information significance can be further obtained based on the peak point grayscale difference and grayscale distribution concentration. That is, the greater the peak point grayscale difference and the greater the grayscale distribution concentration, the more the first suspected license plate area conforms to the grayscale distribution in the real license plate area, and the greater the information significance.

[0027] Preferably, in one embodiment of the present invention, the average value between the representative standard deviations is negatively correlated and normalized to obtain distribution centrality. The method of negative correlation mapping and normalization in the embodiment of the present invention is to first normalize the average value using range normalization, and then negative correlation mapping and normalization can be achieved by subtracting the normalized value from the positive integer 1.

[0028] Preferably, in one embodiment of the present invention, the peak grayscale difference is the absolute value of the difference between the peak points between two representative peaks; and the product of the peak point grayscale difference and the grayscale distribution concentration is taken as the information significance.

[0029] In one embodiment of the present invention, if the information significance of the first suspected license plate area is greater than a preset significance threshold, the information in the first suspected license plate area is considered to conform to the distribution characteristics of the real license plate information and is classified as a real license plate area, otherwise it is a second suspected license plate area for subsequent analysis. In the embodiment of the present invention, after the information significance is normalized, the significance threshold is set to 0.7.

[0030] Step S3: Obtain a suspected character area composed of closed edges in the second suspected license plate area; obtain the rationality of the character position of the second suspected license plate area based on the uniformity of the position distribution of the suspected character area in the second suspected license plate area; perform straight line fitting and circular fitting on the closed edges of the suspected character area, and obtain the rationality of the character shape of the second suspected license plate area based on the fitting deviation; screen out the real license plate area in the second suspected license plate area based on the rationality of the character position and the rationality of the character shape.

[0031] The second suspected license plate area includes non-license plate areas, and also includes areas where character information is not significant due to factors such as lighting, so it is necessary to further filter out areas where real license plate character information exists. Therefore, the embodiment of the present invention first obtains a suspected character area composed of closed edges in the second suspected license plate area. For the character area, it has a fixed position in the license plate and is distributed relatively evenly. For example, the traditional Chinese oil truck license plate consists of 7 characters of uniform size and uniform spacing. Therefore, the rationality of the character position can be first determined based on the uniformity of the position distribution of the suspected character area in the second suspected license plate area. Further considering that the characters on the license plate have a fixed font, whether numbers or text are composed of smooth straight lines or curves, the closed edges of the suspected character area can be linearly fitted and circularly fitted. The larger the fitting deviation, the less the shape of the closed edge does not conform to the font shape of the license plate character. Therefore, the rationality of the character shape of the second suspected license plate area can be obtained based on the fitting deviation. The real license plate area in the second suspected license plate area can be further screened out by the rationality of the character position and the rationality of the character shape.

[0032] Preferably, in an embodiment of the present invention, in order to further ensure the quality of edge information extraction, before edge detection is performed on the second suspected license plate area, an LBP image of the second suspected license plate area is obtained, and edge detection is performed on the LBP image to obtain a closed edge. The LBP image is an image composed of LBP values ​​obtained by using the LBP algorithm to obtain the LBP value of each pixel in the second suspected license plate area. The LBP image has more significant edge features, so more significant edge information can be obtained. The edge detection algorithm in the embodiment of the present invention adopts the canny edge detection algorithm, which is a technical means well known to those skilled in the art and will not be described in detail here.

[0033] Preferably, in one embodiment of the present invention, considering that the characters of a real license plate have fixed positions, when obtaining the rationality of the character positions, the position information on the real license plate template may be compared to obtain the rationality of the character positions, specifically including: The vector between the center point of the suspected character area and the center point of the second suspected license plate area is used as the position vector of each suspected character area; the difference between the position vector of each suspected character area and the position vector of the corresponding character of the standard license plate is counted to obtain a representative difference, and the representative difference is negatively correlated and normalized to obtain the rationality of the character position. In an embodiment of the present invention, the difference between the position vectors is the cosine distance of the two vectors. The minimum cosine distance in all suspected character areas is counted as the representative difference. That is, the smaller the representative difference, the more similar the position of the suspected character area is to the character position in the standard license plate, the more likely the second suspected license plate area is to be a real license plate, and the greater the rationality of the character position.

[0034] It should be noted that the normalization and negative correlation normalization methods in the embodiments of the present invention are technical means well known to those skilled in the art. In the embodiments of the present invention, the normalization or negative correlation mapping in different steps can be achieved by a fixed method, and the details will not be repeated.

[0035] Preferably, in an embodiment of the present invention, the method for obtaining the rationality of character shape includes: For any suspected character area, since the closed edge may be the edge of the character, and the edge of the character has a certain trend, in order to analyze the font shape of the suspected character area, the closed edge needs to be divided into multiple independent edges. In the embodiment of the present invention, the corner point of the closed edge can be obtained by a corner point detection algorithm, and multiple independent edges can be obtained by using the corner point as a segmentation point.

[0036] The independent edge is fitted with a straight line and a circle to obtain the fitting determination coefficient under the two fitting processes, as well as the mean square error between the fitting value and the true value. The fitting determination coefficient is a parameter generated during the fitting process. The larger the coefficient, the better the current fitting effect; the mean square error is the fitting deviation, which represents the distance difference between the true value and the fitting value. The larger the mean square error, the less the independent edge is a straight line or a smooth circular curve. It should be noted that the linear fitting and the circular fitting embodiments of the present invention can be fitted using the least squares method. The specific fitting method is well known to those skilled in the art and will not be described in detail here.

[0037] For each fitting process, the fitting effect is obtained based on the fitting determination coefficient and mean square error. Because there are two fitting processes for each independent edge, the maximum fitting effect of the two fitting processes is selected as the edge shape rationality of the independent edge.

[0038] All suspected character regions and independent edges are further counted, and the average value of the edge shape rationality of all independent edges is used as the initial character shape rationality of the suspected character region. The average value of the initial character shape rationality of all suspected character regions in the second suspected license plate region is normalized to obtain the character shape rationality.

[0039] Preferably, in the embodiment of the present invention, the overall character rationality of the second suspected license plate area is further obtained based on the character position rationality and the character shape rationality. If the overall character rationality is greater than a preset rationality threshold, the second suspected license plate area is taken as the real license plate area. The overall character rationality can be normalized for processing, and then the rationality threshold is set to 0.6.

[0040] In the embodiment of the present invention, the overall character rationality is the average value of the character position rationality and the character shape rationality.

[0041] Step S4: predicting the movement trajectory of other vehicles based on the real license plate area, and adjusting the parking strategy of the target vehicle based on the movement trajectory of other vehicles.

[0042] In an embodiment of the present invention, after the license plates of other vehicles are identified, the movement trajectories of other vehicles can be predicted, mainly through the data stored in the parking lot database, such as the vehicle entry time, historical behavior, etc. For example, if other vehicles are detected to stay for a short time near the target parking space of the target vehicle, and the system predicts that the vehicle may leave, the target vehicle can dynamically adjust the path to detour or wait. When the vehicles traveling in the opposite direction at that time are vehicles with high frequency of occupying lanes in the database, the avoidance strategy can be used preferentially to reduce the risk of congestion, where high frequency of occupying lanes vehicles are work vehicles such as logistics vehicles. If it is detected that the target parking space of the target vehicle is illegally occupied by other vehicles, an abnormal information can be sent to the parking lot and a spare parking space can be searched.

[0043] In summary, the embodiment of the present invention uses the gray value distribution in the first suspected license plate area to obtain information significance, and performs screening to obtain the real license plate area and the second suspected license plate area. The real license plate in the second suspected license plate area is screened out based on the rationality of the character position and the rationality of the character shape of the suspected character area in the second suspected license plate area. The movement trajectory of other vehicles in the target vehicle environment is predicted based on the real license plate, and then the parking strategy of the target vehicle is adjusted. The present invention obtains accurate license plate information to assist the parking strategy of the target vehicle by accurately identifying the license plates of other vehicles in the environment.

[0044] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0045] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A vehicle autonomous parking navigation method based on panoramic perception, characterized in that: The method comprises: Obtain an environment video frame captured by a panoramic image acquisition device of the target vehicle, and obtain a first suspected license plate area of ​​other vehicles in the environment video frame; Obtaining a grayscale histogram of the first suspected license plate area, and obtaining information significance according to the position distribution of peaks in the grayscale histogram; dividing the first suspected license plate area into a real license plate area and a second suspected license plate area according to the information significance; Obtain a suspected character region composed of closed edges in the second suspected license plate region; obtain the rationality of the character position of the second suspected license plate region based on the uniformity of the position distribution of the suspected character region in the second suspected license plate region; perform linear fitting and circular fitting on the closed edge of the suspected character region, and obtain the rationality of the character shape of the second suspected license plate region based on the fitting deviation; screen out the real license plate region in the second suspected license plate region based on the rationality of the character position and the rationality of the character shape; The moving trajectories of other vehicles are predicted according to the real license plate area, and the parking strategy of the target vehicle is adjusted according to the moving trajectories of other vehicles.

2. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that: The method for obtaining the information saliency includes: Select the first two largest peaks in the grayscale histogram as representative peaks according to the peak value; obtain the peak point grayscale difference between the two representative peaks; Using the Otsu threshold algorithm to obtain a grayscale segmentation threshold of the first suspected license plate area, the grayscale segmentation threshold divides the grayscale histogram into two parts, and uses the grayscale standard deviation of the two parts as a representative standard deviation; according to the size of the representative standard deviation, the grayscale distribution concentration is obtained; The information significance is obtained according to the peak point grayscale difference and the grayscale distribution concentration.

3. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that: If the information significance of the first suspected license plate region is greater than a preset significance threshold, it is classified as a real license plate region, otherwise it is a second suspected license plate region.

4. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that: The method for obtaining the closed edge includes: Obtain an LBP image of the second suspected license plate area, perform edge detection on the LBP image, and obtain the closed edge.

5. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that: The method for obtaining the rationality of the character position includes: The vector between the center point of the suspected character area and the center point of the second suspected license plate area is used as the position vector of each suspected character area; the difference between the position vector of each suspected character area and the position vector of the corresponding character of the standard license plate is counted to obtain the representative difference, the representative difference is negatively correlated mapped and normalized to obtain the rationality of the character position.

6. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that: The method for obtaining the rationality of the character shape comprises: For any suspected character area, the closed edge is divided into multiple independent edges, and the independent edges are fitted with straight lines and circles to obtain the fitting determination coefficients under the two fitting processes, as well as the mean square error between the fitting value and the true value; for each fitting process, the fitting effect is obtained according to the fitting determination coefficient and the mean square error; the maximum fitting effect of the two fitting processes is selected as the edge shape rationality of the independent edge; the average value of the edge shape rationality of all independent edges is taken as the initial character shape rationality of the suspected character area; The average value of the initial character shape rationality of all the suspected character regions of the second suspected license plate region is normalized to obtain the character shape rationality.

7. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that: The overall character rationality of the second suspected license plate area is obtained according to the character position rationality and the character shape rationality. If the overall character rationality is greater than a preset rationality threshold, the second suspected license plate area is taken as the real license plate area.

8. The vehicle autonomous parking navigation method based on panoramic perception according to claim 7, characterized in that: The overall character rationality is the average of the character position rationality and the character shape rationality.

9. The method for autonomous parking navigation based on panoramic perception according to claim 2, characterized in that: The obtaining of the information significance according to the peak point grayscale difference and the grayscale distribution concentration includes: The peak point grayscale difference is the absolute value of the difference between the peak points of two representative peaks; the product of the peak point grayscale difference and the grayscale distribution concentration is used as the information significance.

10. The vehicle autonomous parking navigation method based on panoramic perception according to claim 2, characterized in that: The method for obtaining the grayscale distribution centrality includes: The average values ​​between the representative standard deviations are negatively correlated and normalized to obtain the grayscale distribution centrality.

Images