Vehicle autonomous parking navigation method based on panoramic perception

Abstract

The present invention relates to the technical field of vehicle traffic control, and particularly relates to a vehicle autonomous parking navigation method based on panoramic perception. This method obtains information saliency using the gray value distribution in the first suspected license plate area, and performs screening to obtain the true license plate area and the second suspected license plate area. The true license plate in the second suspected license plate area is screened out according to the rationality of the character positions and the rationality of the character shapes of the suspected character areas in the second suspected license plate area. The moving trajectories of other vehicles in the target vehicle environment are predicted based on the true license plate, and then the parking strategy of the target vehicle is adjusted. The present invention accurately identifies the license plates of other vehicles in the environment to obtain accurate license plate information to assist the parking strategy of the target vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle traffic control, and particularly to a vehicle autonomous parking navigation method based on panoramic perception. Background Art

[0002] In intelligent large-scale scenarios such as intelligent factories, automatic parking of vehicles is an important technology in intelligent transportation systems. In actual scenarios, a target vehicle can determine the planned route of each vehicle through license plate recognition and vehicle networking technology, and then assist the target vehicle in making a safe and reasonable route plan. Existing technologies can use panoramic perception to obtain videos of the surrounding environment through the vehicle's own 360-degree camera, and then determine the vehicles existing in the environment. However, when determining the license plate information of environmental vehicles, due to factors such as camera distortion and light interference, areas that do not belong to license plate characters will be recognized as license plate areas, resulting in incorrect license plate recognition, which leads to incorrect automatic parking route planning of the target vehicle and potential safety hazards. Summary of the Invention

[0003] In order to solve the technical problem that in the prior art, due to factors such as environmental interference, incorrect license plate areas are recognized as license plate areas during license plate recognition, resulting in the inability to accurately determine the trajectories of other vehicles in the environment during automatic parking, and further leading to the inability of the target vehicle to effectively plan an automatic parking route, the purpose of the present invention is to provide a vehicle autonomous parking navigation method based on panoramic perception, and the specific technical solution adopted is as follows:

[0004] The present invention proposes a vehicle autonomous parking navigation method based on panoramic perception, and the method includes:

[0005] Obtain an environmental video frame captured by a panoramic image acquisition device of a target vehicle, and obtain a first suspected license plate area of other vehicles in the environmental video frame;

[0006] Obtain the gray-level histogram of the first suspected license plate area, and obtain the information saliency according to the position distribution of peaks in the gray-level histogram; divide the first suspected license plate area into a real license plate area and a second suspected license plate area according to the information saliency;

[0007] Obtain a suspected character area composed of closed edges in the second suspected license plate area; obtain the character position rationality of the second suspected license plate area according to the position distribution uniformity of the suspected character area in the second suspected license plate area; perform linear fitting and circular fitting on the closed edges of the suspected character area, and obtain the character shape rationality of the second suspected license plate area according to the fitting deviation; screen out the real license plate area in the second suspected license plate area according to the character position rationality and the character shape rationality;

[0008] Predict the movement trajectories of other vehicles based on the real license plate region, and adjust the parking strategy of the target vehicle according to the movement trajectories of other vehicles.

[0009] Further, the method for obtaining the information saliency includes:

[0010] Select the first two largest peaks in the grayscale histogram as representative peaks according to the peak magnitudes; obtain the grayscale difference of the peak points between the two representative peaks;

[0011] Obtain the grayscale segmentation threshold of the first suspected license plate region by using the Otsu threshold algorithm. The grayscale segmentation threshold divides the grayscale histogram into two parts, and use the grayscale standard deviations of the two parts as representative standard deviations; obtain the grayscale distribution concentration according to the magnitudes of the representative standard deviations.

[0012] Obtain the information saliency according to the grayscale difference of the peak points and the grayscale distribution concentration.

[0013] Further, if the information saliency of the first suspected license plate region is greater than the preset saliency threshold, it is divided into a real license plate region; otherwise, it is a second suspected license plate region.

[0014] Further, the method for obtaining the closed edge includes:

[0015] Obtain the LBP image of the second suspected license plate region, perform edge detection on the LBP image, and obtain the closed edge.

[0016] Further, the method for obtaining the character position rationality includes:

[0017] Take the vector between the center point of the suspected character region and the center point of the second suspected license plate region as the position vector of each suspected character region; count the differences between the position vectors of each suspected character region and the position vectors of the corresponding characters on the standard license plate to obtain a representative difference, perform a negative correlation mapping and normalization on the representative difference to obtain the character position rationality.

[0018] Further, the method for obtaining the character shape rationality includes:

[0019] For any suspected character region, divide the closed edge into multiple independent edges, perform linear and circular fittings on the independent edges to obtain the fitting determination coefficients under the two fitting processes, as well as the mean square error between the fitting values and the true values; for each fitting process, obtain the fitting effect according to the fitting determination coefficient and the mean square error; select the maximum fitting effect of the two fitting processes as the edge shape rationality of the independent edge; take the average value of the edge shape rationalities of all independent edges as the initial character shape rationality of the suspected character region;

[0020] Normalize the average of the initial character shape rationalities of all suspected character regions in the second suspected license plate region to obtain the character shape rationality.

[0021] Furthermore, obtain the overall character rationality of the second suspected license plate region based on the character position rationality and the character shape rationality. If the overall character rationality is greater than a preset rationality threshold, then regard the second suspected license plate region as the true license plate region.

[0022] Furthermore, the overall character rationality is the average of the character position rationality and the character shape rationality.

[0023] Furthermore, the obtaining of the information saliency based on the peak point gray level difference and the gray level distribution concentration includes:

[0024] The peak point gray level difference is the absolute value of the difference between peak points between two representative peaks; take the product of the peak point gray level difference and the gray level distribution concentration as the information saliency.

[0025] Furthermore, the method for obtaining the gray level distribution concentration includes:

[0026] Perform a negative correlation mapping and normalization on the average between the representative standard deviations to obtain the gray level distribution concentration.

[0027] The present invention has the following beneficial effects:

[0028] The present invention takes into account that there are non-license plate regions in the initially obtained first suspected license plate region. Since there are obvious visual differences between the characters and the background in the license plate region, first, the information saliency is used to screen the first suspected license plate region. That is, the greater the information saliency, the more obvious the character information in the first suspected license plate region. Therefore, a part of the true license plate regions can be directly screened out. For the second suspected license plate region, due to environmental factors such as light, its detail information may be missing and the saliency is low. The character edges distributed in the region may also be the true license plate character edges. Therefore, for the second suspected license plate region, edge analysis is further performed to determine the suspected character regions. Since the character regions in the license plate have fixed position distribution rules and font styles, further, the true license plate regions can be screened out through the character position rationality and the character shape rationality, and finally, the moving trajectories of other vehicles are obtained to assist the target vehicle in adjusting its own parking strategy. Description of the Drawings

[0029] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative work, other drawings can be obtained based on these drawings.

[0030] Figure 1 Flowchart of a vehicle autonomous parking navigation method based on panoramic perception provided by an embodiment of the present invention. Detailed implementation manners

[0031] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in conjunction with the drawings and preferred embodiments, details the specific implementation manners, structures, features, and effects of a vehicle autonomous parking navigation method based on panoramic perception proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0033] The following specifically describes the specific solution of a vehicle autonomous parking navigation method based on panoramic perception provided by the present invention in conjunction with the drawings.

[0034] Please refer to Figure 1 , which shows a flowchart of a vehicle autonomous parking navigation method based on panoramic perception provided by an embodiment of the present invention. The method includes:

[0035] Step S1: Obtain the environmental video frames captured by the panoramic image acquisition device of the target vehicle, and obtain the first suspected license plate area of other vehicles in the environmental video frames.

[0036] In the embodiments of the present invention, the target vehicle can collect the panoramic video of the surrounding environment through a surround view camera, that is, a 360-degree imaging camera. The video frames in the panoramic video are the environmental video frames. The first suspected license plate area of other vehicles in the environmental video frames can be obtained by using existing license plate area recognition algorithms. It should be noted that in order to improve the image quality for subsequent algorithm processing, the obtained first suspected license plate area can also be gray-scaled and filtered to remove noise. This is a well-known technical means for those skilled in the art and will not be elaborated here.

[0037] In the embodiments of the present invention, a vehicle image in an environmental video frame can be recognized through a vehicle recognition algorithm in OpenCV, and then a license plate area can be recognized through an existing semantic segmentation network. Both OpenCV and the semantic segmentation network are well-known technical means to those skilled in the art, and will not be defined and elaborated herein.

[0038] Step S2: Obtain the gray-scale histogram of the first suspected license plate area, and obtain the information saliency according to the position distribution of the peaks in the gray-scale histogram; divide the first suspected license plate area into a real license plate area and a second suspected license plate area according to the information saliency.

[0039] For a license plate, there is a significant color difference between the characters on the license plate and the background. Therefore, the license plate area can be regarded as composed of two types of pixel points with large gray-scale differences. Therefore, in the embodiments of the present invention, in order to analyze the gray-scale distribution of the first suspected license plate area, a gray-scale histogram is obtained. The peaks in the gray-scale histogram can represent the distribution characteristics of the main gray-scale values. The farther the position distance between the peaks is, the more it conforms to the gray-scale distribution relationship between the characters and the background; and the more concentrated the peaks are, the purer the pixel information in the area is, and the area may only contain character and background information. Therefore, the information saliency can be obtained based on the position distribution of the peaks in the gray-scale histogram. The greater the information saliency is, the more the information in the first suspected license plate area conforms to the character plus background information of the real license plate; otherwise, it means that the first suspected license plate area is not a real license plate, or the information is not significant due to factors such as illumination, and further analysis is required. Therefore, in the embodiments of the present invention, the first suspected license plate area is first divided into a real license plate area and a second suspected license plate area according to the information saliency, and the second suspected license plate area is further analyzed.

[0040] Preferably, in the embodiments of the present invention, considering that the background in the license plate area is low-gray-scale information and the characters are high-gray-scale information, the gray-scale histogram can be analyzed in two parts. Therefore, the method for obtaining the information saliency includes:

[0041] For the gray-scale histogram, the horizontal axis is the gray-scale value and the vertical axis is the frequency of the occurrence of pixel points. Therefore, for the license plate area, there should be two relatively obvious peaks in its gray-scale histogram. Both peaks are peaks with relatively large peak values, and there is an obvious gray-scale difference between the two peaks, that is, the distance between the two peaks is far. Therefore, the first two largest peaks in the gray-scale histogram are selected as representative peaks according to the peak values, and the gray-scale difference between the peak points of the two representative peaks is obtained. The greater the gray-scale difference between the peak points is, the more the position distribution between the two representative peaks conforms to the distribution characteristics of the real license plate.

[0042] Further considering that the two peaks in the grayscale histogram of the real license plate area should be relatively smooth, because the license plate area only includes character and background information and there will be no interference from other information, the Otsu threshold algorithm is used to obtain the grayscale segmentation threshold of the first suspected license plate area. The grayscale segmentation threshold divides the grayscale histogram into two parts, and the grayscale standard deviations of the two parts are used as the representative standard deviations. For the grayscale standard deviations of these two parts, if the grayscale standard deviations between the two parts are both small, it indicates that the information in the low-grayscale area and the high-grayscale area of the grayscale histogram is relatively single, which conforms to the information distribution of the real license plate area. Therefore, the grayscale distribution concentration can be obtained according to the representative standard deviations, that is, the smaller the two representative standard deviations, the more concentrated the grayscale distribution and the more single the information.

[0043] Furthermore, the information saliency can be obtained based on the peak point grayscale difference and the grayscale distribution concentration. That is, the greater the peak point grayscale difference and the greater the grayscale distribution concentration, the more the first suspected license plate area conforms to the grayscale distribution in the real license plate area, and the greater the information saliency.

[0044] Preferably, in an embodiment of the present invention, the average value between the representative standard deviations is subjected to negative correlation mapping and normalization to obtain the distribution concentration. The method of negative correlation mapping and normalization in the embodiment of the present invention is to first normalize the average value by range normalization, and then subtract the normalized value from the positive integer 1 to achieve negative correlation mapping and normalization.

[0045] Preferably, in an embodiment of the present invention, the peak grayscale difference is the absolute value of the difference between the peak points of the two representative peaks; the product of the peak point grayscale difference and the grayscale distribution concentration is used as the information saliency.

[0046] In an embodiment of the present invention, if the information saliency of the first suspected license plate area is greater than the preset saliency threshold, it is considered that the information in the first suspected license plate area conforms to the real license plate information distribution characteristics, and it is classified as the real license plate area, otherwise it is the second suspected license plate area for subsequent step analysis. In the embodiment of the present invention, after normalizing the information saliency, the saliency threshold is set to 0.7.

[0047] Step S3: Obtain the suspected character areas formed by the closed edges in the second suspected license plate area; obtain the character position rationality of the second suspected license plate area according to the uniform distribution of the suspected character areas in the second suspected license plate area; perform linear fitting and circular fitting on the closed edges of the suspected character areas, and obtain the character shape rationality of the second suspected license plate area according to the fitting deviation; screen out the real license plate area in the second suspected license plate area according to the character position rationality and the character shape rationality.

[0048] The second suspected license plate area contains non-license plate areas and also areas where character information is not prominent due to factors such as lighting. Therefore, it is necessary to further screen out the areas with real license plate character information. Therefore, in the embodiment of the present invention, first, the suspected character areas composed of closed edges in the second suspected license plate area are obtained. For character areas, they have fixed positions in the license plate and are relatively evenly distributed. For example, traditional Chinese fuel vehicle license plates are composed of 7 characters of the same size and evenly spaced. Therefore, the rationality of character positions can be determined first according to the uniform distribution of the suspected character areas in the second suspected license plate area. Further considering that the characters on the license plate have fixed fonts, whether they are numbers or characters, they are composed of smooth straight lines or curves. Therefore, linear fitting and circular fitting can be performed on the closed edges of the suspected character areas. The greater the fitting deviation, the more the shape of the closed edge does not conform to the font shape of the license plate characters. Therefore, the rationality of the character shapes in the second suspected license plate area can be obtained according to the fitting deviation. Through the rationality of character positions and the rationality of character shapes, the real license plate area in the second suspected license plate area can be further screened out.

[0049] Preferably, in the embodiment of the present invention, in order to further ensure the quality of edge information extraction, before performing edge detection on the second suspected license plate area, the LBP image of the second suspected license plate area is obtained, and edge detection is performed on the LBP image to obtain closed edges. The LBP image is an image formed by the LBP values of each pixel point in the second suspected license plate area obtained by using the LBP algorithm. The LBP image has more prominent edge features, so more prominent edge information can be obtained. The edge detection algorithm in the embodiment of the present invention adopts the canny edge detection algorithm, which is a well-known technical means for those skilled in the art and will not be elaborated here.

[0050] Preferably, in an embodiment of the present invention, considering that the characters of a real license plate have fixed positions, when obtaining the rationality of character positions, it can be compared with the position information on the real license plate template, and then the rationality of character positions can be obtained, which specifically includes:

[0051] The vector between the center point of the suspected character area and the center point of the second suspected license plate area is used as the position vector of each suspected character area; the differences between the position vectors of each suspected character area and the position vectors of the corresponding characters on the standard license plate are statistically obtained to obtain a representative difference. The representative difference is negatively correlated and normalized to obtain the rationality of character positions. In the embodiment of the present invention, the difference between the position vectors is the cosine distance between the two vectors. The minimum cosine distance among all suspected character areas is statistically obtained as the representative difference. That is, the smaller the representative difference, the more similar the position of the suspected character area is to the character position on the standard license plate, and the more likely the second suspected license plate area is a real license plate, and the greater the rationality of character positions.

[0052] It should be noted that methods such as normalization and negative correlation normalization in the embodiments of the present invention are all well-known technical means to those skilled in the art. In the embodiments of the present invention, a fixed method can be used to achieve normalization or negative correlation mapping in different steps, and details are not described herein.

[0053] Preferably, in the embodiments of the present invention, the method for obtaining the rationality of the character shape includes:

[0054] For any suspected character region, since the closed edge may be the edge of the character and the edge of the character has a certain trend, in order to analyze the font shape of the suspected character region, the closed edge needs to be divided into multiple independent edges. In the embodiments of the present invention, the corner points of the closed edge can be obtained through a corner point detection algorithm, and multiple independent edges can be obtained with the corner points as the segmentation points.

[0055] Perform linear and circular fitting on the independent edges to obtain the fitting determination coefficients under the two fitting processes, as well as the mean square error between the fitting values and the true values. The fitting determination coefficient is a parameter generated during the fitting process, and the larger the coefficient, the better the current fitting effect; the mean square error is the fitting deviation, representing the distance difference between the true value and the fitting value. The larger the mean square error, the less likely the independent edge is a straight line or a smooth circular curve. It should be noted that both linear fitting and circular fitting in the embodiments of the present invention can be performed using the least squares method, and the specific fitting method is well-known to those skilled in the art and will not be elaborated herein.

[0056] For each fitting process, obtain the fitting effect based on the fitting determination coefficient and the mean square error. Since there are two fitting processes for each independent edge, select the maximum fitting effect of the two fitting processes as the edge shape rationality of the independent edge.

[0057] Further, count all suspected character regions and independent edges, and use the average value of the edge shape rationality of all independent edges as the initial character shape rationality of the suspected character region. Normalize the average value of the initial character shape rationality of all suspected character regions in the second suspected license plate region to obtain the character shape rationality.

[0058] Preferably, in the embodiments of the present invention, further obtain the overall character rationality of the second suspected license plate region based on the character position rationality and the character shape rationality. If the overall character rationality is greater than the preset rationality threshold, then regard the second suspected license plate region as the real license plate region. Among them, the overall character rationality can be normalized, and the rationality threshold is set to 0.6 accordingly.

[0059] In the embodiments of the present invention, the overall character rationality is the average value of the character position rationality and the character shape rationality.

[0060] Step S4: Predict the moving trajectories of other vehicles based on the real license plate region, and adjust the parking strategy of the target vehicle according to the moving trajectories of other vehicles.

[0061] In the embodiment of the present invention, after identifying the license plates of other vehicles, the moving trajectories of other vehicles can be predicted, mainly by using the data stored in the parking lot database, such as the vehicle entry time, historical behaviors, etc. For example, if it is detected that another vehicle stays near the target parking space of the target vehicle for a short time, the system predicts that the vehicle may drive away, and then the target vehicle can dynamically adjust its path to detour or wait. When it is identified that an oncoming vehicle belongs to the vehicles that frequently occupy lanes in the database, an avoidance strategy can be preferentially adopted to reduce the risk of congestion, where the vehicles that frequently occupy lanes are work vehicles such as logistics vehicles. If it is detected that the target parking space of the target vehicle is illegally occupied by another vehicle, an abnormal message can be sent to the parking lot and a spare parking space can be searched.

[0062] In summary, the embodiment of the present invention obtains the information saliency by using the gray value distribution in the first suspected license plate region, and performs screening to obtain the real license plate region and the second suspected license plate region. The real license plate in the second suspected license plate region is screened out according to the rationality of the character positions and the rationality of the character shapes of the suspected character regions in the second suspected license plate region. The moving trajectories of other vehicles in the environment of the target vehicle are predicted based on the real license plate, and then the parking strategy of the target vehicle is adjusted. The present invention accurately identifies the license plates of other vehicles in the environment to obtain accurate license plate information to assist the parking strategy of the target vehicle.

[0063] It should be noted that: the above sequence of the embodiments of the present invention is only for description, and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0064] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A vehicle autonomous parking navigation method based on panoramic perception, characterized in that, The method includes: Obtaining an environmental video frame captured by a panoramic image acquisition device of a target vehicle, and obtaining a first suspected license plate area of other vehicles in the environmental video frame; Obtaining a grayscale histogram of the first suspected license plate area, and obtaining an information saliency according to the position distribution of peaks in the grayscale histogram; Dividing the first suspected license plate area into a real license plate area and a second suspected license plate area according to the information saliency; Obtaining a suspected character area composed of closed edges in the second suspected license plate area; Obtaining the character position rationality of the second suspected license plate area according to the position distribution uniformity of the suspected character area in the second suspected license plate area; Performing linear fitting and circular fitting on the closed edges of the suspected character area, and obtaining the character shape rationality of the second suspected license plate area according to the fitting deviation; Screening out the real license plate area in the second suspected license plate area according to the character position rationality and the character shape rationality; Predicting the movement trajectory of other vehicles according to the real license plate area, and adjusting the parking strategy of the target vehicle according to the movement trajectory of other vehicles; The method for obtaining the information saliency includes: Selecting the first two largest peaks in the grayscale histogram as representative peaks according to the peak magnitudes; Obtaining the grayscale difference of peak points between the two representative peaks; Obtaining a grayscale segmentation threshold of the first suspected license plate area by using the Otsu threshold algorithm, the grayscale segmentation threshold divides the grayscale histogram into two parts, and taking the grayscale standard deviations of the two parts as representative standard deviations; Obtaining the grayscale distribution concentration according to the magnitudes of the representative standard deviations; Obtaining the information saliency according to the grayscale difference of peak points and the grayscale distribution concentration; If the information saliency of the first suspected license plate area is greater than a preset saliency threshold, it is divided into a real license plate area, otherwise it is a second suspected license plate area.

2. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that The method for obtaining the closed edges includes: Obtaining an LBP image of the second suspected license plate area, performing edge detection on the LBP image, and obtaining the closed edges.

3. The vehicle autonomous parking navigation method based on panoramic perception according to claim 1, wherein, The method for obtaining the character position rationality includes: Taking the vector between the center point of the suspected character area and the center point of the second suspected license plate area as the position vector of each suspected character area; Counting the differences between the position vectors of each suspected character area and the position vectors of the corresponding characters of the standard license plate, obtaining a representative difference, performing negative correlation mapping and normalization on the representative difference, and obtaining the character position rationality.

4. A vehicle autonomous parking navigation method based on panoramic perception according to claim 1, wherein The method for obtaining the character shape rationality includes: For any suspected character area, dividing the closed edge into multiple independent edges, performing linear and circular fitting on the independent edges, obtaining the fitting determination coefficients under the two fitting processes, and the mean square error of the fitting values and the true values; For each fitting process, obtaining the fitting effect according to the fitting determination coefficient and the mean square error; Selecting the maximum fitting effect of the two fitting processes as the edge shape rationality of the independent edge; Taking the average value of the edge shape rationalities of all independent edges as the initial character shape rationality of the suspected character area; Normalize the average of the initial character shape rationalities of all suspected character regions in the second suspected license plate region to obtain the character shape rationality.

5. A vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that Obtain the overall character rationality of the second suspected license plate region based on the character position rationality and the character shape rationality. If the overall character rationality is greater than a preset rationality threshold, then use the second suspected license plate region as the true license plate region.

6. The vehicle autonomous parking navigation method based on panoramic perception according to claim 5, wherein, The overall character rationality is the average of the character position rationality and the character shape rationality.

7. A vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that, The obtaining of the information saliency according to the peak point gray level difference and the gray level distribution concentration includes: The peak point gray level difference is the absolute value of the difference between peak points between two representative wave peaks; use the product of the peak point gray level difference and the gray level distribution concentration as the information saliency.

8. A vehicle autonomous parking navigation method based on panoramic perception according to claim 1, characterized in that The method for obtaining the gray level distribution concentration includes: Perform a negative correlation mapping and normalization on the average between the representative standard deviations to obtain the gray level distribution concentration.

Images