Calculation method and device for linear parking space recognition precision, and storage medium
By conducting deep learning model training and testing on the real-life images of parking spaces in different scenarios, the accuracy of parking space corner points is calculated, which solves the problem that the existing technology cannot directly calculate the accuracy of parking space corner points, and improves the accuracy of line parking space recognition and parking effect.
Patent Information
- Application Number
- CN202411272179.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art cannot directly calculate the accuracy index of the parking space corner point, resulting in poor line parking space identification accuracy and large parking deviation.
By collecting real-life images of parking spaces in different scenarios, training and testing the deep learning model, and using predicted corner point positions and marked corner point positions to calculate corner point recognition accuracy, and then evaluating parking space recognition accuracy.
It is realized that without excessive modification and adaptation of the deep learning model, the accuracy of the parking space corner point is calculated, the model training results are directly evaluated, the recognition accuracy of the parking space recognition scene is clarified, and the parking effect is improved.
Smart Images
Figure CN119445527A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automatic parking, and in particular to a method, device and storage medium for calculating linear parking space recognition accuracy. Background Art
[0002] At present, the perception part of the automatic parking solution includes visual perception and spatial perception. Among them, visual perception generally uses deep learning methods to identify vertical line parking spaces, horizontal line parking spaces, and diagonal line parking spaces. The accuracy of line parking spaces recognized by visual perception will directly affect the final parking posture. Severe accuracy deviation may even cause the final parking posture to cross the line. Among them, line parking spaces refer to parking spaces formed by parking space marking lines on the road surface, which can be recognized by visual perception.
[0003] Visual perception usually uses four fisheye cameras installed on the front, back, left and right sides of the vehicle body. The images taken by the fisheye cameras will be processed to generate a spliced bird's-eye view. Visual detection of linear parking spaces is carried out on the bird's-eye view.
[0004] Most of the existing visual detection solutions based on the YOLO series of deep learning use the detection box method, using the bounding_box box (bounding box) of the target detection to frame the parking space, so that the learned parking space recognition rate is guaranteed to a certain extent. When using the YOLO series of solutions to identify parking spaces, the parking space type recognition rate can be measured based on evaluation indicators such as precision rate and recall rate. There is a lack of indicators to directly evaluate the accuracy of parking space corner points, so the parking space accuracy cannot be accurately measured, which directly affects the final parking effect. There are generally two ways to calculate the accuracy of parking space corner points:
[0005] 1. Manual measurement. Due to the large number of parking scenarios, various factors such as site, weather, and scenario are involved. This measurement method requires a lot of manpower to count. And this method cannot measure the recognition accuracy of running vehicles. And due to the measurement tools and human operation factors, this method has certain measurement errors.
[0006] Second, transplant the OKS (Object Key point Similarity) indicator into the deep learning model. There is a certain amount of adaptation work for the deep learning model. And this indicator is a statistical ratio of the overall key point similarity. However, this ratio cannot intuitively give the specific value of the current corner point recognition accuracy.
[0007] Therefore, the limitation of the existing technology is that under the premise that the parking space type recognition rate is guaranteed to a certain extent, it is impossible to directly calculate the accuracy index of the parking space corner point, and thus it is impossible to improve the line parking space recognition accuracy. Summary of the invention
[0008] The present invention provides a method, device and storage medium for calculating the accuracy of linear parking space recognition, which solves the technical problem that the existing linear parking space recognition scheme cannot directly calculate the accuracy index of the parking space corner point, resulting in poor recognition accuracy and large parking deviation.
[0009] In order to solve the above technical problems, the present invention provides a method for calculating the line parking space recognition accuracy, comprising:
[0010] Collecting real-life parking space images in different scenarios, annotating the real-life parking space images, and dividing the annotated real-life parking space images into a training set and a test set;
[0011] Training the deep learning model based on the training set and the test set;
[0012] Input the test set into the trained deep learning model, output the predicted corner point positions, and obtain the labeled corner point positions;
[0013] Calculating corner point recognition accuracy based on the predicted corner point positions and the marked corner point positions;
[0014] The parking space recognition accuracy is evaluated according to the corner point recognition accuracy and the parking space type recognition accuracy.
[0015] This basic solution is based on a deep learning model to identify real-life images of parking spaces in different scenarios. Through training with a large amount of data, it can automatically learn the features in the image, thereby covering parking space recognition scenarios in a variety of different scenarios. The corner point recognition accuracy is calculated by comparing the predicted corner point positions and the marked corner point positions, and the corner point recognition is added to the evaluation index of parking space recognition. Without making too many modifications and adaptations to the deep learning model, the parking space corner point accuracy can also be calculated. On the one hand, the training results of the model can be directly evaluated, and on the other hand, the recognition accuracy of the current parking space recognition scenario can be clarified to better serve the regulatory control algorithm, thereby ensuring the parking effect of linear parking spaces.
[0016] In a further embodiment, the collecting of real-life images of parking spaces in different scenarios includes:
[0017] Based on different parking scenarios, the real-life parking images captured by four fisheye cameras on the front, back, left and right sides of the vehicle are collected;
[0018] Based on the preset detection range of the bird's-eye view, the real-scene parking space images are stitched to obtain the original stitched bird's-eye view;
[0019] The original spliced bird's-eye view image is identified to perform scene classification, and an image collection of real-life parking space images in different scenes is obtained.
[0020] This solution collects different parking scenarios, trains and tests the deep learning model, and can improve the model's recognition accuracy for different scenarios, thereby expanding the coverage of application scenarios.
[0021] In a further embodiment, after acquiring the real-life image of the parking space, the method further includes:
[0022] The preset detection range of the bird's-eye view and the image pixels of the real-life parking space image are obtained, and the actual distance represented by each pixel is calculated. The calculation formula of the actual distance is as follows:
[0023]
[0024] In the formula, the width and height of the detection range are W cm , H cm , the width and height of the image pixel are W pix , H pix .
[0025] In a further embodiment, the training of the deep learning model based on the training set and the test set comprises:
[0026] Based on scene division, the real-life images of parking spaces in different scenes in the training set are input into a deep learning model for training;
[0027] Based on scene division, the real-life images of parking spaces in different scenes in the test set are input into the deep learning model to obtain prediction results, and the parking space recall rate and recognition accuracy are determined according to the prediction results;
[0028] Determine whether the parking space recall rate and recognition accuracy meet the qualified indicators. If so, proceed to the next step to perform corner point recognition accuracy calculation. If not, re-acquire the training set to train the deep learning model.
[0029] After model training, this solution adds a test set to obtain the prediction results, and then calculates the corresponding parking space recall rate and recognition accuracy to determine whether the training effect is qualified. If not, the model is retrained to automatically optimize the model, improve the generalization ability of the model, and ensure the performance and reliability of the model.
[0030] In a further embodiment, the step of inputting the test set into a trained deep learning model, outputting predicted corner point positions, and obtaining labeled corner point positions comprises:
[0031] Establishing an image coordinate system based on image pixels of the parking space real scene image;
[0032] Acquire parking space annotation data of each parking space real scene image from the test set, acquire corner point coordinate data of four corner points of the parking space from the parking space annotation data, and obtain the position of the annotated corner point;
[0033] The real-life image of each parking space is input into the trained deep learning model to perform parking space recognition and output the corner point coordinate data of the four corner points of the parking space to obtain the predicted corner point position.
[0034] In a further embodiment, the calculating the corner point recognition accuracy according to the predicted corner point positions and the marked corner point positions includes:
[0035] Get the one-to-one correspondence between the marked corner point positions and the predicted corner point positions, perform Euclidean distance on each corner point, and calculate the predicted corner point difference;
[0036] Get all the corner point prediction differences generated in the test set and calculate the average error:
[0037] The corner point recognition accuracy is calculated based on the actual distance represented by each pixel and the average error.
[0038] Based on the image coordinate system, this scheme digitizes the spatial positions of the marked corner point positions and the predicted corner point positions, and then obtains the corner point prediction difference by taking the Euclidean distance for each corner point to determine the offset distance between the predicted position and the calibrated position. Finally, the error average of the corner point prediction difference is statistically calculated to reduce the impact of abnormal data fluctuations, which can better represent the overall situation of the data and improve the accuracy of corner point recognition.
[0039] In a further embodiment, the calculating the corner point recognition accuracy according to the predicted corner point positions and the marked corner point positions includes:
[0040] Obtain the one-to-one correspondence between the marked corner point positions and the predicted corner point positions, and calculate the corner point prediction difference;
[0041] Get all the corner point prediction differences generated by each scene in the test set, and calculate the average error:
[0042] The actual distance and error average of each pixel in the same scene are obtained, and then the recognition accuracy of all corner points in different scenes is calculated.
[0043] In a further implementation scheme, the parking space scenes include one or more of different parking space type scenes, different weather scenes, and different parking stage scenes. The parking space type scenes include vertical parking spaces, horizontal parking spaces, and diagonal parking spaces; the weather scenes include sunny days, rainy days, and nights; and the parking stage scenes include a parking space search stage and a parking stage.
[0044] This solution subdivides different parking scenarios for the regulation and control algorithm to perform refined control, and adopts different strategies for different parking spaces and different ranges, thereby specifically improving the corner point accuracy of the model.
[0045] The present invention provides a device, comprising:
[0046] one or more processors;
[0047] at least one visual sensor, configured to generate a real-view image of a parking space related to the parking space, and send the real-view image of the parking space to a processor;
[0048] A storage device is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the method for calculating the line parking space recognition accuracy.
[0049] The present invention also provides a storage medium on which a computer program is stored, wherein the computer program is used to implement the multi-dimensional pixel fusion method for environmental perception as described above. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM). BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 It is a workflow diagram of a method for calculating line parking space recognition accuracy provided by an embodiment of the present invention;
[0051] Figure 2 is a schematic diagram of parking space marking provided by an embodiment of the present invention;
[0052] Figure 3 It is a schematic diagram of marking corner point positions and predicting corner point positions provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0053] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and cannot be understood as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.
[0054] Example 1
[0055] The present invention provides a method for calculating the line parking space recognition accuracy, such as Figure 1 , Figure 2 , Figure 3 As shown, in this embodiment, it includes:
[0056] S1. Collect real-life images of parking spaces in different scenarios, including:
[0057] Based on different parking scenarios, the real-life parking images captured by four fisheye cameras on the front, back, left and right sides of the vehicle are collected;
[0058] Based on the preset detection range of the bird's-eye view, the real-life parking space images are stitched to obtain the original stitched bird's-eye view.
[0059] The original spliced bird's-eye view image is identified to perform scene classification, and an image collection of real-life parking space images in different scenes is obtained.
[0060] Among them, the collected data is classified according to different scenarios, and tens of thousands of parking space data are obtained.
[0061] This solution collects different parking scenarios, trains and tests the deep learning model, and can improve the model's recognition accuracy for different scenarios, thereby expanding the coverage of application scenarios.
[0062] After obtaining the real-life image of the parking space, the following steps are also included:
[0063] S2. Obtain the preset detection range of the bird's-eye view and the image pixels of the real-view parking space image, and calculate the actual distance represented by each pixel. The calculation formula of the actual distance is as follows:
[0064]
[0065] In the formula, the width and height of the detection range are W cm , H cm , the width and height of the image pixel are W pix , H pix .
[0066] S3, performing data enhancement on the real-life parking space image, and marking the parking space on the real-life parking space image; see Figure 2 , mark the parking space lines and parking space corner points, strictly control the marking quality during the marking process, and control the marked point error at the pixel level.
[0067] S4, dividing the annotated parking space real scene images into a training set and a test set;
[0068] Specifically, the real-life parking space images are divided according to different parking scenarios, and then classified into training sets and test sets in equal proportions. The training set is used to train the parking space model, and the test set is used to verify the model's indicators. The specific ratio coefficient can be set according to actual needs, such as 7:3.
[0069] S5. Training the deep learning model based on the training set and the test set, including:
[0070] Based on scene division, the real-life images of parking spaces in different scenes in the training set are input into a deep learning model for training;
[0071] Based on scene division, the real-life images of parking spaces in different scenes in the test set are input into the deep learning model to obtain prediction results, and the parking space recall rate and recognition accuracy are determined according to the prediction results;
[0072] Determine whether the parking space recall rate and recognition accuracy meet the qualified indicators. If so, proceed to the next step to perform corner point recognition accuracy calculation. If not, re-acquire the training set to train the deep learning model.
[0073] After model training, this solution adds a test set to obtain the prediction results, and then calculates the corresponding parking space recall rate and recognition accuracy to determine whether the training effect is qualified. If not, the model is retrained to automatically optimize the model, improve the generalization ability of the model, and ensure the performance and reliability of the model.
[0074] S6. Input the test set into the trained deep learning model, output the predicted corner point positions, and obtain the labeled corner point positions, including:
[0075] Establishing an image coordinate system based on image pixels of the parking space real scene image;
[0076] Obtain the parking space annotation data of each parking space real scene image from the test set, obtain the corner point coordinate data of the four corner points of the parking space from the parking space annotation data, and obtain the position of the annotated corner point, such as Figure 3 The parking space corner points of each parking space marked in Figure 3 The white dots near A, B, C, and D in the figure are recorded as (A, B, C, D);
[0077] Input the real-life image of each parking space into the trained deep learning model, perform parking space recognition and output the corner point coordinate data of the four corner points of the parking space to obtain the predicted corner point position, such as Figure 3 The parking space corner points of each parking space marked in Figure 3 Middle near A ′ , B ′ , C ′ , D ′ The white dots), recorded as (A ′ ,B ′ ,C ′ ,D ′ ).
[0078] S7, calculating the corner point recognition accuracy according to the predicted corner point positions and the marked corner point positions;
[0079] Specifically, taking the corner point recognition accuracy of the calculation model as an example, it includes:
[0080] S71A, obtaining one-to-one corresponding marked corner point positions and predicted corner point positions, performing Euclidean distance on each corner point, and calculating the predicted corner point difference;
[0081] At the pixel level, calculate the Euclidean distance for each corner point. Take point A as an example:
[0082]
[0083] Correspondingly, each parking space real scene image contains the corner point prediction difference d of 4 corner points A ,d B ,d C ,d D .
[0084] S72A, obtaining all the corner point prediction differences generated in the test set, and calculating the average error value, the calculation formula is as follows:
[0085]
[0086] Similarly, the maximum error err can be obtained max and the minimum error err min , according to the maximum error err max and / or minimum error err min Set a fluctuation threshold. If it exceeds the fluctuation threshold, retrain the model and recalculate the error average.
[0087] S73A, calculating the corner point recognition accuracy according to the actual distance represented by each pixel and the average error, the calculation formula is as follows;
[0088] Corner point recognition accuracy = err × Rate
[0089] Based on the image coordinate system, this scheme digitizes the spatial positions of the marked corner point positions and the predicted corner point positions, and then obtains the corner point prediction difference by taking the Euclidean distance for each corner point to determine the offset distance between the predicted position and the calibrated position. Finally, the error average of the corner point prediction difference is statistically calculated to reduce the impact of abnormal data fluctuations, which can better represent the overall situation of the data and improve the accuracy of corner point recognition.
[0090] Specifically, taking the corner point recognition accuracy of the calculation model for different scenes as an example, it includes:
[0091] S71B, obtaining one-to-one corresponding marked corner point positions and predicted corner point positions, and calculating the corner point prediction difference;
[0092] S72B, obtain all the corner point prediction differences generated by each scene in the test set, and calculate the average error value:
[0093] S73B, obtain the actual distance and error average value represented by each pixel in the same scene, and then calculate the recognition accuracy of all corner points in different scenes.
[0094] In this embodiment, the parking space scene includes one or more of different parking space type scenes, different weather scenes, different parking stage scenes, and different side parking spaces. The parking space type scenes include vertical parking spaces, horizontal parking spaces, and diagonal parking spaces; the weather scenes include sunny days, rainy days, and nights; and the parking stage scenes include the parking space search stage and the parking stage. For example, the accuracy of the parking space corners of the left and right parking spaces of the vehicle and the accuracy of the parking space corners of parking spaces within different ranges (for example, within 1m, within 3m, and within 5m) are counted.
[0095] This solution subdivides different parking scenarios for the regulation and control algorithm to perform refined control, and adopts different strategies for different parking spaces and different ranges, thereby specifically improving the corner point accuracy of the model.
[0096] S8. Evaluate parking space recognition accuracy according to the corner point recognition accuracy and the parking space type recognition accuracy.
[0097] Specifically, two different evaluation indicators are set according to the corner point recognition accuracy and the parking space type recognition accuracy, and the corresponding parking space recognition accuracy level range is divided based on their actual values. When evaluating, the parking space recognition accuracy is determined by referring to the level table. Alternatively, a weight formula is used to fuse the corner point recognition accuracy and the parking space type recognition accuracy to calculate the parking space recognition accuracy containing a specific value. This embodiment is not significant.
[0098] Example 2
[0099] An embodiment of the present invention provides a device, including:
[0100] one or more processors;
[0101] at least one visual sensor, configured to generate a real-view image of a parking space related to the parking space, and send the real-view image of the parking space to a processor;
[0102] A storage device is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the method for calculating the line parking space recognition accuracy.
[0103] Example 3
[0104] The embodiment of the present invention further provides a storage medium on which a computer program is stored, and the computer program is used to implement a multi-dimensional pixel fusion method for environmental perception as described in the above embodiment 1. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).
[0105] The embodiment of the present invention recognizes real-life images of parking spaces in different scenarios based on a deep learning model. Through training with a large amount of data, it can automatically learn the features in the image, thereby covering parking space recognition scenarios in a variety of different scenarios. The corner point recognition accuracy is calculated by comparing the predicted corner point positions and the marked corner point positions, and the corner point recognition is added to the evaluation index of parking space recognition. Without making too many modifications and adaptations to the deep learning model, no additional manual workload is added, and the parking space corner point accuracy can be calculated. On the one hand, the training results of the model can be directly evaluated, and on the other hand, the recognition accuracy of the current parking space recognition scenario can be clarified to better serve the regulation and control algorithm, thereby ensuring the parking effect of the linear parking space.
[0106] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.
Claims
1. A method for calculating the accuracy of line parking space recognition, characterized in that: include: Collecting real-life parking space images in different scenarios, annotating the real-life parking space images, and dividing the annotated real-life parking space images into a training set and a test set; Training the deep learning model based on the training set and the test set; Input the test set into the trained deep learning model, output the predicted corner point positions, and obtain the labeled corner point positions; Calculating corner point recognition accuracy based on the predicted corner point positions and the marked corner point positions; The parking space recognition accuracy is evaluated according to the corner point recognition accuracy and the parking space type recognition accuracy.
2. The method for calculating the line parking space recognition accuracy according to claim 1, characterized in that: The acquisition of real-life parking space images in different scenarios includes: Based on different parking scenarios, the real-life parking images captured by four fisheye cameras on the front, back, left and right sides of the vehicle are collected; Based on the preset detection range of the bird's-eye view, the real-scene parking space images are stitched to obtain the original stitched bird's-eye view; The original spliced bird's-eye view image is identified to perform scene classification, and an image collection of real-life parking space images in different scenes is obtained.
3. The method for calculating the line parking space recognition accuracy according to claim 2, characterized in that: After obtaining the real-life image of the parking space, the following steps are also included: The preset detection range of the bird's-eye view and the image pixels of the real-life parking space image are obtained, and the actual distance represented by each pixel is calculated. The calculation formula of the actual distance is as follows: In the formula, the width and height of the detection range are W cm , H cm , the width and height of the image pixel are W pix , H pix .
4. The method for calculating the line parking space recognition accuracy according to claim 1, characterized in that: The training of the deep learning model based on the training set and the test set includes: Based on scene division, the real-life images of parking spaces in different scenes in the training set are input into a deep learning model for training; Based on scene division, the real-life images of parking spaces in different scenes in the test set are input into the deep learning model to obtain prediction results, and the parking space recall rate and recognition accuracy are determined according to the prediction results; Determine whether the parking space recall rate and recognition accuracy meet the qualified indicators. If so, proceed to the next step to perform corner point recognition accuracy calculation. If not, re-acquire the training set to train the deep learning model.
5. The method for calculating the line parking space recognition accuracy according to claim 4, characterized in that: The step of inputting the test set into the trained deep learning model, outputting the predicted corner point positions, and obtaining the labeled corner point positions includes: Establishing an image coordinate system based on image pixels of the parking space real scene image; Acquire parking space annotation data of each parking space real scene image from the test set, acquire corner point coordinate data of four corner points of the parking space from the parking space annotation data, and obtain the position of the annotated corner point; The real-life image of each parking space is input into the trained deep learning model to perform parking space recognition and output the corner point coordinate data of the four corner points of the parking space to obtain the predicted corner point position.
6. The method for calculating the line parking space recognition accuracy according to claim 3, characterized in that: The calculating the corner point recognition accuracy according to the predicted corner point positions and the marked corner point positions includes: Obtain the one-to-one correspondence between the marked corner point positions and the predicted corner point positions, and calculate the corner point prediction difference; Get all the corner point prediction differences generated in the test set and calculate the average error: The corner point recognition accuracy is calculated based on the actual distance represented by each pixel and the average error.
7. The method for calculating the line parking space recognition accuracy according to claim 1, characterized in that: The calculating the corner point recognition accuracy according to the predicted corner point positions and the marked corner point positions includes: Get the one-to-one correspondence between the marked corner point positions and the predicted corner point positions, perform Euclidean distance on each corner point, and calculate the predicted corner point difference; Get all the corner point prediction differences generated by each scene in the test set, and calculate the average error: The actual distance and error average of each pixel in the same scene are obtained, and then the recognition accuracy of all corner points in different scenes is calculated.
8. The method for calculating the line parking space recognition accuracy according to claim 2, characterized in that: The parking space scenes include one or more of different parking space type scenes, different weather scenes, and different parking stage scenes. The parking space type scenes include vertical parking spaces, horizontal parking spaces, and diagonal parking spaces; the weather scenes include sunny days, rainy days, and nights; and the parking stage scenes include a parking space search stage and a parking stage.
9. A device, characterized in that: include: one or more processors; at least one visual sensor, configured to generate a real-view image of a parking space related to the parking space, and send the real-view image of the parking space to a processor; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the method for calculating the line parking space recognition accuracy described in any one of claims 1-8.
10. A storage medium having a computer program stored thereon, characterized in that: The computer program is used to implement a multi-dimensional pixel fusion method for environmental perception as described in any one of claims 1-8.