Roadside parking space number recognition method and device

Abstract

The present application discloses a roadside parking space number recognition method and device. The method comprises: training a target detection model according to a plurality of roadside parking space number images to obtain a trained target detection model; detecting the roadside parking space number image to be detected according to the trained target detection model to obtain character detection frame coordinate information and character category information of a plurality of parking space number characters; obtaining the center point coordinates of each character detection frame according to the character detection frame coordinate information of each parking space number character, performing straight line fitting according to each center point coordinate, and obtaining a fitting straight line equation of each parking space number character; obtaining the grouping distance from each center point coordinate to each fitting straight line equation according to each center point coordinate and each fitting straight line equation, grouping each center point coordinate, and obtaining a plurality of parking space number groups; sorting according to the character category information and the center point coordinates of each parking space number character to obtain a roadside parking space number sequence.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for identifying roadside parking space numbers. Background Art

[0002] With the development of "Internet + parking", smart parking, by mastering parking big data, relying on its high frequency and rigid demand advantages, realizes intelligent parking, visualized management and efficient operation, and can provide car owners with valuable experience such as electronic payment, parking space query, parking space reservation, parking space navigation, and surrounding information push. Smart parking created by using Internet technology, big data and cloud computing has many advantages and has become a feasible solution to quickly solve the problems of difficult and chaotic parking, bringing changes to the entire city's parking and travel.

[0003] In the process of solving the problems of difficult and chaotic parking, the recognition of roadside parking space numbers plays a key role. The traditional recognition method detects the key points of the image to be tested based on the key point detection and positioning model, corrects the image according to the key points, obtains the corrected image, and recognizes the corrected image based on the deep learning algorithm. However, the detection error of the key points is high, which makes the recognition accuracy of the traditional recognition method low, which is not conducive to the management of roadside parking fees. Summary of the invention

[0004] The purpose of this application is to solve the technical problem that the traditional recognition method has low recognition accuracy and is not conducive to roadside parking fee management. To achieve the above purpose, this application provides a

[0005] The present application provides a roadside parking space number identification method, comprising:

[0006] Acquire a plurality of roadside parking space number images, each of the roadside parking space number images including real character detection frame coordinate information and real character category information of a plurality of parking space number characters;

[0007] Training and optimizing the target detection model according to the plurality of roadside parking space number images to obtain a trained target detection model;

[0008] According to the trained target detection model, the roadside parking space number image to be detected is detected to obtain character detection frame coordinate information and character category information of multiple parking space number characters;

[0009] According to the coordinate information of the character detection frame of each berth number character, the center point coordinates of each character detection frame are obtained, and a straight line fitting is performed according to each center point coordinate to obtain a fitting straight line equation of each berth number character;

[0010] According to each of the center point coordinates and each of the fitted straight line equations, a group distance from each of the center point coordinates to each of the fitted straight line equations is obtained, and each of the center point coordinates is grouped according to each of the group distances to obtain a plurality of berth number groups;

[0011] A roadside parking space number sequence is obtained by sorting the character category information and the center point coordinates of each parking space number character in each parking space number group.

[0012] In one embodiment, after obtaining the center point coordinates of each character detection frame according to the coordinate information of the character detection frame of each berth number character, and performing straight line fitting according to each center point coordinate to obtain the fitted straight line equation of each berth number character, obtaining the grouping distance between each center point coordinate and each fitted straight line equation according to each center point coordinate and each fitted straight line equation, and grouping each center point coordinate according to each grouping distance, before obtaining multiple berth number groups, the method further includes:

[0013] Calculating a slope difference according to the slopes of any two of the fitted straight line equations, and determining whether the slope difference is less than a first threshold;

[0014] If the slope difference is less than the first threshold, calculating the intercept difference according to the intercepts of the two fitted straight line equations, and determining whether the intercept difference is less than a second threshold;

[0015] If the intercept difference is less than the second threshold, then calculating the average slope and the average intercept of the two fitted straight line equations;

[0016] A new fitting straight line equation is obtained according to the slope average value and the intercept average value, and the two fitting straight line equations are replaced with the new fitting straight line equation to obtain the new fitting straight line equation corresponding to the berth number character.

[0017] In one embodiment, the method further comprises:

[0018] If the slope difference is greater than the first threshold, the two fitted straight line equations are two independent straight lines;

[0019] If the intercept difference is greater than the second threshold, the two fitted straight line equations are two independent straight lines;

[0020] The first threshold ranges from 3 to 7, and the second threshold ranges from 17 to 24.

[0021] In one embodiment, grouping each of the center point coordinates according to each of the grouping distances to obtain a plurality of berth number groups includes:

[0022] Determine whether each of the grouping distances is less than a third threshold;

[0023] If the grouping distance is less than the third threshold, dividing the center point coordinates corresponding to the grouping distance into the same group;

[0024] Forming each berth number group according to the berth number characters corresponding to each center point coordinate in the same group;

[0025] The third threshold value ranges from 75 to 85.

[0026] In one embodiment, the obtaining of a roadside parking space number sequence by sorting the character category information and the center point coordinates of each parking space number character in each parking space number group includes:

[0027] The roadside parking space number sequence is obtained by arranging the character category information of each parking space number character in each parking space number group and the x-axis coordinate of the center point coordinate in ascending order.

[0028] In one embodiment, the training and optimizing of the target detection model according to the plurality of roadside parking space number images to obtain the trained target detection model includes:

[0029] Inputting the plurality of roadside parking space number images into a target detection model, and outputting predicted character detection frame coordinate information and predicted character category information of each parking space number character;

[0030] A loss function is constructed based on the real character detection box coordinate information, the real character category information, the predicted character position information and the predicted character category information, and the target detection model is trained and optimized based on the loss function to obtain a trained target detection model.

[0031] In one embodiment, the present application provides a roadside parking space number recognition device, comprising:

[0032] A data acquisition module, used to acquire a plurality of roadside parking space number images, each of which includes real character detection frame coordinate information and real character category information of a plurality of parking space number characters;

[0033] A model generation module, used to train and optimize the target detection model according to the plurality of roadside parking space number images to obtain a trained target detection model;

[0034] A character information acquisition module, used to detect the roadside parking space number image to be detected according to the trained target detection model, and obtain character detection frame coordinate information and character category information of multiple parking space number characters;

[0035] A fitting straight line equation acquisition module, used to obtain the center point coordinates of each character detection frame according to the character detection frame coordinate information of each berth number character, and perform straight line fitting according to each center point coordinate to obtain the fitting straight line equation of each berth number character;

[0036] A grouping module, used for obtaining a group distance from each center point coordinate to each of the fitted straight line equations according to each of the center point coordinates and each of the fitted straight line equations, and grouping each of the center point coordinates according to each of the grouping distances to obtain a plurality of berth number groups;

[0037] The roadside parking space number sequence generating module is used to obtain a roadside parking space number sequence by sorting the character category information and the center point coordinates of each of the parking space number characters in each of the parking space number groups.

[0038] In one embodiment, the roadside parking space number recognition device further includes:

[0039] A first judgment module, used for calculating a slope difference according to the slopes of any two of the fitted straight line equations, and judging whether the slope difference is less than a first threshold;

[0040] A second judgment module, configured to calculate an intercept difference according to the intercepts of the two fitted straight line equations if the slope difference is less than the first threshold, and judge whether the intercept difference is less than a second threshold;

[0041] an average value acquisition module, used for calculating the slope average value and the intercept average value of the two fitted straight line equations if the intercept difference is less than the second threshold value;

[0042] The new fitting straight line equation acquisition module is used to obtain the new fitting straight line equation according to the slope average value and the intercept average value, and replace the two fitting straight line equations with the new fitting straight line equation to obtain the new fitting straight line equation corresponding to the berth number character.

[0043] In one embodiment, the roadside parking space number recognition device further includes:

[0044] A third judgment module, configured to determine that if the slope difference is greater than the first threshold, the two fitted straight line equations are two independent straight lines;

[0045] A fourth judgment module, configured to determine that the two fitted straight line equations are two independent straight lines if the intercept difference is greater than the second threshold;

[0046] The first threshold ranges from 3 to 7, and the second threshold ranges from 17 to 24.

[0047] In one embodiment, the grouping module includes:

[0048] A fifth judging module, used for judging whether each of the grouping distances is less than a third threshold;

[0049] A center point coordinate division module, configured to divide the center point coordinates corresponding to the grouping distance into the same group if the grouping distance is less than the third threshold;

[0050] A berth number group forming module, used for forming each berth number group according to the berth number characters corresponding to each center point coordinate in the same group;

[0051] The third threshold value ranges from 75 to 85.

[0052] In one embodiment, the roadside parking space number sequence generation module includes:

[0053] The sorting module is used to arrange the character category information of each of the berth number characters in each of the berth number groups and the x-axis coordinate of the center point coordinate in ascending order to obtain the roadside berth number sequence.

[0054] In one embodiment, the model generation module includes:

[0055] A target detection model building module, used for inputting the plurality of roadside parking space number images into the target detection model, and outputting predicted character detection frame coordinate information and predicted character category information of each parking space number character;

[0056] A training optimization module is used to construct a loss function based on the real character detection box coordinate information, the real character category information, the predicted character position information and the predicted character category information, and to train and optimize the target detection model based on the loss function to obtain a trained target detection model.

[0057] In the above-mentioned roadside parking space number recognition method and device, target detection model training and optimization are performed based on multiple roadside parking space number images to obtain a trained target detection model. The roadside parking space number image to be tested is detected according to the trained target detection model to obtain the corresponding character detection frame coordinate information and character category information of multiple parking space number characters. Further, the center point coordinates, fitting straight line equation and grouping distance are obtained in sequence according to the character detection frame coordinate information. Thus, grouping is performed according to the one-to-one correspondence between the center point coordinates, the fitting straight line equation and the grouping distance, and divided into multiple parking space number groups. In each parking space number group, the corresponding roadside parking space number sequence is obtained according to the character category information and the center point coordinates of the parking space number characters. Through the roadside parking space number recognition method provided by the present application, the target detection model is trained and parameter optimized to obtain a trained target detection model, and linear fitting, parking space number grouping and parking space sequence generation are performed based on the output result of the trained target detection model to obtain the final parking space number sequence. Through the roadside parking space number recognition method, the output results of the trained target detection model can be subjected to multi-level recognition screening, which improves the recognition accuracy and enables real-time and accurate recognition, thereby improving the accuracy of roadside parking fee management using mobile patrol vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the step flow of the roadside parking space number identification method provided in this application.

[0059] Figure 2 It is a structural schematic diagram of the roadside parking space number identification device provided in this application. DETAILED DESCRIPTION

[0060] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments.

[0061] See also Figure 1 The present application provides a method for identifying a roadside parking space number, comprising:

[0062] S10, acquiring a plurality of roadside parking space number images, each roadside parking space number image including real character detection frame coordinate information and real character category information of a plurality of parking space number characters;

[0063] S20, training and optimizing the target detection model according to the plurality of roadside parking space number images to obtain a trained target detection model;

[0064] S30, detecting the roadside parking space number image to be detected according to the trained target detection model, and obtaining character detection frame coordinate information and character category information of multiple parking space number characters;

[0065] S40, obtaining the center point coordinates of each character detection frame according to the character detection frame coordinate information of each berth number character, and performing straight line fitting according to each center point coordinate to obtain the fitting straight line equation of each berth number character;

[0066] S50, according to each center point coordinate and each fitted straight line equation, obtaining a group distance from each center point coordinate to each fitted straight line equation, and grouping each center point coordinate according to each group distance to obtain a plurality of berth number groups;

[0067] S60, sorting the characters of each berth number in each berth number group according to the character category information and the center point coordinates to obtain a roadside berth number sequence.

[0068] In S10, each roadside parking space number image includes a roadside parking space number. Each roadside parking space number includes multiple parking space number characters. The roadside parking space number can be understood as a number representing the parking space location printed next to the parking space. Multiple roadside parking space number images can be obtained by collecting roadside parking space number data using the camera on the mobile inspection device. The mobile inspection device usually includes three cameras, facing the front, right, and rear sides of the inspection device's travel direction, respectively. The three cameras of the mobile inspection device are used to collect front video data in the direction of travel, right parking space video information, and rear video data in the direction of travel, respectively. The camera facing the right side of the mobile inspection device will capture a clearer and more complete roadside parking space number, collect the video data captured by the right camera, and extract the image frame containing the roadside parking space number as the roadside parking space number image.

[0069] Each roadside parking space number image includes the real character detection frame coordinate information and real character category information of multiple parking space number characters. The real character detection frame coordinate information and real character category information of each parking space number character can be obtained through data annotation. In one embodiment, all parking space number characters in each roadside parking space number image are annotated with a rectangular frame annotation tool for target detection. The character detection frame coordinate information includes the coordinate position information of the upper left corner and the lower right corner of the parking space number character. The character category information includes 11 categories including Arabic numerals 0-9 and the character "-".

[0070] In S20, multiple roadside parking space number images form a training set of the target detection model. Multiple roadside parking space number images are input into the target detection model to perform parking space number character target detection, and obtain target detection results of all parking space number characters of each roadside parking space number image. The target detection model is trained and optimized by the data training set formed by multiple roadside parking space number images to form a trained target detection model for predicting the roadside parking space number image to be tested. The target detection model can be formed by a convolutional neural network, a two-stage or a single-stage target detection method.

[0071] In S30, the roadside parking space number image to be tested can be understood as the roadside parking space number image to be tested. The roadside parking space number image to be tested is input into the trained object detection model, and the character detection frame coordinate information and character category information of each parking space number character can be correspondingly output.

[0072] In S40, the character detection frame coordinate information can represent the coordinate information of the character outline, specifically, the coordinate position information of the upper left corner and the lower right corner of the berth number character. The center point coordinates of each character detection frame can be calculated based on the character detection frame coordinate information. The center point coordinates can be expressed as (x, y). The corresponding fitting straight line equation can be fitted based on the center point coordinates. The fitting method can be a Hough transform algorithm, a least squares method, or a RANSAC algorithm. According to the Hough transform algorithm, the least squares method, or the RANSAC algorithm, a straight line is fitted for each center point coordinate to obtain the fitted two-point coordinates (x1, y1, x2, y2) representing the straight line. According to the two-point coordinates (x1, y1, x2, y2), the corresponding fitting straight line equation y=kx+b can be obtained.

[0073] In S50, the group distance can be understood as the distance from each center point coordinate to each fitted straight line equation, which can be calculated by the point-to-straight line distance calculation formula. One group distance corresponds to one center point coordinate, one fitted straight line equation, and one berth number character. By dividing the group distance, the center point coordinate can be divided, and then the fitted straight line equation can be divided, and then the berth number characters can be divided to form multiple berth number groups.

[0074] In S60, after training the target detection model with a data set of character category information annotated in a plurality of roadside parking space number images, a trained target detection model is obtained. After detecting the roadside parking space number image to be detected with the trained target detection model, character category information is obtained. The center point coordinates include X-axis coordinate information and Y-axis coordinate information. The sorting of each parking space number character can be achieved according to the X-axis coordinate information and the Y-axis coordinate information. In each parking space number group, the corresponding roadside parking space number sequence can be obtained according to the sorting of the parking space number characters.

[0075] The roadside parking space number recognition method provided by the present application performs target detection model training and optimization based on multiple roadside parking space number images to obtain a trained target detection model. According to the trained target detection model, the roadside parking space number image to be tested is detected to obtain the character detection frame coordinate information and character category information of the corresponding multiple parking space number characters. Furthermore, the center point coordinates, the fitted straight line equation and the grouping distance are obtained in sequence according to the character detection frame coordinate information. Thus, according to the one-to-one correspondence between the center point coordinates, the fitted straight line equation and the grouping distance, the groups are grouped and divided into multiple parking space number groups. In each parking space number group, according to the character category information and the center point coordinates of the parking space number characters, the corresponding roadside parking space number sequence is obtained. Through the roadside parking space number recognition method provided by the present application, the target detection model is trained and parameter optimized to obtain a trained target detection model, and based on the output result of the trained target detection model, straight line fitting, parking space number grouping and parking space sequence generation are performed to obtain the final parking space number sequence. Through the roadside parking space number recognition method, the output results of the trained target detection model can be subjected to multi-level recognition screening, which improves the recognition accuracy and enables real-time and accurate recognition, thereby improving the accuracy of roadside parking fee management using mobile patrol vehicles.

[0076] In one embodiment, after obtaining a plurality of roadside parking space number images at S10, each roadside parking space number image including real character detection frame coordinate information and real character category information of a plurality of parking space number characters, at S20, training and optimizing a target detection model according to the plurality of roadside parking space number images, before obtaining the trained target detection model, the roadside parking space number recognition method further includes:

[0077] S101, performing data enhancement on multiple roadside parking space number images.

[0078] In this embodiment, data enhancement of the multiple roadside parking space number images includes data enhancement such as rotation transformation, brightness transformation, and cropping transformation of the multiple roadside parking space number images. By performing rotation transformation on the multiple roadside parking space number images to obtain multiple roadside parking space number images with different rotation angles, the number of images in the image data set can be increased, and the tilted roadside parking space number image data in the real shooting scene can be simulated.

[0079] By performing brightness transformation on multiple roadside parking space number images at different rotation angles to obtain multiple roadside parking space number images with different brightnesses, the number of images in the image data set can be increased, and the roadside parking space number image data under different brightness conditions in the morning, noon and evening in a real roadside parking scene can be simulated.

[0080] By randomly cropping multiple roadside parking space number images with different brightness to obtain multiple roadside parking space number images with different occlusion degrees, the number of images in the image data set can be increased, and roadside parking space number image data in situations such as vehicles occluding parking space numbers in real roadside parking scenes can be simulated.

[0081] In one embodiment, S20, training and optimizing the target detection model according to a plurality of roadside parking space number images to obtain a trained target detection model includes:

[0082] S210, inputting a plurality of roadside parking space number images into a target detection model, and outputting predicted character detection frame coordinate information and predicted character category information of each parking space number character;

[0083] S220, construct a loss function based on the real character detection box coordinate information, the real character category information, the predicted character detection box coordinate information and the predicted character category information, and train and optimize the target detection model based on the loss function to obtain a trained target detection model.

[0084] In this embodiment, each roadside parking space number image includes the real character position information and real character category information of multiple parking space number characters. According to the real character detection frame coordinate information, the real character category information, the predicted character detection frame coordinate information and the predicted character category information, the target detection model is trained to obtain a stable target detection model.

[0085] In one embodiment, the target detection model includes a backbone network, a feature pyramid network, and a target detection network. The backbone network includes but is not limited to convolutional neural networks such as ResNet and VGG. The target detection network includes but is not limited to two-stage or single-stage target detection methods such as Faster-RCNN, YOLO, and SSD.

[0086] The feature pyramid network is integrated into the backbone network to extract and fuse features of multiple roadside parking space number images to obtain multiple feature fusion number images. Multiple feature fusion number images are input into the target detection network for target detection, and the predicted character detection box coordinate information and predicted character category information of each parking space number character are output. Adding the feature pyramid network structure to the backbone network structure can extract high-level semantic features from different scales and fuse the extracted features. Feature Pyramid Networks (FPN) can solve multi-scale problems.

[0087] The multiple feature fusion number images include fused features. The fused features are the features obtained by the feature pyramid network that fuse feature maps of different resolutions. By increasing the network connections of the feature pyramid network, the detection performance can be improved without increasing the model calculation amount of the convolutional neural network. The feature pyramid network can fuse the features with lower resolution but stronger semantic features and the features with higher resolution but weaker semantic features in multiple roadside parking space number images through top-down paths and lateral connections, realize the fusion of different features, and thus improve the detection accuracy.

[0088] Integrating the feature pyramid network into the backbone network can realize feature extraction and fusion, and solve the problem of different sizes of photographed numbers caused by different distances from the berths and different shooting angles when mobile inspection equipment collects data, thereby improving the recognition accuracy of the roadside berth number recognition method.

[0089] In one embodiment, each feature-fused number image is input into the classification branch and regression branch of the target detection network, and the predicted character detection frame coordinate information and predicted character category information of each berth number character are output. A position regression loss function is constructed based on the predicted character detection frame coordinate information and the real character detection frame coordinate information of each berth number character. A category classification loss function is constructed based on the predicted character category information and the real character category information of each berth number character.

[0090] In this embodiment, each berth number character can be identified through the target detection network, which is more targeted and can improve the recognition accuracy. The classification branch of the target detection network is used to output the predicted character category information of each berth number character. The regression branch of the target detection network is used to output the predicted character detection frame coordinate information of each berth number character. The loss function of the target detection model is constructed based on the position regression loss function and the category classification loss function, and the target detection model is trained and parameter optimized. The loss function of the target detection model is L det =λ1L reg +λ2L cls .

[0091] Among them, L reg represents the position regression loss function, L cls It represents the category classification loss function, λ1 and λ2 represent the weight coefficients of the model loss function, which can be set to 1 or the weight coefficient ratio can be set according to the actual application situation.

[0092] In one embodiment, the position regression loss function includes but is not limited to the use of mean absolute error loss function L1Loss, mean square error loss function L2Loss, intersection over union loss function IoU Loss and other position regression loss functions. The category classification loss function includes but is not limited to the use of Cross Entropy loss function, Focal Loss loss function and other classification loss functions. The target detection model is trained and parameter optimized through the position regression loss function and the category classification loss function.

[0093] In one embodiment, the roadside parking space number recognition method further includes removing some redundant detection frames through a non-maximum suppression (NMS) algorithm, which can further improve the recognition detection accuracy and obtain the final category and position of each parking space number area.

[0094] In one embodiment, after obtaining the center point coordinates of each character detection frame according to the character detection frame coordinate information of each berth number character at step S40, and performing straight line fitting according to each center point coordinate to obtain the fitted straight line equation of each berth number character, at step S50, obtaining the grouping distance between each center point coordinate and each fitted straight line equation according to each center point coordinate and each fitted straight line equation, and grouping each center point coordinate according to each grouping distance to obtain multiple berth number groups, the roadside berth number recognition method further includes:

[0095] S401, calculating a slope difference according to the slopes of any two fitted straight line equations, and determining whether the slope difference is less than a first threshold;

[0096] S402, if the slope difference is less than the first threshold, calculating the intercept difference according to the intercepts of the two fitted straight line equations, and determining whether the intercept difference is less than the second threshold;

[0097] S403, if the intercept difference is less than the second threshold, calculating the average slope and the average intercept of the two fitted straight line equations;

[0098] S404, obtaining a new fitted straight line equation according to the average slope and the average intercept, and replacing the two fitted straight line equations with the new fitted straight line equation to obtain a new fitted straight line equation corresponding to the berth number character.

[0099] In this embodiment, the numerical range of the first threshold and the second threshold can be limited according to actual conditions. The slopes of two fitted linear equations are arbitrarily selected from multiple fitted linear equations for difference calculation to obtain the slope difference. Each slope represents the characteristics of each fitted linear equation. By comparing the slopes of the two fitted linear equations, it can be determined whether the two fitted linear equations are independent, which can also be understood as determining whether the two fitted linear equations are the same fitted linear equation. By calculating the intercepts of the two fitted linear equations, the situation where the two fitted linear equations are parallel can be excluded, and further determining whether the two fitted linear equations are independent. When the slope difference is less than the first threshold and the intercept difference is less than the second threshold, the two fitted linear equations are determined to be a straight line. Furthermore, the slopes and intercepts of the two fitted linear equations are respectively calculated as the mean values, which are used as the slope and intercept of the new fitted linear equation, and the two fitted linear equations are replaced by the new fitted linear equation, and multiple fitted linear equations are screened. The new fitted linear equation is applied to the S50 step for group distance calculation.

[0100] By calculating and judging the slope difference and the intercept difference in this embodiment, multiple fitted straight line equations can be screened, and then two fitted straight line equations with similar distances can be screened out, and a new fitted straight line equation is obtained for replacement, so as to obtain the fitted straight line equation corresponding to each parking space number character. Through S401 to S404 in this embodiment, redundant fitted straight line equations are screened, which improves the accuracy of roadside parking space number recognition and is more conducive to roadside parking fee management.

[0101] In one embodiment, the roadside parking space number identification method further includes:

[0102] S405, if the slope difference is greater than a first threshold, the two fitted straight line equations are two independent straight lines;

[0103] S406, if the intercept difference is greater than a second threshold, the two fitted straight line equations are two independent straight lines;

[0104] The first threshold value ranges from 3 to 7, and the second threshold value ranges from 17 to 24.

[0105] In this embodiment, the two independent straight lines can be understood as two independent straight lines, not the same fitted straight line equation. If the slope difference is less than the first threshold and the intercept difference is greater than the second threshold, the two fitted straight line equations are considered to be two independent straight lines, not the same fitted straight line equation. By setting the range of the first threshold and the range of the second threshold, it can be better adapted to the size of each number character in the spray-type parking space number and the spacing between adjacent number characters, making the roadside parking space number recognition method more practical and improving the recognition accuracy.

[0106] In one embodiment, in S50, each center point coordinate is grouped according to each grouping distance to obtain multiple berth number groups, including:

[0107] S510, determining whether each group distance is less than a third threshold;

[0108] S520, if the grouping distance is less than a third threshold, dividing the center point coordinates corresponding to the grouping distance into the same group;

[0109] S530, forming each berth number group according to the berth number characters corresponding to each center point coordinate in the same group;

[0110] The third threshold value ranges from 75 to 85.

[0111] In this embodiment, the grouping distance can be understood as the distance from the center point to the fitted straight line equation, and the distance from all center point coordinates to each fitted straight line equation can be calculated. The third threshold can be limited according to actual conditions. Each grouping distance corresponds to a center point coordinate. By screening the grouping distance, the grouping of different berth number groups can be achieved, and the berth number characters can be further screened. In addition, in this embodiment, the range of the third threshold is set to 75 to 85, which can better divide the grouping distance and better group all center point coordinates. By setting the first threshold, the second threshold and the third threshold, the recognition process can be screened three times, thereby improving the recognition accuracy.

[0112] In one embodiment, S60, sorting the character category information and the center point coordinates of each berth number character in each berth number group to obtain a roadside berth number sequence includes:

[0113] S610: Arrange in ascending order the character category information of each berth number character in each berth number group and the x-axis coordinate of the center point coordinate to obtain a roadside berth number sequence.

[0114] In this embodiment, the character category information of each berth number character includes Arabic numerals 0-9 and the character "-", totaling 11 categories. According to the character category information, the specific category of each berth number character in each berth number group can be known. The center point coordinates are represented as (x, y), and can be arranged in ascending order according to the size of the x-axis coordinate to form a corresponding roadside berth number sequence.

[0115] See also Figure 2In one embodiment, the present application provides a roadside parking space number recognition device 100. The roadside parking space number recognition device 100 includes a data acquisition module 10, a model generation module 20, a character information acquisition module 30, a fitting straight line equation acquisition module 40, a grouping module 50, and a roadside parking space number sequence generation module 60. The data acquisition module 10 is used to acquire multiple roadside parking space number images, each roadside parking space number image includes real character detection frame coordinate information and real character category information of multiple parking space number characters. The model generation module 20 is used to train and optimize the target detection model according to the multiple roadside parking space number images to obtain a trained target detection model. The character information acquisition module 30 is used to detect the roadside parking space number image to be tested according to the trained target detection model, and obtain the character detection frame coordinate information and character category information of multiple parking space number characters.

[0116] The fitting straight line equation acquisition module 40 is used to obtain the center point coordinates of each character detection frame according to the character detection frame coordinate information of each berth number character, and to perform straight line fitting according to each center point coordinate to obtain the fitting straight line equation of each berth number character. The grouping module 50 is used to obtain the grouping distance from each center point coordinate to each fitting straight line equation according to each center point coordinate and each fitting straight line equation, and to group each center point coordinate according to each grouping distance to obtain multiple berth number groups. The roadside berth number sequence generation module 60 is used to sort the character category information and center point coordinates of each berth number character in each berth number group to obtain a roadside berth number sequence.

[0117] In this embodiment, the relevant description of the data acquisition module 10 can refer to the relevant description of S10 in the above embodiment. The relevant description of the model generation module 20 can refer to the relevant description of S20 in the above embodiment. The relevant description of the character information acquisition module 30 can refer to the relevant description of S30 in the above embodiment. The relevant description of the fitting straight line equation acquisition module 40 can refer to the relevant description of S40 in the above embodiment. The relevant description of the grouping module 50 can refer to the relevant description of S50 in the above embodiment. The relevant description of the roadside parking space number sequence generation module 60 can refer to the relevant description of S60 in the above embodiment.

[0118] In one embodiment, the roadside parking space number recognition device 100 further includes a first judgment module (not labeled in the figure), a second judgment module (not labeled in the figure), an average value acquisition module (not labeled in the figure) and a new fitting straight line equation acquisition module (not labeled in the figure). The first judgment module is used to calculate the slope difference according to the slopes of any two fitting straight line equations, and to determine whether the slope difference is less than a first threshold. The second judgment module is used to calculate the intercept difference according to the intercepts of the two fitting straight line equations if the slope difference is less than the first threshold, and to determine whether the intercept difference is less than a second threshold. The average value acquisition module is used to calculate the slope average value and the intercept average value of the two fitting straight line equations if the intercept difference is less than the second threshold. The new fitting straight line equation acquisition module is used to obtain a new fitting straight line equation according to the slope average value and the intercept average value, and replace the two fitting straight line equations with the new fitting straight line equation to obtain a new fitting straight line equation corresponding to the parking space number character.

[0119] In this embodiment, the relevant description of the first judgment module can refer to the relevant description of S401 in the above embodiment. The relevant description of the second judgment module can refer to the relevant description of S402 in the above embodiment. The relevant description of the average value acquisition module can refer to the relevant description of S403 in the above embodiment. The relevant description of the new fitting straight line equation acquisition module can refer to the relevant description of S404 in the above embodiment.

[0120] In one embodiment, the roadside parking space number recognition device 100 further includes a third judgment module (not labeled in the figure) and a fourth judgment module (not labeled in the figure). The third judgment module is used to determine that if the slope difference is greater than the first threshold, the two fitted straight line equations are two independent straight lines. The fourth judgment module (not labeled in the figure) is used to determine that if the intercept difference is greater than the second threshold, the two fitted straight line equations are two independent straight lines. The first threshold ranges from 3 to 7, and the second threshold ranges from 17 to 24.

[0121] In this embodiment, the description of the third judgment module may refer to the description of S405 in the above embodiment. The description of the fourth judgment module may refer to the description of S406 in the above embodiment.

[0122] In one embodiment, the grouping module 50 includes a fifth judgment module (not labeled in the figure), a center point coordinate division module (not labeled in the figure), and a berth number group formation module (not labeled in the figure). The fifth judgment module is used to judge whether each group distance is less than a third threshold. The center point coordinate division module is used to divide the center point coordinates corresponding to the group distance into the same group if the group distance is less than the third threshold. The berth number group formation module is used to form each berth number group according to the berth number characters corresponding to each center point coordinate in the same group. Wherein, the range of the third threshold is 75 to 85.

[0123] In this embodiment, the description of the fifth determination module can refer to the description of S510 in the above embodiment. The description of the center point coordinate division module can refer to the description of S520 in the above embodiment. The description of the berth number group formation module can refer to the description of S530 in the above embodiment.

[0124] In one embodiment, the roadside parking space number sequence generation module 60 includes a sorting module (not marked in the figure) for obtaining a roadside parking space number sequence by sorting the character category information of each parking space number character in each parking space number group and the x-axis coordinate of the center point coordinate in ascending order.

[0125] In this embodiment, the relevant description of the sorting module may refer to the relevant description of S610 in the above embodiment.

[0126] In one embodiment, the model generation module 20 includes a target detection model construction module (not labeled in the figure) and a training optimization module (not labeled in the figure). The target detection model construction module is used to input multiple roadside parking space number images into the target detection model, and output the predicted character detection frame coordinate information and predicted character category information of each parking space number character. The training optimization module is used to construct a loss function based on the real character detection frame coordinate information, the real character category information, the predicted character position information and the predicted character category information, and train and optimize the target detection model based on the loss function to obtain a trained target detection model.

[0127] In this embodiment, the description of the target detection model building module can refer to the description of S210 in the above embodiment. The description of the training optimization module can refer to the description of S220 in the above embodiment.

[0128] In the above-mentioned various embodiments, the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims provide the elements of various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0129] Those skilled in the art may also understand that the various illustrative logical blocks, modules and steps listed in the embodiments of the present application may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, modules and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0130] The various illustrative logic blocks or modules described in the embodiments of the present application can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or the design of any combination of the above functions. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.

[0131] The steps of the method or algorithm described in the embodiments of the present application can be directly embedded in the software module executed by the hardware, the processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0132] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for identifying roadside parking lot numbers, characterized in that: include: Acquire a plurality of roadside parking space number images, each of the roadside parking space number images including real character detection frame coordinate information and real character category information of a plurality of parking space number characters; Training and optimizing the target detection model according to the plurality of roadside parking space number images to obtain a trained target detection model; According to the trained target detection model, the roadside parking space number image to be detected is detected to obtain character detection frame coordinate information and character category information of multiple parking space number characters; According to the character detection frame coordinate information of each of the berth number characters, the center point coordinates of each of the character detection frames are obtained, and a straight line fitting is performed according to each of the center point coordinates to obtain a fitting straight line equation of each of the berth number characters; According to each of the center point coordinates and each of the fitted straight line equations, a group distance from each of the center point coordinates to each of the fitted straight line equations is obtained, and each of the center point coordinates is grouped according to each of the group distances to obtain a plurality of berth number groups; A roadside parking space number sequence is obtained by sorting the character category information and the center point coordinates of each parking space number character in each parking space number group.

2. The method for identifying roadside parking spaces according to claim 1, characterized in that: After obtaining the center point coordinates of each character detection frame according to the coordinate information of the character detection frame of each berth number character, and performing straight line fitting according to each center point coordinate to obtain the fitted straight line equation of each berth number character, obtaining the grouping distance between each center point coordinate and each fitted straight line equation according to each center point coordinate and each fitted straight line equation, and grouping each center point coordinate according to each grouping distance to obtain multiple berth number groups, the method further includes: Calculating a slope difference according to the slopes of any two of the fitted straight line equations, and determining whether the slope difference is less than a first threshold; If the slope difference is less than the first threshold, calculating the intercept difference according to the intercepts of the two fitted straight line equations, and determining whether the intercept difference is less than a second threshold; If the intercept difference is less than the second threshold, then calculating the average slope and the average intercept of the two fitted straight line equations; A new fitting straight line equation is obtained according to the average slope value and the average intercept value, and the two fitting straight line equations are replaced with the new fitting straight line equation to obtain the new fitting straight line equation corresponding to the berth number character.

3. The method for identifying roadside parking spaces according to claim 2, characterized in that: The method further comprises: If the slope difference is greater than the first threshold, the two fitted straight line equations are two independent straight lines; If the intercept difference is greater than the second threshold, the two fitted straight line equations are two independent straight lines; The first threshold ranges from 3 to 7, and the second threshold ranges from 17 to 24.

4. The method for identifying roadside parking spaces according to claim 1, characterized in that: The step of grouping each of the center point coordinates according to each of the grouping distances to obtain a plurality of berth number groups includes: Determine whether each of the grouping distances is less than a third threshold; If the grouping distance is less than the third threshold, dividing the center point coordinates corresponding to the grouping distance into the same group; forming each berth number group according to the berth number characters corresponding to each center point coordinate in the same group; The third threshold value ranges from 75 to 85.

5. The method for identifying roadside parking spaces according to claim 4, characterized in that: The step of obtaining a roadside parking space number sequence by sorting the character category information of each parking space number character in each parking space number group and the center point coordinates includes: The roadside parking space number sequence is obtained by arranging the character category information of each parking space number character in each parking space number group and the x-axis coordinate of the center point coordinate in ascending order.

6. The berth number identification method according to claim 1, characterized in that: The training and optimizing of the target detection model according to the plurality of roadside parking space number images to obtain the trained target detection model includes: Inputting the plurality of roadside parking space number images into a target detection model, and outputting predicted character detection frame coordinate information and predicted character category information of each parking space number character; A loss function is constructed based on the real character detection box coordinate information, the real character category information, the predicted character position information and the predicted character category information, and the target detection model is trained and optimized based on the loss function to obtain a trained target detection model.

7. A roadside parking space number recognition device, characterized in that: include: A data acquisition module, used to acquire a plurality of roadside parking space number images, each of which includes real character detection frame coordinate information and real character category information of a plurality of parking space number characters; A model generation module, used to train and optimize the target detection model according to the plurality of roadside parking space number images to obtain a trained target detection model; A character information acquisition module, used to detect the roadside parking space number image to be detected according to the trained target detection model, and obtain character detection frame coordinate information and character category information of multiple parking space number characters; A fitting straight line equation acquisition module, used to obtain the center point coordinates of each character detection frame according to the character detection frame coordinate information of each berth number character, and perform straight line fitting according to each center point coordinate to obtain the fitting straight line equation of each berth number character; A grouping module, used for obtaining a group distance from each center point coordinate to each of the fitted straight line equations according to each of the center point coordinates and each of the fitted straight line equations, and grouping each of the center point coordinates according to each of the grouping distances to obtain a plurality of berth number groups; The roadside parking space number sequence generating module is used to obtain a roadside parking space number sequence by sorting the character category information and the center point coordinates of each of the parking space number characters in each of the parking space number groups.

8. The roadside parking space number recognition device according to claim 7, characterized in that: The roadside parking space number recognition device further comprises: A first judgment module, used for calculating a slope difference according to the slopes of any two of the fitted straight line equations, and judging whether the slope difference is less than a first threshold; A second judgment module, configured to calculate an intercept difference according to the intercepts of the two fitted straight line equations if the slope difference is less than the first threshold, and judge whether the intercept difference is less than a second threshold; an average value acquisition module, used for calculating the slope average value and the intercept average value of the two fitted straight line equations if the intercept difference is less than the second threshold value; The new fitting straight line equation acquisition module is used to obtain the new fitting straight line equation according to the slope average value and the intercept average value, and replace the two fitting straight line equations with the new fitting straight line equation to obtain the new fitting straight line equation corresponding to the berth number character.

9. The roadside parking space number recognition device according to claim 8, characterized in that: The roadside parking space number recognition device further comprises: A third judgment module, configured to determine that if the slope difference is greater than the first threshold, the two fitted straight line equations are two independent straight lines; A fourth judgment module, configured to determine that the two fitted straight line equations are two independent straight lines if the intercept difference is greater than the second threshold; The first threshold ranges from 3 to 7, and the second threshold ranges from 17 to 24.

10. The roadside parking space number recognition device according to claim 7, characterized in that: The grouping module comprises: A fifth judging module, used for judging whether each of the grouping distances is less than a third threshold; A center point coordinate division module, configured to divide the center point coordinates corresponding to the grouping distance into the same group if the grouping distance is less than the third threshold; A berth number group forming module, used for forming each berth number group according to the berth number characters corresponding to each center point coordinate in the same group; The third threshold value ranges from 75 to 85.

11. The roadside parking space number recognition device according to claim 10, characterized in that: The roadside parking space number sequence generation module includes: The sorting module is used to arrange the character category information of each of the berth number characters in each of the berth number groups and the x-axis coordinate of the center point coordinate in ascending order to obtain the roadside berth number sequence.

12. The berth number identification device according to claim 7, characterized in that: The model generation module comprises: A target detection model building module, used for inputting the plurality of roadside parking space number images into the target detection model, and outputting predicted character detection frame coordinate information and predicted character category information of each parking space number character; A training optimization module is used to construct a loss function based on the real character detection box coordinate information, the real character category information, the predicted character position information and the predicted character category information, and to train and optimize the target detection model based on the loss function to obtain a trained target detection model.

Images