Multi-node class roadside parking stall character recognition method and system

Abstract

The application discloses a multi-node category roadside parking space character recognition method and system. The method comprises the following steps: acquiring a roadside parking space character image dataset, and performing target detection labeling on each parking space character in the roadside parking space character image dataset to obtain a labeled character image dataset; performing node classification labeling on each parking space character in the labeled character image dataset to obtain a node-labeled character image dataset; training a parking space character detection model according to the labeled character image dataset to obtain a trained parking space character detection model; training a graph convolutional neural network model according to the node-labeled character image dataset to obtain a trained graph convolutional neural network model; acquiring a to-be-tested parking space character image, and identifying the to-be-tested parking space character image according to the trained parking space character detection model and the trained graph convolutional neural network model to obtain a parking space character sequence.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for recognizing characters in multi-node category roadside parking spaces. Background Technology

[0002] In recent years, parking difficulties and disorderly parking have become one of the major problems that urgently need to be solved in the development of urban transportation in my country. Traffic management departments in various regions are also actively promoting solutions to the problems of roadside parking difficulties and disorderly parking. In order to effectively alleviate this problem, traffic management departments in various regions have tried and implemented various intelligent parking management technologies.

[0003] However, traditional methods determine the location of mobile inspection devices by using GPS positioning, and then determine the corresponding roadside parking space character sequence based on that location. Thus, obtaining the current parking space location via GPS positioning yields the roadside parking space character sequence, which is then matched with the vehicle information within the space to manage parking fees. However, the GPS positioning on mobile inspection devices is prone to errors within the meter range, leading to inaccurate roadside parking space character sequences determined by the device's location. When adjacent parking spaces are full or parking is improper, the system cannot correctly match parked vehicles with the roadside parking space character sequence, resulting in errors in parking fee management. Summary of the Invention

[0004] The purpose of this application is to solve the technical problem of incorrect confirmation of roadside parking space character sequences due to meter-level errors in GPS positioning on mobile inspection equipment. To achieve the above objective, this application provides a multi-node category roadside parking space character recognition method and system.

[0005] This application provides a method for recognizing characters in multi-node category roadside parking spaces, including:

[0006] Obtain a roadside parking space character image dataset, and perform target detection annotation on each parking space character in the roadside parking space character image dataset to obtain an annotated character image dataset;

[0007] Perform node classification and annotation on each of the berth characters in the annotated character image dataset to obtain a node-annotated character image dataset;

[0008] Based on the labeled character image dataset, the berth character detection model is trained to obtain the trained berth character detection model;

[0009] Based on the node-annotated character image dataset, the graph convolutional neural network model is trained to obtain the trained graph convolutional neural network model.

[0010] A berth character image to be tested is acquired. Based on the trained berth character detection model and the trained graph convolutional neural network model, the berth character image to be tested is identified to obtain a berth character sequence.

[0011] In one embodiment, the step of performing node classification and annotation on each of the berth characters in the annotated character image dataset to obtain a node-annotated character image dataset includes:

[0012] Based on the character position of each berth character, the berth characters in the labeled character image dataset are divided into columns or rows to form berth characters of the first node category, berth characters of the second node category, and berth characters of the third node category.

[0013] Based on the first node category berth character, the second node category berth character, and the third node category berth character, node category annotation is performed on each berth character in the annotated character image dataset to form the node-annotated character image dataset.

[0014] In one embodiment, training a graph convolutional neural network model based on the node-annotated character image dataset to obtain a trained graph convolutional neural network model includes:

[0015] Based on the character position of each berth character, the node-annotated character image in the node-annotated character image dataset is extracted and transformed to obtain a single-character image dataset;

[0016] Based on the character position of each berth character, obtain the distance feature of a single character;

[0017] The image features of each single character image in the single character image dataset are concatenated with the distance features of the single character to obtain the input feature vector of each single character image.

[0018] In one embodiment, training the graph convolutional neural network model based on the node-annotated character image dataset to obtain the trained graph convolutional neural network model further includes:

[0019] Based on the node category of each single character image in the single character image dataset, obtain the node classification feature matrix of the character detection box corresponding to each single character image;

[0020] Based on the character position of each single character image in the single character image dataset, obtain the degree matrix of the character detection box corresponding to each single character image;

[0021] Obtain the identity matrix of the character detection box corresponding to each single character image, and obtain the adjacency matrix of the character detection box corresponding to each single character image based on the identity matrix, the node classification feature matrix and the degree matrix;

[0022] The graph convolutional neural network model is trained based on the adjacency matrix of the character detection box corresponding to each single character image and the input feature vector to obtain the trained graph convolutional neural network model.

[0023] The graph convolutional neural network model is as follows:

[0024] Where X represents the input feature vector of each single-character image, A represents the node classification feature matrix of the character detection box corresponding to each single-character image, and I represents the identity matrix of the character detection box corresponding to each single-character image. The degree matrix represents the character detection box corresponding to each of the single character images. W represents the adjacency matrix of the character detection boxes corresponding to each single character image. (0) With W (1) The parameters of the graph convolutional neural network model are represented by , softmax represents the normalization function, and ReLU represents the non-linear activation function.

[0025] In one embodiment, acquiring the berth character image to be tested, and recognizing the berth character image to obtain the berth character sequence based on the trained berth character detection model and the trained graph convolutional neural network model, includes:

[0026] The character image of the berth to be tested is obtained, and the character image of the berth to be tested is input into the trained berth character detection model to obtain the character category and character position of each character of the berth to be tested;

[0027] Based on the character position of each character, the character image of the berth to be tested is extracted and transformed to obtain multiple single character images to be tested. The distance feature of the single character to be tested is obtained based on the character position of each character. The image feature of each single character image to be tested is concatenated with the distance feature of the single character to be tested to obtain the input feature vector of each single character image to be tested.

[0028] The input feature vector of each of the single-character images to be tested is input into the trained graph convolutional neural network model, and the node classification feature matrix of the character detection box corresponding to each single-character image to be tested is output.

[0029] The berth character sequence is obtained based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image to be tested for each berth character.

[0030] In one embodiment, obtaining the berth character sequence based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image to be tested includes:

[0031] Based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each character image to be tested, all the characters to be tested are sorted in ascending or descending order to obtain the berth character sequence.

[0032] In one embodiment, this application provides a roadside parking space character recognition system based on a graph convolutional neural network, comprising:

[0033] The data acquisition module is used to acquire a roadside parking space character image dataset, and to perform target detection and annotation on each parking space character in the roadside parking space character image dataset to obtain an annotated character image dataset;

[0034] The node classification module is used to perform node classification and annotation on each of the berth characters in the labeled character image dataset to obtain the node-annotated character image dataset.

[0035] The first model training module is used to train the berth character detection model based on the labeled character image dataset to obtain the trained berth character detection model.

[0036] The second model training module is used to train the graph convolutional neural network model based on the node-annotated character image dataset to obtain the trained graph convolutional neural network model.

[0037] The berth character sequence acquisition module is used to acquire the berth character image to be tested, and to identify the berth character image to be tested based on the trained berth character detection model and the trained graph convolutional neural network model to obtain the berth character sequence.

[0038] In one embodiment, the node classification module includes:

[0039] The segmentation module is used to divide each of the berth characters in the labeled character image dataset into columns or rows according to the character position of each berth character, forming berth characters of the first node category, berth characters of the second node category, and berth characters of the third node category.

[0040] The annotation module is used to annotate each berth character in the annotated character image dataset with a node category according to the first node category berth character, the second node category berth character, and the third node category berth character, thereby forming the node-annotated character image dataset.

[0041] In one embodiment, the second model training module includes:

[0042] The single-character image acquisition module is used to extract and transform each node-labeled character image in the node-labeled character image dataset according to the character position of each berth character to obtain a single-character image dataset;

[0043] The distance feature acquisition module is used to acquire the distance feature of a single character based on the character position of each of the berth characters;

[0044] The input feature vector acquisition module is used to concatenate the image features of each single character image in the single character image dataset with the distance features of the single character to obtain the input feature vector of each single character image.

[0045] In one embodiment, the second model training module further includes:

[0046] The node classification feature matrix acquisition module is used to obtain the node classification feature matrix of the character detection box corresponding to each single character image based on the node category of each single character image in the single character image dataset.

[0047] The degree matrix acquisition module is used to obtain the degree matrix of the character detection box corresponding to each single character image based on the character position of each single character image in the single character image dataset;

[0048] The adjacency matrix acquisition module is used to acquire the identity matrix of the character detection box corresponding to each single character image, and to obtain the adjacency matrix of the character detection box corresponding to each single character image based on the identity matrix, the node classification feature matrix and the degree matrix.

[0049] The graph convolutional neural network model construction module is used to train the graph convolutional neural network model based on the adjacency matrix of the character detection box corresponding to each single character image and the input feature vector, so as to obtain the trained graph convolutional neural network model.

[0050] The graph convolutional neural network model is as follows:

[0051] Where X represents the input feature vector of each single-character image, A represents the node classification feature matrix of the character detection box corresponding to each single-character image, and I represents the identity matrix of the character detection box corresponding to each single-character image. The degree matrix represents the character detection box corresponding to each of the single character images. W represents the adjacency matrix of the character detection boxes corresponding to each single character image. (0) With W (1) The parameters of the graph convolutional neural network model are represented by , softmax represents the normalization function, and ReLU represents the non-linear activation function.

[0052] In one embodiment, the berth character sequence acquisition module includes:

[0053] The berth character information acquisition module is used to acquire the character image of the berth to be tested, input the character image of the berth to be tested into the trained berth character detection model, and obtain the character category and character position of each character of the berth to be tested.

[0054] The module for obtaining the input feature vector to be tested is used to extract and transform the character image of the berth to be tested according to each character position to obtain multiple single character images to be tested, obtain the distance feature of the single character to be tested according to each character position, and concatenate the image feature of each single character image to be tested with the distance feature of the single character to be tested to obtain the input feature vector of each single character image to be tested.

[0055] The module for obtaining the classification feature matrix of the node to be tested is used to input the input feature vector of each single character image to be tested into the trained graph convolutional neural network model, and output the node classification feature matrix of the character detection box corresponding to each single character image to be tested;

[0056] The berth character sequence generation module is used to obtain the berth character sequence based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image to be tested for each berth character.

[0057] In one embodiment, the berth character sequence generation module includes:

[0058] The sorting module is used to sort all the characters of the berth to be tested in ascending or descending order according to the character category, character position and node classification feature matrix of the character detection box corresponding to each single character image to be tested, so as to obtain the berth character sequence.

[0059] The aforementioned multi-node category roadside parking space character recognition method and system constructs an annotated character image dataset and a node-annotated character image dataset to train the parking space character detection model and the graph convolutional neural network model, obtaining the trained parking space character detection model and graph convolutional neural network model. Based on the trained parking space character detection model and graph convolutional neural network model, the system detects and recognizes the parking space character images to be tested, obtaining the character category, character position, and character node category, and then obtaining the parking space character sequence based on the character category, character position, and character node category. This multi-node category roadside parking space character recognition method allows for real-time recognition of roadside parking space characters, eliminating the need to rely solely on GPS positioning to obtain the roadside parking space character sequence, thus avoiding the problem of incorrect matching between the roadside parking space character sequence and the vehicle caused by GPS positioning errors. Furthermore, this method can obtain vehicle information based on the roadside parking space character image and match the vehicle with the parking space character sequence, further improving the accuracy of parking space character matching, enhancing the precision of parking fee management, and improving user experience satisfaction. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating the steps of the multi-node category roadside parking space character recognition method provided in this application.

[0061] Figure 2 This is a schematic diagram illustrating the classification of node categories in one embodiment provided in this application.

[0062] Figure 3 This is a schematic diagram of the multi-node category roadside parking space character recognition system provided in this application. Detailed Implementation

[0063] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0064] Please see Figure 1 This application provides a method for recognizing roadside parking space characters based on graph convolutional neural networks, including:

[0065] S10, obtain the roadside parking space character image dataset, and perform target detection annotation on each parking space character in the roadside parking space character image dataset to obtain the annotated character image dataset;

[0066] S20, perform node classification and labeling on each berth character in the labeled character image dataset to obtain the node-labeled character image dataset;

[0067] S30, Train the berth character detection model based on the labeled character image dataset to obtain the trained berth character detection model;

[0068] S40, Train the graph convolutional neural network model based on the node-annotated character image dataset to obtain the trained graph convolutional neural network model;

[0069] S50: Obtain the character image of the berth to be tested. Based on the trained berth character detection model and the trained graph convolutional neural network model, identify the character image of the berth to be tested to obtain the berth character sequence.

[0070] In S10, the roadside parking space character image dataset includes multiple roadside parking space character images. These images are collected using cameras on a mobile inspection device. The mobile inspection device has three cameras, facing forward, to the right, and backward of the device's direction of travel, respectively capturing video data from the front, right, and rear of the parking spaces. Roadside parking space characters are typically printed next to the parking spaces. The mobile inspection device uses a down-facing camera to capture clear and complete images of these characters. By collecting video data from the down-facing cameras, image frames containing roadside parking space characters are extracted to form the roadside parking space character image dataset. A bounding box annotation tool is used to perform target detection and annotation on all parking space characters in the roadside parking space character images. The annotation information includes the coordinates of the top-left and bottom-right corners of the characters and their category information. The coordinates of each parking space character can also be understood as its position. The character position is represented by its coordinates. The berth character category information includes Arabic numerals 0-9 and the character "-", totaling 11 categories. However, the specific category information is not limited to these 11 categories and can be added as needed. After labeling each berth character in the roadside berth character image dataset, the berth character detection label is obtained, forming a labeled character image dataset. The labeled character image dataset includes multiple roadside berth character images containing labeling information.

[0071] In S20, node classification can be understood as further dividing the sequence of parking space characters formed by roadside parking space characters, based on the printing position of the parking space characters in the actual roadside scene. The node classification categories can include left-side category parking space characters, right-side category parking space characters, and middle category parking space characters, etc. The specific categories are not limited to the three categories listed above; other categories can be included, and additional categories can be added as needed. Left-side category parking space characters can be understood as parking space numbers located on the left edge of the parking space frame, with the character sequence read from top to bottom. Right-side category parking space characters can be understood as parking space numbers located on the right edge of the parking space frame, with the character sequence read from bottom to top. Middle category parking space characters can be understood as parking space numbers located on the bottom edge of the parking space frame, with the character sequence read from left to right. Node classification and annotation of all characters in the labeled character image dataset can indicate the specific position of each character relative to the parking space frame, which is more helpful in determining which parking space a parking space character belongs to among multiple adjacent parking spaces, and can solve errors caused by GPS positioning errors.

[0072] In S30, the labeled character image dataset includes multiple roadside berth character images containing character position and category information. This dataset serves as the model training set for training the berth character detection model, optimizing parameters, and obtaining a trained model. The trained model is used to detect and recognize the character positions and categories of berth characters. Each labeled character image in the dataset is input into the berth character detection model for target detection, obtaining the target detection results for all characters in the labeled character images. In one embodiment, the berth character detection model includes a backbone network and a target classification and regression network. The backbone network is used for feature extraction from the labeled character images. The backbone network includes, but is not limited to, convolutional neural networks such as ResNet and VGG. The target classification and regression network is used to classify and regress targets based on the extracted features, obtaining the character category and position corresponding to the detection box for each character. Target detection networks include, but are not limited to, two-stage or single-stage target detection methods such as Faster-RCNN, YOLO, and SSD.

[0073] In S40, the node-annotated character image dataset includes multiple roadside berth character images containing node category, character position, and character category information. This dataset serves as the model training set for training a graph convolutional neural network (GCN) model, optimizing parameters, and obtaining a trained GCN model. The trained GCN model is then used to identify the node categories of the berth characters. Each node-annotated character image from the dataset is input into the GCN model to identify the node categories of the berth characters, obtaining the node categories of all characters in the node-annotated character images. In one embodiment, the GCN model can be a Graph Convolutional Network (GCN) for processing unstructured data. This application represents the relationships between berth characters as graph-structured data, leveraging the powerful graph-structured data learning capabilities of the GCN to achieve the task of classifying the node categories of the berth characters.

[0074] In S50, the image of the parking space character to be tested can be understood as the image of parking space character to be detected and recognized. The trained parking space character detection model detects and recognizes the parking space character image, obtaining the character position and character category of all characters in the image. The trained graph convolutional neural network model then performs node category detection on the parking space character image, obtaining the node category of all characters in the image. Based on the character position and character category of all characters, the parking space character sequence is obtained. Therefore, based on the character position, character category, and node category of all characters, the parking space character sequence for each parking space can be determined.

[0075] The multi-node category roadside parking space character recognition method provided in this application constructs an annotated character image dataset and a node-annotated character image dataset, and trains a parking space character detection model and a graph convolutional neural network model to obtain the trained parking space character detection model and graph convolutional neural network model. Based on the trained parking space character detection model and graph convolutional neural network model, the method detects and recognizes the parking space character images to be tested, obtaining the character category, character position, and character node category, and then obtaining the parking space character sequence based on the character category, character position, and character node category. This multi-node category roadside parking space character recognition method allows for real-time recognition of roadside parking space characters, eliminating the need to rely solely on GPS positioning to obtain the roadside parking space character sequence, thus avoiding the problem of incorrect matching between the roadside parking space character sequence and the vehicle caused by GPS positioning errors. Furthermore, this method can obtain vehicle information based on the roadside parking space character image and perform vehicle-parking space character sequence matching, further improving the accuracy of parking space character matching, the precision of parking fee management, and user experience satisfaction.

[0076] In one embodiment, S20, node classification and annotation are performed on each berth character in the labeled character image dataset to obtain a node-annotated character image dataset, including:

[0077] S210, based on the character position of each berth character, divide each berth character in the labeled character image dataset into columns or rows to form berth characters of the first node category, berth characters of the second node category, and berth characters of the third node category;

[0078] S220, based on the first node category berth character, the second node category berth character, and the third node category berth character, perform node category labeling on each berth character in the labeled character image dataset to form a node-labeled character image dataset.

[0079] In this embodiment, the character position of the berth character can be understood as the coordinate information of the upper left and lower right corners of the character detection box. Based on the character position of each berth character, it can be determined which characters in the labeled character image dataset belong to the same row or column. Column division or row division can be understood as dividing characters into the same row or column. Classifying characters located in the same row and the same column forms the first node category berth character, the second node category berth character, and the third node category berth character. In one embodiment, the first node category berth character can be the left-side category berth character, the second node category berth character can be the right-side category berth character, and the third node category berth character can be the middle category berth character (see [reference]). Figure 2(As shown). The positional descriptions of the left, right, and middle categories of parking space characters are relative to the parking space bounding box and are not limited to left, right, or center positions. By labeling all parking space characters in the labeled character image dataset with node categories, each character can be divided into different node categories. This further limits the sequence position based on the character position of the parking space characters, enabling more accurate detection and recognition of parking space character sequences. It is also more helpful in determining which parking space the parking space character sequence belongs to among multiple adjacent parking spaces, thus resolving errors caused by GPS positioning.

[0080] In one embodiment, after performing node classification and annotation on each berth character in the labeled character image dataset to obtain the node-annotated character image dataset, and then training the berth character detection model based on the labeled character image dataset in S20, before obtaining the trained berth character detection model, the method further includes:

[0081] Rotate each node-annotated character image in the node-annotated character image dataset to obtain multiple tilted node-annotated character images;

[0082] Perform brightness transformation on each node-annotated character image in the node-annotated character image dataset to obtain multiple brightness-transformed node-annotated character images;

[0083] Cropping is performed on each node-annotated character image in the node-annotated character image dataset to obtain multiple occluded node-annotated character images;

[0084] A data-augmented node-annotated character image dataset is formed by using character images annotated with multiple tilted nodes, multiple brightness transformation nodes, and multiple occlusion nodes.

[0085] In this embodiment, the character image labeled at each node is rotated to obtain tilted character images with different rotation angles, which can simulate tilted character images encountered in real shooting scenarios. Brightness transformation is applied to each character image labeled at each node to simulate character images under different brightness conditions in the morning, noon, and evening in a real roadside parking scenario. Random cropping of each character image labeled at each node can simulate character images in a real roadside parking scenario where vehicles obscure parking space numbers.

[0086] In one embodiment, S30, the berth character detection model is trained based on the labeled character image dataset to obtain the trained berth character detection model. The berth character detection model includes a backbone network, a feature pyramid network, and an object detection network. The backbone network includes, but is not limited to, convolutional neural networks such as ResNet and VGG. The object detection network includes, but is not limited to, two-stage or single-stage object detection methods such as Faster-RCNN, YOLO, and SSD.

[0087] By integrating a feature pyramid network into the backbone network, features are extracted and fused from multiple labeled character images to obtain multiple feature-fused number images. These multiple feature-fused number images are then input into an object detection network for object detection, yielding the predicted character detection box coordinates and predicted character category information for each character. Adding a feature pyramid network structure to the backbone network allows for the extraction of high-level semantic features at different scales, and the extracted features are then fused. Feature Pyramid Networks (FPNs) effectively address multi-scale issues.

[0088] The fused number image includes the fused features. These fused features are obtained by fusing feature maps of different resolutions through a feature pyramid network. By increasing the network connections of the feature pyramid network, detection performance can be improved without increasing the computational cost of the convolutional neural network model. The feature pyramid network, through top-down paths and lateral connections, can fuse lower-resolution but semantically strong features and higher-resolution but semantically weak features from multiple labeled character images, achieving feature fusion and thus improving the accuracy of object detection.

[0089] Integrating the feature pyramid network into the backbone network enables feature extraction and fusion, solving the problem of inconsistent number sizes in roadside parking space identification caused by different distances from the parking space and different shooting angles when mobile inspection equipment collects data, thereby improving the accuracy of the roadside parking space number identification method.

[0090] In one embodiment, S30, the berth character detection model is trained based on the labeled character image dataset. The trained berth character detection model obtains multiple feature-fused number images by extracting and fusing features from multiple labeled character images through a feature pyramid network and a backbone network. Each feature-fused number image is input into the classification and regression branches of the target detection network, outputting the predicted character detection box coordinates and predicted character category information for each character. A positional regression loss function can be constructed based on the predicted and true character detection box coordinates for each character. A category classification loss function is constructed based on the predicted and true character category information for each character.

[0091] The loss function of the berth character detection model is L. det =λ1L reg +λ2L cls .

[0092] Among them, L reg L represents the location regression loss function. cls This represents the category classification loss function, where λ1 and λ2 represent weight coefficients, which can be set to 1 or adjusted according to the specific application.

[0093] Location regression loss functions include, but are not limited to, mean absolute error loss (L1 Loss), mean squared error loss (L2 Loss), and intersection-over-union (IoU) loss. Classification loss functions include, but are not limited to, cross-entropy loss and focal loss. The object detection model is trained and its parameters optimized using both location regression and classification loss functions.

[0094] In one embodiment, S40, the graph convolutional neural network model is trained based on the node-annotated character image dataset to obtain the trained graph convolutional neural network model, including:

[0095] S410, Based on the character position of each berth character, extract and transform the character image of each node in the node labeled character image dataset to obtain a single character image dataset;

[0096] S420: Obtain the distance feature of a single character based on the character position of each berth character;

[0097] S430: The image features of each single character image in the single character image dataset are concatenated with the distance features of the single character to obtain the input feature vector of each single character image.

[0098] In this embodiment, the character position of each berth character can be obtained through the coordinate information of the character detection box. The image corresponding to each detection box is extracted from the node-annotated character image, resulting in multiple single-character images. Each single-character image contains one character. Multiple single-character images of different sizes are resized to the same size. The dimensions of the resized single-character image are N*C*H*W, where N represents the number of character detection boxes, H*W represents the width and height of the character detection boxes, and C represents the number of channels, which can be RGB three channels.

[0099] Based on the node classification of berth characters into three categories (first, second, and third), berth characters belonging to different node categories have different character orientations and different image features, which can be used as one of the features learned for node classification. Please see [link / reference]. Figure 2 For example, taking a certain berth as a reference, the berth type characters 01-002-03 on the left face east, the berth type characters 04-005-06 on the right face west, and the berth type characters 07-008-09 in the middle face north.

[0100] The distance feature of each single character can be represented as N*4*H*W, where N is the number of character detection boxes, 4 represents the X, Y, W, and H information of the character position, and H*W is the width and height information of the single character image after standardization. By using the distance features of single characters, the relationship between adjacent berth characters with close proximity can be learned. Characters with close proximity have a higher probability of belonging to the same node category, which is more beneficial for node classification learning.

[0101] The N*3*W*H image features and the N*4*H*W distance features are concatenated along the channel dimension to obtain features of N*7*W*H dimensions.

[0102] In one embodiment, the N*7*H*W dimension features after feature concatenation are input into multiple convolutional layers to obtain the image features after feature transformation, which can be represented as N*C*H*W, with the number of channels C being 128. This allows for further extraction of features of the detection box image and distance.

[0103] In one embodiment, the image features after feature transformation are input into a fully connected layer network for calculation to obtain a feature dimension of N*C*1*1, i.e., N*C, where C is 128. This feature is used as the input feature vector for each single character image and is then input into a graph convolutional neural network model for training.

[0104] In one embodiment, S40, training a graph convolutional neural network model based on a node-annotated character image dataset to obtain a trained graph convolutional neural network model further includes:

[0105] S440, Based on the node category of each single character image in the single character image dataset, obtain the node classification feature matrix of the character detection box corresponding to each single character image;

[0106] S450: Based on the character position of each single character image in the single character image dataset, obtain the degree matrix of the character detection box corresponding to each single character image;

[0107] S460, obtain the identity matrix of the character detection box corresponding to each single character image, and obtain the adjacency matrix of the character detection box corresponding to each single character image based on the identity matrix, the node classification feature matrix and the degree matrix.

[0108] S470, The graph convolutional neural network model is trained based on the adjacency matrix of the character detection box corresponding to each single character image and the input feature vector to obtain the trained graph convolutional neural network model.

[0109] The graph convolutional neural network model is as follows:

[0110] Where X represents the input feature vector of each single character image, A represents the node classification feature matrix of the character detection box corresponding to each single character image, and I represents the identity matrix of the character detection box corresponding to each single character image. This represents the degree matrix of the character detection box corresponding to each single character image. W represents the adjacency matrix of the character detection boxes corresponding to each single character image. (0) With W (1) represents the parameters of the graph convolutional neural network model, softmax represents the normalization function, and ReLU represents the non-linear activation function.

[0111] In this embodiment, the graph convolutional neural network model includes multiple convolutional layers and nonlinear activation layers to learn the features of berth character nodes. Node categories are divided into three categories: first category berth character, second category berth character, and third category berth character. Based on the node category of each single-character image, the relationship between the corresponding character detection box and other detection boxes can be determined. If each single-character image belongs to the same node category as other single-character images, it can be understood that the corresponding character detection box of each single-character image belongs to the same node category as the corresponding character detection boxes of other single-character images, and thus has a value of 1 in the node classification feature matrix. If they do not belong to the same node category, the value in the matrix is ​​0, forming the node classification feature matrix for the character detection boxes corresponding to each single-character image. I represents the identity matrix; the identity matrix of the character detection boxes corresponding to each single-character image can be used to increase the feature representation of each character detection box itself. represents the degree matrix of the character detection boxes corresponding to each single character image, indicating the degree of the nodes. This can also be understood as the number of detection boxes connected to other character detection boxes. Connections can be understood as belonging to the same node category. Z represents a two-layer graph convolutional neural network with an output feature dimension of N*N. This is a first-layer graph convolutional neural network with an output feature dimension of N*N. Let W be an adjacency matrix with N*N feature dimensions.(0) W represents the learnable parameters in the first layer of the graph convolutional neural network, with a feature dimension of 128*N, and ReLU() denotes the non-linear activation function. (1) represents the learnable parameters in the second-layer graph convolutional neural network, with a feature dimension of 128*N, and softmax() represents the softmax normalization function.

[0112] In one embodiment, S40, training a graph convolutional neural network model based on a node-annotated character image dataset to obtain a trained graph convolutional neural network model further includes:

[0113] S480, construct the loss function of the graph convolutional neural network model based on the node classification feature matrix of the character detection box corresponding to each single character image and the predicted node classification feature matrix.

[0114] In this embodiment, based on the graph convolutional neural network model, the predicted node classification feature matrix of the character detection box corresponding to each single character image can be predicted, which means it can be determined whether each character detection box belongs to the same node category as other character detection boxes. The predicted node classification feature matrix is ​​represented by the values ​​1 or 0. A value of 1 indicates that a character detection box belongs to the same node category as other character detection boxes, and a value of 0 indicates that a character detection box does not belong to the same node category as other character detection boxes. Therefore, based on the predicted node classification feature matrix obtained from the graph convolutional neural network model, it can be determined whether each character detection box belongs to the same node category, and thus the parking space sequence number of each parking space can be determined. In this embodiment, the L2 norm loss function is used for model training. The L2 norm loss function is also called the least squared error loss function. The loss function of the graph convolutional neural network model is:

[0115]

[0116] in, Let y represent the predicted node classification feature matrix of the model, which can also be understood as the true node classification feature matrix. m represents the total number of samples, which can also be understood as the sample data corresponding to the m character detection boxes. The loss function calculates the sum of squares of the predicted and true values ​​for each sample, and then sums the squares over all samples. The objective of the loss function is to minimize the sum of squares.

[0117] In one embodiment, S50, an image of the berth character to be tested is acquired. Based on the trained berth character detection model and the trained graph convolutional neural network model, the image of the berth character to be tested is recognized to obtain a berth character sequence, including:

[0118] S510, acquire the character image of the berth to be tested, input the character image of the berth to be tested into the trained berth character detection model, and obtain the character category and character position of each character of the berth to be tested;

[0119] S520: Extract and transform the image of the character to be tested based on the character position of each character to be tested to obtain multiple single character images to be tested. Obtain the distance feature of the single character to be tested based on the character position of each character to be tested. Then, concatenate the image feature of each single character image to be tested with the distance feature of the single character to be tested to obtain the input feature vector of each single character image to be tested.

[0120] S530: Input the input feature vector of each single character image to be tested into the trained graph convolutional neural network model, and output the node classification feature matrix of the character detection box corresponding to each single character image to be tested;

[0121] S540: Obtain the berth character sequence based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each character image to be tested.

[0122] In this embodiment, in S510, the trained berth character detection model can obtain the character category of each character to be tested, such as the Arabic numerals 0-9 and the character "-", totaling 11 categories. The trained berth character detection model can also obtain the character position of each character to be tested, which can be understood as the character's coordinate position. In one embodiment, the NMS (Non-Maximum Suppression) algorithm is used to remove some redundant boxes, retaining the category and position information of the berth character detection output box.

[0123] In S520, the relevant descriptions in this step can be referred to the relevant descriptions in S410 to S430 of the above embodiments. In S530, through the trained graph convolutional neural network model, the node classification feature matrix corresponding to each character detection box can be obtained, and then it can be determined whether each character detection box belongs to the same node category as other detection boxes based on the node classification feature matrix. A single character image to be tested corresponds to a character to be tested, a character category, a character position, a character detection box, and a node classification feature matrix. There is a one-to-one correspondence between them, which can be explained and illustrated. The node classification feature matrix can be understood as the category relationship between a certain character detection box and other character detection boxes, or it can be further understood as the category relationship between a certain berth character and other berth characters. In S540, based on the character category, character position, and corresponding node classification feature matrix of each berth character to be tested, the berth character sequence formed by the arrangement of multiple berth characters to be tested can be obtained.

[0124] In one embodiment, S540, based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each character image to be tested, a berth character sequence is obtained, including:

[0125] S541, based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each character image to be tested, all characters to be tested are sorted in ascending or descending order to obtain the berth character sequence.

[0126] In this embodiment, the specific position of each character in a berth to be tested can be determined based on its character position. The category relationship between the characters in each berth to be tested can be determined based on the node classification feature matrix of the character detection box corresponding to each character, identifying which characters belong to the same category. Based on the character category of each character, the specific number of each character can be determined. Therefore, by sorting the characters in the same category in ascending or descending order according to their character positions, the corresponding complete berth character sequence can be obtained.

[0127] Therefore, the multi-node category roadside berth character recognition method provided in this application uses object detection methods to detect each berth character. By constructing a berth character detection model and using a loss function for model training and parameter optimization, a trained berth character detection model can be obtained. This allows for real-time detection of roadside berth characters, obtaining the character category and position of each berth character. Furthermore, the multi-node category roadside berth character recognition method provided in this application classifies nodes based on the object detection results of each berth character and uses a graph convolutional neural network model to classify each berth character node. Berth character nodes belonging to the same berth sequence are grouped into one category, and a complete berth character sequence is obtained based on berth characters of the same category.

[0128] Please see Figure 3 This application provides a roadside parking space character recognition system 100 based on a graph convolutional neural network. The multi-node category roadside parking space character recognition system 100 includes a data acquisition module 10, a node classification module 20, a first model training module 30, a second model training module 40, and a parking space character sequence acquisition module 50. The data acquisition module 10 acquires a roadside parking space character image dataset and performs target detection annotation on each parking space character in the dataset to obtain an annotated character image dataset. The node classification module 20 performs node classification annotation on each parking space character in the annotated character image dataset to obtain a node-annotated character image dataset. The first model training module 30 trains a parking space character detection model based on the annotated character image dataset to obtain a trained parking space character detection model. The second model training module 40 trains a graph convolutional neural network model based on the node-annotated character image dataset to obtain a trained graph convolutional neural network model. The berth character sequence acquisition module 50 is used to acquire the berth character image to be tested. Based on the trained berth character detection model and the trained graph convolutional neural network model, it identifies the berth character image to be tested and obtains the berth character sequence.

[0129] In this embodiment, the description of the data acquisition module 10 can be referred to the description of S10 in the above embodiment. The description of the node classification module 20 can be referred to the description of S20 in the above embodiment. The description of the first model training module 30 can be referred to the description of S30 in the above embodiment. The description of the second model training module 40 can be referred to the description of S40 in the above embodiment. The description of the berth character sequence acquisition module 50 can be referred to the description of S50 in the above embodiment.

[0130] In one embodiment, the node classification module 20 includes a segmentation module and an annotation module. The segmentation module is used to divide each berth character in the annotated character image dataset into columns or rows according to the character position of each berth character, forming berth characters of a first node category, berth characters of a second node category, and berth characters of a third node category. The annotation module is used to annotate each berth character in the annotated character image dataset with a node category based on the first node category berth characters, the second node category berth characters, and the third node category berth characters, forming a node-annotated character image dataset.

[0131] In this embodiment, the description of the segmentation module can be found in the description of S210 in the above embodiment. The description of the annotation module can be found in the description of S220 in the above embodiment.

[0132] In one embodiment, the second model training module 40 includes a single-character image acquisition module, a distance feature acquisition module, and an input feature vector acquisition module. The single-character image acquisition module extracts and transforms each node-annotated character image in the node-annotated character image dataset based on the character position of each berth character, obtaining a single-character image dataset. The distance feature acquisition module obtains the distance feature of each single character based on its character position. The input feature vector acquisition module concatenates the image features of each single-character image in the single-character image dataset with the distance feature of the single character, obtaining the input feature vector of each single-character image.

[0133] In this embodiment, the description of the single-character image acquisition module can be found in the description of S410 in the above embodiment. The description of the distance feature acquisition module can be found in the description of S420 in the above embodiment. The description of the input feature vector acquisition module can be found in the description of S430 in the above embodiment.

[0134] In one embodiment, the second model training module 40 further includes a node classification feature matrix acquisition module, a degree matrix acquisition module, an adjacency matrix acquisition module, and a graph convolutional neural network model construction module. The node classification feature matrix acquisition module is used to obtain the node classification feature matrix of the character detection box corresponding to each single-character image based on the node category of each single-character image in the single-character image dataset. The degree matrix acquisition module is used to obtain the degree matrix of the character detection box corresponding to each single-character image based on the character position of each single-character image in the single-character image dataset.

[0135] The adjacency matrix acquisition module obtains the identity matrix of the character detection boxes corresponding to each single-character image. Based on the identity matrix, node classification feature matrix, and degree matrix, it obtains the adjacency matrix of the character detection boxes corresponding to each single-character image. The graph convolutional neural network model construction module trains the graph convolutional neural network model based on the adjacency matrix of the character detection boxes corresponding to each single-character image and the input feature vector, obtaining the trained graph convolutional neural network model.

[0136] The graph convolutional neural network model is as follows:

[0137] Where X represents the input feature vector of each single character image, A represents the node classification feature matrix of the character detection box corresponding to each single character image, and I represents the identity matrix of the character detection box corresponding to each single character image. This represents the degree matrix of the character detection box corresponding to each single character image. W represents the adjacency matrix of the character detection boxes corresponding to each single character image. (0) With W (1) represents the parameters of the graph convolutional neural network model, softmax represents the normalization function, and ReLU represents the non-linear activation function.

[0138] In this embodiment, the description of the node classification feature matrix acquisition module can be found in the description of S440 in the above embodiment. The description of the degree matrix acquisition module can be found in the description of S450 in the above embodiment. The description of the adjacency matrix acquisition module can be found in the description of S460 in the above embodiment. The description of the graph convolutional neural network model construction module can be found in the description of S470 in the above embodiment.

[0139] In one embodiment, the berth character sequence acquisition module 50 includes a berth character information acquisition module, a test input feature vector acquisition module, a test node classification feature matrix acquisition module, and a berth character sequence generation module. The berth character information acquisition module acquires berth character images and inputs them into a trained berth character detection model to obtain the character category and position of each berth character. The test input feature vector acquisition module extracts and transforms the berth character images based on each character position to obtain multiple test single-character images. It acquires the distance features of each test single-character image based on its position and concatenates the image features of each test single-character image with the distance features to obtain the input feature vector of each test single-character image. The test node classification feature matrix acquisition module inputs the input feature vector of each test single-character image into a trained graph convolutional neural network model and outputs the node classification feature matrix of the character detection box corresponding to each test single-character image. The berth character sequence generation module is used to obtain the berth character sequence based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image.

[0140] In this embodiment, the description of the module for obtaining the character information of the berth under test can be found in the description of S510 in the above embodiment. The description of the module for obtaining the input feature vector under test can be found in the description of S520 in the above embodiment. The description of the module for obtaining the classification feature matrix of the node under test can be found in the description of S530 in the above embodiment. The description of the module for generating the berth character sequence can be found in the description of S540 in the above embodiment.

[0141] In one embodiment, the berth character sequence generation module includes an arrangement module. The arrangement module is used to arrange all the berth characters to be tested in ascending or descending order according to the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image, to obtain the berth character sequence.

[0142] In this embodiment, the relevant description of the arrangement module can be referred to the relevant description of S541 in the above embodiment.

[0143] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to a specific order or hierarchy.

[0144] Those skilled in the art will also understand that the various illustrative logical blocks, modules, and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, modules, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0145] The various illustrative logic blocks or modules described in the embodiments of this application can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0146] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0147] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for recognizing characters in multi-node category roadside parking spaces, characterized in that, include: Obtain a roadside parking space character image dataset, and perform target detection annotation on each parking space character in the roadside parking space character image dataset to obtain an annotated character image dataset; Perform node classification and annotation on each of the berth characters in the annotated character image dataset to obtain a node-annotated character image dataset; Based on the labeled character image dataset, the berth character detection model is trained to obtain the trained berth character detection model; Based on the node-annotated character image dataset, the graph convolutional neural network model is trained to obtain the trained graph convolutional neural network model. Obtain the berth character image to be tested, and identify the berth character image according to the trained berth character detection model and the trained graph convolutional neural network model to obtain the berth character sequence; The node classification further divides the berth character sequence formed by the roadside berth characters. The classification is based on the printing position of the roadside berth characters in the actual roadside scene. The node classification categories include left-side berth characters, right-side berth characters, and middle-side berth characters.

2. The multi-node category roadside parking space character recognition method according to claim 1, characterized in that, The step of performing node classification and annotation on each of the berth characters in the annotated character image dataset to obtain a node-annotated character image dataset includes: Based on the character position of each berth character, the berth characters in the labeled character image dataset are divided into columns or rows to form berth characters of the first node category, berth characters of the second node category, and berth characters of the third node category. Based on the first node category berth character, the second node category berth character, and the third node category berth character, node category annotation is performed on each berth character in the annotated character image dataset to form the node-annotated character image dataset.

3. The multi-node category roadside parking space character recognition method according to claim 2, characterized in that, The step of training a graph convolutional neural network model based on the node-annotated character image dataset to obtain a trained graph convolutional neural network model includes: Based on the character position of each berth character, the node-annotated character image in the node-annotated character image dataset is extracted and transformed to obtain a single-character image dataset; Based on the character position of each berth character, obtain the distance feature of a single character; The image features of each single character image in the single character image dataset are concatenated with the distance features of the single character to obtain the input feature vector of each single character image.

4. The multi-node category roadside parking space character recognition method according to claim 3, characterized in that, The step of training the graph convolutional neural network model based on the node-annotated character image dataset to obtain the trained graph convolutional neural network model further includes: Based on the node category of each single character image in the single character image dataset, obtain the node classification feature matrix of the character detection box corresponding to each single character image; Based on the character position of each single character image in the single character image dataset, obtain the degree matrix of the character detection box corresponding to each single character image; Obtain the identity matrix of the character detection box corresponding to each single character image, and obtain the adjacency matrix of the character detection box corresponding to each single character image based on the identity matrix, the node classification feature matrix and the degree matrix; The graph convolutional neural network model is trained based on the adjacency matrix of the character detection box corresponding to each single character image and the input feature vector to obtain the trained graph convolutional neural network model. The graph convolutional neural network model is as follows: Where X represents the input feature vector of each single-character image, A represents the node classification feature matrix of the character detection box corresponding to each single-character image, and I represents the identity matrix of the character detection box corresponding to each single-character image. The degree matrix represents the character detection box corresponding to each of the single character images. W represents the adjacency matrix of the character detection boxes corresponding to each single character image. (0) With W (1) This represents the parameters of the graph convolutional neural network model. softmax This represents the normalization function, or ReLU nonlinear activation function.

5. The multi-node category roadside parking space character recognition method according to claim 4, characterized in that, The step of acquiring the berth character image to be tested, and recognizing the berth character image to be tested based on the trained berth character detection model and the trained graph convolutional neural network model to obtain the berth character sequence, includes: The character image of the berth to be tested is obtained, and the character image of the berth to be tested is input into the trained berth character detection model to obtain the character category and character position of each character of the berth to be tested; Based on the character position of each character, the character image of the berth to be tested is extracted and transformed to obtain multiple single character images to be tested. The distance feature of the single character to be tested is obtained based on the character position of each character. The image feature of each single character image to be tested is concatenated with the distance feature of the single character to be tested to obtain the input feature vector of each single character image to be tested. The input feature vector of each of the single-character images to be tested is input into the trained graph convolutional neural network model, and the node classification feature matrix of the character detection box corresponding to each single-character image to be tested is output. The berth character sequence is obtained based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image to be tested for each berth character.

6. The multi-node category roadside parking space character recognition method according to claim 5, characterized in that, The step of obtaining the berth character sequence based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image to be tested includes: Based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each character image to be tested, all the characters to be tested are sorted in ascending or descending order to obtain the berth character sequence.

7. A multi-node category roadside parking space character recognition system, characterized in that, include: The data acquisition module is used to acquire a roadside parking space character image dataset, and to perform target detection and annotation on each parking space character in the roadside parking space character image dataset to obtain an annotated character image dataset; The node classification module is used to perform node classification and annotation on each berth character in the labeled character image dataset to obtain a node-labeled character image dataset; the node classification is further divided into berth character sequences formed by roadside berth characters, and the classification is based on the printing position of the roadside berth characters in the actual roadside scene. The node classification categories include left-side category berth characters, right-side category berth characters, and middle category berth characters; The first model training module is used to train the berth character detection model based on the labeled character image dataset to obtain the trained berth character detection model. The second model training module is used to train the graph convolutional neural network model based on the node-annotated character image dataset to obtain the trained graph convolutional neural network model. The berth character sequence acquisition module is used to acquire the berth character image to be tested, and to identify the berth character image to be tested based on the trained berth character detection model and the trained graph convolutional neural network model to obtain the berth character sequence.

8. The multi-node category roadside parking space character recognition system according to claim 7, characterized in that, The node classification module includes: The segmentation module is used to divide each of the berth characters in the labeled character image dataset into columns or rows according to the character position of each berth character, forming berth characters of the first node category, berth characters of the second node category, and berth characters of the third node category. The annotation module is used to annotate each berth character in the annotated character image dataset with a node category according to the first node category berth character, the second node category berth character, and the third node category berth character, thereby forming the node-annotated character image dataset.

9. The multi-node category roadside parking space character recognition system according to claim 8, characterized in that, The second model training module includes: The single-character image acquisition module is used to extract and transform each node-annotated character image in the node-annotated character image dataset according to the character position of each berth character to obtain a single-character image dataset; The distance feature acquisition module is used to acquire the distance feature of a single character based on the character position of each of the berth characters; The input feature vector acquisition module is used to concatenate the image features of each single character image in the single character image dataset with the distance features of the single character to obtain the input feature vector of each single character image.

10. The multi-node category roadside parking space character recognition system according to claim 9, characterized in that, The second model training module also includes: The node classification feature matrix acquisition module is used to obtain the node classification feature matrix of the character detection box corresponding to each single character image based on the node category of each single character image in the single character image dataset. The degree matrix acquisition module is used to obtain the degree matrix of the character detection box corresponding to each single character image based on the character position of each single character image in the single character image dataset; The adjacency matrix acquisition module is used to acquire the identity matrix of the character detection box corresponding to each single character image, and to obtain the adjacency matrix of the character detection box corresponding to each single character image based on the identity matrix, the node classification feature matrix and the degree matrix. The graph convolutional neural network model construction module is used to train the graph convolutional neural network model based on the adjacency matrix of the character detection box corresponding to each single character image and the input feature vector, so as to obtain the trained graph convolutional neural network model. The graph convolutional neural network model is as follows: Where X represents the input feature vector of each single-character image, A represents the node classification feature matrix of the character detection box corresponding to each single-character image, and I represents the identity matrix of the character detection box corresponding to each single-character image. The degree matrix represents the character detection box corresponding to each of the single character images. W represents the adjacency matrix of the character detection boxes corresponding to each single character image. (0) With W (1) This represents the parameters of the graph convolutional neural network model. softmax This represents the normalization function, or ReLU nonlinear activation function.

11. The multi-node category roadside parking space character recognition system according to claim 10, characterized in that, The berth character sequence acquisition module includes: The berth character information acquisition module is used to acquire the character image of the berth to be tested, input the character image of the berth to be tested into the trained berth character detection model, and obtain the character category and character position of each character of the berth to be tested. The module for obtaining the input feature vector to be tested is used to extract and transform the character image of the berth to be tested according to each character position to obtain multiple single character images to be tested, obtain the distance feature of the single character to be tested according to each character position, and concatenate the image feature of each single character image to be tested with the distance feature of the single character to be tested to obtain the input feature vector of each single character image to be tested. The module for obtaining the classification feature matrix of the node to be tested is used to input the input feature vector of each single character image to be tested into the trained graph convolutional neural network model, and output the node classification feature matrix of the character detection box corresponding to each single character image to be tested; The berth character sequence generation module is used to obtain the berth character sequence based on the character category, character position, and node classification feature matrix of the character detection box corresponding to each single character image to be tested for each berth character.

12. The multi-node category roadside parking space character recognition system according to claim 11, characterized in that, The berth character sequence generation module includes: The sorting module is used to sort all the characters of the berth to be tested in ascending or descending order according to the character category, character position and node classification feature matrix of the character detection box corresponding to each single character image to be tested, so as to obtain the berth character sequence.

Images