A road parking license plate recognition method and system based on 5G neural network

CN118865351BActive Publication Date: 2024-11-29NANJING SMART PARKING CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411328443.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-24
Publication Date
2024-11-29
Estimated Expiration
2044-09-24

AI Technical Summary

Technical Problem

The prior art is difficult to achieve fast and accurate license plate recognition on handheld PDA devices, which is limited by poor hardware resources and environmental adaptability, resulting in inefficient parking management.

Method used

Using a 5G neural network-based road parking license plate recognition method, the license plate image is collected through handheld PDA terminals and transmitted to edge computing nodes, the super-resolution model and YOLO model are used for clarity enhancement and positioning, combined with U-Net and federated learning technology License plate recognition realizes real-time transmission of identification information and cloud optimization.

Benefits of technology

It improves the accuracy and real-timeness of license plate recognition, reduces the deployment cost and operation and maintenance difficulty of traditional fixed equipment, and ensures the accuracy of identification and the continuity and reliability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118865351B_ABST
    Figure CN118865351B_ABST
Patent Text Reader

Abstract

The present invention discloses a road parking license plate recognition method and system based on 5G neural network, and relates to the field of road parking technology. The method comprises: collecting license plate images through the camera of a handheld PDA terminal, and transmitting them to an edge computing node deployed in a 5G base station through a 5G network; enhancing the clarity of the license plate image through a super-resolution model at the edge computing node, and inputting the processed license plate image into a trained license plate recognition model for recognition to obtain the recognized license plate information; transmitting the license plate information back to the handheld PDA terminal through the 5G network for visual display, and manually confirming whether the license plate recognition is correct. The present invention combines 5G technology and edge computing to achieve real-time acquisition and rapid transmission of license plate images, uses U‑Net and YOLO models to improve recognition accuracy, introduces federated learning to protect data privacy, and designs local cache and offline models to ensure system continuity and reliability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of road parking technology, and specifically to a road parking license plate recognition method and system based on 5G neural network. Background Art

[0002] In road parking scenarios where vehicles frequently enter and exit, road parking managers are usually equipped with handheld PDA (Personal Digital Assistant) devices for collecting, processing and uploading license plate information. The algorithm principle of license plate recognition is to locate, segment, identify and verify vehicle license plates through image processing technology and pattern recognition methods. First, the vehicle image is collected through the camera, and the image is preprocessed to remove noise and enhance image quality; secondly, the license plate is located and separated from the background through edge detection, color segmentation and other technologies; then, the separated license plate is segmented and each character is recognized; finally, the recognized license plate information is compared with the standard license plate in the database to achieve automatic recognition and identity authentication of the vehicle.

[0003] The license plate area accounts for a very small proportion of the entire image, and the color, size, and position of the license plate are uncertain. The recognition distance must take into account the safe shooting distance of the toll collector (about 3 meters) and the close-range recognition requirements of some sections with tight parking spaces (about 1 meter). The positioning algorithm must be able to overcome the influence of complex environments (such as night, rainy days, weak light, strong light, license plate damage, occlusion, deformation, etc.), and must also take into account accuracy and real-time performance. However, PDA equipment has limited hardware resources, and traditional license plate recognition algorithms have low operating efficiency on these devices. The recognition speed and accuracy are difficult to meet actual needs, resulting in low parking management efficiency and prone to disputes and management loopholes. Therefore, it is difficult to quickly and accurately locate license plates on PDA devices.

[0004] In recent years, the development of neural network technology, especially deep learning, has brought new solutions to license plate recognition. Deep learning models can automatically learn and extract complex features of license plates through multi-layer neural network structures, and have high recognition accuracy and generalization capabilities. However, the efficient operation of deep learning models usually requires powerful computing resources and high-speed data transmission. Due to the limitations of hardware performance, handheld PDA devices are difficult to meet these requirements in traditional network environments, resulting in a significant reduction in license plate recognition speed and accuracy.

[0005] With the popularization of 5G communication technology, its high bandwidth, low latency and large-scale connection characteristics provide technical support for the application of handheld PDA terminals in license plate recognition. The high speed and low latency characteristics of the 5G network enable handheld PDA devices to receive, process and upload large amounts of image data in real time, while using the powerful computing power of remote servers to perform complex neural network model reasoning and training, thereby greatly improving the speed and accuracy of license plate recognition.

[0006] Therefore, how to combine 5G communication technology and neural network technology on handheld PDA devices to develop a flexible and efficient license plate recognition system to improve the speed and accuracy of license plate recognition has become an urgent problem to be solved. Summary of the invention

[0007] In view of the problems of limited hardware resources of handheld PDA devices and poor environmental adaptability of license plate recognition algorithms, the present invention proposes a road parking license plate recognition method and system based on 5G neural network, which can not only improve the accuracy and real-time performance of recognition, but also reduce the deployment cost and operation and maintenance difficulty of traditional fixed equipment, and is particularly suitable for road parking scenarios.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for road parking license plate recognition based on 5G neural network, the method comprising:

[0010] The vehicle license plate is photographed or video scanned by the camera of the handheld PDA terminal to collect the license plate image;

[0011] Transmitting the license plate image to an edge computing node deployed in a 5G base station via a 5G network;

[0012] At the edge computing node, the license plate image is enhanced in clarity by using a super-resolution model, and the processed license plate image is input into a trained license plate recognition model for recognition to obtain recognized license plate information;

[0013] The license plate information is transmitted back to the handheld PDA terminal via the 5G network for visual display, and manual confirmation is made as to whether the license plate recognition is correct. If the recognition is wrong, the recognition is performed again, and the error information is reported to the cloud for subsequent model training and optimization.

[0014] As a preferred solution of the present invention, the method for collecting license plate images includes:

[0015] Enable low-light enhancement, autofocus, and anti-shake functions, continuously capture multiple frames of vehicle images containing license plate information in dynamic scenes, and use image denoising and multi-frame synthesis algorithms to generate a single-frame vehicle image;

[0016] Use the pre-trained YOLO model to detect the license plate position in real time on the vehicle image, automatically mark and output the bounding box coordinates of the license plate. The formula is:

[0017] ;

[0018] Where (x, y) is the actual center coordinate of the bounding box, w and h are the actual width and height of the bounding box respectively; is the output value of the YOLO model, indicating the offset of the center point of the bounding box relative to the upper left corner of the grid cell , and the logarithmic offset of the width and height of the predicted box relative to the anchor box ; , is an exponential function used to convert the logarithmic offset of the model prediction into a positive scaling factor; Represents the width and height of the predefined anchor box; Represents the absolute coordinates of the upper left corner of the grid cell, which is used to determine the position of the predicted bounding box relative to the entire image; To use the Sigmoid function to limit the prediction offset between 0 and 1;

[0019] The bounding box coordinates of the license plate are used as input data, and the license plate image is cropped from the single-frame vehicle image using a U-Net segmentation model based on MobileNetV2.

[0020] As a preferred solution of the present invention, the definition of the license plate image is enhanced by a super-resolution model, and the implementation method includes:

[0021] The collected license plate images are standardized and scaled, and a convolutional neural network is used to preprocess the license plate images. LR Perform feature extraction and generate the initial feature map F LR , the formula is:

[0022] ;

[0023] In the formula, is the convolution kernel weight, is the bias term, represents the convolution operation, is the activation function;

[0024] The feature map F LR The input is sent to the feature enhancement module that includes triple attention of channel attention, spatial attention, pixel attention and residual connection blocks. The character and layout features in the license plate image are enhanced through self-attention mechanism and residual connection, and the variable convolution layer is integrated. According to the feature map F LRThe shape and position of the convolution kernel are dynamically adjusted based on the content of the convolution kernel, and the enhanced feature map F is output. LA ;

[0025] The enhanced feature map F LA Input character feature upsampling module, use deconvolution operation for upsampling, and output super-resolution license plate image I SR , the formula is:

[0026] ;

[0027] In the formula, is the deconvolution kernel weight, is the bias term;

[0028] The training strategy of generative adversarial network is adopted. The generator is used to generate super-resolution license plate images. The discriminator is an OCR model that gradually adjusts and updates the convolution kernel weights. , autonomy weights and deconvolution kernel weights , optimize the super-resolution model.

[0029] As a preferred solution of the present invention, a composite loss function L is defined for the super-resolution model: 总 , until the composite loss function L 总 Converges to the preset threshold range, the composite loss function L 总 The calculation formula is:

[0030] ;

[0031] in,

[0032] ;

[0033] L P = ∑ n = 1 N [ D ( x n ) ⋅ A ( y n ) + A ( x n ) ⋅ D ( y n ) ] ;

[0034] ;

[0035] In the formula, is the cross entropy loss of character classification, which is used to represent the classification error of the model for license plate characters; N represents the total number of characters contained in the license plate image; is the weight of the nth character; is the input feature, is the true label of the nth character; For a given input When the nth character The predicted probability of

[0036] is the character positioning loss, which is used to represent the error of the model in character positioning; among them, the function and They are used to calculate whether the character type is a number or a letter. If it is a number or a letter, the value is , otherwise the value is 0; is a predefined penalty coefficient;

[0037] is the structural similarity loss, which is used to represent the structural similarity error between the generated image and the real image; and are the pixel blocks of the generated image and the real image respectively; is the structural similarity index used to measure the generated image With real image The structural similarity between them.

[0038] As a preferred solution of the present invention, the license plate recognition model is trained and updated in the cloud, and the edge computing node downloads the license plate recognition model from the cloud for license plate recognition; the construction process of the license plate recognition model includes:

[0039] Vehicle image data is collected from multiple data sources to form a dataset. In the federated learning environment, the total number of images is divided according to the number of clients to ensure that each client uses a different data subset during local training. The number of clients represents the number of devices or nodes participating in the federated learning training.

[0040] Use data augmentation techniques to expand each client’s dataset by scaling, normalizing, horizontally and vertically offsetting, and centering the original images in the dataset.

[0041] The definition of the expanded image is enhanced through a super-resolution model to generate a high-resolution image;

[0042] Use the U-Net model to segment the license plate image from the high-resolution image, use the Tesseract OCR model to binarize and extract features for each character in the segmented license plate image, and use the convolutional neural network to achieve character classification;

[0043] The Dice similarity coefficient is used as a metric to evaluate image segmentation performance, and the formula is:

[0044] ;

[0045] In the formula, is the Dice similarity coefficient; is the real mask pixel, To predict the mask pixels, is a constant to prevent the denominator from being 0;

[0046] The F1 score is used to evaluate the classification results. The formula is:

[0047] ;

[0048] In the formula, P is the precision and R is the recall.

[0049] As a preferred solution of the present invention, the scaling, feature normalization, horizontal and vertical offset, and feature centering of the original image in the data set are specifically:

[0050] Scaling: Define the scaling factor s∈[0.8,1.2] to randomly scale the image size;

[0051] Feature standardization: Use mean μ and standard deviation δ to standardize the image pixel values;

[0052] Offset: Randomly generate offset values ​​(Δx, Δy) and apply them to image coordinates to achieve random cropping;

[0053] Feature Centering: Symmetrically adjust each image so that the center of each image is aligned to a fixed reference point.

[0054] As a preferred solution of the present invention, the U-Net model includes an encoder and a decoder, and the specific implementation steps are:

[0055] The encoder part gradually downsamples the input image through convolutional layers and pooling layers to generate a set of feature maps;

[0056] The decoder part gradually restores the image spatial resolution through upsampling layers and skip connections and generates segmentation masks;

[0057] The federated learning method is used for decentralized training. i Aggregate and update the global model W, the formula is:

[0058] ;

[0059] Where G is the number of clients participating in the training, W j represents the local model parameters of the j-th client.

[0060] As a preferred solution of the present invention, in the 5G network, HEVC / H.265 or JPEG XL is used to compress the license plate image, the TLS / SSL encryption protocol is used to ensure the security of data during transmission, and the QUIC or HTTP / 3 efficient transmission protocol is used to reduce transmission delay.

[0061] As a preferred solution of the present invention, the handheld PDA terminal is provided with a local caching mechanism. When the network is unstable or disconnected, the collected license plate image is temporarily saved locally, and the mode is automatically switched to offline recognition mode. The locally cached license plate image is recognized offline through a lightweight offline recognition model. When the network is restored, the retry mechanism is automatically or manually triggered to upload the cached license plate image to the edge computing node for processing.

[0062] A road parking license plate recognition system based on a 5G neural network, based on the road parking license plate recognition method based on a 5G neural network as described above, the system comprises:

[0063] A camera is used to take a photo or video scan of the vehicle license plate to collect the license plate image;

[0064] 5G communication module, used for data communication between handheld PDA terminals and edge computing nodes;

[0065] The edge computing node is used to enhance the clarity of the license plate image by using a super-resolution model, and input the processed license plate image into a trained license plate recognition model for recognition to obtain recognized license plate information;

[0066] A cloud server for training and updating the license plate recognition model;

[0067] The local cache module is used to temporarily save the collected license plate images locally when the network is unstable or disconnected;

[0068] The offline recognition module is used in offline recognition mode to perform offline recognition on the locally cached license plate images through a lightweight offline recognition model.

[0069] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: by utilizing 5G communication technology and edge computing nodes, real-time acquisition and rapid transmission of license plate image data are realized, and preliminary processing (such as super-resolution enhancement and YOLO model detection) is performed on the edge computing nodes, reducing dependence on the cloud, reducing data transmission delays and bandwidth consumption; by working together between edge computing nodes and cloud servers, the U-Net segmentation model is used to accurately segment and identify the license plate area, thereby improving recognition accuracy; by introducing a federated learning module, data privacy protection is achieved in a distributed environment, the risk of data leakage is avoided, and continuous optimization of the recognition model is ensured; a local cache module is designed to store license plate image data and its recognition results, and a lightweight offline recognition model is introduced, which is used for offline recognition when the network is unavailable, thereby ensuring the continuity and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0071] Figure 1 is a flow chart of the method of the present invention;

[0072] Figure 2 It is a system modular structure diagram of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0074] like Figure 1 As shown in FIG. 1 , an embodiment of the present invention is provided, which provides a method for road parking license plate recognition based on a 5G neural network, comprising the following steps:

[0075] S1: Take a photo or video scan of the vehicle license plate through the camera of the handheld PDA terminal to collect the license plate image;

[0076] In one embodiment, step S1 comprises:

[0077] S11: Enables low-light enhancement, autofocus, and anti-shake functions, continuously captures multiple frames of vehicle images containing license plate information in dynamic scenes, and uses image denoising and multi-frame synthesis algorithms to generate a single-frame vehicle image;

[0078] The low-light enhancement function can improve the image quality in low-light conditions, ensuring that the license plate information is clearly visible; the autofocus and anti-shake functions ensure clear images and reduce blur caused by vibration of the handheld device; in dynamic scenes, continuous shooting can capture the best license plate image frames, increasing the chance of capturing clear license plate images, especially under moving conditions; image denoising and multi-frame synthesis algorithms make the generated single-frame images clearer, which helps the subsequent license plate detection and recognition accuracy;

[0079] S12: Use the pre-trained YOLO model to detect the license plate position in real time on the vehicle image, automatically mark and output the bounding box coordinates of the license plate. The formula is:

[0080] ;

[0081] Where (x, y) is the actual center coordinate of the bounding box, w and h are the actual width and height of the bounding box respectively; is the output value of the YOLO model, indicating the offset of the center point of the bounding box relative to the upper left corner of the grid cell , and the logarithmic offset of the width and height of the predicted box relative to the anchor box ; , is an exponential function used to convert the logarithmic offset of the model prediction into a positive scaling factor; Represents the width and height of the predefined anchor box; Represents the absolute coordinates of the upper left corner of the grid cell, which is used to determine the position of the predicted bounding box relative to the entire image; To use the Sigmoid function to limit the prediction offset between 0 and 1;

[0082] S13: Using the bounding box coordinates of the license plate as input data, a U-Net segmentation model based on MobileNetV2 is used to crop the license plate image from a single frame of the vehicle image;

[0083] MobileNetV2 is a lightweight network suitable for running on mobile devices; U-Net is an effective image segmentation network. Combining the two can accurately segment license plate images while maintaining efficient computing, and is suitable for running on resource-constrained PDA devices.

[0084] S2: Transmit the license plate image to the edge computing node deployed in the 5G base station through the 5G network;

[0085] Specifically, in 5G networks, HEVC / H.265 or JPEG XL is used to compress license plate images, TLS / SSL encryption protocols are used to ensure data security during transmission, and QUIC or HTTP / 3 efficient transmission protocols are used to reduce transmission delays;

[0086] Considering that the device performance of handheld PDA terminals may be limited, and clarity enhancement usually requires strong computing power, it is more appropriate to set the clarity enhancement on the edge computing nodes deployed in the 5G base stations. In this way, the efficient computing power of the edge computing nodes can be used to enhance the clarity of the license plate images, while avoiding excessive high performance requirements for the PDA terminals.

[0087] S3: At the edge computing node, the license plate image is enhanced in clarity through a super-resolution model, and the processed license plate image is input into a trained license plate recognition model for recognition to obtain the recognized license plate information;

[0088] The license plate recognition model is trained and updated in the cloud, and the edge computing node downloads the license plate recognition model from the cloud for license plate recognition;

[0089] Specifically, the clarity of the license plate image is enhanced by a super-resolution model, and the implementation method includes:

[0090] S31: Standardize and resize the collected license plate images to eliminate image variations under different capture conditions and ensure the generalization ability of the model; use a convolutional neural network to perform preprocessed license plate images I LR Perform feature extraction, deeply explore the essential features of the image, and generate the initial feature map F LR , the formula is:

[0091] ;

[0092] In the formula, is the convolution kernel weight, is the bias term, represents the convolution operation, is the activation function;

[0093] S32: The feature map F LR The input is sent to the feature enhancement module which includes triple attention and residual connection blocks including channel attention, spatial attention and pixel attention. The character and layout features in the license plate image, such as character edges and contrast, are enhanced through self-attention mechanism and residual connection. The variable convolution layer is integrated to obtain the feature map F. LR The shape and position of the convolution kernel are dynamically adjusted based on the content of the convolution kernel, and the enhanced feature map F is output. LA ;

[0094] S33: The enhanced feature map F LA The character feature upsampling module is input and upsampled using the deconvolution operation. Deconvolution can effectively enlarge the image size while increasing the image details and clarity, making the characters on the license plate easier to be accurately recognized by the subsequent OCR model, recovering high-resolution details from the low-resolution license plate image, and outputting a super-resolution license plate image I SR , the formula is:

[0095] ;

[0096] In the formula, is the deconvolution kernel weight, is the bias term;

[0097] S34: Using the training strategy of generative adversarial network, the generator is used to generate super-resolution license plate images, and the discriminator is an OCR model that gradually adjusts and updates the convolution kernel weights. , autonomy weights and deconvolution kernel weights , optimize the super-resolution model;

[0098] GAN can generate high-quality images that are very close to real license plates. At the same time, the adversarial nature of the training process enables the model to continuously improve its performance, improving the realism and practicality of super-resolution images.

[0099] S35: Define a composite loss function L for the super-resolution model 总 , until the composite loss function L 总 Converges to the preset threshold range, the composite loss function L 总 The calculation formula is:

[0100] ;

[0101] in,

[0102] ;

[0103] L P = ∑ n = 1 N [ D ( x n ) ⋅ A ( y n ) + A ( x n ) ⋅ D ( y n ) ] ;

[0104] ;

[0105] In the formula, is the cross entropy loss of character classification, which is used to represent the classification error of the model for license plate characters; N represents the total number of characters contained in the license plate image; is the weight of the nth character; is the input feature, is the true label of the nth character; For a given input When the nth character The predicted probability of

[0106] is the character positioning loss, which is used to represent the error of the model in character positioning; among them, the function and They are used to calculate whether the character type is a number or a letter. If it is a number or a letter, the value is , otherwise the value is 0; is a predefined penalty coefficient;

[0107] is the structural similarity loss, which is used to represent the structural similarity error between the generated image and the real image; and are the pixel blocks of the generated image and the real image respectively; is the structural similarity index used to measure the generated image With real image The closer the value is to 1, the more similar it is. The structural difference is obtained by calculating 1-SSIM and then divided by 2 to ensure that the loss range is between [0, 0.5].

[0108] Furthermore, the construction process of the license plate recognition model includes:

[0109] The vehicle image data is collected from multiple data sources to form a data set. In the federated learning environment, the total number of images is divided according to the number of clients to ensure that each client uses a different data subset during local training. The number of clients represents the number of devices or nodes participating in the federated learning training. By dividing the data according to the number of clients in the federated learning environment, the diversity and independence of each client's training data are ensured, reducing the risk of information leakage. Through multi-node collaborative learning, the generalization ability of the model is improved.

[0110] In order to improve the accuracy of the model and prevent overfitting, data augmentation techniques are used to expand the dataset of each client. The original images in the dataset are scaled, feature normalized, horizontally and vertically offset, and feature centered. Specifically:

[0111] Scaling: Define the scaling factor s∈[0.8,1.2] to randomly scale the image size;

[0112] Feature standardization: Use mean μ and standard deviation δ to standardize the image pixel values;

[0113] Offset: Randomly generate offset values ​​(Δx, Δy) and apply them to image coordinates to achieve random cropping;

[0114] Feature centering: symmetrically adjust each image so that the center of each image is aligned with a fixed reference point;

[0115] The definition of the expanded image is enhanced through a super-resolution model to generate a high-resolution image;

[0116] The U-Net model is used to segment the license plate image from the high-resolution image. U-Net performs well in processing low-light and irregular license plate images, significantly improving the recognition accuracy. In order to protect privacy, the Federated Learning (FL) method is used. This method performs decentralized model training on the customer device, and the data does not leave the device, thus ensuring data privacy.

[0117] The U-Net model includes an encoder and a decoder. The specific implementation steps are as follows:

[0118] The encoder part gradually downsamples the input image through convolutional layers and pooling layers to generate a set of feature maps;

[0119] The decoder part gradually restores the image spatial resolution through upsampling layers and skip connections and generates segmentation masks;

[0120] The federated learning method is used for decentralized training. i Aggregate and update the global model W, the formula is:

[0121] ;

[0122] Where G is the number of clients participating in the training, W j represents the local model parameters of the j-th client.

[0123] The encoder extracts image features through convolutional layers and pooling layers, while the decoder restores detailed information of the image through upsampling and skip connections. This structure can effectively capture the important features of the license plate image and keep the boundaries clear during image segmentation.

[0124] By combining the U-Net model with the federated learning method, the localization and memory allocation problems existing in YOLO V3 and YOLO V4 are overcome, especially the accuracy of license plate recognition under night conditions is improved;

[0125] Binarize and extract features of each character in the segmented license plate image through the OCR model, and classify the characters through the convolutional neural network.

[0126] The Dice similarity coefficient is used as a metric to evaluate image segmentation performance, and the formula is:

[0127] ;

[0128] In the formula, is the Dice similarity coefficient; is the real mask pixel, To predict the mask pixels, is a constant to prevent the denominator from being 0;

[0129] DSC is often used to measure the overlap accuracy of image segmentation and is suitable for segmentation tasks of objects of uneven sizes.

[0130] The F1 score is used to evaluate the classification results. The formula is:

[0131] ;

[0132] In the formula, P is the precision and R is the recall.

[0133] Comprehensive consideration of precision and recall helps optimize model performance, balance missed detections and false detections, and improve recognition accuracy.

[0134] S4: The license plate information is transmitted back to the handheld PDA terminal via the 5G network for visual display, and manual confirmation is made as to whether the license plate recognition is correct;

[0135] If there is a recognition error, the recognition will be performed again and the error information will be reported to the cloud for subsequent model training and optimization.

[0136] Preferably, the handheld PDA terminal in this embodiment is provided with a local cache mechanism. When the network is unstable or disconnected, the collected license plate image is temporarily saved locally, and the mode is automatically switched to offline recognition mode. The locally cached license plate image is offline recognized by a lightweight offline recognition model. Compared with the main recognition model (such as the YOLO model), the model has the characteristics of small computational complexity and low resource consumption. Therefore, when the network is unavailable, the limited computing resources of the local device can be used for license plate recognition to ensure the continuous operation of the system.

[0137] When the network is restored, the retry mechanism is triggered automatically or manually to upload the cached license plate image to the edge computing node for processing, ensuring that the cached data can continue to be used when the network is restored, reducing repeated acquisition and processing of data.

[0138] like Figure 2 As shown, another embodiment of the present invention provides a road parking license plate recognition system based on a 5G neural network, based on the above-mentioned road parking license plate recognition method based on a 5G neural network, specifically including:

[0139] A camera is used to take a photo or video scan of the vehicle license plate to collect the license plate image;

[0140] 5G communication module, used for data communication between handheld PDA terminals and edge computing nodes;

[0141] The edge computing node is used to enhance the clarity of the license plate image by using a super-resolution model, and input the processed license plate image into a trained license plate recognition model for recognition to obtain recognized license plate information;

[0142] A cloud server for training and updating the license plate recognition model;

[0143] The local cache module is used to temporarily save the collected license plate images locally when the network is unstable or disconnected;

[0144] The offline recognition module is used in offline recognition mode to perform offline recognition on the locally cached license plate images through a lightweight offline recognition model.

[0145] In summary, the present invention utilizes 5G communication technology and edge computing nodes to realize real-time acquisition and rapid transmission of license plate image data, and performs preliminary processing (such as super-resolution enhancement and YOLO model detection) on the edge computing nodes, thereby reducing dependence on the cloud, reducing data transmission delays and bandwidth consumption; by working together between edge computing nodes and cloud servers, the U-Net segmentation model is used to accurately segment and identify the license plate area, thereby improving recognition accuracy; by introducing a federated learning module, data privacy protection is achieved in a distributed environment, the risk of data leakage is avoided, and continuous optimization of the recognition model is ensured; a local cache module is designed to store license plate image data and its recognition results, and a lightweight offline recognition model is introduced, which is used for offline recognition when the network is unavailable, thereby ensuring the continuity and reliability of the system.

[0146] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0147] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0148] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A road parking license plate recognition method based on 5G neural network, characterized in that: The method comprises: The vehicle license plate is photographed or video scanned by the camera of the handheld PDA terminal to collect the license plate image; Transmitting the license plate image to an edge computing node deployed in a 5G base station via a 5G network; At the edge computing node, the license plate image is enhanced in clarity by using a super-resolution model, and the processed license plate image is input into a trained license plate recognition model for recognition to obtain recognized license plate information; The method of enhancing the clarity of the license plate image by using a super-resolution model includes: The collected license plate images are standardized and scaled, and a convolutional neural network is used to preprocess the license plate images. LR Perform feature extraction and generate the initial feature map F LR , the formula is: ; In the formula, is the convolution kernel weight, is the bias term, represents the convolution operation, is the activation function; The feature map F LR The input is sent to the feature enhancement module that includes triple attention of channel attention, spatial attention, pixel attention and residual connection blocks. The character and layout features in the license plate image are enhanced through self-attention mechanism and residual connection, and the variable convolution layer is integrated. According to the feature map F LR The shape and position of the convolution kernel are dynamically adjusted based on the content of the convolution kernel, and the enhanced feature map F is output. LA ; The enhanced feature map F LA Input character feature upsampling module, use deconvolution operation for upsampling, and output super-resolution license plate image I SR , the formula is: ; In the formula, is the deconvolution kernel weight, is the bias term; The training strategy of generative adversarial network is adopted. The generator is used to generate super-resolution license plate images. The discriminator is an OCR model that gradually adjusts and updates the convolution kernel weights. , autonomy weights and deconvolution kernel weights , optimize the super-resolution model; The license plate information is transmitted back to the handheld PDA terminal via the 5G network for visual display, and manual confirmation is made as to whether the license plate recognition is correct. If the recognition is wrong, the recognition is performed again, and the error information is reported to the cloud for subsequent model training and optimization.

2. According to the method of claim 1, the method is characterized in that: The method for collecting license plate images comprises: Enable low-light enhancement, autofocus, and anti-shake functions, continuously capture multiple frames of vehicle images containing license plate information in dynamic scenes, and use image denoising and multi-frame synthesis algorithms to generate a single-frame vehicle image; Use the pre-trained YOLO model to detect the license plate position in real time on the vehicle image, automatically mark and output the bounding box coordinates of the license plate. The formula is: ; Where (x, y) is the actual center coordinate of the bounding box, w and h are the actual width and height of the bounding box respectively; The output value of the YOLO model, which indicates the offset of the center point of the bounding box relative to the upper left corner of the grid cell. , and the logarithmic offset of the width and height of the predicted box relative to the anchor box ; , is an exponential function used to convert the logarithmic offset of the model prediction into a positive scaling factor; Represents the width and height of the predefined anchor box; Represents the absolute coordinates of the upper left corner of the grid cell, which is used to determine the position of the predicted bounding box relative to the entire image; To use the Sigmoid function to limit the prediction offset between 0 and 1; The bounding box coordinates of the license plate are used as input data, and the license plate image is cropped from the single-frame vehicle image using a U-Net segmentation model based on MobileNetV2.

3. According to the method of claim 1, the method is characterized in that: The composite loss function L is defined for the super-resolution model 总 , until the composite loss function L 总 Converges to the preset threshold range, the composite loss function L 总 The calculation formula is: ; in, ; ; ; In the formula, is the cross entropy loss of character classification, which is used to represent the classification error of the model for license plate characters; N represents the total number of characters contained in the license plate image; is the weight of the nth character; is the input feature, is the true label of the nth character; For a given input When the nth character The predicted probability of is the character positioning loss, which is used to represent the error of the model in character positioning; among them, the function and They are used to calculate whether the character type is a number or a letter. If it is a number or a letter, the value is , otherwise the value is 0; is a predefined penalty coefficient; is the structural similarity loss, which is used to represent the structural similarity error between the generated image and the real image; and are the pixel blocks of the generated image and the real image respectively; is the structural similarity index used to measure the generated image With real image The structural similarity between them.

4. According to claim 3, a method for road parking license plate recognition based on 5G neural network is characterized in that: The license plate recognition model is trained and updated in the cloud, and the edge computing node downloads the license plate recognition model from the cloud for license plate recognition; The construction process of the license plate recognition model includes: Vehicle image data is collected from multiple data sources to form a dataset. In the federated learning environment, the total number of images is divided according to the number of clients to ensure that each client uses a different data subset during local training. The number of clients represents the number of devices or nodes participating in the federated learning training. Use data augmentation techniques to expand each client’s dataset by scaling, normalizing, horizontally and vertically offsetting, and centering the original images in the dataset. The definition of the expanded image is enhanced through a super-resolution model to generate a high-resolution image; Use the U-Net model to segment the license plate image from the high-resolution image, use the Tesseract OCR model to binarize and extract features for each character in the segmented license plate image, and use the convolutional neural network to achieve character classification; The Dice similarity coefficient is used as a metric to evaluate image segmentation performance, and the formula is: ; In the formula, is the Dice similarity coefficient; is the real mask pixel, To predict the mask pixels, is a constant to prevent the denominator from being 0; The F1 score is used to evaluate the classification results. The formula is: ; In the formula, P is the precision and R is the recall.

5. According to claim 4, a method for road parking license plate recognition based on 5G neural network is characterized in that: The original images in the data set are scaled, feature normalized, horizontally and vertically offset, and feature centered, specifically: Scaling: Define the scaling factor s∈[0.8,1.2] to randomly scale the image size; Feature standardization: Use mean μ and standard deviation δ to standardize the image pixel values; Offset: Randomly generate offset values ​​(Δx, Δy) and apply them to image coordinates to achieve random cropping; Feature Centering: Symmetrically adjust each image so that the center of each image is aligned to a fixed reference point.

6. A method for road parking license plate recognition based on 5G neural network according to claim 4, characterized in that: The U-Net model includes an encoder and a decoder, and the specific implementation steps are: The encoder part gradually downsamples the input image through convolutional layers and pooling layers to generate a set of feature maps; The decoder part gradually restores the image spatial resolution through upsampling layers and skip connections and generates segmentation masks; The federated learning method is used for decentralized training. i Aggregate and update the global model W, the formula is: ; Where G is the number of clients participating in the training, W j represents the local model parameters of the j-th client.

7. The method for road parking license plate recognition based on 5G neural network according to claim 1 is characterized in that: In the 5G network, HEVC / H.265 or JPEG XL is used to compress the license plate image, the TLS / SSL encryption protocol is used to ensure the security of data during transmission, and the QUIC or HTTP / 3 efficient transmission protocol is used to reduce transmission delay.

8. The method for road parking license plate recognition based on 5G neural network according to claim 1 is characterized in that: The handheld PDA terminal is equipped with a local cache mechanism. When the network is unstable or disconnected, the collected license plate image is temporarily saved locally, and the mode is automatically switched to offline recognition mode. The locally cached license plate image is recognized offline through a lightweight offline recognition model. When the network is restored, the retry mechanism is automatically or manually triggered to upload the cached license plate image to the edge computing node for processing.

9. A recognition system for a road parking license plate recognition method based on a 5G neural network according to any one of claims 1 to 8, characterized in that: The system comprises: A camera is used to take a photo or video scan of the vehicle license plate to collect the license plate image; 5G communication module, used for data communication between handheld PDA terminals and edge computing nodes; The edge computing node is used to enhance the clarity of the license plate image by using a super-resolution model, and input the processed license plate image into a trained license plate recognition model for recognition to obtain recognized license plate information; A cloud server for training and updating the license plate recognition model; The local cache module is used to temporarily save the collected license plate images locally when the network is unstable or disconnected; The offline recognition module is used in offline recognition mode to perform offline recognition on the locally cached license plate images through a lightweight offline recognition model.

Citation Information

Patent Citations

  • The invention discloses an intelligent supervisory system and supervisory method for a construction site mud-head vehicle

    CN109829450A

  • License plate positioning and recognition method based on YOLO model

    CN110781882A