License plate recognition method and system based on feature fusion segmentation and splicing

Abstract

The invention discloses a license plate recognition method and system based on feature fusion segmentation and splicing, and relates to the technical field of image recognition. The method comprises the following steps: constructing a license plate recognition network which comprises a feature extraction module, a feature fusion module, a feature segmentation and splicing module and a recognition sequence output module; preprocessing the collected license plate image to obtain a standard image; performing multi-level feature extraction through a feature extraction module to obtain a multi-feature sequence; a feature fusion module performs splicing and fusion in a channel dimension to generate a fusion feature map; performing sequence segmentation and splicing through a feature segmentation and splicing module to obtain a recombined feature map; and inputting the recombined feature map into a recognition sequence output module, and outputting a license plate recognition result. The technical problem that the recognition accuracy and generalization performance are insufficient due to the fact that the license plate recognition technology cannot effectively capture the correlation features between the uplink characters and the downlink characters when facing double-layer license plates is solved, and the technical effect of improving the accuracy and robustness of license plate recognition through multi-level feature extraction and intelligent feature fusion is achieved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a license plate recognition method and system based on feature fusion, segmentation, and stitching. Background Technology

[0002] With the large-scale application of intelligent transportation, urban security, electronic toll collection, and smart parking, license plate recognition, as a core technology connecting the sensing end and the business end, is required to maintain high accuracy and real-time performance in more diverse and complex real-world environments. However, license plates in real-world scenarios are not always regular single-layer blue plates; there are a large number of double-layer license plates, special license plates, license plates of different colors and specifications, and license plate images affected by factors such as changes in lighting, dirt occlusion, shooting angle shifts, and insufficient equipment clarity. Existing mainstream deep learning-based license plate recognition methods are mostly designed around single-line character layouts, lacking the ability to model the correlation features between the upper and lower lines of double-layer license plates. They often can only recognize the upper and lower lines separately or treat them as independent sequences, resulting in a significant decrease in recognition accuracy in scenarios such as double-layer license plates and yellow plates. At the same time, differences in character size, line spacing, and background interference in different license plates also make it difficult for single-scale feature extraction networks to simultaneously take into account local details and overall structure, resulting in insufficient feature representation. In addition, complex traffic environments also present interference such as uneven lighting, reflection, blurring, occlusion, and dirt, which makes the recognition system need to have stronger feature robustness and generalization ability. Summary of the Invention

[0003] This application provides a license plate recognition method and system based on feature fusion, segmentation and splicing, which solves the technical problem of insufficient recognition accuracy and generalization performance when license plate recognition technology faces dual-layer license plates because it cannot effectively capture the correlation features between characters in the upper and lower lines.

[0004] The first aspect of this application provides a license plate recognition method based on feature fusion, segmentation, and stitching, the method comprising:

[0005] A license plate recognition network is constructed, comprising a feature extraction module, a feature fusion module, a feature segmentation and stitching module, and a recognition sequence output module. The acquired license plate image is preprocessed to obtain a standard license plate image. Multi-level feature extraction is performed on the standard license plate image using the feature extraction module to obtain a multi-feature sequence of the license plate image. The multi-feature sequence of the license plate image is then stitched and fused along the channel dimension using the feature fusion module to obtain a fused license plate feature map. The fused license plate feature map is then segmented and stitched using the feature segmentation and stitching module to obtain a reconstructed license plate feature map. The reconstructed license plate feature map is then input into the recognition sequence output module to obtain the license plate recognition result.

[0006] A second aspect of this application provides a license plate recognition system based on feature fusion, segmentation, and stitching, the system comprising:

[0007] The system comprises the following components: a license plate recognition network construction unit (including a feature extraction module, a feature fusion module, a feature segmentation and stitching module, and a recognition sequence output module); an image preprocessing unit (preprocessing the acquired license plate image to obtain a standard license plate image); a feature extraction unit (performing multi-level feature extraction on the standard license plate image using the feature extraction module to obtain a multi-feature sequence of the license plate image); a feature fusion unit (performing channel-dimensional stitching and fusion of the multi-feature sequence of the license plate image using the feature fusion module to obtain a fused license plate feature map); a feature segmentation unit (performing sequence feature segmentation and stitching of the fused license plate feature map using the feature segmentation and stitching module to obtain a reconstructed license plate feature map); and a license plate recognition unit (inputting the reconstructed license plate feature map into the recognition sequence output module to obtain the license plate recognition result).

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] First, the acquired license plate images are preprocessed to standardize their size and brightness, resulting in a standardized input. Then, a feature extraction module performs multi-level feature extraction, capturing everything from global structure to local character details, forming a multi-feature sequence. Next, a feature fusion module stitches and fuses features from different levels along the channel dimension, generating a more comprehensive fused feature map. Then, a feature segmentation and stitching module segments and reassembles the fused feature map, unfolding the up and down features of a double-layer license plate into a single-layer sequence, achieving structural reconstruction. Finally, a recognition sequence output module maps the reassembled feature map to the character space, outputting the complete license plate recognition result, achieving high-precision recognition of single-layer, double-layer, and complex license plates. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic flowchart of the license plate recognition method based on feature fusion, segmentation, and stitching provided in the embodiments of this application.

[0012] Figure 2 This is a schematic diagram of the structure of a license plate recognition system based on feature fusion, segmentation, and stitching provided in an embodiment of this application.

[0013] Figure labeling: Recognition network construction unit 11, image preprocessing unit 12, feature extraction unit 13, feature fusion unit 14, feature segmentation unit 15, license plate recognition unit 16. Detailed Implementation

[0014] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0015] Example 1, as Figure 1 As shown, this application provides a license plate recognition method based on feature fusion, segmentation, and stitching. The method includes:

[0016] A license plate recognition network is constructed, which includes a feature extraction module, a feature fusion module, a feature segmentation and splicing module, and a recognition sequence output module.

[0017] In this embodiment, a license plate recognition network is first constructed. This license plate recognition network is an end-to-end deep learning network structure, which consists of a feature extraction module, a feature fusion module, a feature segmentation and stitching module, and a recognition sequence output module connected in sequence. It is used to automatically convert the input license plate image into a recognizable character sequence. The feature extraction module is used to perform multi-scale and multi-level feature learning on the preprocessed standard license plate image, and consists of a three-level branch structure of Feature1, Feature2, and Feature3. The feature fusion module is used to uniformly align and stitch the feature maps output by the above multiple branches in the channel dimension, and can use the convolution and downsampling added after each branch to merge the feature maps. The spatial dimensions are adjusted to be consistent and spliced ​​in the channel direction; the feature segmentation and splicing module is used to convert the fused feature map into a sequence feature suitable for license plate character order recognition. It can use serialized convolution or lightweight feature transformation to compress the two-dimensional features into row sequence features, and then splice the two parts after segmentation in the width dimension in turn, expanding the original two rows of license plate features into a continuous single row sequence feature; the recognition sequence output module is used to map the above-mentioned recombined single row sequence features to the license plate character category space. It is usually composed of one or more layers of convolution / fully connected and sequence rearrangement, and can output a probability sequence corresponding to the length of the license plate characters, so as to realize the unified automatic recognition of single-layer license plates, double-layer license plates and multi-system license plates.

[0018] Furthermore, building a license plate recognition network includes:

[0019] Based on the actual application scenario, the license plate recognition network is deployed to the actual application platform; the license plate recognition network is optimized and transformed according to the platform configuration information of the actual application platform.

[0020] Preferably, the platform configuration information, such as computing resources, memory capacity, and inference framework support, is first determined based on the type of platform involved in the actual application scenario, such as server-side, embedded terminal, edge computing device, or mobile application. If deployed on a server-side, a high-performance GPU environment, such as the NVIDIA Tesla series, is typically used, equipped with a deep learning framework like PyTorch or TensorFlow. If deployed on an embedded platform, the limitation of computing power needs to be considered, and a lightweight inference engine, such as TensorRT, NCNN, or OpenVINO, should be selected. Next, the weight file of the license plate recognition network is exported from the original framework format, such as .pth or .ckpt, to the universal intermediate format ONNX, and then converted to the appropriate format according to the inference engine supported by the actual application platform, such as TensorRT Engine, OpenVINO IR, or CoreML. During the conversion process, the network structure is pruned, parameters are quantized, and fusion optimizations are performed to reduce model size, accelerate inference speed, and maintain stable accuracy. Finally, multi-threading optimization, memory management optimization, and batch inference parameter adjustments are performed for the actual operating environment. By combining real-world license plate video streams or image inputs, the recognition latency, frame rate, and accuracy are evaluated to ensure the network achieves a balance between real-time performance and accuracy. Finally, the optimized license plate recognition network is embedded into the core engine of the actual application platform, and interfaces with image acquisition devices and databases via API interfaces or SDKs to achieve an end-to-end automated recognition process, meeting the real-time recognition needs of intelligent transportation, urban security, and vehicle management systems.

[0021] The captured license plate images are preprocessed to obtain standard license plate images.

[0022] In one embodiment, the captured license plate images are adjusted to a predetermined standard size according to a predetermined image tensor, such as 3×48×96, to ensure that the detailed information of the license plate characters can be effectively captured and to avoid the distortion caused by different sizes of license plate images. Then, the images are subjected to pixel normalization processing, that is, the range of each pixel value from 0 to 255 is mapped to the range of 0 to 1, to accelerate the subsequent training process, improve the stability and convergence speed of network training, and avoid gradient explosion or vanishing problems caused by the numerical range of input data being too large or too small, thus laying the foundation for subsequent feature extraction and license plate recognition.

[0023] Furthermore, the captured license plate images are preprocessed to obtain standard license plate images, including:

[0024] The license plate image is adjusted based on a predetermined image tensor to obtain an adjusted license plate image; the adjusted license plate image is then normalized to generate a standard license plate image.

[0025] Preferably, a pre-set image tensor is first obtained. This pre-set image tensor is typically set to 3×48×96, where "3" represents the number of color channels (RGB three-color channels), 48 represents the image height, and 96 represents the image width. Since the original size and aspect ratio of each license plate image may differ, the system uses image scaling algorithms, such as bilinear interpolation, to precisely adjust the image height to 48 pixels and the width to 96 pixels. The scaled original image is typically stored in memory in a (height, width, number of channels) format. Subsequently, to adapt to the standard requirements of tensors in mainstream deep learning frameworks such as PyTorch and TensorFlow, a dimension transformation operation is performed, reorganizing the image data into a (number of channels, height, width) order, thereby converting the image data into a canonical tensor with dimensions of 3×48×96, which is used to adjust the license plate image. Then, each pixel value in the adjusted license plate image is divided by 255, thereby normalizing these pixel values ​​to floating-point numbers and mapping them to the range [0,1], ensuring that all image data have consistent scale. The image after normalization is the standard license plate image, providing high-quality input for the subsequent feature extraction module.

[0026] The feature extraction module performs multi-level feature extraction on the standard license plate image to obtain a multi-feature sequence of the license plate image.

[0027] In one embodiment, after obtaining the standard license plate image, the feature extraction module calls the internal Feature1 branch to perform initial feature extraction, the Feature2 branch to perform deep feature extraction, and the Feature3 branch to perform abstract feature extraction. Through the collaborative work of these three branches, the final multi-feature sequence of the license plate image will contain multi-level feature information, laying a data foundation for subsequent network processing and improving the accuracy and robustness of license plate character recognition.

[0028] Furthermore, the feature extraction module performs multi-level feature extraction on the standard license plate image to obtain a multi-feature sequence of the license plate image, including:

[0029] The feature extraction module is activated, which includes a Feature1 branch, a Feature2 branch, and a Feature3 branch. Initial feature extraction is performed on the standard license plate image using the Feature1 branch to obtain an initial image feature sequence. Deep feature extraction is performed on the standard license plate image using the Feature2 branch to obtain a deep image feature sequence. Abstract feature extraction is performed on the standard license plate image using the Feature3 branch to obtain an abstract image feature sequence. A multi-feature sequence of the license plate image is generated based on the initial image feature sequence, the deep image feature sequence, and the abstract image feature sequence.

[0030] Preferably, the feature extraction module is activated first. This module consists of three branches: Feature1, Feature2, and Feature3, which are used to progressively extract the initial features, deep features, and abstract features of the image. When the feature extraction module receives the standard license plate image, it first passes the standard license plate image to the Feature1 branch. The Feature1 branch uses a large convolutional kernel (e.g., 5×5), a ReLU activation function, and a batch normalization layer to perform convolution operations on the license plate image, capturing the global features of the license plate, such as the outline, position, background, and contrast between the license plate and the background. Then, a 1×1 convolutional kernel is used to reduce the number of channels, and a max pooling layer is used for dimensionality reduction to reduce the number of channels while preserving spatial information, thus obtaining the initial feature sequence of the standard license plate image. Subsequently, the standard license plate image is passed to the Feature2 branch. Feature2 uses multiple smaller convolutional kernels (e.g., 3×3), ReLU activation functions, and batch normalization layers to further process the initial feature sequence extracted by Feature1, extracting local detail features of the license plate image, such as the edges, strokes, and font details of the license plate characters. Then, dimensionality reduction is performed using a similar method to obtain the deep feature sequence of the image. Afterward, the standard license plate image is passed to the Feature3 branch. Feature3 focuses on deeper features of the image, using convolutional layers, ReLU activation functions, batch normalization layers, and pooling layers to perform higher-level processing, extracting abstract features of the license plate image. These features involve relationships between characters, contextual information, and the global semantic structure of the image. For example, Feature3 can identify high-level semantic features such as the arrangement of license plate characters and the spatial distribution between characters, forming an abstract feature sequence of the image. Finally, the initial feature sequences, deep feature sequences, and abstract feature sequences extracted from the Feature1, Feature2, and Feature3 branches are merged to obtain a multi-feature sequence of a license plate image. This provides rich input data for the subsequent license plate character recognition module, enhances the model's ability to recognize different license plate images, and improves the accuracy and robustness of the recognition.

[0031] The feature fusion module performs channel-dimensional splicing and fusion of the multi-feature sequence of the license plate image to obtain a license plate fusion feature map.

[0032] In one embodiment, after obtaining the multi-feature sequence of the license plate image, the feature fusion module performs convolution and downsampling operations on the initial image feature sequence and the deep image feature sequence in the multi-feature sequence of the license plate image to obtain corresponding optimized features. Then, these optimized features are concatenated and fused with the abstract image feature sequence in the multi-feature sequence of the license plate image along the channel dimension, thereby combining the feature information extracted from each branch to generate a license plate fusion feature map containing features at all levels. This license plate fusion feature map not only retains the global information, detailed information and semantic information from each branch, but also effectively fuses image features at different scales, improving the recognition accuracy and robustness of subsequent models.

[0033] Furthermore, the feature fusion module performs channel-dimensional splicing and fusion of the multi-feature sequence of the license plate image to obtain a license plate fusion feature map, including:

[0034] The initial feature sequence of the image is convolved and downsampled to obtain an optimized initial feature sequence; the deep feature sequence of the image is convolved and downsampled to obtain an optimized deep feature sequence; the optimized initial feature sequence, the optimized deep feature sequence, and the image abstract feature sequence are concatenated and fused along the channel dimension according to the feature fusion module to generate the license plate fusion feature map.

[0035] Preferably, after receiving the multi-feature sequence of the license plate image, the feature fusion module first applies a convolution operation to the initial feature sequence of the image, typically using a 1×1 convolution kernel. The main purpose is to precisely adjust the number of channels to a predetermined target and simultaneously integrate information between channels. Then, a downsampling operation is performed, usually through a convolutional layer with a stride of 2 or a max-pooling layer, aiming to reduce its spatial size to match the deepest abstract feature sequence of the image. Similarly, the deep feature sequence of the image also undergoes similar 1×1 convolution and downsampling operations to unify its number of channels and spatial size. After this processing, the initial image feature sequence and the deep image feature sequence are transformed into optimized initial feature sequences and optimized deep feature sequences with uniform specifications, respectively. Subsequently, the feature fusion module concatenates the optimized initial feature sequence, the optimized deep feature sequence, and the image abstract feature sequence (which, while not adjusted in this step, already possesses highly abstract semantics) along the channel dimension. This concatenation operation directly connects the three feature sequences along the channel direction while maintaining their spatial dimensions. For example, if the optimized initial feature sequence and the optimized deep feature sequence both have 128 channels, the image abstract feature sequence has 256 channels, and all three are aligned to a spatial dimension of 4×20, then the concatenation will generate a license plate fusion feature map with a significantly increased number of channels (512). The resulting license plate fusion feature map is a highly information-dense composite representation, encompassing shallow, precise spatial details, mid-level local structural features, and deep, global semantic information. This provides a feature foundation with rich detail and strong semantic understanding capabilities for subsequent processing, enhancing the model's ability to express complex license plate structures.

[0036] The license plate fusion feature map is segmented and spliced ​​using the feature segmentation and splicing module to obtain a license plate reconstructed feature map.

[0037] In one embodiment, after obtaining the license plate fusion feature map, the feature segmentation and stitching module extracts sequence features from the fusion feature map to obtain a license plate sequence feature map. This license plate sequence feature map is then segmented in height dimension to obtain an upper-layer and lower-layer feature map. By stitching the upper-layer and lower-layer feature maps in width dimension, a continuous license plate reconstructed feature map containing all feature information is generated. At this point, the upper and lower-layer features of the license plate image are fused, forming a single, unified feature representation, thereby simplifying subsequent recognition tasks and improving the accuracy and robustness of license plate recognition.

[0038] Furthermore, the license plate fusion feature map is segmented and stitched according to the feature segmentation and stitching module to obtain a license plate reconstructed feature map, including:

[0039] The feature segmentation and splicing module extracts sequence features from the license plate fusion feature map to obtain a license plate sequence feature map; the feature segmentation and splicing module performs height dimension segmentation on the license plate sequence feature map to obtain an upper-layer feature map and a lower-layer feature map; the feature segmentation and splicing module splices the upper-layer feature map and the lower-layer feature map in width dimension to generate the reconstructed license plate feature map.

[0040] Preferably, the obtained license plate fusion feature map is typically a tensor with dimensions [512, 4, 20], representing a feature map with 512 channels, a height of 4 pixels, and a width of 20 pixels. For this license plate fusion feature map, the feature segmentation and stitching module first processes it through a sequence feature extractor composed of multiple convolutional layers, batch normalization layers, and ReLU activation functions. These convolutional layers are responsible for deepening feature fusion, strengthening feature representation, and aggregating spatial information and expanding the receptive field through slight downsampling. Batch normalization layers ensure training stability, and the ReLU activation function introduces a non-linear transformation. After this series of operations, the dimensions of the license plate fusion feature map are converted to [512, 2, 18], where the height is compressed from 4 to 2, allowing each pixel row to cover a larger vertical area in the original image, and the width is adjusted from 20 to 18 to better match the length of the final output sequence, thereby generating a refined license plate sequence feature map. Subsequently, the feature segmentation and concatenation module performs height-dimensional segmentation on the license plate sequence feature map, that is, it divides it into two independent parts along its height dimension (the second dimension). Typically, the 0th pixel row is taken as the upper-layer feature map, with dimensions [512, 1, 18], and the 1st pixel row is taken as the lower-layer feature map, also with dimensions [512, 1, 18]. This segmentation operation physically separates the features that might represent the upper and lower rows of characters on a double-layer license plate. For a single-layer license plate, it is considered a special case where the upper and lower parts have the same features. Finally, the feature segmentation and concatenation module concatenates the segmented upper-layer and lower-layer feature maps along the width dimension (the third dimension), connecting the two sequences, each with a length of 18. After concatenation, the width of the feature map becomes 36, while the number of channels and the height remain unchanged, resulting in a reconstructed license plate feature map with dimensions [512, 1, 36]. This reconstructed feature map provides more comprehensive input information for subsequent character recognition, ensuring the accuracy and robustness of license plate recognition.

[0041] The reconstructed license plate feature map is input into the recognition sequence output module to obtain the license plate recognition result.

[0042] In one embodiment, after obtaining the license plate reconstructed feature map, it is passed as input to the recognition sequence output module. The recognition sequence output module processes the reconstructed feature map according to the license plate recognition model, outputs the category probability distribution of each character, and outputs the final license plate character sequence as the license plate recognition result based on these probability values. This recognition sequence output module provides accurate and fast character recognition for the entire license plate recognition system, ensuring high-efficiency license plate recognition performance.

[0043] Furthermore, the construction steps of the recognition sequence output module include:

[0044] Data preprocessing is performed on the license plate recognition sample set to obtain a license plate recognition record set; the license plate recognition record set is dynamically adjusted based on progressive data augmentation technology to obtain a dynamic license plate recognition record set; supervised training is performed on the dynamic license plate recognition record set based on the CTC loss function and the license plate type classification loss function to construct a license plate recognition model; the license plate recognition model is optimized and trained according to the Adam optimization algorithm to generate the recognition sequence output module.

[0045] Preferably, the license plate recognition sample set extracted from the sample library is first subjected to data preprocessing similar to that described above. The original images of varying sizes are uniformly scaled and converted into a predetermined image tensor format with dimensions of 3×48×96. Simultaneously, pixel normalization is performed, and the license plate string and type label are converted into a model-readable format, thereby generating a structured license plate recognition record set. Subsequently, to improve the model's generalization ability and robustness in complex real-world scenarios, a progressive data augmentation technique is used to dynamically adjust this record set. In the early stages of training, basic geometric transformations such as ±15° random rotation and 90%–110% random scaling are applied to help the model initially establish adaptability to deformation. As training progresses, more challenging photometric distortions, such as brightness jitter ±30% and contrast adjustment ±20%, are gradually added to simulate complex lighting conditions such as strong light and backlight. This phased, dynamic augmentation strategy continuously generates a rich and progressively more difficult dynamic record set for license plate recognition, effectively preventing model overfitting and strengthening its environmental resistance. During the model training phase, a hybrid supervision mechanism is designed to jointly optimize the two tasks of sequence recognition and vehicle type discrimination. After forward propagation, the model output simultaneously calculates two losses. The first is a connection-temporal classification loss function (CTC), which does not require pre-aligned labels and can directly handle the mapping relationship between the character probability sequence output by the network and the real variable-length license plate string, solving the sequence recognition problem where the number of license plate characters is not fixed and the position does not need to be precisely labeled. The second is a license plate type classification loss function, which uses cross-entropy loss to urge the network to learn global features that distinguish different license plate types, thereby enhancing the representation ability and discriminative power of the backbone features. The two losses are weighted and summed according to predetermined weights to form the total loss, which jointly guides the update of model parameters and constructs the license plate recognition model. Based on this, the Adam optimization algorithm, which integrates the advantages of momentum and adaptive learning rate, is selected for backpropagation and parameter optimization, and a step-decreasing learning rate scheduling strategy is adopted to balance training efficiency and convergence accuracy. Through multiple rounds of iterative training, the model parameters are continuously fine-tuned, ultimately generating a stable, highly accurate, and powerful generalization-capable recognition sequence output module, thus completing the construction of the entire license plate recognition network.

[0046] In summary, the embodiments of this application have at least the following technical effects:

[0047] First, a license plate recognition network is constructed, comprising a feature extraction module, a feature fusion module, a feature segmentation and stitching module, and a recognition sequence output module. Next, the acquired license plate image is preprocessed to obtain a standard license plate image. Then, the feature extraction module performs multi-level feature extraction on the standard license plate image to obtain a multi-feature sequence of the license plate image. Next, the feature fusion module performs channel-dimensional stitching and fusion on the multi-feature sequence of the license plate image to obtain a fused feature map. Then, the feature segmentation and stitching module performs sequence feature segmentation and stitching on the fused feature map to obtain a reconstructed license plate feature map. Finally, the reconstructed feature map is input into the recognition sequence output module to obtain the license plate recognition result. This method solves the technical problem of insufficient recognition accuracy and generalization performance in license plate recognition technology when dealing with dual-layer license plates, due to the inability to effectively capture the correlation features between characters in the upper and lower lines. It achieves the technical effect of improving the accuracy and robustness of license plate recognition through multi-level feature extraction and intelligent feature fusion.

[0048] Example 2, based on the same inventive concept as the license plate recognition method based on feature fusion segmentation and splicing in the previous examples, such as... Figure 2 As shown, this application provides a license plate recognition system based on feature fusion, segmentation, and stitching. The system includes:

[0049] Recognition Network Construction Unit 11: Constructs a license plate recognition network, which includes a feature extraction module, a feature fusion module, a feature segmentation and splicing module, and a recognition sequence output module; Image Preprocessing Unit 12: Preprocesses the acquired license plate image to obtain a standard license plate image; Feature Extraction Unit 13: Performs multi-level feature extraction on the standard license plate image according to the feature extraction module to obtain a multi-feature sequence of the license plate image; Feature Fusion Unit 14: Performs channel-dimensional splicing and fusion on the multi-feature sequence of the license plate image according to the feature fusion module to obtain a fused license plate feature map; Feature Segmentation Unit 15: Performs sequence feature segmentation and splicing on the fused license plate feature map according to the feature segmentation and splicing module to obtain a reconstructed license plate feature map; License Plate Recognition Unit 16: Inputs the reconstructed license plate feature map into the recognition sequence output module to obtain the license plate recognition result.

[0050] Furthermore, the identification network construction unit 11 is used to perform the following method:

[0051] Based on the actual application scenario, the license plate recognition network is deployed to the actual application platform; the license plate recognition network is optimized and transformed according to the platform configuration information of the actual application platform.

[0052] Furthermore, the image preprocessing unit 12 is configured to perform the following method:

[0053] The license plate image is adjusted based on a predetermined image tensor to obtain an adjusted license plate image; the adjusted license plate image is then normalized to generate a standard license plate image.

[0054] Furthermore, the feature extraction unit 13 is used to perform the following method:

[0055] The feature extraction module is activated, which includes a Feature1 branch, a Feature2 branch, and a Feature3 branch. Initial feature extraction is performed on the standard license plate image using the Feature1 branch to obtain an initial image feature sequence. Deep feature extraction is performed on the standard license plate image using the Feature2 branch to obtain a deep image feature sequence. Abstract feature extraction is performed on the standard license plate image using the Feature3 branch to obtain an abstract image feature sequence. A multi-feature sequence of the license plate image is generated based on the initial image feature sequence, the deep image feature sequence, and the abstract image feature sequence.

[0056] Furthermore, the feature fusion unit 14 is used to perform the following method:

[0057] The initial feature sequence of the image is convolved and downsampled to obtain an optimized initial feature sequence; the deep feature sequence of the image is convolved and downsampled to obtain an optimized deep feature sequence; the optimized initial feature sequence, the optimized deep feature sequence, and the image abstract feature sequence are concatenated and fused along the channel dimension according to the feature fusion module to generate the license plate fusion feature map.

[0058] Furthermore, the feature segmentation unit 15 is used to perform the following method:

[0059] The feature segmentation and splicing module extracts sequence features from the license plate fusion feature map to obtain a license plate sequence feature map; the feature segmentation and splicing module performs height dimension segmentation on the license plate sequence feature map to obtain an upper-layer feature map and a lower-layer feature map; the feature segmentation and splicing module splices the upper-layer feature map and the lower-layer feature map in width dimension to generate the reconstructed license plate feature map.

[0060] Furthermore, the license plate recognition unit 16 is used to perform the following method:

[0061] Data preprocessing is performed on the license plate recognition sample set to obtain a license plate recognition record set; the license plate recognition record set is dynamically adjusted based on progressive data augmentation technology to obtain a dynamic license plate recognition record set; supervised training is performed on the dynamic license plate recognition record set based on the CTC loss function and the license plate type classification loss function to construct a license plate recognition model; the license plate recognition model is optimized and trained according to the Adam optimization algorithm to generate the recognition sequence output module.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A license plate recognition method based on feature fusion segmentation splicing, characterized in that, The method comprises: constructing a license plate recognition network, the license plate recognition network comprising a feature extraction module, a feature fusion module, a feature segmentation and splicing module, and an identification sequence output module; preprocessing a license plate collection image to obtain a license plate standard image; performing multi-level feature extraction on the license plate standard image according to the feature extraction module to obtain a license plate image multi-feature sequence; performing channel dimension splicing fusion on the license plate image multi-feature sequence according to the feature fusion module to obtain a license plate fusion feature map; performing sequence feature segmentation and splicing on the license plate fusion feature map according to the feature segmentation and splicing module to obtain a license plate reorganized feature map; inputting the license plate reorganized feature map into the identification sequence output module to obtain a license plate recognition result.

2. The license plate recognition method based on feature fusion segmentation and splicing according to claim 1, characterized in that, The preprocessing of the license plate collection image to obtain the license plate standard image comprises: performing basic adjustment on the license plate collection image according to a predetermined image tensor to obtain an adjusted license plate image; performing pixel normalization processing on the adjusted license plate image to generate the license plate standard image.

3. The license plate recognition method based on feature fusion segmentation and splicing according to claim 1, characterized in that, The multi-level feature extraction on the license plate standard image according to the feature extraction module to obtain the license plate image multi-feature sequence comprises: activating the feature extraction module, the feature extraction module comprising a Feature1 branch, a Feature2 branch, and a Feature3 branch; performing initial feature extraction on the license plate standard image according to the Feature1 branch to obtain an image initial feature sequence; performing deep feature extraction on the license plate standard image according to the Feature2 branch to obtain an image deep feature sequence; performing abstract feature extraction on the license plate standard image according to the Feature3 branch to obtain an image abstract feature sequence; generating the license plate image multi-feature sequence according to the image initial feature sequence, the image deep feature sequence, and the image abstract feature sequence.

4. The license plate recognition method based on feature fusion segmentation and splicing according to claim 1, characterized in that, The channel dimension splicing fusion on the license plate image multi-feature sequence according to the feature fusion module to obtain the license plate fusion feature map comprises: performing convolution operation and down-sampling on the image initial feature sequence to obtain an optimized initial feature sequence; performing convolution operation and down-sampling on the image deep feature sequence to obtain an optimized deep feature sequence; performing channel dimension splicing fusion on the optimized initial feature sequence, the optimized deep feature sequence, and the image abstract feature sequence according to the feature fusion module to generate the license plate fusion feature map.

5. The license plate recognition method based on feature fusion segmentation and splicing according to claim 1, characterized in that, The sequence feature segmentation and splicing on the license plate fusion feature map according to the feature segmentation and splicing module to obtain the license plate reorganized feature map comprises: performing sequence feature extraction on the license plate fusion feature map according to the feature segmentation and splicing module to obtain a license plate sequence feature map; performing height dimension segmentation on the license plate sequence feature map according to the feature segmentation and splicing module to obtain a license plate upper layer feature map and a license plate lower layer feature map; performing width dimension splicing on the license plate upper layer feature map and the license plate lower layer feature map according to the feature segmentation and splicing module to generate the license plate reorganized feature map.

6. The license plate recognition method based on feature fusion segmentation and splicing according to claim 1, characterized in that, The construction steps of the identification sequence output module comprise: According to the license plate recognition sample set, data preprocessing is performed to obtain a license plate recognition record set; Based on the progressive data augmentation technology, the license plate recognition record set is dynamically adjusted to obtain a license plate recognition dynamic record set; Based on the CTC loss function and the license plate type classification loss function, the license plate recognition dynamic record set is supervised trained to construct a license plate recognition model; According to the Adam optimization algorithm, the license plate recognition model is optimized and trained to generate the recognition sequence output module.

7. The license plate recognition method based on feature fusion segmentation and splicing according to claim 1, characterized in that, The license plate recognition network is constructed, including: According to the actual application scenario, the license plate recognition network is deployed to an actual application platform; According to the platform configuration information of the actual application platform, the license plate recognition network is optimized and converted.

8. The license plate recognition system based on feature fusion segmentation splicing, characterized in that, The system for implementing the license plate recognition method based on feature fusion segmentation and splicing according to any one of claims 1-7, comprising: A recognition network construction unit: constructing a license plate recognition network, the license plate recognition network comprising a feature extraction module, a feature fusion module, a feature segmentation and splicing module, and a recognition sequence output module; An image preprocessing unit: preprocessing the license plate collection image to obtain a standard license plate image; A feature extraction unit: performing multi-level feature extraction on the standard license plate image according to the feature extraction module to obtain a license plate image multi-feature sequence; A feature fusion unit: performing channel dimension splicing and fusion on the license plate image multi-feature sequence according to the feature fusion module to obtain a license plate fusion feature map; A feature segmentation unit: performing sequence feature segmentation and splicing on the license plate fusion feature map according to the feature segmentation and splicing module to obtain a license plate reorganized feature map; A license plate recognition unit: inputting the license plate reorganized feature map into the recognition sequence output module to obtain a license plate recognition result.

Images