Double-layer license plate character hierarchical division method and system

Abstract

The invention discloses a double-layer license plate character hierarchical division method and system, and belongs to the field of image processing and computer vision. The method comprises the steps of obtaining a license plate image; identifying the license plate image through the target detection network, and judging the license plate structure type; if the license plate structure type is a double-row license plate, character density field construction and gravity center thermodynamic diagram generation are carried out, and a fusion density field and a thermodynamic diagram are obtained; obtaining a character level cutting line according to the character density field and the gravity center thermodynamic diagram; cutting the license plate image according to the character level cutting line to obtain an upper character region and a lower character region; and horizontally splicing the upper character region and the lower character region into a single-line character image. The method supports complex structure license plate recognition, and is high in adaptability; the double-layer license plate character division accuracy and recognition stability are improved; structural type perception and character space distribution information are innovatively combined, and an intelligent recognition process is realized.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and computer vision, and specifically to a character segmentation and recognition method for double-layer license plates based on character density field and center of gravity heat map analysis, which is suitable for applications such as license plate recognition and urban monitoring in intelligent transportation systems. Background Art

[0002] With the development of intelligent transportation and video surveillance technologies, license plate recognition systems have been widely deployed in various real-world scenarios. However, many double-layer license plates have complex and irregular character structures. Traditional character segmentation methods (such as those based on projection and edge detection) often suffer from character overlap and inaccurate segmentation when processing such license plates, seriously affecting recognition accuracy.

[0003] In existing methods, the entire license plate image is often fed into a character recognition model, such as a CRNN or Transformer. However, for a two-layer structure, direct recognition without hierarchical distinction results in disordered character order or confusion between upper and lower characters, affecting recognition accuracy and interpretability.

[0004] Therefore, there is an urgent need for a method that can automatically identify the license plate structure and accurately divide the double-layer structure characters into different levels to improve the overall performance of the recognition system. Summary of the Invention

[0005] The purpose of the present invention is to provide a double-layer license plate character hierarchical segmentation method based on character density field and center of gravity heat map analysis, combined with a lightweight object detection network (LeYOLO) to perform license plate structure classification (single row / double row), and under the double-row structure, combined with the statistical characteristics of character density distribution and spatial center of gravity heat map, complete the separation and recognition of the upper and lower character layers, and solve the difficulties of traditional methods in character segmentation and sequential modeling.

[0006] To solve the above technical problems, the present invention provides a method for hierarchical division of double-layer license plate characters, comprising the following steps:

[0007] Get the license plate image;

[0008] Identify license plate images through the target detection network and determine the license plate structure type;

[0009] If the license plate structure type is a double-row license plate, the character density field is constructed and the center of gravity heat map is generated to obtain a fused density field and heat map;

[0010] According to the character density field and the center of gravity heat map, the character level cutting line is obtained;

[0011] Cut the license plate image according to the character level cutting line to obtain the upper character area and the lower character area;

[0012] The upper character area and the lower character area are horizontally spliced ​​into a single-line character image.

[0013] Preferably, identifying the license plate image through the target detection network and determining the license plate structure type specifically includes the following steps:

[0014] The license plate area in the license plate image is detected through the LeYOLO network, and the license plate structure type is determined through category branches; the license plate structure type includes single-row license plates and double-row license plates.

[0015] Preferably, the character density field is constructed, specifically comprising the following steps:

[0016] In the license plate area, the grayscale image is binarized and projected to obtain the vertical character density distribution map:

[0017]

[0018] Where: D(y) is the character density; I(x,y) represents the pixel intensity of the license plate image at coordinate (x,y); W is the image width.

[0019] Preferably, the center of gravity heat map is generated, specifically comprising the following steps:

[0020] Calculate the geometric center coordinates of each character area in the license plate area, and use the Gaussian kernel function to generate a probability density distribution map as the center of gravity heat map;

[0021]

[0022] Where: y c.i is the center of the kernel function; σ is the width parameter of the function.

[0023] Preferably, obtaining character-level cutting lines according to the character density field and the center-of-gravity heat map specifically includes the following steps:

[0024] The density change rate of the character density field is detected through a sliding window, and when the gradient value exceeds the threshold, it is marked as a candidate cutting line;

[0025] K-means is used to cluster the center points of the centroid heat map, and the silhouette coefficient is used to verify the clustering effect and determine the cluster boundaries of the upper and lower layers;

[0026] The candidate cutting lines obtained by gradient analysis are spatially compared with the cluster boundaries, the central axis of the overlapping area of ​​the two is selected, and dynamically adjusted through confidence weighting to obtain the character-level cutting lines.

[0027] Preferably, the character-level cutting line is obtained by dynamically adjusting the confidence weight, which specifically includes the following steps:

[0028] Fusion density field and heat map, construct the energy function E(y):

[0029] E(y)=αD(y)+(1-α)H(y)

[0030] Where: α is the weight coefficient;

[0031] The minimum value of E(y) is taken as the character-level cutting line.

[0032] Preferably, the method further comprises the following steps:

[0033] The upper character area and the lower character area are added with position encoding information, and then sent into the CRNN model for character recognition to obtain the recognition result.

[0034] Preferably, the method further comprises the following steps:

[0035] If the license plate structure type is a single-row license plate, the license plate image will be sent to the CRNN model for character recognition to obtain the recognition result.

[0036] Preferably, the LeYOLO network includes a STEM module, a Backbone network, an FPAN Neck structure, a DNiNHead structure and a Scaling strategy;

[0037] The STEM module performs two convolutions and one depthwise convolution on the input image, reducing the input size from, for example, 640×640 to 160×160;

[0038] The Backbone network performs multi-level semantic extraction based on an improved Inverted Bottleneck module; it applies the information bottleneck principle to reduce information redundancy and maintain key features; the number of inter-layer channels is limited to no more than 6 times, reducing cross-layer information loss;

[0039] The FPAN Neck structure uses PANet and BiFPN to fuse multi-scale features. It also proposes a lightweight fusion strategy to avoid repeated convolution and channel redundancy in traditional FPN.

[0040] The DNiN Head architecture uses a decoupled head structure: classification and regression use two independent pointwise networks; only lightweight 1×1 (pointwise) and 3×3 (depthwise) convolutions are used; each pixel independently outputs its category and bounding box prediction through this structure;

[0041] The Scaling strategy provides multiple scale models from LeYOLO-Nano to LeYOLO-Large.

[0042] The present invention also provides a double-layer license plate character hierarchical division system, comprising:

[0043] Acquisition module, used to obtain license plate images;

[0044] The recognition module is used to identify the license plate image through the target detection network and determine the license plate structure type;

[0045] The analysis module is used to construct a character density field and generate a center of gravity heat map if the license plate structure type is a double-row license plate, and obtain a fused density field and heat map;

[0046] Dynamic boundary line extraction module, used to obtain character-level cutting lines based on character density field and center of gravity heat map;

[0047] A cutting module is used to cut the license plate image according to the character level cutting line to obtain the upper character area and the lower character area;

[0048] The reorganization module is used to horizontally splice the upper character area and the lower character area into a single-line character image.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. Support complex structure license plate recognition and strong adaptability;

[0051] 2. Improve the accuracy of double-layer license plate character segmentation and recognition stability;

[0052] 3. Innovatively combine structural type perception and character spatial distribution information to realize intelligent recognition process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0054] Figure 1 This is a flow chart of the overall structure of the double-layer license plate character hierarchical division method based on character density field and center of gravity heat map analysis of the present invention;

[0055] Figure 2 Lightweight Object Detection Network (LeYOLO) module flow chart;

[0056] Figure 3 It is the flow chart of the character level division module. DETAILED DESCRIPTION

[0057] The following description sets forth numerous specific details to facilitate a thorough understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific implementations disclosed below.

[0058] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0059] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0060] The present invention is described in further detail below with reference to the accompanying drawings:

[0061] The present invention provides a method for hierarchical division of characters of a double-layer license plate, comprising the following steps:

[0062] (1) Use the target detection network to detect the license plate area and determine the structure type (single row / double row);

[0063] (2) If it is a two-row license plate, character density field construction and center of gravity heat map analysis are performed;

[0064] (3) Obtaining character-level cutting lines based on the joint energy function;

[0065] (4) Separate the character levels and identify and reorganize them into license plate character sequences.

[0066] Preferably, the character density field construction is achieved by vertical projection of the image;

[0067] Preferably, the centroid heat map is obtained by Gaussian superposition of the centroids of the character regions;

[0068] Preferably, the cutting line is the minimum energy point of the joint density function;

[0069] Preferably, if the license plate structure is determined to be a single-row license plate, it is directly sent to the character recognition network without the need for hierarchical division;

[0070] Preferably, character layer position information encoding is integrated into the character recognition model input to enhance recognition accuracy and sequential modeling capabilities.

[0071] In order to better illustrate the technical effects of the present invention, the present invention provides the following specific examples to illustrate the above technical process:

[0072] Example 1: A method for dividing characters into hierarchical levels on a double-layer license plate, comprising the following steps:

[0073] 1. License Plate Structure Perception Module

[0074] The LeYOLO network is used to detect the license plate area in the image and determine the license plate structure type (single or double lines) through category branches. This structure type information is used to dynamically select the subsequent processing path.

[0075] 2. Character density field construction

[0076] In the detected license plate area, the grayscale image is binarized and projected to obtain the vertical character density distribution map:

[0077]

[0078] Where: I(x,y) represents the pixel intensity of the license plate image at coordinate (x,y), and W is the image width.

[0079] 3. Center of gravity heat map analysis

[0080] Extract all candidate connected domains of characters, calculate their centroid coordinates (xc, yc), and construct a heat map distribution in the y direction:

[0081]

[0082] The peak positions of the upper and lower distributions of characters can be observed through the heat map, and a joint analysis map can be constructed in combination with the density field as a basis for cutting.

[0083] 4. Dynamic dividing line extraction

[0084] Fusion density field and heat map, construct the energy function E(y):

[0085] E(y)=αD(y)+(1-α)H(y)

[0086] Where: α is the weight coefficient;

[0087] Take the minimum value of E(y) as the dividing line ysplit for character level division.

[0088] 5. Layered Character Recombination and Recognition

[0089] The cut upper and lower character regions (upper and lower character regions) are horizontally spliced ​​into a single-line character image and fed into the CRNN model for character recognition. To enhance recognition accuracy, position encoding information can be added at the image or network input stage (for example, binary encoding [0] / [1] for the upper and lower layers).

[0090] The beneficial effects of the hierarchical division method of double-layer license plate characters in this embodiment are:

[0091] 1. Support complex structure license plate recognition and strong adaptability;

[0092] 2. Improve the accuracy of double-layer license plate character segmentation and recognition stability;

[0093] 3. The modular structure is lightweight and has good portability, suitable for embedded and cloud environments;

[0094] 4. Innovatively combine structural type perception and character spatial distribution information to realize intelligent recognition process.

[0095] The overall structure of the method for dividing characters into two-layer license plates is as follows: Figure 1 As shown, the following steps are included:

[0096] 1. Use LeYOLO to detect the license plate area and determine whether the structure is single row or double row;

[0097] 2. If it is a double line, enter the character level division module:

[0098] (1) Construct a character density field map (calculate the density distribution of characters in the vertical direction);

[0099] (2) Calculate the centroid heat map of the character center point (to assist in dividing the boundary line);

[0100] (3) Determine the optimal cutting line through density gradient and center of gravity clustering to achieve separation of upper and lower character layers;

[0101] 3. Splice the two layers of characters into a single line and feed it into the recognition model (such as CRNN);

[0102] 4. If it is a single-lane license plate, it is directly sent to the recognition model;

[0103] 5. Output standard character sequence.

[0104] The lightweight object detection network (LeYOLO) module process of the present invention is as follows Figure 2 Shown, including:

[0105] 1. STEM module

[0106] Perform two convolutions and one depthwise convolution on the input image, quickly reducing the input size from, for example, 640×640 to 160×160, saving FLOPs;

[0107] Use a low channel count to keep computational resource consumption low.

[0108] 2. Backbone network

[0109] Multi-level semantic extraction based on the improved Inverted Bottleneck module;

[0110] The Information Bottleneck Principle is applied to reduce information redundancy and maintain key features;

[0111] The expansion of the number of inter-layer channels is limited to a reasonable range (no more than 6 times) to reduce cross-layer information loss.

[0112] 3. FPAN (Fast PANet) Neck Structure

[0113] Refer to PANet and BiFPN designs to perform multi-scale feature fusion;

[0114] A lightweight fusion strategy is proposed to avoid repeated convolution and channel redundancy in traditional FPN;

[0115] The key is to use P4 as the main input and P3 and P5 as supplements to reduce lock waiting and computing costs.

[0116] 4.DNiN Head (Decoupled Network-in-Network Head)

[0117] Adopt a decoupled head structure: classification and regression use two independent pointwise networks;

[0118] Only lightweight 1×1 (pointwise) and 3×3 (depthwise) convolutions are used;

[0119] Each pixel point independently outputs its category and bounding box prediction through this structure.

[0120] 5. Scaling strategy

[0121] Provide multiple scale models from LeYOLO-Nano to LeYOLO-Large;

[0122] Adaptable to different scenarios from <1GFLOPs to >8GFLOPs, covering embedded, mobile and industrial scenarios.

[0123] The character level division module process of the present invention is as follows Figure 3 As shown, the following steps are included:

[0124] 1. Character density field map construction

[0125] By vertically projecting the pixel density of each column, a histogram is generated that reflects the vertical distribution of characters. Density peaks correspond to character areas, while valleys correspond to the blank areas between lines.

[0126] 2. Center of gravity heat map generation

[0127] The geometric center coordinates of each connected region (character) are calculated, and a Gaussian kernel function is used to generate a probability density distribution map. The highlighted areas of the heat map reflect the natural clustering centers of character rows.

[0128] 3. Two-branch joint analysis

[0129] (1) Density gradient branch: The density change rate is detected through a sliding window, and when the gradient value exceeds the threshold, it is marked as a candidate cutting line (such as an area where the density drops by 50%).

[0130] (2) Centroid clustering branch: K-means (K=2) is used to cluster the center points, and the clustering effect is verified by combining the silhouette coefficient to determine the center distribution boundaries of the upper and lower layers.

[0131] 4. Result fusion and cutting line optimization

[0132] The candidate cutting lines obtained from the gradient analysis are spatially aligned with the cluster boundaries, with the central axis of the overlapping area being preferred. If there is a conflict, it is dynamically adjusted by confidence weighting (gradient strength × cluster compactness).

[0133] 5. Split Execution and Verification

[0134] After segmenting the image along the final cut line, verify the connected domains of the upper and lower layers to ensure that no characters are cut across rows. If an anomaly is found (such as a single character being cut), return to step 3 for iterative optimization.

[0135] In the several embodiments provided herein, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules, subassemblies, or units is merely a logical functional division. In actual implementation, other divisions may be used, such as combining or integrating multiple units, subassemblies, or components into another device, or some features may be omitted or not implemented.

[0136] The units may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0137] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0138] In particular, according to an embodiment disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above.

[0139] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention shall be covered by the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for dividing characters into two-layer license plates, characterized in that: The following steps are involved: Get the license plate image; Identify license plate images through the target detection network and determine the license plate structure type; If the license plate structure type is a double-row license plate, the character density field is constructed and the center of gravity heat map is generated to obtain a fused density field and heat map; According to the character density field and the center of gravity heat map, the character level cutting line is obtained; Cut the license plate image according to the character level cutting line to obtain the upper character area and the lower character area; The upper character area and the lower character area are horizontally spliced ​​into a single-line character image.

2. The method for dividing characters of a double-layer license plate according to claim 1, characterized in that: The target detection network is used to identify the license plate image and determine the license plate structure type, which specifically includes the following steps: The license plate area in the license plate image is detected through the LeYOLO network, and the license plate structure type is determined through category branches; the license plate structure type includes single-row license plates and double-row license plates.

3. The method for dividing characters of a double-layer license plate into different levels according to claim 2, wherein: The character density field is constructed, specifically including the following steps: In the license plate area, the grayscale image is binarized and projected to obtain the vertical character density distribution map: Where: D(y) is the character density; I(x,y) represents the pixel intensity of the license plate image at coordinate (x,y); W is the image width.

4. The method for dividing characters of a double-layer license plate into different levels according to claim 3, wherein: The center of gravity heat map generation includes the following steps: Calculate the geometric center coordinates of each character area in the license plate area, and use the Gaussian kernel function to generate a probability density distribution map as the center of gravity heat map; Where: y c.i is the center of the kernel function; σ is the width parameter of the function.

5. The method for dividing characters of a double-layer license plate into different levels according to claim 4, wherein: According to the character density field and the center of gravity heat map, the character level cutting line is obtained, which specifically includes the following steps: The density change rate of the character density field is detected through a sliding window, and when the gradient value exceeds the threshold, it is marked as a candidate cutting line; K-means is used to cluster the center points of the centroid heat map, and the silhouette coefficient is used to verify the clustering effect and determine the cluster boundaries of the upper and lower layers; The candidate cutting lines obtained by gradient analysis are spatially compared with the cluster boundaries, the central axis of the overlapping area of ​​the two is selected, and dynamically adjusted through confidence weighting to obtain the character-level cutting lines.

6. The method for dividing characters of a double-layer license plate according to claim 5, characterized in that: The character-level cutting line is obtained by dynamic adjustment of confidence weights, which specifically includes the following steps: Fusion density field and heat map, construct the energy function E(y): E(y)=αD(y)+(1-α)H(y) Where: α is the weight coefficient; The minimum value of E(y) is taken as the character-level cutting line.

7. The method for dividing characters of a double-layer license plate according to claim 6, characterized in that: The following steps are also included: The upper character area and the lower character area are added with position encoding information, and then sent into the CRNN model for character recognition to obtain the recognition result.

8. The method for dividing characters of a double-layer license plate according to claim 7, characterized in that: The following steps are also included: If the license plate structure type is a single-row license plate, the license plate image will be sent to the CRNN model for character recognition to obtain the recognition result.

9. The method for hierarchical division of double-layer license plate characters according to claim 8, characterized in that: The LeYOLO network includes a STEM module, a Backbone network, an FPAN Neck structure, a DNiN Head structure, and a Scaling strategy; The STEM module performs two convolutions and one depthwise convolution on the input image, reducing the input size from, for example, 640×640 to 160×160; The Backbone network performs multi-level semantic extraction based on the Inverted Bottleneck module; applies the information bottleneck principle to reduce information redundancy and maintain key features; the number of inter-layer channels is limited to no more than 6 times, reducing cross-layer information loss; The FPAN Neck structure uses PANet and BiFPN to fuse multi-scale features. It also proposes a lightweight fusion strategy to avoid repeated convolution and channel redundancy in traditional FPN. The DNiN Head architecture uses a decoupled head structure: classification and regression use two independent pointwise networks; only lightweight 1×1 (pointwise) and 3×3 (depthwise) convolutions are used; each pixel independently outputs its category and bounding box prediction through this structure; The Scaling strategy provides multiple scale models from LeYOLO-Nano to LeYOLO-Large.

10. A system for hierarchical character division of double-layer license plates, for implementing the method for hierarchical character division of double-layer license plates according to any one of claims 1 to 9, characterized in that: include: Acquisition module, used to obtain license plate images; The recognition module is used to identify the license plate image through the target detection network and determine the license plate structure type; The analysis module is used to construct a character density field and generate a center of gravity heat map if the license plate structure type is a double-row license plate, and obtain a fused density field and heat map; Dynamic boundary line extraction module, used to obtain character-level cutting lines based on character density field and center of gravity heat map; A cutting module is used to cut the license plate image according to the character level cutting line to obtain the upper character area and the lower character area; The reorganization module is used to horizontally splice the upper character area and the lower character area into a single-line character image.

Images