License plate information generation method and device, electronic equipment and computer readable medium
Through the hierarchical feature information generation model and the decoupled detection head model, the problem of insufficient output of the YOLO v5 detection head model is solved, and the accurate identification of license plate information is achieved, and the accuracy of identification of license plate information is improved.
Patent Information
- Application Number
- CN202310586949.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
The output layer of YOLO v5's detection head model outputs license plate character classification information and regression information at the same time, resulting in the license plate information being inaccurate enough and unable to effectively distinguish the feature learning focus of classification tasks and regression tasks.
The hierarchical feature information generation model is used to generate hierarchical license plate feature information and input it into the decoupled detection head model to ensure that each output layer outputs a single type of character information, and avoid the output layer of the detection head model performing relevant regression tasks and classification at the same time. tasks, thereby improving the output accuracy of the output layer.
Through the decoupled detection model, the relevant classification tasks and regression tasks of license plate characters are distinguished and learned, which improves the recognition accuracy of license plate information and ensures that the output character information set is more accurate.
Smart Images

Figure CN119027929A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, and computer-readable medium for generating license plate information. Background Art
[0002] At present, license plate recognition technology is a technology for identifying license plate information in vehicle images and has been widely used in various application scenarios. For the generation of license plate information, the usual method is to use YOLO (You Only Look Once, object detection model) v5 as a model for detecting license plate characters to generate license plate information for vehicle images.
[0003] However, the inventors have found that when the above method is used to generate license plate information, the following technical problems often occur:
[0004] The feature learning for the classification task and regression task of license plate characters has different focuses, and the output layer of the YOLO v5 detection head model outputs both license plate character classification information and license plate character regression information, resulting in the character information output by the YOLO v5 detection head model being inaccurate.
[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention
[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0007] Some embodiments of the present disclosure propose a license plate information generation method, device, electronic device, computer-readable medium and program product to solve the technical problems mentioned in the above background technology section.
[0008] In a first aspect, some embodiments of the present disclosure provide a method for generating license plate information, comprising: acquiring a target license plate image; inputting the target license plate image into a hierarchical feature information generation model in a license plate recognition model to generate hierarchical license plate feature information; inputting the hierarchical license plate feature information into a decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image, wherein the decoupled detection head model comprises: an output layer set, each output layer outputs a single type of character information; and generating license plate information based on the character information set.
[0009] Optionally, the above-mentioned hierarchical license plate feature information includes: a multi-level license plate feature information set, the above-mentioned decoupled detection head model includes: a multi-level detection head model set; and the above-mentioned input of the above-mentioned hierarchical license plate feature information into the decoupled detection head model included in the above-mentioned license plate recognition model to output a character information set for the above-mentioned target license plate image, including: inputting each multi-level license plate feature information in the above-mentioned multi-level license plate feature information set into the corresponding multi-level detection head model to output a character information group to obtain a character information group set, wherein the multi-level detection head model includes: an output layer group, the output layer group is a subset of the above-mentioned output layer set; the above-mentioned character information group set is determined as the above-mentioned character information set.
[0010] Optionally, the multi-level detection head model includes at least one of the following: a character category output layer, a character position output layer, and a character probability output layer; and the above-mentioned inputting each multi-level license plate feature information in the multi-level license plate feature information set into the corresponding multi-level detection head model to output a character information group, including: inputting the multi-level license plate feature information into the corresponding character category output layer to output character category information as character information; inputting the multi-level license plate feature information into the corresponding character position output layer to output character position information as character information; inputting the multi-level license plate feature information into the corresponding character probability output layer to output character probability information as character information.
[0011] Optionally, the character category output layer includes: a first convolution layer, a first depth-separable convolution layer, a second convolution layer, and a category output layer; and inputting the multi-level license plate feature information into the corresponding character category output layer to output character category information, including: inputting the multi-level license plate feature information into the corresponding first convolution layer to generate first convolution information; inputting the first convolution information into the corresponding first depth-separable convolution layer to output second convolution information; inputting the second convolution information into the corresponding second convolution layer to output third convolution layer information; inputting the third convolution information into the corresponding category output layer to output character category information.
[0012] Optionally, the character position output layer includes: a first convolution layer, a second depth-separable convolution layer, a third convolution layer, and a position output layer; and inputting the multi-level license plate feature information into the corresponding character position output layer to output the character position information, including: inputting the multi-level license plate feature information into the corresponding first convolution layer to output the first convolution information; inputting the first convolution information into the corresponding second depth-separable convolution layer to output the fourth convolution information; inputting the fourth convolution information into the corresponding third convolution layer to output the fifth convolution layer information; inputting the fifth convolution information into the corresponding position output layer to output the character position information.
[0013] Optionally, the character probability output layer includes: a first convolution layer, a second depth-separable convolution layer, a fourth convolution layer, and a probability output layer; and inputting the multi-level license plate feature information into the corresponding character probability output layer to output character probability information, including: inputting the multi-level license plate feature information into the corresponding first convolution layer to output the first convolution information; inputting the first convolution information into the corresponding second depth-separable convolution layer to output the fourth convolution information; inputting the fourth convolution information into the corresponding fourth convolution layer to output the sixth convolution layer information; inputting the sixth convolution information into the corresponding probability output layer to output the character probability information.
[0014] Optionally, generating the license plate information according to the character information set includes: determining a license plate character set for the target license plate image according to the character information set; and performing character combination on the license plate character set to generate the license plate information.
[0015] Optionally, the above-mentioned license plate recognition model is trained through the following steps: obtaining a sample data set; performing sample data splicing enhancement on the sample data in the above-mentioned sample data set to generate an enhanced data set; using the above-mentioned sample data set and the above-mentioned enhanced data set as training data sets to perform model training on the initial license plate recognition model to generate a license plate recognition model.
[0016] In a second aspect, some embodiments of the present disclosure provide a license plate information generating device, comprising: an acquisition unit configured to acquire a target license plate image; a first input unit configured to input the above target license plate image into a hierarchical feature information generation model in a license plate recognition model to generate hierarchical license plate feature information; a second input unit configured to input the above hierarchical license plate feature information into a decoupled detection head model included in the above license plate recognition model to output a character information set for the above target license plate image, wherein the above decoupled detection head model comprises: an output layer set, each output layer outputs a single type of character information; a generation unit configured to generate license plate information based on the above character information set.
[0017] Optionally, the above-mentioned hierarchical license plate feature information includes: a multi-level license plate feature information set, the above-mentioned decoupled detection head model includes: a multi-level detection head model set; and the first input unit can be configured to: input each multi-level license plate feature information in the above-mentioned multi-level license plate feature information set into the corresponding multi-level detection head model to output a character information group to obtain a character information group set, wherein the multi-level detection head model includes: an output layer group, the output layer group is a subset of the above-mentioned output layer set; and the above-mentioned character information group set is determined as the above-mentioned character information set.
[0018] Optionally, the multi-level detection head model includes at least one of the following: a character category output layer, a character position output layer, and a character probability output layer; and the first input unit can be configured to: input the multi-level license plate feature information into the corresponding character category output layer to output the character category information as character information; input the multi-level license plate feature information into the corresponding character position output layer to output the character position information as character information; input the multi-level license plate feature information into the corresponding character probability output layer to output the character probability information as character information.
[0019] Optionally, the above-mentioned character category output layer includes: a first convolution layer, a first depth-separable convolution layer, a second convolution layer, and a category output layer; and the first input unit can be configured to: input the above-mentioned multi-level license plate feature information into the corresponding first convolution layer to generate first convolution information; input the above-mentioned first convolution information into the corresponding first depth-separable convolution layer to output second convolution information; input the above-mentioned second convolution information into the corresponding second convolution layer to output third convolution layer information; input the above-mentioned third convolution information into the corresponding category output layer to output character category information.
[0020] Optionally, the above-mentioned character position output layer includes: a first convolution layer, a second depth-separable convolution layer, a third convolution layer, and a position output layer; and the first input unit can be configured to: input the above-mentioned multi-level license plate feature information into the corresponding first convolution layer to output the first convolution information; input the above-mentioned first convolution information into the corresponding second depth-separable convolution layer to output the fourth convolution information; input the above-mentioned fourth convolution information into the corresponding third convolution layer to output the fifth convolution layer information; input the above-mentioned fifth convolution information into the corresponding position output layer to output the character position information.
[0021] Optionally, the character probability output layer includes: a first convolution layer, a second depth-separable convolution layer, a fourth convolution layer, and a probability output layer; and the first input unit can be configured to: input the multi-level license plate feature information into the corresponding first convolution layer to output the first convolution information; input the first convolution information into the corresponding second depth-separable convolution layer to output the fourth convolution information; input the fourth convolution information into the corresponding fourth convolution layer to output the sixth convolution layer information; input the sixth convolution information into the corresponding probability output layer to output the character probability information.
[0022] Optionally, the generating unit may be configured to: determine a license plate character set for the target license plate image according to the character information set; and perform character combination on the license plate character set to generate license plate information.
[0023] Optionally, the above-mentioned license plate recognition model is trained through the following steps: obtaining a sample data set; performing sample data splicing enhancement on the sample data in the above-mentioned sample data set to generate an enhanced data set; using the above-mentioned sample data set and the above-mentioned enhanced data set as training data sets to perform model training on the initial license plate recognition model to generate a license plate recognition model.
[0024] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.
[0025] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.
[0026] In a fifth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner in the above-mentioned first aspect when executed by a processor.
[0027] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the license plate information generation method of some embodiments of the present disclosure can accurately and efficiently identify the license plate information in the license plate image. Specifically, the reason why the relevant license plate information is not accurate enough is that the feature learning focus of the classification task and the regression task of the license plate character is different, and the output layer of the detection head model of yolov5 outputs the license plate character classification information and the license plate character regression information at the same time, resulting in the character information output by the detection head model of YOLOv5 not being accurate enough. Based on this, the license plate information generation method of some embodiments of the present disclosure first obtains the target license plate image to subsequently identify the license plate information in the target license plate image. Then, the target license plate image is input into the hierarchical feature information generation model in the license plate recognition model to generate hierarchical license plate feature information. Here, the hierarchical feature information generation model can generate accurate hierarchical feature information for the target license plate image to improve the accuracy of subsequent license plate recognition. Furthermore, the hierarchical license plate feature information is input into the decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image. Among them, the above-mentioned decoupled detection head model includes: an output layer set. Each output layer outputs a single type of character information. Here, by decoupling the detection head model in the license plate recognition model to obtain a model including an output layer set, each output layer outputs one type of character information, which not only avoids the output layer of the detection head model from performing the relevant regression task and the relevant classification task for the license plate characters at the same time, but also avoids the problem that the final character information output is not accurate enough due to the different emphasis of character feature learning. As a result, the task features of the corresponding task are focused on learning for each output layer, which greatly improves the output accuracy of the output layer, and the obtained character information set is also more accurate. Finally, according to the above-mentioned character information set, the license plate information can be accurately generated. In summary, through the decoupled detection model, the feature learning of the relevant classification task and the relevant regression task for the license plate characters is distinguished and learned, and each output layer is limited to output a single type of character information, which greatly improves the recognition accuracy of the license plate information. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0029] Figure 1 is a schematic diagram of an application scenario of a method for generating license plate information according to some embodiments of the present disclosure;
[0030] Figure 2is a flow chart of some embodiments of the method for generating license plate information according to the present disclosure;
[0031] Figure 3 is a flow chart of other embodiments of the method for generating license plate information according to the present disclosure;
[0032] Figure 4 is a schematic diagram of the network structure of the character category output layer, the character probability output layer and the character position output layer in some embodiments of the license plate information generation method disclosed in the present invention;
[0033] Figure 5 is a schematic diagram of the structure of some embodiments of the vehicle license plate information generating device according to the present disclosure;
[0034] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0035] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0036] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0037] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0038] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0039] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0040] For operations such as the collection, storage, and use of information (such as license plate information) involved in this disclosure, before performing the corresponding operations, relevant organizations or individuals shall fulfill obligations including conducting information security impact assessments, fulfilling the obligation of informing the information subject, and obtaining the prior authorization and consent of the information subject.
[0041] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0042] Figure 1 It is a schematic diagram of an application scenario of a license plate information generation method according to some embodiments of the present disclosure.
[0043] In Figure 1 In the application scenario, first, the electronic device 101 can obtain the target license plate image 102. Then, the electronic device 101 can input the above target license plate image 102 into the hierarchical feature information generation model 104 in the license plate recognition model 103 to generate hierarchical license plate feature information 105. Next, the electronic device 101 can input the above hierarchical license plate feature information 105 into the decoupled detection head model 106 included in the above license plate recognition model 103 to output a character information set 109 for the above target license plate image 102. Among them, the above decoupled detection head model 106 includes: an output layer set. Each output layer outputs character information of a single type. In this application scenario, the output layer set may include: output layer 107 and output layer 108. The character information set 109 includes: character information 1091 and character information 1092. The character information 1091 may be "Character: Jing, A, 8, 0, 3, 7". The character information 1092 may be "Character: Jing, A, 8, 0, 3, 5, 7". Finally, the electronic device 101 can generate license plate information 110 according to the above character information set 109. In this application scenario, the license plate information 110 may be "Jing A80**3".
[0044] It should be noted that the above electronic device 101 can be hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or can be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it can be installed in the above-listed hardware devices. It can be implemented as, for example, multiple software or software modules for providing distributed services, or can be implemented as a single software or software module. No specific limitation is made here.
[0045] It should be understood that Figure 1 the number of electronic devices in
[0046] Continue to refer to Figure 2, shows a process 200 of some embodiments of the license plate information generation method according to the present disclosure. The license plate information generation method comprises the following steps:
[0047] Step 201, obtaining a target license plate image.
[0048] In some embodiments, the execution subject of the above license plate information generation method (for example Figure 1 The electronic device 101 shown can obtain the target license plate image through a wired connection or a wireless connection. The target license plate image can be an image to be subjected to license plate recognition. In practice, the target license plate image can be an image of a license plate in front of a vehicle or an image of a license plate behind a vehicle.
[0049] Step 202: input the target license plate image into the hierarchical feature information generation model in the license plate recognition model to generate hierarchical license plate feature information.
[0050] In some embodiments, the above-mentioned execution entity may input the above-mentioned target license plate image into the hierarchical feature information generation model in the license plate recognition model to generate hierarchical license plate feature information. Among them, the license plate recognition model may be a model for recognizing license plate character information. License plate characters may be the smallest splitting units of license plate numbers. In practice, for the license plate number "Jing A28**12", the corresponding license plate character set may include: "Jing", "A", "2", "8", "*", "*", "1", "2". The above-mentioned hierarchical feature information model may be a model for generating specific hierarchical feature information in the license plate image. The above-mentioned specific hierarchical feature information may include, but is not limited to, at least one of the following: high-level feature information, middle-level feature information, and low-level feature information. The overall feature representation of the license plate image corresponding to the above-mentioned high-level feature information is stronger than that of the middle-level feature information, and the corresponding detail feature of the license plate image is lower than that of the middle-level feature information. Similarly, the overall feature representation of the license plate image corresponding to the above-mentioned middle-level feature information is stronger than that of the low-level feature information, and the corresponding detail feature representation of the license plate image is lower than that of the low-level feature information. The overall feature representation of the license plate image may represent the overall feature situation of the license plate image. In practice, the overall feature representation of the license plate image may be contour feature information. The detail feature representation of the license plate image may represent the detail feature situation of the license plate image. In practice, the detail feature representation of the license plate image may be license plate texture information. In practice, the above-mentioned hierarchical license plate feature information may be in the form of a vector. The above-mentioned hierarchical feature information generation model may include: the input network of YOLO v5, the backbone network (Backone network) of YOLO v5, and the Neck network of YOLO v5. Among them, the backbone network may be a convolutional neural network (Convolutional Neural Networks, CNN) that aggregates and forms image features at different image granularities. The Neck network may be a series of network layers that mix and combine image features.
[0051] In some optional implementation manners of some embodiments, the above-mentioned license plate recognition model is trained through the following steps:
[0052] First step, obtain a sample data set.
[0053] Among them, the sample data set includes: a license plate data set and a corresponding license plate number label set.
[0054] Second step, perform sample data splicing enhancement on the sample data in the above-mentioned sample data set to generate an enhanced data set.
[0055] As an example, the execution subject may perform a Mosaic operation (image cropping and splicing operation) on the sample data in the sample data set to obtain the sample data set after the operation. Then, the sample data set after the operation is enhanced by Mix Up to generate enhanced data to obtain an enhanced data set. The probability of using the Mix Up operation may be 0.5. The Mix Up operation may adjust the transparency of at least two sample data, and then perform an operation of superimposing the sample data.
[0056] The third step is to use the above sample data set and the above enhanced data set as training data sets to train the initial license plate recognition model to generate a license plate recognition model.
[0057] Step 203: input the hierarchical license plate feature information into the decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image.
[0058] In some embodiments, the execution subject may input the hierarchical license plate feature information into the decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image. The decoupled detection head model may be a model obtained by decoupling the output layer of the detection head model. The output result of the output layer of the detection head model may include: relevant classification task information for license plate characters and relevant regression information for license plate characters. In practice, the relevant classification task information may include: the character type of the license plate characters and the place of ownership of the license plate characters. The relevant regression information of the license plate characters may include: the accuracy of the license plate characters. The decoupled detection head model includes an output layer set, and for each classification task of the license plate characters, there is a corresponding separate output layer for classification output. Similarly, for each regression task of the license plate characters, there is a corresponding separate output layer for regression prediction. The single type may be the output result type of executing a single character task. That is, different character tasks correspond to different output character types. Each character information in the above character information set may be the execution result of different character tasks, and the corresponding output character types may be different. The character information set may include but is not limited to at least one of the following: character category information, character position information. The character category information may represent the content of the license plate characters. The character position information may represent the position information of the license plate characters.
[0059] As an example, the decoupled detection head model can set up multiple serially connected convolutional neural networks as multiple output layers according to different character tasks to generate a character information set for the above-mentioned target license plate image based on the hierarchical license plate feature information.
[0060] Step 204: Generate license plate information based on the above character information set.
[0061] In some embodiments, the execution subject may generate license plate information according to the character information set, wherein the license plate information may be a predicted license plate number in the target license plate image.
[0062] As an example, first, the execution subject can perform corresponding splicing on each character information in the character information set to obtain a character splicing information set. Then, each character splicing information in the character splicing information set is vectorized to obtain a splicing vector. Next, the target license plate image is image encoded to obtain a coding vector. Finally, the character splicing information with the highest similarity between the corresponding splicing vector and the coding vector is selected from the character splicing information set as the license plate information.
[0063] In some optional implementations of some embodiments, generating license plate information according to the above character information set may include the following steps:
[0064] In the first step, the execution subject can determine the license plate character set for the target license plate image according to the character information set, wherein the license plate characters can be characters in the license plate, corresponding to the character category.
[0065] As an example, for each character position, the execution subject may determine a character category set corresponding to the character position, and then select the character category with the largest number from the character category set as the target character category to obtain the license plate character.
[0066] As another example, the execution subject may use a non-maximum suppression algorithm to filter out the corresponding license plate characters at each position from the character information set to obtain a license plate character set.
[0067] In the second step, the execution entity may combine characters of the license plate character set to generate license plate information.
[0068] As an example, the execution subject may sort and combine the license plate characters in sequence according to the order of the positions of the characters to obtain the license plate information.
[0069] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the license plate information generation method of some embodiments of the present disclosure can accurately and efficiently identify the license plate information in the license plate image. Specifically, the reason why the relevant license plate information is not accurate enough is that the feature learning focus of the classification task and the regression task of the license plate character is different, and the output layer of the detection head model of yolov5 outputs the license plate character classification information and the license plate character regression information at the same time, resulting in the character information output by the detection head model of YOLOv5 not being accurate enough. Based on this, the license plate information generation method of some embodiments of the present disclosure first obtains the target license plate image to subsequently identify the license plate information in the target license plate image. Then, the target license plate image is input into the hierarchical feature information generation model in the license plate recognition model to generate hierarchical license plate feature information. Here, the hierarchical feature information generation model can generate accurate hierarchical feature information for the target license plate image to improve the accuracy of subsequent license plate recognition. Furthermore, the hierarchical license plate feature information is input into the decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image. Among them, the above-mentioned decoupled detection head model includes: an output layer set. Each output layer outputs a single type of character information. Here, by decoupling the detection head model in the license plate recognition model to obtain a model including an output layer set, each output layer outputs one type of character information, which not only avoids the output layer of the detection head model from performing the relevant regression task and the relevant classification task for the license plate characters at the same time, but also avoids the problem that the final character information output is not accurate enough due to the different emphasis of character feature learning. As a result, the task features of the corresponding task are focused on learning for each output layer, which greatly improves the output accuracy of the output layer, and the obtained character information set is also more accurate. Finally, according to the above-mentioned character information set, the license plate information can be accurately generated. In summary, through the decoupled detection model, the feature learning of the relevant classification task and the relevant regression task for the license plate characters is distinguished and learned, and each output layer is limited to output a single type of character information, which greatly improves the recognition accuracy of the license plate information.
[0070] Further references Figure 3 , shows a process 300 of another embodiment of the method for generating license plate information according to the present disclosure. The method for generating license plate information comprises the following steps:
[0071] Step 301, obtaining a target license plate image.
[0072] Step 302: input the target license plate image into the hierarchical feature information generation model in the license plate recognition model to generate hierarchical license plate feature information.
[0073] Step 303: input each multi-level license plate feature information in the multi-level license plate feature information set into a corresponding multi-level detection head model to output a character information group to obtain a character information group set.
[0074] In some embodiments, an execution entity (e.g. Figure 1 The electronic device 101 shown) can input each multi-level license plate feature information in the above-mentioned multi-level license plate feature information set into the corresponding multi-level detection head model to output a character information group to obtain a character information group set. Among them, the above-mentioned hierarchical license plate feature information includes: a multi-level license plate feature information set. The above-mentioned decoupled detection head model includes: a multi-level detection head model set. The multi-level detection head model includes: an output layer group. That is, each multi-level detection head model has a corresponding output layer group, and each output layer in the output layer group outputs a single type of character information. The output layer group is a subset of the above-mentioned output layer set. The output layer group set is equivalent to the output layer set.
[0075] In practice, the multi-level license plate feature information set may include: high-level license plate feature information, mid-level license plate feature information, and low-level license plate feature information. The high-level license plate feature information, mid-level license plate feature information, and low-level license plate feature information will not be described in detail. For details, please refer to the relevant explanations of high-level feature information, mid-level feature information, and low-level feature information. There is a one-to-one correspondence between the multi-level license plate feature information in the multi-level license plate feature information set and the multi-level detection heads in the multi-level detection head model set. That is, the multi-level detection head model set includes: a multi-level detection head model for input of high-level license plate feature information, a multi-level detection head model for input of mid-level license plate feature information, and a multi-level detection head model for input of low-level license plate feature information.
[0076] In some optional implementations of some embodiments, the multi-level detection head model includes at least one of the following: a character category output layer, a character position output layer, and a character probability output layer. Among them, the character category output layer can be a network layer that outputs the character category. Among them, the character category can be the letter content corresponding to the character or the digital content corresponding to the character. The character position output layer can be a network layer that outputs the character position information. The character position can be the position information of the character on the license plate image. The above-mentioned character position information can be represented by multiple coordinates. The above-mentioned character probability output layer can be a network layer that outputs the character probability. The character probability can represent the character accuracy. The larger the character probability, the higher the accuracy of the character representation. In practice, the character category output layer can be a multi-layer serially connected residual neural network. The character position output layer can be a multi-layer serially connected convolutional neural network. The character probability output layer can be a multi-layer serially connected residual neural network + a multi-layer serially connected convolutional neural network.
[0077] Optionally, the step of inputting each multi-level license plate feature information in the multi-level license plate feature information set into a corresponding multi-level detection head model to output a character information group may include the following steps:
[0078] In the first step, the execution entity may input the multi-level license plate feature information into the corresponding character category output layer to output the character category information as character information.
[0079] In a second step, the execution entity may input the multi-level license plate feature information into a corresponding character position output layer to output character position information as character information.
[0080] In a third step, the execution entity may input the multi-level license plate feature information into a corresponding character probability output layer to output character probability information as character information.
[0081] In some optional implementations of some embodiments, the character category output layer includes: a first convolution layer, a first depthwise separable convolution layer (Depthwise Convolution), a second convolution layer, and a category output layer. The category output layer may be a network layer that outputs character categories based on convolution features. In practice, the category output layer may be a fully connected layer with multiple layers connected in series.
[0082] Optionally, the step of inputting the multi-level license plate feature information into the corresponding character category output layer to output the character category information may include the following steps:
[0083] In the first step, the execution subject may input the multi-level license plate feature information into the corresponding first convolution layer to generate first convolution information, wherein the first convolution information may represent the feature information of the multi-level license plate.
[0084] In a second step, the execution entity may input the first convolution information into a corresponding first depth-separable convolution layer to output second convolution information.
[0085] In the third step, the execution entity may input the second convolution information into the corresponding second convolution layer to output third convolution layer information.
[0086] In a fourth step, the execution entity may input the third convolution information into a corresponding category output layer to output character category information.
[0087] In some optional implementations of some embodiments, the character position output layer includes: a first convolution layer, a second depth-separable convolution layer, a third convolution layer, and a position output layer. The position output layer may be a network layer that outputs the character position based on the convolution feature. In practice, the position output layer may be a fully connected layer with multiple layers connected in series.
[0088] Optionally, the step of inputting the multi-layered license plate feature information into the corresponding character position output layer to output the character position information may include the following steps:
[0089] In the first step, the execution entity may input the multi-level license plate feature information into the corresponding first convolution layer to output the first convolution information.
[0090] In a second step, the execution entity may input the first convolution information into a corresponding second depth-separable convolution layer to output fourth convolution information.
[0091] In the third step, the execution entity may input the fourth convolution information into the corresponding third convolution layer to output the fifth convolution layer information.
[0092] In the fourth step, the execution entity may input the fifth convolution information into the corresponding position output layer to output character position information.
[0093] In some optional implementations of some embodiments, the character probability output layer includes: a first convolution layer, a second depth-separable convolution layer, a fourth convolution layer, and a probability output layer. The probability output layer can be a network layer that outputs character accuracy probabilities based on convolution features. In practice, the probability output layer can be a fully connected layer with multiple layers connected in series.
[0094] Optionally, the step of inputting the multi-level license plate feature information into the corresponding character probability output layer to output character probability information may include the following steps:
[0095] In the first step, the execution entity may input the multi-level license plate feature information into the corresponding first convolution layer to output the first convolution information.
[0096] In a second step, the execution entity may input the first convolution information into a corresponding second depth-separable convolution layer to output fourth convolution information.
[0097] In the third step, the execution entity may input the fourth convolution information into the corresponding fourth convolution layer to output the sixth convolution layer information.
[0098] In a fourth step, the execution entity may input the sixth convolution information into a corresponding probability output layer to output character probability information.
[0099] As an example, the network structure of the specific character category output layer, character probability output layer and character position output layer can be found in Figure 4 .
[0100] Step 304: determine the above character information group set as the above character information set.
[0101] In some embodiments, the execution subject may determine the character information group set as the character information set. The multi-level detection head model includes: an output layer group, which is a subset of the output layer set.
[0102] Step 305: Generate license plate information based on the above character information set.
[0103] In some embodiments, the specific implementation of steps 301-302 and 305 and the technical effects thereof can be referred to in Figure 2 The steps 201 - 202 and 204 in the corresponding embodiment are not described in detail here.
[0104] from Figure 3 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 3 The process 300 of the license plate information generation method in some corresponding embodiments highlights the generation of a character information set based on a multi-level license plate feature information set and a multi-level detection head model set. In the process of generating the character information set, the license plate feature information at various levels is fully considered, and the corresponding multi-level detection head model is used to accurately output the character information set.
[0105] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a license plate information generating device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the license plate information generating device can be specifically applied to various electronic devices.
[0106] like Figure 5 As shown, a license plate information generating device 500 includes: an acquisition unit 501, a first input unit 502, a second input unit 503 and a generating unit 504. The acquisition unit 501 is configured to acquire a target license plate image; the first input unit 502 is configured to input the target license plate image into a hierarchical feature information generating model in a license plate recognition model to generate hierarchical license plate feature information; the second input unit 503 is configured to input the hierarchical license plate feature information into a decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image, wherein the decoupled detection head model includes: an output layer set, each output layer outputting a single type of character information; the generating unit 504 is configured to generate license plate information according to the character information set.
[0107] In some optional implementations of some embodiments, the above-mentioned hierarchical license plate feature information includes: a multi-level license plate feature information set, the above-mentioned decoupled detection head model includes: a multi-level detection head model set; and the first input unit 502 can be further configured to: input each multi-level license plate feature information in the above-mentioned multi-level license plate feature information set into the corresponding multi-level detection head model to output a character information group to obtain a character information group set, wherein the multi-level detection head model includes: an output layer group, the output layer group is a subset of the above-mentioned output layer set; and the above-mentioned character information group set is determined as the above-mentioned character information set.
[0108] In some optional implementations of some embodiments, the above-mentioned multi-level detection head model includes at least one of the following: a character category output layer, a character position output layer, and a character probability output layer; and the first input unit 502 can be further configured to: input the above-mentioned multi-level license plate feature information into the corresponding character category output layer to output the character category information as character information; input the above-mentioned multi-level license plate feature information into the corresponding character position output layer to output the character position information as character information; input the above-mentioned multi-level license plate feature information into the corresponding character probability output layer to output the character probability information as character information.
[0109] In some optional implementations of some embodiments, the above-mentioned character category output layer includes: a first convolution layer, a first depth-separable convolution layer, a second convolution layer, and a category output layer; and the first input unit 502 can be further configured to: input the above-mentioned multi-level license plate feature information into the corresponding first convolution layer to generate first convolution information; input the above-mentioned first convolution information into the corresponding first depth-separable convolution layer to output second convolution information; input the above-mentioned second convolution information into the corresponding second convolution layer to output third convolution layer information; input the above-mentioned third convolution information into the corresponding category output layer to output character category information.
[0110] In some optional implementations of some embodiments, the above-mentioned character category output layer includes: a first convolution layer, a second depth-separable convolution layer, a third convolution layer, and a position output layer; and the first input unit 502 can be further configured to: input the above-mentioned first convolution information into the corresponding first convolution layer to output the first convolution information; input the above-mentioned first convolution information into the corresponding second depth-separable convolution layer to output the fourth convolution information; input the above-mentioned fourth convolution information into the corresponding third convolution layer to output the fifth convolution layer information; input the above-mentioned fifth convolution information into the corresponding position output layer to output the character position information.
[0111] In some optional implementations of some embodiments, the above-mentioned character category output layer includes: a first convolution layer, a second depth-separable convolution layer, a fourth convolution layer, and a probability output layer; and the first input unit 502 can be further configured to: input the above-mentioned first convolution information into the corresponding first convolution layer to output the first convolution information; input the above-mentioned first convolution information into the corresponding second depth-separable convolution layer to output the fourth convolution information; input the above-mentioned fourth convolution information into the corresponding fourth convolution layer to output the sixth convolution layer information; input the above-mentioned sixth convolution information into the corresponding probability output layer to output the character probability information.
[0112] In some optional implementations of some embodiments, the generating unit 502 may be further configured to: determine a license plate character set for the target license plate image according to the character information set; and perform character combination on the license plate character set to generate license plate information.
[0113] In some optional implementations of some embodiments, the above-mentioned license plate recognition model is trained through the following steps: obtaining a sample data set; performing sample data splicing enhancement on the sample data in the above-mentioned sample data set to generate an enhanced data set; using the above-mentioned sample data set and the above-mentioned enhanced data set as training data sets to perform model training on the initial license plate recognition model to generate a license plate recognition model.
[0114] It is understandable that the units recorded in the license plate information generating device 500 are similar to those in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the license plate information generating device 500 and the units included therein, and will not be described in detail here.
[0115] Reference below Figure 6 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the electronic device 101)600. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0116] like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage device 608 into a random access memory 603. Various programs and data required for the operation of the electronic device 600 are also stored in the random access memory 603. The processing device 601, the read-only memory 602, and the random access memory 603 are connected to each other via a bus 604. An input / output interface 605 is also connected to the bus 604.
[0117] Typically, the following devices may be connected to the input / output interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0118] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a read-only memory 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0119] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0120] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0121] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a target license plate image; inputs the target license plate image into a hierarchical feature information generation model in a license plate recognition model to generate hierarchical license plate feature information; inputs the hierarchical license plate feature information into a decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image, wherein the decoupled detection head model includes: an output layer set, each output layer outputs a single type of character information; and generates license plate information according to the character information set.
[0122] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0123] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0124] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor includes an acquisition unit, a first input unit, a second input unit, and a generation unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring a target license plate image".
[0125] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0126] Some embodiments of the present disclosure also provide a computer program product, including a computer program, which implements any of the above-mentioned license plate information generation methods when executed by a processor.
[0127] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
Claims
1. A method for generating license plate information, comprising: Get the target license plate image; Inputting the target license plate image into a hierarchical feature information generation model in a license plate recognition model to generate hierarchical license plate feature information; Inputting the hierarchical license plate feature information into a decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image, wherein the decoupled detection head model includes: an output layer set, each output layer outputting a single type of character information; The license plate information is generated according to the character information set.
2. The method according to claim 1, wherein: The hierarchical license plate feature information includes: a multi-level license plate feature information set, and the decoupled detection head model includes: a multi-level detection head model set; and The step of inputting the hierarchical license plate feature information into the decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image comprises: Input each multi-level license plate feature information in the multi-level license plate feature information set into a corresponding multi-level detection head model to output a character information group to obtain a character information group set, wherein the multi-level detection head model includes: an output layer group, which is a subset of the output layer set; The character information group set is determined as the character information set.
3. The method according to claim 2, wherein: The multi-level detection head model includes at least one of the following: a character category output layer, a character position output layer, and a character probability output layer; as well as The step of inputting each multi-level license plate feature information in the multi-level license plate feature information set into a corresponding multi-level detection head model to output a character information group includes: Inputting the multi-level license plate feature information into a corresponding character category output layer to output character category information as character information; Inputting the multi-level license plate feature information into a corresponding character position output layer to output character position information as character information; The multi-level license plate feature information is input into the corresponding character probability output layer to output character probability information as character information.
4. The method according to claim 3, wherein: The character category output layer includes: a first convolution layer, a first depth-separable convolution layer, a second convolution layer, and a category output layer; and The step of inputting the multi-level license plate feature information into a corresponding character category output layer to output character category information includes: Inputting the multi-level license plate feature information into the corresponding first convolution layer to generate first convolution information; Inputting the first convolution information into a corresponding first depthwise separable convolution layer to output second convolution information; Inputting the second convolution information into the corresponding second convolution layer to output third convolution layer information; The third convolution information is input into the corresponding category output layer to output character category information.
5. The method according to claim 3, wherein: The character position output layer includes: a first convolution layer, a second depth-separable convolution layer, a third convolution layer, and a position output layer; and The step of inputting the multi-level license plate feature information into a corresponding character position output layer to output character position information as character information includes: Inputting the multi-level license plate feature information into the corresponding first convolution layer to output first convolution information; Inputting the first convolution information into a corresponding second depthwise separable convolution layer to output fourth convolution information; Inputting the fourth convolution information into the corresponding third convolution layer to output fifth convolution layer information; The fifth convolution information is input into the corresponding position output layer to output character position information.
6. The method according to claim 3, wherein: The character probability output layer includes: a first convolution layer, a second depth-separable convolution layer, a fourth convolution layer, and a probability output layer; and The step of inputting the multi-level license plate feature information into a corresponding character probability output layer to output character probability information includes: Inputting the multi-level license plate feature information into the corresponding first convolution layer to output first convolution information; Inputting the first convolution information into a corresponding second depthwise separable convolution layer to output fourth convolution information; Inputting the fourth convolution information into the corresponding fourth convolution layer to output sixth convolution layer information; The sixth convolution information is input into the corresponding probability output layer to output character probability information.
7. The method according to claim 1, wherein: The step of generating license plate information according to the character information set includes: Determining a license plate character set for the target license plate image according to the character information set; Character combination is performed on the license plate character set to generate license plate information.
8. The method according to claim 1, wherein: The license plate recognition model is trained by the following steps: Get a sample dataset; Performing sample data splicing enhancement on the sample data in the sample data set to generate an enhanced data set; The sample data set and the enhanced data set are used as training data sets to perform model training on the initial license plate recognition model to generate a license plate recognition model.
9. A license plate information generating device, comprising: An acquisition unit is configured to acquire a target license plate image; A first input unit is configured to input the target license plate image into a hierarchical feature information generation model in a license plate recognition model to generate hierarchical license plate feature information; The second input unit is configured to input the hierarchical license plate feature information into the decoupled detection head model included in the license plate recognition model to output a character information set for the target license plate image, wherein the decoupled detection head model includes: an output layer set, each output layer outputting a single type of character information; The generating unit is configured to generate license plate information according to the character information set.
10. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.
11. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
12. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
License plate identification method, device and equipment and readable storage medium
CN108564088A
License plate character recognition method based on deep learning
CN109753914A
End-to-end license plate recognition method and device based on lightweight network
CN112215229A
License plate number recognition method and device, electronic device and storage medium
WO2021103897A1
License plate character recognition method and apparatus, electronic device, and storage medium
WO2021196873A1
