License plate recognition method, license plate recognition device, electronic equipment and storage medium

Abstract

The invention discloses a license plate recognition method, a license plate recognition device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a license plate image; a license plate image is input to a corner coordinate detection model, corresponding license plate corner coordinates are obtained, and the corner coordinate detection model comprises a target detection model and a feature point detection model; determining a target license plate area corresponding to the license plate image according to the license plate corner coordinates; and performing optical character recognition on the target license plate area to obtain a corresponding target license plate number. According to the method, optical character recognition is carried out on the target license plate area corresponding to the license plate image in the license plate recognition process, the corresponding target license plate number is obtained, license plate number recognition errors caused by recognition of numbers of non-license plate areas are avoided, and the recognition accuracy of license plate recognition is improved.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular relates to a license plate recognition method, a license plate recognition device, an electronic device and a storage medium. Background Art

[0002] With the gradual development of the economy, the number of vehicles has increased year by year, and the urban traffic pressure we face is also increasing. How to efficiently manage traffic has become an urgent problem that needs to be solved. License plate recognition technology plays a very important role in traffic management. From traffic violations to accident monitoring, the ability to automatically recognize license plates is one of the key tools used by law enforcement agencies in various places. License plate recognition is not only widely used in road traffic management, but also more widely used in parking lots, community security, stolen vehicle wanted, etc.

[0003] At present, in the process of license plate recognition, the corresponding license plate number is obtained by performing image recognition on the collected license plate image. However, the collected license plate image usually contains a license plate area and a non-license plate area. When there is a number in the non-license plate area, the license plate number obtained by performing image recognition on the license plate image consists of the real license plate number corresponding to the license plate area and the non-real license plate number corresponding to the non-license plate area. The license plate number recognition error leads to low recognition accuracy of the license plate. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a license plate recognition method, a license plate recognition device, an electronic device and a storage medium to overcome or at least partially solve the above problems of the prior art.

[0005] In a first aspect, an embodiment of the present application provides a license plate recognition method, including: acquiring a license plate image; inputting the license plate image into a corner point coordinate detection model to obtain corresponding license plate corner point coordinates, the corner point coordinate detection model including a target detection model and a feature point detection model; determining a target license plate area corresponding to the license plate image according to the license plate corner point coordinates; performing optical character recognition on the target license plate area to obtain the corresponding target license plate number.

[0006] In the second aspect, the embodiment of the present application provides a license plate recognition device, including an acquisition module, an input module, a region determination module and a license plate region recognition module. The acquisition module is used to acquire a license plate image; the input module is used to input the license plate image into a corner point coordinate detection model to obtain the corresponding license plate corner point coordinates, and the corner point coordinate detection model includes a target detection model and a feature point detection model; the region determination module is used to determine the target license plate region corresponding to the license plate image according to the license plate corner point coordinates; the license plate region recognition module is used to perform optical character recognition on the target license plate region to obtain the corresponding target license plate number.

[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory; one or more processors coupled to the memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the license plate recognition method provided in the first aspect above.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the license plate recognition method provided in the first aspect above.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes the license plate recognition method provided in the first aspect above.

[0010] The solution provided by the present application obtains the corresponding license plate corner point coordinates by acquiring a license plate image and inputting the license plate image into a corner point coordinate detection model. The corner point coordinate detection model includes a target detection model and a feature point detection model. According to the license plate corner point coordinates, the target license plate area corresponding to the license plate image is determined, and optical character recognition is performed on the target license plate area to obtain the corresponding target license plate number. In the license plate recognition process, optical character recognition is performed on the target license plate area corresponding to the license plate image to obtain the corresponding target license plate number, thereby avoiding incorrect license plate number recognition caused by recognizing numbers in non-license plate areas and improving the recognition accuracy of license plate recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A schematic diagram of a scenario of a license plate recognition system provided in an embodiment of the present application is shown.

[0013] Figure 2 A flow chart of a license plate recognition method provided in an embodiment of the present application is shown.

[0014] Figure 3 A structural schematic diagram of a corner point coordinate detection model in the license plate recognition method provided in an embodiment of the present application is shown.

[0015] Figure 4Another flow chart of the license plate recognition method provided in an embodiment of the present application is shown.

[0016] Figure 5 A structural block diagram of a license plate recognition device provided in an embodiment of the present application is shown.

[0017] Figure 6 A functional block diagram of an electronic device provided in an embodiment of the present application is shown.

[0018] Figure 7 A computer-readable storage medium provided in an embodiment of the present application for storing or carrying a program code for implementing a license plate recognition method provided in an embodiment of the present application is shown.

[0019] Figure 8 A computer program product provided in an embodiment of the present application and used to store or carry program codes for implementing the license plate recognition method provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0022] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates other working conditions.

[0023] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0024] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0025] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0026] With the gradual development of the economy, the number of vehicles has increased year by year, and the urban traffic pressure we face is also increasing. How to efficiently manage traffic has become an urgent problem that needs to be solved. License plate recognition technology plays a very important role in traffic management. From traffic violations to accident monitoring, the ability to automatically recognize license plates is one of the key tools used by law enforcement agencies in various places. License plate recognition is not only widely used in road traffic management, but also more widely used in parking lots, community security, stolen vehicle wanted, etc.

[0027] At present, in the process of license plate recognition, the corresponding license plate number is obtained by performing image recognition on the collected license plate image. However, the collected license plate image usually contains a license plate area and a non-license plate area. When there is a number in the non-license plate area, the license plate number obtained by performing image recognition on the license plate image consists of the real license plate number corresponding to the license plate area and the non-real license plate number corresponding to the non-license plate area. The license plate number recognition error leads to low recognition accuracy of the license plate.

[0028] In response to the above problems, the inventors have conducted a long period of research and proposed the license plate recognition method, license plate recognition device, electronic device and storage medium provided in the embodiments of the present application. The license plate image is acquired and input into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates. The corner point coordinate detection model includes a target detection model and a feature point detection model. According to the license plate corner point coordinates, the target license plate area corresponding to the license plate image is determined, and optical character recognition is performed on the target license plate area to obtain the corresponding target license plate number. In the license plate recognition process, optical character recognition is performed on the target license plate area corresponding to the license plate image to obtain the corresponding target license plate number, thereby avoiding the recognition error of the license plate number caused by the recognition of the number in the non-license plate area, and improving the recognition accuracy of the license plate.

[0029] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0030] See also Figure 1 , which shows a schematic diagram of an application scenario of the license plate recognition system provided in an embodiment of the present application, which may include a processing device 100 and a vehicle 200. The processing device 100 can obtain a license plate image of the vehicle 200, and perform optical character recognition (OCR) on the license plate area of ​​the license plate image to obtain a corresponding license plate number.

[0031] The processing device 100 may be a terminal device or a server, etc. The type of the processing device 100 is not limited here and may be specifically configured according to actual needs.

[0032] The terminal device can be a mobile terminal device (for example, any one of a mobile phone, a personal digital assistant (PDA), a tablet personal computer (Tablet PC), a laptop computer, a smart watch, a smart bracelet or a wearable device, etc.), or a fixed terminal device (desktop computer, smart panel, etc.), etc.

[0033] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or any one of the cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), big data or artificial intelligence platforms, etc.

[0034] Vehicle 200 can be an electric vehicle (for example, an electric car or a battery car, etc.), a hybrid vehicle (for example, a hybrid electric vehicle (HEV)), a fuel vehicle or a gas vehicle, etc. The type of vehicle 200 is not limited here and can be set according to actual needs.

[0035] See also Figure 2 , which shows a flow chart of a license plate recognition method provided by an embodiment of the present application. In a specific embodiment, the license plate recognition method can be applied to Figure 1 The processing device 100 in the license plate recognition system shown in FIG. 1 is taken as an example to describe the processing device 100 in the following. Figure 2 The process shown is described in detail, and the license plate recognition method may include the following steps S110 to S140.

[0036] Step S110: Acquire a license plate image.

[0037] In an embodiment of the present application, when a user needs to identify the license plate number of a vehicle, a recognition instruction can be sent to a processing device, and the processing device receives and responds to the recognition instruction to obtain an image of the license plate of the vehicle.

[0038] In some embodiments, the processing device may be provided with a camera. When a user needs to identify a vehicle's license plate number, an identification instruction may be sent to the processing device. The processing device receives and responds to the identification instruction, controls the camera to capture an image of the vehicle's license plate, and obtains a corresponding license plate image.

[0039] Among them, the camera can be any one of a white light camera, an infrared camera, a laser camera, etc. The type of camera is not limited here and can be set according to actual needs.

[0040] In some embodiments, the license plate recognition system may further include a camera, which is connected to the processing device via a network and exchanges data with the processing device via the network.

[0041] When the user needs to identify the vehicle's license plate number, he can send an identification instruction to the processing device. The processing device receives and responds to the identification instruction and sends an image acquisition instruction to the camera via the network. The camera receives and responds to the image acquisition instruction, acquires the image of the vehicle's license plate, obtains the corresponding license plate image, and sends the license plate image to the processing device via the network. The processing device receives the license plate image returned by the camera.

[0042] Among them, the network can be any one of a ZigBee network, a Bluetooth (BT) network, a Wireless Fidelity (Wi-Fi) network, a Thread network, a Long Range Radio (LoRa) network, a Low-Power Wide-Area Network (LPWAN), an infrared network, a Narrow Band Internet of Things (NB-IoT), a Controller Area Network (CAN), a Digital Living Network Alliance (DLNA) network, a Wide Area Network (WAN), a Local Area Network (LAN), a Metropolitan Area Network (MAN) or a Wireless Personal Area Network (WPAN), etc. The type of network is not limited here and can be set according to actual needs.

[0043] In some embodiments, the processing device may include an input panel. When a user needs to identify a vehicle license plate number, the user may input an identification instruction on the input panel, and the processing device receives the identification instruction through the input panel.

[0044] In some embodiments, the processing device may include a voice recognition module. When a user needs to identify a vehicle's license plate number, a voice message carrying a recognition instruction can be sent within the voice collection range of the voice recognition module. The processing device collects the voice information through the voice recognition module, performs voice recognition on the voice information, and obtains the recognition instruction.

[0045] In some implementations, the license plate recognition system may further include a client, which is connected to the processing device via a network and performs data exchange with the processing device via the network.

[0046] When the user needs to identify the vehicle's license plate number, an identification instruction can be sent to the client. The client receives and responds to the identification instruction and forwards the identification instruction to the processing device through the network. The processing device receives the identification instruction forwarded by the client.

[0047] Among them, the client can be any one of a mobile client (for example, a mobile phone client, a personal digital assistant (PDA) client, a tablet computer (Tablet Personal Computer, Tablet PC) client, a laptop computer client, a smart watch client, a smart bracelet client or a wearable client, etc.) or a fixed client (for example, a desktop computer client, a smart panel client, etc.). The type of client is not limited here and can be set according to actual needs.

[0048] Step S120: input the license plate image into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates.

[0049] In an embodiment of the present application, after acquiring the license plate image, the processing device can input the license plate image into the corner point coordinate detection model. The corner point coordinate detection model receives and responds to the license plate image, detects the license plate corner point coordinates corresponding to the license plate image, obtains the license plate corner point coordinates, and outputs the license plate corner point coordinates to the processing device. The processing device receives the license plate corner point coordinates output by the corner point coordinate detection model.

[0050] The corner point coordinate detection model may include a target detection model and a landmark detection model. The target detection model may be any one of a YOLO model, an SSD model, a RetinaNet model, or a Faster R-CNN model. The type of the target detection model is not limited here and may be set according to actual needs.

[0051] In one application scenario, such as Figure 3 As shown, the corner point coordinate detection model may include a first branch, a second branch, and a third branch. The target detection model includes a first branch and a second branch, and the landmark detection model includes a third branch. The first branch, the second branch, and the third branch of the target detection model may be fused to obtain a corner point coordinate detection model.

[0052] The first branch can be used to detect the license plate type of the license plate image to obtain the corresponding license plate type information (classification, cls). CLS can include any of ordinary blue license plate, single-layer yellow license plate, training license plate, police license plate, new energy white and green license plate, new energy yellow and green license plate, hanging license plate, double-layer yellow license plate, temporary ticket, etc., without limitation here.

[0053] The second branch can be used to detect the license plate frame information in the license plate image to obtain the corresponding license plate frame information. The license plate frame information can include the license plate frame value (rectangle, reg) and the probability that the license plate frame is an object (object, obj). reg can include the coordinates of the center point of the license plate frame (x0, y0) and the height of the license plate frame (w, h), etc. The value range of obj is [0, 1].

[0054] The third branch can be used to detect the coordinates of the corner points of the license plate in the license plate image to obtain the corresponding license plate corner point coordinate information (lmk). LMK may include the coordinates of the four corner points of the license plate in the license plate image, for example, the coordinates of the upper left corner point of the license plate (x1, y1), the coordinates of the lower left corner point of the license plate (x2, y2), the coordinates of the upper right corner point of the license plate (x3, y3), and the coordinates of the lower right corner point of the license plate (x4, y4).

[0055] Step S130: Determine the target license plate area corresponding to the license plate image according to the coordinates of the license plate corner points.

[0056] In the embodiment of the present application, after the processing device inputs the license plate image into the corner point coordinate detection model and obtains the corresponding license plate corner point coordinates, it can determine the target license plate area corresponding to the license plate image according to the license plate corner point coordinates. The target license plate area can be used to characterize the image area defined by the license plate corner point coordinates in the license plate image.

[0057] Specifically, after the processing device inputs the license plate image into the corner point coordinate detection model and obtains the corresponding license plate corner point coordinates, it can connect the license plate corner point coordinates in sequence to obtain the area surrounded by the license plate corner point coordinates, which is the target license plate area.

[0058] In some embodiments, due to image distortion during the image acquisition process, after the processing device inputs the license plate image into the corner point coordinate detection model and obtains the corresponding license plate corner point coordinates, it can determine the initial license plate area corresponding to the license plate image based on the license plate corner point coordinates, and perform perspective transformation on the initial license plate area to obtain the corresponding target license plate area, thereby achieving image correction of the license plate area corresponding to the license plate image, ensuring that the target license plate area is closer to the real license plate area, reducing image distortion of the target license plate area, and facilitating improving the recognition accuracy of the target license plate area when recognizing the license plate.

[0059] Among them, the processing device can correct and crop the initial license plate area according to the preset license plate aspect ratio and use perspective transformation technology to obtain the corresponding target license plate area.

[0060] The preset license plate aspect ratio may be an aspect ratio value preset by the user, or an aspect ratio value automatically generated by the processing device according to a recognition process of multiple license plate recognitions, etc. For example, the preset license plate aspect ratio may be 22:7, the preset license plate aspect ratio may be 20:11, the preset license plate aspect ratio may be 44:19, etc., which is not limited here.

[0061] Step S140: performing optical character recognition on the target license plate area to obtain the corresponding target license plate number.

[0062] In an embodiment of the present application, the processing device may include a preset optical character recognition application program interface (Optical Character Recognition Application Program Interface, OCR API), the OCR API is connected to the processing device, the processing device can call the corresponding optical character recognition application (Optical Character Recognition Application, OCR APP) through the OCR API, and the OCR APP can be used to perform optical character recognition (Optical Character Recognition, OCR) on the license plate image.

[0063] After the processing device determines the target license plate area corresponding to the license plate image according to the coordinates of the license plate corner points, it can call the corresponding OCR APP through the OCR API and send OCR instructions to the OCR APP. The OCR APP receives and responds to the OCR instructions, performs OCR on the target license plate area, and obtains the corresponding target license plate number. In the license plate recognition process, OCR is performed on the target license plate area corresponding to the license plate image to obtain the corresponding target license plate number, avoiding the recognition of license plate numbers caused by recognizing numbers in non-license plate areas, thereby improving the recognition accuracy of license plates.

[0064] In some implementations, the license plate OCR process is a process of performing OCR on a text area. When the text area overlaps or is disconnected, the text area is OCRed and the obtained license plate number format is incorrect, resulting in license plate recognition failure.

[0065] After the processing device determines the target license plate area corresponding to the license plate image based on the coordinates of the license plate corner points, it can obtain multiple character frames corresponding to the target license plate area, merge multiple character frames belonging to the same row to obtain the corresponding target text area, and perform OCR on the target text area to obtain the corresponding target license plate number. This can avoid license plate recognition failure due to overlapping or disconnected text areas during the license plate recognition process, thereby improving the recognition success rate of license plate recognition.

[0066] Among them, the processing device stores a pre-trained text detection model. The processing device can extract and input the target license plate area image corresponding to the target license plate area into the text detection model. The text detection model receives and responds to the target license plate area image, performs text detection on the target license plate area image, and obtains corresponding multiple character boxes.

[0067] The text detection model can be a Convolutional Neural Networks (CNN) model, a Deep Belief Networks (DBN) model, a Stacked AutoEncoder Networks (SAE) model, a Recurrent Neural Networks (RNN) model, a Deep Neural Networks (DNN) model, a Long Short-Term Memory (LSTM) network model or a Gated Recurring Units (GRU) model, etc. The type of text detection model is not limited here and can be set according to actual needs.

[0068] The processing device can determine the row sequence of each character frame according to the arrangement order of multiple character frames from top to bottom and from left to right, and perform fusion correction on the character frames belonging to the same row to obtain the corresponding target text area.

[0069] In some embodiments, the acquisition angle and / or environmental factors of the license plate image may cause shadows to be occluded in the license plate area or the image to be blurred. When performing OCR on the license plate area with shadows or blurred images, license plate recognition may fail.

[0070] After the processing device determines the target license plate area corresponding to the license plate image according to the coordinates of the license plate corner points, it can obtain multiple character frames corresponding to the target license plate area, and merge the multiple character frames belonging to the same row to obtain the corresponding target text area, and perform OCR on the target text area to obtain the corresponding target license plate number, and determine whether the format of the target license plate number is correct. When it is determined that the format of the target license plate number is incorrect, the target text area is corrected to obtain the corresponding corrected text area, and the corrected text area is performed OCR to obtain the corresponding corrected license plate number. This realizes correction of the target text area when the license plate recognition is wrong, and the license plate recognition is performed again on the corrected corrected text area to obtain the corresponding corrected license plate number. This can avoid the failure of license plate recognition due to shadow obstruction or image blur in the license plate area, thereby improving the recognition success rate of the license plate.

[0071] The processing device can determine whether the format of the target license plate number is correct according to the number of digits of the target license plate number. When the number of digits of the target license plate number is the number of digits of the preset license plate number, the format of the target license plate number is determined to be correct; when the number of digits of the target license plate number is not the number of digits of the preset license plate number, the format of the target license plate number is determined to be incorrect.

[0072] The preset number of digits of the license plate number can be a number of digits pre-set by the user, or a number of digits automatically generated by the processing device according to the recognition process of multiple license plates. For example, the preset number of digits of the license plate number can be 7 digits, or the preset number of digits of the license plate number can be 8 digits, etc., which is not limited here.

[0073] When the processing device determines that the format of the target license plate number is incorrect, it can determine the license plate type corresponding to the target text area, and correct the target text area according to the text box size corresponding to the license plate type to obtain the corresponding corrected text area.

[0074] Specifically, the processing device may be provided with a plurality of corresponding preset license plate templates according to the preset license plate type, and each preset license plate template corresponds to a preset PHash value.

[0075] When the processing device determines that the format of the target license plate number is incorrect, it can calculate the target PHash value corresponding to the target text area, determine a target preset PHash value that matches the target PHash value from multiple preset PHash values, and determine the preset license plate type corresponding to the target preset PHash value as the target license plate type of the target license plate area.

[0076] The solution provided in this embodiment obtains the license plate image and inputs the license plate image into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates, and determines the target license plate area corresponding to the license plate image based on the license plate corner point coordinates, and performs optical character recognition on the target license plate area to obtain the corresponding target license plate number. In the license plate recognition process, optical character recognition is performed on the target license plate area corresponding to the license plate image to obtain the corresponding target license plate number, thereby avoiding incorrect license plate number recognition caused by recognizing numbers in non-license plate areas and improving the recognition accuracy of license plate recognition.

[0077] See also Figure 4 , which shows a flow chart of a license plate recognition method provided by another embodiment of the present application. In a specific embodiment, the license plate recognition method can be applied to Figure 1 The processing device 100 in the license plate recognition system shown in FIG. 1 is taken as an example to describe the processing device 100 in the following. Figure 4 The process shown is described in detail, and the license plate recognition method may include the following steps S210 to S250.

[0078] Step S210: Acquire a license plate image.

[0079] In this embodiment, step S210 may refer to the contents of the corresponding steps in the aforementioned embodiments, which will not be described in detail here.

[0080] Step S220: construct and train a corner point coordinate detection model based on the target detection model and the feature point detection model.

[0081] In this embodiment, the processing device can fuse the target detection model and the landmark detection model to obtain the corresponding corner point coordinate detection model, obtain the historical license plate image, and mark the corner point coordinates of the historical license plate image to obtain the corresponding historical marked license plate image, and divide the historical marked license plate image according to the preset division rules to obtain the corresponding training set and test, and input the training set to the corner point coordinate detection model for training.

[0082] The preset division rule is an artificial division rule. For example, the preset division rule may be an artificial division rule of training set: test set = 8:7. When the number of historically annotated license plate images is 15,000, the training set is 8,000 and the test set is 7,000. The preset division rule may be an artificial division rule of training set: test set = 1:1. When the number of historically annotated license plate images is 20,000, the training set is 10,000 and the test set is 10,000. The division method of the preset division rule is not limited here and can be set according to actual needs.

[0083] In some embodiments, the target detection model is a YOLO model, and the processing device can fuse the YOLO model and the Landmark detection model to obtain a corresponding corner point coordinate detection model, obtain a historical license plate image, and mark the corner point coordinates of the historical license plate image to obtain the corresponding historical marked license plate image, and divide the historical marked license plate image according to preset division rules to obtain a corresponding training set and test, and input the training set to the corner point coordinate detection model for training.

[0084] In some embodiments, the target detection model is an SSD model, and the processing device can fuse the SSD model and the landmark detection model to obtain a corresponding corner point coordinate detection model, obtain historical license plate images, and mark the corner point coordinates of the historical license plate images to obtain the corresponding historical marked license plate images, and divide the historical marked license plate images according to preset division rules to obtain corresponding training sets and tests, and input the training sets to the corner point coordinate detection model for training.

[0085] In some embodiments, the target detection model is a RetinaNet model, and the processing device can fuse the RetinaNet model and the Landmark detection model to obtain a corresponding corner point coordinate detection model, obtain historical license plate images, and mark the corner point coordinates of the historical license plate images to obtain corresponding historical marked license plate images, and divide the historical marked license plate images according to preset division rules to obtain corresponding training sets and tests, and input the training sets into the corner point coordinate detection model for training.

[0086] In some embodiments, the target detection model is a Faster R-CNN model, and the processing device can fuse the Faster R-CNN model and the Landmark detection model to obtain a corresponding corner point coordinate detection model, obtain historical license plate images, and mark the corner point coordinates of the historical license plate images to obtain corresponding historical marked license plate images, and divide the historical marked license plate images according to preset division rules to obtain corresponding training sets and tests, and input the training sets to the corner point coordinate detection model for training.

[0087] In some implementations, after constructing and training the corner point coordinate detection model, the processing device may determine whether the corner point coordinate detection model has converged.

[0088] Specifically, after constructing and training the corner point coordinate detection model, the processing device can input a test set into the corner point coordinate detection model. The corner point coordinate detection model receives and responds to the test set, tests the test set, obtains the corresponding test results, and determines the test accuracy corresponding to the test results based on the test results and the test set, and determines whether the corner point coordinate detection model has converged based on the test accuracy.

[0089] When the test accuracy is greater than or equal to the preset accuracy threshold, it is determined that the corner point coordinate detection model has converged; when the test accuracy is less than the preset accuracy threshold, it is determined that the corner point coordinate detection model has not converged.

[0090] Step S230: When it is determined that the corner point coordinate detection model has converged, the license plate image is input into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates.

[0091] In this embodiment, when the processing device determines that the corner point coordinate detection model has converged, the license plate image can be input into the corner point coordinate detection model. The corner point coordinate detection model receives and responds to the license plate image, detects the license plate corner point coordinates corresponding to the license plate image, and outputs the license plate corner point coordinates to the processing device. The processing device receives the license plate corner point coordinates output by the corner point coordinate detection model, and realizes that when the corner point coordinate detection model is determined to converge according to the test set, the license plate corner point coordinates corresponding to the license plate image are detected, thereby ensuring the stability of the corner point coordinate detection model and improving the credibility of the license plate corner point coordinates.

[0092] Step S240: Determine the target license plate area corresponding to the license plate image according to the coordinates of the license plate corner points.

[0093] Step S250: performing optical character recognition on the target license plate area to obtain the corresponding target license plate number.

[0094] In this embodiment, step S240 and step S250 may refer to the contents of the corresponding steps in the aforementioned embodiment, which will not be described in detail here.

[0095] The solution provided in this embodiment obtains a license plate image, and builds and trains a corner point coordinate detection model based on a target detection model and a feature point detection model. When it is determined that the corner point coordinate detection model converges, the license plate image is input into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates. According to the license plate corner point coordinates, the target license plate area corresponding to the license plate image is determined, and optical character recognition is performed on the target license plate area to obtain the corresponding target license plate number. In the license plate recognition process, optical character recognition is performed on the target license plate area corresponding to the license plate image to obtain the corresponding target license plate number, thereby avoiding the recognition error of the license plate number caused by the recognition of the number in the non-license plate area, and improving the recognition accuracy of the license plate.

[0096] Furthermore, when the corner point coordinate detection model is determined to converge according to the test set, the license plate corner point coordinates corresponding to the license plate image are detected, which ensures the stability of the corner point coordinate detection model and improves the credibility of the license plate corner point coordinates.

[0097] See also Figure 5 , which shows a license plate recognition device 300 provided by an embodiment of the present application. The license plate recognition device 300 can be applied to Figure 1 The processing device 100 in the license plate recognition system shown in FIG. 1 is taken as an example to describe the processing device 100 in the following. Figure 5 The license plate recognition device 300 shown is described in detail. The license plate recognition device 300 may include an acquisition module 310 , an input module 320 , a region determination module 330 and a license plate region recognition module 340 .

[0098] The acquisition module 310 can be used to acquire a license plate image; the input module 320 can be used to input the license plate image into a corner point coordinate detection model to obtain corresponding license plate corner point coordinates, and the corner point coordinate detection model can include a target detection model and a feature point detection model; the area determination module 330 can be used to determine the target license plate area corresponding to the license plate image based on the license plate corner point coordinates; the license plate area recognition module 340 can be used to perform optical character recognition on the target license plate area to obtain the corresponding target license plate number.

[0099] In some embodiments, the target detection model may include a first branch and a second branch, the first branch may be used to detect the license plate type of the license plate image, and the second branch may be used to detect the license plate frame information in the license plate image.

[0100] In some implementations, the feature point detection model may include a third branch, and the third branch may be used to detect the coordinates of the corner points of the license plate in the license plate image.

[0101] In some implementations, the license plate region recognition module 340 may include an acquisition unit, a fusion unit, and a recognition unit.

[0102] The acquisition unit can be used to acquire multiple character frames corresponding to the target license plate area; the fusion unit can be used to fuse multiple character frames belonging to the same row to obtain the corresponding target text area; the recognition unit can be used to perform optical character recognition on the target text area to obtain the corresponding target license plate number.

[0103] In some implementations, the license plate recognition 300 may further include a format determination module, a correction module, and a text region recognition module.

[0104] The format determination module can be used to determine whether the format of the target license plate number is correct; the correction module can be used to correct the target text area when it is determined that the format of the target license plate number is incorrect, and obtain the corresponding corrected text area; the text area recognition module can be used to perform optical character recognition on the corrected text area to obtain the corresponding corrected license plate number.

[0105] In some embodiments, the correction module may include a first determining unit and a correction unit.

[0106] The first determination unit can be used to determine the license plate type corresponding to the target text area when it is determined that the format of the target license plate number is incorrect; the correction unit can be used to correct the target text area according to the text box size corresponding to the license plate type to obtain the corresponding corrected text area.

[0107] In some implementations, the first determining unit may include a calculating subunit and a determining subunit.

[0108] The calculation subunit can be used to calculate the target PHash value corresponding to the target text area when it is determined that the format of the target license plate number is incorrect; the determination subunit can be used to determine the target license plate type of the target license plate area based on the target PHash value.

[0109] In some implementations, the license plate recognition 300 may further include a building block.

[0110] The construction module can be used for inputting the license plate image into the corner point coordinate detection model through the input module 320. Before obtaining the corresponding license plate corner point coordinates, the corner point coordinate detection model is constructed and trained according to the target detection model and the feature point detection model.

[0111] In some implementations, the input module 320 may include an input unit.

[0112] The input unit can be used to input the license plate image into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates when it is determined that the corner point coordinate detection model has converged.

[0113] In some implementations, the region determination module 330 may include a second determination unit and a transformation processing unit.

[0114] The second determination unit can be used to determine the initial license plate area corresponding to the license plate image according to the coordinates of the license plate corner points; the transformation processing unit can be used to perform perspective transformation processing on the initial license plate area to obtain the corresponding target license plate area.

[0115] The solution provided in this embodiment obtains the license plate image and inputs the license plate image into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates, and determines the target license plate area corresponding to the license plate image based on the license plate corner point coordinates, and performs optical character recognition on the target license plate area to obtain the corresponding target license plate number. In the license plate recognition process, optical character recognition is performed on the target license plate area corresponding to the license plate image to obtain the corresponding target license plate number, thereby avoiding incorrect license plate number recognition caused by recognizing numbers in non-license plate areas and improving the recognition accuracy of license plate recognition.

[0116] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated one by one in the device embodiment.

[0117] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0118] See also Figure 6, which shows a functional block diagram of an electronic device 400 provided by an embodiment of the present application, and the electronic device 400 may include one or more of the following components: a memory 410, a processor 420, and one or more applications, wherein the one or more applications may be stored in the memory 410 and configured to be executed by the one or more processors 420, and the one or more applications are configured to execute the method described in the aforementioned method embodiment.

[0119] The memory 410 may include a random access memory (RAM) or a read-only memory (ROM). The memory 410 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 410 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as obtaining a license plate image, inputting a license plate image, obtaining license plate corner point coordinates, obtaining an angle coordinate detection model, determining a target license plate area, image recognition of a target license plate area, obtaining a target license plate number, detecting a license plate type, detecting license plate frame information, detecting license plate corner point coordinates, obtaining a character frame, fusing multiple character frames, obtaining a target text area, optical character recognition of a target text area, determining whether the format of the target license plate number is correct, determining whether the format of the target license plate number is incorrect, correcting the target text area, obtaining a corrected text area, optical character recognition of a corrected text area, obtaining a corrected license plate number, determining a license plate type, calculating a PHash value, constructing a corner point coordinate detection model, training a corner point coordinate detection model, determining an initial license plate area, perspective transformation processing, and obtaining a target license plate area, etc.), instructions for implementing the following various method embodiments, etc. The data storage area can also store data created by the electronic device 400 during use (such as license plate images, corner point coordinate detection models, license plate corner point coordinates, target detection models, feature point detection models, target license plate areas, target license plate numbers, first branches, second branches, license plate types, license plate frame information, third branches, character boxes, target text areas, license plate number formats, correction text areas, correction license plate numbers, license plate types, text box sizes, PHash values, corner point coordinate detection models, and initial license plate areas), etc.

[0120] The processor 420 may include one or more processing cores. The processor 420 uses various interfaces and lines to connect the various parts of the entire electronic device 400, and executes various functions and processes data of the electronic device 400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 410, and calling data stored in the memory 410. Optionally, the processor 420 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 420 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 420, but may be implemented separately through a communication chip.

[0121] Please refer to Figure 7 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 500 stores a program code 510, which can be called by a processor to execute the method described in the above method embodiment.

[0122] The computer-readable storage medium 500 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 510 can be compressed, for example, in an appropriate form.

[0123] Please refer to Figure 8, which shows a structural block diagram of a computer program product 600 provided in an embodiment of the present application. The computer program product 600 includes a computer program / instruction 610, and the computer program / instruction 610 is stored in a computer-readable storage medium of a computer device. When the computer program product 600 is run on a computer device, a processor of the computer device reads the computer program / instruction 610 from the computer-readable storage medium, and the processor executes the computer program / instruction 610, so that the computer device performs the method described in the above method embodiment.

[0124] The solution provided in this embodiment obtains the license plate image and inputs the license plate image into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates, and determines the target license plate area corresponding to the license plate image based on the license plate corner point coordinates, and performs optical character recognition on the target license plate area to obtain the corresponding target license plate number. In the license plate recognition process, optical character recognition is performed on the target license plate area corresponding to the license plate image to obtain the corresponding target license plate number, thereby avoiding incorrect license plate number recognition caused by recognizing numbers in non-license plate areas and improving the recognition accuracy of license plate recognition.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A license plate recognition method, characterized in that: include: Get the license plate image; Input the license plate image to a corner point coordinate detection model to obtain corresponding license plate corner point coordinates, wherein the corner point coordinate detection model includes a target detection model and a feature point detection model; Determining a target license plate area corresponding to the license plate image according to the license plate corner point coordinates; Optical character recognition is performed on the target license plate area to obtain a corresponding target license plate number.

2. The license plate recognition method according to claim 1, characterized in that: The target detection model includes a first branch and a second branch, the first branch is used to detect the license plate type of the license plate image, and the second branch is used to detect the license plate frame information in the license plate image.

3. The license plate recognition method according to claim 1, characterized in that: The feature point detection model includes a third branch, and the third branch is used to detect the coordinates of the license plate corner points in the license plate image.

4. The license plate recognition method according to claim 1, characterized in that: The performing optical character recognition on the target license plate area to obtain the corresponding target license plate number includes: Obtain multiple character frames corresponding to the target license plate area; Merge multiple character boxes belonging to the same line to obtain the corresponding target text area; Optical character recognition is performed on the target text area to obtain a corresponding target license plate number.

5. The license plate recognition method according to claim 4, characterized in that: Also includes: Determining whether the format of the target license plate number is correct; When it is determined that the format of the target license plate number is wrong, correcting the target text area to obtain a corresponding corrected text area; Optical character recognition is performed on the correction text area to obtain a corresponding correction license plate number.

6. The license plate recognition method according to claim 5, characterized in that: When it is determined that the format of the target license plate number is wrong, correcting the target text area to obtain a corresponding corrected text area includes: When it is determined that the format of the target license plate number is incorrect, determining the license plate type corresponding to the target text area; The target text area is corrected according to the text box size corresponding to the license plate type to obtain a corresponding corrected text area.

7. The license plate recognition method according to claim 6, characterized in that: When it is determined that the format of the target license plate number is incorrect, determining the type of license plate corresponding to the target text area includes: When it is determined that the format of the target license plate number is wrong, calculating the target PHash value corresponding to the target text area; Based on the target PHash value, a target license plate type of the target license plate area is determined.

8. The license plate recognition method according to claim 1, characterized in that: Before inputting the license plate image into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates, the license plate recognition method further includes: Based on the target detection model and feature point detection model, build and train the corner point coordinate detection model; The step of inputting the license plate image into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates includes: When it is determined that the corner point coordinate detection model has converged, the license plate image is input into the corner point coordinate detection model to obtain the corresponding license plate corner point coordinates.

9. The license plate recognition method according to any one of claims 1 to 8, characterized in that: The step of determining the target license plate area corresponding to the license plate image according to the license plate corner point coordinates includes: Determining an initial license plate area corresponding to the license plate image according to the license plate corner point coordinates; The initial license plate area is subjected to perspective transformation processing to obtain a corresponding target license plate area.

10. A license plate recognition device, characterized in that: include: An acquisition module is used to acquire a license plate image; An input module, used to input the license plate image into a corner point coordinate detection model to obtain corresponding license plate corner point coordinates, wherein the corner point coordinate detection model includes a target detection model and a feature point detection model; An area determination module, used to determine a target license plate area corresponding to the license plate image according to the coordinates of the license plate corner points; The license plate area recognition module is used to perform optical character recognition on the target license plate area to obtain the corresponding target license plate number.

11. An electronic device, characterized in that: include: Memory; One or more processors coupled to the memory; One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by one or more processors, and the one or more applications are configured to execute the license plate recognition method according to any one of claims 1 to 7.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and the program code can be called by a processor to execute the license plate recognition method according to any one of claims 1 to 7.

Images