License plate information identification method and device, equipment, storage medium and program product

Abstract

The invention relates to a license plate information identification method, device and equipment, a storage medium and a program product, the method is applied to a camera with a built-in NPU, and the method comprises the following steps: obtaining a vehicle image collected in the driving process of a vehicle and a zoomed low-resolution image; positioning the vehicle in the low-resolution image by using a vehicle positioning thread to obtain the position of the vehicle; performing license plate positioning on the vehicle image based on the vehicle position by using a license plate positioning thread to obtain a license plate position; positioning a license plate area in the vehicle image based on the license plate position by using a license plate character recognition thread, and performing character recognition on the license plate area to obtain a character recognition result; performing color identification on the license plate area by using a license plate color identification thread to obtain a color identification result; and determining license plate information of the vehicle according to the character recognition result and the color recognition result. The method can effectively improve the accuracy of license plate information identification.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a license plate information recognition method, apparatus, device, storage medium and program product. Background Art

[0002] As a vehicle's unique identifier, the license plate has become a crucial solution for effective traffic management. Accurate license plate recognition can effectively address issues such as highway toll collection, vehicle safety management, and the handling of traffic violations. However, despite continuous improvements in license plate recognition technology, its efficiency and accuracy remain low, requiring urgent improvement. Summary of the Invention

[0003] Based on this, it is necessary to provide a license plate information recognition method, device, equipment, storage medium and program product to address the above technical problems, which can effectively improve the accuracy of license plate information recognition.

[0004] In a first aspect, the present application provides a license plate information recognition method, which is applied to a camera with a built-in NPU, and the method includes:

[0005] Acquire a vehicle image and a scaled low-resolution image captured while the vehicle is traveling; the vehicle image is captured with a fill light on, the fill light being mounted on a traffic gantry and positioned at an angle with the horizontal plane within a target range to prevent glare to the driver;

[0006] Using a vehicle positioning thread to locate the vehicle in the low-resolution image to obtain a vehicle position;

[0007] Using a license plate positioning thread to perform license plate positioning on the vehicle image based on the vehicle position to obtain a license plate position;

[0008] Using a license plate character recognition thread to locate a license plate area in the vehicle image based on the license plate position, and performing character recognition on the license plate area to obtain a character recognition result;

[0009] Performing color recognition on the license plate area using a license plate color recognition thread to obtain a color recognition result;

[0010] The license plate information of the vehicle is determined according to the character recognition result and the color recognition result.

[0011] In one embodiment, acquiring the vehicle image and the scaled low-resolution image collected while the vehicle is traveling includes:

[0012] Acquire a road environment image, and perform light detection based on the road environment image to obtain brightness;

[0013] When the brightness is lower than a preset brightness, during the process of capturing an image of a moving vehicle, the fill light is controlled to turn on to provide fill light, thereby obtaining an image of the vehicle;

[0014] The copy of the vehicle image is scaled to obtain a low-resolution image.

[0015] In one embodiment, the method further comprises:

[0016] converting the vehicle position into a vehicle position in the vehicle image using an image preprocessing thread, cropping the vehicle image based on the vehicle position in the vehicle image, and scaling and normalizing the cropped vehicle image to obtain a preprocessed image;

[0017] The license plate positioning thread is used to perform license plate positioning on the vehicle image based on the vehicle position to obtain the license plate position, which includes:

[0018] The license plate positioning thread is used to perform license plate positioning on the pre-processed image to obtain the license plate position.

[0019] In one embodiment, the locating the vehicle in the low-resolution image using a vehicle positioning thread to obtain the vehicle position includes:

[0020] When there are multiple vehicles in the low-resolution image, positioning the multiple vehicles in the low-resolution image using a vehicle positioning thread to obtain positions of the vehicles to be selected;

[0021] From the candidate vehicle positions, a vehicle position that is located in the lane corresponding to the camera and at the bottom of the low-resolution image is selected.

[0022] In one embodiment, before determining the license plate information of the vehicle based on the character recognition result and the color recognition result, the method further includes:

[0023] If there are multiple vehicle images obtained, then selecting the character with the highest frequency of occurrence and the color with the highest frequency of occurrence at each character position from the character recognition results and the color recognition results of the multiple vehicle images and combining them to obtain a first combination result;

[0024] The determining of the vehicle license plate information according to the character recognition result and the color recognition result includes:

[0025] The license plate information of the vehicle is determined according to the first combination result.

[0026] In one embodiment, the method further comprises:

[0027] If the character with the highest frequency of occurrence does not meet the specification requirements, selecting the character with the second highest frequency of occurrence and the color with the highest frequency of occurrence at each character position from the character recognition results and the color recognition results of the plurality of vehicle images, and combining them to obtain a second combination result;

[0028] The determining of the vehicle license plate information according to the character recognition result and the color recognition result includes:

[0029] The license plate information of the vehicle is determined according to the second combination result.

[0030] In a second aspect, the present application further provides a license plate information recognition device, which is applied to a camera with a built-in NPU, and includes:

[0031] an acquisition module, configured to acquire a vehicle image and a scaled low-resolution image captured while the vehicle is in motion; the vehicle image is captured with a fill light on, the fill light being mounted on a traffic gantry and positioned at an angle with the horizontal plane to prevent glare to the driver;

[0032] A vehicle positioning module, configured to locate the vehicle in the low-resolution image using a vehicle positioning thread to obtain a vehicle position;

[0033] A license plate positioning module, configured to perform license plate positioning on the vehicle image based on the vehicle position using a license plate positioning thread to obtain a license plate position;

[0034] a character recognition module, configured to locate the license plate region in the vehicle image based on the license plate position using a license plate character recognition thread, and perform character recognition on the license plate region to obtain a character recognition result;

[0035] A color recognition module is used to perform color recognition on the license plate area using a license plate color recognition thread to obtain a color recognition result;

[0036] A determination module is used to determine the license plate information of the vehicle according to the character recognition result and the color recognition result.

[0037] In one embodiment, the acquisition module is further used to acquire a road environment image and perform light detection based on the road environment image to obtain brightness; when the brightness is lower than a preset brightness, during the image acquisition of a moving vehicle, the fill light is controlled to turn on for fill light to obtain a vehicle image; and a copy of the vehicle image is scaled to obtain a low-resolution image.

[0038] In one embodiment, the apparatus further comprises:

[0039] a processing module, configured to convert the vehicle position into a vehicle position in the vehicle image using an image preprocessing thread, crop the vehicle image based on the vehicle position in the vehicle image, and scale and normalize the cropped vehicle image to obtain a preprocessed image;

[0040] The license plate positioning module is further used to use the license plate positioning thread to perform license plate positioning on the pre-processed image to obtain the license plate position.

[0041] In one embodiment, the vehicle positioning module is further used to, when there are multiple vehicles in the low-resolution image, use the vehicle positioning thread to locate the multiple vehicles in the low-resolution image to obtain the positions of the selected vehicles; and select the vehicle position of the vehicle located in the lane corresponding to the camera and at the bottom of the low-resolution image from the selected vehicle positions.

[0042] In one embodiment, the apparatus further comprises:

[0043] a selection module configured to select, from the character recognition results and the color recognition results of the plurality of vehicle images, a character with the highest frequency of occurrence and a color with the highest frequency of occurrence at each character position, and combine them to obtain a first combination result if a plurality of vehicle images are obtained;

[0044] The determination module is further configured to determine the license plate information of the vehicle according to the first combination result.

[0045] In one embodiment, the selection module is further configured to select, from the character recognition results and color recognition results of the plurality of vehicle images, the character with the second highest frequency of occurrence and the color with the highest frequency of occurrence at each character position, and combine them to obtain a second combination result if the character with the highest frequency of occurrence does not meet specification requirements;

[0046] The determining module is further configured to determine the license plate information of the vehicle according to the second combination result.

[0047] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the license plate information recognition method when executing the computer program.

[0048] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the license plate information recognition method are implemented.

[0049] In a fifth aspect, the present application also provides a computer program product, which includes a computer program that implements the steps of the license plate information recognition method when executed by a processor.

[0050] The above-mentioned license plate information recognition method, device, computer equipment, storage medium and computer program product obtain a vehicle image and a scaled low-resolution image collected during the vehicle's driving process, and can quickly locate the vehicle using the low-resolution image to obtain the vehicle position; based on the vehicle position, the license plate is located on the vehicle image to obtain the license plate position; based on the license plate position, the license plate area in the vehicle image is located, and character recognition is performed on the license plate area to obtain a character recognition result; color recognition is performed on the license plate area to obtain a color recognition result; the vehicle's license plate information is determined based on the character recognition result and the color recognition result, so that the vehicle can be accurately identified. The license plate information of the vehicle can be quickly recognized. In addition, when performing vehicle positioning, license plate positioning, character recognition and color recognition, corresponding threads are used to execute them respectively, which can help speed up the recognition of license plate information. Moreover, the license plate information recognition method is used for cameras with built-in NPU, and NPU can provide high computing power support for cameras. Therefore, when facing high-speed vehicles, the camera can quickly recognize the license plate information of the vehicle. Finally, the fill light is installed on the traffic gantry, and the angle with the horizontal plane is within the target range to avoid dazzling the driver. Therefore, turning on the fill light when collecting vehicle images will not cause traffic safety risks to the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A diagram showing an application environment of a license plate information recognition method in one embodiment;

[0052] Figure 2 1 is a flow chart of a license plate information recognition method according to an embodiment;

[0053] Figure 3 This is a product schematic diagram of a heating module in one embodiment;

[0054] Figure 4 A schematic diagram of a camera fastened to a heating module in one embodiment;

[0055] Figure 5 A schematic diagram of a vehicle information identification page in one embodiment;

[0056] Figure 6 is a flow chart of a license plate information recognition method according to another embodiment;

[0057] Figure 7 is a structural block diagram of a license plate information recognition device in one embodiment;

[0058] Figure 8 is a structural block diagram of a license plate information recognition device in another embodiment;

[0059] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] The license plate information recognition method provided in the embodiment of the present application can be applied to Figure 1 In the illustrated application environment, camera 102 with a built-in NPU executes a license plate recognition method to obtain the license plate information of vehicle 104. During the license plate recognition process, fill light 106 can be activated based on light levels or when night is approaching, thereby capturing a sufficiently bright vehicle image. Furthermore, fill light 106 and camera 102 can be mounted on a traffic gantry at an angle within a target range relative to the horizontal. Within this target range, fill light 106 can prevent glare to the driver, such as at an angle of 40 to 45 degrees relative to the horizontal.

[0062] The vehicle 104 may be any type of vehicle, such as a family car, a truck, a professional vehicle, or any other type of vehicle (eg, a passenger car).

[0063] In one embodiment, Figure 2 As shown, a license plate information recognition method is provided, which is applied to Figure 1 The camera 102 with a built-in NPU includes the following steps:

[0064] S202 , obtaining a vehicle image and a scaled low-resolution image collected during the vehicle's driving process.

[0065] The vehicle image may be captured with a fill light on. The fill light is mounted on a traffic gantry and positioned at an angle to the horizontal plane within a target range to prevent glare on the driver. This fill light is used to illuminate the vehicle in dimly lit environments (i.e., low-light environments) to ensure the captured vehicle image is sufficiently bright. Furthermore, the camera and fill light may be mounted on the traffic gantry together, with their angle to the horizontal plane also within the target range.

[0066] The camera may include a lens, a power module, a built-in high-computing NPU (Neural Processing Unit), and other components. The NPU can be a standalone NPU or a NPU within an AI processor. The AI processor also includes a CPU and a GPU. This means the camera's AI processor utilizes a CPU+NPU+GPU SoC architecture. The NPU can have at least 2T of computing power. If the NPU is a standalone NPU, the other components may be the CPU and GPU.

[0067] The camera lens can be a zoom lens installed at the front end of the camera with a focal length of 5mm-16mm, usually set to 8mm, used to capture vehicle images and identify vehicle license plate information; the power module is used to power the camera and heating module; the CPU is used for temperature monitoring and logic control.

[0068] For example, a camera and fill light are mounted on a traffic gantry using a mounting bracket at a 40- or 45-degree angle. The camera and vehicle are positioned relative to each other, and the fill light is positioned 0.3-0.5 meters from the camera. The camera covers a single lane. The near-end of the captured vehicle image is 3 meters from the traffic gantry, and the far-end is 10 meters from the gantry. Therefore, the vehicle is 7 meters within the gantry's detection area. The vehicle's following blind spot can be calculated using the formula: A = B / tan(θ). Here, A is the vehicle's following blind spot, B is the height of the preceding vehicle, and θ is the camera's downward angle. Based on this formula, for the same vehicle, the larger the camera's downward angle θ, the smaller the vehicle's following blind spot A, and the greater the probability that the camera will detect the following vehicle in the following scenario.

[0069] The camera in this application has a built-in high-performance NPU and increases its frame rate to 50 frames per second (20 milliseconds per frame). The number of frames recognized can be calculated using the formula N = S / V / 0.02, where N is the number of frames recognized, S is the detection area distance (in meters), V is the vehicle's speed (in meters per second), and 0.02 is the camera's acquisition rate (0.02 seconds per frame). For example, if a vehicle passes a traffic gantry at 150 km / h, the camera can capture 8.5 frames of data according to the formula.

[0070] In this application, the housing of the camera can be tightly fastened to a heating module, such as Figure 3 As shown in Figures (a) and (b), the whole behind the heating module can be called the temperature drift compensation device of the camera lens, as shown in Figures (a) and (b). Figure 4As shown in Figures (a) and (b); in addition, a thermally conductive silicone sheet is provided between the heating module and the camera housing. On the one hand, this can prevent the heat generated by the heating module from directly acting on the camera housing and causing local overheating. On the other hand, it can evenly transfer the heat generated by the heating module to the camera housing, and then transfer the heat to the camera's CPU and / or NPU through the housing.

[0071] The camera is connected to the heating module via an RS485 serial control line to transmit control commands (such as heating instructions and heating stop instructions) to the heating module, thereby controlling whether the heating module performs heating work.

[0072] In one embodiment, the camera captures vehicle images, and when each frame of the vehicle image is captured, the vehicle image of the frame can be scaled to obtain a corresponding low-resolution image, thereby performing two-way output to obtain a vehicle image with original resolution and a corresponding low-resolution image.

[0073] For example, each frame of the captured vehicle image is output in two ways through the ISP (Image Signal Processor) hardware. One output is a vehicle image with the original resolution of 4096*2208, and the other is a low-resolution image that is pre-processed and scaled to 320*320 using its hardware interpolation unit and padding logic.

[0074] In one embodiment, a camera can acquire a road environment image and perform light detection based on the road environment image to obtain brightness; when the brightness is lower than a preset brightness, during the image acquisition of a moving vehicle, the fill light is controlled to turn on for fill light to obtain a vehicle image; a copy of the vehicle image is scaled to obtain a low-resolution image.

[0075] In addition, in addition to turning on the fill light by detecting light, the fill light can also be turned on according to time, such as turning on the fill light near night, so that a sufficiently bright vehicle image can be obtained when collecting vehicle images.

[0076] After obtaining the vehicle image and the corresponding low-resolution image, the camera can perform vehicle location, license plate location, character recognition, and license plate color recognition through cross-threaded parallel processing to speed up processing. For details, see S204 to S210 of this application. In addition, before performing license plate location, the vehicle image can also be pre-processed, such as cropping, scaling, and normalization, to further speed up license plate location.

[0077] S204: Utilize the vehicle positioning thread to locate the vehicle in the low-resolution image to obtain the vehicle position.

[0078] The vehicle positioning thread is mainly responsible for positioning the vehicle, that is, the thread used for vehicle positioning. The vehicle position can be the position coordinates of the vehicle.

[0079] In one embodiment, when there are multiple vehicles in the low-resolution image, the camera can use the vehicle positioning thread to locate multiple vehicles in the low-resolution image to obtain the positions of selected vehicles; from the candidate vehicle positions corresponding to the multiple vehicles, the vehicle position of the vehicle located in the lane corresponding to the camera and at the bottom of the low-resolution image is selected.

[0080] Among them, the vehicle position to be selected can be the vehicle position of each vehicle in the low-resolution image. During the actual license plate information recognition, it is necessary to select one of the vehicles in the current lane for recognition. Therefore, the vehicle position of the vehicle in the lane corresponding to the camera and at the bottom of the low-resolution image can be selected from the vehicle position to be selected.

[0081] For example, when there are multiple vehicles in the scene at the same time, the vehicle positioning thread will locate the vehicle positions of multiple vehicles. The vehicle positioning thread selects the vehicle position that is within the lane line and at the bottom of the screen.

[0082] In one embodiment, the camera can use the YOLOv5s model in the vehicle localization thread to locate each vehicle in the scaled low-resolution image to obtain the position of the candidate vehicle. For example, the model size is modified to 320*320, and the modified YOLOv5s model is used to locate the vehicle in the scaled 320*320 resolution image, outputting the upper left coordinates (X1, Y1) and lower right coordinates (X2, Y2) of each vehicle in the image.

[0083] In another embodiment, the camera can traverse the low-resolution image with sliding windows at different zoom ratios (e.g., 0.5-2.0) in the vehicle positioning thread, and calculate the similarity between the features in each sliding window and the features of the template, such as the similarity with respect to the number of feature point matches or the HOG cosine distance; set a similarity threshold (e.g., matching points > 15), filter out potential vehicle areas, and then merge overlapping candidate boxes through non-maximum suppression (NMS), retaining the area with the highest confidence as the candidate area; use an edge detection operator (e.g., Canny operator) in the candidate area to extract the vehicle outline, so as to obtain the bounding box coordinates (e.g., center point) and width and height of the vehicle.

[0084] In one embodiment, after obtaining the vehicle position, the camera stores the vehicle image and the positioning position in an image pre-processing queue or a license plate positioning queue.

[0085] S206 , using the license plate positioning thread to perform license plate positioning on the vehicle image based on the vehicle position to obtain the license plate position.

[0086] The license plate position may be the position coordinates of the license plate in the vehicle.

[0087] In one embodiment, the server utilizes the license plate positioning thread to obtain the vehicle image and positioning position from the license plate positioning queue, and performs license plate positioning on the vehicle image based on the vehicle position to obtain the license plate position.

[0088] In another embodiment, the camera uses an image preprocessing thread to convert the vehicle position into the position of the vehicle in the vehicle image, and crops the vehicle image based on the position of the vehicle in the vehicle image, and scales and normalizes the cropped vehicle image to obtain a preprocessed image; then, the license plate positioning thread is used to locate the license plate on the preprocessed image to obtain the license plate position.

[0089] The camera may first utilize an image preprocessing thread to obtain a vehicle image and a positioning position from an image preprocessing queue, and then convert the vehicle position into a position of the vehicle in the vehicle image.

[0090] In one embodiment, after obtaining the preprocessed image, the camera can use the optimized detection model in the license plate positioning thread (such as the optimized YOLOv5s model, with the model size modified to 256*256) to perform license plate positioning to obtain the license plate position.

[0091] For example, the image preprocessing thread converts the vehicle position to its original resolution image position, such as the upper left crop coordinates (A1, B1) and the lower right crop coordinates (A2, B2), where A1 = (X2-X1) / 2 - 256, A2 = (X2-X1) / 2 + 256, B1 = Y2+512, and B2 = Y2+512. The original resolution vehicle image is then cropped to a 512*512 image based on these converted coordinates. Since the minimum number of pixels for a license plate in the original vehicle image is 80, and the minimum number of pixels required for a clearly discernible license plate is 30 (empirically), the ISP hardware interpolation unit and padding logic are used to preprocess the cropped image to 256*256 (the minimum number of pixels required for a resized image is 40). The pixel values are then normalized to produce the preprocessed image. Finally, the optimized YOLOv5s model is used to locate the license plate and determine its position.

[0092] In one embodiment, after obtaining the license plate position, the camera may store the vehicle image and the license plate position in a character recognition queue.

[0093] S208 , using the license plate character recognition thread to locate the license plate area in the vehicle image based on the license plate position, and performing character recognition on the license plate area to obtain a character recognition result.

[0094] The character recognition result may be a recognized license plate number.

[0095] In one embodiment, the camera can use the character recognition thread to obtain the vehicle image and license plate position from the character recognition queue, perform character segmentation on the license plate area at the license plate position in the vehicle image, and then use CRNN (convolutional recurrent network) or Vision Transformer to perform sequence recognition to obtain the character recognition result.

[0096] For example, the camera can use the license plate recognition thread to obtain the vehicle image and license plate position from the character recognition queue, extract the license plate area at the license plate position in the vehicle image, and obtain the license plate image; then use CRNN or VisionTransformer for sequence recognition to obtain the character recognition result.

[0097] S210, performing color recognition on the license plate area using the license plate color recognition thread to obtain a color recognition result.

[0098] The color recognition result may be the color of the recognized license plate.

[0099] In one embodiment, the camera can use the color recognition thread to obtain the vehicle image and license plate position from the character recognition queue, extract the license plate area at the license plate position in the vehicle image, and obtain the license plate image; extract the HSV color space histogram from the license plate image, and obtain the license plate color through the color threshold segmentation method.

[0100] It should be noted that for each vehicle, the camera will capture multiple frames of vehicle images, and the above steps S202 to S210 are used to obtain the character recognition result and color recognition result of each frame of the vehicle image.

[0101] S212: Determine the vehicle license plate information based on the character recognition result and the color recognition result.

[0102] The license plate information may include the license plate number and the license plate type, which may include blue, green, and yellow plates.

[0103] In one embodiment, if the number of vehicle images obtained is multiple, the camera selects the character with the highest frequency of occurrence and the color with the highest frequency of occurrence at each character position from the character recognition results and color recognition results of the multiple vehicle images, and combines them to obtain a first combination result; and determines the vehicle license plate information based on the first combination result.

[0104] In one embodiment, if the character with the highest frequency of occurrence does not meet the specification requirements, the character with the second highest frequency of occurrence and the color with the highest frequency of occurrence at each character position are selected from the character recognition results and color recognition results of multiple vehicle images and combined to obtain a second combination result; the vehicle license plate information is determined based on the second combination result.

[0105] Specifically, the camera's license plate voting thread votes on character recognition and color recognition results from multiple consecutive frames (default 5 frames, configurable). The voting method selects the most frequently occurring character and color at each character position, then combines them to obtain the initial license plate information. Finally, according to license plate standards, illegal results are filtered out. For example, if a character at a certain position does not meet the standards, the next most frequently occurring character is selected to obtain the vehicle's license plate information. This continuous multi-frame license plate recognition and license plate voting mechanism improves the reliability of license plate recognition.

[0106] For example, the recognition results of 5 consecutive frames, all the recognition results of each character are: the first position (Shanghai is 4 times, black is 1 time), the second position (F is 3 times, E is 2 times), the third position (Q is 5 times, P is 0 times), the fourth position (5 is 4 times, 8 is 1 time), and so on, until the recognition results of the 5th to 7th positions are obtained, and the color recognition results are (yellow is 4 times, blue is 1 time). According to the voting mechanism, the license plate information of the vehicle can be obtained as Huang Hu FQ5XXX. Figure 5 shown.

[0107] In the above embodiment, a vehicle image and a scaled low-resolution image collected during the driving process of the vehicle are obtained. The low-resolution image can be used to quickly locate the vehicle and obtain the vehicle position; the license plate is located on the vehicle image based on the vehicle position to obtain the license plate position; the license plate area in the vehicle image is located based on the license plate position, and character recognition is performed on the license plate area to obtain a character recognition result; color recognition is performed on the license plate area to obtain a color recognition result; the vehicle's license plate information is determined based on the character recognition result and the color recognition result, so that the vehicle's license plate information can be accurately identified; in addition, Vehicle positioning, license plate positioning, character recognition, and color recognition are each executed using a corresponding thread, which can help speed up the recognition of license plate information. Moreover, the license plate information recognition method is used for cameras with built-in NPUs, and the NPU can provide high computing power support for the camera. Therefore, when facing high-speed vehicles, the camera can quickly identify the vehicle's license plate information. Finally, the fill light is installed on the traffic gantry, and the angle with the horizontal plane is within the target range to avoid dazzling the driver. Therefore, turning on the fill light when collecting vehicle images will not pose a safety risk to the driver's traffic.

[0108] In one embodiment, during the process of acquiring images and recognizing license plate information, the camera may also obtain the temperature of the camera's first AI processor and the ambient temperature; select an AI processor heating threshold that matches the ambient temperature, and compare the first AI processor temperature with the AI processor heating threshold; if the first AI processor temperature is less than the AI processor heating threshold, send a heating instruction to a heating module fastened to the camera housing; a thermally conductive silicone sheet is provided between the heating module and the camera housing; and control at least one heater of the heating module to perform heating through the heating instruction, so that during the heating process, the heating module transfers the heat generated by the heating to the camera's AI processor through the thermally conductive silicone sheet, and during the AI processor heating process, transfers the heat of the AI processor to the camera's lens through the camera's thermal conductive device, so that the lens can maintain a constant temperature (e.g., 15 to 20 degrees), thereby effectively ensuring the image effect of the vehicle.

[0109] In one embodiment, the camera can send a heating instruction to the main control unit in the heating module tightly attached to the outer shell of the camera; controlling at least one heater of the heating module to perform heating work through the heating instruction includes: controlling the main control unit of the heating module to schedule at least one heater among multiple heaters to perform heating work through the heating instruction.

[0110] In one embodiment, the camera can obtain the test environment temperature within different temperature ranges and the AI processor usage parameters of the camera; based on each test environment temperature, the AI processor usage parameters and the temperature correction coefficient, the AI processor heating threshold at each test environment temperature is calculated; the AI processor heating threshold at each test environment temperature is configured as a parameter in the camera; selecting the AI processor heating threshold that matches the ambient temperature includes: selecting the AI processor heating threshold that matches the ambient temperature from the parameters configured in the camera.

[0111] In one embodiment, if the camera detects a change in ambient temperature, it can determine the temperature increase value after the ambient temperature changes; compare the temperature increase value with the temperature adjustment threshold; if the temperature increase value is greater than or equal to the temperature adjustment threshold, control the main control unit to turn off some heaters in working state, so that the heat obtained by heating offsets the heat loss of the lens, so that the lens is in a constant temperature state.

[0112] In the above embodiment, the first AI processor temperature and the ambient temperature of the camera are obtained; an AI processor heating threshold that matches the ambient temperature is selected, and the first AI processor temperature is compared with the AI processor heating threshold; if the first AI processor temperature is less than the AI processor heating threshold, a heating instruction is sent to a heating module tightly attached to the camera housing; a thermally conductive silicone sheet is provided between the heating module and the camera housing; at least one heater of the heating module is controlled by the heating instruction to perform heating, so that during the heating process, the heating module transfers the heat generated by the heating to the AI processor of the camera through the thermally conductive silicone sheet, and during the heating process of the AI processor, the heat of the AI processor is transferred to the camera lens through the thermal conductive device of the camera, so that only a heating module needs to be installed on the camera housing, without changing the structure of the lens and without increasing the difficulty of focusing the lens. The lens can be heated in a low temperature environment to ensure the temperature of the lens, thereby effectively ensuring the image effect without affecting the accuracy of license plate recognition.

[0113] As an example, combining Figure 6 The solution of this application is described as a whole, as follows:

[0114] The camera captures vehicle images at a rate of 50 frames per second, outputting both the original-resolution vehicle image and the corresponding low-resolution image in a dual-channel output format. The low-resolution image is used to locate the vehicle and determine whether it has been located. If not, the next frame is processed. If so, the number of vehicle coordinates is determined to be greater than one. If so, the coordinates at the bottom of the vehicle image, within the lane line, are selected and the original-resolution vehicle image is cropped using the selected coordinates to obtain a cropped vehicle image. If not, the original-resolution vehicle image is directly cropped using the selected coordinates to obtain a cropped vehicle image. The cropped vehicle image is scaled to further reduce its size (i.e., to 256*256 pixels). The reduced image is then used to locate the license plate and output the license plate location. Using this license plate location, character and color recognition are performed on the license plate area in the vehicle image, resulting in character and color recognition results. The character and color recognition results are voted to determine the highest-voted result. The system then determines whether it meets license plate specifications and, if so, outputs the license plate information.

[0115] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0116] Based on the same inventive concept, embodiments of the present application also provide a license plate information recognition device for implementing the aforementioned license plate information recognition method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more license plate information recognition device embodiments provided below can be found in the aforementioned limitations of the license plate information recognition method and will not be further elaborated here.

[0117] In one embodiment, Figure 7 As shown, a license plate information recognition device is provided, which is applied to a camera with a built-in NPU. The device includes: an acquisition module 702, a vehicle positioning module 704, a license plate positioning module 706, a character recognition module 708, a color recognition module 710 and a determination module 712, wherein:

[0118] The acquisition module 702 is configured to acquire vehicle images and scaled low-resolution images captured while the vehicle is in motion. The vehicle images are captured with a fill light on, which is mounted on a traffic gantry and positioned at an angle to the horizontal plane that prevents glare to the driver.

[0119] The vehicle positioning module 704 is used to locate the vehicle in the low-resolution image using the vehicle positioning thread to obtain the vehicle position;

[0120] The license plate positioning module 706 is used to perform license plate positioning on the vehicle image based on the vehicle position using the license plate positioning thread to obtain the license plate position;

[0121] The character recognition module 708 is used to locate the license plate area in the vehicle image based on the license plate position using the license plate character recognition thread, and perform character recognition on the license plate area to obtain a character recognition result;

[0122] The color recognition module 710 is used to perform color recognition on the license plate area using the license plate color recognition thread to obtain a color recognition result;

[0123] The determination module 712 is used to determine the license plate information of the vehicle according to the character recognition result and the color recognition result.

[0124] In one embodiment, the acquisition module 702 is further used to acquire a road environment image and perform light detection based on the road environment image to obtain brightness; when the brightness is lower than a preset brightness, during the image acquisition of a moving vehicle, the fill light is controlled to turn on for fill light to obtain a vehicle image; and a copy of the vehicle image is scaled to obtain a low-resolution image.

[0125] In one embodiment, Figure 8 As shown, the device also includes:

[0126] a processing module 714 configured to convert the vehicle position into the vehicle position in the vehicle image using an image preprocessing thread, crop the vehicle image based on the vehicle position in the vehicle image, and scale and normalize the cropped vehicle image to obtain a preprocessed image;

[0127] The license plate positioning module 706 is further configured to use the license plate positioning thread to perform license plate positioning on the pre-processed image to obtain the license plate position.

[0128] In one embodiment, the vehicle positioning module 704 is further used to, when there are multiple vehicles in the low-resolution image, use the vehicle positioning thread to locate the multiple vehicles in the low-resolution image to obtain the positions of the selected vehicles; and select the vehicle position of the vehicle located in the lane corresponding to the camera and at the bottom of the low-resolution image from the candidate vehicle positions.

[0129] In one embodiment, Figure 8 As shown, the device also includes:

[0130] A selection module 716 is configured to select, from the character recognition results and color recognition results of the multiple vehicle images, the character with the highest frequency of occurrence and the color with the highest frequency of occurrence at each character position, and combine them to obtain a first combination result.

[0131] The determination module 712 is further configured to determine the license plate information of the vehicle according to the first combination result.

[0132] In one embodiment, the selection module 716 is further configured to select, from the character recognition results and color recognition results of the plurality of vehicle images, the character with the second highest frequency of occurrence and the color with the highest frequency of occurrence at each character position, and combine them to obtain a second combination result if the character with the highest frequency of occurrence does not meet the specification requirements;

[0133] The determination module 712 is further configured to determine the license plate information of the vehicle according to the second combination result.

[0134] In the above embodiment, a vehicle image and a scaled low-resolution image collected during the driving process of the vehicle are obtained. The low-resolution image can be used to quickly locate the vehicle and obtain the vehicle position; the license plate is located on the vehicle image based on the vehicle position to obtain the license plate position; the license plate area in the vehicle image is located based on the license plate position, and character recognition is performed on the license plate area to obtain a character recognition result; color recognition is performed on the license plate area to obtain a color recognition result; the vehicle's license plate information is determined based on the character recognition result and the color recognition result, so that the vehicle's license plate information can be accurately identified; in addition, Vehicle positioning, license plate positioning, character recognition, and color recognition are each executed using a corresponding thread, which can help speed up the recognition of license plate information. Moreover, the license plate information recognition method is used for cameras with built-in NPUs, and the NPU can provide high computing power support for the camera. Therefore, when facing high-speed vehicles, the camera can quickly identify the vehicle's license plate information. Finally, the fill light is installed on the traffic gantry, and the angle with the horizontal plane is within the target range to avoid dazzling the driver. Therefore, turning on the fill light when collecting vehicle images will not pose a safety risk to the driver's traffic.

[0135] Each module in the license plate recognition device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0136] In one embodiment, a computer device is provided. The computer device may be a camera, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store image data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a license plate information recognition method is implemented.

[0137] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0138] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above-mentioned license plate information recognition method when executing the computer program.

[0139] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned license plate information recognition method are implemented.

[0140] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above-mentioned license plate information recognition method when executed by a processor.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0142] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processors (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), and other data processing logic devices.

[0143] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A license plate information recognition method, characterized in that: The method is applied to a camera with a built-in NPU, and the method includes: Acquire a vehicle image and a scaled low-resolution image captured while the vehicle is traveling; the vehicle image is captured with a fill light on, the fill light being mounted on a traffic gantry and positioned at an angle with the horizontal plane within a target range to prevent glare to the driver; Using a vehicle positioning thread to locate the vehicle in the low-resolution image to obtain a vehicle position; Using a license plate positioning thread to perform license plate positioning on the vehicle image based on the vehicle position to obtain a license plate position; Using a license plate character recognition thread to locate a license plate area in the vehicle image based on the license plate position, and performing character recognition on the license plate area to obtain a character recognition result; Performing color recognition on the license plate area using a license plate color recognition thread to obtain a color recognition result; The license plate information of the vehicle is determined according to the character recognition result and the color recognition result.

2. The method according to claim 1, characterized in that The step of acquiring the vehicle image and the scaled low-resolution image collected during the vehicle's travel includes: Acquire a road environment image, and perform light detection based on the road environment image to obtain brightness; When the brightness is lower than a preset brightness, during the process of capturing an image of a moving vehicle, the fill light is controlled to turn on to provide fill light, thereby obtaining an image of the vehicle; The copy of the vehicle image is scaled to obtain a low-resolution image.

3. The method according to claim 1, characterized in that The method further comprises: converting the vehicle position into a vehicle position in the vehicle image using an image preprocessing thread, cropping the vehicle image based on the vehicle position in the vehicle image, and scaling and normalizing the cropped vehicle image to obtain a preprocessed image; The license plate positioning thread is used to perform license plate positioning on the vehicle image based on the vehicle position to obtain the license plate position, which includes: The license plate positioning thread is used to perform license plate positioning on the pre-processed image to obtain the license plate position.

4. The method according to claim 1, wherein The method of using the vehicle positioning thread to locate the vehicle in the low-resolution image to obtain the vehicle position includes: When there are multiple vehicles in the low-resolution image, positioning the multiple vehicles in the low-resolution image using a vehicle positioning thread to obtain positions of the vehicles to be selected; From the candidate vehicle positions, a vehicle position that is located in the lane corresponding to the camera and at the bottom of the low-resolution image is selected.

5. The method according to any one of claims 1 to 4, characterized in that Before determining the license plate information of the vehicle according to the character recognition result and the color recognition result, the method further includes: If there are multiple vehicle images obtained, then selecting the character with the highest frequency of occurrence and the color with the highest frequency of occurrence at each character position from the character recognition results and the color recognition results of the multiple vehicle images and combining them to obtain a first combination result; The determining of the vehicle license plate information according to the character recognition result and the color recognition result includes: The license plate information of the vehicle is determined according to the first combination result.

6. The method according to claim 5, characterized in that The method further comprises: If the character with the highest frequency of occurrence does not meet the specification requirements, selecting the character with the second highest frequency of occurrence and the color with the highest frequency of occurrence at each character position from the character recognition results and the color recognition results of the plurality of vehicle images, and combining them to obtain a second combination result; The determining of the vehicle license plate information according to the character recognition result and the color recognition result includes: The license plate information of the vehicle is determined according to the second combination result.

7. A license plate information recognition device, characterized in that: The device is applied to a camera with a built-in NPU, and the device includes: an acquisition module, configured to acquire a vehicle image and a scaled low-resolution image captured while the vehicle is in motion; the vehicle image is captured with a fill light on, the fill light being mounted on a traffic gantry and positioned at an angle with the horizontal plane to prevent glare to the driver; A vehicle positioning module, configured to locate the vehicle in the low-resolution image using a vehicle positioning thread to obtain a vehicle position; A license plate positioning module, configured to perform license plate positioning on the vehicle image based on the vehicle position using a license plate positioning thread to obtain a license plate position; a character recognition module, configured to locate the license plate region in the vehicle image based on the license plate position using a license plate character recognition thread, and perform character recognition on the license plate region to obtain a character recognition result; A color recognition module is used to perform color recognition on the license plate area using a license plate color recognition thread to obtain a color recognition result; A determination module is used to determine the license plate information of the vehicle according to the character recognition result and the color recognition result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images