A gate license plate recognition method and system based on infrared triggering

Abstract

A gate license plate recognition method based on infrared triggering, comprising the following steps: setting a first infrared sensing area, and determining the opening mode of the gate based on the continuous sensing time length of the first infrared sensing area; wherein the opening mode includes: a first mode, using a first camera device in front of the gate to acquire images, obtaining vehicle license plate information and opening the gate to release; a second mode, using an infrared camera to identify images, obtaining vehicle license plate information, marking the coordinates of the vehicle, and determining whether to open the gate to release based on the coordinate information of the vehicle; when the continuous sensing time length is less than a time threshold, the first mode is adopted; when the continuous sensing time length is greater than or equal to the time threshold, the second mode is adopted. When there is congestion, the technology of pre-acquiring license plate information and vehicle tracking is used to quickly open the gate, improve the passing efficiency of the gate, and reduce the degree of parking lot congestion.

Description

Technical Field

[0001] This invention relates to the field of gate control technology, and in particular to a gate license plate recognition method and system based on infrared triggering. Background Technology

[0002] In current traffic management and parking operation scenarios, license plate recognition timing systems are widely used in parking lots, highway toll stations, and other locations. Currently, in parking lot operation and management, when a vehicle needs to enter a specific location or road section, it is necessary to first obtain the vehicle's license plate information through a camera, and then bind the license plate information with the entry duration to calculate the vehicle's parking time.

[0003] Traditional camera image acquisition and processing is easily affected by external environmental factors, especially in strong light, backlight, or rainy conditions. In these situations, license plate information may not be accurately recognized. Often, vehicles need to be moved back and forth so the camera can acquire license plate information from different angles. Especially during holidays, with numerous vehicles entering and exiting parking lots, unrecognizable license plates can cause congestion at the parking lot entrance. Vehicles unable to move can also become stuck, leading to further congestion. This necessitates manual gate opening by management personnel, significantly increasing the difficulty of managing parking lot entrances and exits. Summary of the Invention

[0004] To address the aforementioned shortcomings, the present invention aims to propose a gate license plate recognition method and system based on infrared triggering, thereby solving the problem of vehicles blocking parking lot gates due to license plate recognition issues in poor lighting conditions.

[0005] To achieve this objective, the present invention adopts the following technical solution: a gate license plate recognition method based on infrared triggering, comprising the following steps: A first infrared sensing area is set, and the opening mode of the gate is determined based on the continuous sensing duration of the first infrared sensing area. The opening modes include: a first mode in which the first camera device in front of the gate acquires an image, obtains the vehicle license plate information, and opens the gate to allow passage; The second mode uses an infrared camera to perform image recognition, obtain vehicle license plate information, mark the vehicle's coordinates, and determine whether to open the gate to allow passage based on the vehicle's coordinate information. When the continuous sensing duration is less than the time threshold, the first mode is used; when the continuous sensing duration is greater than or equal to the time threshold, the second mode is used.

[0006] Preferably, the steps for obtaining vehicle license plate information through image recognition using an infrared camera are as follows: Step A1: Preprocess the infrared image acquired by the infrared camera to obtain a preprocessed image; Step A2: Set high and low thresholds to extract pixels with drastic gradient changes in the image, obtain the license plate outline based on the Canny edge detection algorithm, and crop the license plate outline to obtain a preliminary license plate image; Step A3: Filter the preliminary license plate images based on the geometric features of the license plate to obtain the license plate image; Step A4: Perform perspective distortion correction on the license plate image to obtain the first processed image; Step A5: Perform character segmentation on the first processed image to obtain individual characters, and input the individual characters into the trained deep convolutional neural network to recognize the individual characters; Preliminary license plate information is obtained by combining the extracted characters in the order they were extracted. Step A6: Statistically analyze the preliminary license plate information of multiple frames within the same second, and select the license plate number with the highest frequency of occurrence as the license plate information.

[0007] Preferably, the preprocessing step in step A1 includes: Step A11: Convert the infrared image from color space to a single-channel grayscale image; Step A12: Perform histogram equalization or adaptive histogram equalization on the single-channel grayscale image to obtain a stretched image; Step A13: Perform Gaussian filtering and / or median filtering on the stretched image to obtain the preprocessed image.

[0008] Preferably, when the deep convolutional neural network cannot identify the initial license plate information, the following steps need to be performed before executing step A12: Step B1: Activate the second camera device to acquire a second image; wherein the second camera device is located next to the infrared camera, and the second camera device and the infrared camera are at the same shooting angle; Step B2: Perform grayscale conversion on the second image to obtain a grayscale image; Step B3: Perform geometric registration on the grayscale image and the single-channel grayscale image to align the grayscale image with the single-channel grayscale image; Step B4: Input the grayscale image and the single-channel grayscale image into the Laplacian pyramid, and obtain the first feature of the grayscale image and the second feature of the single-channel grayscale image respectively; Step B5: Selectively fuse the first feature and the second feature to obtain the reconstructed features; the fusion rules are as follows: The features of the non-last layer of the Laplace pyramid are fused using the following formula: ; in This represents the k-th first feature. This represents the k-th second feature; The features of the last layer of the Laplace pyramid were fused using a weighted fusion method; Step B6: Starting from the highest layer, the reconstructed features are reconstructed layer by layer upwards to obtain the updated single-channel grayscale image.

[0009] The preferred method for marking vehicle coordinates is as follows: Step C1: Obtain the coordinates (u,v) of the license plate information in the infrared image, and correct the coordinates (u,v) to undistorted coordinates. ; The formula for obtaining the distortion-free coordinates is as follows: , To remove distortion functions, The distortion coefficient; Step C2: Based on the intrinsic parameter matrix K of the infrared camera, convert the distortion-free coordinates... This is converted into a downlink normalized direction vector Pc from the infrared camera. , ; Step C3: Assume the license plate information is located on a plane in the world coordinate system. The depth factor is obtained based on the pre-calibrated camera extrinsic parameters, rotation matrix R and translation vector t. ; ; The rotation matrix R is a 3×3 orthogonal matrix. Let be the third row vector of the rotation matrix R; Step C4: Based on the depth factor Obtain the three-dimensional coordinates of the license plate information in the infrared camera coordinate system. and three-dimensional coordinates Transform to world coordinates Obtain the ground coordinates of the license plate information ( );

[0010] ; in World coordinates The first two components.

[0011] An infrared-triggered gate license plate recognition system, using the aforementioned infrared-triggered gate license plate recognition method, includes: Setting module: Used to set the first infrared sensing area, and to determine the gate opening mode based on the continuous sensing duration of the first infrared sensing area; The opening modes include: a first mode in which the first camera device in front of the gate acquires an image, obtains the vehicle license plate information, and opens the gate to allow passage; The second mode uses an infrared camera to perform image recognition, obtain vehicle license plate information, mark the vehicle's coordinates, and determine whether to open the gate to allow passage based on the vehicle's coordinate information. The selection module is used to use the first mode when the continuous sensing duration is less than the time threshold, and the second mode when the continuous sensing duration is greater than or equal to the time threshold.

[0012] Preferably, the setting module includes a license plate acquisition module; The license plate acquisition module is used to preprocess the infrared image acquired by the infrared camera to obtain a preprocessed image; High and low thresholds are set to extract pixels with drastic gradient changes in the image, and the license plate outline is obtained based on the Canny edge detection algorithm. The license plate outline is then cropped to obtain a preliminary license plate image. The initial license plate image is filtered based on the geometric features of the license plate to obtain the license plate image; Perspective distortion correction is performed on the license plate image to obtain the first processed image; The first processed image is segmented into individual characters, and these individual characters are then input into a trained deep convolutional neural network for recognition. Preliminary license plate information is obtained by combining the extracted characters in the order they were extracted. The preliminary license plate information of multiple frames within the same second is statistically analyzed, and the license plate number with the highest frequency of occurrence is selected as the license plate information.

[0013] Preferably, the license plate acquisition module includes a preprocessing submodule; The preprocessing submodule is used to convert the infrared image from color space into a single-channel grayscale image; The single-channel grayscale image is subjected to histogram equalization or adaptive histogram equalization to obtain a stretched image; The stretched image is subjected to Gaussian filtering and / or median filtering to obtain the preprocessed image.

[0014] Preferably, the license plate acquisition module further includes a reflection processing submodule; The reflection processing submodule is used to invoke a second camera device to acquire a second image; wherein the second camera device is located next to the infrared camera, and the second camera device and the infrared camera have the same shooting angle; The second image is converted to grayscale to obtain a grayscale image; Geometric registration is performed on the grayscale image and the single-channel grayscale image to align the grayscale image with the single-channel grayscale image; The grayscale image and the single-channel grayscale image are input into the Laplacian pyramid, and the first feature of the grayscale image and the second feature of the single-channel grayscale image are obtained respectively. The first and second features are selected and fused to obtain the reconstructed features; The reconstructed features are reconstructed starting from the highest layer and proceeding upwards layer by layer to obtain an updated single-channel grayscale image.

[0015] Preferably, the setting module includes a coordinate acquisition module; The coordinate acquisition module is used to acquire the coordinates (u,v) of the license plate information in the infrared image, and correct the coordinates (u,v) to distortion-free coordinates. ; Based on the intrinsic parameter matrix K of the infrared camera, distortion-free coordinates are... This is converted into a downlink normalized direction vector Pc from the infrared camera. Assume the license plate information is located on a plane in the world coordinate system. The depth factor is obtained based on the pre-calibrated camera extrinsic parameters, rotation matrix R and translation vector t. ; The rotation matrix R is a 3×3 orthogonal matrix. Let be the third row vector of the rotation matrix R; Based on the depth factor Obtain the three-dimensional coordinates of the license plate information in the infrared camera coordinate system. and three-dimensional coordinates Transform to world coordinates Obtain the ground coordinates of the license plate information ( ).

[0016] One of the above technical solutions has the following advantages or beneficial effects: In this invention, the first infrared sensing area is used as the trigger condition to determine whether there is a blockage in front of the gate. Based on the blockage, the gate opening mode is changed to improve the flexibility of gate management. When there is a blockage, the gate is opened quickly by using the technology of pre-acquiring license plate information and vehicle tracking, thereby improving the gate's passage efficiency and reducing the degree of parking lot congestion. Attached Figure Description

[0017] Figure 1 This is a flowchart of one embodiment of the method of the present invention.

[0018] Figure 2 This is a schematic diagram of the structure of one embodiment of the system of the present invention. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0020] In the description of embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0022] like Figures 1-2 As shown, a gate license plate recognition method based on infrared triggering includes the following steps: A first infrared sensing area is set, and the opening mode of the gate is determined based on the continuous sensing duration of the first infrared sensing area. The opening modes include: a first mode in which the first camera device in front of the gate acquires an image, obtains the vehicle license plate information, and opens the gate to allow passage; The second mode uses an infrared camera to perform image recognition, obtain vehicle license plate information, mark the vehicle's coordinates, and determine whether to open the gate to allow passage based on the vehicle's coordinate information. When the continuous sensing duration is less than the time threshold, the first mode is used; when the continuous sensing duration is greater than or equal to the time threshold, the second mode is used.

[0023] To effectively address the issue of vehicles smoothly entering parking lots during inclement weather, this invention incorporates a first infrared sensing area, positioned 6-8 meters in front of the parking lot gate. When the continuous sensing duration of the first infrared sensing area exceeds a time threshold, it indicates a queue forming in front of the parking lot gate. If the traditional first mode is used for control, the first camera device will be unable to recognize license plates, leading to continued congestion and hindering the control and management of the parking lot gate. The first camera device in this first mode is a standard camera installed on one side of the gate.

[0024] Therefore, when the continuous sensing duration exceeds or equals the time threshold, the gate's opening mode needs to be changed to the second mode. In the second mode, an infrared camera is used for image recognition. The infrared camera is positioned between the gate and the first infrared sensing area, ahead of the first camera device, allowing it to obtain vehicle license plate information in advance. Since the infrared images obtained by the infrared camera are unaffected by light, image information can be effectively extracted even in bright light, backlight, or dark environments such as rainy days, thus obtaining the vehicle's license plate information. While obtaining vehicle information via license plate, the vehicle's coordinate information is also obtained and continuously tracked. Because some roads are connected to the parking lot gate entrance, vehicles may be unable to enter the road when the parking lot entrance is blocked. Therefore, coordinate tracking can determine whether the vehicle is entering the parking lot or another road. Since the license plate information has already been obtained through the infrared camera, once it is determined that the vehicle has entered the parking lot, it is bound to the license plate. When the vehicle reaches the corresponding position, the gate can be opened directly without needing to take a picture through the first camera device, improving the gate's communication efficiency.

[0025] In this invention, the first infrared sensing area is used as the trigger condition to determine whether there is a blockage in front of the gate. Based on the blockage situation, the gate opening mode is changed to improve the flexibility of gate management. When there is a blockage, the gate is opened quickly by using the technology of pre-acquiring license plate information and vehicle tracking, thereby improving the gate's passage efficiency and reducing the degree of parking lot congestion.

[0026] Preferably, the steps for obtaining vehicle license plate information through image recognition using an infrared camera are as follows: Step A1: Preprocess the infrared image acquired by the infrared camera to obtain a preprocessed image; This is to reduce unnecessary data processing and to maintain license plate information, so that subsequent steps can proceed normally.

[0027] Step A2: Set high and low thresholds to extract pixels with drastic gradient changes in the image, obtain the license plate outline based on the Canny edge detection algorithm, and crop the license plate outline to obtain a preliminary license plate image; After preprocessing, the license plate area in the preprocessed image exhibits high contrast between characters and the background (with dramatic gradient changes at character edges). Therefore, the gradient magnitude of character edge pixels often exceeds the high threshold, resulting in strong edges. Edges such as the license plate border and rivets have slightly weaker gradients but remain between the high and low thresholds, thus becoming weak edges. Therefore, high and low thresholds can be obtained through multiple experiments. These thresholds are then used to acquire the license plate outline. Since these edges are spatially connected to the strong edges (character edges), they are preserved along with the strong edges (character edges) when using the Canny edge detection algorithm to acquire the license plate outline, forming a complete character outline.

[0028] Step A3: Filter the preliminary license plate images based on the geometric features of the license plate to obtain the license plate image; The initial license plate image extracted using the Canny edge detection algorithm is based on a dual threshold method (high and low thresholds). Therefore, non-license plate images can be extracted from the initial license plate image. Thus, the initial license plate image needs to be filtered. In this invention, the filtering is based on the geometric features of the license plate, such as the length-to-width ratio and the number of pixels. For example, the standard license plate size for a small car is 440 mm long and 140 mm wide. Due to the angle of the infrared camera, the length may not meet the 440 mm standard. Therefore, the length-to-width ratio can be set to 220-340:140, and the number of pixels in both length and width must also meet certain requirements before it can be identified as a license plate.

[0029] Step A4: Perform perspective distortion correction on the license plate image to obtain the first processed image; Due to the shooting angle of infrared cameras, license plates are not captured head-on. Directly using the license plate image for deep convolutional neural network recognition can easily lead to misidentification. Therefore, perspective distortion correction is necessary. Specifically, Harris corner detection can be used to obtain the four corners of the license plate image. Based on the correspondence between the four vertices and the desired frontal rectangle, the perspective transformation matrix is ​​calculated. Finally, the transformation matrix is ​​applied to resample the candidate region (commonly using bilinear interpolation) to generate a first processed image with a frontal view and rectangular shape.

[0030] Step A5: Perform character segmentation on the first processed image to obtain individual characters, and input the individual characters into the trained deep convolutional neural network to recognize the individual characters; Preliminary license plate information is obtained by combining the extracted characters in the order they were extracted. Step A6: Statistically analyze the preliminary license plate information of multiple frames within the same second, and select the license plate number with the highest frequency of occurrence as the license plate information.

[0031] Since single-shot recognition may contain errors, this invention performs steps A1-A5 on multiple frames of infrared images within the same second to obtain multiple preliminary license plate information. Finally, the license plate number with the highest frequency of occurrence is selected as the final license plate information. This improves the stability of license plate recognition.

[0032] Preferably, the preprocessing step in step A1 includes: Step A11: Convert the infrared image from color space to a single-channel grayscale image; Step A12: Perform histogram equalization or adaptive histogram equalization on the single-channel grayscale image to obtain a stretched image; Step A13: Perform Gaussian filtering and / or median filtering on the stretched image to obtain the preprocessed image.

[0033] Infrared images acquired under conditions of strong light or backlight have significant brightness information, and direct processing would increase computational load and introduce unnecessary parameters. Therefore, during preprocessing, the infrared image is first converted from RGB or infrared pseudo-color space to a single-channel grayscale image. The grayscale values ​​are then used to filter out strong light or backlight information, thereby reducing computational load and preserving normal brightness information.

[0034] For darker environments such as nighttime or rainy days, this invention performs histogram equalization or adaptive histogram equalization on the single-channel grayscale image. This stretches the grayscale distribution within the single-channel grayscale image, increasing the difference between the license plate characters and the background, thus aiding the subsequent Canny edge detection algorithm and improving the license plate extraction rate. However, since the license plate characters may become blurry after stretching, Gaussian filtering can remove fine noise, while median filtering can remove black and white noise, maintaining the sharpness of the license plate characters and facilitating font recognition by the subsequent deep convolutional neural network.

[0035] Preferably, when the deep convolutional neural network cannot identify the initial license plate information, the following steps need to be performed before executing step A12: Step B1: Activate the second camera device to acquire a second image; wherein the second camera device is located next to the infrared camera, and the second camera device and the infrared camera are at the same shooting angle; Step B2: Perform grayscale conversion on the second image to obtain a grayscale image; Step B3: Perform geometric registration on the grayscale image and the single-channel grayscale image to align the grayscale image with the single-channel grayscale image; Step B4: Input the grayscale image and the single-channel grayscale image into the Laplacian pyramid, and obtain the first feature of the grayscale image and the second feature of the single-channel grayscale image respectively; Step B5: Selectively fuse the first feature and the second feature to obtain the reconstructed features; the fusion rules are as follows: The features of the non-last layer of the Laplace pyramid are fused using the following formula: ; in This represents the k-th first feature. This represents the k-th second feature; The features of the last layer of the Laplace pyramid were fused using a weighted fusion method; Step B6: Starting from the highest layer, the reconstructed features are reconstructed layer by layer upwards to obtain the updated single-channel grayscale image.

[0036] In this invention, the second mode is used to control the gate in some situations with strong light. Due to the paint on the license plate, some strong light may be reflected at a specific angle to the infrared camera. In this case, the license plate will appear severely overexposed in the infrared image acquired by the infrared camera, becoming an unrecognizable white bright spot, thus making it impossible for the infrared camera to identify. Therefore, in this invention, a second camera device is provided on one side of the infrared camera. When the deep convolutional neural network cannot identify the initial license plate information, the image acquired by the second camera device is fused with the infrared image to update the single-channel grayscale image.

[0037] Specifically, firstly, a visible light image (i.e., the second image) is acquired using the second camera device. Then, the second image is converted to grayscale to obtain a grayscale image. This grayscale image is then registered with the single-channel grayscale image to align the license plates in the two images at the pixel level. At this point, although the license plate area in the infrared image is overexposed and appears as a white blank, the information in this area is complete in the visible light second image. In subsequent processing, the information from the license plate area in the second image can be used for fusion to compensate for the overexposed license plate area. During the fusion process, this invention uses the Laplacian pyramid for fusion because the overexposed area appears as a large, saturated, uniform white spot in the infrared image, and its high-frequency information is basically lost. However, in the visible light image, the high-frequency details (character edges, rivet textures) in the same area still exist. The multi-scale decomposition of the Laplacian pyramid can separate these high-frequency components and selectively extract the second feature from the visible light. When fusing the first and second features, the Laplacian coefficient of the overexposed license plate area in the infrared image (excluding the last layer of the Laplacian pyramid, which is higher) approaches 0, indicating almost no detail. In contrast, the absolute value of the Laplacian coefficient at the same location in the visible light image is much larger, such as at character edges. The fusion rule, which selects the larger absolute value, naturally prioritizes visible light details while not ignoring some details in the infrared image, thus bringing clear license plate characters into the final image. In the last layer of the Laplacian pyramid, the overall illumination outline is represented. The low-frequency layer of the infrared image has extremely high brightness at the license plate location, while the low-frequency layer of the visible light is more reasonable. During fusion, the weight of the infrared low frequencies can be reduced to avoid the final image being too bright overall. Finally, the reconstructed features are reconstructed layer by layer from the highest layer upwards, resulting in an updated single-channel grayscale image.

[0038] The preferred method for marking vehicle coordinates is as follows: Step C1: Obtain the coordinates (u,v) of the license plate information in the infrared image, and correct the coordinates (u,v) to undistorted coordinates. ; The formula for obtaining the distortion-free coordinates is as follows: , To remove distortion functions, The distortion coefficient; Step C2: Based on the intrinsic parameter matrix K of the infrared camera, convert the distortion-free coordinates... This is converted into a downlink normalized direction vector Pc from the infrared camera. , ; Step C3: Assume the license plate information is located on a plane in the world coordinate system. The depth factor is obtained based on the pre-calibrated camera extrinsic parameters, rotation matrix R and translation vector t. ; ; The rotation matrix R is a 3×3 orthogonal matrix. Let be the third row vector of the rotation matrix R; Step C4: Based on the depth factor Obtain the three-dimensional coordinates of the license plate information in the infrared camera coordinate system. and three-dimensional coordinates Transform to world coordinates Obtain the ground coordinates of the license plate information ( );

[0039] ; in World coordinates The first two components.

[0040] An infrared-triggered gate license plate recognition system, using the aforementioned infrared-triggered gate license plate recognition method, includes: Setting module: Used to set the first infrared sensing area, and to determine the gate opening mode based on the continuous sensing duration of the first infrared sensing area; The opening modes include: a first mode in which the first camera device in front of the gate acquires an image, obtains the vehicle license plate information, and opens the gate to allow passage; The second mode uses an infrared camera to perform image recognition, obtain vehicle license plate information, mark the vehicle's coordinates, and determine whether to open the gate to allow passage based on the vehicle's coordinate information. The selection module is used to use the first mode when the continuous sensing duration is less than the time threshold, and the second mode when the continuous sensing duration is greater than or equal to the time threshold.

[0041] Preferably, the setting module includes a license plate acquisition module; The license plate acquisition module is used to preprocess the infrared image acquired by the infrared camera to obtain a preprocessed image; High and low thresholds are set to extract pixels with drastic gradient changes in the image, and the license plate outline is obtained based on the Canny edge detection algorithm. The license plate outline is then cropped to obtain a preliminary license plate image. The initial license plate image is filtered based on the geometric features of the license plate to obtain the license plate image; Perspective distortion correction is performed on the license plate image to obtain the first processed image; The first processed image is segmented into individual characters, and these individual characters are then input into a trained deep convolutional neural network for recognition. Preliminary license plate information is obtained by combining the extracted characters in the order they were extracted. The preliminary license plate information of multiple frames within the same second is statistically analyzed, and the license plate number with the highest frequency of occurrence is selected as the license plate information.

[0042] Preferably, the license plate acquisition module includes a preprocessing submodule; The preprocessing submodule is used to convert the infrared image from color space into a single-channel grayscale image; The single-channel grayscale image is subjected to histogram equalization or adaptive histogram equalization to obtain a stretched image; The stretched image is subjected to Gaussian filtering and / or median filtering to obtain the preprocessed image.

[0043] Preferably, the license plate acquisition module further includes a reflection processing submodule; The reflection processing submodule is used to invoke a second camera device to acquire a second image; wherein the second camera device is located next to the infrared camera, and the second camera device and the infrared camera have the same shooting angle; The second image is converted to grayscale to obtain a grayscale image; Geometric registration is performed on the grayscale image and the single-channel grayscale image to align the grayscale image with the single-channel grayscale image; The grayscale image and the single-channel grayscale image are input into the Laplacian pyramid, and the first feature of the grayscale image and the second feature of the single-channel grayscale image are obtained respectively. The first and second features are selected and fused to obtain the reconstructed features; The reconstructed features are reconstructed starting from the highest layer and proceeding upwards layer by layer to obtain an updated single-channel grayscale image.

[0044] Preferably, the setting module includes a coordinate acquisition module; The coordinate acquisition module is used to acquire the coordinates (u,v) of the license plate information in the infrared image, and correct the coordinates (u,v) to distortion-free coordinates. ; Based on the intrinsic parameter matrix K of the infrared camera, distortion-free coordinates are... This is converted into a downlink normalized direction vector Pc from the infrared camera. Assume the license plate information is located on a plane in the world coordinate system. The depth factor is obtained based on the pre-calibrated camera extrinsic parameters, rotation matrix R and translation vector t. ; The rotation matrix R is a 3×3 orthogonal matrix. Let be the third row vector of the rotation matrix R; Based on the depth factor Obtain the three-dimensional coordinates of the license plate information in the infrared camera coordinate system. and three-dimensional coordinates Transform to world coordinates Obtain the ground coordinates of the license plate information ( ).

[0045] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0046] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A gate license plate recognition method based on infrared triggering, characterized in that, Includes the following steps: A first infrared sensing area is set, and the opening mode of the gate is determined based on the continuous sensing duration of the first infrared sensing area. The opening modes include: a first mode in which the first camera device in front of the gate acquires an image, obtains the vehicle license plate information, and opens the gate to allow passage; The second mode uses an infrared camera to perform image recognition, obtain vehicle license plate information, mark the vehicle's coordinates, and determine whether to open the gate to allow passage based on the vehicle's coordinate information. When the continuous sensing duration is less than the time threshold, the first mode is used; when the continuous sensing duration is greater than or equal to the time threshold, the second mode is used.

2. A gate license plate recognition method based on infrared triggering according to claim 1, characterized in that, The steps to obtain vehicle license plate information through image recognition using an infrared camera are as follows: Step A1: Preprocess the infrared image acquired by the infrared camera to obtain a preprocessed image; Step A2: Set high and low thresholds to extract pixels with drastic gradient changes in the image, obtain the license plate outline based on the Canny edge detection algorithm, and crop the license plate outline to obtain a preliminary license plate image; Step A3: Filter the preliminary license plate images based on the geometric features of the license plate to obtain the license plate image; Step A4: Perform perspective distortion correction on the license plate image to obtain the first processed image; Step A5: Perform character segmentation on the first processed image to obtain individual characters, and input the individual characters into the trained deep convolutional neural network to recognize the individual characters; Preliminary license plate information is obtained by combining the extracted characters in the order they were extracted. Step A6: Statistically analyze the preliminary license plate information of multiple frames within the same second, and select the license plate number with the highest frequency of occurrence as the license plate information.

3. A gate license plate recognition method based on infrared triggering according to claim 2, characterized in that, The preprocessing steps in step A1 include: Step A11: Convert the infrared image from color space to a single-channel grayscale image; Step A12: Perform histogram equalization or adaptive histogram equalization on the single-channel grayscale image to obtain a stretched image; Step A13: Perform Gaussian filtering and / or median filtering on the stretched image to obtain the preprocessed image.

4. A gate license plate recognition method based on infrared triggering according to claim 3, characterized in that, When the deep convolutional neural network fails to recognize the initial license plate information, the following steps need to be performed before executing step A12: Step B1: Activate the second camera device to acquire a second image; wherein the second camera device is located next to the infrared camera, and the second camera device and the infrared camera are at the same shooting angle; Step B2: Perform grayscale conversion on the second image to obtain a grayscale image; Step B3: Perform geometric registration on the grayscale image and the single-channel grayscale image to align the grayscale image with the single-channel grayscale image; Step B4: Input the grayscale image and the single-channel grayscale image into the Laplacian pyramid, and obtain the first feature of the grayscale image and the second feature of the single-channel grayscale image respectively; Step B5: Select and fuse the first feature and the second feature to obtain the reconstructed features; The fusion rules are as follows: The features of the non-last layer of the Laplace pyramid are fused using the following formula: ； in This represents the k-th first feature. This represents the k-th second feature; The features of the last layer of the Laplace pyramid were fused using a weighted fusion method; Step B6: Starting from the highest layer, the reconstructed features are reconstructed layer by layer upwards to obtain the updated single-channel grayscale image.

5. A gate license plate recognition method based on infrared triggering according to claim 1, characterized in that, The method for marking vehicle coordinates is as follows: Step C1: Obtain the coordinates (u,v) of the license plate information in the infrared image, and correct the coordinates (u,v) to undistorted coordinates. ; The formula for obtaining the distortion-free coordinates is as follows: , To remove distortion functions, The distortion coefficient; Step C2: Based on the intrinsic parameter matrix K of the infrared camera, convert the distortion-free coordinates... This is converted into a downlink normalized direction vector Pc from the infrared camera. ， ； Step C3: Assume the license plate information is located on a plane in the world coordinate system. The depth factor is obtained based on the pre-calibrated camera extrinsic parameters, rotation matrix R and translation vector t. ; ； The rotation matrix R is a 3×3 orthogonal matrix. Let be the third row vector of the rotation matrix R; Step C4: Based on the depth factor Obtain the three-dimensional coordinates of the license plate information in the infrared camera coordinate system. and three-dimensional coordinates Transform to world coordinates Obtain the ground coordinates of the license plate information ( ); ； in World coordinates The first two components.

6. A gate license plate recognition system based on infrared triggering, using the gate license plate recognition method based on infrared triggering as described in any one of claims 1 to 5, characterized in that, include Setting module: Used to set the first infrared sensing area, and to determine the gate opening mode based on the continuous sensing duration of the first infrared sensing area; The opening modes include: a first mode in which the first camera device in front of the gate acquires an image, obtains the vehicle license plate information, and opens the gate to allow passage; The second mode uses an infrared camera to perform image recognition, obtain vehicle license plate information, mark the vehicle's coordinates, and determine whether to open the gate to allow passage based on the vehicle's coordinate information. The selection module is used to use the first mode when the continuous sensing duration is less than the time threshold, and the second mode when the continuous sensing duration is greater than or equal to the time threshold.

7. A gate license plate recognition system based on infrared triggering according to claim 6, characterized in that, The setting module includes a license plate acquisition module; The license plate acquisition module is used to preprocess the infrared image acquired by the infrared camera to obtain a preprocessed image; High and low thresholds are set to extract pixels with drastic gradient changes in the image, and the license plate outline is obtained based on the Canny edge detection algorithm. The license plate outline is then cropped to obtain a preliminary license plate image. The initial license plate image is filtered based on the geometric features of the license plate to obtain the license plate image; Perspective distortion correction is performed on the license plate image to obtain the first processed image; The first processed image is segmented into individual characters, and these individual characters are then input into a trained deep convolutional neural network for recognition. Preliminary license plate information is obtained by combining the extracted characters in the order they were extracted. The preliminary license plate information of multiple frames within the same second is statistically analyzed, and the license plate number with the highest frequency of occurrence is selected as the license plate information.

8. A gate license plate recognition system based on infrared triggering according to claim 7, characterized in that, The license plate acquisition module includes a preprocessing submodule; The preprocessing submodule is used to convert the infrared image from color space into a single-channel grayscale image; The single-channel grayscale image is subjected to histogram equalization or adaptive histogram equalization to obtain a stretched image; The stretched image is subjected to Gaussian filtering and / or median filtering to obtain the preprocessed image.

9. A gate license plate recognition system based on infrared triggering according to claim 7, characterized in that, The license plate acquisition module also includes a reflection processing submodule; The reflection processing submodule is used to invoke a second camera device to acquire a second image; wherein the second camera device is located next to the infrared camera, and the second camera device and the infrared camera have the same shooting angle; The second image is converted to grayscale to obtain a grayscale image; Geometric registration is performed on the grayscale image and the single-channel grayscale image to align the grayscale image with the single-channel grayscale image; The grayscale image and the single-channel grayscale image are input into the Laplacian pyramid, and the first feature of the grayscale image and the second feature of the single-channel grayscale image are obtained respectively. The first and second features are selected and fused to obtain the reconstructed features; The reconstructed features are reconstructed starting from the highest layer and proceeding upwards layer by layer to obtain an updated single-channel grayscale image.

10. A gate license plate recognition system based on infrared triggering according to claim 6, characterized in that, The settings module includes a coordinate acquisition module; The coordinate acquisition module is used to acquire the coordinates (u,v) of the license plate information in the infrared image, and correct the coordinates (u,v) to distortion-free coordinates. ; Based on the intrinsic parameter matrix K of the infrared camera, distortion-free coordinates are... This is converted into a downlink normalized direction vector Pc from the infrared camera. Assume the license plate information is located on a plane in the world coordinate system. The depth factor is obtained based on the pre-calibrated camera extrinsic parameters, rotation matrix R and translation vector t. ; The rotation matrix R is a 3×3 orthogonal matrix. Let be the third row vector of the rotation matrix R; Based on the depth factor Obtain the three-dimensional coordinates of the license plate information in the infrared camera coordinate system. and three-dimensional coordinates Transform to world coordinates Obtain the ground coordinates of the license plate information ( ).

Images