Automatic barrier gate control system and method based on computer vision

Abstract

The invention relates to the field of barrier gate control, and particularly discloses a barrier gate automatic control system and method based on computer vision, and the method comprises the steps: firstly obtaining a barrier gate entrance vehicle image collected by a visual sensor, then carrying out the feature extraction and correlation analysis of the image through a deep learning technology, and finally obtaining a classification result through a classifier. Therefore, a corresponding control signal is sent to the barrier gate, so that the requirements of manual operation and inspection are reduced, the passing efficiency is improved, the congestion phenomenon is avoided, and the experience and the satisfaction degree of a user are improved.

Description

Technical Field

[0001] The present application relates to the field of barrier gate control, and more specifically, to a barrier gate automatic control system and method based on computer vision. Background Art

[0002] The barrier is an automated device widely used in parking lots, community gates, industrial zone entrances and other areas to manage vehicle access. Its main function is to block or release vehicles by raising and lowering the control lever, thereby ensuring that only authorized vehicles can pass.

[0003] With the advancement of technology, parking management systems are becoming more intelligent, and the entrance and exit control system is particularly critical. At present, many systems still rely on manual management to identify vehicles and control the lifting and lowering of the gate. This method is not only inefficient, but also prone to management errors, which may affect user experience and satisfaction.

[0004] Therefore, a computer vision-based automatic gate control system and method are desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a barrier gate automatic control system and method based on computer vision, which first obtains the barrier gate entrance vehicle image collected by the visual sensor, then uses deep learning technology to perform feature extraction and association analysis, and finally obtains the classification result through the classifier to determine whether the barrier gate entrance vehicle is an authorized vehicle, thereby sending a corresponding control signal to the barrier gate, thereby reducing the need for manual operation and inspection, improving traffic efficiency, avoiding congestion, and enhancing user experience and satisfaction.

[0006] According to one aspect of the present application, a computer vision-based automatic gate control system is provided, which includes:

[0007] A gate entrance image acquisition module is used to acquire a vehicle image at the gate entrance collected by a visual sensor;

[0008] A gate entrance data extraction module, used to extract a gate entrance vehicle appearance enhancement feature vector and a gate entrance vehicle license plate grayscale feature vector from the gate entrance vehicle image collected by the visual sensor;

[0009] The authorized vehicle judgment module is used to judge whether the vehicle at the gate entrance is an authorized vehicle based on the appearance enhancement feature vector of the vehicle at the gate entrance and the grayscale feature vector of the license plate of the vehicle at the gate entrance.

[0010] According to another aspect of the present application, a computer vision-based automatic gate control method is provided, which includes:

[0011] Acquire vehicle images at the gate entrance collected by the visual sensor;

[0012] Extracting a gate entrance vehicle appearance enhancement feature vector and a gate entrance vehicle license plate grayscale feature vector from the gate entrance vehicle image collected by the visual sensor;

[0013] Based on the appearance enhancement feature vector of the vehicle at the gate entrance and the license plate grayscale feature vector of the vehicle at the gate entrance, it is determined whether the vehicle at the gate entrance is an authorized vehicle.

[0014] Compared with the prior art, the present application provides a computer vision-based barrier gate automatic control system and method, which first obtains the vehicle image at the barrier gate entrance collected by the visual sensor, then uses deep learning technology to perform feature extraction and association analysis, and finally obtains the classification result through the classifier to determine whether the vehicle at the barrier gate entrance is an authorized vehicle, thereby sending a corresponding control signal to the barrier gate, thereby reducing the need for manual operation and inspection, improving traffic efficiency, avoiding congestion, and enhancing user experience and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 It is a block diagram of a computer vision-based automatic gate control system according to an embodiment of the present application.

[0017] Figure 2 It is a block diagram of a barrier entrance data extraction module in a barrier automatic control system based on computer vision according to an embodiment of the present application.

[0018] Figure 3 It is a block diagram of a vehicle appearance feature extraction unit at a barrier entrance in a barrier automatic control system based on computer vision according to an embodiment of the present application.

[0019] Figure 4 It is a block diagram of an authorized vehicle judgment module in a computer vision-based barrier gate automatic control system according to an embodiment of the present application.

[0020] Figure 5 Flow chart of a barrier gate automatic control method based on computer vision according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0022] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0023] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0024] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0025] Figure 1 FIG. 1 is a block diagram of a computer vision-based automatic gate control system according to an embodiment of the present application. Figure 1 As shown, according to the computer vision-based barrier gate automatic control system 100 of the embodiment of the present application, it includes: a barrier gate entrance image acquisition module 110, which is used to acquire the barrier gate entrance vehicle image collected by the visual sensor; a barrier gate entrance data extraction module 120, which is used to extract the barrier gate entrance vehicle appearance enhancement feature vector and the barrier gate entrance vehicle license plate grayscale feature vector from the barrier gate entrance vehicle image collected by the visual sensor; an authorized vehicle judgment module 130, which is used to judge whether the barrier gate entrance vehicle is an authorized vehicle based on the barrier gate entrance vehicle appearance enhancement feature vector and the barrier gate entrance vehicle license plate grayscale feature vector.

[0026] In the above-mentioned automatic gate control system 100 based on computer vision, the gate entrance image acquisition module 110 is used to acquire the gate entrance vehicle image collected by the visual sensor. It should be understood that the gate is a common automation equipment, which is widely used in parking lots, community entrances and industrial park gates, etc., to control the entry and exit of vehicles. Its main function is to block or release vehicles by lifting and lowering control rods, thereby ensuring that only authorized vehicles can pass. With the development of science and technology, parking management systems are becoming more and more intelligent, among which the role of entrance and exit control systems is particularly prominent. At present, many parking management systems still rely on manual identification of vehicles and operation of the gate. However, this manual method is inefficient and prone to errors, which may affect the user experience and satisfaction. Therefore, in the technical solution of the present application, by acquiring the gate entrance vehicle image collected by the visual sensor, and combining the deep learning technology, it is determined whether the vehicle at the gate entrance is authorized to send a corresponding control signal to the gate. This automated processing reduces the reliance on manual operation and inspection, thereby improving traffic efficiency, reducing congestion, and enhancing user experience and satisfaction.

[0027] Specifically, the visual sensor can automatically identify the vehicle by capturing the vehicle image at the gate entrance, thereby analyzing the features in the vehicle image, such as license plate number, vehicle appearance, and color. Through the analysis and application of image data, management strategies and system functions can be further optimized, bringing a more comprehensive and intelligent solution to parking lot management.

[0028] In the above-mentioned automatic gate control system 100 based on computer vision, the gate entrance data extraction module 120 is used to extract the gate entrance vehicle appearance enhancement feature vector and the gate entrance vehicle license plate grayscale feature vector from the gate entrance vehicle image collected by the visual sensor. This is important for improving recognition accuracy, supporting multiple recognition scenarios, improving the system intelligence level, supporting data analysis and optimization, and enhancing user experience, so as to achieve more accurate and efficient vehicle recognition and management.

[0029] Figure 2 FIG. 1 is a block diagram of a gate entrance data extraction module in a gate automatic control system based on computer vision according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the barrier entrance data extraction module 120 includes: a barrier entrance vehicle appearance feature extraction unit 121, which is used to extract vehicle appearance features from the barrier entrance vehicle image collected by the visual sensor to obtain the barrier entrance vehicle appearance enhancement feature vector; a barrier entrance vehicle license plate feature extraction unit 122, which is used to extract license plate features from the barrier entrance vehicle image collected by the visual sensor to obtain the barrier entrance vehicle license plate grayscale feature vector.

[0030] It should be understood that vehicle appearance feature extraction can capture information such as the vehicle's color, shape, brand logo, body outline, etc., which are important features for identifying vehicles. The appearance-enhanced feature vector can significantly improve the recognition accuracy of the system in complex environments by accurately encoding these details. For example, when the license plate is difficult to recognize due to damage or poor lighting conditions, the appearance features can still provide effective information to help identify the vehicle. The extracted appearance-enhanced feature vectors are not only used for vehicle identification, but can also support more complex functions such as vehicle classification (such as sedans, SUVs, trucks), behavior analysis (such as parking location analysis), etc. Through deep learning algorithms, these feature vectors can perform in-depth pattern recognition, giving the system more intelligent functions to meet various management needs.

[0031] Furthermore, the license plate feature extraction of the vehicle image at the gate entrance collected by the visual sensor is to achieve efficient and accurate vehicle identification and management. License plate recognition is one of the core functions in the vehicle management system. License plate grayscale feature vector extraction involves converting the license plate image into grayscale values ​​to capture the subtle features of the characters and numbers on the license plate. These grayscale features can effectively distinguish different parts of the license plate, such as the shape and arrangement of letters and numbers, and the overall layout of the license plate. This fine feature extraction can greatly improve the accuracy of license plate recognition, especially when the license plate clarity is limited or the environmental conditions are poor. License plate grayscale feature vector extraction can cope with the impact of different lighting, weather, license plate damage and other factors on image quality. Grayscale feature extraction excludes color information and focuses on shape and structural features, which enables the system to maintain high recognition performance in different environments. Even if the license plate is reflected by light or covered with dirt, the system can find the character information on the license plate through grayscale feature extraction, thereby improving the robustness of recognition.

[0032] Figure 3 FIG. 1 is a block diagram of a vehicle appearance feature extraction unit at a gate entrance in a gate automatic control system based on computer vision according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the vehicle appearance feature extraction unit 121 at the gate entrance includes: a vehicle appearance feature preprocessing subunit 1211 at the gate entrance, which is used to perform feature preprocessing on the vehicle image at the gate entrance collected by the visual sensor to obtain a Canny edge detection map of the vehicle appearance enhanced at the gate entrance; and a vehicle appearance feature encoding subunit 1212 at the gate entrance, which is used to perform feature encoding on the Canny edge detection map of the vehicle appearance enhanced at the gate entrance to obtain a feature vector of the vehicle appearance enhanced at the gate entrance.

[0033] It should be understood that there may be factors such as lighting changes, shadows, and license plate damage in vehicle images, which may interfere with vehicle recognition. Canny edge detection effectively reduces the impact of these interfering factors on recognition by extracting edge features. Edge detection can provide a more stable and consistent feature representation, allowing the recognition algorithm to maintain high robustness and stability when facing complex environmental conditions, thereby improving the overall performance of the system. Specifically, the Canny edge detection algorithm is a classic image processing technique used to identify edge features in images. By applying Canny edge detection to vehicle images, the vehicle outline and other important edge information can be effectively extracted. This enhanced edge information is crucial for identifying the appearance characteristics of the vehicle, because the shape and edge features of the vehicle are key visual information in various recognition tasks. By highlighting the edges of the vehicle, the system can capture the vehicle's geometry more clearly, thereby improving the accuracy of feature extraction.

[0034] Furthermore, the original edge detection map is still a two-dimensional image data, and it may be complicated to analyze it directly. Through feature encoding, these edge features can be converted into one-dimensional feature vectors. This encoding form not only simplifies the data structure. Among them, feature encoding expresses the important information in the image in a structured form by numerically processing the edge detection map. During the encoding process, the system can extract key information such as the vehicle's geometric features and edge distribution, and represent this information in the form of feature vectors. This method can enhance the expressiveness of features, allowing the recognition algorithm to more accurately capture and utilize the appearance features of the vehicle, thereby improving the performance of the recognition system.

[0035] In a specific embodiment of the present application, the vehicle appearance feature preprocessing subunit 1211 at the gate entrance includes: passing the vehicle image at the gate entrance collected by the visual sensor through a vehicle appearance image pixel enhancer to obtain a vehicle appearance pixel enhanced image at the gate entrance; performing Canny edge detection on the vehicle appearance pixel enhanced image at the gate entrance to obtain the vehicle appearance enhanced Canny edge detection image at the gate entrance.

[0036] It should be understood that the original vehicle image at the gate entrance may be affected by factors such as insufficient lighting, low contrast, and noise interference, resulting in unclear image details. The image pixel enhancer can significantly improve the visual effect of the image by adjusting the brightness, contrast, and clarity of the image, making the details of the vehicle more obvious and easy to identify. Among them, the pixel enhancer can make these important visual information more prominent by optimizing key areas in the image, such as the outline of the vehicle, license plate, and other identification features. This enhancement makes the edges and details more distinct, helping the recognition system to better extract and identify the appearance features of the vehicle. Specifically, the gate entrance vehicle image collected by the visual sensor is input into the encoder of the gate entrance vehicle appearance image pixel enhancer, wherein the encoder uses a convolution layer to explicitly spatially encode the gate entrance vehicle image collected by the visual sensor to obtain image features; and the image features are input into the decoder of the gate entrance vehicle appearance image pixel enhancer, wherein the decoder uses a deconvolution layer to deconvolve the image features to obtain the gate entrance vehicle appearance pixel enhanced image.

[0037] Furthermore, although pixel enhancers can improve the overall quality of an image, there may still be some noise or interference in details. The Canny edge detection algorithm can effectively suppress the influence of noise and accurately extract edge features through multi-level processing (such as Gaussian filtering, gradient calculation, and non-maximum suppression) during the edge detection process. This helps ensure that the detection results are more stable and reliable, thereby reducing errors caused by noise or image defects.

[0038] In a specific embodiment of the present application, the barrier entrance vehicle appearance feature encoding subunit 1212 includes: passing the barrier entrance vehicle appearance enhanced Canny edge detection map through a barrier entrance vehicle appearance enhanced Canny edge detection feature encoder based on a spatial attention mechanism to obtain a barrier entrance vehicle appearance enhanced feature map; pooling the barrier entrance vehicle appearance enhanced feature map to obtain the barrier entrance vehicle appearance enhanced feature vector.

[0039] It should be understood that although the edge information extracted from the Canny edge detection image provides the basic features of the vehicle's contour and structure, this edge information may still contain a lot of irrelevant or minor details. The feature encoder based on the spatial attention mechanism can automatically identify and enhance the important feature areas in the image, focusing on key areas such as the body contour and license plate position, and reducing interference from background noise. This feature enhancement helps to improve the accuracy of vehicle recognition and positioning. Among them, the spatial attention mechanism weights the key information by dynamically adjusting the weights of each area of ​​the image, thereby generating a more accurate and rich feature representation. This mechanism can better capture the important details in the image, so that the feature map can more clearly reflect the actual appearance characteristics of the vehicle, improve the expressiveness of the feature map, and support more complex recognition and classification tasks. Specifically, the convolution encoding part of the vehicle appearance enhancement Canny edge detection feature encoder for the barrier entrance based on the spatial attention mechanism is used to perform deep convolution encoding on the vehicle appearance enhancement Canny edge detection map at the barrier entrance to obtain an initial convolution feature map; the initial convolution feature map is input into the spatial attention part of the vehicle appearance enhancement Canny edge detection feature encoder for the barrier entrance based on the spatial attention mechanism to obtain a spatial attention map; the spatial attention map is passed through a Softmax activation function to obtain a spatial attention feature map; and the positional point multiplication of the spatial attention feature map and the initial convolution feature map is calculated to obtain the vehicle appearance enhancement feature map at the barrier entrance. More specifically, the initial convolutional feature map is input into the spatial attention part of the barrier entrance vehicle appearance enhancement Canny edge detection feature encoder based on the spatial attention mechanism to obtain a spatial attention map, including: performing average pooling and maximum pooling along the channel dimension on the initial convolutional feature map to obtain an average feature matrix and a maximum feature matrix; cascading and channel-adjusting the average feature matrix and the maximum feature matrix to obtain a channel feature matrix; and using the convolution layer of the spatial attention feature map to convolutionally encode the channel feature matrix to obtain a spatial attention map.

[0040] Furthermore, feature maps usually contain high-dimensional data, representing feature information at multiple locations in the image. Pooling these feature maps can compress their spatial dimensions and reduce the dimensionality of the data. The pooling operation divides the image into small blocks and applies a summary operation (such as the maximum or average) to each small block, thereby converting a larger feature map into a smaller feature vector. This not only reduces the amount of computation in subsequent processing, but also alleviates storage requirements. Pooling operations can help extract and retain the main information in the feature map. The Max Pooling method selects the maximum value in each pooling area, while the Average Pooling method calculates the average value of each pooling area. These operations can extract the most significant features from local areas and reduce the impact caused by changes in local details, thereby removing unnecessary details while maintaining key information. By pooling high-dimensional feature maps into low-dimensional feature vectors, the computational complexity is significantly reduced. This simplification makes subsequent processing and analysis more efficient, especially when large amounts of data processing or real-time applications are required, the pooling operation can improve the processing speed and response time of the system. For example, in vehicle recognition or classification tasks, processing the pooled feature vector is faster and more efficient than processing the original feature map.

[0041] In a specific embodiment of the present application, the vehicle license plate feature extraction unit 122 at the gate entrance includes: performing grayscale processing on the vehicle image at the gate entrance collected by the visual sensor to obtain a grayscale image of the vehicle license plate at the gate entrance; passing the grayscale image of the vehicle license plate at the gate entrance through a vehicle license plate area target detection model to obtain a region of interest for the vehicle license plate at the gate entrance; passing the region of interest for the vehicle license plate at the gate entrance through a convolutional neural network-based vehicle license plate grayscale feature encoder to obtain a grayscale feature vector of the vehicle license plate at the gate entrance.

[0042] It should be understood that the original vehicle image is usually in color, containing three color channels: red, green, and blue. Each channel has a large amount of data to be processed, which makes the image processing task complex and consumes a lot of computing resources. By converting the image to a grayscale image, all color information can be compressed into a single channel of data, which greatly simplifies the complexity of the image data. Grayscale images only retain brightness information and remove color information, making subsequent image processing tasks more direct and efficient. In this way, converting the image to a grayscale image can highlight the edges and details of the license plate characters, reduce the interference of color on the recognition process, and improve the accuracy of license plate recognition. Grayscale images can show the outline and texture of the characters more clearly, which is helpful for character extraction and recognition.

[0043] Furthermore, in a vehicle image, the license plate is usually a relatively small and specific location area. The main task of the vehicle license plate area target detection model is to identify and locate the exact location of the license plate from the overall image. This process analyzes the image to find the area where the license plate is located and marks it as a region of interest (ROI). Among them, the license plate area target detection model can filter out the license plate part from the entire vehicle image, avoiding the complex process of character recognition on the entire image. By extracting the license plate area in the image, the model only needs to process this specific area, greatly reducing the amount of calculation and processing time. This targeted processing makes license plate recognition more efficient, especially when processing a large number of vehicle images, it can significantly improve the overall processing speed of the system. Specifically, the target anchor layer of the vehicle license plate area target detection model is used to slide the anchor frame B to process the grayscale image of the vehicle license plate at the gate entrance, so as to frame the vehicle license plate region of interest in the grayscale image of the vehicle license plate at the gate entrance, thereby obtaining the vehicle license plate region of interest at the gate entrance. Accordingly, in a specific example of the present application, the vehicle license plate area target detection model is Fast R-CNN, Faster R-CNN or RetinaNet. More specifically, the vehicle license plate area target detection model is used to process the grayscale image of the vehicle license plate at the gate entrance using the following target detection formula to obtain the vehicle license plate region of interest at the gate entrance;

[0044] Among them, the target detection formula is:

[0045] ROI=(cls(ψ det ,B),Rear(ψ det ,B)

[0046] Among them, ψ det is the grayscale image of the vehicle license plate at the gate entrance, B is the anchor frame, ROI is the region of interest of the vehicle license plate at the gate entrance, cls(ψ det ,B) indicates classification, Regr(ψ det ,B) indicates regression.

[0047] Furthermore, the image of the license plate area is processed by a convolutional neural network, and high-level feature information can be automatically extracted from it. Through multiple convolutional layers, pooling layers, and activation layers, the convolutional neural network can capture important features in the image, such as edges, textures, and shapes, which are crucial for the recognition of license plate characters. Through these network layers, CNN can extract effective feature representations from complex image data to generate a grayscale feature vector of the license plate, which can highly summarize the core information of the license plate. Among them, the convolutional neural network can convert the two-dimensional image data of the license plate area of ​​interest into a high-dimensional feature vector. This feature vector is a condensed expression of the characteristics of the license plate area and has a digital representation of the key information in the image. The length and dimension of the feature vector can vary depending on the design of the network structure. It compresses the information of the license plate area into a vector of fixed length for subsequent processing and comparison. This method not only reduces data redundancy, but also improves computational efficiency. Specifically, each layer of the vehicle license plate grayscale feature of interest encoder at the barrier entrance based on the convolutional neural network performs convolution processing, mean pooling processing based on the local feature matrix, and nonlinear activation processing on the input data in the forward pass of the layer, so that the last layer of the vehicle license plate grayscale feature of interest encoder at the barrier entrance based on the convolutional neural network outputs the vehicle license plate grayscale feature vector at the barrier entrance, wherein the input of the vehicle license plate grayscale feature of interest encoder at the barrier entrance based on the convolutional neural network is the vehicle license plate region of interest at the barrier entrance.

[0048] In the above-mentioned automatic barrier control system 100 based on computer vision, the authorized vehicle judgment module 130 is used to judge whether the vehicle at the barrier entrance is an authorized vehicle based on the appearance enhancement feature vector of the vehicle at the barrier entrance and the grayscale feature vector of the license plate of the vehicle at the barrier entrance. It should be understood that the vehicle appearance enhancement feature vector and the license plate grayscale feature vector provide two important dimensions of vehicle identity. The appearance enhancement feature vector helps the system identify and distinguish different vehicles by processing visual features such as the shape, color, and size of the vehicle. The license plate grayscale feature vector focuses on the specific information of the license plate characters, which is crucial for verifying the uniqueness and legitimacy of the vehicle. By combining these two features, the system can more comprehensively identify the vehicle and reduce the possibility of misidentification.

[0049] Figure 4 FIG. 1 is a block diagram of an authorized vehicle determination module in a computer vision-based automatic gate control system according to an embodiment of the present application. Figure 4As shown, in a specific embodiment of the present application, the authorized vehicle judgment module 130 includes: a gate entrance vehicle feature fusion unit 131, which is used to perform back propagation error correction based on synchronous micro-dimension on the gate entrance vehicle appearance enhancement feature vector and the gate entrance vehicle license plate grayscale feature vector to obtain an authorized vehicle judgment classification feature vector; a gate entrance vehicle authorization judgment classification unit 132, which is used to pass the authorized vehicle judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether the gate entrance vehicle is an authorized vehicle.

[0050] It should be understood that by integrating the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance, the vehicle identification and authorization judgment process can be intelligent. This fusion method can simplify subsequent data processing, reduce manual intervention, and improve management efficiency. The system can quickly generate the authorized vehicle judgment classification feature vector, automatically perform vehicle identity verification, and improve the efficiency and accuracy of the overall operation.

[0051] In particular, in the technical solution of the present application, the vehicle appearance image captures the overall morphology and texture information of the vehicle through pixel enhancement, edge detection and feature encoding. These features are usually based on the spatial structure and edge information of the image, and have a strong descriptive power for the appearance and state of the vehicle. The license plate image, on the other hand, is subjected to grayscale processing, target detection and feature encoding, focusing on the regional features of the license plate and the information of the license plate number. These features mainly focus on the accuracy of details, such as the edges and shapes of the license plate characters. The details and information dimensions of the two features are different during the processing process, which leads to inconsistencies in their dimensional and fine-grained expressions in the feature space, resulting in feature fine-grained dimension offset. Furthermore, due to the differences in feature dimensions and representation details between the vehicle appearance enhancement feature vector and the license plate grayscale feature vector, local structural information (such as the precise shape of the license plate characters and the overall texture of the vehicle appearance) may be diluted or lost in a simple fusion process. Specifically, the global features of the vehicle appearance may not be able to effectively supplement the license plate details, and the license plate details are also difficult to enhance the overall recognition effect of the vehicle. This information mismatch and structural destruction may result in the final feature vector being unable to accurately reflect all the features of the vehicle, affecting the judgment effect of the classifier. Therefore, in the technical solution of the present application, the authorized vehicle judgment classification feature vector is obtained by performing back propagation error correction based on the synchronous microscopic dimension on the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance.

[0052] Among them, the appearance enhancement feature vector of the vehicle at the gate entrance and the grayscale feature vector of the vehicle license plate at the gate entrance are subjected to back propagation error correction based on the synchronous microscopic dimension to obtain the authorized vehicle judgment classification feature vector, including: calculating the position difference between the appearance enhancement feature vector of the vehicle at the gate entrance and the grayscale feature vector of the vehicle license plate at the gate entrance to obtain the back propagation error correction difference feature vector; calculating the position point multiplication between the appearance enhancement feature vector of the vehicle at the gate entrance and the grayscale feature vector of the vehicle license plate at the gate entrance to obtain the back propagation error correction position point multiplication feature vector; calculating the position addition between the appearance enhancement feature vector of the vehicle at the gate entrance and the grayscale feature vector of the vehicle license plate at the gate entrance to obtain the back propagation error correction position addition feature vector; and adding the back propagation error correction difference feature vector and the back propagation error correction position point multiplication feature vector. The vector and the back propagation error correction position-added feature vector are cascaded and input into a one-dimensional convolution layer and maximum pooling is performed to obtain a back propagation error correction convolution coding pooling feature vector; the square root of the two norm of the back propagation error correction convolution coding pooling feature vector is weighted and added with the one norm of the back propagation error correction convolution coding pooling feature vector to obtain a back propagation error correction factor; the back propagation error correction position-based dot product feature vector and the square root of the length of the vehicle appearance enhancement feature vector at the gate entrance are added positionally to obtain a back propagation error correction intermediate vector; the back propagation error correction factor is used as the first weighting coefficient and the second weighted hyperparameter is used as the second weighting coefficient to calculate the position-weighted sum of the back propagation error correction position-added feature vector and the back propagation error correction intermediate vector to obtain the authorized vehicle judgment classification feature vector.

[0053] Among them, the appearance enhancement feature vector of the vehicle at the gate entrance and the gray feature vector of the vehicle license plate at the gate entrance are corrected by back propagation error based on the synchronous microscopic dimension to obtain the authorized vehicle judgment classification feature vector, including:

[0054]

[0055]

[0056] Among them, V1 represents the appearance enhancement feature vector of the vehicle at the gate entrance, v2 represents the grayscale feature vector of the vehicle license plate at the gate entrance, represents positional subtraction, ⊙ represents positional multiplication, represents positional addition, conv1D represents a one-dimensional convolutional layer, MaxPool represents maximum pooling, V μrepresents the back-propagation error-corrected convolutional coding pooling feature vector, ||·||1 represents the first norm, ||·||2 represents the second norm, [·] represents the cascade, α represents the first weighted hyperparameter, τ represents the back-propagation error correction factor, L represents the length of the vehicle appearance enhancement feature vector at the gate entrance, β represents the second weighted hyperparameter, V f Represents the authorized vehicle judgment classification feature vector.

[0057] In the technical solution of the present application, in the process of fusing the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance, the feature fine-grained dimension offset between the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance is taken into account, which may cause the fused authorized vehicle judgment classification feature vector to have local structural collapse. Based on this, in the technical solution of the present application, the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance are subjected to back propagation error correction based on synchronized microscopic dimensions.

[0058] Specifically, based on the restricted expression of the foreground-background structure distinction of the high-dimensional manifold of the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector by the amplitude of the semantic difference vector between the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance, the key feature fine-grained correspondence between the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance is modeled, and then the active fine-grained correspondence adjustment propagation is performed through the back-propagation displacement compensation information between the corresponding eigenvalues ​​of the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance, so as to avoid the imbalance at the feature vector level of the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance in a focusing manner, thereby synchronizing the microscopic dimensions of the two feature vectors, ensuring that in the fusion process, the information between the feature vectors can be effectively aligned and combined, thereby avoiding the abnormal accumulation of local structures.

[0059] Furthermore, the authorized vehicle judgment classification feature vector is input into the classifier, which can comprehensively consider multiple feature dimensions (such as vehicle appearance and license plate information) and analyze the combination of these features to determine the authorization status of the vehicle. The classifier uses complex algorithms to process these features, which can capture complex patterns and relationships, improve the comprehensiveness of judgment, thereby improving the efficiency and accuracy of vehicle management at the gate entrance and ensuring the stability and adaptability of the management system.

[0060] In another embodiment of the present application, a smart control panel is provided. Specifically, the tidal light function in the smart control panel includes a tidal two-way indicator light, which can be self-controlled and automatically adjust the light color according to the license plate recognition status. That is, when the license plate is recognized / entering or exiting the venue, the light is green and the reverse indicator light is controlled to be red; when there is no recognized license plate, the light returns to green. More specifically, the smart control panel has an anti-following function. When the barrier gate is not closed, the anti-following function automatically closes the gate when it times out (the default is 0 seconds without control, and the software configuration is such as 15 seconds), and intelligently judges the following vehicle situation, automatically memorizes the number of gate openings, and ensures that the vehicle can pass normally, thereby effectively solving various following vehicle problems. Among them, when the smart control panel receives a remote control gate opening signal, it will execute the gate pole and push the gate opening record to the cloud background and link the IMS warning. When the queue mode signal is received, the barrier pole is supported to open immediately and remain in the normally open state. In addition, the smart control panel can also monitor the barrier signal, update the current barrier pole open or closed state in real time, and support real-time temperature detection and reporting. Among them, the intelligent control panel supports special demand functions. When encountering serious faults such as sudden power failure of the equipment, the equipment can automatically open the gate bar. Furthermore, the intelligent control panel can remotely solve the power failure and restart of the equipment when the on-site identification machine and radar equipment have abnormal conditions. Furthermore, after the vehicle entry / exit recognition bar is raised, if the license plate does not enter or leave the parking lot, the intelligent control panel supports automatic processing of invalid orders. It also supports timed detection of the gate bar status, and supports voice reminders and automatic drop of the bar when it is timed out.

[0061] In summary, the embodiment of the present application first obtains the vehicle image at the gate entrance collected by the visual sensor, then uses deep learning technology to perform feature extraction and association analysis, and finally obtains the classification result through the classifier to determine whether the vehicle at the gate entrance is an authorized vehicle, thereby sending a corresponding control signal to the gate, thereby reducing the need for manual operation and inspection, improving traffic efficiency, avoiding congestion, and enhancing user experience and satisfaction.

[0062] As described above, the automatic gate control system 100 based on computer vision according to the embodiment of the present application can be implemented in various terminal devices. In one example, the automatic gate control system 100 based on computer vision can be integrated into the terminal device as a software module and / or a hardware module. For example, the automatic gate control system 100 based on computer vision can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the automatic gate control system 100 based on computer vision can also be one of the many hardware modules of the terminal device.

[0063] Alternatively, in another example, the computer vision-based barrier gate automatic control system 100 and the terminal device may also be separate devices, and the computer vision-based barrier gate automatic control system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0064] Figure 5 FIG. 1 is a flow chart of a method for automatically controlling a gate based on computer vision according to an embodiment of the present application. Figure 5 As shown, according to the computer vision-based automatic barrier gate control method of an embodiment of the present application, it includes: S110, acquiring a barrier gate entrance vehicle image captured by a visual sensor; extracting a barrier gate entrance vehicle appearance enhancement feature vector and a barrier gate entrance vehicle license plate grayscale feature vector from the barrier gate entrance vehicle image captured by the visual sensor; based on the barrier gate entrance vehicle appearance enhancement feature vector and the barrier gate entrance vehicle license plate grayscale feature vector, determining whether the barrier gate entrance vehicle is an authorized vehicle.

[0065] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned automatic gate control method based on computer vision have been referred to above. Figures 1 to 4 The description of the computer vision-based barrier gate automatic control system has been introduced in detail, and therefore, its repeated description will be omitted.

[0066] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0067] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0068] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0069] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0070] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0071] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.

Claims

1. A gate automatic control system based on computer vision, It is characterized in that include: A gate entrance image acquisition module is used to acquire a vehicle image at the gate entrance collected by a visual sensor; A gate entrance data extraction module, used to extract a gate entrance vehicle appearance enhancement feature vector and a gate entrance vehicle license plate grayscale feature vector from the gate entrance vehicle image collected by the visual sensor; The authorized vehicle judgment module is used to judge whether the vehicle at the gate entrance is an authorized vehicle based on the appearance enhancement feature vector of the vehicle at the gate entrance and the grayscale feature vector of the license plate of the vehicle at the gate entrance.

2. According to the computer vision-based automatic gate control system of claim 1, It is characterized in that The gate entrance data extraction module includes: A vehicle appearance feature extraction unit at a gate entrance is used to extract vehicle appearance features from the vehicle image at the gate entrance collected by the visual sensor to obtain an enhanced feature vector of the vehicle appearance at the gate entrance; The vehicle license plate feature extraction unit at the gate entrance is used to extract the license plate features of the vehicle image at the gate entrance collected by the visual sensor to obtain the grayscale feature vector of the vehicle license plate at the gate entrance.

3. According to the computer vision-based automatic gate control system of claim 2, It is characterized in that The gate entrance vehicle appearance feature extraction unit comprises: A gate entrance vehicle appearance feature preprocessing subunit, used for performing feature preprocessing on the gate entrance vehicle image collected by the visual sensor to obtain a gate entrance vehicle appearance enhanced Canny edge detection image; The gate entrance vehicle appearance feature encoding subunit is used to perform feature encoding on the gate entrance vehicle appearance enhancement Canny edge detection image to obtain the gate entrance vehicle appearance enhancement feature vector.

4. According to the computer vision-based automatic gate control system of claim 3, It is characterized in that The gate entrance vehicle appearance feature preprocessing subunit includes: The gate entrance vehicle image collected by the visual sensor is passed through the gate entrance vehicle appearance image pixel intensifier to obtain a gate entrance vehicle appearance pixel enhanced image; Canny edge detection is performed on the pixel-enhanced image of the vehicle appearance at the gate entrance to obtain an enhanced Canny edge detection image of the vehicle appearance at the gate entrance.

5. According to the computer vision-based automatic gate control system of claim 4, It is characterized in that The gate entrance vehicle appearance feature encoding subunit includes: The gate entrance vehicle appearance enhancement Canny edge detection image is passed through a gate entrance vehicle appearance enhancement Canny edge detection feature encoder based on a spatial attention mechanism to obtain a gate entrance vehicle appearance enhancement feature image; The vehicle appearance enhancement feature map at the gate entrance is pooled to obtain the vehicle appearance enhancement feature vector at the gate entrance.

6. According to the computer vision-based automatic gate control system of claim 5, It is characterized in that The vehicle license plate feature extraction unit at the gate entrance includes: Performing grayscale processing on the vehicle image at the gate entrance collected by the visual sensor to obtain a grayscale image of the vehicle license plate at the gate entrance; The grayscale image of the vehicle license plate at the gate entrance is passed through a vehicle license plate area target detection model to obtain a region of interest for the vehicle license plate at the gate entrance; The vehicle license plate region of interest at the gate entrance is passed through a vehicle license plate grayscale feature of interest encoder based on a convolutional neural network to obtain a vehicle license plate grayscale feature vector at the gate entrance.

7. According to the computer vision-based automatic gate control system of claim 6, It is characterized in that The authorized vehicle determination module includes: The gate entrance vehicle feature fusion unit is used to perform back propagation error correction based on the synchronous micro-dimension on the gate entrance vehicle appearance enhancement feature vector and the gate entrance vehicle license plate grayscale feature vector to obtain the authorized vehicle judgment classification feature vector; The gate entrance vehicle authorization judgment classification unit is used to pass the authorized vehicle judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to judge whether the gate entrance vehicle is an authorized vehicle.

8. According to the computer vision-based automatic gate control system of claim 7, It is characterized in that The gate entrance vehicle feature fusion unit comprises: Calculating the position difference between the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance to obtain a back-propagation error correction differential feature vector; Calculate the position point multiplication between the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance to obtain a back-propagation error correction position point multiplication feature vector; Calculate the position-based addition between the vehicle appearance enhancement feature vector at the gate entrance and the vehicle license plate grayscale feature vector at the gate entrance to obtain a back-propagation error correction position-based addition feature vector; The back-propagation error correction differential feature vector, the back-propagation error correction position point product feature vector and the back-propagation error correction position addition feature vector are cascaded and input into a one-dimensional convolution layer and maximum pooling is performed to obtain a back-propagation error correction convolutional coding pooling feature vector; The square root value of the two-norm of the back-propagation error correction convolutional coding pooling feature vector is weighted and added to the one-norm of the back-propagation error correction convolutional coding pooling feature vector to obtain a back-propagation error correction factor; The back-propagation error correction is performed by adding the position point multiplication feature vector and the square root of the length of the gate entrance vehicle appearance enhancement feature vector to obtain a back-propagation error correction intermediate vector; Using the back propagation error correction factor as the first weighting coefficient and the second weighted hyperparameter as the second weighting coefficient, the position-weighted sum of the back propagation error correction position-added feature vector and the back propagation error correction intermediate vector is calculated to obtain the authorized vehicle judgment classification feature vector.

9. A gate automatic control method based on computer vision, It is characterized in that include: Acquire vehicle images at the gate entrance collected by the visual sensor; Extracting a gate entrance vehicle appearance enhancement feature vector and a gate entrance vehicle license plate grayscale feature vector from the gate entrance vehicle image collected by the visual sensor; Based on the appearance enhancement feature vector of the vehicle at the gate entrance and the license plate grayscale feature vector of the vehicle at the gate entrance, it is determined whether the vehicle at the gate entrance is an authorized vehicle.

10. The automatic gate control method based on computer vision according to claim 9, It is characterized in that Extracting a gate entrance vehicle appearance enhancement feature vector and a gate entrance vehicle license plate grayscale feature vector from the gate entrance vehicle image collected by the visual sensor, including: Extracting vehicle appearance features from the vehicle image at the gate entrance collected by the visual sensor to obtain an enhanced feature vector of the vehicle appearance at the gate entrance; The license plate feature is extracted from the vehicle image at the gate entrance collected by the visual sensor to obtain a grayscale feature vector of the vehicle license plate at the gate entrance.

Images