A highway vehicle identification method based on an optimized monocular vision algorithm

Abstract

The application discloses a highway vehicle identification method based on an optimized monocular vision algorithm, and comprises the following steps: vehicle detection is performed on a detection area to obtain a plurality of vehicle passing data; wherein the detection area comprises a license plate identification area and a wheel counting area; one vehicle corresponds to one piece of vehicle passing data; lane toll display transaction table video and toll display serial port data are analyzed to obtain a plurality of vehicle transaction data; wherein one vehicle corresponds to one piece of vehicle transaction data; the plurality of vehicle passing data and the plurality of vehicle transaction data are compared to determine whether there is an abnormal vehicle; and the application aims to solve the problems of low vehicle identification precision and insufficient reliability caused by poor scene adaptability, missed detection of small wheel targets, repeated counting of reverse vehicles and inability to split dense following vehicles in the prior art.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method for highway vehicle recognition based on an optimized monocular vision algorithm. Background Technology

[0002] With the rapid development of intelligent transportation, automatic vehicle identification on highways has become a core component of intelligent transportation systems. Currently, visual recognition solutions based on monocular cameras are widely used in highway vehicle detection and identification due to their low cost and flexible deployment. However, existing recognition solutions based on general-purpose visual algorithms (such as YOLO and DeepSORT) still have the following significant technical problems when facing actual highway operation scenarios: First, highways are entirely outdoor scenes with drastic changes in lighting conditions. The alternation of day and night leads to low illumination, direct headlights from vehicles and road reflections cause localized overexposure, and rain, snow, and fog cause image blurring and reduced contrast. Traditional image preprocessing algorithms use fixed global parameters and cannot dynamically adjust according to local scenes, resulting in enhanced images that either lose details in dark areas or are overexposed in highlight areas, directly affecting the accuracy of subsequent license plate localization and wheel detection. Second, monocular cameras are installed on the side of the lane, and distant wheels only occupy a dozen or so pixels in the image, resulting in extremely limited features. Traditional DeepSORT tracking algorithms mainly rely on single appearance features for target association, which is insufficient when dealing with small vehicles. When targeting wheels, appearance feature matching is prone to failure. Furthermore, in highway scenarios, the high speed of wheel movement and drastic changes in lighting make it difficult for traditional algorithms to maintain a stable tracking ID, frequently resulting in target drift and ID jumps. Additionally, when vehicles are extremely close together (less than 1 meter), occlusion of the following vehicle by the preceding vehicle and the phenomenon of vehicles behind each other further exacerbate wheel detection errors and false positives. Thirdly, in highway toll lanes, frequent starts, stops, and reversing to adjust position are common phenomena; existing recognition schemes use a unidirectional incremental counting method, which cannot handle the problem of repeated counting caused by reversing. Moreover, in densely following scenarios, multiple vehicles are often intertwined in the image, making it difficult for existing algorithms to accurately separate vehicles, leading to data confusion for individual vehicles. Clearly, existing highway vehicle recognition solutions still have significant shortcomings in terms of adaptability to complex scenarios, small target detection and tracking, vehicle behavior recognition, and data collection security. There is an urgent need for a recognition method that can overcome these problems. Summary of the Invention

[0003] This invention aims to provide a highway vehicle recognition method based on an optimized monocular vision algorithm, in order to solve the problems of low vehicle recognition accuracy and insufficient reliability in the existing technology due to poor scene adaptability, missed detection of small targets such as wheels, repeated counting of reversing vehicles, and inability to separate dense following vehicles.

[0004] To achieve the above objectives, this invention provides a highway vehicle recognition method based on an optimized monocular vision algorithm, comprising the following steps: Step S1: Detect vehicles in the detection area and obtain multiple vehicle passage data; wherein, the detection area includes a license plate recognition area and a wheel counting area; one vehicle corresponds to one vehicle passage data; the vehicle passage data includes license plate number, vehicle color, wheel count, vehicle type, and passenger capacity; Step S2: Analyze the desktop video and serial port data of the lane toll display to obtain multiple vehicle transaction data; one vehicle corresponds to one vehicle transaction data; the vehicle transaction data includes the vehicle transaction time, vehicle type, license plate number, amount, entrance toll station, and vehicle status of each vehicle; Step S3: Compare the multiple vehicle passage data and the multiple vehicle transaction data to determine whether there are any abnormal vehicles.

[0005] According to specific embodiments provided by the present invention, the present invention achieves the following technical effects: This invention overcomes the complex lighting interference on highways through a preprocessing algorithm that employs adaptive contrast stretching and highlight suppression, ensuring image quality in subsequent recognition stages. Based on gradient frame motion state determination, first-round reference coordinate verification, and ID error correction mechanisms, coupled with bidirectional dynamic counting logic, it solves problems in traditional solutions such as missed detection of small targets like wheels, repeated counting during reversing, and ID jumps, accurately obtaining the number of vehicle axles. By integrating the cumulative value of license plate visual verification and the cumulative value of wheel counts into a dual-threshold determination mechanism, it achieves vehicle segmentation in complex, interconnected scenarios such as vehicles without license plates followed by vehicles with plates or long-distance tailgating. Dual-source data acquisition using video OCR and serial port parsing cross-verification ensures the integrity of transaction information. The introduction of a matching cost function based on edit distance enables fault-tolerant comparison of license plate numbers, reducing false alarm rates. Furthermore, based on sliding window matching state verification, it can automatically output various abnormal labels such as vehicles without license plates, vehicles attempting to cross lanes, and vehicles swiping cards across lanes, thereby efficiently and accurately identifying vehicles. Attached Figure Description

[0006] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same parts.

[0007] Figure 1 This is a flowchart illustrating a highway vehicle recognition method based on an optimized monocular vision algorithm provided in this embodiment. Figure 2 This is a schematic diagram illustrating the process of acquiring multiple vehicle passage data in this embodiment; Figure 3This is a schematic diagram illustrating the process of obtaining multiple vehicle transaction data in this embodiment; Figure 4 This is a schematic diagram of the data comparison process in this embodiment. Detailed Implementation

[0008] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0009] Example 1: Please refer to Figure 1 A highway vehicle recognition method based on an optimized monocular vision algorithm is proposed and applied to the highway lane vehicle traffic recognition scenario. It solves the technical problems of traditional recognition methods such as poor adaptability in complex high-speed scenarios, easy omission of small targets such as wheels, repeated recognition due to vehicle reversing, and inability to separate densely following vehicles through image recognition algorithms.

[0010] The method in this embodiment includes the following steps: Step S1: Detect vehicles in the detection area and obtain multiple vehicle passage data; the detection area includes a license plate recognition area and a wheel counting area; one vehicle corresponds to one vehicle passage data; the vehicle passage data includes license plate number, vehicle color, wheel count, vehicle type, and passenger capacity. The wheel count represents the number of valid wheels identified by the system during the identification process of a single vehicle. This wheel count is used to determine the number of axles, thus assisting in vehicle type classification. Specifically, for a standard vehicle, each axle corresponds to two wheels, so the number of axles is equal to the wheel count divided by 2 and rounded up. Based on the axle count information, combined with the vehicle type identification results, it can be further determined whether the vehicle exhibits abnormal behavior such as "large vehicle with small label". For example, when the wheel count is 4, the number of axles is 2, corresponding to a regular passenger car or light truck; when the wheel count is 6, the number of axles is 3, corresponding to a medium-sized truck or bus; when the wheel count is greater than or equal to 8, the number of axles is 4 or more, corresponding to a heavy truck or trailer.

[0011] Please see Figure 2 The steps to obtain multiple vehicle passage data include: Original lane images are captured using a monocular camera. Adaptive image preprocessing is then performed on the original images to generate enhanced images. Adaptive image preprocessing includes contrast stretching and highlight suppression based on the grayscale distribution of local image regions. Based on enhanced images, vehicles are identified and tracked using monocular vision algorithms to obtain vehicle passing data; the identification and tracking includes license plate visual recognition, wheel status tracking, and vehicle body information visual extraction.

[0012] It should be noted that the license plate recognition area and the wheel counting area are obtained by calibrating the enhanced image through the software system; the monocular camera is installed on the side of the lane, and the camera's field of view is adjusted to cover the entire detection area by adjusting the installation angle; the license plate recognition area is located at the front of the detection area and is used to capture the image of the license plate at the front of the vehicle; the wheel counting area is located in the middle and rear of the detection area and is used to capture the image of the wheels on the side of the vehicle; the two areas partially overlap in spatial position to ensure that the license plate information and wheel information of the same vehicle can be correctly associated.

[0013] The steps of contrast stretching and highlight suppression based on the gray-level distribution of local image regions include: Each pixel in the original lane image Construct a local window centered on the element and calculate the average gray value of the local window. and standard deviation ; According to standard deviation Adjust contrast gain factor Including: when Below the preset low light threshold At that time, increase To enhance details in shadow areas; Set high brightness threshold and specular suppression coefficient For grayscale values Higher than For each pixel, calculate its excess portion. Multiply the excess by The highlight suppression amount is obtained, and then subtracted from the enhanced grayscale value of the pixel to obtain the final enhanced grayscale value. as follows:

[0014] in, To prevent division by zero constant; when hour, Take the maximum value Otherwise, take the smaller value. ;when At that time, the highlight suppression term Activated.

[0015] It should be noted that due to the complex outdoor imaging environment of highways, there are interference factors such as day and night cycles, direct headlights, road surface reflections, and rain, snow, and fog. The original images often appear too dark (at night, on cloudy days, at tunnel entrances) or overexposed (direct sunlight, road surface reflections), which directly affects the accuracy of subsequent license plate localization and wheel detection. Therefore, this embodiment uses the above-mentioned adaptive preprocessing algorithm to enhance the quality of each frame of the original image before inputting it into the subsequent recognition process, thus ensuring image quality from the source. So-called contrast gain factor This parameter represents the intensity of image contrast stretching; the larger the value, the more significant the local contrast enhancement effect. The so-called high-gloss threshold... This represents the threshold value for determining whether a pixel belongs to an overexposed area, typically set between 200-220 (0-255 for 8-bit grayscale images). Pixels with values ​​greater than this are considered potentially affected by strong light; this is known as the highlight suppression coefficient. This represents the intensity of suppression of overexposed areas, and its value typically ranges from 0.3 to 0.7. In the formula, the first term Preserve the original image information, the second item Achieve local contrast stretching, i.e., when the local standard deviation is... A smaller value indicates low contrast in that area, requiring greater stretching to reveal details; the third item Suppression is applied to the highlight areas to eliminate overexposed white spots; the combination of these three methods ensures that the final enhanced image retains the original information, reveals details in the dark areas, and suppresses overexposure in the bright areas, providing high-quality image input for subsequent recognition. In a preferred embodiment The value is 15. The value is 2.0. The value is 0.5. The value is 210. The value is 0.5.

[0016] The steps involved in license plate visual recognition include: Based on the enhanced image, the YOLOv5-face algorithm is used to locate the license plate position within the license plate recognition area and capture a close-up image of the license plate. A CRNN model is used to perform character and color recognition on close-up images of license plates, and the license plate number and vehicle color are output. The similarity of the recognition results of the same vehicle in multiple consecutive frames is checked. The current recognition result is output as valid license plate data only when the similarity of the license plate numbers in at least two consecutive frames is greater than or equal to the preset similarity threshold.

[0017] It should be noted that the YOLOv5-face algorithm is a lightweight object detection network. By pruning and optimizing the structure and parameters of the YOLOv5 network, it can run in real time on embedded devices and is specifically designed for detecting faces or face-like targets in images. In this embodiment, it is applied to the license plate detection task, utilizing its efficient detection capabilities to achieve rapid license plate localization. The CRNN model is an end-to-end sequence recognition network composed of convolutional layers (CNN), recurrent layers (RNN), and transcription layers (CTC), capable of recognizing character sequences of variable length. The so-called similarity check refers to calculating the Levenshtein edit distance between each pair of license plate strings identified in multiple consecutive frames for the same vehicle and normalizing it to a similarity value. Only when the recognition results of at least two consecutive frames have high similarity is the license plate number output as a valid result, in order to filter out accidental recognition errors in a single frame. In a preferred embodiment, a license plate recognition result cache queue is maintained for each vehicle entering the detection area. This queue stores the license plate recognition results and their confidence scores for the most recent N frames (e.g., N=10). Whenever a license plate number is recognized from the current frame, the system calculates the similarity between the license plate number and the historical license plate number with the highest confidence score in the cache queue. If the similarity score is greater than or equal to 0.8, the license plate similarity count is incremented by 1; otherwise, the license plate DF check count is incremented by 1. When the license plate similarity count reaches 3, the current license plate recognition is deemed successful, the image with the highest confidence score is confirmed as the target image, and the license plate number is output. When the license plate DF check count reaches 6, it indicates that the license plate area has changed significantly (possibly due to obstruction by a following vehicle or the departure of the preceding vehicle). The system then ends the recognition of the preceding vehicle and begins to recognize the following vehicle, thereby achieving accurate segmentation of license plates of different vehicles in dense following scenarios.

[0018] Furthermore, wheel state tracking is based on gradient frame determination, including: Get the same wheel in the wheel counting region in the current frame Previous frame and previous frame The coordinates of the bottom center of the frame are denoted as follows: ,in This is the preset frame interval number; Define motion state discrimination factor for:

[0019] in, For symbolic functions; according to With the preset dynamic displacement threshold The comparison results determine the motion state of the wheels. :

[0020] in, =-1 represents the reverse position. =0 represents a static state. =1 represents the forward state; dynamic displacement threshold. It adaptively adjusts based on the pixel height of the wheel in the image.

[0021] It should be noted that the so-called bottom edge center coordinates refer to the pixel coordinates of the midpoint of the bottom edge of the wheel detection bounding box. This point approximately corresponds to the contact point between the wheel and the ground, and its horizontal displacement during vehicle movement best reflects the actual motion state of the wheel. The motion state discrimination factor reflects the consistency of the wheel's motion trend within the first L frames. Indicates the direction of motion of the current frame relative to the previous frame. This indicates the direction of motion of the previous frame relative to the Lth frame before it. Multiplying the two can detect whether the directions of motion are consistent. The smaller of the two displacement values ​​is used as the measure of motion intensity to avoid misjudgment caused by single-frame displacement anomalies. Dynamic displacement threshold ,in The pixel height of the current wheel target in the image. The scaling factor ranges from 0.1 to 0.3. Since the size of the wheel in the image varies at different distances, an adaptive threshold ensures that the algorithm has consistent sensitivity in judging wheels at different distances. In a preferred embodiment, The value is 0.15, and L is 8; The gradient frame determination method in this embodiment obtains accurate wheel motion states (forward, stationary, and reverse), which can effectively distinguish various driving behaviors such as normal driving, temporary parking, and reversing, providing a state basis for subsequent bidirectional dynamic counting.

[0022] Wheel status tracking also includes the steps of first-round coordinate reference verification and ID error correction: ID error correction refers to the system's forced correction of tracking results when the identity (ID) of the same wheel target in a multi-target tracking algorithm changes incorrectly. This correction uses reference coordinate information to re-associate the trajectory segments before and after the change as a continuous trajectory of the same target. In complex scenarios such as close following and vehicle occlusion, the conventional DeepSORT algorithm may swap the wheel IDs of the two vehicles in front and behind, leading to a deviation in the wheel count of a single vehicle. This embodiment uses the locking and forced association mechanism of the first wheel's reference coordinates to suppress the counting error caused by ID changes.

[0023] Record the coordinates of the center of the bottom edge of the first wheel to enter the wheel counting area. This serves as a reference coordinate, and the width of the reference wheel is also recorded. and height ; For any subsequent tracked wheel, determine the coordinates of its bottom edge center. Define the attribution determination function :

[0024] in, and These are the preset horizontal and vertical scale thresholds, respectively. The overall direction of motion of the vehicle at present, when moving forward =+1, when going backward =-1, This represents the logical AND operation; Instantaneous state independent of individual wheels Specifically, the calculation starts from the location of the first wheel's reference coordinates and continuously tracks the position change of that first wheel across multiple consecutive frames. When the cumulative displacement exceeds a preset threshold, the position is determined based on the displacement direction. The value of ; when When true, the wheel is determined to belong to the current vehicle; when a change in the tracking ID is detected, the reference coordinates are used to determine the wheel's position. Force association between wheel trajectory segments before and after the ID transition to correct counting errors caused by ID transitions.

[0025] It should be noted that the attribution determination function By employing a triple spatial constraint to progressively filter through different levels, it determines whether a tracked wheel belongs to the current vehicle, thereby correctly associating wheels of the same vehicle and eliminating interfering targets. The formula consists of three logical conditions, all of which must be satisfied simultaneously: 1. Lateral position constraint, which is the horizontal distance between the new wheel and the reference wheel. Divide by the reference wheel width This yields a normalized offset independent of the imaging scale, which is required to be less than a preset threshold. The physical meaning is that all wheels of the same vehicle should be concentrated within a limited lateral range under the chassis, and cannot deviate indefinitely.

[0026] 2. Vertical position constraint, similar to horizontal position constraint, is the vertical distance... Divide by the reference wheel height It is required that it is less than the threshold. The first two constraints together define a rectangular region of ownership centered on the reference wheel in the image.

[0027] 3. Directional consistency constraint, which requires that the new wheel appear in front of the current overall direction of vehicle movement, rather than behind. This constraint ensures that the vehicle can track correctly when reversing, while eliminating interference from wheels that have already left the vehicle behind.

[0028] The three conditions form a layer-by-layer filtering mechanism; if any condition is not met, the candidate is excluded, ensuring high reliability of the attribution determination; ultimately... The output is a true / false judgment, which serves as a prerequisite for wheel counting in bidirectional dynamic counting and as the basis for associating trajectory segments in ID error correction.

[0029] So-called based on reference coordinates Forced association of wheel trajectory segments before and after the jump means that when the system detects a jump in the tracking ID of a wheel target (e.g., ID jumps from #5 to #8), it does not directly regard it as a new target, but calculates the positional deviation of the two trajectory segments before and after the jump in the reference coordinate space; if the deviation is less than the preset association threshold, it is considered that the two are actually the same wheel target, and they are merged into the same trajectory, with the first wheel reference coordinate as the spatial anchor point of the trajectory; This embodiment solves the problem of multiple counts of the same vehicle caused by ID jumps in multi-target tracking by verifying the coordinate reference in the first round and correcting the ID. It realizes the accurate accumulation of wheel counts for a single vehicle and provides a data foundation for subsequent axle count calculation.

[0030] Wheel state tracking also includes a bidirectional dynamic counting step: Based on the determined wheel motion state Update the current vehicle's wheel count according to the following rules. :

[0031] Among the increments Determined by the following formula:

[0032] in, This is the lifecycle value for which the current wheel is continuously tracked, i.e., the number of frames since the wheel was first detected in the current frame; This is the preset minimum stable tracking frame threshold.

[0033] It should be noted that the increment This reflects the change in wheel count determined by the wheel's motion state: a positive increase (moving forward and passing the attribution check), a negative rollback (reversing and not yet stable), or no change; the current vehicle's cumulative wheel count. This reflects the total number of valid wheels belonging to the vehicle that the system has confirmed from the moment the vehicle enters the recognition area until the current frame. This count increases when the vehicle is moving forward and decreases when the vehicle is reversing and the wheels are unstable, thus ensuring that the wheel count always reflects the actual number of axles of the vehicle at the current moment. Compared with the traditional one-way incremental counting, this two-way dynamic counting method can effectively cope with actual scenarios such as frequent starts and stops and reversing to adjust the position of the vehicle in the toll lane, and avoid duplicate counting caused by reversing.

[0034] In a preferred embodiment, the method further includes a cumulative value of valid license plate data that has undergone similarity verification and a cumulative wheel count. The dual threshold approach employs scenario-based vehicle segmentation logic to specifically address the challenges of vehicle segmentation in three complex, interconnected scenarios: vehicles without license plates in front and vehicles with license plates behind, dense following, and long-distance tailgating. It achieves accurate differentiation of multiple vehicles through visual feature differences, eliminating vehicle data confusion.

[0035] Specifically, it simultaneously monitors the cumulative value of visual verification of license plates. And wheel count cumulative value When the license plate verification passes a certain number of times Reaching a preset first threshold (e.g., 3 times) and the lifecycle of all tracking wheels. minimum value Greater than the preset maximum lifespan threshold When all identification information of the current vehicle has been collected, the system triggers vehicle segmentation and data output; or, when the license plate confidence level between adjacent frames jumps and the jump exceeds a preset threshold, and a vehicle without a license plate is detected entering the license plate recognition area, it indicates that the current vehicle may be a vehicle without a license plate and the following vehicle may be a vehicle with a license plate. The system immediately ends the recognition of the current vehicle and initializes the next vehicle. Through the above dual threshold determination mechanism, the vehicle segmentation problem under the conditions of dense following and mixed traffic of vehicles without license plates is solved.

[0036] The steps for visually extracting vehicle body information include: Using the axle video screenshots saved during wheel state tracking, extract the side image of the vehicle body within the wheel counting area; Perform local OCR enhancement recognition on the side image of the vehicle body: automatically crop the text area on the door, perform contrast stretching and noise filtering, and then call the OCR model to recognize the text content; Based on a pre-defined traffic vehicle keyword database, the system matches text content to extract the vehicle's approved passenger capacity or approved load capacity, which is then stored as the approved passenger capacity in the vehicle passage data.

[0037] It should be noted that the so-called local OCR enhancement recognition refers to targeted OCR processing performed on a specific area on the side of the vehicle body (usually the "Approved Passenger Capacity X people" or "Total Mass XXXX kg" markings sprayed on the door or lower part of the vehicle body). Since the text on the vehicle body is often subject to interference such as oil stains, scratches, and rust, directly calling a general OCR model results in low recognition accuracy. This embodiment improves the success rate of extracting the vehicle's rated load information by performing targeted image enhancement on the recognition area (including contrast stretching to enhance the distinction between text and background, median filtering to remove salt-and-pepper noise, and bilateral filtering to preserve text edges). For example, for a bus, the words "Approved passenger capacity 47 people" are painted on the side of the door. After the system locates the text area through edge detection, it performs contrast stretching (linearly mapping the grayscale value to the full range of [0,255]) and median filtering (3×3 window), and then calls the OCR model to recognize it. Finally, the passenger capacity "47" is extracted and stored in the vehicle passage data.

[0038] Step S2: Analyze the desktop video and serial port data of the lane toll display to obtain multiple vehicle transaction data; one vehicle corresponds to one vehicle transaction data; the vehicle transaction data includes the vehicle transaction time, vehicle type, license plate number, amount, entrance toll station, and vehicle status of each vehicle; Please see Figure 3 The steps to obtain multiple vehicle transaction data include: The video of the lane toll display transaction desktop is captured by a split screen device and an encoder. The split screen device is physically isolated from the toll display device in the toll booth. Perform layout partitioning OCR recognition on each frame of the transaction desktop video: use image segmentation algorithm to automatically divide the fee display area, license plate information area and transaction field area, and perform binarization and character recognition on each partition respectively; At the same time, the serial port data of the fee display is obtained through the serial port protocol, and the transaction field is obtained by parsing the serial port data according to the protocol. The OCR recognition results are cross-validated and complementarily integrated with the serial port parsing results to form a complete vehicle transaction data entry.

[0039] It should be noted that the so-called splitter is a type of video signal distribution device, which duplicates the VGA or HDMI output signal of the toll display device into two paths: one path is transmitted to the toll display screen for normal display, and the other path is transmitted to the encoder for collection by the audit system. The two paths of signals are physically isolated from each other, so the data collection operation will not affect the normal operation of the toll collector and the stable operation of the transaction system; the so-called encoder is a device that converts analog video signals or digital video signals into network-transmittable IP code streams. In this embodiment, the encoder converts the video signal output by the splitter into an H.264 or H.265 encoded network stream for real-time analysis by the back-end OCR recognition module; the so-called toll display serial port data refers to the original data packet of transaction information sent outward by the toll display device through the RS232 or RS485 serial port protocol, which contains the original codes of fields such as license plate number, vehicle type, and amount; The so-called layout partition OCR recognition refers to dividing the screen picture of the toll display into a plurality of functional areas through image semantic segmentation technology, and then performing text recognition respectively. Specifically, the toll display screen usually has a fixed interface layout, that is, the upper left corner is the toll station name and lane number, the middle part is the license plate number and vehicle type, and the right side or lower part is the transaction amount and vehicle condition information. This embodiment uses an image segmentation algorithm (for example, layout analysis based on projection method or deep learning-based object detection) to automatically locate the boundary of each area, and then performs binarization and character recognition on each partition independently, so as to avoid mutual interference of texts between different areas. For example, when the toll display interface shows "Chuan A12345 Passenger 1 15.00 Yuan", the system divides the picture into a license plate area ("Chuan A12345"), a vehicle type area ("Passenger 1") and an amount area ("15.00 Yuan"), which are respectively recognized and then combined into a complete transaction data entry; The so-called serial port protocol is a physical layer and data link layer protocol for data communication between the toll display device and the industrial personal computer, and usually adopts MODBUS or a custom ASCII protocol; the so-called performing protocol parsing on the serial port data to obtain transaction fields refers to extracting the original values of fields such as license plate number, transaction amount, and vehicle type code from the original data packet according to the predefined field offset and length in accordance with the protocol specification, and converting them into a readable text format; The process of the so-called cross-check and complementary fusion includes: when both the OCR recognition result and the serial port parsing result exist, the two are compared at the field level, that is, if the license plate number and the amount are completely consistent, they are directly merged into a piece of high-confidence transaction data; if there is an inconsistency, the field value with higher confidence is selected according to the confidence score (for example, the character recognition confidence of OCR and the correctness of the check code of the serial port data), or it is marked as requiring manual review; when data exists in only one party (for example, the serial port data is missing due to a hardware failure), the OCR recognition result is directly used as a supplement. Through the above mechanism, redundant backup and complementary enhancement of the two data sources are realized, ensuring the integrity and reliability of transaction data collection.

[0040] Step S3: Compare multiple vehicle passage data and multiple vehicle transaction data to determine if there are any abnormal vehicles.

[0041] Please refer to Figure 4 The steps for comparing multiple vehicle passage records and multiple vehicle transaction records include: The vehicle passage data sequence is denoted as... The vehicle transaction data sequence is denoted as Calculate each pair of data Matching cost :

[0042] in, Edit the distance function for Levenshtein. and These are license plate number strings from vehicle pass data and transaction data, respectively. and Indicates the length of the string; and For vehicle passage timestamp and transaction timestamp; This is the preset maximum allowable time difference; For indicator functions, and Vehicle model code; , , As a weighting factor, and ; Globally, iterate through and compute each pair. Matching cost When the matching cost Less than the preset matching threshold At that time, Mark as a confirmed match pair; All confirmed match pairs are arranged in chronological order. Adjacent confirmed match pairs are used as the first and last anchor points of a sliding window. The sequence of all vehicle passage data between these first and last anchor points is then processed. Vehicle transaction data sequence Add them to the same window to be matched; The system checks the matching status of the data within the window. If there is data within the window that was not matched during the global matching phase, the system outputs the corresponding anomaly identification results based on the data source and field characteristics. The vehicle recognition result at least includes: the number of vehicle axles based on wheel counting, the license plate number and color based on license plate recognition, the permitted number of passengers extracted based on vehicle body vision, and an abnormal behavior label; the abnormal behavior label is determined based on the abnormal recognition result, including an unlicensed vehicle identifier, a vehicle following to rush through the barrier identifier, a reversed license plate sequence identifier, and a cross-lane card swiping identifier.

[0043] It should be noted that the so-called Levenshtein edit distance function is a measurement method for measuring the degree of difference between two strings, and is defined as the minimum number of single-character edit operations (insertion, deletion or replacement) required to convert one string into another; for example, the edit distance between "Chuan A12345" and "Chuan A12346" is 1 (the last character is replaced), and the normalized similarity is 1-1 / max(7,7)=0.857; in this embodiment, the Levenshtein distance is used to realize fault-tolerant matching of license plate numbers, and even if there is an individual character error in OCR recognition, the corresponding transaction data can still be correctly matched. Indicator function outputs 0 or 1, which is used to judge whether the vehicle type code in the vehicle passing data is consistent with the vehicle type code in the transaction data, and when they are inconsistent, add an additional cost; in a preferred embodiment, =0.60, =0.20, =0.20; The preset matching threshold is pre-calibrated by a statistical analysis method: a large number of vehicle passing-transaction data pairs known to be normally matched are collected, and the distribution of their matching costs is calculated, and the threshold is set as the mean value of the normal matching cost plus twice the standard deviation, so that the cost of the vast majority of normally matched data is less than the threshold; in a preferred embodiment, the value is 0.45, which is obtained through statistics of 2000 groups of manually labeled matching data, wherein the mean value of the cost of normally matched data pairs is 0.28, and the standard deviation is 0.08, and 0.44 is obtained by calculating the mean value plus twice the standard deviation, which is rounded up to 0.45.

[0044] The unlicensed vehicle identifier means that when there is a record in the vehicle passing data whose license plate number is a preset default license plate number (for example, an all-zero string "000000" or an empty string), and the record fails to form a confirmed matching pair with any vehicle transaction data in the global matching stage, and at the same time all vehicle transaction data in the sliding window have formed confirmed matching pairs with other vehicle passing data, it is determined that the vehicle corresponding to is an unlicensed vehicle, and the unlicensed vehicle identifier is output. ​

[0045] The "follow-the-vehicle obstacle" indicator indicates that a record exists in the vehicle passing data. Its license plate number Normal (not the default value), but this record failed to trade with any vehicle data during the global matching phase. When a confirmed match is formed, and all vehicle transaction data within the sliding window has formed a confirmed match with other vehicle transfer data, the transaction is determined to be valid. If the corresponding vehicle has a valid passage record but no legal transaction record, a "following vehicle to break through" flag will be displayed.

[0046] The license plate order is reversed, indicating that there are two passing vehicle records within the sliding window. , With two transaction data , The total cost of cross matching is less than the total cost of sequential matching, i.e. + < + ,and < , < If the order of transaction data recording is inconsistent with the actual order of vehicle passage, a license plate order reversal indicator will be output.

[0047] Cross-lane card swipe identifier, which means that there is a record in the vehicle transaction data. Its license plate number Normal, but this record failed to match any vehicle data during the global matching phase. When a confirmed matching pair is formed, and all vehicle passage data within the sliding window has formed a confirmed matching pair with other transaction data, it is determined that the vehicle corresponding to the transaction record comes from an adjacent lane or is missed by the vision system, and a cross-lane card swipe mark is output.

[0048] The highway vehicle recognition method based on an optimized monocular vision algorithm provided in this embodiment improves image quality under complex lighting conditions through adaptive image preprocessing, accurately identifies wheel movement through gradient frame wheel state determination, ensures accurate wheel counting through first-round benchmark verification and ID error correction, adapts to special driving behaviors such as vehicle reversing through bidirectional dynamic counting, solves the vehicle segmentation problem in dense following scenarios through dual-threshold vehicle segmentation, achieves automatic extraction of vehicle load information through vehicle body OCR enhanced recognition, achieves pure visual acquisition of transaction data through layout partitioning OCR, achieves fault-tolerant comparison of vehicle passing data and transaction data through a matching cost function based on edit distance, and finally achieves automatic recognition of various abnormal vehicle behaviors through a sliding window mechanism and anomaly discrimination logic. It solves key technical problems in previous applications, such as poor adaptability of general vision algorithms to complex high-speed scenarios, easy omission of small wheel targets, duplicate counting due to vehicle reversing, and inability to segment dense following, and achieves a significant improvement in the accuracy and stability of highway vehicle recognition. The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A highway vehicle recognition method based on an optimized monocular vision algorithm, characterized in that, Includes the following steps: Step S1: Detect vehicles in the detection area and obtain multiple vehicle passage data; wherein, the detection area includes a license plate recognition area and a wheel counting area; one vehicle corresponds to one vehicle passage data; the vehicle passage data includes license plate number, vehicle color, wheel count, vehicle type, and passenger capacity; Step S2: Analyze the desktop video and serial port data of the lane toll display to obtain multiple vehicle transaction data; one vehicle corresponds to one vehicle transaction data; the vehicle transaction data includes the vehicle transaction time, vehicle type, license plate number, amount, entrance toll station, and vehicle status of each vehicle; Step S3: Compare the multiple vehicle passage data and the multiple vehicle transaction data to determine whether there are any abnormal vehicles.

2. The highway vehicle identification method according to claim 1, characterized in that, The step of obtaining multiple vehicle passage data in step S1 includes: Original lane images are acquired using a monocular camera, and adaptive image preprocessing is performed on the original images to generate enhanced images. The adaptive image preprocessing includes contrast stretching and highlight suppression based on the grayscale distribution of local areas of the image. Based on the enhanced image, vehicles are identified and tracked using a monocular vision algorithm to obtain vehicle passing data; the identification and tracking includes license plate visual recognition, wheel status tracking, and vehicle body information visual extraction.

3. The highway vehicle identification method according to claim 2, characterized in that, The steps of contrast stretching and highlight suppression based on the grayscale distribution of local image regions include: Each pixel in the original lane image Construct a local window centered on the specified area and calculate the average gray value of the local window. and standard deviation ; According to the standard deviation Adjust contrast gain factor Including: when Below the preset low light threshold At that time, increase To enhance details in shadow areas; Set high brightness threshold and specular suppression coefficient For grayscale values Higher than For each pixel, calculate its excess portion. Multiply the excess portion by The specular suppression amount is obtained, and then subtracted from the enhanced grayscale value of the pixel to obtain the final enhanced grayscale value. as follows: in, To prevent division by zero constant; when hour, Take the maximum value Otherwise, take the smaller value. ;when At that time, the highlight suppression term Activated.

4. The highway vehicle identification method according to claim 2, characterized in that, The steps of the license plate visual recognition include: Based on the enhanced image, the YOLOv5-face algorithm is used to locate the license plate within the license plate recognition area, and a close-up image of the license plate is captured. A CRNN model is used to perform character and color recognition on the close-up image of the license plate, and the license plate number and vehicle color are output. The similarity of the recognition results of the same vehicle in multiple consecutive frames is checked. The current recognition result is output as valid license plate data only when the similarity of the license plate numbers in at least two consecutive frames is greater than or equal to the preset similarity threshold.

5. The highway vehicle identification method according to claim 2, characterized in that, The wheel state tracking is based on gradient frame determination, including: Get the same wheel in the wheel counting region in the current frame Previous frame and previous frame The coordinates of the bottom center of the frame are denoted as follows: ,in This is the preset frame interval number; Define motion state discrimination factor for: in, For symbolic functions; according to With the preset dynamic displacement threshold The comparison results determine the motion state of the wheels. : in, =-1 represents the reverse position. =0 represents a static state. =1 represents the forward state; the dynamic displacement threshold It adaptively adjusts based on the pixel height of the wheel in the image.

6. The highway vehicle identification method according to claim 5, characterized in that, The wheel status tracking also includes the first-round coordinate reference verification and ID error correction steps: Record the coordinates of the center of the bottom edge of the first wheel to enter the wheel counting area. This serves as a reference coordinate, and the width of the reference wheel is also recorded. and height ; For any subsequent tracked wheel, determine the coordinates of its bottom edge center. Define the attribution determination function : in, and These are the preset horizontal and vertical scale thresholds, respectively. The overall direction of motion of the vehicle at present, when moving forward =+1, when going backward =-1, This represents the logical AND operation; when When true, the wheel is determined to belong to the current vehicle; when a change in the tracking ID is detected, the reference coordinates are used as the basis for determining the wheel's position. Force association between wheel trajectory segments before and after the ID transition to correct counting errors caused by ID transitions.

7. The highway vehicle identification method according to claim 6, characterized in that, The wheel state tracking also includes a bidirectional dynamic counting step: Based on the determined wheel motion state Update the current vehicle's wheel count according to the following rules. : Among the increments Determined by the following formula: in, This is the lifecycle value for which the current wheel is continuously tracked, i.e., the number of frames since the wheel was first detected in the current frame; This is the preset minimum stable tracking frame threshold.

8. The highway vehicle identification method according to claim 2, characterized in that, The steps for visually extracting vehicle information include: Using the axle video screenshots saved during wheel state tracking, extract the side image of the vehicle body within the wheel counting area; The vehicle side image is subjected to local OCR enhancement recognition: the text area on the door is automatically cropped, contrast stretching and noise filtering are performed, and then the OCR model is called to recognize the text content; Based on a preset traffic vehicle keyword database, the text content is matched to extract the vehicle's approved passenger capacity or approved load capacity, which is then stored as the approved passenger capacity in the vehicle passage data.

9. The highway vehicle identification method according to claim 1, characterized in that, The steps in step S2 to obtain multiple vehicle transaction data include: The lane toll display transaction desktop video is captured by a split screen device and an encoder, and the split screen device is physically isolated from the toll display device in the toll booth. For each frame of the transaction desktop video, perform layout partitioning OCR recognition: use image segmentation algorithm to automatically divide the fee display area, license plate information area and transaction field area, and perform binarization and character recognition on each partition respectively; At the same time, the serial port data of the fee display is obtained through the serial port protocol, and the transaction field is obtained by parsing the serial port data according to the protocol. The OCR recognition results are cross-validated and complementarily integrated with the serial port parsing results to form a complete vehicle transaction data entry.

10. The highway vehicle identification method according to claim 1, characterized in that, The step S3, which compares the multiple vehicle passage data and the multiple vehicle transaction data, includes: The vehicle passage data sequence is denoted as... The vehicle transaction data sequence is denoted as Calculate each pair of data Matching cost : in, Edit the distance function for Levenshtein. and These are license plate number strings from vehicle pass data and transaction data, respectively. and Indicates the length of the string; and For vehicle passage timestamp and transaction timestamp; This is the preset maximum allowable time difference; For indicator functions, and Vehicle model code; , , As a weighting factor, and ; Globally, iterate through and compute each pair. Matching cost When the matching cost Less than the preset matching threshold At that time, Mark as a confirmed match pair; All confirmed match pairs are arranged in chronological order. Adjacent confirmed match pairs are used as the first and last anchor points of a sliding window. The sequence of all vehicle passage data between these first and last anchor points is then processed. Vehicle transaction data sequence Add them to the same window to be matched; The system checks the matching status of the data within the window. If there is data within the window that was not matched during the global matching phase, the system outputs the corresponding anomaly identification results based on the data source and field characteristics. The vehicle identification results include at least: the number of vehicle axles based on wheel counting, the license plate number and color based on license plate recognition, the number of passengers allowed based on vehicle body visual extraction, and abnormal behavior tags; the abnormal behavior tags are determined based on the abnormal identification results and include no-license-plate vehicle tags, following other vehicles to break through checkpoints tags, license plate number reversed tags, and cross-lane card swiping tags.

Images