A multi-vehicle recognition method and system for a self-service toll station

Abstract

The application provides a multi-vehicle identification method and system of a self-service toll station, and the method comprises the following steps: obtaining a final vehicle area from a vehicle image to determine whether there is a following behavior; if there is, obtaining a category probability vector of each pixel point in the final vehicle area to determine a first front vehicle pixel point, a first rear vehicle pixel point, a background pixel point and a sticking pixel point; obtaining an initial front vehicle area and an initial rear vehicle area through the first front vehicle pixel point and the first rear vehicle pixel point to separate the sticking pixel point into a second front vehicle pixel point and a second rear vehicle pixel point, and obtaining a final front vehicle area through the first front vehicle pixel point and the second front vehicle pixel point; obtaining a vehicle moving track through two final front vehicle areas under adjacent time frames to determine whether a front vehicle drives away. Through the above method, when the following is close and the vehicle sensing coil cannot identify, accurate cooperative determination of whether the front vehicle drives away or not is provided, and the normal traffic of the toll station is ensured.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a multi-vehicle recognition method and system for self-service toll stations. Background Technology

[0002] Toll booths are facilities used to collect tolls from passing vehicles. They are generally located in two ways: one is directly on the main line, also known as a roadblock toll booth, which is mostly used at the beginning and end of the main line toll section; the other is on grade-separated intersections or connecting lines, which are generally used at grade-separated intersections between main line toll sections to control the entry and exit of vehicles on the intersecting roads from the main line to collect tolls.

[0003] ETC is an electronic toll collection system that does not require staff to be on duty. It uses dedicated short-range microwave communication between an on-board electronic tag installed on the vehicle's windshield and a microwave antenna in the ETC lane at the toll station, and uses computer networking technology for background settlement processing, so that vehicles can pay tolls without stopping when passing through the toll station.

[0004] Toll booths typically detect vehicle arrivals and departures based on signals from underground vehicle sensor loops. When the signal disappears, the vehicle is considered to have left. However, when multiple vehicles pass through a toll booth using ETC, if the following distance is close, it's possible for a vehicle to have already entered the loop before the preceding vehicle has left. This makes it impossible to distinguish whether a vehicle that has paid the toll has left the toll booth, affecting the normal passage of vehicles within the toll booth. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a multi-vehicle identification method and system for self-service toll stations. This system addresses the technical problem in existing technologies where, when multiple vehicles pass through a toll station via ETC, if the following distance is too close, the following vehicle may have already entered the vehicle sensor loop before the preceding vehicle has left the loop, making it impossible to distinguish whether a vehicle that has paid the toll has left the toll station, thus affecting the normal passage of vehicles within the toll station.

[0006] To achieve the above objectives, in a first aspect, embodiments of this application provide a multi-vehicle identification method for self-service toll stations, comprising the following steps:

[0007] Acquire a vehicle image, obtain a final vehicle region from the vehicle image, and determine whether following behavior exists based on the final vehicle region;

[0008] If following behavior exists, the category probability vector of each pixel in the final vehicle region is obtained. The category probability vector includes the probability vector of the preceding vehicle, the probability vector of the following vehicle, and the background probability vector. Based on the category probability vector, the first preceding vehicle pixel, the first following vehicle pixel, the background pixel, and the connected pixels are determined.

[0009] The first front vehicle pixels are combined into an initial front vehicle region, and the first rear vehicle pixels are combined into an initial rear vehicle region. Based on the initial front vehicle region and the initial rear vehicle region, the stuck pixels are separated into a second front vehicle pixel and a second rear vehicle pixel. The final front vehicle region is obtained through the first front vehicle pixel and the second front vehicle pixel.

[0010] The vehicle movement trajectory is obtained by using the two final preceding vehicle regions in adjacent time frames, and the preceding vehicle is determined to have left based on the vehicle movement trajectory.

[0011] Furthermore, the step of obtaining the final vehicle region from the vehicle image and determining whether following behavior exists based on the final vehicle region includes:

[0012] An initial vehicle region is obtained from the vehicle image, the initial vehicle region is magnified to obtain a final vehicle region, and the final vehicle region is binarized to separate the final vehicle region into a foreground region and a background region.

[0013] The foreground pixel density and background pixel density of the foreground region and the background region are obtained respectively, so as to obtain the pixel density difference based on the foreground pixel density and the background pixel density;

[0014] The pixel density difference is compared with a density threshold. If the pixel density difference is greater than the density threshold, it is determined that there is a following behavior.

[0015] Furthermore, after comparing the pixel density difference with a density threshold, and determining that following behavior exists if the pixel density difference is greater than the density threshold, the method further includes:

[0016] Obtain the minimum bounding rectangle corresponding to the foreground region, and obtain the area ratio by combining the area of ​​the foreground region and the area of ​​the minimum bounding rectangle;

[0017] Obtain the frame length and frame width of the minimum bounding rectangle, and obtain the aspect ratio using the frame length and frame width;

[0018] The area ratio is compared with a first threshold, and the aspect ratio is compared with a second threshold. If the area ratio is less than the first threshold and the aspect ratio is greater than the second threshold, then a second determination is made that there is following behavior.

[0019] Furthermore, the step of obtaining the category probability vector of each pixel in the final vehicle region includes:

[0020] The semantic feature vector and texture feature vector corresponding to each pixel in the final vehicle region are obtained by using the UNet segmentation model.

[0021] The semantic feature vector and the texture feature vector are fused into a pixel feature vector, and the category probability vector is obtained through the pixel feature vector.

[0022] Furthermore, the formula for obtaining the category probability vector is:

[0023] ,

[0024] in, This represents the probability vector of the vehicle ahead at pixel (i, j). This represents the probability vector of the following vehicle at pixel (i, j). This represents the background probability vector of pixel (i, j). This represents the pixel feature vector of pixel (i, j). Represents the weight matrix. This represents the bias vector. Represents matrix multiplication. This represents the probability normalization function.

[0025] Furthermore, the step of determining the first front vehicle pixel, the first rear vehicle pixel, the background pixel, and the adhered pixels based on the category probability vector includes:

[0026] The pixels whose probability vector of the preceding vehicle is greater than the probability threshold of the preceding vehicle are selected as the first preceding vehicle pixel, the pixels whose probability vector of the following vehicle is greater than the probability threshold of the following vehicle are selected as the first following vehicle pixel, the pixels whose probability vector of the background is greater than the probability threshold of the background are selected as background pixels, and the remaining pixels are selected as the first points to be identified.

[0027] Obtain the vector difference between the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified, and select the maximum probability vector from the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified.

[0028] The first point to be identified that has a vector difference less than the difference threshold and the highest probability vector is in the threshold range is selected as the sticky pixel point, and the remaining first points to be identified are selected as the second points to be identified.

[0029] By comparing the probability vectors of the preceding and following vehicles of the second identification point, if the probability vector of the preceding vehicle is greater than that of the following vehicle, the second identification point is selected as the first preceding vehicle pixel; if the probability vector of the following vehicle is greater than that of the preceding vehicle, the second identification point is selected as the first following vehicle pixel.

[0030] Furthermore, the step of dividing the adhered pixels into second front vehicle pixels and second rear vehicle pixels based on the initial front vehicle region and the initial rear vehicle region includes:

[0031] Obtain the centroid coordinates of the front vehicle image and the front vehicle radar corresponding to the initial front vehicle region, and obtain the centroid coordinates of the rear vehicle image and the rear vehicle radar corresponding to the initial rear vehicle region.

[0032] The probability vector of the stuck pixels is corrected to a correction vector by using the centroid coordinates of the front vehicle image and the centroid coordinates of the front vehicle radar, and the probability vector of the stuck pixels is corrected to a correction vector by using the centroid coordinates of the rear vehicle image and the centroid coordinates of the rear vehicle radar.

[0033] Obtain a first vector difference between the correction vector of the preceding vehicle and the correction vector of the following vehicle, and obtain a second vector difference between the correction vector of the following vehicle and the correction vector of the preceding vehicle. Select the sticky pixel points whose first vector difference is greater than the determination threshold as the second preceding vehicle pixel points, and select the sticky pixel points whose second vector difference is greater than the determination threshold as the second following vehicle pixel points.

[0034] Furthermore, the formula for obtaining the forward vehicle correction vector is:

[0035] ,

[0036] in, Indicates sticky pixels The forward vehicle correction vector, Indicates sticky pixels The probability vector of the vehicle ahead, Indicates the correction weight. Indicates the centroid coordinates of the image of the vehicle in front. Indicates the coordinates of the center of gravity of the radar of the vehicle in front. Indicates coordinate projection transformation, Indicates the distance attenuation coefficient. This represents an exponential function.

[0037] Furthermore, the step of obtaining the vehicle movement trajectory through two final preceding vehicle regions in adjacent time frames includes:

[0038] The state vector and appearance feature vector of the two final front vehicle regions are obtained respectively;

[0039] The first similarity is obtained by using the two state vectors, and the Euclidean distance between the two appearance feature vectors is obtained. The comprehensive correlation is obtained by using the first similarity and the second similarity.

[0040] If the overall correlation degree is greater than the correlation threshold, it is determined that the two final preceding vehicle regions represent the same vehicle, and the vehicle movement trajectory is constructed by the change of the centroid of the two final preceding vehicle regions.

[0041] Secondly, embodiments of this application provide a multi-vehicle recognition system for self-service toll stations, applied to the multi-vehicle recognition method for self-service toll stations as described in the first aspect above, the system comprising:

[0042] The recognition module is used to acquire vehicle images, obtain a final vehicle region from the vehicle images, and determine whether following behavior exists based on the final vehicle region.

[0043] The first analysis module is used to obtain the category probability vector of each pixel in the final vehicle area if there is a following behavior. The category probability vector includes the probability vector of the preceding vehicle, the probability vector of the following vehicle, and the background probability vector. Based on the category probability vector, the first preceding vehicle pixel, the first following vehicle pixel, the background pixel, and the sticky pixel are determined.

[0044] The second analysis module is used to combine the first front vehicle pixels into an initial front vehicle region and combine the first rear vehicle pixels into an initial rear vehicle region, and based on the initial front vehicle region and the initial rear vehicle region, to separate the adhered pixels into second front vehicle pixels and second rear vehicle pixels, and to obtain the final front vehicle region through the first front vehicle pixels and the second front vehicle pixels.

[0045] The execution module is used to obtain the vehicle movement trajectory through the two final preceding vehicle regions in adjacent time frames, and to determine whether the preceding vehicle has left based on the vehicle movement trajectory.

[0046] Thirdly, embodiments of this application provide a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-vehicle identification method for self-service toll stations as described in the first aspect above.

[0047] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the multi-vehicle identification method for self-service toll stations as described in the first aspect above.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows: by making a first judgment on following behavior based on the proportion of the vehicle in the same shooting area, a basis is provided for the recognition accuracy of the vehicle sensing coil. Further, a second judgment is made based on the shooting shape of the vehicle in the shooting area, which improves the accuracy of following behavior recognition. When following behavior exists, the overlapping pixels are identified by the category probability vector, and the overlapping intersection area of ​​the two vehicles is extracted. Then, by distinguishing the overlapping pixels, the accurate division of different vehicles is completed. On this basis, in different vehicle images, the position change of the final leading vehicle area is used to determine whether the leading vehicle has left. When the following vehicle is close and the vehicle sensing coil cannot be recognized, accurate collaborative judgment on whether the leading vehicle has left is provided to ensure the normal passage of the toll station. Attached Figure Description

[0049] Figure 1 This is a flowchart of the multi-vehicle identification method for self-service toll stations in the first embodiment of the present invention;

[0050] Figure 2 This is a structural block diagram of the multi-vehicle recognition system for self-service toll stations in the second embodiment of the present invention;

[0051] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0052] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0053] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0055] Please see Figure 1The multi-vehicle identification method for self-service toll stations provided in the first embodiment of the present invention includes the following steps:

[0056] S10: Acquire a vehicle image, obtain a final vehicle region from the vehicle image, and determine whether following behavior exists based on the final vehicle region;

[0057] The vehicle image is acquired by a camera. In this embodiment, the camera is a radar-visual integrated machine. While acquiring the vehicle image, the radar-visual integrated machine can also acquire the corresponding radar coordinates. It is understood that by setting up the camera at a lane of a toll gate, it only captures images within that lane, and subsequent judgments are only made for vehicles within that lane. Therefore, it is only necessary to identify whether two vehicles in the same lane are following each other.

[0058] Step S10 includes:

[0059] S110: Obtain an initial vehicle region from the vehicle image, enlarge the initial vehicle region to obtain a final vehicle region, and binarize the final vehicle region to separate the final vehicle region into a foreground region and a background region.

[0060] In this embodiment, key regions in the vehicle image are obtained through the dynamic query vector of the transformer decoder, thereby forming the initial vehicle region that includes the vehicle. The four sides of the initial vehicle region are parallel to the four sides of the vehicle image. The magnification process essentially enlarges the width and height of the rectangle that selects the vehicle, avoiding the exclusion of some edge information of the vehicle and ensuring that the entire vehicle is selected, forming the final vehicle region. Understandably, the foreground region is the area where the vehicle is located, and the background region is the area other than the vehicle.

[0061] S120: Obtain the foreground pixel density and background pixel density of the foreground region and the background region respectively, so as to obtain the pixel density difference based on the foreground pixel density and the background pixel density;

[0062] The foreground pixel density and the background pixel density refer to the number of pixels per unit area in the foreground region and the background region, respectively.

[0063] S130: Compare the pixel density difference with the density threshold. If the pixel density difference is greater than the density threshold, then determine that there is following behavior.

[0064] The density threshold can be determined based on multiple trials and can be adjusted as needed. Understandably, if the pixel density difference is less than the density threshold, it is determined that there is no following behavior, and no further steps are required; the vehicle's departure is determined based on the recognition result of the vehicle sensor coil.

[0065] Preferably, step S10 further includes:

[0066] S140: Obtain the minimum bounding rectangle corresponding to the foreground region, and obtain the area ratio by combining the area of ​​the foreground region and the area of ​​the minimum bounding rectangle;

[0067] The minimum bounding rectangle is formed based on the foreground region. The difference between the minimum bounding rectangle and the initial vehicle region is that the minimum bounding rectangle is selected according to the shape of the vehicle, and its four sides do not necessarily have to be parallel to the vehicle image.

[0068] S150: Obtain the frame length and frame width of the minimum bounding rectangle, and obtain the aspect ratio through the frame length and frame width;

[0069] S160: Compare the area ratio with a first threshold and the length-to-width ratio with a second threshold. If the area ratio is less than the first threshold and the length-to-width ratio is greater than the second threshold, then a second determination is made that there is following behavior.

[0070] The first and second thresholds can also be set as needed. When a single vehicle overlaps with two other vehicles, the area ratio within the minimum bounding rectangle and the shape of the resulting minimum bounding rectangle are different. Based on this, secondary recognition of following behavior can be performed, improving the accuracy of following behavior recognition.

[0071] S20: If there is a following behavior, obtain the category probability vector of each pixel in the final vehicle area. The category probability vector includes the probability vector of the preceding vehicle, the probability vector of the following vehicle, and the background probability vector. Based on the category probability vector, determine the first preceding vehicle pixel, the first following vehicle pixel, the background pixel, and the sticky pixel.

[0072] Step S20 includes:

[0073] S210: Obtain the semantic feature vector and texture feature vector corresponding to each pixel in the final vehicle region using the UNet segmentation model;

[0074] The UNet segmentation model includes an encoder and a decoder. The encoder downsamples and extracts features step by step through convolutional and pooling layers to compress spatial size and extract high-level semantic features, i.e., to identify the whole vehicle and obtain the semantic feature vector. The decoder transposes the convolution to gradually restore spatial resolution and recover detailed information, i.e., to separate the sticky parts of the two vehicles and obtain the texture feature vector.

[0075] S220: The semantic feature vector and the texture feature vector are fused into a pixel feature vector, and the category probability vector is obtained through the pixel feature vector;

[0076] The formula for obtaining the category probability vector is:

[0077] ,

[0078] in, This represents the probability vector of the vehicle ahead at pixel (i, j). This represents the probability vector of the following vehicle at pixel (i, j). This represents the background probability vector of pixel (i, j). This represents the pixel feature vector of pixel (i, j). Represents the weight matrix. This represents the bias vector. Represents matrix multiplication. This represents the probability normalization function. Understandably, the weight matrix contains three sets of independent weight vectors, and the bias vector contains three independent bias terms. The probability normalization function transforms the three results respectively, thereby obtaining three probability vectors based on the pixel feature vector.

[0079] S230: Select the pixel corresponding to the probability vector of the preceding vehicle being greater than the probability threshold of the preceding vehicle as the first preceding vehicle pixel, select the pixel corresponding to the probability vector of the following vehicle being greater than the probability threshold of the following vehicle as the first following vehicle pixel, select the pixel corresponding to the probability vector of the background being greater than the background probability threshold as the background pixel, and select the remaining pixels as the first point to be identified.

[0080] In this embodiment, the probability thresholds for both the preceding and following vehicles are 0.7, and the background probability threshold is 0.5. After this identification step, the first point to be identified may be the boundary area between the two vehicles, or it may be an area reflecting light from the vehicle bumper, the ground, etc. Therefore, further identification is required.

[0081] S240: Obtain the vector difference between the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified, and select the maximum probability vector from the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified.

[0082] S250: Select the first point to be identified that has a vector difference less than the difference threshold and the highest probability vector is in the threshold interval as the sticky pixel point, and select the remaining first points to be identified as the second points to be identified.

[0083] In this embodiment, the difference threshold is 0.3, and the threshold range is 0.4~0.6. When the vector difference of a certain first point to be identified is less than the difference threshold, and the maximum probability vector is within the threshold range, the first point to be identified is determined to be located in the boundary area between the two vehicles. The remaining first points to be identified are determined to be reflective areas, etc., and are assigned according to step S260.

[0084] S260: Compare the probability vectors of the preceding vehicle and the following vehicle of the second point to be identified. If the probability vector of the preceding vehicle is greater than the probability vector of the following vehicle, then the second point to be identified is selected as the first preceding vehicle pixel. If the probability vector of the following vehicle is greater than the probability vector of the preceding vehicle, then the second point to be identified is selected as the first following vehicle pixel.

[0085] S30: Combine the first front vehicle pixels into an initial front vehicle region, and combine the first rear vehicle pixels into an initial rear vehicle region. Based on the initial front vehicle region and the initial rear vehicle region, divide the adhered pixels into a second front vehicle pixel and a second rear vehicle pixel. Obtain the final front vehicle region through the first front vehicle pixel and the second front vehicle pixel.

[0086] Step 30 includes:

[0087] S310: Obtain the centroid coordinates of the front vehicle image and the front vehicle radar corresponding to the initial front vehicle region, and obtain the centroid coordinates of the rear vehicle image and the rear vehicle radar corresponding to the initial rear vehicle region.

[0088] Understandably, the centroid coordinates of the front vehicle radar and the rear vehicle radar are acquired synchronously during the shooting process by the aforementioned radar-visual integrated machine.

[0089] S320: Correct the front vehicle probability vector of the stuck pixel points to a front vehicle correction vector using the centroid coordinates of the front vehicle image and the centroid coordinates of the front vehicle radar, and correct the rear vehicle probability vector of the stuck pixel points to a rear vehicle correction vector using the centroid coordinates of the rear vehicle image and the centroid coordinates of the rear vehicle radar.

[0090] After obtaining the initial front vehicle region and the initial rear vehicle region, the probability vector of the second point to be identified can be corrected by the deviation between the radar coordinates and the image coordinates, so as to avoid errors in the classification of pixel points.

[0091] The formula for obtaining the forward vehicle correction vector is:

[0092] ,

[0093] in, Indicates sticky pixels The forward vehicle correction vector, Indicates sticky pixels The probability vector of the vehicle ahead, Indicates the correction weight. Indicates the centroid coordinates of the image of the vehicle in front. Indicates the coordinates of the center of gravity of the radar of the vehicle in front. Indicates coordinate projection transformation, Indicates the distance attenuation coefficient. This represents an exponential function. The method for obtaining the rear vehicle correction vector is the same as the method for obtaining the front vehicle correction vector, and will not be repeated here.

[0094] S330: Obtain the first vector difference between the front vehicle correction vector and the rear vehicle correction vector, and obtain the second vector difference between the rear vehicle correction vector and the front vehicle correction vector. Select the sticky pixel points corresponding to the first vector difference being greater than the determination threshold as the second front vehicle pixel points, and select the sticky pixel points corresponding to the second vector difference being greater than the determination threshold as the second rear vehicle pixel points.

[0095] In this embodiment, the determination threshold is 0.3. By introducing radar centroid coordinates to correct the probability vector, the accuracy of segmenting the adhesion region between the two vehicles is effectively improved.

[0096] S40: Obtain the vehicle movement trajectory through the two final preceding vehicle regions in adjacent time frames, and determine whether the preceding vehicle has left based on the vehicle movement trajectory;

[0097] Step S40 includes:

[0098] S410: Obtain the state vector and appearance feature vector of the two final front vehicle regions respectively;

[0099] The state vector includes the horizontal and vertical coordinates, aspect ratio, height, and corresponding velocity variables of the centroid of the final preceding vehicle region. In this embodiment, the Faster-RCNN object detection algorithm is used to acquire the state vector of the final preceding vehicle region at different time frames. While acquiring the state vector of the previous time frame, it simultaneously outputs the predicted state vector of the next time frame. Furthermore, a deep residual network is used as a feature extractor to extract the appearance feature vector.

[0100] S420: Obtain the first similarity between the two state vectors and the Euclidean distance between the two appearance feature vectors, and obtain the comprehensive correlation degree through the first similarity and the second similarity;

[0101] Understandably, the first similarity is between the predicted state vector of the next time frame obtained based on the state vector of the previous time frame and the state vector of the next time frame.

[0102] The formula for obtaining the first similarity is:

[0103] ,

[0104] in, This represents the first similarity between the predicted state vector obtained based on the state vector in the m-th time frame and the state vector in the n-th time frame. This represents the state vector in the nth time frame. This represents the predicted state vector obtained based on the state vector at the m-th time frame. Indicates the transpose operator. This represents the error covariance matrix of the prediction space. It should be noted that the m-th time frame and the n-th time frame are two adjacent time frames.

[0105] The formula for obtaining the second similarity is:

[0106] ,

[0107] in, This represents the second similarity between two appearance feature vectors at the m-th and n-th time frames. This represents the appearance feature vector at the nth time frame. This represents the appearance feature vector at the m-th time frame.

[0108] S430: If the overall correlation degree is greater than the correlation threshold, it is determined that the two final preceding vehicle regions represent the same vehicle, and the vehicle movement trajectory is constructed by the change of the centroid of the two final preceding vehicle regions.

[0109] The centroid coordinates of the two final preceding vehicle regions are obtained respectively. Based on the time frame difference between the two adjacent final preceding vehicle regions and the distance between the centroid coordinates, the vehicle movement trajectory can be constructed to determine whether the preceding vehicle has left.

[0110] By determining the following behavior based on the proportion of the vehicle within the same shooting area, a basis for the accuracy of vehicle sensor coil recognition is provided. Further, a second determination is made based on the shape of the vehicle within the shooting area, improving the accuracy of following behavior recognition. When following behavior is present, the overlapping pixels are identified using the category probability vector, extracting the overlapping area between the two vehicles. By distinguishing these overlapping pixels, accurate classification of different vehicles is achieved. Based on this, in different vehicle images, the position change of the final preceding vehicle area determines whether the preceding vehicle has left. When following too closely and the vehicle sensor coil cannot recognize the vehicle, accurate collaborative determination of whether the preceding vehicle has left is provided, ensuring normal passage at toll stations.

[0111] Please see Figure 2 The second embodiment of the present invention provides a multi-vehicle recognition system for self-service toll stations. This system is applied to the multi-vehicle recognition method for self-service toll stations described in the above embodiments, and will not be repeated hereafter. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0112] The system includes:

[0113] The recognition module 10 is used to acquire vehicle images, obtain a final vehicle region from the vehicle images, and determine whether following behavior exists based on the final vehicle region.

[0114] The identification module 10 includes:

[0115] The first unit is used to obtain an initial vehicle region from the vehicle image, enlarge the initial vehicle region to obtain a final vehicle region, and perform binarization on the final vehicle region to separate the final vehicle region into a foreground region and a background region.

[0116] The second unit is used to obtain the foreground pixel density and background pixel density of the foreground region and the background region respectively, so as to obtain the pixel density difference based on the foreground pixel density and the background pixel density;

[0117] The third unit is used to compare the pixel density difference with a density threshold. If the pixel density difference is greater than the density threshold, it is determined that there is a following behavior.

[0118] Preferably, the acquisition module 10 further includes:

[0119] The fourth unit is used to obtain the minimum bounding rectangle corresponding to the foreground region, and to obtain the area ratio by the area of ​​the foreground region and the area of ​​the minimum bounding rectangle;

[0120] The fifth unit is used to obtain the frame length and frame width of the minimum bounding rectangle, and to obtain the aspect ratio through the frame length and frame width;

[0121] The sixth unit is used to compare the area ratio with a first threshold and the aspect ratio with a second threshold. If the area ratio is less than the first threshold and the aspect ratio is greater than the second threshold, then a second determination is made that there is following behavior.

[0122] The first analysis module 20 is used to obtain the category probability vector of each pixel in the final vehicle area if there is a following behavior. The category probability vector includes the probability vector of the preceding vehicle, the probability vector of the following vehicle, and the background probability vector. Based on the category probability vector, the first preceding vehicle pixel, the first following vehicle pixel, the background pixel, and the sticky pixel are determined.

[0123] The first analysis module 20 includes:

[0124] The seventh unit is used to obtain the semantic feature vector and texture feature vector corresponding to each pixel in the final vehicle region through the UNet segmentation model;

[0125] The eighth unit is used to fuse the semantic feature vector and the texture feature vector into a pixel feature vector, and obtain a category probability vector through the pixel feature vector;

[0126] The ninth unit is used to select the pixel corresponding to the preceding vehicle probability vector being greater than the preceding vehicle probability threshold as the first preceding vehicle pixel, select the pixel corresponding to the following vehicle probability vector being greater than the following vehicle probability threshold as the first following vehicle pixel, select the pixel corresponding to the background probability vector being greater than the background probability threshold as the background pixel, and select the remaining pixels as the first point to be identified.

[0127] The tenth unit is used to obtain the vector difference between the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified, and to select the maximum probability vector from the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified.

[0128] The eleventh unit is used to select the first identification point corresponding to the vector difference being less than the difference threshold and the maximum probability vector being in the threshold interval as the sticky pixel point, and to select the remaining first identification points as the second identification points.

[0129] The twelfth unit is used to compare the probability vectors of the preceding vehicle and the following vehicle of the second identification point. If the probability vector of the preceding vehicle is greater than the probability vector of the following vehicle, the second identification point is selected as the first preceding vehicle pixel. If the probability vector of the following vehicle is greater than the probability vector of the preceding vehicle, the second identification point is selected as the first following vehicle pixel.

[0130] The second analysis module 30 is used to combine the first front vehicle pixels into an initial front vehicle region and combine the first rear vehicle pixels into an initial rear vehicle region, and based on the initial front vehicle region and the initial rear vehicle region, to separate the adhered pixels into a second front vehicle pixel and a second rear vehicle pixel, and to obtain the final front vehicle region through the first front vehicle pixel and the second front vehicle pixel.

[0131] The second analysis module 30 includes:

[0132] The thirteenth unit is used to obtain the centroid coordinates of the front vehicle image and the front vehicle radar corresponding to the initial front vehicle area, and to obtain the centroid coordinates of the rear vehicle image and the rear vehicle radar corresponding to the initial rear vehicle area.

[0133] The fourteenth unit is used to correct the front vehicle probability vector of the stuck pixel points into a front vehicle correction vector by using the centroid coordinates of the front vehicle image and the centroid coordinates of the front vehicle radar, and to correct the rear vehicle probability vector of the stuck pixel points into a rear vehicle correction vector by using the centroid coordinates of the rear vehicle image and the centroid coordinates of the rear vehicle radar.

[0134] The fifteenth unit is used to obtain a first vector difference between the front vehicle correction vector and the rear vehicle correction vector, and to obtain a second vector difference between the rear vehicle correction vector and the front vehicle correction vector. The sticky pixel points corresponding to the first vector difference being greater than the determination threshold are selected as the second front vehicle pixel points, and the sticky pixel points corresponding to the second vector difference being greater than the determination threshold are selected as the second rear vehicle pixel points.

[0135] Execution module 40 is used to obtain the vehicle movement trajectory through the two final preceding vehicle regions in adjacent time frames, and determine whether the preceding vehicle has left based on the vehicle movement trajectory;

[0136] The execution module 40 includes:

[0137] The sixteenth unit is used to obtain the state vector and appearance feature vector of the two final front vehicle regions respectively;

[0138] The seventeenth unit is used to obtain a first similarity through the two state vectors, and to obtain the Euclidean distance between the two appearance feature vectors, and to obtain a comprehensive correlation degree through the first similarity and the second similarity;

[0139] The eighteenth unit is used to determine that if the comprehensive correlation degree is greater than the correlation threshold, the two final preceding vehicle regions represent the same vehicle, and to construct the vehicle movement trajectory by the change of the centroid of the two final preceding vehicle regions.

[0140] The present invention also provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-vehicle identification method for self-service toll stations as described in the above technical solutions.

[0141] The present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the multi-vehicle identification method for self-service toll stations as described in the above technical solutions.

[0142] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0143] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for multi-vehicle identification at a self-service toll station, characterized in that, Includes the following steps: Acquire a vehicle image, obtain a final vehicle region from the vehicle image, and determine whether following behavior exists based on the final vehicle region; The step of obtaining the final vehicle region from the vehicle image and determining whether following behavior exists based on the final vehicle region includes: An initial vehicle region is obtained from the vehicle image, the initial vehicle region is magnified to obtain a final vehicle region, and the final vehicle region is binarized to separate the final vehicle region into a foreground region and a background region. The foreground pixel density and background pixel density of the foreground region and the background region are obtained respectively, so as to obtain the pixel density difference based on the foreground pixel density and the background pixel density; The pixel density difference is compared with a density threshold. If the pixel density difference is greater than the density threshold, it is determined that there is a following behavior. Obtain the minimum bounding rectangle corresponding to the foreground region, and obtain the area ratio by combining the area of ​​the foreground region and the area of ​​the minimum bounding rectangle; Obtain the frame length and frame width of the minimum bounding rectangle, and obtain the aspect ratio using the frame length and frame width; The area ratio is compared with a first threshold, and the aspect ratio is compared with a second threshold. If the area ratio is less than the first threshold and the aspect ratio is greater than the second threshold, then a second determination is made that there is following behavior. If following behavior exists, the category probability vector of each pixel in the final vehicle region is obtained. The category probability vector includes the probability vector of the preceding vehicle, the probability vector of the following vehicle, and the background probability vector. Based on the category probability vector, the first preceding vehicle pixel, the first following vehicle pixel, the background pixel, and the connected pixels are determined. The first front vehicle pixels are combined into an initial front vehicle region, and the first rear vehicle pixels are combined into an initial rear vehicle region. Based on the initial front vehicle region and the initial rear vehicle region, the stuck pixels are separated into a second front vehicle pixel and a second rear vehicle pixel. The final front vehicle region is obtained through the first front vehicle pixel and the second front vehicle pixel. The vehicle movement trajectory is obtained by using the two final preceding vehicle regions in adjacent time frames, and the preceding vehicle is determined to have left based on the vehicle movement trajectory.

2. The multi-vehicle identification method for self-service toll stations according to claim 1, characterized in that, The step of obtaining the category probability vector of each pixel in the final vehicle region includes: The semantic feature vector and texture feature vector corresponding to each pixel in the final vehicle region are obtained by using the UNet segmentation model. The semantic feature vector and the texture feature vector are fused into a pixel feature vector, and the category probability vector is obtained through the pixel feature vector.

3. The multi-vehicle identification method for self-service toll stations according to claim 2, characterized in that, The formula for obtaining the category probability vector is: ， in, This represents the probability vector of the vehicle ahead at pixel (i, j). This represents the probability vector of the following vehicle at pixel (i, j). This represents the background probability vector of pixel (i, j). This represents the pixel feature vector of pixel (i, j). Represents the weight matrix. This represents the bias vector. Represents matrix multiplication. This represents the probability normalization function.

4. The multi-vehicle identification method for self-service toll stations according to claim 1, characterized in that, The step of determining the first front vehicle pixel, the first rear vehicle pixel, the background pixel, and the connected pixels based on the category probability vector includes: The pixels whose probability vector of the preceding vehicle is greater than the probability threshold of the preceding vehicle are selected as the first preceding vehicle pixel, the pixels whose probability vector of the following vehicle is greater than the probability threshold of the following vehicle are selected as the first following vehicle pixel, the pixels whose probability vector of the background is greater than the probability threshold of the background are selected as background pixels, and the remaining pixels are selected as the first points to be identified. Obtain the vector difference between the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified, and select the maximum probability vector from the probability vector of the vehicle in front and the probability vector of the vehicle behind the first point to be identified. The first point to be identified that has a vector difference less than the difference threshold and the highest probability vector is in the threshold range is selected as the sticky pixel point, and the remaining first points to be identified are selected as the second points to be identified. By comparing the probability vectors of the preceding and following vehicles of the second identification point, if the probability vector of the preceding vehicle is greater than that of the following vehicle, the second identification point is selected as the first preceding vehicle pixel; if the probability vector of the following vehicle is greater than that of the preceding vehicle, the second identification point is selected as the first following vehicle pixel.

5. The multi-vehicle identification method for self-service toll stations according to claim 1, characterized in that, The step of dividing the adhered pixels into second front vehicle pixels and second rear vehicle pixels based on the initial front vehicle region and the initial rear vehicle region includes: Obtain the centroid coordinates of the front vehicle image and the front vehicle radar corresponding to the initial front vehicle region, and obtain the centroid coordinates of the rear vehicle image and the rear vehicle radar corresponding to the initial rear vehicle region. The probability vector of the stuck pixels is corrected to a correction vector by using the centroid coordinates of the front vehicle image and the centroid coordinates of the front vehicle radar, and the probability vector of the stuck pixels is corrected to a correction vector by using the centroid coordinates of the rear vehicle image and the centroid coordinates of the rear vehicle radar. Obtain a first vector difference between the correction vector of the preceding vehicle and the correction vector of the following vehicle, and obtain a second vector difference between the correction vector of the following vehicle and the correction vector of the preceding vehicle. Select the sticky pixel points whose first vector difference is greater than the determination threshold as the second preceding vehicle pixel points, and select the sticky pixel points whose second vector difference is greater than the determination threshold as the second following vehicle pixel points.

6. The multi-vehicle identification method for self-service toll stations according to claim 5, characterized in that, The formula for obtaining the forward vehicle correction vector is: ， in, Indicates sticky pixels The forward vehicle correction vector, Indicates sticky pixels The probability vector of the vehicle ahead, Indicates the correction weight. Indicates the centroid coordinates of the image of the vehicle in front. Indicates the coordinates of the center of gravity of the radar of the vehicle in front. Indicates coordinate projection transformation, Indicates the distance attenuation coefficient. This represents an exponential function.

7. The multi-vehicle identification method for self-service toll stations according to claim 1, characterized in that, The step of obtaining the vehicle movement trajectory through two final preceding vehicle regions in adjacent time frames includes: The state vector and appearance feature vector of the two final front vehicle regions are obtained respectively; A first similarity is obtained by using the two state vectors, and a second similarity is obtained by using the two appearance feature vectors. A comprehensive correlation is obtained by using the first similarity and the second similarity. The formula for obtaining the second similarity is: ， in, This represents the second similarity between two appearance feature vectors at the m-th and n-th time frames. This represents the appearance feature vector at the nth time frame. This represents the appearance feature vector at the m-th time frame; If the overall correlation degree is greater than the correlation threshold, it is determined that the two final preceding vehicle regions represent the same vehicle, and the vehicle movement trajectory is constructed by the change of the centroid of the two final preceding vehicle regions.

8. A multi-vehicle identification system for a self-service toll station, applied to the multi-vehicle identification method for a self-service toll station as described in any one of claims 1 to 7, characterized in that, The system includes: The recognition module is used to acquire vehicle images, obtain a final vehicle region from the vehicle images, and determine whether following behavior exists based on the final vehicle region. The identification module includes: The first unit is used to obtain an initial vehicle region from the vehicle image, enlarge the initial vehicle region to obtain a final vehicle region, and perform binarization on the final vehicle region to separate the final vehicle region into a foreground region and a background region. The second unit is used to obtain the foreground pixel density and background pixel density of the foreground region and the background region respectively, so as to obtain the pixel density difference based on the foreground pixel density and the background pixel density; The third unit is used to compare the pixel density difference with a density threshold. If the pixel density difference is greater than the density threshold, it is determined that there is a following behavior. The fourth unit is used to obtain the minimum bounding rectangle corresponding to the foreground region, and to obtain the area ratio by the area of ​​the foreground region and the area of ​​the minimum bounding rectangle; The fifth unit is used to obtain the frame length and frame width of the minimum bounding rectangle, and to obtain the aspect ratio through the frame length and frame width; The sixth unit is used to compare the area ratio with a first threshold and the aspect ratio with a second threshold. If the area ratio is less than the first threshold and the aspect ratio is greater than the second threshold, then a second determination is made that there is following behavior. The first analysis module is used to obtain the category probability vector of each pixel in the final vehicle area if there is a following behavior. The category probability vector includes the probability vector of the preceding vehicle, the probability vector of the following vehicle, and the background probability vector. Based on the category probability vector, the first preceding vehicle pixel, the first following vehicle pixel, the background pixel, and the sticky pixel are determined. The second analysis module is used to combine the first front vehicle pixels into an initial front vehicle region and combine the first rear vehicle pixels into an initial rear vehicle region, and based on the initial front vehicle region and the initial rear vehicle region, to separate the adhered pixels into second front vehicle pixels and second rear vehicle pixels, and to obtain the final front vehicle region through the first front vehicle pixels and the second front vehicle pixels. The execution module is used to obtain the vehicle movement trajectory through the two final preceding vehicle regions in adjacent time frames, and to determine whether the preceding vehicle has left based on the vehicle movement trajectory.

Images