Methods and systems for early warning of abnormal toll evasion behavior of vehicles on highways
By collecting real-time images at the entrances and exits of the expressway and identifying face and license plates, the problems of high cost of evasion and poor timeliness in the existing technology are solved, and real-time early warning and prevention of evasion behavior are achieved.
Patent Information
- Application Number
- CN202210541263.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-17
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-05-17
AI Technical Summary
When dealing with highway vehicle evasion, the existing technology mainly involves post-event investigation, which is costly and timely, lacks full-process vehicle detection and inspection methods, and cannot effectively prevent evasion.
Use a high-speed camera to collect real-time images at the entrances and exits of the expressway, use the YOLO network to perform face recognition and license plate positioning, combine SVM model for character recognition, generate face and license plate mapping relationship, verify and mark abnormal behavior in real time, and send warnings Information to the monitoring end.
Real-time marking and early warning of vehicles that may cause evasion, reduce the cost of evasion and tracking, and improve timeliness and preventive effects.
Smart Images

Figure CN114758402B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of abnormal toll evasion behavior processing, and in particular to an abnormal toll evasion behavior early warning method and system for vehicles on expressways. Background Art
[0002] The phenomenon of highway toll evasion is common, which has had a serious impact on highway operations. For example, a vehicle that used the method of "running long and buying short" to evade tolls was successfully intercepted at a toll station in the jurisdiction. After verification, the vehicle evaded tolls more than 130 times, involving more than 10,000 yuan in tolls. In order to reduce the occurrence of this phenomenon, it is necessary to establish a system for preventing toll evasion.
[0003] Existing fare evasion is usually handled after the fact, that is, the fare evasion is tracked and traced after the fare evasion has occurred, which results in high cost, low efficiency and failure to achieve the expected results. Preventive measures are basically to detect the historical data of licensed vehicles, lacking a comprehensive and full-process vehicle detection and screening method, and there is a huge gap in timeliness and accuracy. Summary of the invention
[0004] In order to overcome the above-mentioned technical defects, the purpose of the present invention is to provide a method and system for early warning of abnormal behavior of toll evasion by vehicles on highways, which is used to overcome the problem that the existing toll evasion behavior is mostly tracked and traced after the toll evasion behavior has occurred, which is costly and has poor timeliness.
[0005] The present invention discloses a method for early warning of abnormal behavior of toll evasion by vehicles on highways, comprising:
[0006] After receiving a vehicle passing signal, the high-speed camera is started to collect real-time images at the entrance and exit of the highway respectively to obtain a first image set and a second image set;
[0007] For the first image set and the second image set, a pre-trained recognition model is used to perform face recognition at a preset position of each real-time image to obtain face data at the driving position in the real-time image, and a first face data set and a second face data set are correspondingly obtained; wherein the recognition model is established based on a YOLO network;
[0008] Searching for facial data that matches the second facial data set in the first facial data set, generating a facial record for each facial data that matches, and leaving the record empty if the match fails, to generate a facial record set;
[0009] For the first image set and the second image set, the detection model is used to locate the license plate of each real-time image to obtain the license plate area, and the characters of the license plate area are segmented to obtain the segmented image, and the SVM model is used to perform character recognition on the segmented image. If the recognition fails, it is recorded as empty to obtain the first license plate set and the second license plate set;
[0010] Associating the face record set with the first license plate set and the first vehicle set based on the real-time image to form a to-be-verified information set having a mapping relationship between a plurality of face records, a first license plate and a second license plate;
[0011] For any information to be verified in the set of information to be verified:
[0012] Check whether the face data corresponding to each real-time image exceeds one, and if so, mark it as abnormal data;
[0013] Check whether there is a data segment in the information to be verified that is empty, and if so, mark it as abnormal behavior data;
[0014] Retrieving vehicle data from a preset database according to the face record corresponding to the information to be verified;
[0015] When there is historical violation data in the vehicle data, it is marked as attention information;
[0016] When the vehicle data is verified to be inconsistent with the first license plate or the second license plate in the information to be verified, it is marked as abnormal behavior data;
[0017] Generate warning information based on the abnormal behavior data and the attention information, and send it to the monitoring terminal.
[0018] Preferably, the method of performing face recognition on a preset position of each real-time image using a pre-trained recognition model to obtain face data at the driving position in the real-time image includes:
[0019] Build an initial model based on the YOLO network, and obtain the target model after training;
[0020] For any real-time image, in the target model, the real-time image is sampled by using a starting module;
[0021] The facial landmarks are regressed and located using loss functions to identify key points of the face in the sampled real-time image.
[0022] A spatial pyramid pooling module with a preset kernel is used for pooling, and the face data at the driving position in the real-time image is output accordingly.
[0023] Preferably, the use of the detection model to locate the license plate of each real-time image to obtain the license plate area includes:
[0024] For any real-time image, perform enhancement preprocessing on the real-time image and then binarize it to obtain a first image;
[0025] Using an edge detection operator to perform edge detection on the first image, and performing region expansion to obtain a second image containing multiple connected domains;
[0026] A connected domain including the license plate is identified based on the second image to obtain a license plate area.
[0027] Preferably, the use of the detection model to locate the license plate of each real-time image to obtain the license plate area includes:
[0028] For any real-time image, edge detection is performed on the real-time image, and binarization is performed to obtain an edge image;
[0029] The edge image is divided into a number of sub-blocks, and the sub-blocks are collected using a sliding window and the license plate boundary is determined based on a neural network to obtain a license plate area.
[0030] Preferably, the performing character segmentation on the license plate area to obtain a segmented image includes:
[0031] Binarizing the license plate area to obtain a binary image and calculating a smooth curve of vertical integral projection;
[0032] Determine a trough position sequence according to the smooth curve, and extract two adjacent troughs in the trough position sequence as left and right boundaries respectively to obtain a plurality of rectangular areas;
[0033] Each rectangular area is split or merged according to preset rules to obtain a segmented image.
[0034] Preferably, the adopting of the SVM model to perform character recognition on the segmented image comprises:
[0035] Obtain a preset data set, and mark each data in the data set to generate a training set;
[0036] The training set is used to call the training instruction to train the SVM model and store it as a target file;
[0037] The target file is called to perform character recognition on the segmented image based on the directional gradient histogram of the segmented image.
[0038] Preferably, in the verification information, path information corresponding to each face record is generated according to the mapping relationship between the face record, the first license plate and the second license plate and the real-time image, wherein the path information includes location data and time data at the entrance and exit of the expressway;
[0039] After a preset time period, the payment record corresponding to the path information in the vehicle data is queried, and when the payment record is empty, it is marked as abnormal behavior data.
[0040] Preferably, the method further comprises: receiving a feedback signal sent by the monitoring terminal to confirm the fee evasion behavior, and acquiring corresponding abnormal behavior data based on the feedback signal to update the preset database.
[0041] The present invention also provides a highway vehicle toll evasion abnormal behavior early warning system, comprising:
[0042] The acquisition module is used to start the high-speed camera to collect real-time images at the entrance and exit of the expressway after receiving the vehicle passing signal, so as to obtain a first image set and a second image set;
[0043] A face recognition module, for the first image set and the second image set, using a pre-trained recognition model to perform face recognition at a preset position of each real-time image, to obtain face data at the driving position in the real-time image, and correspondingly obtain a first face data set and a second face data set; wherein the recognition model is established based on a YOLO network;
[0044] A processing module, used for searching the first face data set for face data that matches the second face data set, generating a face record for each face data that matches, and leaving the record empty if the match fails, so as to generate a face record set;
[0045] A license plate recognition module is used to locate the license plate of each real-time image using a detection model for the first image set and the second image set to obtain a license plate area, perform character segmentation on the license plate area to obtain a segmented image, perform character recognition on the segmented image using a SVM model, and record empty if recognition fails to obtain a first license plate set and a second license plate set;
[0046] An association module, configured to associate the face record set with the first license plate set and the first vehicle set based on the real-time image, so as to form a to-be-verified information set having a mapping relationship between a plurality of face records, a first license plate and a second license plate;
[0047] The early warning module is used to verify whether the facial data corresponding to each real-time image exceeds one, and if so, mark it as abnormal data; verify whether there is an empty data segment in the information to be verified, and if so, mark it as abnormal behavior data; call vehicle data from a preset database according to the facial record corresponding to the information to be verified; when there is historical violation data in the vehicle data, mark it as attention information; when it is verified based on the vehicle data that it is inconsistent with the first license plate or the second license plate in the information to be verified, mark it as abnormal behavior data; generate warning information based on the abnormal behavior data and the attention information, and send it to the monitoring end.
[0048] Preferably, it also includes:
[0049] The early warning module generates path information corresponding to each face record in the verification information according to the face record, the first license plate and the second license plate mapping relationship and the real-time image, wherein the path information includes the location data and time data of the entrance and exit of the expressway;
[0050] After a preset time period, the payment record corresponding to the path information in the vehicle data is queried, and when the payment record is empty, it is marked as abnormal behavior data.
[0051] Compared with the prior art, the above technical solution has the following beneficial effects:
[0052] The present invention collects real-time images, and through image analysis means (including face recognition and license plate recognition), marks vehicles that may evade fares for abnormal behavior, and sends relevant data to a monitoring terminal in real time for early warning so that the monitoring terminal can take measures in real time, thereby overcoming the problem that the existing practice of evading fares is mostly to investigate and track evasion after the evasion has occurred, which is costly and less timely. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 This is a flow chart of Embodiment 1 of a method for early warning of abnormal behavior of toll evasion by vehicles on highways according to the present invention;
[0054] Figure 2 This is a module diagram of a second embodiment of a highway vehicle toll evasion abnormal behavior early warning system according to the present invention.
[0055] Reference numerals:
[0056] 10-Highway vehicle toll evasion abnormal behavior warning system; 101-acquisition module; 102-recognition module; 103-processing module; 104-license plate recognition module; 105-association module; 106-warning module. DETAILED DESCRIPTION
[0057] The advantages of the present invention are further described below in conjunction with the accompanying drawings and specific embodiments.
[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0059] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0060] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0061] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0062] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.
[0063] In the following description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module" and "component" can be used interchangeably.
[0064] Embodiment 1: This embodiment discloses a method for warning abnormal behavior of toll evasion by vehicles on highways. Figure 1 ,include:
[0065] S100: after receiving a vehicle passing signal, starting a high-speed camera to collect real-time images at the entrance and exit of the highway respectively, to obtain a first image set and a second image set;
[0066] In the above steps, real-time image acquisition begins after receiving a vehicle passing signal, that is, in the absence of abnormalities, each real-time image should contain an image of a vehicle passing. It should also be noted that the real-time image contains the parameters of the acquired high-speed camera, based on which the location and time of the real-time image acquisition can be determined. When shooting a moving object, the most important problem to overcome is ghosting. Ghosting is formed when there is relative motion between the shooting target and the camera system during exposure. Because of this relative motion, the image formed on the chip is constantly changing. The pixels in various parts are affected by imaging from different positions of the object during the exposure process. The final picture is a superposition of pictures in a continuously changing image space. In the subsequent steps of this embodiment, it is necessary to determine the face and license plate, so a high-speed camera is used to obtain a real-time image with better color reproduction.
[0067] S200: For the first image set and the second image set, a pre-trained recognition model is used to perform face recognition at a preset position of each real-time image to obtain face data at the driving position in the real-time image, and a first face data set and a second face data set are correspondingly obtained; wherein the recognition model is established based on a YOLO network;
[0068] In the above steps, the first face data set includes face data of each vehicle at the entrance of the highway, and the second face data set includes face data of each vehicle at the exit of the highway. Specifically, the Yolov5face framework in the open source platform is used for face recognition processing, wherein the improvements made relative to the existing Yolov5 framework include: adding a key point regression head (regression face) to the YOLOv5 network, using Wingloss as the loss function, replacing the Focus layer of YOLOv5 with the Stem block structure, modifying the SPP block, using a smaller kernel, adding a P6 output block with a stride of 64, and removing the up-down flipping layer. It should also be noted that the recognition model autonomously identifies the facial data of the driver's seat in the real-time image, and can determine the vehicle position in combination with target detection, and then determine the driving position based on the preset range of the vehicle position, or because the driving position is usually on the left side of the real-time image, the facial information of the left half of the real-time image is identified, or in most cases, when there is only one car passing by, the driver's seat face should be the clearest, then the clearest face closest to the camera is determined to be the driver's seat face, and when there are multiple vehicles or multiple driving seat faces, abnormal toll evasion may occur.
[0069] Specifically, the method of performing face recognition on a preset position of each real-time image using a pre-trained recognition model to obtain face data at the driving position in the real-time image includes:
[0070] S210: Build an initial model based on the YOLO network, and obtain the target model after training;
[0071] As mentioned above, in the above steps, an initial model is established based on the existing Yolov5 network. In order to improve the recognition accuracy, a large number of face photos are set as a data set. The data set is marked as YOLO mode and a TXT file is generated. The TXT file format contains object category, object coordinates, image width, image height, etc. The data set is divided into a training set and a validation set and passed into the neural network for training.
[0072] S220: For any real-time image, in the target model, use a starting module to sample the real-time image;
[0073] In the above steps, it should be noted that, as mentioned above, since the initial model is established based on the existing Yolov5 network, the structures of the initial model and the target model are similar to the Yolov5 network. The starting module (i.e., the above-mentioned Stem block) is used for sampling instead of the Focus layer in the Yolov5 network, which increases the generalization ability of the network and reduces the complexity of the calculation without reducing the performance. Removing the up-down flipping of the feature map will improve the performance. When using small images, Mosaic data enhancement will reduce performance, and random cropping will help improve performance, so it can be set selectively during the sampling process.
[0074] S230: extracting features after adopting, and identifying key points of the face on the sampled real-time image by using regression facial landmark positioning and loss function;
[0075] In the above steps, after sampling, the CSP network is used to extract features, and then the facial landmark positioning and loss function are implemented using the key point regression head (regression face) and Wing loss. Wing loss is the loss function of the facial key point regression method, and the prediction value range is 0-1. During training, when the error of Wing loss is particularly large, the gradient is constant. When the error is relatively small, the gradient is larger than the mean absolute error and the mean square error.
[0076] S240: Performing pooling using a spatial pyramid pooling module with a preset kernel, and outputting corresponding face data at the driving position in the real-time image.
[0077] In the above steps, the feature map identified by the key points of the face is pooled using a spatial pyramid pooling module to speed up the calculation and prevent overfitting, and a P6 output block with a stride of 64 is added to increase the ability to detect large faces. Based on the above steps S210-S240, compared with the existing Yolov5 network, the average precision (mAP) and speed are improved, which is more suitable for face recognition and detection at high speed in this embodiment.
[0078] S300: searching for facial data that matches the second facial data set in the first facial data set, generating a facial record for each facial data that matches, and leaving the record empty if the match fails, to generate a facial record set;
[0079] Based on the recognition model used in the above S200 to recognize the faces in each real-time image, it should be noted that the first face data are all faces at the entrance of the highway, that is, any vehicle entering the highway, and the second face data are all faces at the exit of the highway, that is, any vehicle leaving the highway. If there is no abnormal behavior of evading fees, the faces in the first face data set and the second face data set should correspond to each other one by one. However, when abnormal behavior of evading fees occurs, such as following a car, the second face data cannot be recognized. At this time, it will be determined as abnormal behavior of evading fees and sent to the monitoring end for subsequent warning of abnormal behavior. It should be noted that in this process, face data that cannot be matched also generates a face record, but a face record with an empty data segment (i.e., a blank data segment) is generated, so that this situation can be determined as abnormal behavior in the subsequent step S600.
[0080] S400: For the first image set and the second image set, a detection model is used to locate the license plate of each real-time image to obtain a license plate area, and the license plate area is segmented to obtain a segmented image, and a SVM model is used to perform character recognition on the segmented image, and if the recognition fails, it is recorded as empty to obtain a first license plate set and a second license plate set;
[0081] In this embodiment, in addition to collecting human faces, license plate positioning is also performed. The positioning results may have three results: a. the license plate exists, b. a temporary license plate exists, and c. the license plate position cannot be obtained. Only the license plate exists, which is not an abnormal behavior. The other two may be abnormal behaviors of evading fees, and both may require the generation of warning information. Therefore, in this embodiment, edge detection or neural network is used to locate the license plate to achieve license plate recognition.
[0082] Specifically, the use of the detection model to locate the license plate of each real-time image to obtain the license plate area includes:
[0083] S411: for any real-time image, perform enhancement preprocessing on the real-time image and then binarize it to obtain a first image;
[0084] S412: performing edge detection on the first image using an edge detection operator, and performing region expansion to obtain a second image including a plurality of connected domains;
[0085] S413: Based on the second image, a connected domain including the license plate is identified to obtain a license plate area.
[0086] The above steps S411-S413 are a license plate positioning method based on edge detection: before positioning the license plate, the real-time image is first pre-processed by grayscale transformation, histogram equalization and other enhancements, then binarized, and finally the edge detection operator is used to detect the edge of the image. After the edge is detected, the region is expanded and irrelevant small objects are corroded. At this time, the image will present multiple connected judgment areas, and finally the one that is most likely to be the license plate among all the connected domains can be found. More specifically, the most likely license plate can be judged based on parameters such as position and size.
[0087] Optionally, the use of the detection model to locate the license plate of each real-time image to obtain the license plate area includes:
[0088] S421: For any real-time image, edge detection is performed on the real-time image, and binarization is performed to obtain an edge image;
[0089] S422: Divide the edge image into several sub-blocks, collect the sub-blocks using a sliding window, and determine the license plate boundary based on a neural network to obtain a license plate area.
[0090] The above steps S421-S422 are to realize license plate positioning by using neural network, which can be realized by directly perceiving grayscale images, that is, using a sliding window as a sampling window (a long strip or narrow sliding window can be selected according to the characteristics of the license plate), moving sequentially on the grayscale image, and using the image block covered by the window as the input of the neural network. The BP network used is a 3-layer fully connected feedforward network, the number of neurons in its input layer is the size of the sliding window, and the number of neurons in its output layer is 1. When the output exceeds 1 / 2, it means that the image block under the sliding window belongs to the license plate area; when the output is less than 1 / 2, it means that the image block under the sliding window belongs to the background area.
[0091] The above method first requires repeated sampling of the license plate, and various lighting conditions, background complexity and license plate colors should be included in the real-time image as much as possible to facilitate the generalization of the network, which can enhance the fault tolerance of the network; since the license plate is generally located in the middle and lower part of the real-time image, it is generally traversed from bottom to top, so that it is not easy to mistakenly locate the vehicle model logo as the license plate area, and when more than one candidate license plate area appears after traversing the image, the candidate license plate area located at the bottom should also be given priority. In addition, for license plates with large tilt angles, horizontal correction preprocessing should be performed before neural network processing.
[0092] The edge detection algorithm used above is simple and clear; while the application scenarios of the neural network are comprehensive and the accuracy is high, and the appropriate application method can be selected according to the actual scenario.
[0093] Specifically, performing character segmentation on the license plate area to obtain a segmented image includes:
[0094] S431: Binarize the license plate area to obtain a binary image and calculate a smooth curve of vertical integral projection;
[0095] S432: determining a trough position sequence according to the smooth curve, and extracting two adjacent troughs in the trough position sequence as left and right boundaries respectively to obtain a plurality of rectangular regions;
[0096] S433: Split or merge each rectangular area according to a preset rule to obtain a segmented image.
[0097] In the above steps, the character segmentation in this embodiment first calculates the smooth curve of the vertical integral projection on the binary image, searches for the local minimum of the smooth curve as a trough position, and all trough positions form a trough position sequence; then a group of rectangular areas are extracted by taking two adjacent troughs as the left and right boundaries respectively; finally, according to preset rules, the rectangular areas are subjected to operations including but not limited to deletion, splitting, merging and resizing, thereby realizing single character segmentation of the license plate area.
[0098] Specifically, the use of the SVM model to perform character recognition on the segmented image includes:
[0099] S441: Obtain a preset data set, and mark each data in the data set to generate a training set;
[0100] S442: Using the training set to call the training instruction to train the SVM model, and store it as a target file;
[0101] S443: Calling the target file to perform character recognition on the segmented image based on the directional gradient histogram of the segmented image.
[0102] In the above steps, the SVM model is used to realize the recognition of each single character segmented. First, the data is extracted and labeled: It should be noted that the labeled labels here are deskew and hog, both of which are commonly used APIs in OpenCV, which can reduce the size of the image feature values and speed up the recognition speed; call the train in the SVM model to train the data and generate an XML file; finally, after restoring the SVM model from the XML file, the feature values (the directional gradient histogram of the above segmented image) are still needed for matching before recognition, and then call the SVMPredict that comes with OpenCV for character recognition.
[0103] S500: Associating the face record set with the first license plate set and the first vehicle set based on the real-time image to form a to-be-verified information set having a mapping relationship between a plurality of face records, a first license plate and a second license plate;
[0104] In the above steps, based on the above recognition models and detection models, the facial data and license plates at the entrance and exit of the highway can be obtained. Based on this, the facial data and license plates appearing in the same real-time image can be associated into one piece of data, and then based on the facial records generated in the above step S300, several associated data can be generated, which is the above information to be verified.
[0105] For any information to be verified in the set of information to be verified:
[0106] S600: Check whether the face data corresponding to each real-time image exceeds one, if so, mark it as abnormal data; check whether there is a data segment in the information to be verified that is empty, if so, mark it as abnormal behavior data;
[0107] In the above steps, when the number of facial data recognized in the real-time image exceeds one, based on the above-mentioned known facial data recognized in the driver's seat in this solution, if the number exceeds one, it means that more than one vehicle has passed by at this time, and there may be abnormal behavior of taking a ride to evade fees; and if there are multiple facial data, there may be a situation where the data segment is empty when generating the facial record, but the fee has actually been evaded, and whether there is an empty data segment in the information to be verified, it may be that any of the first license plate data / second license plate data / first face data / second face data cannot be recognized, and at this time there may be a behavior of the license plate being blocked or taking a ride to evade fees, which are all marked as abnormal behavior data.
[0108] S700: Retrieving vehicle data from a preset database according to the face record corresponding to the information to be verified;
[0109] It should be noted that the preset database stores several driver records, and each driver record includes personal information, facial images, vehicle data, whether there are any abnormal behavior records and other information; the present embodiment also includes: receiving a feedback signal sent by the monitoring end confirming the toll evasion behavior, obtaining the corresponding abnormal behavior data based on the feedback signal to update the preset database, and after being marked as abnormal behavior data and sent to the monitoring end, it is confirmed as abnormal behavior data by the staff, and its corresponding facial data and license plate number will be recorded in the preset database to realize data update. It can also be based on this accumulation and establishment of a database of duplicate license plate vehicles, a database of suspected vehicles, a database of historical toll evasion, etc., to improve detection efficiency and accuracy.
[0110] S800: When there is historical violation data in the vehicle data, it is marked as attention information; when it is verified based on the vehicle data that it is inconsistent with the first license plate or the second license plate in the information to be verified, it is marked as abnormal behavior data;
[0111] In the above steps, when the driver record consistent with the face record is matched from the preset database and the corresponding vehicle data is obtained, the vehicle data contains information such as license plate and payment record. If the recognized license plate is inconsistent with the license plate recorded in the vehicle data, there may be a situation of hitchhiking or using a fake license plate. Therefore, the monitoring end needs to confirm whether the fee is evaded, so it is marked as abnormal behavior data.
[0112] S900: Generate warning information based on the abnormal behavior data and the concerned information, and send it to the monitoring terminal for early warning.
[0113] In this implementation, all the above-mentioned abnormal behaviors (including but not limited to hitchhiking, blocked license plates, fake license plates, etc.) will be marked as abnormal behaviors. At the same time, historical records of fare evasion will also be marked as attention information so that they can be sent to the monitoring end. Vehicles with abnormal behaviors can be intercepted directly, or staff can be prompted to pay special attention. When the staff receives the signal, they can obtain real-time image confirmation again to intercept the abnormal fare evasion behavior in real time. Overcome the problem that the existing fare evasion behavior is mostly tracked and traced after the fare evasion behavior has occurred, which is costly and time-poor
[0114] In the above implementation, the early warning method further includes:
[0115] In the verification information, path information corresponding to each face record is generated based on the face record, the mapping relationship between the first license plate and the second license plate and the real-time image, wherein the path information includes location data and time data at the entrance and exit of the highway; after a preset time period, the payment record corresponding to the path information in the vehicle data is queried, and when the payment record is empty, it is marked as abnormal behavior data.
[0116] In the above steps, a payment record should be generated every time a vehicle passes through the entrance and exit of the highway. In order to further improve the accuracy of the warning of abnormal behavior of toll evasion, it is possible to confirm in real time whether there is a payment record corresponding to the path information (location data of the entrance and exit of the highway) when the vehicle leaves the highway exit. If not, there may be toll evasion. In this case, it is marked as abnormal behavior data, and a warning message is generated as described in S900 above and sent to the monitoring end.
[0117] This implementation uses image analysis (including face recognition and license plate recognition) to mark vehicles that may evade tolls for abnormal behavior, and notifies the monitoring end (which can be a nearby toll station) of relevant data in real time through the early warning system of the smart platform for early warning, so that nearby toll stations can take corresponding measures in advance. This reduces the cost of tracing toll evasion, is more timely, and has a better prevention effect.
[0118] Embodiment 2: This embodiment also provides a highway vehicle toll evasion abnormal behavior warning system 10, see Figure 2 ,include:
[0119] The acquisition module 101 is used to start the high-speed camera to collect real-time images at the entrance and exit of the expressway after receiving the vehicle passing signal, so as to obtain a first image set and a second image set;
[0120] Specifically, after receiving a vehicle passing signal, real-time image collection begins, that is, in the absence of an abnormality, each real-time image should contain an image of a vehicle passing.
[0121] The face recognition module 102 is used to perform face recognition on the preset positions of each real-time image using a pre-trained recognition model for the first image set and the second image set, obtain face data at the driving position in the real-time image, and correspondingly obtain the first face data set and the second face data set; wherein the recognition model is established based on the YOLO network;
[0122] The first face data set includes face data of each vehicle at the entrance of the highway, and the second face data set includes face data of each vehicle at the exit of the highway. Specifically, the Yolov5face framework in the open source platform is used for face recognition processing. It should also be noted that the recognition model autonomously identifies the face data of the driving position in the real-time image, and can determine the vehicle position in combination with target detection, and then determine the driving position based on the preset range of the vehicle position, or because the driving position is usually on the left side of the real-time image, the face information of the left half of the real-time image is identified.
[0123] The processing module 103 is used to search the first face data set for face data that matches the second face data set, generate a face record for each face data that matches, and leave the record empty if the match fails, so as to generate a face record set;
[0124] The license plate recognition module 104 is used to locate the license plate of each real-time image using the detection model for the first image set and the second image set to obtain the license plate area, perform character segmentation on the license plate area to obtain a segmented image, perform character recognition on the segmented image using the SVM model, and record the character as empty if the recognition fails, to obtain the first license plate set and the second license plate set;
[0125] Specifically, edge detection or neural network can be used to locate the license plate to realize license plate recognition, and the appropriate license plate recognition method can be selected according to the actual usage scenario.
[0126] An association module 105, configured to associate the face record set with the first license plate set and the first vehicle set based on the real-time image, to form a to-be-verified information set having a mapping relationship between a plurality of face records, a first license plate and a second license plate;
[0127] Specifically, the facial data and license plate appearing in the same real-time image can be associated into one piece of data, and then based on the facial record generated by the processing module, several pieces of associated data can be generated, which are the information to be verified.
[0128] The early warning module 106 is used to verify whether the facial data corresponding to each real-time image exceeds one, and if so, mark it as abnormal data; verify whether there is an empty data segment in the information to be verified, and if so, mark it as abnormal behavior data; call vehicle data from a preset database according to the facial record corresponding to the information to be verified; when there is historical violation data in the vehicle data, mark it as attention information; when it is verified based on the vehicle data that it is inconsistent with the first license plate or the second license plate in the information to be verified, mark it as abnormal behavior data; generate warning information based on the abnormal behavior data and the attention information, and send it to the monitoring end.
[0129] It also includes: the early warning module generates path information corresponding to each face record in the verification information according to the face record, the first license plate and the second license plate mapping relationship and the real-time image, wherein the path information includes location data and time data at the entrance and exit of the highway; after a preset time period, the payment record corresponding to the path information in the vehicle data is queried, and when the payment record is empty, it is marked as abnormal behavior data.
[0130] In this embodiment, when more than one facial data is recognized in the real-time image, based on the above-mentioned known facial data recognized in the driver's seat in this scheme, if more than one is recognized, it means that more than one vehicle has passed by at this time, and there may be abnormal behavior of hitchhiking to evade fees; and if there is a data segment in the information to be verified that is empty, it may be that any of the first license plate data / second license plate data / first face data / second face data cannot be recognized, and there may be a behavior of the license plate being blocked or hitchhiking to achieve the purpose of evading fees. If the recognized license plate is inconsistent with the license plate recorded in the vehicle data, hitchhiking or using a duplicate license plate may occur. When the payment record corresponding to the path information in the vehicle data is queried after a preset time period, when the payment record is empty, it means that the payment information is inconsistent with the path, and there may also be evasion of fees. Therefore, it is necessary to issue an early warning message for autonomous interception, or the monitoring end confirms whether evasion of fees occurs, and it is therefore marked as abnormal behavior data so that the abnormal evasion of fees can be intercepted in real time. This overcomes the problem that the existing evasion investigation and tracking is mostly carried out after the evasion has occurred, which is costly and has a poor timeliness.
[0131] It should be noted that the embodiments of the present invention have better practicability and do not impose any form of limitation on the present invention. Any technician familiar with the field may use the technical content disclosed above to change or modify it into an equivalent effective embodiment. However, any modification or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. A method for early warning of abnormal behavior of toll evasion by vehicles on highways, characterized in that: include: After receiving a vehicle passing signal, the high-speed camera is started to collect real-time images at the entrance and exit of the highway respectively to obtain a first image set and a second image set; For the first image set and the second image set, a pre-trained recognition model is used to perform face recognition at a preset position of each real-time image to obtain face data at the driving position in the real-time image, and a first face data set and a second face data set are correspondingly obtained; wherein the recognition model is established based on a YOLO network; Searching for facial data that matches the second facial data set in the first facial data set, generating a facial record for each facial data that matches, and leaving the record empty if the match fails, to generate a facial record set; For the first image set and the second image set, the detection model is used to locate the license plate of each real-time image to obtain the license plate area, and the license plate area is segmented to obtain the segmented image, and the SVM model is used to perform character recognition on the segmented image. If the recognition fails, the record is empty to obtain the first license plate set and the second license plate set; Associating the face record set with the first license plate set and the second license plate set based on the real-time image to form a to-be-verified information set having a mapping relationship between a plurality of face records, the first license plate and the second license plate; For any information to be verified in the set of information to be verified: Check whether the face data corresponding to each real-time image exceeds one, and if so, mark it as abnormal data; Check whether there is a data segment in the information to be verified that is empty, and if so, mark it as abnormal behavior data; Retrieving vehicle data from a preset database according to the face record corresponding to the information to be verified; When there is historical violation data in the vehicle data, it is marked as attention information; When the vehicle data is verified to be inconsistent with the first license plate or the second license plate in the information to be verified, it is marked as abnormal behavior data; Generate warning information based on the abnormal behavior data and the attention information, and send it to the monitoring terminal.
2. The early warning method according to claim 1, characterized in that: The method of performing face recognition on a preset position of each real-time image using a pre-trained recognition model to obtain face data at a driving position in the real-time image includes: Build an initial model based on the YOLO network, and obtain the target model after training; For any real-time image, in the target model, the real-time image is sampled by using a starting module; The facial landmarks are regressed and the loss function is used to identify the key points of the face in the sampled real-time image. A spatial pyramid pooling module with a preset kernel is used for pooling, and the face data at the driving position in the real-time image is output accordingly.
3. The early warning method according to claim 1, characterized in that: The method of using the detection model to locate the license plate of each real-time image to obtain the license plate area includes: For any real-time image, perform enhancement preprocessing on the real-time image and then binarize it to obtain a first image; Using an edge detection operator to perform edge detection on the first image, and performing region expansion to obtain a second image containing multiple connected domains; A connected domain including the license plate is identified based on the second image to obtain a license plate area.
4. The early warning method according to claim 1, characterized in that: The method of using the detection model to locate the license plate of each real-time image to obtain the license plate area includes: For any real-time image, edge detection is performed on the real-time image, and binarization is performed to obtain an edge image; The edge image is divided into a number of sub-blocks, and the sub-blocks are collected using a sliding window and the license plate boundary is determined based on a neural network to obtain a license plate area.
5. The early warning method according to claim 1, characterized in that: The step of performing character segmentation on the license plate area to obtain a segmented image includes: Binarizing the license plate area to obtain a binary image and calculating a smooth curve of vertical integral projection; Determine a trough position sequence according to the smooth curve, and extract two adjacent troughs in the trough position sequence as left and right boundaries respectively to obtain a plurality of rectangular areas; Each rectangular area is split or merged according to preset rules to obtain a segmented image.
6. The early warning method according to claim 1, characterized in that: The method of using the SVM model to perform character recognition on the segmented image includes: Obtain a preset data set, and mark each data in the data set to generate a training set; The training set is used to call the training instruction to train the SVM model and store it as a target file; The target file is called to perform character recognition on the segmented image based on the directional gradient histogram of the segmented image.
7. The early warning method according to claim 1, characterized in that: In the verification information, path information corresponding to each face record is generated according to the face record, the mapping relationship between the first license plate and the second license plate and the real-time image, wherein the path information includes location data and time data at the entrance and exit of the expressway; After a preset time period, the payment record corresponding to the path information in the vehicle data is queried, and when the payment record is empty, it is marked as abnormal behavior data.
8. The early warning method according to claim 1, characterized in that: Also includes: A feedback signal confirming the fee evasion behavior is received from the monitoring terminal, and corresponding abnormal behavior data is acquired based on the feedback signal to update the preset database.
9. A highway vehicle toll evasion abnormal behavior warning system, characterized in that: include: The acquisition module is used to start the high-speed camera to collect real-time images at the entrance and exit of the expressway after receiving the vehicle passing signal, so as to obtain a first image set and a second image set; A face recognition module, for the first image set and the second image set, using a pre-trained recognition model to perform face recognition at a preset position of each real-time image, to obtain face data at the driving position in the real-time image, and correspondingly obtain a first face data set and a second face data set; wherein the recognition model is established based on a YOLO network; A processing module, used for searching the first face data set for face data that matches the second face data set, generating a face record for each face data that matches, and leaving the record empty if the match fails, so as to generate a face record set; A license plate recognition module is used to locate the license plate of each real-time image using a detection model for the first image set and the second image set to obtain a license plate area, and to perform character segmentation on the license plate area to obtain a segmented image, and to perform character recognition on the segmented image using a SVM model, and record a null value if recognition fails, to obtain a first license plate set and a second license plate set; an associating module, configured to associate the face record set with the first license plate set and the second license plate set based on the real-time image, so as to form a to-be-verified information set having a mapping relationship between a plurality of face records, the first license plate and the second license plate; The early warning module is used to verify whether the facial data corresponding to each real-time image exceeds one, and if so, mark it as abnormal data; verify whether there is an empty data segment in the information to be verified, and if so, mark it as abnormal behavior data; call vehicle data from a preset database according to the facial record corresponding to the information to be verified; when there is historical violation data in the vehicle data, mark it as attention information; when it is verified based on the vehicle data that it is inconsistent with the first license plate or the second license plate in the information to be verified, mark it as abnormal behavior data; generate warning information based on the abnormal behavior data and the attention information, and send it to the monitoring end.
10. The early warning system according to claim 9, characterized in that: Also includes: The early warning module generates path information corresponding to each face record in the verification information according to the face record, the first license plate and the second license plate mapping relationship and the real-time image, wherein the path information includes the location data and time data of the entrance and exit of the expressway; After a preset time period, the payment record corresponding to the path information in the vehicle data is queried, and when the payment record is empty, it is marked as abnormal behavior data.
Citation Information
Patent Citations
Method for preventing fee evasion on highways on basis of face recognition technologies
CN104574547A
Multi-mode driver and vehicle illegal behavior monitoring method and system
CN111444843A
Intelligent license plate character recognition system
CN111652210A
Method and system for detecting and identifying illegal ETC use behaviors of others
CN113313051A
Guiding type station crossing prevention intersection for highway, control method and storage medium
CN114457637A
