A highway two passengers and one dangerous vehicle monitoring method and system

Abstract

The application provides a highway two-passenger and dangerous goods vehicle supervision method and system, relates to the two-passenger and dangerous goods vehicle management technical field, and comprises the following steps: collecting an initial image of a vehicle at a highway entrance; performing license plate recognition and vehicle feature recognition according to the initial image to obtain license plate information and a vehicle feature image; collecting images at multiple places on the highway to obtain a plurality of real-time images; performing license plate recognition and vehicle feature recognition on the real-time images to obtain real-time license plate data and real-time vehicle feature images; if the license plate recognition is successful, determining whether the vehicle is a two-passenger and dangerous goods vehicle according to the real-time license plate data to obtain real-time vehicle data; if the license plate recognition fails, performing multi-size feature extraction matching on the real-time vehicle feature images and obtaining real-time vehicle data; performing path verification to generate a real-time path and store the real-time path to the dynamic database, thereby solving the problem that there is a lack of position tracking and supervision of two-passenger and dangerous goods vehicles.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of two-passenger and one-dangerous vehicle management, and particularly relates to a highway two-passenger and one-dangerous vehicle supervision method and system. BACKGROUND

[0002] In recent years, with the great progress of highway construction in China, the highway construction mileage is getting longer and longer. The safe driving of the highway also faces great challenges, especially in the complex terrain areas such as Yunnan, Guizhou and Sichuan. The real-time supervision of the two-passenger and one-dangerous vehicle on the highway will also face greater challenges.

[0003] At present, the trajectory of the two-passenger and one-dangerous vehicle depends on the vehicle-mounted GPS. The GPS data is uploaded to the national key vehicle supervision platform, and then the platform issues. The process causes the delay of positioning data, so that the highway operation personnel cannot track and supervise the position of the two-passenger and one-dangerous vehicle. SUMMARY

[0004] In order to overcome the above technical defects, the purpose of the present application is to provide a highway two-passenger and one-dangerous vehicle supervision method and system, which solves the problem that the existing two-passenger and one-dangerous vehicle lacks position tracking and supervision.

[0005] The present application discloses a highway two-passenger and one-dangerous vehicle supervision method, comprising:

[0006] Collecting an initial image of a vehicle at a highway entrance;

[0007] Performing license plate recognition and vehicle feature recognition according to the initial image, obtaining license plate information and vehicle feature images, and corresponding storage in a dynamic database;

[0008] Image collection at multiple places on the highway to obtain a plurality of real-time images;

[0009] Performing license plate recognition and vehicle feature recognition on the real-time images to obtain real-time license plate data and real-time vehicle feature images;

[0010] If the license plate recognition is successful, determining whether the vehicle is a two-passenger and one-dangerous vehicle according to the real-time license plate data;

[0011] When it is determined that the vehicle is a two-passenger and one-dangerous vehicle, obtaining real-time vehicle data according to the real-time images;

[0012] If the license plate recognition fails, performing multi-size feature extraction on the real-time vehicle feature images, and performing similarity matching in the dynamic database based on the results obtained by feature extraction, when the matching is successful, obtaining real-time vehicle data according to the real-time images;

[0013] The path is checked based on each real-time vehicle data, and a real-time path is generated and stored in the dynamic database.

[0014] Preferably, the multi-size feature extraction is performed on the real-time vehicle feature image, and a similar match is performed in the dynamic database based on the obtained results of the feature extraction, including:

[0015] The real-time vehicle data is cropped to obtain real-time vehicle data of three sizes respectively;

[0016] The real-time vehicle data of three sizes is feature-extracted and quantized to obtain a first feature set;

[0017] Any vehicle feature image is obtained from the dynamic database, and a corresponding feature set is obtained as a second feature set;

[0018] The first feature set and the second feature set are inverse quantized, the real-time vehicle data and the reference image of the same size are feature-compared, and the similarity of three sizes is obtained respectively;

[0019] The similarities of the three sizes are weighted and averaged to obtain a similarity result;

[0020] When the similarity result exceeds a threshold value, the real-time vehicle data matches the reference image.

[0021] Preferably, the real-time vehicle data of three sizes is feature-extracted and quantized to obtain a first feature set respectively, including:

[0022] The real-time vehicle data of three sizes is input into the VGG-16 network, and the output features after the third convolution of the fourth convolution block of the VGG-16 network are obtained and quantized to obtain the first feature set.

[0023] Preferably, the path is checked based on each real-time vehicle data, and a real-time path is generated and stored in the dynamic database, including:

[0024] A plurality of initial paths are generated according to a plurality of real-time vehicle data, the length and time of each interval in each initial path are obtained to calculate the vehicle speed in each interval;

[0025] For any interval, it is judged whether the vehicle speed is within a preset range;

[0026] If yes, the offset is calculated according to the length and time of the interval; wherein the offset is the difference between the vehicle speed in the interval and the average speed of the path;

[0027] The offset of each interval is obtained in a loop to obtain the offset of each initial path;

[0028] adopting the initial path corresponding to the least offset as the real-time path, and storing the path data in the dynamic database;

[0029] If not, deleting the corresponding real-time vehicle data.

[0030] Preferably, the offset is calculated according to the following formula:

[0031]

[0032] wherein x is the offset; L i is the length of the ith interval, T i is the time of the ith interval.

[0033] i=1, 2, …, n, wherein n is the number of intervals.

[0034] Preferably, the determining whether the vehicle is a two-passenger-one-dangerous vehicle according to the real-time license plate data comprises:

[0035] acquiring the color and number in the real-time license plate data;

[0036] determining whether the vehicle is a two-passenger-one-dangerous vehicle according to the color or number.

[0037] Preferably, after the feature extraction and quantification of the real-time vehicle data under three sizes to obtain the first feature set, the method further comprises:

[0038] uploading the first feature set to the cloud for storage.

[0039] The application further provides a highway two-passenger-one-dangerous vehicle supervision system, comprising:

[0040] a preprocessing module, configured to collect an initial image of a vehicle at an entrance of a highway; perform license plate recognition and vehicle feature recognition according to the initial image to obtain license plate information and a vehicle feature image, and store the same in a dynamic database correspondingly;

[0041] a collecting module, configured to collect images at multiple positions on the highway to obtain a plurality of real-time images;

[0042] an identifying module, configured to perform license plate recognition and vehicle feature recognition on the real-time images to obtain real-time license plate data and real-time vehicle feature images;

[0043] a first matching module, configured to determine whether the vehicle is a two-passenger-one-dangerous vehicle according to the real-time license plate data if the license plate recognition is successful; and acquire real-time vehicle data according to the real-time images when it is determined that the vehicle is a two-passenger-one-dangerous vehicle.

[0044] a second matching module, configured to, if the license plate recognition fails, perform multi-size feature extraction on the real-time vehicle feature image, and perform similar matching in the dynamic database based on a result obtained through the feature extraction, and when the matching succeeds, acquire real-time vehicle data according to the real-time image;

[0045] a processing module, configured to perform path verification based on each real-time vehicle data to generate a real-time path and store the real-time path to the dynamic database.

[0046] Preferably, the method further comprises:

[0047] a cloud server, configured to provide the dynamic database and store the first feature set.

[0048] After the above technical solution is adopted, compared with the prior art, the following beneficial effects are achieved:

[0049] In the scheme, first, information of each two-passenger-and-dangerous-goods vehicle entering a high-speed highway entrance is obtained as basic information, then a plurality of real-time images are collected during high-speed highway travel, real-time license plate data and real-time vehicle feature images are recognized and obtained, license plate matching is performed, if the matching fails, vehicle feature matching is performed, if the matching succeeds, real-time vehicle data can be acquired, finally, the acquired real-time vehicle data is subjected to path verification, unreasonable real-time vehicle data caused by overspeed or errors of the algorithm itself is eliminated, finally, a real-time path is generated and stored in the dynamic database, and real-time dynamic supervision of the two-passenger-and-dangerous-goods vehicle is realized. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 a flowchart of the high-speed highway two-passenger-and-dangerous-goods vehicle supervision method embodiment one of the application;

[0051] Figure 2 a structural schematic diagram of a network for feature comparison similarity in the flowchart of the high-speed highway two-passenger-and-dangerous-goods vehicle supervision method embodiment one of the application;

[0052] Figure 3 a flowchart for embodying path verification based on each real-time vehicle data and generating a real-time path stored to the dynamic database in the flowchart of the high-speed highway two-passenger-and-dangerous-goods vehicle supervision method embodiment one of the application;

[0053] Figure 4 a module schematic diagram of the high-speed highway two-passenger-and-dangerous-goods vehicle supervision system embodiment two of the application.

[0054] REFERENCE SIGNS:

[0055] 9highway two passengers and dangerous goods vehicle supervision system; 91-preprocessing module; 92-acquisition module; 93-identification module; 94-first matching module; 95-second matching module; 96-processing module; 97-cloud server. DETAILED DESCRIPTION

[0056] The advantages of the present application are further set forth in the description that follows, and will be appreciated by those skilled in the art upon reading and understanding the following detailed description.

[0057] Exemplary embodiments are described herein below with reference to the accompanying drawings. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more embodiments. It can be evident, however, that the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring the description of the one or more embodiments.

[0058] The terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the description of the present application and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It also will be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0059] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy among the information. These terms are used only to distinguish one from another. For example, a first information can be termed a second information, and, similarly, a second information can be termed a first information, without departing from the scope of the present application. As used herein, the term "if' can be construed to mean "when" or "upon" or "in response to determining" terms denoting the occurrence of an action.

[0060] In the description of the present application, it should be understood that the terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like refer to the orientation or positional relationship in the drawings shown by the present application, and are only used to facilitate the description of the present application and simplify the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0061] In the description of the present application, unless otherwise specified and limited, it is necessary to explain that the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be mechanical connection or electrical connection, it can be the communication inside two elements, it can be direct connection or indirect connection through intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0062] In the following description, the suffix such as "module", "component" or "unit" used to represent elements is only for the convenience of the description of the present application, and has no specific meaning in itself. Therefore, "module" and "component" can be used mixedly.

[0063] Embodiment one: a highway two passengers and dangerous goods vehicle supervision method, refer to Figures 1-3 , comprising the following steps:

[0064] S100: collecting the initial image of the vehicle at the entrance of the highway;

[0065] Specifically, the present embodiment is used for monitoring two passengers and dangerous goods vehicles, that is, managing their real-time paths, so the image is collected at the entrance of the highway, that is, the initial image. Generally, due to the particularity of the entrance of the highway, that is, the setting of the toll station, the image collection device such as camera is arranged at the toll station, so that the front and clear image of the vehicle can be collected, and the accurate information of the vehicle can be obtained at this time.

[0066] S200: performing license plate recognition and vehicle feature recognition according to the initial image, obtaining license plate information and vehicle feature image, and corresponding storage in dynamic database;

[0067] In the present embodiment, the target detection network in the prior art can be used for identification and vehicle recognition, including but not limited to RCNN series, SSD, Yolo series network. It needs to be explained here that collection can be controlled according to the charging condition of the toll station, or it can be determined whether to store in the dynamic database according to the color of the recognized license plate. The license plate information and vehicle feature image generated here are basic information, so that the subsequent collected images can find the matched confirmation of the same vehicle and obtain its driving path.

[0068] S300: image collection at multiple places on the highway to obtain several real-time images;

[0069] In the above steps, the image collection device set on the existing highway can be directly obtained, that is, the real-time image of the vehicle in the process is obtained, so as to determine the path in the following steps and store it in the dynamic database, so as to realize the supervision of two passengers and dangerous goods vehicles.

[0070] S400: performing license plate recognition and vehicle feature recognition on the real-time image to obtain real-time license plate data and real-time vehicle feature image;

[0071] In the above steps, the same target detection network as in step S200 can be used for license plate recognition and vehicle feature recognition, or a different target detection network can be used for recognition. The target detection network in the prior art can be used, and specific details are not described here. The purpose is to obtain license plate data and match it with the license plate data recognized at the entrance to determine the driving path of the vehicle. However, it should be noted that since the image is obtained during vehicle driving at this time, there may be a blur, so the recognition of the license plate may fail, and therefore different processing methods are proposed for different situations in the following embodiment.

[0072] It should be particularly noted that the above real-time vehicle data includes but is not limited to generating vehicle number (unique number), license plate, license plate color, latitude and longitude, and stake number (used to determine the distance of driving into the highway entrance).

[0073] S500: If the license plate recognition is successful, determining whether the vehicle is a two-passenger and one-dangerous vehicle according to the real-time license plate data;

[0074] Specifically, the determination of whether the vehicle is a two-passenger and one-dangerous vehicle according to the real-time license plate data includes:

[0075] Obtaining the color and number in the real-time license plate data; determining whether the vehicle is a two-passenger and one-dangerous vehicle according to the color or number. As an explanation, a two-passenger and one-dangerous vehicle has a special license plate color and number. After extracting the real-time license plate data, it is matched with the preset license plate color or number. If the match is consistent, it is, if the match is not consistent, it is not, if it is, record the position information to generate the driving track finally.

[0076] S600: When it is determined that the vehicle is a two-passenger and one-dangerous vehicle, obtaining real-time vehicle data according to the real-time image;

[0077] Specifically, in the above steps, the position of the vehicle can be determined according to the vehicle data, and the path data of the vehicle can be generated according to the position. Therefore, when the license plate data can be directly recognized, the real-time vehicle data can be directly obtained from the real-time image, so that the real-time vehicle data can be used to directly generate a real-time path, which is convenient for operation and can be obtained from the dynamic data road at any time to realize real-time supervision.

[0078] S700: If the license plate recognition fails, performing multi-size feature extraction on the real-time vehicle feature image, and performing similar matching in the dynamic database based on the results obtained by feature extraction. When the matching is successful, obtaining real-time vehicle data according to the real-time image;

[0079] In the embodiment, the multi-size feature extraction refers to real-time vehicle data (represented as images) in three sizes. When the size meets the small size (size_s) as an example, the bounding box detected is cropped from the image. According to the method, small (size_s), medium (size_m) and large (size_l) size vehicle appearance images are obtained, which represent the vehicle appearance captured by the camera in the long distance, medium distance and close distance. According to the method, a more comprehensive vehicle appearance can be obtained, and the finally extracted vehicle features are more comprehensive.

[0080] Specifically, the multi-size feature extraction is performed on the real-time vehicle feature image, and similarity matching is performed in the dynamic database based on the result obtained by feature extraction, including:

[0081] S710: cropping the real-time vehicle data to obtain real-time vehicle data in three sizes, respectively;

[0082] In the above step, based on the above, the cropped real-time vehicle data containing the long distance, medium distance and close distance of the vehicle can be obtained. After cropping, it can be selectively adjusted to a preset size to facilitate subsequent processing.

[0083] S720: performing feature extraction and quantization on the real-time vehicle data in three sizes to obtain a first feature set;

[0084] Specifically, the feature extraction and quantization are performed on the real-time vehicle data in three sizes to obtain a first feature set, including: inputting the real-time vehicle data in three sizes into the VGG-16 network, and obtaining the output features after the third convolution of the fourth convolution block of the VGG-16 network and quantizing to obtain the first feature set.

[0085] In the above step, the input to the VGG-16 network obtains the output features after the third convolution of the fourth convolution block of the network as the final features in the corresponding size, and then the features of the three sizes are quantized, and the quantized results of the three features are saved as the feature set (i.e. the above feature set) of the corresponding vehicle. It should be noted that the feature quantization is to reduce the demand of data on bandwidth, i.e. to increase the speed of data transmission process, to facilitate data processing of operators and to improve efficiency.

[0086] Specifically, after the feature extraction and quantization of the real-time vehicle data in three sizes to obtain the first feature set, the first feature set can also be uploaded to the cloud for storage, so that the corresponding data can be directly obtained when needed in the future.

[0087] S730: Obtain any vehicle feature image from the dynamic database and obtain the corresponding feature set as the second feature set;

[0088] As an illustration, any vehicle feature image in the dynamic database corresponds to a feature set, which is processed in step S720 as described above, and is also a feature set generated after multi-size feature extraction. As described above, feature extraction is performed on it in the process of obtaining vehicle feature data, so it can be directly stored for obtaining here. For the initial image obtained, that is, obtained at the entrance of the highway, its corresponding feature set can be obtained in advance according to the above steps, and it can be directly obtained here, which is convenient for operation.

[0089] S740: Inverse quantization is performed on the first feature set and the second feature set, and feature comparison is performed on real-time vehicle data and reference images under the same size, and similarity under three sizes is obtained respectively;

[0090] S750: Weighted average is performed on the similarity under the three sizes to obtain a similarity result;

[0091] S760: When the similarity result exceeds a threshold value, the real-time vehicle data matches the reference image.

[0092] Based on steps S740-S760 described above, when vehicle feature comparison is performed, the feature sets obtained from two vehicles (i.e., the current vehicle and the vehicle corresponding to the vehicle feature image obtained from the dynamic database) are inverse quantized to restore the three-size feature images corresponding to each vehicle, and then the features under the corresponding sizes of the two vehicles are input into a feature comparison model as a group, and the similarity of the feature images of the two vehicles under three sizes is obtained through the feature comparison model. The value obtained by weighted average of the similarity under three sizes is the final similarity of the two vehicles.

[0093] As an example but not limited, see Figure 2 The network for feature comparison similarity described above can be set as a simple twin network, and the network structures on the left and right sides are exactly the same. It has two inputs, which are the features under the corresponding sizes of the two vehicles. The network has three convolution blocks in total, and the similarity of the output features of the two is calculated after each convolution block. Finally, the three similarities obtained are averaged to obtain the final similarity of the two under the size.

[0094] S800: Based on each real-time vehicle data, path verification is performed to generate real-time paths and store them in the dynamic database.

[0095] In the above steps, if the matching is successful, it means that the real-time vehicle data and the vehicle feature image obtained from the dynamic database are data of the same vehicle, and based on this, the path data of the vehicle can be generated. If the matching fails, it means that it is not consistent, and it may not belong to the two passengers and dangerous vehicle, so it can be discarded, so the images collected by the collection device in the highway may contain several images that do not belong to the two passengers and dangerous vehicle. The path verification is performed to reduce the problem of inconsistent paths generated due to errors such as overspeed and visual recognition algorithm, and to store the actual inconsistent paths in the dynamic database. It should be noted that the path verification will generate two results, reasonable and unreasonable. The unreasonable path can be set to directly trigger the warning information, but in the present scheme, it will not be stored in the dynamic database in order to monitor, and only the reasonable one will generate the real-time path.

[0096] Specifically, the path verification is performed based on each real-time vehicle data to generate a real-time path and store it to the dynamic database, referring to Figure 3 , comprising:

[0097] S810: Generate a plurality of initial paths according to a plurality of real-time vehicle data, obtain the length and time of each interval in each initial path to calculate the vehicle speed in each interval;

[0098] Based on the above, the vehicle data includes latitude, longitude, stake number, etc., so that the entire initial path can be divided into several intervals according to the stake number, i.e. the path between two adjacent stakes is taken as an interval.

[0099] S820: For any interval, determine whether the vehicle speed is within a preset range;

[0100] Specifically, the preset range depends on the actual road section speed limit. If it is not within the range, it means that the speed is too high, i.e. the path is unreasonable, at which time a warning message can be issued and the route can be discarded, and not stored in the dynamic database. In this process, as an option, the unreasonable path data can also be removed, and if the remaining reasonable path data is greater than 2, the next step is entered, otherwise, the only reasonable path data is directly added to the dynamic library for subsequent comparison.

[0101] S830: If yes, calculate the offset according to the length and time of the interval; wherein the offset is the difference between the vehicle speed in the interval and the average speed of the path; the initial path corresponding to the smallest offset is taken as the real-time path, and stored in the path data in the dynamic database;

[0102] It should be noted that both the initial path and the real-time path need to match the license plate data and the pre-existing license plate data or vehicle data in the dynamic database to ensure that the path of a certain vehicle is generated.

[0103] Specifically, the offset is calculated according to the following formula:

[0104]

[0105] wherein x is the offset; L i is the length of the ith interval, T i is the time of the ith interval;

[0106] i = 1, 2, …, n.

[0107] As an example to describe the calculation process of the above offset:

[0108] The following table is a real-time vehicle data and the dynamic database to obtain vehicle data:

[0109] Serial number Vehicle number License plate Latitude and longitude Pile number Features Record time Record format 0001 XC002 Cloud c·xxxx2 103.12222,27.33333 K1+200 xxx 1 / 9 10:21:45 OLD 0002 XC002 Cloud c·xxxx2 103.16788,27.324344 K4+200 xxx 1 / 9 10:23:55 OLD 0004 XC002 Unknown 103.16798,27.356434 K10+300 xxx 1 / 9 10:26:45 NEW

[0110] Note: K2+400 processing for 2.4KM, other similar. Time difference converted to hours, other similar

[0111] The time is taken as the x-axis, and the time recorded at the nth point is T n . The driving mileage (pile number) is taken as the Y-axis, and the pile number recorded at the Nth point is L n , and the difference between the recent interval speed and the average value of the historical speed is recorded as the offset x.

[0112] The calculation results of the above data are as follows:

[0113] The matching result is that the offset x of the XC001 vehicle is 1.5KM / h

[0114] The matching result is that the offset x of the XC002 vehicle is 46.1KM / h

[0115] The absolute value comparison is performed. |1.5|<|46.1|. The local XC001 trajectory data is added to the dynamic library (including vehicle feature data), and the local XC002 trajectory data is deleted. Based on the above calculation of the offset, the analysis error caused by the visual algorithm of vehicle body recognition and license plate recognition is reduced, and the accuracy of the path data obtained is further increased, so as to improve the accuracy of the supervision, and the calculation of each interval is performed in this way. Discard unreasonable real-time vehicle data, and finally generate complete trajectory point records. Using the vehicle dynamic library data, combined with the road network coordinates, complete trajectory information can be generated.

[0116] S840: If not, delete the corresponding real-time vehicle data. As described above, delete the real-time vehicle data that is unreasonable or has a large offset, so as to generate a real-time path with high matching degree with the actual path. Optionally, warning information can be sent to the operator to punish the illegal behavior such as overspeed.

[0117] The management method provided by the embodiment can realize real-time dynamic management and control of "two passengers and one dangerous" vehicles based on existing high-speed cameras without adding additional roadside equipment and vehicle-mounted equipment. Real-time path generation after verification of real-time feature data can reduce vehicle misjudgment and improve the efficiency of vehicle management and control. After multi-size feature extraction of the vehicle, the problem of being unable to confirm the vehicle trajectory when the license plate cannot be monitored by the camera is solved.

[0118] Embodiment two: the present embodiment provides a highway two passengers and one dangerous vehicle management system 9, referring to Figure 4 , comprising:

[0119] The preprocessing module 91 is configured to collect an initial image of a vehicle at the entrance of the highway; perform license plate recognition and vehicle feature recognition based on the initial image to obtain license plate information and vehicle feature images, and store them in a dynamic database;

[0120] Specifically, the generated license plate information and vehicle feature images are basic information, so that the subsequent collected images can find the matching information to confirm the same vehicle and obtain its driving path.

[0121] The acquisition module 92 is configured to acquire images at multiple locations on the highway to obtain a plurality of real-time images;

[0122] Specifically, the acquisition device on the highway can be directly used.

[0123] The identification module 93 is configured to perform license plate recognition and vehicle feature recognition on the real-time images to obtain real-time license plate data and real-time vehicle feature images;

[0124] Specifically, the above step is aimed at obtaining license plate data and matching it with the license plate data identified at the entrance to determine the driving path of a vehicle.

[0125] The first matching module 94 is configured to determine whether the vehicle is a two passengers and one dangerous vehicle according to the real-time license plate data if the license plate recognition is successful; and acquire real-time vehicle data according to the real-time images when it is determined that the vehicle is a two passengers and one dangerous vehicle.

[0126] The second matching module 95 is configured to perform multi-size feature extraction on the real-time vehicle feature images if the license plate recognition fails, and perform similarity matching in the dynamic database based on the results obtained by the feature extraction, and acquire real-time vehicle data according to the real-time images when the matching is successful.

[0127] Specifically, the multi-size feature extraction refers to obtaining vehicle appearance images of small (size_s), medium (size_m) and large (size_l) sizes, which represent the vehicle appearance captured by the camera under three conditions of long distance, medium distance and short distance for feature extraction and comparison to confirm whether it is the same vehicle.

[0128] The processing module 96 is configured to perform path verification based on the real-time vehicle data to generate a real-time path and store the real-time path in the dynamic database.

[0129] The path verification is performed to reduce the problem paths such as overspeed and the error paths generated due to the visual recognition algorithm and inconsistent with the actual paths stored in the dynamic database. Specifically, an initial path can be generated based on the real-time vehicle data, and the vehicle speed in each interval is determined whether it is overspeed, and the path with the least deviation of the vehicle speed in the interval formed by the adjacent stakes and the average speed of the path is calculated as a real-time path to be stored in the dynamic database, so that the operation personnel can directly obtain the driving route of the two-passenger-one-dangerous vehicle from the dynamic database for supervision.

[0130] Specifically, the method further comprises:

[0131] The cloud server 97 is configured to provide the dynamic database and store the first feature set. The first feature set can be directly stored in the dynamic database or stored in another database.

[0132] In the embodiment, the pre-processing module is used to obtain the information of each two-passenger-one-dangerous vehicle entering the high-speed highway entrance as basic information, and then the acquisition module is used to obtain a plurality of real-time images during the driving process on the highway, and real-time license plate data and real-time vehicle feature images are recognized. First, the first matching module is used for license plate verification, and if the verification is successful, the real-time vehicle data is obtained, and if the verification fails, the second matching module is used for vehicle feature verification, and if the verification is successful, the real-time vehicle data can be obtained. Finally, the processing module is used to perform path verification on the obtained real-time vehicle data, to eliminate unreasonable real-time vehicle data caused by overspeed or errors in the algorithm itself, and finally to generate a real-time path and store the real-time path in the dynamic database, to realize real-time dynamic supervision of the two-passenger-one-dangerous vehicle.

[0133] It should be noted that the embodiments of the present application have better implementation, and do not limit the present application in any form. Any skilled person in the art can change or modify the above disclosed technical content to equivalent effective embodiments, as long as it does not deviate from the technical solution of the present application. Any modification or equivalent change and modification of the above embodiments according to the technical essence of the present application are still within the scope of the technical solution of the present application.

Claims

1. A method for supervising passenger and hazardous goods vehicles on highways, characterized in that, include: Acquire initial images of the vehicle located at the highway entrance; Based on the initial image, license plate recognition and vehicle feature recognition are performed to obtain license plate information and vehicle feature images, which are then stored in a dynamic database. Image acquisition was conducted at multiple locations along the highway, resulting in several real-time images. The real-time images are subjected to license plate recognition and vehicle feature recognition to obtain real-time license plate data and real-time vehicle feature images; If the license plate recognition is successful, determine whether the vehicle is a passenger vehicle or a dangerous goods vehicle based on the real-time license plate data. Once the vehicle is determined to be a passenger vehicle and a dangerous goods vehicle, real-time vehicle data is obtained based on the real-time image. If license plate recognition fails, multi-size feature extraction is performed on the real-time vehicle feature image, and similarity matching is performed in the dynamic database based on the results of feature extraction. When the matching is successful, real-time vehicle data is obtained based on the real-time image. Route verification is performed based on real-time vehicle data to generate real-time routes that are stored in the dynamic database.

2. The regulatory method according to claim 1, characterized in that, The step of extracting multi-size features from the real-time vehicle feature image and performing similarity matching in the dynamic database based on the results of feature extraction includes: The real-time vehicle data is cropped to obtain real-time vehicle data in three different sizes. Feature extraction and quantization are performed on real-time vehicle data at three different sizes to obtain the first feature set; Obtain any vehicle feature image from the dynamic database, and obtain the corresponding feature set as the second feature set; The first and second feature sets are reverse quantized, and the features of real-time vehicle data and reference images at the same size are compared to obtain the similarity at the three sizes respectively. The similarity scores at the three dimensions are weighted and averaged to obtain the similarity result. When the similarity result exceeds the threshold, the real-time vehicle data is matched with the reference image.

3. The regulatory method according to claim 2, characterized in that, Feature extraction and quantization are performed on real-time vehicle data at three different scales to obtain the first feature set, including: Real-time vehicle data at three different sizes are input into the VGG-16 network, and the output features of the third convolution of the fourth convolutional block in the VGG-16 network are obtained and quantized to obtain the first feature set.

4. The regulatory method according to claim 1, characterized in that, The step of verifying the route based on real-time vehicle data and generating a real-time route which is then stored in the dynamic database includes: Multiple initial paths are generated based on multiple real-time vehicle data, and the length and time of each interval in each initial path are obtained to calculate the vehicle speed in each interval. For any given interval, determine whether the vehicle speed is within a preset range; If so, the offset is calculated based on the length of the interval and the time; wherein the offset is the difference between the vehicle speed and the average speed of the path within the interval; Loop through each interval to obtain the offset of each initial path; The initial path with the smallest offset is used as the real-time path to store path data in the dynamic database; If not, delete the corresponding real-time vehicle data.

5. The regulatory method according to claim 4, characterized in that: The offset is calculated using the following formula: Where x is the offset; L i Let T be the length of the i-th interval. i Let i be the time interval of the i-th interval; i = 1, 2, ..., n, where n is the number of intervals.

6. The regulatory method according to claim 1, characterized in that, The step of determining whether a vehicle is a passenger vehicle or a dangerous goods vehicle based on real-time license plate data includes: Obtain the color and number from the real-time license plate data; The vehicle is determined to be a passenger vehicle or a dangerous goods vehicle based on its color or number.

7. The regulatory method according to claim 2, characterized in that, After extracting and quantizing features from real-time vehicle data at three different scales to obtain the first feature set, the following is included: The first feature set is uploaded to the cloud for storage.

8. A monitoring system for passenger and hazardous goods vehicles on highways, characterized in that, include: The preprocessing module is used to acquire initial images of vehicles located at highway entrances; Based on the initial image, license plate recognition and vehicle feature recognition are performed to obtain license plate information and vehicle feature images, which are then stored in a dynamic database. The acquisition module is used to acquire images at multiple locations along the highway and obtain several real-time images. The recognition module is used to perform license plate recognition and vehicle feature recognition on the real-time image to obtain real-time license plate data and real-time vehicle feature images; The first matching module is used to determine whether the vehicle is a passenger vehicle or a dangerous goods vehicle based on real-time license plate data if the license plate recognition is successful; when the vehicle is determined to be a passenger vehicle or a dangerous goods vehicle, real-time vehicle data is obtained based on the real-time image. The second matching module is used to perform multi-size feature extraction on the real-time vehicle feature image if the license plate recognition fails, and to perform similar matching in the dynamic database based on the result of feature extraction. When the matching is successful, real-time vehicle data is obtained based on the real-time image. The processing module is used to perform path verification based on real-time vehicle data to generate real-time paths and store them in the dynamic database.

9. The monitoring system according to claim 8, characterized in that, Also includes: A cloud server is used to provide a dynamic database and store the first feature set.

Images