Vision-based road section vehicle information detection method and related device

Abstract

The present application discloses a method and related apparatus for detecting vehicle information on a road section based on vision. The method comprises obtaining video clips at both ends of the road section to be detected; performing multi-target tracking on each video clip to obtain a vehicle trajectory set; drawing a target determination line on the video interface of the camera, and determining the camera's incoming vehicle information set and outgoing vehicle information set based on the target determination line and the vehicle trajectory set, respectively extracting features from each representative image of each vehicle in the incoming vehicle information set and the outgoing vehicle information set to obtain an incoming vehicle feature set and an outgoing vehicle feature set; and determining the road section vehicle information of the road section to be detected based on each incoming vehicle feature set and the outgoing vehicle feature set. The present application determines the road section vehicle information by processing the video clips captured by existing cameras. This allows detection of the road section vehicle information without the need for additional deployment points or maintenance, thereby saving the resources and personnel costs required for road condition detection.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a vision-based road section vehicle information detection method and related devices. Background Art

[0002] Urban roads often have designated roadside parking spaces and illegal parking in non-parking spaces. To detect illegal parking in these spaces, dedicated cameras are currently deployed in parking spaces or along the road. These cameras are then aimed at parking spaces or locations where illegal parking may occur, and parking detection is performed based on the video images captured by the dedicated cameras. However, this detection method requires additional work such as selecting locations, purchasing cameras, and maintaining these locations, which incurs significant resource and labor costs.

[0003] Therefore existing technology still needs to be improved and improved. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a vision-based road section vehicle information detection method and related devices in response to the shortcomings of the existing technology.

[0005] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides a method for detecting vehicle information on a road section based on vision, the method comprising:

[0006] Obtain video clips taken by cameras installed at both ends of the road section to be detected;

[0007] Perform multi-target tracking on each acquired video clip to obtain a set of vehicle trajectories corresponding to each video clip;

[0008] For each camera, a target determination line is drawn on the video interface of the camera, and based on the target determination line and the vehicle trajectory set corresponding to the camera, an incoming vehicle information set and an outgoing vehicle information set corresponding to the camera are determined, wherein the incoming vehicle information set and the outgoing vehicle information set both include vehicle representative images;

[0009] For each camera, feature extraction is performed on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain an incoming vehicle feature set and an outgoing vehicle feature set, wherein the incoming vehicle feature set and the outgoing vehicle feature set both include vehicle features and license plate information;

[0010] Based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera, the road section vehicle information of the road section to be detected is determined.

[0011] The vision-based road section vehicle information detection method, wherein the vehicle trajectory set includes the vehicle trajectory of each target vehicle in the video clip, the vehicle tracking identifier, the vehicle detection frame image of each trajectory point in each vehicle trajectory, and the video timestamp.

[0012] The vision-based road section vehicle information detection method, wherein the determining of the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera based on the target determination line and the vehicle trajectory set corresponding to the camera specifically includes:

[0013] For each vehicle trajectory in the vehicle trajectory set, determining whether the vehicle trajectory intersects the target determination line;

[0014] When the vehicle trajectory intersects the target determination line, the driving direction of the vehicle trajectory is obtained;

[0015] Adding the vehicle trajectory with the driving direction being the driving direction to the candidate driving-in trajectory set, and adding the vehicle trajectory with the driving direction being the driving direction to the candidate driving-out trajectory set;

[0016] For each candidate vehicle trajectory in the candidate entry trajectory set and the candidate exit trajectory set, determining a trajectory speed corresponding to the candidate vehicle trajectory according to the starting coordinates, the ending coordinates, and the number of trajectory points of the candidate vehicle trajectory;

[0017] A candidate vehicle trajectory is selected whose trajectory speed satisfies a speed condition and whose number of trajectory points satisfies a number condition, and an incoming vehicle information set and an outgoing vehicle information set are determined according to the selected candidate vehicle trajectory.

[0018] The vision-based road section vehicle information detection method, wherein the vehicle representative image is the vehicle detection frame image with the largest detection frame among the vehicle detection frame images of each track point of the vehicle track.

[0019] The vision-based road section vehicle information detection method, wherein the feature extraction of each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain the incoming vehicle feature set and the outgoing vehicle feature set specifically includes:

[0020] For each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera, extracting a feature vector of the vehicle representative image using a vehicle recognition algorithm, and detecting the license plate information of the vehicle representative image using a license plate detection algorithm;

[0021] The set consisting of the feature vectors and license plate information of each target vehicle in the incoming vehicle information set is used as the incoming vehicle feature set, and the set consisting of the feature vectors and license plate information of each outgoing vehicle in the outgoing vehicle information set is used as the outgoing vehicle feature set.

[0022] The vision-based road section vehicle information detection method, wherein determining the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera specifically includes:

[0023] For each target vehicle in the incoming vehicle information set of each camera, select a reference vehicle whose exit time is later than the target vehicle's entry time from the outgoing vehicle information set of each camera, and calculate the similarity between each reference vehicle and the target vehicle based on the feature vector and license plate information to obtain a similarity matrix between each incoming vehicle information set and each outgoing vehicle information set;

[0024] A bipartite graph matching algorithm is used to perform vehicle matching on the similarity matrix between each incoming vehicle information set and each outgoing vehicle information set to obtain a vehicle matching dataset corresponding to each incoming vehicle information set. The vehicle matching dataset includes incoming vehicles that match outgoing vehicles, incoming vehicles that do not match outgoing vehicles, and outgoing vehicles that do not match incoming vehicles.

[0025] The road section vehicle information of the road section to be detected is determined based on the matching data set.

[0026] The vision-based road segment vehicle information detection method, wherein the incoming vehicle information set and the outgoing vehicle information set both include a video timestamp and a vehicle tracking identifier; and determining the road segment vehicle information of the road segment to be detected based on the matching data set specifically includes:

[0027] For each incoming vehicle, determining the exit time of the outgoing vehicle corresponding to the incoming vehicle based on the matching data set;

[0028] The section vehicle information of the road section to be detected is determined based on the entry time of each entering vehicle and its corresponding exit time, wherein the section vehicle information includes one or more of the vehicle stay time, the traffic volume of the section, the average travel time and the number of stops on the section.

[0029] A second aspect of an embodiment of the present application provides a vision-based road section vehicle information detection system, the system comprising:

[0030] An acquisition module is used to acquire video clips taken by cameras installed at both ends of the road section to be detected;

[0031] A tracking module is used to perform multi-target tracking on each acquired video clip to obtain a set of vehicle trajectories corresponding to each video clip;

[0032] a drawing module configured to draw, for each camera, a target determination line on a video interface of the camera, and determine, based on the target determination line and a vehicle trajectory set corresponding to the camera, an incoming vehicle information set and an outgoing vehicle information set corresponding to the camera, wherein the incoming vehicle information set and the outgoing vehicle information set both include vehicle representative images;

[0033] an extraction module configured to, for each camera, perform feature extraction on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera, respectively, to obtain an incoming vehicle feature set and an outgoing vehicle feature set, wherein the incoming vehicle feature set and the outgoing vehicle feature set both include vehicle features and license plate information;

[0034] The determination module is used to determine the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera.

[0035] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the vision-based road section vehicle information detection methods described above.

[0036] A fourth aspect of an embodiment of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor;

[0037] The communication bus realizes the connection and communication between the processor and the memory;

[0038] When the processor executes the computer-readable program, the steps in any of the above-described vision-based road section vehicle information detection methods are implemented.

[0039] Beneficial effects: Compared with the prior art, the present application provides a vision-based road section vehicle information detection method and related devices, the method comprising obtaining video clips shot by cameras installed at both ends of the road section to be detected; performing multi-target tracking on each obtained video clip to obtain a vehicle trajectory set corresponding to each video clip; drawing a target determination line on the video interface of the camera, and based on the target determination line and the vehicle trajectory set corresponding to the camera, determining the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera, respectively performing feature extraction on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain an incoming vehicle feature set and an outgoing vehicle feature set; determining the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera. The present application determines the road section vehicle information by processing the video clips captured by existing cameras, so that the road section vehicle information can be detected without the need for additional deployment points or maintenance, thereby saving the resources and personnel costs required for road condition detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0041] Figure 1 This is a flowchart of the vision-based road section vehicle information detection method provided in this application.

[0042] Figure 2 This is an example flowchart of the vision-based road section vehicle information detection method provided in this application.

[0043] Figure 3 This is a schematic diagram of the structural principle of the vision-based road section vehicle information detection system provided in this application.

[0044] Figure 4 This is a schematic diagram of the terminal device structure provided in this application. DETAILED DESCRIPTION

[0045] This application provides a method and related apparatus for detecting vehicle information on a road section based on vision. To make the purpose, technical solution, and effects of this application more clear and explicit, this application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to explain this application and are not intended to limit this application.

[0046] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0047] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0048] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not imply the order of execution. The order of execution of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of this application.

[0049] After research, the inventors discovered that urban roads often have designated roadside parking spaces and illegal parking in non-parking spaces. To detect illegal parking in roadside parking spaces and non-parking spaces, dedicated cameras are currently deployed in parking spaces or along the road. These cameras are then aimed at parking spaces or locations where illegal parking may occur, and parking detection is then performed based on the video images captured by the dedicated cameras. However, this detection method requires additional work such as selecting locations, purchasing cameras, and maintaining the locations, which requires significant resources and labor costs.

[0050] In order to solve the above problems, in an embodiment of the present application, video clips taken by cameras installed at both ends of the road section to be detected are obtained; multi-target tracking is performed on each obtained video clip to obtain a vehicle trajectory set corresponding to each video clip; a target determination line is drawn on the video interface of the camera, and based on the target determination line and the vehicle trajectory set corresponding to the camera, the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera are determined, and feature extraction is performed on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain an incoming vehicle feature set and an outgoing vehicle feature set; based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera, the road section vehicle information of the road section to be detected is determined. The present application determines the road section vehicle information by processing the video clips collected by existing cameras, so that the road section vehicle information can be detected without the need for additional deployment points or maintenance, thereby saving the resources and personnel costs required for road condition detection.

[0051] The application content will be further explained below through description of embodiments in conjunction with the accompanying drawings.

[0052] This embodiment provides a method for detecting road section vehicle information based on vision, such as Figure 1 and Figure 2 As shown, the method includes:

[0053] S10: Obtain video clips captured by cameras installed at both ends of the road section to be detected.

[0054] Specifically, the road section to be inspected is a closed road section. It is understood that the road section to be inspected has only two exits at either end, and vehicles entering through one end can only exit through that end or the other end. For example, the road section to be inspected has entrances and exits A and B, and vehicles entering through entrance and exit A can only exit through entrances A or B. Cameras are respectively installed at the front and rear ends of the road section to be inspected. For example, camera C1 is installed at the front end of the road section to be inspected, and camera C2 is installed at the rear end of the road section to be inspected. The video clips obtained then include the video clips captured by camera C1 and the video clips captured by camera C2.

[0055] In this embodiment, the two video clips are shot at the same time. For example, both video clips are shot during time period T. Of course, in actual applications, the shooting times of the two video clips may also be different. For example, video clip A is shot during time period T, while video clip B is shot one minute later than video clip A. Furthermore, the shooting time of the video clips may be greater than a preset duration threshold, where the preset duration threshold may be based on the time required for a vehicle to pass through the road section to be inspected at a target speed. The target speed may be determined based on actual needs. For example, the preset duration threshold may be a vehicle speed of 30 km / s, 20 km / s, etc.

[0056] S20: Perform multi-target tracking on each acquired video clip to obtain a vehicle trajectory set corresponding to each video clip.

[0057] Specifically, a multi-target tracking algorithm can be used to track multiple targets in each acquired video clip, for example, Yolo5+deepsort, FairMoT, etc., and each video clip is processed by the multi-target tracking algorithm so that each video clip has a corresponding vehicle trajectory set, wherein the vehicle trajectory set includes the vehicle trajectory of each target vehicle in the video clip, the vehicle tracking identifier, the vehicle detection frame image of each trajectory point in each vehicle trajectory, and the video timestamp.

[0058] S30. For each camera, draw a target determination line on the video interface of the camera, and determine an incoming vehicle information set and an outgoing vehicle information set corresponding to the camera based on the target determination line and the vehicle trajectory set corresponding to the camera.

[0059] Specifically, the target determination line is used to reflect the location information of the road end of the road section to be detected. It can be understood that the target determination line can be used to determine whether a vehicle enters or exits the road section to be detected. In one implementation, the target determination line is the position in the video interface of the camera at the road end of the road section to be detected, with one side of the target determination line indicating the inside of the road section to be detected and the other side indicating the outside of the road section to be detected.

[0060] Furthermore, both the incoming vehicle information set and the outgoing vehicle information set include vehicle representative images. That is, the incoming vehicle information set includes vehicle representative images of each target vehicle entering the road section to be detected, and the outgoing vehicle information set includes vehicle representative images of each target vehicle exiting the road section to be detected. The vehicle representative image is any vehicle detection frame image of the target vehicle captured by the camera, or it can be the vehicle detection frame image with the largest detection frame among the vehicle detection frame images of each track point in the vehicle trajectory of the target vehicle, or it can be the vehicle detection frame image with the highest definition among the vehicle detection frame images of each track point in the vehicle trajectory of the target vehicle. In a typical implementation, the vehicle representative image is the vehicle detection frame image with the largest detection frame among the vehicle detection frame images of each track point in the vehicle trajectory.

[0061] In one implementation, determining the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera based on the target determination line and the vehicle trajectory set corresponding to the camera specifically includes:

[0062] S31. For each vehicle trajectory in the vehicle trajectory set, determine whether the vehicle trajectory intersects the target determination line;

[0063] S32. When the vehicle trajectory intersects the target determination line, obtaining the driving direction of the vehicle trajectory;

[0064] S33, adding the vehicle trajectory with the driving direction being the driving direction to the candidate driving-in trajectory set, and adding the vehicle trajectory with the driving direction being the driving direction to the candidate driving-out trajectory set;

[0065] S34. For each candidate vehicle trajectory in the candidate entry trajectory set and the candidate exit trajectory set, determine a trajectory speed corresponding to the candidate vehicle trajectory according to the starting coordinates, the ending coordinates, and the number of trajectory points of the candidate vehicle trajectory;

[0066] S35 . Select a candidate vehicle trajectory whose trajectory speed satisfies a speed condition and whose number of trajectory points satisfies a number condition, and determine an incoming vehicle information set and an outgoing vehicle information set according to the selected candidate vehicle trajectory.

[0067] Specifically, the driving direction is the driving-in direction or the driving-out direction, wherein the driving-in direction refers to the direction from the area outside the road section to the area inside the road section, and the driving-out direction refers to the direction from the area inside the road section to the area outside the road section. The candidate driving-in trajectory set is a set consisting of vehicle trajectories whose driving direction is the driving-in direction of all vehicle trajectories, and the candidate driving-out trajectory set is a set consisting of vehicle trajectories whose driving direction is the driving-out direction of all vehicle trajectories. It can be understood that each camera will correspond to one candidate driving-in trajectory set and one candidate driving-out trajectory set, so that two candidate driving-in trajectory sets and two candidate driving-out trajectory sets can be obtained. For example, a camera C1 is provided at the front end of the road section to be detected, and a camera C2 is provided at the back end of the road section to be detected, then the candidate driving-in trajectory set PTraj of camera C1 can be obtained. 1_in , the candidate exit trajectory set PTraj of camera C1 1_out , the candidate entry trajectory set PTraj of camera C2 2_in , the candidate exit trajectory set PTraj of camera C2 2_out .

[0068] Furthermore, after obtaining the candidate entry trajectory set and the candidate exit trajectory set, the starting coordinates, ending coordinates and number of trajectory points of each candidate vehicle trajectory can be obtained. After obtaining the starting coordinates, ending coordinates and number of trajectory points, the trajectory speed corresponding to the candidate vehicle trajectory can be determined. For example, let the starting point coordinates of the trajectory be (x s ,y s ), the coordinates of the ending point are (x e ,y e ), the number of trajectory points is n, then the calculation formula of the trajectory speed can be:

[0069]

[0070] After obtaining the trajectory speed, the candidate vehicle trajectories are screened according to the preset speed condition and number condition to obtain the candidate vehicle trajectories that meet the speed condition and number condition. The speed condition and number condition can be set according to the actual situation. For example, the upper limit value of the point threshold t can be preset. mmax and the lower limit t min , the upper limit of trajectory speed s max and the lower limit s mmin , and then determine whether the trajectory speed is at the upper limit t mmax and the lower limit t min Is the number of trajectory points within the upper limit s? max and the lower limit s min When the trajectory speed is at the upper limit t muax and the lower limit t min When the number of trajectory points is between s and s, it means that the trajectory speed meets the speed condition.max and the lower limit s min When , it means that the number of trajectory points meets the number condition.

[0071] Furthermore, when selecting a candidate vehicle trajectory whose trajectory speed satisfies the speed condition and whose number of trajectory points satisfies the number condition, an incoming vehicle information set is determined based on the candidate vehicle trajectory whose driving direction is the incoming direction, and an outgoing vehicle information set is determined based on the candidate vehicle trajectory whose driving direction is the outgoing direction. It can be understood that determining the incoming vehicle information set based on the candidate vehicle trajectory whose driving direction is the incoming direction means selecting one from the vehicle detection frame image of each trajectory point in the candidate vehicle trajectory whose driving direction is the incoming direction as a vehicle representative image and adding it to the incoming vehicle information set. Correspondingly, determining the outgoing vehicle information set based on the candidate vehicle trajectory whose driving direction is the outgoing direction means selecting one from the vehicle detection frame image of each trajectory point in the candidate vehicle trajectory whose driving direction is the outgoing direction as a vehicle representative image and adding it to the outgoing vehicle information set. In a typical implementation, the vehicle representative image is the vehicle detection frame image with the largest detection frame among the vehicle detection frame images of each trajectory point of the candidate vehicle trajectory.

[0072] Furthermore, to facilitate road segment vehicle information detection, the incoming vehicle information set and the outgoing vehicle information set may also include a vehicle tracking identifier for the vehicle trajectory and a video timestamp corresponding to the vehicle representative image, so as to facilitate identification of the target vehicle corresponding to the vehicle trajectory and the time when the target vehicle entered or exited the road segment to be detected. Of course, in actual applications, the incoming vehicle information set and the outgoing vehicle information set may also include other information, such as two representative vehicle images, one of which is the vehicle detection frame image with the largest detection frame, and the other is the vehicle detection frame image with the highest resolution.

[0073] S40. For each camera, feature extraction is performed on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain an incoming vehicle feature set and an outgoing vehicle feature set, wherein the incoming vehicle feature set and the outgoing vehicle feature set both include vehicle features and license plate information.

[0074] Specifically, the feature vector is used to reflect the vehicle characteristics of the target vehicle corresponding to the vehicle trajectory, and the target vehicle can be determined based on the feature vector. The license plate information is used to identify the license plate number of the target vehicle corresponding to the vehicle trajectory. The license plate information may include the license plate number information, or the license plate number information and the license plate number confidence level. In a typical implementation, the license plate information includes the license plate number information and the license plate number confidence level. The license plate number confidence level reflects the credibility of the license plate number information. A higher license plate number confidence level indicates a higher credibility of the license plate information, while a lower license plate number confidence level indicates a lower credibility of the license plate information.

[0075] In one implementation of this embodiment, extracting features from each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain the incoming vehicle feature set and the outgoing vehicle feature set specifically includes:

[0076] For each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera, extracting a feature vector of the vehicle representative image using a vehicle recognition algorithm, and detecting the license plate information of the vehicle representative image using a license plate detection algorithm;

[0077] The set consisting of the feature vectors and license plate information of each target vehicle in the incoming vehicle information set is used as the incoming vehicle feature set, and the set consisting of the feature vectors and license plate information of each outgoing vehicle in the outgoing vehicle information set is used as the outgoing vehicle feature set.

[0078] Specifically, when extracting features from representative vehicle images, a vehicle re-identification algorithm or a trained vehicle recognition model can be used. The extracted vehicle features can be feature vectors or vehicle feature maps. License plate information can be obtained by detecting representative vehicle images using a license plate detection and recognition algorithm, where the license plate number information is a predicted string p. When no license plate information is detected, string p is set to None, and the license plate number confidence level is set to 0. Furthermore, the license plate detection and recognition algorithm can utilize existing algorithms or a trained detection network model.

[0079] S50 , determining the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera.

[0080] Specifically, the road section vehicle information includes one or more of the following: vehicle dwell time, road section traffic volume, average travel time, and number of stops on the road section. It is understood that after determining the entry vehicle feature set and the exit vehicle feature set, it can be determined based on the entry vehicle feature set and the exit vehicle feature set whether the vehicle exits the road section after entering the road section to be detected, thereby determining whether the target vehicle remains on the road section to be detected.

[0081] In one implementation of this embodiment, determining the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera specifically includes:

[0082] For each target vehicle in the incoming vehicle information set of each camera, select a reference vehicle whose exit time is later than the target vehicle's entry time from the outgoing vehicle information set of each camera, and calculate the similarity between each reference vehicle and the target vehicle based on the feature vector and license plate information to obtain a similarity matrix between each incoming vehicle information set and each outgoing vehicle information set;

[0083] Using a bipartite graph matching algorithm, the similarity matrix between each incoming vehicle information set and each outgoing vehicle information set is used to perform vehicle matching to obtain a vehicle matching data set corresponding to each incoming vehicle information set;

[0084] The road section vehicle information of the road section to be detected is determined based on the matching data set.

[0085] Specifically, the similarity is the similarity distance between the target vehicle in the incoming vehicle information set and the reference vehicle in the outgoing vehicle information set. The calculation formula of the similarity distance can be:

[0086]

[0087]

[0088] Among them, w1 and w2 are custom weights, v q represents the target vehicle, v g represents the reference vehicle, represents the confidence of the target vehicle, represents the confidence of the parameter vehicle, represents the vehicle characteristics of the reference vehicle, represents the vehicle characteristics of the reference vehicle, and i represents the i-th target vehicle.

[0089] After obtaining the similarity matrix between the incoming vehicle information set and each outgoing vehicle information set, a bipartite graph matching algorithm, such as the Hungarian algorithm, is used to perform vehicle matching on the similarity matrix between the incoming vehicle information set and each outgoing vehicle information set to obtain a vehicle matching dataset. The vehicle matching dataset includes instances where an incoming vehicle matches an outgoing vehicle, an incoming vehicle does not match an outgoing vehicle, and an outgoing vehicle does not match an incoming vehicle. Furthermore, to quickly determine the vehicle matching dataset, a vehicle matching determination table can be pre-set and used to determine vehicle matching data. The vehicle matching determination table can be shown in Table 1.

[0090] Table 1 Vehicle matching judgment table

[0091] <![CDATA[Camera C1]]> <![CDATA[Camera C2]]> Matching results judge enter out match Passing section out enter match Passing section - In, Out match U-turn within the road section In, Out - match U-turn within the road section out - Unmatched Vehicles parked on the original road section exit the road section - out Unmatched Vehicles parked on the original road section exit the road section enter - Unmatched Stay within the road section - enter Unmatched Stay within the road section

[0092] In one implementation of this embodiment, the incoming vehicle information set and the outgoing vehicle information set both include a video timestamp and a vehicle tracking identifier; and determining the road segment vehicle information of the road segment to be detected based on the matching data set specifically includes:

[0093] For each incoming vehicle, determining the exit time of the outgoing vehicle corresponding to the incoming vehicle based on the matching data set;

[0094] The road section vehicle information of the road section to be detected is determined based on the entry time of each entering vehicle and its corresponding exit time.

[0095] Specifically, the exit time refers to the time when the target vehicle leaves the road section to be detected, and the entry time refers to the time when the target vehicle enters the road section to be detected. Furthermore, in practical applications, for each detected target vehicle, the vehicle information of incoming vehicles that are not matched to exiting vehicles can be stored. This vehicle information can include vehicle characteristics and entry time. Thus, in subsequent detections, for outgoing vehicles that are not identified as entering vehicles, a matching incoming vehicle can be searched for within the stored vehicle information of incoming vehicles to determine the target vehicle's residence time on the road section to be detected. Furthermore, after obtaining the entry time and its corresponding exit time, the vehicle's residence time can be calculated based on the entry time and its corresponding exit time. Furthermore, the average travel time can be determined based on the vehicle's residence time and the length of the road section to be detected. The traffic volume on the road section and the number of parked vehicles on the road section can also be determined.

[0096] In summary, this embodiment provides a method for detecting vehicle information on a road section based on vision, the method comprising obtaining video clips taken by cameras installed at both ends of the road section to be detected; performing multi-target tracking on each of the obtained video clips to obtain a vehicle trajectory set corresponding to each video clip; drawing a target determination line on the video interface of the camera, and based on the target determination line and the vehicle trajectory set corresponding to the camera, determining the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera, respectively performing feature extraction on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain an incoming vehicle feature set and an outgoing vehicle feature set; determining the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera. This application determines the road section vehicle information by processing the video clips captured by existing cameras, so that the road section vehicle information can be detected without the need for additional deployment points or maintenance, thereby saving the resources and personnel costs required for road condition detection.

[0097] Based on the above-mentioned road section vehicle information detection method based on vision, this embodiment provides a road section vehicle information detection system based on vision, such as Figure 3 As shown, the system includes:

[0098] An acquisition module 100 is used to acquire video clips captured by cameras installed at both ends of the road section to be detected;

[0099] The tracking module 200 is used to perform multi-target tracking on each acquired video clip to obtain a vehicle trajectory set corresponding to each video clip;

[0100] a drawing module 300 for drawing, for each camera, a target determination line on the video interface of the camera, and determining, based on the target determination line and the vehicle trajectory set corresponding to the camera, an incoming vehicle information set and an outgoing vehicle information set corresponding to the camera, wherein the incoming vehicle information set and the outgoing vehicle information set both include vehicle representative images;

[0101] An extraction module 400 is configured to perform feature extraction on each representative image of a vehicle in the incoming vehicle information set and the outgoing vehicle information set corresponding to each camera, respectively, to obtain an incoming vehicle feature set and an outgoing vehicle feature set, wherein the incoming vehicle feature set and the outgoing vehicle feature set both include vehicle features and license plate information;

[0102] The determination module 500 is configured to determine the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera.

[0103] Based on the above-mentioned vision-based road section vehicle information detection method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the vision-based road section vehicle information detection method as described in the above-mentioned embodiment.

[0104] Based on the above-mentioned road section vehicle information detection method based on vision, the present application also provides a terminal device, such as Figure 4 As shown, it includes at least one processor 20; a display screen 21; and a memory 22. It may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via bus 24. The display screen 21 is configured to display a preset user guidance interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0105] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0106] The memory 22, as a computer-readable storage medium, can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the software programs, instructions, or modules stored in the memory 22 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0107] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal device. In addition, the memory 22 may include high-speed random access memory and non-volatile memory. For example, various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

[0108] In addition, the specific process of loading and executing the multiple instructions in the storage medium and the processor in the terminal device has been described in detail in the above method and will not be described here one by one.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting road section vehicle information based on vision, characterized in that: The method comprises: Obtain video clips taken by cameras installed at both ends of the road section to be detected; Perform multi-target tracking on each acquired video clip to obtain a set of vehicle trajectories corresponding to each video clip; For each camera, a target determination line is drawn on the video interface of the camera, and based on the target determination line and the vehicle trajectory set corresponding to the camera, an incoming vehicle information set and an outgoing vehicle information set corresponding to the camera are determined, wherein the incoming vehicle information set and the outgoing vehicle information set both include vehicle representative images; For each camera, feature extraction is performed on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain an incoming vehicle feature set and an outgoing vehicle feature set, wherein the incoming vehicle feature set and the outgoing vehicle feature set both include vehicle features and license plate information; Based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera, the road section vehicle information of the road section to be detected is determined.

2. The method for detecting road section vehicle information based on vision according to claim 1, characterized in that: The vehicle trajectory set includes the vehicle trajectory of each target vehicle in the video clip, a vehicle tracking identifier, a vehicle detection frame image of each trajectory point in each vehicle trajectory, and a video timestamp.

3. The method for detecting road section vehicle information based on vision according to claim 1, characterized in that: The determining of the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera based on the target determination line and the vehicle trajectory set corresponding to the camera specifically includes: For each vehicle trajectory in the vehicle trajectory set, determining whether the vehicle trajectory intersects the target determination line; When the vehicle trajectory intersects the target determination line, the driving direction of the vehicle trajectory is obtained; Adding the vehicle trajectory with the driving direction being the driving direction to the candidate driving-in trajectory set, and adding the vehicle trajectory with the driving direction being the driving direction to the candidate driving-out trajectory set; For each candidate vehicle trajectory in the candidate entry trajectory set and the candidate exit trajectory set, determining a trajectory speed corresponding to the candidate vehicle trajectory according to the starting coordinates, the ending coordinates, and the number of trajectory points of the candidate vehicle trajectory; A candidate vehicle trajectory is selected whose trajectory speed satisfies a speed condition and whose number of trajectory points satisfies a number condition, and an incoming vehicle information set and an outgoing vehicle information set are determined according to the selected candidate vehicle trajectory.

4. The method for detecting road section vehicle information based on vision according to claim 1 or 3, characterized in that: The vehicle representative image is a vehicle detection frame image with the largest detection frame among the vehicle detection frame images of each track point of the vehicle track.

5. The method for detecting road section vehicle information based on vision according to claim 1, characterized in that: The extracting features of each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera to obtain the incoming vehicle feature set and the outgoing vehicle feature set specifically includes: For each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera, extracting a feature vector of the vehicle representative image using a vehicle recognition algorithm, and detecting the license plate information of the vehicle representative image using a license plate detection algorithm; The set consisting of the feature vectors and license plate information of each target vehicle in the incoming vehicle information set is used as the incoming vehicle feature set, and the set consisting of the feature vectors and license plate information of each outgoing vehicle in the outgoing vehicle information set is used as the outgoing vehicle feature set.

6. The method for detecting road section vehicle information based on vision according to claim 5, characterized in that: The determining of the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera specifically includes: For each target vehicle in the incoming vehicle information set of each camera, select a reference vehicle whose exit time is later than the target vehicle's entry time from the outgoing vehicle information set of each camera, and calculate the similarity between each reference vehicle and the target vehicle based on the feature vector and license plate information to obtain a similarity matrix between each incoming vehicle information set and each outgoing vehicle information set; Using a bipartite graph matching algorithm, vehicle matching is performed on the similarity matrix of each incoming vehicle information set and each outgoing vehicle information set to obtain a vehicle matching dataset corresponding to each incoming vehicle information set, wherein the vehicle matching dataset includes incoming vehicles that match outgoing vehicles, incoming vehicles that do not match outgoing vehicles, and outgoing vehicles that do not match incoming vehicles; The road section vehicle information of the road section to be detected is determined based on the matching data set.

7. The method for detecting road section vehicle information based on vision according to claim 6, characterized in that: The incoming vehicle information set and the outgoing vehicle information set both include a video timestamp and a vehicle tracking identifier; The determining of the road section vehicle information of the road section to be detected based on the matching data set specifically includes: For each incoming vehicle, determining the exit time of the outgoing vehicle corresponding to the incoming vehicle based on the matching data set; The section vehicle information of the road section to be detected is determined based on the entry time of each entering vehicle and its corresponding exit time, wherein the section vehicle information includes one or more of the vehicle stay time, the traffic volume of the section, the average travel time and the number of stops on the section.

8. A road section vehicle information detection system based on vision, characterized in that: The system comprises: An acquisition module is used to acquire video clips taken by cameras installed at both ends of the road section to be detected; A tracking module is used to perform multi-target tracking on each acquired video clip to obtain a set of vehicle trajectories corresponding to each video clip; a drawing module configured to draw, for each camera, a target determination line on a video interface of the camera, and determine, based on the target determination line and a vehicle trajectory set corresponding to the camera, an incoming vehicle information set and an outgoing vehicle information set corresponding to the camera, wherein the incoming vehicle information set and the outgoing vehicle information set both include vehicle representative images; an extraction module configured to, for each camera, perform feature extraction on each vehicle representative image in the incoming vehicle information set and the outgoing vehicle information set corresponding to the camera, respectively, to obtain an incoming vehicle feature set and an outgoing vehicle feature set, wherein the incoming vehicle feature set and the outgoing vehicle feature set both include vehicle features and license plate information; The determination module is used to determine the road section vehicle information of the road section to be detected based on the incoming vehicle feature set and the outgoing vehicle feature set corresponding to each camera.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the vision-based road section vehicle information detection method as described in any one of claims 1-7.

10. A terminal device, characterized in that: include: processor, memory, and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the processor implements the steps in the vision-based road section vehicle information detection method as described in any one of claims 1-7.

Images