Automatic labeling and detection method, device and computer equipment for vehicle driving detection

Abstract

The present invention discloses an automatic marking and detection method, device and computer equipment for vehicle driving detection. The method comprises: acquiring road image data; extracting a vehicle trajectory set of a target vehicle within a preset time based on the road image data, wherein the vehicle trajectory set comprises a driving trajectory of the target vehicle and a vehicle detection frame of the target vehicle at a preset detection point of the driving trajectory; obtaining a direction vector between a vehicle detection frame of the target vehicle at each preset detection point and a vehicle detection frame of the preset marking point based on the vehicle detection frame of the target vehicle at the preset marking point; the preset marking point is one of the preset detection points; obtaining a calibrated direction vector of the target vehicle based on the direction vector and an initial direction vector; and marking the driving direction of the target vehicle based on the calibrated direction vector.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to an automatic marking and detection method, device and computer equipment for vehicle driving detection. Background Art

[0002] Current vehicle driving detection solutions all rely on sample data formed by manual annotation of road directions. However, with the explosive growth of the video surveillance market and the increasing number of cameras, current solutions lack adaptive road direction annotation for complex scenes. Manual annotation is required for each adjustment, making it difficult to achieve higher intelligence in large-scale deployments, resulting in low accuracy and processing efficiency in the process of vehicle driving detection. Summary of the invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defects of low accuracy and processing efficiency in the process of vehicle driving detection caused by manual labeling of road direction in existing vehicle reverse detection, thereby providing an automatic labeling and detection method, device and computer equipment for vehicle driving detection.

[0004] According to a first aspect, an embodiment of the present invention discloses an automatic labeling method for vehicle driving detection, comprising: acquiring road image data; extracting a vehicle trajectory set of a target vehicle within a preset time based on the road image data, the vehicle trajectory set comprising a driving trajectory of the target vehicle, and a vehicle detection frame of the target vehicle at a preset detection point of the driving trajectory; obtaining a direction vector between a vehicle detection frame of the target vehicle at each of the preset detection points and a vehicle detection frame of the preset labeling point based on the vehicle detection frame of the target vehicle at the preset labeling point; the preset labeling point is one of the preset detection points; obtaining a calibrated direction vector of the target vehicle based on the direction vector and an initial direction vector; and labeling the driving direction of the target vehicle based on the calibrated direction vector.

[0005] Optionally, the obtaining of the direction vector between the vehicle detection frame of the target vehicle at each preset detection point and the vehicle detection frame of the preset marking point based on the vehicle detection frame of the target vehicle at the preset marking point includes: obtaining a preset number of vehicle detection frames in the first direction and the second direction of the preset marking point in the driving trajectory; connecting the geometric center of the vehicle detection frame in the first direction to the geometric center of the vehicle detection frame of the preset marking point to obtain a first direction vector; connecting the geometric center of the vehicle detection frame in the second direction to the geometric center of the vehicle detection frame of the preset marking point to obtain a second direction vector; obtaining the direction vector based on the first direction vector and the second direction vector.

[0006] Optionally, the first direction vector includes a preset number of first sub-direction vectors, and the second direction vector includes a preset number of second sub-direction vectors, and obtaining the direction vector based on the first direction vector and the second direction vector includes: screening sub-direction vectors that form an acute angle between the first sub-direction vector and the second sub-direction vector as possible direction vectors of the preset marking point; removing vectors on both side edges of the angle of the possible direction vectors; and calculating the vector sum of the remaining possible direction vectors to obtain the direction vector.

[0007] Optionally, obtaining the calibrated direction vector of the target vehicle based on the direction vector and the initial direction vector includes: aligning the vehicle detection frame of the preset annotation point with the corresponding road image data by geometric center and performing geometric scaling to a preset ratio; based on the scaled road image data, the initial direction vector and the direction vector, obtaining the calibrated direction vector of all pixel points of the target vehicle in the scaled road image data.

[0008] According to the second aspect, an embodiment of the present invention further discloses a vehicle driving detection method, comprising: obtaining a driving marked direction of the target vehicle based on image data of the target vehicle, wherein the driving marked direction is obtained using an automatic marking method for vehicle driving detection as described in the first aspect and any optional implementation scheme of the first aspect; detecting a driving trend of the target vehicle based on the driving marked direction.

[0009] Optionally, the detecting the driving trend of the target vehicle based on the driving marked direction includes: obtaining the position of the determination point based on one of the vehicle trajectories in the vehicle trajectory set, the frame interval of the vehicle trajectory and a preset number of determination points; obtaining the direction vector of the geometric center of each of the determination points based on the position of the determination point; and detecting the driving trend of the target vehicle based on the angular relationship between the direction vector of the geometric center of the determination point and the calibrated direction vector.

[0010] Optionally, the driving trend of the target vehicle is detected based on the angle relationship between the direction vector of the geometric center of the judgment point and the calibrated direction vector, including: counting the wrong-moving situations of the target vehicle in all judgment points; if the number of wrong-moving situations of the target vehicle is greater than a preset proportion of the number of judgment points, the target vehicle is determined to be wrong-moving; if the number of wrong-moving situations of the target vehicle is less than or equal to the preset proportion of the number of judgment points, the target vehicle is determined to be not wrong-moving.

[0011] According to the third aspect, an embodiment of the present invention further discloses an automatic labeling device for vehicle driving detection, comprising: an acquisition module, used to acquire road image data; a vehicle trajectory set extraction module, used to extract a vehicle trajectory set of a target vehicle within a preset time based on the road image data, wherein the vehicle trajectory set includes the driving trajectory of the target vehicle and a vehicle detection frame of the target vehicle at a preset detection point of the driving trajectory; a direction vector module, used to obtain a direction vector between the vehicle detection frame of the target vehicle at each preset detection point and the vehicle detection frame of the preset labeling point based on the vehicle detection frame of the target vehicle at the preset labeling point; the preset labeling point is one of the preset detection points; a calibration direction vector module, used to obtain a calibration direction vector of the target vehicle based on the direction vector and an initial direction vector; and a labeling module, used to label the driving direction of the target vehicle based on the calibration direction vector.

[0012] According to the fourth aspect, an embodiment of the present invention further discloses a vehicle driving detection device, including: a driving annotation direction module, used to obtain the driving annotation direction of the target vehicle based on the image data of the target vehicle, and the driving annotation direction is obtained using the automatic annotation method for vehicle driving detection as described in the first aspect and any optional real-time method of the first aspect; a driving trend module, used to detect the driving trend of the target vehicle based on the driving annotation direction.

[0013] According to the fifth aspect, an embodiment of the present invention also discloses a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the automatic labeling method for vehicle driving detection as described in the first aspect or any optional embodiment of the first aspect, or the automatic labeling method for vehicle driving detection as described in the second aspect or any optional embodiment of the second aspect.

[0014] The technical solution of the present invention has the following advantages:

[0015] The automatic marking and detection method, device and computer equipment for vehicle driving detection provided by the present invention include: obtaining road image data; extracting a vehicle trajectory set of a target vehicle within a preset time based on the road image data, the vehicle trajectory set including the driving trajectory of the target vehicle and the vehicle detection frame of the target vehicle at the preset detection point of the driving trajectory; obtaining the direction vector between the vehicle detection frame of the target vehicle at each preset detection point and the vehicle detection frame of the preset marking point based on the vehicle detection frame of the target vehicle at the preset marking point; the preset marking point is one of the preset detection points; obtaining the calibrated direction vector of the target vehicle based on the direction vector and the initial direction vector; and marking the driving direction of the target vehicle based on the calibrated direction vector. By extracting the vehicle driving trajectory in the road image, the automatic marking of the driving direction of the target vehicle is obtained through the vector relationship between the preset marking point and the preset detection point in the vehicle driving trajectory, the mode of relying on manual marking of the vehicle driving direction is broken away, and the marking can be performed adaptively, thereby improving the accuracy and processing efficiency of marking the driving direction of the target vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flowchart of a specific example of an automatic labeling method for vehicle driving detection in an embodiment of the present invention;

[0018] Figure 2 is a flowchart of a specific example of a vehicle driving detection method in an embodiment of the present invention;

[0019] Figure 3 is a principle block diagram of a specific example of an automatic labeling device for vehicle driving detection in an embodiment of the present invention;

[0020] Figure 4 is a principle block diagram of a specific example of a vehicle driving detection device in an embodiment of the present invention;

[0021] Figure 5 A specific example diagram of a computer device in an embodiment of the present invention;

[0022] Figure 6 is a schematic diagram of a specific example of a vehicle driving detection method in an embodiment of the present invention;

[0023] Figure 7 A schematic diagram of a specific example of an automatic labeling method for vehicle driving detection in an embodiment of the present invention;

[0024] Figure 8 The figure is a schematic diagram of a specific example of an automatic labeling method for vehicle driving detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0027] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0029] The embodiment of the present invention discloses an automatic labeling method for vehicle driving detection, such as Figure 1 As shown, the method comprises the following steps:

[0030] Step 101: Obtain road image data. Exemplarily, the road image data may be obtained by photographing a camera device set up at a detection point above or on both sides of the road, and the road image data may be extracted from a section of road video data photographed by the camera device. The embodiment of the present invention does not limit the type of road image data, and those skilled in the art may determine it according to actual needs.

[0031] Step 102: extracting a vehicle trajectory set of a target vehicle within a preset time based on the road image data, wherein the vehicle trajectory set includes a driving trajectory of the target vehicle and a vehicle detection frame of the target vehicle at a preset detection point of the driving trajectory.

[0032] Exemplarily, the target vehicle is a vehicle that needs to be marked with a vehicle direction, the preset time is the time span for obtaining the road image data, the vehicle trajectory set is the driving trajectory of all vehicles in the road image data within the preset time, and the vehicle detection box is marked for the corresponding vehicle at different detection points in each corresponding driving trajectory. For example, within the preset time, the FairMot algorithm is used to track the road image data to obtain a vehicle trajectory set T = {t(1), t(2), ..., t(i), ...}, i∈{1,2,3.......∞}, t represents a trajectory, and t = {box(1), box(2)...box(i)...}, i∈{1,2,3.......∞}, box(i) represents the corresponding vehicle detection box detected at the i-th detection point of the trajectory, the order of i is arranged in chronological order, box = (x1, y1, x2, y2), (x1, y1) represents the coordinates of the upper left vertex of the vehicle detection box, and (x2, y2) represents the coordinates of the lower right vertex of the frame. The embodiment of the present invention does not limit the type of vehicle tracking algorithm, and those skilled in the art can determine it according to actual needs.

[0033] Step 103: Based on the vehicle detection frame of the target vehicle at the preset marking point, obtain the direction vector between the vehicle detection frame of the target vehicle at each of the preset detection points and the vehicle detection frame of the preset marking point; the preset marking point is one of the preset detection points. Exemplarily, the preset marking point is the detection point that needs to be marked with the vehicle direction, and the preset detection point is the detection point on both sides of the preset marking point, and the direction vector corresponding to the preset marking point is obtained according to the vector relationship between the detection points on both sides and the marking point.

[0034] Step 104: Obtain a calibrated direction vector of the target vehicle based on the direction vector and the initial direction vector. Exemplarily, the initial direction vector is an initial direction vector with a size of 0 set for each pixel point in the road image data and an uncertain direction. The vector obtained by adding the direction vector obtained in step 103 to the initial direction vector is the calibrated direction vector of the target vehicle at the preset mark point.

[0035] Step 105: Annotate the driving direction of the target vehicle based on the calibration direction vector. Exemplarily, the calibration direction vector corresponding to the target vehicle is obtained and the driving direction of the target vehicle is annotated, so the driving direction of the target vehicle is the direction of the calibration direction vector.

[0036] The automatic annotation method for vehicle driving detection provided by the present invention includes: acquiring road image data; extracting a vehicle trajectory set of a target vehicle within a preset time based on the road image data, wherein the vehicle trajectory set includes the driving trajectory of the target vehicle and the vehicle detection frame of the target vehicle at the preset detection point of the driving trajectory; obtaining the direction vector between the vehicle detection frame of the target vehicle at each preset detection point and the vehicle detection frame of the preset annotation point based on the vehicle detection frame of the target vehicle at the preset annotation point; the preset annotation point is one of the preset detection points; obtaining the calibrated direction vector of the target vehicle based on the direction vector and the initial direction vector; and annotating the driving direction of the target vehicle based on the calibrated direction vector. By extracting the vehicle driving trajectory in the road image, the annotation of the driving direction of the target vehicle is obtained through the vector relationship between the preset annotation point and the preset detection point in the vehicle driving trajectory, the mode of relying on manual annotation of the vehicle driving direction is eliminated, and annotation can be performed adaptively, thereby improving the accuracy and processing efficiency of the annotation of the driving direction of the target vehicle.

[0037] As an optional embodiment of the present invention, step 103 includes: obtaining a preset number of vehicle detection frames in the first direction and the second direction of the preset marked points in the driving trajectory; connecting the geometric center of the vehicle detection frame in the first direction with the geometric center of the vehicle detection frame of the preset marked point to obtain a first direction vector; connecting the geometric center of the vehicle detection frame in the second direction with the geometric center of the vehicle detection frame of the preset marked point to obtain a second direction vector; and obtaining the direction vector based on the first direction vector and the second direction vector.

[0038] As an optional embodiment of the present invention, the first direction vector includes a preset number of first sub-direction vectors, and the second direction vector includes a preset number of second sub-direction vectors. The step 103 also includes: screening the sub-direction vectors that form an acute angle between the first sub-direction vector and the second sub-direction vector as possible direction vectors of the preset marking point; removing the vectors on both sides of the edge of the possible direction vector angle; and calculating the vector sum of the remaining possible direction vectors to obtain the direction vector.

[0039] Exemplarily, the first direction and the second direction of the preset annotation point of the driving track are respectively the preset detection points on both sides of the preset annotation point, which may be the preset detection points on both sides of the preset annotation point, or on both sides of the left and right sides, and the preset number of vehicle detection frames are the preset number of detection points in the first direction and the second direction, for example, the vehicle detection frames extracted from the 5 detection points in the front and back. The embodiment of the present invention does not limit the number of the preset number of vehicle detection frames, and those skilled in the art may determine it according to actual needs.

[0040] Specifically, the calculation method of the direction vector is introduced in the following specific embodiment. For each box(i) in the trajectory t, the geometric center b(i) of the vehicle detection box with the pre-set annotation point is selected. The geometric centers of the first five vehicle detection boxes are selected to b(i), and the unit direction vector is obtained by connecting the lines. Then, the unit direction vector is obtained by connecting b(i) to the center b(i) of the last five detection boxes. The range of less than 45 degrees formed by these 10 vectors is collected as the possible direction vector range of the detection box. The two vectors at the edge of the vector range are removed, and the acute angle range formed by the remaining eight vectors is used as the vector range to be selected. The sum of these eight unit vectors is calculated to obtain the direction vector, which is the direction vector of box(i). Figure 7 This is a schematic diagram of determining the direction vector. Due to the limited space of the drawing, Figure 7 Box 1 and box 2 represent the first five boxes, box 4 and box 5 represent the last five boxes, box 3 represents the vehicle detection box of the preset annotation points, and the step of removing the two vectors at the edge of the vector range is omitted. The large black arrow is the driving direction, and the thick arrow is the final unit direction vector. Among them, the second step to the third step is mainly the step of normalizing each unit vector.

[0041] As an optional embodiment of the present invention, step 104 includes: aligning the vehicle detection frame of the preset marking point with the corresponding road image data by geometric center and scaling them proportionally to a preset ratio; based on the scaled road image data, the initial direction vector and the direction vector, obtaining the calibrated direction vector of all pixel points of the target vehicle in the scaled road image data.

[0042] Exemplarily, the vehicle detection frame with the preset annotation points is aligned with the road image data with the geometric center of the vehicle detection frame, and the preset ratio can be specifically set according to the size of the actual target vehicle, for example, it can be set to 80% or 90%. After the scaled vehicle detection frame completely overlaps with the vehicle area in the corresponding road image data, the pixel points in the vehicle detection frame area in the scaled road image data are directional-annotated to obtain the calibration direction vectors of all pixel points, wherein the calibration direction vector is consistent with the direction vector of the vehicle detection frame. The embodiment of the present invention does not limit the size of the preset ratio, and those skilled in the art can determine it according to actual needs.

[0043] Specifically, the pixel area corresponding to box(i) in the road image data is denoted as B1, box(i) is scaled by 80% and the geometric centers are aligned. The corresponding pixel area in the obtained road image data is denoted as B2. For each pixel point in B2, the direction vector obtained in step 103 is added to the initial direction vector to obtain the calibrated direction vector of the current iteration of the pixel point.

[0044] The embodiment of the present invention also discloses a vehicle driving detection method, such as Figure 2 As shown, the method comprises the following steps:

[0045] Step 201: Acquire a marked driving direction of the target vehicle based on image data of the target vehicle, wherein the marked driving direction is obtained using the automatic marking method for vehicle driving detection as described in any of the above embodiments.

[0046] Step 202: Detect the driving trend of the target vehicle based on the marked driving direction. Exemplarily, the target vehicle is a vehicle that needs to be determined for its driving trend, where the driving trend of the vehicle can be the vehicle's driving direction, such as forward or reverse, or the vehicle is changing lanes.

[0047] The vehicle driving detection method provided by an embodiment of the present invention includes: obtaining the driving annotation direction of the target vehicle based on the image data of the target vehicle, wherein the driving annotation direction is obtained using the automatic annotation method for vehicle driving detection as described in any of the above embodiments. Detecting the driving trend of the target vehicle based on the driving annotation direction. Obtaining the formal driving annotation direction in the vehicle driving trajectory based on the acquired vehicle image data, and according to the obtained driving annotation direction, the driving trend of the vehicle can be obtained more accurately.

[0048] As an optional embodiment of the present invention, step 202 includes: obtaining the position of the determination point based on one of the vehicle trajectories in the vehicle trajectory set, the frame interval of the vehicle trajectory and a preset number of determination points; obtaining the direction vector of the geometric center of each of the determination points based on the position of the determination point; detecting the driving trend of the target vehicle based on the angle relationship between the direction vector of the geometric center of the determination point and the calibrated direction vector. Counting the retrograde conditions of the target vehicle in all determination points; if the number of retrograde conditions of the target vehicle is greater than the number of determination points of the preset ratio, the target vehicle is determined to be retrograde; if the number of retrograde conditions of the target vehicle is less than or equal to the number of determination points of the preset ratio, the target vehicle is determined to be non-retrograde.

[0049] Exemplarily, the vehicle trajectory set is all vehicle trajectories in the image data, the frame interval of the vehicle trajectory is the interval of the collection frames in the vehicle trajectory collection, the preset number of determination points are the detection points that need to be used to determine the driving trend of the target vehicle, and the number of determination points can be selected to be an odd number. When the angle between the geometric center of the vehicle detection bounding box of the determination point and the calibrated direction vector in the corresponding driving annotation direction is an obtuse angle, it is determined that the vehicle driving trend of the current determination point is retrograde.

[0050] Specifically, the determination of the driving trend of the target vehicle is introduced in the following specific implementation method. For each trajectory, an odd number of determination points are selected, and the number and position of the determination points are determined by the following rules: suppose the trajectory has n frames, select a frame interval of m, if n is an odd number and m is divided by m, then the number of points is n / m, if it is an integer, then the number of determination points is n / m+1. The number of determination points is x, and the positions of the determination points are respectively located at 1 / (x+1), 2 / (x+1)...x / (x+1). For several centers b(i) of each determination point on a trajectory, the direction vector of each determination point is obtained according to the above-mentioned automatic marking method for vehicle driving detection, and the angle between the direction vector of each determination point and the calibrated direction vector of the corresponding position is calculated. When the angle is greater than 45 degrees, it is recorded as a reverse movement. The reverse movement of x determination points is counted. When more than 50% of the determination points are marked as reverse movement, the vehicle on the trajectory is marked as reverse movement. Figure 8 The figure is a schematic diagram of a specific example of determining whether a vehicle is traveling in the wrong direction. The solid arrow is the calibrated direction vector of the pixel point, and the dotted arrow is the upper and lower bounds. The vehicle direction vectors within the 90-degree range surrounded by the upper and lower bounds of the vector are all traveling normally, and the angles from the upper and lower bounds of the vector to the calibrated direction vector are both 45 degrees. The vehicle direction vectors that exceed this 90-degree range are all traveling in the wrong direction.

[0051] Specifically, according to the above-mentioned automatic marking method and vehicle driving detection method for vehicle driving detection, the method for determining the vehicle driving trend can be introduced according to the following embodiments. Selecting road image data of a preset time of 30 minutes to mark the road direction, and using the road image data of the next 30 minutes to determine the vehicle driving trend, such as Figure 6 The real-time detection results shown in FIG. 1 are as follows: arrows 12, 22 and 32 are the directions of the vehicle's trajectory at this time; arrows 11, 21 and 31 are the calibrated directions of the vehicle's geometric center at this time:

[0052] The arrows 11 and 12 of car No. 1 are almost coincident, and the car is driving normally; car No. 2 is about to pull over and is not completely parallel to the road direction, but the angle between the calibrated direction vector arrow 21 of the car's center point and the vehicle's direction vector at this time is less than 45 degrees, so it is not judged as going against the flow; car No. 3 wants to change lanes to the left, so the calibrated direction vector of the center point deviates from the vehicle's direction vector at this time, but it is also within 45 degrees, so it is not judged as going against the flow either.

[0053] The embodiment of the present invention also discloses an automatic labeling device for vehicle driving detection, such as Figure 3 As shown, the device comprises:

[0054] The acquisition module 301 is used to acquire road image data. For example, the details are shown in step 101 in the above-mentioned automatic annotation method embodiment for vehicle driving detection, which will not be described in detail here.

[0055] The vehicle trajectory set extraction module 302 is used to extract the vehicle trajectory set of the target vehicle within a preset time based on the road image data, and the vehicle trajectory set includes the driving trajectory of the target vehicle and the vehicle detection frame of the target vehicle at the preset detection point of the driving trajectory. For example, the details are shown in step 102 in the above-mentioned automatic annotation method embodiment for vehicle driving detection, which will not be repeated here.

[0056] The direction vector module 303 is used to obtain the direction vector between the vehicle detection frame of the target vehicle at each of the preset detection points and the vehicle detection frame of the preset annotation point based on the vehicle detection frame of the target vehicle at the preset annotation point; the preset annotation point is one of the preset detection points. For example, the details are shown in step 103 of the above-mentioned embodiment of the automatic annotation method for vehicle driving detection, which will not be repeated here.

[0057] The calibration direction vector module 304 is used to obtain the calibration direction vector of the target vehicle based on the direction vector and the initial direction vector. For example, the details are shown in step 104 in the above-mentioned automatic labeling method embodiment for vehicle driving detection, which will not be repeated here.

[0058] The marking module 305 is used to mark the driving direction of the target vehicle based on the calibrated direction vector. For example, the details are shown in step 105 in the above-mentioned automatic marking method embodiment for vehicle driving detection, which will not be repeated here.

[0059] The automatic annotation device for vehicle driving detection provided by the present invention includes: an acquisition module 301, which is used to acquire road image data. A vehicle trajectory set extraction module 302, which is used to extract a vehicle trajectory set of a target vehicle within a preset time based on the road image data, and the vehicle trajectory set includes the driving trajectory of the target vehicle and the vehicle detection frame of the target vehicle at the preset detection point of the driving trajectory. A direction vector module 303, which is used to obtain the direction vector between the vehicle detection frame of the target vehicle at each preset detection point and the vehicle detection frame of the preset annotation point based on the vehicle detection frame of the target vehicle at the preset annotation point; the preset annotation point is one of the preset detection points. A calibration direction vector module 304, which is used to obtain the calibration direction vector of the target vehicle based on the direction vector and the initial direction vector. A labeling module 305, which is used to label the driving direction of the target vehicle based on the calibration direction vector. By extracting the vehicle's driving trajectory in the road image and obtaining the target vehicle's driving direction annotation through the vector relationship between the preset annotation points and the preset detection points in the vehicle's driving trajectory, the method breaks away from the mode of relying on manual annotation of the vehicle's driving direction and can perform annotation adaptively, thereby improving the accuracy and processing efficiency of the target vehicle's driving direction annotation.

[0060] As an optional embodiment of the present invention, the direction vector module 303 includes: a preset detection frame acquisition module 301, which is used to obtain a preset number of vehicle detection frames in the first direction and the second direction of the preset annotation point in the driving trajectory; a first direction vector module, which is used to connect the geometric center of the vehicle detection frame in the first direction with the geometric center of the vehicle detection frame of the preset annotation point to obtain a first direction vector; a second direction vector module, which is used to connect the geometric center of the vehicle detection frame in the second direction with the geometric center of the vehicle detection frame of the preset annotation point to obtain a second direction vector; a combination module, which is used to obtain the direction vector based on the first direction vector and the second direction vector. Exemplarily, the details are shown in step 103 of the automatic annotation method for vehicle driving detection in the above embodiment, which will not be repeated here.

[0061] As an optional embodiment of the present invention, the first direction vector includes a preset number of first sub-direction vectors, the second direction vector includes a preset number of second sub-direction vectors, and the direction vector module 303 further includes: a screening module for screening the sub-direction vectors that form an acute angle between the first sub-direction vector and the second sub-direction vector as possible direction vectors of the preset annotation point; an edge removal module for removing the vectors of the edges on both sides of the possible direction vector angle; and a vector sum module for calculating the vector sum of the remaining possible direction vectors to obtain the direction vector. For example, the details are shown in step 103 of the automatic annotation method for vehicle driving detection in the above embodiment, which will not be repeated here.

[0062] As an optional embodiment of the present invention, the calibration direction vector module 304 includes: a scaling module, which is used to align the vehicle detection frame of the preset annotation point with the corresponding road image data by geometric center and perform geometric scaling of a preset ratio; a calibration direction vector submodule, which is used to obtain the calibration direction vector of all pixel points of the target vehicle in the scaled road image data based on the scaled road image data, the initial direction vector and the direction vector. For example, the details are shown in step 104 of the automatic annotation method for vehicle driving detection in the above embodiment, which will not be repeated here.

[0063] The embodiment of the present invention also discloses a vehicle driving detection device, such as Figure 4 As shown, the device comprises:

[0064] The driving direction marking module 401 is used to obtain the driving direction marking of the target vehicle based on the image data of the target vehicle, and the driving direction marking is obtained by using the automatic marking method for vehicle driving detection described in the above method embodiment. For example, the details are shown in step 201 of the vehicle driving detection method in the above embodiment, which will not be repeated here.

[0065] The driving trend module 402 is used to detect the driving trend of the target vehicle based on the driving marked direction. For example, the details are shown in step 202 of the vehicle driving detection method in the above embodiment, which will not be repeated here.

[0066] The automatic labeling device for vehicle driving detection provided by the present invention includes: a driving labeling direction module 401, which is used to obtain the driving labeling direction of the target vehicle based on the image data of the target vehicle, and the driving labeling direction is obtained by using the automatic labeling method for vehicle driving detection described in the above method embodiment; a driving trend module 402, which is used to detect the driving trend of the target vehicle based on the driving labeling direction. The formal driving labeling direction in the vehicle driving trajectory is obtained according to the acquired vehicle image data, and the driving trend of the vehicle can be obtained more accurately according to the already obtained driving labeling direction.

[0067] As an optional embodiment of the present invention, the driving trend module 402 includes: a determination point position module, which is used to obtain the position of the determination point based on one of the vehicle trajectories in the vehicle trajectory set, the frame interval of the vehicle trajectory and a preset number of determination points; a determination point direction module, which is used to obtain the direction vector of the geometric center of each determination point based on the position of the determination point; a first trend determination module, which is used to detect the driving trend of the target vehicle based on the angle relationship between the direction vector of the geometric center of the determination point and the calibration direction vector. A statistical module is used to count the retrograde conditions of the target vehicle in all determination points; a first determination module is used to determine that the target vehicle is retrograde if the number of retrograde conditions of the target vehicle is greater than the number of determination points of the preset ratio; a second determination module is used to determine that the target vehicle is not retrograde if the number of retrograde conditions of the target vehicle is less than or equal to the number of determination points of the preset ratio. For example, the details are shown in step 202 of the vehicle driving detection method in the above embodiment, which will not be repeated here.

[0068] The embodiment of the present invention also provides a computer device, such as Figure 5 As shown, the computer device may include a processor 501 and a memory 502, wherein the processor 501 and the memory 502 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0069] The processor 501 may be a central processing unit (CPU). The processor 501 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0070] The memory 502 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the automatic labeling method for vehicle driving detection and the program instructions / modules corresponding to the vehicle driving detection method in the embodiment of the present invention. The processor 501 executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory 502, that is, the automatic labeling method for vehicle driving detection and the vehicle driving detection method in the above method embodiment are implemented.

[0071] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 501, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 502 may optionally include a memory remotely arranged relative to the processor 501, and these remote memories may be connected to the processor 501 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0072] The one or more modules are stored in the memory 502, and when executed by the processor 501, the following is performed: Figure 1 or Figure 2 The automatic labeling method for vehicle driving detection and the vehicle driving detection method in the illustrated embodiment.

[0073] For details of the above computer equipment, please refer to Figure 1 or Figure 2 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0074] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0075] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An automatic labeling method for vehicle driving detection, characterized in that: include: Acquire road image data; Extracting a vehicle trajectory set of a target vehicle within a preset time based on the road image data, wherein the vehicle trajectory set includes a driving trajectory of the target vehicle and a vehicle detection frame of the target vehicle at a preset detection point of the driving trajectory; Based on the vehicle detection frame of the target vehicle at the preset marked point, a direction vector between the vehicle detection frame of the target vehicle at each of the preset detection points and the vehicle detection frame of the preset marked point is obtained; the preset marked point is one of the preset detection points; Obtaining a calibrated direction vector of the target vehicle based on the direction vector and an initial direction vector; Annotating the travel direction of the target vehicle based on the calibrated direction vector; The obtaining, based on the vehicle detection frame of the target vehicle at the preset marked point, a direction vector between the vehicle detection frame of the target vehicle at each preset detection point and the vehicle detection frame of the preset marked point comprises: Acquire a preset number of vehicle detection frames in a first direction and a second direction of a preset marked point in the driving trajectory; Respectively connecting the geometric center of the vehicle detection frame in the first direction with the geometric center of the vehicle detection frame of the preset marking point to obtain a first direction vector; Respectively connecting the geometric center of the vehicle detection frame in the second direction with the geometric center of the vehicle detection frame of the preset marking point to obtain a second direction vector; Obtaining the direction vector based on the first direction vector and the second direction vector; The first direction and the second direction of the preset marking point of the driving track are respectively the front and rear sides or the left and right sides of the preset marking point.

2. The method according to claim 1, characterized in that The first direction vector includes a preset number of first sub-direction vectors, the second direction vector includes a preset number of second sub-direction vectors, The obtaining the direction vector based on the first direction vector and the second direction vector comprises: Selecting a sub-direction vector that forms an acute angle between the first sub-direction vector and the second sub-direction vector as a possible direction vector of the preset marking point; Remove the vectors at both sides of the possible direction vector angle; The vector sum of the remaining possible direction vectors is calculated to obtain the direction vector.

3. The method according to claim 1, characterized in that The step of obtaining the calibrated direction vector of the target vehicle based on the direction vector and the initial direction vector includes: Aligning the vehicle detection frame of the preset marked point with the corresponding road image data by geometric center and scaling them by a preset ratio; Based on the scaled road image data, the initial direction vector and the direction vector, a calibrated direction vector of all pixel points of the target vehicle in the scaled road image data is obtained.

4. A vehicle driving detection method, characterized in that: include: Acquire a driving annotation direction of the target vehicle based on the image data of the target vehicle, wherein the driving annotation direction is obtained by using the automatic annotation method for vehicle driving detection according to any one of claims 1 to 3; The driving trend of the target vehicle is detected based on the marked driving direction.

5. The method according to claim 4, characterized in that The detecting the driving trend of the target vehicle based on the driving marked direction includes: Obtaining the position of the decision point based on one of the vehicle trajectories in the vehicle trajectory set, the frame interval of the vehicle trajectory, and a preset number of decision points; Obtaining a direction vector of a geometric center of each of the decision points based on the positions of the decision points; The driving trend of the target vehicle is detected based on the angle relationship between the direction vector of the geometric center of the determination point and the calibration direction vector.

6. The method according to claim 5, characterized in that The detecting the driving trend of the target vehicle based on the angle relationship between the direction vector of the geometric center of the determination point and the calibration direction vector comprises: Counting the wrong-travel situations of the target vehicles at all the decision points; If the number of wrong-way situations of the target vehicle is greater than the number of determination points of a preset ratio, the target vehicle is determined to be wrong-way; If the number of wrong-way situations of the target vehicle is less than or equal to the number of determination points of the preset ratio, the target vehicle is determined to be not wrong-way.

7. An automatic labeling device for vehicle driving detection, characterized in that: include: An acquisition module, used for acquiring road image data; A vehicle trajectory set extraction module, used to extract a vehicle trajectory set of a target vehicle within a preset time based on the road image data, wherein the vehicle trajectory set includes a driving trajectory of the target vehicle and a vehicle detection frame of the target vehicle at a preset detection point of the driving trajectory; A direction vector module, for obtaining a direction vector between a vehicle detection frame of the target vehicle at each of the preset detection points and a vehicle detection frame of the preset marking point based on a vehicle detection frame of the target vehicle at a preset marking point; the preset marking point is one of the preset detection points; A calibration direction vector module, used for obtaining a calibration direction vector of the target vehicle based on the direction vector and an initial direction vector; A marking module, used for marking the driving direction of the target vehicle based on the calibrated direction vector; The obtaining, based on the vehicle detection frame of the target vehicle at the preset marked point, a direction vector between the vehicle detection frame of the target vehicle at each preset detection point and the vehicle detection frame of the preset marked point comprises: Acquire a preset number of vehicle detection frames in a first direction and a second direction of a preset marked point in the driving trajectory; Respectively connecting the geometric center of the vehicle detection frame in the first direction with the geometric center of the vehicle detection frame of the preset marking point to obtain a first direction vector; Respectively connecting the geometric center of the vehicle detection frame in the second direction with the geometric center of the vehicle detection frame of the preset marking point to obtain a second direction vector; Obtaining the direction vector based on the first direction vector and the second direction vector; The first direction and the second direction of the preset marking point of the driving track are respectively the front and rear sides or the left and right sides of the preset marking point.

8. A vehicle driving detection device, characterized in that: include: A driving direction labeling module, used for acquiring the driving direction labeling of the target vehicle based on the image data of the target vehicle, wherein the driving direction labeling is obtained by using the automatic labeling method for vehicle driving detection according to any one of claims 1 to 3; A driving trend module is used to detect the driving trend of the target vehicle based on the marked driving direction.

9. A computer device, characterized in that: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the steps of the automatic labeling method for vehicle driving detection as described in any one of claims 1-3 or the vehicle driving detection method as described in any one of claims 4-6.

Images