Vehicle driving direction recognition method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for identifying the driving direction of a vehicle. The method comprises: obtaining a vehicle driving video; tracking a target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence; extracting a head image set and a tail image set from the vehicle tracking sequence; determining the body part of the target vehicle corresponding to each frame of the target vehicle image in the head image set and the tail image set; and identifying the driving direction of the target vehicle according to the body part. The method can improve recognition efficiency and accuracy.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, computer equipment and storage medium for identifying a vehicle's driving direction. Background Art

[0002] Identifying the direction of vehicle travel based on vehicle travel videos is gradually being used in a variety of application scenarios. For example, in video editing scenarios, it is necessary to identify the vehicle travel direction of each target vehicle based on the vehicle travel video, so as to perform video editing based on the identified vehicle travel direction. For example, in the detection scenario of illegal and wrong-way vehicles, it is necessary to identify the vehicle travel direction of each target vehicle based on the vehicle travel video, so as to determine whether each target vehicle has illegal and wrong-way problems based on the identified vehicle travel direction, thereby detecting illegal and wrong-way vehicles. Therefore, how to identify the vehicle travel direction based on the vehicle travel video is a problem worthy of attention. At present, the method of manually observing the vehicle travel video is usually used to identify and determine the vehicle travel direction. However, under this recognition method, the recognition efficiency is low due to the limitation of manual efficiency, and the recognition accuracy will be reduced due to visual fatigue and other reasons, especially when there are many target vehicles to be identified in the vehicle travel direction, the recognition accuracy and efficiency of the vehicle travel direction will be significantly reduced. Summary of the invention

[0003] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for identifying the vehicle's driving direction that can improve recognition efficiency and accuracy in order to address the above technical problems.

[0004] A method for identifying a vehicle driving direction, the method comprising:

[0005] Obtain vehicle driving video;

[0006] Tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence;

[0007] Extracting a first image set and a last image set from the vehicle tracking sequence;

[0008] Determine the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set;

[0009] The driving direction of the target vehicle is identified according to the vehicle body part.

[0010] In one embodiment, the step of tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence includes:

[0011] Extracting a first video frame from the vehicle driving video, and performing vehicle detection on the first video frame to determine a target vehicle to be tracked;

[0012] A second video frame is extracted from the vehicle driving video, and the target vehicle is tracked according to the second video frame to obtain a vehicle tracking sequence.

[0013] In one embodiment, performing vehicle detection on the first video frame to determine a target vehicle to be tracked includes:

[0014] Inputting the first video frame into a trained vehicle detection model to obtain a candidate detection frame corresponding to each candidate vehicle;

[0015] Selecting a target detection frame that meets the tracking condition from the candidate detection frames;

[0016] A target vehicle to be tracked is determined from the candidate vehicles according to the target detection frame.

[0017] In one embodiment, tracking the target vehicle according to the second video frame to obtain a vehicle tracking sequence includes:

[0018] Inputting each of the second video frames into a trained vehicle tracking model to obtain a tracking detection frame corresponding to the target vehicle in each of the second video frames;

[0019] A target vehicle image corresponding to the target vehicle is extracted from the corresponding second video frame according to each tracking detection frame to obtain a vehicle tracking sequence.

[0020] In one embodiment, extracting the first image set and the last image set from the vehicle tracking sequence includes:

[0021] Extracting a first number of target vehicle images from the head of the vehicle tracking sequence to obtain a head image set;

[0022] A second number of target vehicle images are extracted from the tail of the vehicle tracking sequence to obtain a tail image set.

[0023] In one embodiment, determining the body part of the target vehicle corresponding to each frame of the target vehicle image in the head image set and the tail image set includes:

[0024] Each frame of the target vehicle image in the head image set and the tail image set is input into a trained classification model to obtain the body part of the target vehicle corresponding to each frame of the target vehicle image.

[0025] In one embodiment, the identifying the driving direction of the target vehicle according to the vehicle body part includes:

[0026] According to the body part corresponding to the target vehicle in each frame of the target vehicle image of the first image set, voting is performed on each body part, and the body part with the most votes is determined as the body part corresponding to the target vehicle in the first image set;

[0027] According to the body part corresponding to the target vehicle in each frame of the target vehicle image in the tail image set, voting is performed on each body part, and the body part with the most votes is determined as the body part corresponding to the target vehicle in the tail image set;

[0028] The traveling direction of the target vehicle is identified according to the body parts of the target vehicle corresponding to the head image set and the tail image set.

[0029] A device for identifying a vehicle's driving direction, the device comprising:

[0030] An acquisition module, used to acquire vehicle driving video;

[0031] A tracking module, used for tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence;

[0032] An extraction module, used for extracting a head image set and a tail image set from the vehicle tracking sequence;

[0033] A classification module, used to determine the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set;

[0034] The identification module is used to identify the driving direction of the target vehicle according to the vehicle body part.

[0035] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0036] Obtain vehicle driving video;

[0037] Tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence;

[0038] Extracting a first image set and a last image set from the vehicle tracking sequence;

[0039] Determine the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set;

[0040] The driving direction of the target vehicle is identified according to the vehicle body part.

[0041] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0042] Obtain vehicle driving video;

[0043] Tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence;

[0044] Extracting a first image set and a last image set from the vehicle tracking sequence;

[0045] Determine the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set;

[0046] The driving direction of the target vehicle is identified according to the vehicle body part.

[0047] The above-mentioned method, device, computer equipment and storage medium for identifying the driving direction of a target vehicle in a vehicle driving video, when it is necessary to identify the driving direction of a target vehicle in a vehicle driving video, automatically track the target vehicle based on the vehicle driving video to obtain a corresponding vehicle tracking sequence, then extract the head image set and the tail image set corresponding to the target vehicle from the vehicle tracking sequence, and respectively identify the body parts corresponding to the target vehicle in each frame of the target vehicle image in the head image set and the tail image set, so as to quickly and accurately identify the driving direction of the target vehicle based on the body parts corresponding to each frame of the target vehicle image in the head image set and the tail image set. In this way, the vehicle tracking sequence of the target vehicle is determined by automatic tracking, and then the body parts corresponding to the head and tail target vehicle images of the vehicle tracking sequence are automatically identified, and the driving direction of the target vehicle is automatically determined according to the identified body parts, so as to improve the recognition efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of a method for identifying a vehicle driving direction in one embodiment;

[0049] Figure 2 A schematic diagram of the effect of classifying vehicle body parts in a target vehicle image in one embodiment;

[0050] Figure 3 is a schematic diagram of video frames extracted from the head, middle and tail of a vehicle driving video corresponding to a vehicle in the same direction, respectively, in one embodiment;

[0051] Figure 4 is a schematic diagram of video frames extracted from the head, middle and tail of a vehicle driving video corresponding to an oncoming vehicle, respectively, in one embodiment;

[0052] Figure 5 A schematic diagram of the principle of a method for identifying a vehicle's driving direction provided in one embodiment;

[0053] Figure 6 is a structural block diagram of a device for identifying a vehicle's driving direction in one embodiment;

[0054] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] In one embodiment, Figure 1 As shown, a method for identifying the driving direction of a vehicle is provided. This embodiment takes the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0057] Step 102, obtaining vehicle driving video.

[0058] Among them, the vehicle driving video can specifically be an overtaking video, or other types of vehicle driving videos, as long as it is compatible with the vehicle driving direction identification method provided in this application. The overtaking video is a video shot in an overtaking scene. The overtaking scene refers to a vehicle equipped with an image acquisition device in a high-speed running state, and its running speed is higher than the running speed of vehicles in the same direction, so as to continuously surpass vehicles in the same direction. If the vehicle equipped with the image acquisition device is recorded as the photographer's vehicle, then in the process of the photographer's vehicle continuously moving forward, the image acquisition device mounted on the photographer's vehicle dynamically captures the vehicle driving video, and the vehicle driving video includes vehicles in the same direction and / or oncoming vehicles currently traveling on the corresponding highway section, so as to achieve vehicle tracking and shooting.

[0059] In one embodiment, since the shooting vehicle is in a high-speed running state, the tracked vehicles (vehicles in the same direction and / or vehicles in the opposite direction) are also in a continuous moving state, and the image acquisition device can realize continuous tracking and shooting of the vehicle during its moving process, that is, it can capture a tracking video with a longer duration. For any vehicle in the tracking video, it only appears in some video frames in the tracking video. Therefore, by performing preliminary tracking on each vehicle in the tracking video, the vehicle driving video corresponding to each vehicle can be extracted from the tracking video. For example, assuming that the image acquisition device captures a 3-minute tracking video, and the tracking video includes 10 vehicles, then through preliminary tracking, the vehicle driving videos corresponding to each of the 10 vehicles can be extracted from the tracking video. It can be understood that the preliminary tracking process can be implemented by the vehicle tracking model provided in one or more embodiments of the present application, and can also be implemented by the existing vehicle tracking method. The specific tracking method is not specifically limited here, as long as the vehicle driving video corresponding to each vehicle can be extracted from the tracking video.

[0060] In one embodiment, the image acquisition device may be a camera or a webcam. The image acquisition device may be integrated into the terminal as a component of the terminal, or may be deployed separately as a device independent of the terminal and communicate with the terminal through a network.

[0061] Step 104: Track the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence.

[0062] The vehicle tracking sequence is an image sequence obtained by tracking a target vehicle in a vehicle driving video and includes images of the target vehicle corresponding to the target vehicle in each video frame in the vehicle driving video.

[0063] Specifically, the terminal tracks the target vehicle in the vehicle driving video in a time sequence according to each video frame in the vehicle driving video, obtains the target vehicle image corresponding to the target vehicle in each video frame, and obtains the corresponding vehicle tracking sequence according to the target vehicle image corresponding to the target vehicle in each video frame.

[0064] In one embodiment, if a vehicle driving video includes multiple candidate vehicles to be tracked, the terminal can select one as the target vehicle and track the target vehicle according to each video frame to obtain a corresponding vehicle tracking sequence. The terminal can also use all the multiple candidate vehicles as target vehicles and track each target vehicle separately according to each video frame to obtain a vehicle tracking sequence corresponding to each target vehicle.

[0065] Step 106: extracting a head image set and a tail image set from the vehicle tracking sequence.

[0066] Among them, the head image set is an image set including one or more frames of target vehicle images at the head of the vehicle tracking sequence, and the tail image set is an image set including one or more frames of target vehicle images at the tail of the vehicle tracking sequence. The head is relative to the tail, the head corresponds to the starting position, and the tail corresponds to the ending position.

[0067] Specifically, the terminal extracts one or more target vehicle images from the head of the vehicle tracking sequence in time sequence to obtain a head image set, and extracts one or more target vehicle images from the tail of the vehicle tracking sequence to obtain a tail image set.

[0068] In one embodiment, step 106 includes: extracting a first number of target vehicle images from the head of the vehicle tracking sequence to obtain a head image set; and extracting a second number of target vehicle images from the tail of the vehicle tracking sequence to obtain a tail image set.

[0069] The first number and the second number may be the same or different, and may be customized according to needs, such as 1 or 5. Specifically, the terminal extracts a first number of target vehicle images from the head of the vehicle tracking sequence in time sequence to obtain a head image set, and extracts a second number of target vehicle images from the tail of the vehicle tracking sequence to obtain a tail image set.

[0070] In one embodiment, the first number and the second number are determined by the preset first extraction ratio and the preset second extraction ratio, respectively. For example, the first number is obtained by multiplying the total number of frames of the vehicle tracking sequence by the first extraction ratio, and the second number is obtained by multiplying the total number of frames of the vehicle tracking sequence by the second extraction ratio. The first extraction ratio and the second extraction ratio can be the same or different, and can be customized according to actual needs, such as 20%.

[0071] In one embodiment, the terminal extracts a first number of target vehicle images from the head of the vehicle tracking sequence in a time sequence. Specifically, the terminal may extract a first number of continuous target vehicle images from the head, for example, extract five frames of target vehicle images from the first frame to the fifth frame from the vehicle tracking sequence, or may extract a first number of target vehicle images from the head according to a preset step length, for example, extract five frames of target vehicle images from the first frame, the third frame, the fifth frame, the seventh frame and the ninth frame from the vehicle tracking sequence. The method for extracting the target vehicle images in the tail image set is similar and will not be repeated here.

[0072] In one embodiment, the terminal extracts one or more frames of target vehicle images from the first preset position of the vehicle tracking sequence in time sequence to obtain a head image set, and extracts one or more frames of target vehicle images from the second preset position of the vehicle tracking sequence to obtain a tail image set. The first preset position and the second preset position can be dynamically adjusted according to demand, such as the 20% position and the 80% position of the vehicle tracking sequence respectively.

[0073] In the above embodiment, the head image set and the tail image set corresponding to the target vehicle are extracted from the head and the tail of the vehicle tracking sequence respectively, so as to classify the body parts corresponding to the target vehicle in each target vehicle image in the head image set and the tail image set, and identify its driving direction based on the classification result. Compared with classifying the body parts of the target vehicle images in the entire vehicle tracking sequence, it is possible to improve the recognition efficiency and the recognition accuracy.

[0074] Step 108 , determining the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set.

[0075] Among them, the body part of the target vehicle corresponding to the target vehicle image may refer to the part of the body of the target vehicle that appears in the target vehicle image, or may refer to the part of the body of the target vehicle that faces the image acquisition device when the video frame corresponding to the target vehicle image is collected. The body part may also be understood as a body part category, which may specifically include the front, rear, middle of the body, and others. The front / rear may refer to the front / rear of the target vehicle included in the target vehicle image, or may refer to the front / rear of the target vehicle facing the image acquisition device when the video frame corresponding to the target vehicle image is collected. The middle of the body refers to the body / middle of the target vehicle being approximately perpendicular to the image acquisition device when the video frame corresponding to the target vehicle image is collected. When the body part is others, the corresponding target vehicle image may be understood as an illegal input category, which may specifically refer to the video frame corresponding to the target vehicle image being a video frame collected inside the target vehicle, or the target vehicle image does not include any part of the body of the target vehicle.

[0076] Specifically, the terminal classifies the body parts corresponding to the target vehicle in each frame of the target vehicle image in the head image set to determine the body parts corresponding to the target vehicle in each frame of the target vehicle image in the head image set, and classifies the body parts corresponding to the target vehicle in each frame of the target vehicle image in the tail image set to determine the body parts corresponding to the target vehicle in each frame of the target vehicle image in the tail image set.

[0077] Figure 2 FIG. 1 is a schematic diagram showing the effect of classifying the body parts in the target vehicle image in one embodiment. Figure 2, the body parts corresponding to the vehicle in the target vehicle image include the front, the rear, the middle of the body and others, wherein, reference numeral 21 is an example in which the body part corresponding to the vehicle in the target vehicle image is the front of the vehicle, and when the video frame corresponding to the target vehicle image is acquired, the front of the vehicle is facing the lens of the image acquisition device, reference numeral 22 is an example in which the body part corresponding to the vehicle in the target vehicle image is the rear of the vehicle, and when the video frame corresponding to the target vehicle image is acquired, the rear of the vehicle is facing the lens of the image acquisition device, reference numeral 23 is an example in which the body part corresponding to the vehicle in the target vehicle image is the middle of the vehicle, and when the video frame corresponding to the target vehicle image is acquired, the body / middle of the vehicle is approximately perpendicular to the lens of the image acquisition device, reference numeral 24 is an example in which the body part corresponding to the vehicle in the target vehicle image is others, and when the video frame corresponding to the target vehicle image is acquired, the internal components (steering wheel) of the vehicle are facing the lens of the image acquisition device.

[0078] Step 110, identifying the driving direction of the target vehicle according to the vehicle body parts.

[0079] Among them, the driving direction of the target vehicle may specifically refer to the driving direction of the target vehicle relative to the photographer's vehicle, which may specifically include the same direction and the opposite direction. The same direction means that the driving directions of the target vehicle and the photographer's vehicle are the same, and the opposite direction means that the driving directions of the target vehicle and the photographer's vehicle are opposite / opposite.

[0080] Specifically, the terminal identifies the corresponding body parts of the target vehicle in the vehicle driving video based on the corresponding body parts of the target vehicle in each frame of the target vehicle image in the first image set, and the corresponding body parts of the target vehicle in each frame of the target vehicle image in the tail image set, so as to determine the driving direction of the target vehicle compared to the photographer's vehicle.

[0081] In one embodiment, the terminal determines the corresponding body part of the target vehicle in the head image set according to the corresponding body part in each frame of the target vehicle image in the head image set, and determines the corresponding body part of the target vehicle in the tail image set according to the corresponding body part in each frame of the target vehicle image in the tail image set, and determines the corresponding body part of the target vehicle in the vehicle driving video according to the corresponding body parts of the target vehicle in the head image set and the tail image set. In one embodiment, based on the vehicle driving direction recognition method provided in one or more embodiments of the present application, the corresponding driving mode of each target vehicle in the corresponding vehicle driving video can be identified. If the tracking video obtained by tracking shooting includes multiple target vehicles to be tracked, after identifying the driving direction corresponding to each target vehicle, the video clips corresponding to each target vehicle with the driving direction as the target direction can be edited from the tracking video based on the driving direction of each target vehicle, so as to facilitate subsequent processing of the edited video clips. If the target direction is the same direction, for example, the edited video clips are processed with special effects, and if the target direction is an object, for example, the illegal wrong-way vehicle can be identified based on the edited video clips.

[0082] In one embodiment, if the tracking video is a panoramic video, after identifying the driving direction of each target vehicle in the panoramic video through the vehicle driving direction recognition method provided in one or more embodiments of the present application, the panoramic video can be automatically edited into a two-dimensional video such as an overtaking video, a same-direction driving video or a meeting video according to the driving direction of each target vehicle and the video shooting direction / angle.

[0083] In one embodiment, after the driving direction of each target vehicle in the tracking video is identified through the vehicle driving direction identification method provided in one or more embodiments of the present application, it is possible to judge whether each target vehicle in the tracking video has problems such as illegal wrong-way driving based on the driving direction of each target vehicle, thereby being able to detect target vehicles that have illegal wrong-way driving from the tracking video.

[0084] The above-mentioned method for identifying the driving direction of a vehicle, when it is necessary to identify the driving direction of a target vehicle in a vehicle driving video, automatically tracks the target vehicle based on the vehicle driving video to obtain a corresponding vehicle tracking sequence, then extracts the head image set and the tail image set corresponding to the target vehicle from the vehicle tracking sequence, and respectively identifies the body parts corresponding to each frame of the target vehicle image in the head image set and the tail image set, so as to quickly and accurately identify the driving direction of the target vehicle based on the body parts corresponding to each frame of the target vehicle image in the head image set and the tail image set. In this way, the vehicle tracking sequence of the target vehicle is determined by automatic tracking, and then the body parts corresponding to the head and tail target vehicle images of the vehicle tracking sequence are automatically identified, and the driving direction of the target vehicle is automatically determined according to the identified body parts, so as to improve the recognition efficiency and accuracy.

[0085] In one embodiment, step 104 includes: extracting a first video frame from the vehicle driving video, performing vehicle detection on the first video frame to determine a target vehicle to be tracked; extracting a second video frame from the vehicle driving video, and tracking the target vehicle according to the second video frame to obtain a vehicle tracking sequence.

[0086] The vehicle driving video includes a first video frame and a second video frame, the first video frame may be one or more video frames at the head (i.e., the starting position) of the vehicle driving video, and the second video frame may be each video frame in the vehicle driving video except the first video frame. For example, assuming that the vehicle driving video includes 500 video frames, the first video frame in the vehicle driving video may be extracted as the first video frame in time sequence, and the subsequent 499 video frames may be extracted as the second video frames.

[0087] Specifically, the terminal extracts a first video frame and a second video frame from the vehicle driving video, performs vehicle detection on the first video frame to obtain a candidate vehicle to be tracked in the first video frame, and selects a target vehicle to be tracked from the candidate vehicles. Further, the terminal tracks the selected target vehicle according to each second video frame to obtain a vehicle tracking sequence corresponding to the target vehicle.

[0088] In one embodiment, if the first video frame is a single video frame, the terminal directly selects the target vehicle to be tracked from the candidate vehicles in the first video frame. Specifically, a candidate vehicle may be randomly selected as the target vehicle, or it may be selected based on parameters such as the detection frame confidence and / or detection frame area corresponding to each candidate vehicle, such as selecting the candidate vehicle with the highest detection frame confidence as the target vehicle. If the first video frame is a plurality of video frames, the terminal determines the target vehicle to be tracked by comprehensively considering the candidate vehicles in each first video frame, such as summing the detection frame confidence corresponding to each candidate vehicle in each first video frame to obtain the confidence sum value corresponding to each candidate vehicle, and selecting the candidate vehicle with the largest confidence sum value from each candidate vehicle as the target vehicle.

[0089] In the above embodiment, a target vehicle to be tracked is determined based on the first video frame in the vehicle driving video, and the target vehicle is tracked based on the second video frame in the vehicle driving video. The vehicle tracking sequence corresponding to the target vehicle can be accurately obtained, so that when the driving direction of the target vehicle is identified based on the vehicle tracking sequence with higher accuracy, the recognition accuracy can be improved.

[0090] In one embodiment, vehicle detection is performed on the first video frame to determine a target vehicle to be tracked, including: inputting the first video frame into a trained vehicle detection model to obtain a candidate detection frame corresponding to each candidate vehicle; screening a target detection frame that meets the tracking conditions from the candidate detection frame; and determining the target vehicle to be tracked from the candidate vehicles based on the target detection frame.

[0091] Among them, the vehicle detection model is a model obtained by training based on the first sample and can be used to detect the candidate detection frame corresponding to each candidate vehicle from the first video frame. The first training sample set includes multiple first sample images, and a sample detection frame corresponding to each vehicle in each first sample image. The sample detection frame in the first training sample set can be specifically represented by the detection frame data, and the detection frame data can be used to uniquely locate the corresponding vehicle in the first sample image, and specifically may include the coordinates corresponding to the two diagonal vertices of the sample detection frame, or the coordinates corresponding to any vertex of the sample detection frame, and the height and width of the sample detection frame, and may also include the detection frame area corresponding to the sample detection frame. It can be understood that the first sample training set may also include the detection frame confidence corresponding to each sample detection frame.

[0092] In the training stage of the vehicle detection model, the first sample image is used as an input feature, and the corresponding sample detection frame (and the detection frame confidence corresponding to the sample detection frame) is used as the desired output feature for model training to obtain a trained vehicle detection model. The tracking condition is a condition or basis for screening the target detection frame from the candidate detection frame, so as to determine the target vehicle to be tracked from each candidate vehicle based on the screened target detection frame, and may specifically include the detection frame confidence being greater than or equal to the confidence threshold, and may also include the detection frame area being greater than or equal to the first area threshold and less than or equal to the second area threshold, and the first area threshold being less than the second area threshold.

[0093] Specifically, the terminal inputs the first video frame into the trained vehicle detection model for vehicle detection, obtains a candidate detection frame corresponding to each candidate vehicle in the first video frame, compares each candidate detection frame with a preset tracking condition, and selects a target detection frame that meets the tracking condition from the candidate detection frame. And the target vehicle to be tracked is determined from each candidate vehicle according to the selected target detection frame.

[0094] In one embodiment, the terminal performs vehicle detection on the first video frame through a trained vehicle detection model, outputs detection frame data and detection frame confidence corresponding to the candidate detection frame corresponding to each candidate vehicle, extracts the detection frame area from the detection frame data corresponding to each candidate detection frame, and selects the candidate detection frames whose detection frame areas are greater than or equal to the first area threshold and less than or equal to the second area threshold, and whose detection frame confidence is greater than or equal to the confidence threshold from each candidate detection frame as target detection frames. In this way, based on the confidence threshold, low-confidence candidate detection frames are filtered out, high-confidence candidate detection frames are retained, and based on the first area threshold and the second area threshold, too large or too small candidate detection frames are eliminated to obtain a target detection frame, so as to determine the target vehicle to be tracked based on the target detection frame, and when tracking the target vehicle, a more accurate vehicle tracking sequence can be obtained, thereby improving the recognition accuracy of the vehicle's driving direction.

[0095] In one embodiment, the vehicle detection model may be a deep neural network model capable of realizing vehicle detection. The vehicle detection model includes but is not limited to a model trained based on any one of machine learning algorithms such as Faster RCNN, RetinaNet, and CenterNet. The first sample image in the first training sample set may be specifically selected from public data sets such as COCO and OpenImages. It is understandable that images of vehicle categories are selected from public data sets such as COCO and OpenImages as the first sample image.

[0096] In the above embodiment, with the help of the trained vehicle detection model, the candidate detection frames corresponding to the candidate vehicles are quickly and accurately detected from the first video frame, and the target detection frames that meet the tracking conditions are screened from the candidate detection frames with higher accuracy, so that the screened target detection frames can quickly and accurately screen out the target vehicle to be tracked from the candidate vehicles.

[0097] In one embodiment, a target vehicle is tracked according to a second video frame to obtain a vehicle tracking sequence, including: inputting each second video frame into a trained vehicle tracking model to obtain a tracking detection frame corresponding to the target vehicle in each second video frame; and extracting a target vehicle image corresponding to the target vehicle from the corresponding second video frame according to each tracking detection frame to obtain a vehicle tracking sequence.

[0098] The vehicle tracking model is a model trained according to a pre-acquired second training sample set and can be used to track the vehicle to be tracked. The second training sample set includes a plurality of second sample images, and a tracking detection frame corresponding to each vehicle in each second sample image in the second sample image. The tracking detection frame in the second training sample set can be specifically represented by detection frame data, and the detection frame data can be used to uniquely locate the corresponding vehicle in the second sample image, and specifically may include the coordinates corresponding to the two diagonal vertices of the tracking detection frame, or the coordinates corresponding to any vertex of the tracking detection frame, and the height and width of the tracking detection frame.

[0099] Specifically, the terminal tracks each second video frame through the trained vehicle tracking model to obtain the tracking detection frame corresponding to the target vehicle in each second video frame, extracts the target vehicle image corresponding to the target vehicle from the corresponding second video frame according to each tracking detection frame, and obtains the corresponding vehicle tracking sequence according to the time sequence according to the target vehicle images corresponding to the target vehicle.

[0100] In one embodiment, the terminal extracts a partial image corresponding to the corresponding tracking detection frame from each second video frame, and can use the extracted partial image as the target vehicle image corresponding to the target vehicle in the second video frame, or can scale the size of the partial image to the target size to obtain the target vehicle image corresponding to the target vehicle in the second video frame. The target size can be customized according to actual needs, such as 224*224.

[0101] In one embodiment, the terminal saves the target vehicle image extracted from each second video frame corresponding to the target vehicle into a list object until the target vehicle is tracked. For example, assuming that the vehicle driving video includes 200 frames of second video frames, a local image corresponding to the target vehicle is captured from each second video frame, the size of the captured local image is scaled to the target size, the target vehicle image corresponding to the target vehicle is obtained, and each target vehicle image corresponding to the target vehicle is saved into a tensor object of the form [200, 224, 224, 3].

[0102] In one embodiment, the vehicle tracking sequence corresponding to the target vehicle may also include a target vehicle image corresponding to the target vehicle in the first video frame. After the target vehicle to be tracked is determined from each candidate vehicle based on the target detection frame, a partial image corresponding to the target vehicle is captured from the corresponding first video frame according to the target detection frame, and the target vehicle image corresponding to the target vehicle is obtained based on the partial image.

[0103] In one embodiment, the vehicle detection model includes but is not limited to a model trained based on any one of open source machine learning algorithms such as STAPLE, CentreTrack, and SiamRPN.

[0104] In the above embodiment, by tracking the target vehicle with the help of the trained vehicle tracking model, the tracking detection frame corresponding to the target vehicle in each second video frame can be quickly and accurately obtained, so that the vehicle tracking sequence corresponding to the target vehicle can be quickly and accurately obtained based on the tracking detection frame.

[0105] In one embodiment, step 108 includes: inputting each frame of the target vehicle image in the head image set and the tail image set into a trained classification model to obtain the body part of the target vehicle corresponding to each frame of the target vehicle image.

[0106] The classification model is a model trained based on the trained third training sample set and can be used to classify the body parts corresponding to the target vehicle in the target vehicle image. The third training sample set includes multiple frames of third sample images corresponding to multiple vehicles, and the body parts corresponding to each vehicle in each corresponding frame of the third sample image, and may also include the probability value corresponding to each body part. In the training stage of the classification model, the third sample image is used as the input feature, and the corresponding body part (the probability value corresponding to the body part) is used as the desired output feature for model training to obtain a trained classification model.

[0107] Specifically, the terminal inputs each frame of the target vehicle image in the head image set and each frame of the target vehicle image in the tail image set into the trained classification model respectively, and classifies the body parts corresponding to the target vehicle in the input target vehicle image through the classification model to obtain the body parts corresponding to the target vehicle in each frame of the target vehicle image in the head image set and the tail image set.

[0108] In one embodiment, the terminal classifies the body parts in the input target vehicle image through a trained classification model, and can output the classified body parts, and determine the output body parts as the body parts corresponding to the target vehicle in the target vehicle image, or output the probability values ​​corresponding to the classified body parts and each body part, and select the body part with the largest probability value as the body part corresponding to the target vehicle in the target vehicle image.

[0109] In one embodiment, the classification model can be a deep convolutional neural network model that can be used to implement classification. The classification model includes but is not limited to being trained based on any one of the machine learning algorithms such as MobileNetv2, EfficientNet and ResNet. In the training stage of the classification model, the classification model is trained based on any of the above machine learning algorithms using a gradient descent algorithm.

[0110] In one embodiment, during the training phase of the classification model, a verification sample set is also obtained. After the classification model is trained based on the third training sample set, the trained classification model is verified using the verification sample set, and the verified classification model is determined as the trained classification model.

[0111] In the above embodiment, the body parts are classified with the help of the trained classification model, so that the body parts corresponding to the target vehicle in each target vehicle image in the head image set and the tail image can be quickly and accurately determined, so as to improve the recognition accuracy and efficiency when identifying the driving direction of the target vehicle based on the body parts.

[0112] In one embodiment, step 110 includes: voting on each body part according to the body part corresponding to the target vehicle in each frame of the target vehicle image in the head image set, and determining the body part with the largest number of votes as the body part corresponding to the target vehicle in the head image set; voting on each body part according to the body part corresponding to the target vehicle in each frame of the target vehicle image in the tail image set, and determining the body part with the largest number of votes as the body part corresponding to the target vehicle in the tail image set; identifying the driving direction of the target vehicle according to the body parts corresponding to the target vehicle in the head image set and the tail image set.

[0113] In one embodiment, the terminal votes for the body parts corresponding to the target vehicle in the head image set and the tail image set according to the body parts corresponding to each target vehicle image in the head image set and the tail image set, respectively, according to the following voting function, to obtain the body parts corresponding to the target vehicle in the head image set and the tail image set.

[0114] O 首部 =Vote(f θ (I i )|I i ∈ first image set)

[0115] O 尾部 =Vote(f θ (I j )|I j ∈ tail image set)

[0116] Among them, I i with I j Respectively represent the single-frame target vehicle images in the first image set and the last image set, O 首部 With O 尾部 They represent the corresponding body parts of the target vehicle in the first image set and the last image set, Vote() represents the voting function, and f θ () represents the trained classification model, f θ (I i ) and f θ (I j ) represent the trained classification model f θ (), classify the body parts of the target vehicle in the corresponding target vehicle image.

[0117] In one embodiment, the body part corresponding to the target vehicle in the first image set refers to the body part of the target vehicle that appears first in the corresponding vehicle driving video, and the body part corresponding to the target vehicle in the tail image set refers to the body part of the target vehicle that appears later in the corresponding vehicle driving video. Therefore, based on the body parts corresponding to the target vehicle in the first image set and the tail image set, respectively, according to the processing logic in the following table, the driving direction of the target vehicle in the vehicle driving video can be identified.

[0118]

[0119]

[0120] It can be understood that for the combination of the head and tail classification results not listed in the above table, the target vehicle's driving direction cannot be accurately determined, and thus the corresponding recognition result of the target vehicle can be determined as "unable to determine". For the target vehicle with the recognition result of "unable to determine", the terminal can also push the corresponding vehicle driving video to the user and play and display it, so that the user can determine and feedback the target vehicle's driving direction based on the vehicle driving video, and update the recognition result of the target vehicle based on the driving direction fed back by the user.

[0121] Figure 3 FIG. 1 is a schematic diagram of video frames extracted from the head, middle and tail of a vehicle driving video corresponding to a vehicle in the same direction, respectively, in one embodiment. Figure 3 During the tracking and shooting process of the vehicle driving video corresponding to the same-direction vehicle, the image acquisition device for collecting the vehicle driving video is mounted on the photographer's vehicle. The photographer's vehicle drives on one side of the road and continuously overtakes the same-direction vehicle, so as to collect the vehicle driving video corresponding to the same-direction vehicle through the image acquisition device. The single video frames extracted from the head, middle and tail of the vehicle driving video are represented by the numbers 01, 0 and 02 respectively. The corresponding body parts of the target vehicle in the extracted head video frame, middle video frame and tail video frame are the rear, middle and front of the vehicle respectively. It can be seen that for the same-direction vehicle (the target vehicle traveling in the same direction as the photographer's vehicle), the body part that appears first in the corresponding vehicle tracking sequence is usually the rear of the vehicle.

[0122] Figure 4 FIG. 1 is a schematic diagram of video frames extracted from the head, middle and tail of a vehicle driving video corresponding to an oncoming vehicle in an embodiment. Figure 4 During the tracking and shooting process of the vehicle driving video corresponding to the oncoming vehicle, the image acquisition device for collecting the vehicle driving video is mounted on the photographer's vehicle. The photographer's vehicle travels on one side of the road and continuously overtakes the vehicles in the same direction, so as to collect the vehicle driving video corresponding to the oncoming vehicle through the image acquisition device. The single video frames extracted from the head, middle and tail of the vehicle driving video are represented by the numbers 06, 0 and 07 respectively. The corresponding body parts of the target vehicle in the extracted head video frame, middle video frame and tail video frame are the front, middle and tail of the vehicle respectively. It can be seen that for the oncoming vehicle (the target vehicle traveling in the opposite direction of the photographer's vehicle), the body part that appears first in the corresponding vehicle tracking sequence is usually the front of the vehicle.

[0123] In the above embodiment, by analyzing each frame of the target vehicle image in the first image set and the last image set of the vehicle tracking sequence respectively, the body part of the target vehicle that first / last appears in the field of view of the image acquisition device during the vehicle tracking process is determined to identify the driving direction of the target vehicle compared to the photographer's vehicle.

[0124] Figure 5 FIG. 1 is a schematic diagram of the principle of a method for identifying a vehicle's driving direction provided in an embodiment. Figure 5 , obtain a vehicle driving video, perform vehicle detection on the first video frame in the vehicle driving video through a trained vehicle detection model, so as to determine the target vehicle to be tracked according to the vehicle detection result, track the target vehicle according to the second video frame in the vehicle driving video through a trained vehicle tracking model, and obtain a vehicle tracking sequence, extract a head image set and a tail image set from the head and the tail of the vehicle tracking sequence respectively, determine the body part corresponding to the target vehicle in each frame of the target vehicle image in the head image set and the tail image set through a trained classification model, so as to further determine the body part corresponding to the target vehicle in the head image set and the tail image set, and identify the driving direction of the target vehicle based on the body parts corresponding to the target vehicle in the head image set and the tail image set.

[0125] In the above embodiment, a vehicle tracking sequence is obtained by tracking the target vehicle based on the vehicle driving video, and the body parts such as the front and rear of the vehicle are identified to determine the body parts corresponding to the head image set and the rear image set in the vehicle tracking sequence, respectively, so as to determine the direction of the front and rear of the vehicle based on the determined body parts, so as to further intelligently identify the relative travel direction of the target vehicle and the photographer's vehicle, thereby improving the recognition accuracy and the recognition efficiency.

[0126] It should be understood that although Figure 1 and Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 and Figure 5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0127] In one embodiment, Figure 6As shown, a vehicle driving direction recognition device 600 is provided, comprising: an acquisition module 601, a tracking module 602, an extraction module 603, a classification module 604 and a recognition module 605, wherein:

[0128] The acquisition module 601 is used to acquire the vehicle driving video;

[0129] A tracking module 602 is used to track the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence;

[0130] An extraction module 603 is used to extract a first image set and a last image set from a vehicle tracking sequence;

[0131] A classification module 604, used to determine the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set;

[0132] The identification module 605 is used to identify the driving direction of the target vehicle according to the body parts.

[0133] In one embodiment, the tracking module 602 is also used to extract a first video frame from the vehicle driving video, perform vehicle detection on the first video frame to determine the target vehicle to be tracked; extract a second video frame from the vehicle driving video, and track the target vehicle according to the second video frame to obtain a vehicle tracking sequence.

[0134] In one embodiment, the tracking module 602 is also used to input the first video frame into a trained vehicle detection model to obtain a candidate detection frame corresponding to each candidate vehicle; filter a target detection frame that meets the tracking conditions from the candidate detection frame; and determine the target vehicle to be tracked from the candidate vehicles based on the target detection frame.

[0135] In one embodiment, the tracking module 602 is also used to input each second video frame into a trained vehicle tracking model to obtain a tracking detection frame corresponding to the target vehicle in each second video frame; extract a target vehicle image corresponding to the target vehicle from the corresponding second video frame according to each tracking detection frame to obtain a vehicle tracking sequence.

[0136] In one embodiment, the extraction module 603 is further configured to extract a first number of target vehicle images from the head of the vehicle tracking sequence to obtain a head image set; and extract a second number of target vehicle images from the tail of the vehicle tracking sequence to obtain a tail image set.

[0137] In one embodiment, the classification module 604 is further used to input each frame of the target vehicle image in the head image set and the tail image set into a trained classification model to obtain the body part of the target vehicle corresponding to each frame of the target vehicle image.

[0138] In one embodiment, the recognition module 605 is further used to vote for each body part according to the body part corresponding to the target vehicle in each frame of the target vehicle image in the leading image set, and determine the body part with the largest number of votes as the body part corresponding to the target vehicle in the leading image set; vote for each body part according to the body part corresponding to the target vehicle in each frame of the target vehicle image in the trailing image set, and determine the body part with the largest number of votes as the body part corresponding to the target vehicle in the trailing image set; and identify the driving direction of the target vehicle according to the body parts corresponding to the target vehicle in the leading image set and the trailing image set.

[0139] The specific definition of the vehicle driving direction identification device can refer to the definition of the vehicle driving direction identification method mentioned above, which will not be repeated here. Each module in the above-mentioned vehicle driving direction identification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0140] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for identifying the driving direction of a vehicle is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0141] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0142] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in each method embodiment when executing the computer program.

[0143] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in each method embodiment are implemented.

[0144] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying a vehicle's driving direction, It is characterized in that The method comprises: Obtain vehicle driving video; Tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence; Extracting a first image set and a last image set from the vehicle tracking sequence; Determine the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set; Identifying the driving direction of the target vehicle according to the vehicle body part; The step of identifying the driving direction of the target vehicle according to the vehicle body part includes: According to the body part corresponding to the target vehicle in each frame of the target vehicle image of the first image set, voting is performed on each body part, and the body part with the most votes is determined as the body part corresponding to the target vehicle in the first image set; Voting for each body part according to the body part corresponding to the target vehicle in each frame of the target vehicle image in the tail image set, and determining the body part with the most votes as the body part corresponding to the target vehicle in the tail image set; The traveling direction of the target vehicle is identified according to the body parts of the target vehicle corresponding to the head image set and the tail image set.

2. The method according to claim 1, It is characterized in that The step of tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence includes: Extracting a first video frame from the vehicle driving video, and performing vehicle detection on the first video frame to determine a target vehicle to be tracked; A second video frame is extracted from the vehicle driving video, and the target vehicle is tracked according to the second video frame to obtain a vehicle tracking sequence.

3. The method according to claim 2, It is characterized in that The performing vehicle detection on the first video frame to determine a target vehicle to be tracked includes: Inputting the first video frame into a trained vehicle detection model to obtain a candidate detection frame corresponding to each candidate vehicle; Selecting a target detection frame that meets the tracking condition from the candidate detection frames; A target vehicle to be tracked is determined from the candidate vehicles according to the target detection frame.

4. The method according to claim 2, It is characterized in that The step of tracking the target vehicle according to the second video frame to obtain a vehicle tracking sequence includes: Inputting each of the second video frames into a trained vehicle tracking model to obtain a tracking detection frame corresponding to the target vehicle in each of the second video frames; A target vehicle image corresponding to the target vehicle is extracted from the corresponding second video frame according to each tracking detection frame to obtain a vehicle tracking sequence.

5. The method according to any one of claims 1 to 4, It is characterized in that The step of extracting a first image set and a last image set from the vehicle tracking sequence comprises: Extracting a first number of target vehicle images from the head of the vehicle tracking sequence to obtain a head image set; A second number of target vehicle images are extracted from the tail of the vehicle tracking sequence to obtain a tail image set.

6. The method according to any one of claims 1 to 4, It is characterized in that The determining of the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set comprises: Each frame of the target vehicle image in the head image set and the tail image set is input into a trained classification model to obtain the body part of the target vehicle corresponding to each frame of the target vehicle image.

7. A device for identifying the direction of a vehicle, It is characterized in that The device comprises: An acquisition module, used to acquire vehicle driving video; A tracking module, used for tracking the target vehicle according to the vehicle driving video to obtain a vehicle tracking sequence; An extraction module, used for extracting a head image set and a tail image set from the vehicle tracking sequence; A classification module, used to determine the body part of the target vehicle corresponding to each frame of the target vehicle image in the first image set and the last image set; An identification module is used to identify the traveling direction of the target vehicle according to the body parts; the identification module is specifically used to: vote for each body part according to the body part corresponding to the target vehicle in each frame of the target vehicle image in the head image set, and determine the body part with the largest number of votes as the body part corresponding to the target vehicle in the head image set; vote for each body part according to the body part corresponding to the target vehicle in each frame of the target vehicle image in the tail image set, and determine the body part with the largest number of votes as the body part corresponding to the target vehicle in the tail image set; identify the traveling direction of the target vehicle according to the body parts corresponding to the target vehicle in the head image set and the tail image set.

8. The device according to claim 7, It is characterized in that The tracking module is specifically used to: extract a first video frame from the vehicle driving video, perform vehicle detection on the first video frame to determine a target vehicle to be tracked; extract a second video frame from the vehicle driving video, and track the target vehicle according to the second video frame to obtain a vehicle tracking sequence.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images