Target detection method, device and storage medium

Abstract

The application provides a target detection method, device and storage medium, the method comprising: obtaining a current frame image corresponding to a current sampling time in a real-time collected video stream, the current frame image containing a target object; obtaining a first number of historical frame images with a first sampling interval multiple to obtain a first image set, and obtaining a second number of historical frame images with a second sampling interval multiple to obtain a second image set, the first sampling interval multiple being less than the second sampling interval multiple, the first number being less than the second number, and the first image set and the second image set both containing the current frame image. Feature maps corresponding to the multiple frames of images in the first image set and the second image set are respectively extracted, and the position information of the target object in a next frame of image is predicted according to the feature maps of the multiple frames of images in the first image set and the feature maps of the multiple frames of images in the second image set. Through the scheme, accurate detection of the position of the target object in the next frame can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a target detection method, device and storage medium. BACKGROUND

[0002] With the development of the network, people are entering the era of video from the era of pictures, and video data on the network is showing a trend of rapid growth. The Video Object Detection (VOD) task aims to study how to detect the position and category of the target object of interest from an input video.

[0003] However, the traditional video target detection task is offline, that is, it is assumed that the complete video segment can be obtained in advance, and the real-time nature of the video itself is not considered, which does not match the situation of many practical application scenarios, such as in the field of autonomous driving. Therefore, a new sub-task of stream-aware target detection is generated in the video target detection task, which considers the real-time nature of the video and requires the algorithm to have the ability to predict the position of the target object in the future frame image according to the current frame image and the past frame image (i.e. historical frame). SUMMARY

[0004] The embodiments of the present application provide a target detection method, device, equipment and storage medium, which can realize accurate target detection for real-time video stream.

[0005] In a first aspect, the embodiments of the present application provide a target detection method, which comprises:

[0006] obtaining a current frame image corresponding to a current sampling time in a real-time collected video stream, the current frame image containing a target object;

[0007] obtaining a first number of historical frame images with a first sampling interval multiple to obtain a first image set, and obtaining a second number of historical frame images with a second sampling interval multiple to obtain a second image set; wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both contain the current frame image;

[0008] extracting the feature maps corresponding to each of the multiple frames of images in the first image set and the feature maps corresponding to each of the multiple frames of images in the second image set;

[0009] predicting the position information of the target object in the next frame of image according to the feature maps corresponding to each of the multiple frames of images in the first image set and the feature maps corresponding to each of the multiple frames of images in the second image set.

[0010] In a second aspect, an embodiment of the present application provides a target detection device, the device comprising:

[0011] a sampling module configured to acquire a current frame image corresponding to a current sampling time in a real-time collected video stream, the current frame image containing a target object;

[0012] an acquisition module configured to acquire a first number of historical frame images to obtain a first image set at a first sampling interval multiple, and acquire a second number of historical frame images to obtain a second image set at a second sampling interval multiple, wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both contain the current frame image;

[0013] a detection module configured to extract a feature map corresponding to each of a plurality of frame images in the first image set, and extract a feature map corresponding to each of a plurality of frame images in the second image set, and predict position information of the target object in a next frame image according to the feature map corresponding to each of the plurality of frame images in the first image set and the feature map corresponding to each of the plurality of frame images in the second image set.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a communication interface, wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the target detection method according to the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a non-transitory machine readable storage medium, wherein the non-transitory machine readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the target detection method according to the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a target detection method, the method comprising:

[0017] receiving a request triggered by calling a target detection service by a vehicle terminal device, the request containing a video stream collected in real time by the vehicle terminal device on a front road environment;

[0018] using a processing resource corresponding to the target detection service to perform the following steps:

[0019] acquiring a current frame image corresponding to a current sampling time in the video stream, the current frame image containing a target object;

[0020] acquire a first number of historical frame images with a first sampling interval multiple to obtain a first image set, and acquire a second number of historical frame images with a second sampling interval multiple to obtain a second image set; wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both contain the current frame image;

[0021] extract feature maps corresponding to the multiple frame images in the first image set, and extract feature maps corresponding to the multiple frame images in the second image set;

[0022] predict the position information of the target object in the next frame image according to the feature maps corresponding to the multiple frame images in the first image set and the feature maps corresponding to the multiple frame images in the second image set.

[0023] In a sixth aspect, an embodiment of the present application provides a target detection method applied to a vehicle terminal device, and the method comprises:

[0024] acquire a current frame image corresponding to a current sampling time in a video stream collected in real time from a front road environment, and the current frame image contains a target object;

[0025] acquire a first number of historical frame images with a first sampling interval multiple to obtain a first image set, and acquire a second number of historical frame images with a second sampling interval multiple to obtain a second image set; wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both contain the current frame image;

[0026] extract feature maps corresponding to the multiple frame images in the first image set, and extract feature maps corresponding to the multiple frame images in the second image set;

[0027] predict the position information of the target object in the next frame image according to the feature maps corresponding to the multiple frame images in the first image set and the feature maps corresponding to the multiple frame images in the second image set.

[0028] perform automatic driving control processing according to the position information.

[0029] For example, in an automatic driving scenario, the vehicle terminal device can collect a video stream of a front road environment in real time, sample frame images from the video stream, accurately predict the positions of some target objects (such as front vehicles or other road facilities) in the next sampling time in the shooting picture based on the current frame and historical frames, and timely understand the position relationship of the target objects relative to the vehicle, so that the vehicle can avoid danger in advance when encountering danger.

[0030] In order to accurately predict the corresponding position of the target object in the next frame image, the embodiment of the present application provides a long-short dual-branch based flow perception target detection scheme. Specifically, when a current frame image is sampled, a first number of historical frame images are obtained with a first sampling interval multiple to obtain a first image set, and a second number of historical frame images are obtained with a second sampling interval multiple to obtain a second image set. Wherein, the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the current frame image is contained in the first image set and the second image set. As can be seen, the first image set has a shorter time receptive field, and the time length between adjacent two frame images in the first image set is shorter, while the second image set has a longer time receptive field, and the time length between adjacent two frame images in the second image set is longer. By extracting the respective corresponding feature maps of the multiple frame images in the first image set, short-term spatial features can be obtained. By extracting the respective corresponding feature maps of the multiple frame images in the second image set, long-term time sequence features can be obtained, so as to understand the accurate changes of the moving situation of the target object in a longer period of time. Then, the respective corresponding feature maps of the multiple frame images in the first image set and the respective corresponding feature maps of the multiple frame images in the second image set are combined to predict the position information of the target object in the next frame image, and a more accurate prediction result can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A flow chart of a target detection method provided by the embodiment of the present application;

[0033] Figure 2 An application schematic diagram of a target detection method provided by the embodiment of the present application;

[0034] Figure 3 A flow chart of a target detection model training method provided by the embodiment of the present application;

[0035] Figure 4 A schematic diagram of a speed change situation provided by the embodiment of the present application;

[0036] Figure 5 A schematic diagram of a speed change situation provided by the embodiment of the present application;

[0037] Figure 6 An application schematic diagram of a target detection method provided by the embodiment of the present application;

[0038] Figure 7 This is a schematic diagram of the structure of a target detection device provided in an embodiment of the present invention;

[0039] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0042] A current stream-aware target detection method uses video frames acquired at the current and previous sampling times as input to predict the target location at the next sampling time. This method has the following problems:

[0043] Using two frames of images from the current sampling time and the previous sampling time as input cannot fully characterize the motion features of a non-uniformly moving target. For example, if a car is accelerating / decelerating in front, the information from the two frames of images cannot determine whether it is accelerating / decelerating, let alone the car's acceleration.

[0044] Using two frames of images from the current sampling time and the previous sampling time as input cannot fully represent the motion characteristics of non-linear moving targets. For example, for a car that is turning, the information from the two frames of images cannot determine the details of its turn. More information is needed to represent the state of non-linear motion.

[0045] Based on this, this invention proposes a flow-aware target detection scheme based on a long-short dual-branch network model. Specifically, it utilizes long and short branches to transform the temporal receptive field in the temporal dimension, forming a dual-branch network. By using long branches with longer receptive fields and short branches with shorter receptive fields, temporal and spatial features of the video are extracted, ultimately improving the network model's prediction accuracy for target object positions under complex motion conditions such as non-uniform and non-linear motion.

[0046] Figure 1 A flowchart of a target detection method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes the following steps:

[0047] 101、In the real-time acquisition of video stream, the current frame image corresponding to the current sampling time is obtained, and the target object is included in the current frame image.

[0048] 102、A first number of historical frame images are obtained with a first sampling interval multiple to obtain a first image set, and a second number of historical frame images are obtained with a second sampling interval multiple to obtain a second image set, the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the current frame image is included in the first image set and the second image set.

[0049] 103、The feature maps corresponding to the multiple frames of images in the first image set are extracted, and the feature maps corresponding to the multiple frames of images in the second image set are extracted.

[0050] 104、According to the feature maps corresponding to the multiple frames of images in the first image set and the feature maps corresponding to the multiple frames of images in the second image set, the position information of the target object in the next frame image is predicted.

[0051] Optionally, in step 104, the feature maps corresponding to the multiple frames of images in the first image set and the feature maps corresponding to the multiple frames of images in the second image set can be fused, and then the position information of the target object in the next frame image is predicted according to the fused feature maps.

[0052] The target detection method provided by the embodiment of the application can be applied to the application scenario of real-time acquisition of video stream, and there is a moving subject in the application scenario. Or simply, there is an object whose position will move in the video stream. For example, in the automatic driving scene, a vehicle-mounted terminal device for collecting the road environment in front of a vehicle A can be arranged on the vehicle A, the vehicle A is a moving subject, other vehicles B driving in front of the vehicle A are also moving subjects, pedestrians in front are also moving subjects, and because the vehicle A is moving, facilities such as street lamps and road stakes fixedly arranged in front of the vehicle A are also special moving subjects relative to the position of the vehicle A.

[0053] Based on the above example, the task of target detection is to predict the corresponding position of the target object in the next frame image based on the current frame image and the historical frame image. The target object may be, for example, a vehicle B, a pedestrian, a road facility, and the like in the above example.

[0054] When the video stream is collected in real time, one frame of video image included in the video stream is sampled based on the set sampling interval, and the target detection process can be triggered once for each time a frame of image is sampled.

[0055] Specifically, two sampling interval multiples are preset: a first sampling interval multiple Δt1 and a second sampling interval multiple Δt2, for example, Δt1 = 1, indicating that it is set to one times of the sampling interval, and Δt2 = 2, indicating that it is set to two times of the sampling interval. Two parameters for limiting the number of historical frames are also set: a first number and a second number, for example, the first number is 1, and the second number is 5. In actual application, the first number can be set to a small value, such as 1 or 2, and the second number can be set to a relatively large value, such as 5 or 8.

[0056] The larger Δt1 and Δt2 are, the larger the interval between the adjacent two frames is, and the larger the first number and the second number are, indicating that more video frames can be perceived in the time dimension.

[0057] Taking the current sampling time as t, in the above example, after sampling the current frame image corresponding to time t, the first number (assuming 1) of historical frame images are obtained in the previous sampling: one frame image sampled at time t-Δt1, and the second number (assuming 5) of historical frame images are obtained in the previous sampling: images sampled at times t-Δt2, t-2Δt2, t-3Δt2, t-4Δt2, and t-5Δt2, respectively.

[0058] The current frame image sampled at the current sampling time t and the first number of historical frame images form a first image set, and the current frame image sampled at the current sampling time t and the second number of historical frame images form a second image set.

[0059] Then, based on the first image set and the second image set, the position of the target object in the image corresponding to the next sampling time t+1 (i.e., the t+1 sampling time) (i.e., the next frame image) is predicted by using a trained target detection model. Through the setting of the two sampling interval multiples and the two numbers, the target detection model can perceive the spatial features of the shooting environment in the short term, and the target detection model can also perceive the time sequence change features of the target objects in the environment in a long period of time.

[0060] It should be noted that the time length required for obtaining the first image set and the second image set at the current sampling time t to complete the prediction processing of the position of the target object in the next frame image is shorter than the set sampling interval, that is, the prediction of the position of the target object in the next frame image is completed before the next frame image is collected, so that the position information perceived in advance can be used for subsequent processing. For example, when it is determined that there is a risk of collision between the vehicle and a vehicle in front based on the prediction result of the position of the vehicle in the next frame image, timely warning or braking, avoidance, and other operation controls are performed.

[0061] For example, in an automatic driving scenario, after determining the position information of vehicle B in the next sampling time in the corresponding image based on the first image set and the second image set contained in the video collected by vehicle A, since the position information actually describes the vertex positions of the rectangular frame surrounding vehicle B, it can be known that the position area of vehicle B in the corresponding image in the next frame. If the distance between the two vehicles is far, the position area of vehicle B in the image will be small when imaging, and if the distance is close, the position area will be large. A warning area threshold corresponding to the vehicle can be set in advance, and if it is determined according to the prediction result at this time that the position area of vehicle B is less than or equal to the threshold, an avoidance operation corresponding to such a warning condition is triggered, such as braking, turning to avoid, etc.

[0062] In combination Figure 2 The composition and working principle of the target detection model used to complete the above prediction task are shown.

[0063] As Figure 2 shown in the above, the target detection model includes a first backbone network model corresponding to a short branch, a second backbone network model corresponding to a long branch, and a head network model.

[0064] Among them, the so-called short branch refers to a branch with a short time receptive field, and the receptive field of the short branch is determined based on the first sampling interval multiple and the first number; the so-called long branch refers to a branch with a long time receptive field, and the receptive field of the long branch is determined based on the second sampling interval multiple and the second number.

[0065] Among them, as known from the above examples, a short time receptive field refers to only needing to pay attention to the historical frames sampled in a short time before the current sampling time, such as one frame, two frames of images sampled before; a long time receptive field refers to needing to pay attention to the historical frames sampled in a long time before the current sampling time, such as 10 frames sampled before, and the 10 frames can not be 10 consecutive frames, but can be interval extracted.

[0066] As Figure 2 shown in the above, assuming that the first image set includes the tth frame image sampled at the current sampling time and the t-Δt1th frame image, inputting the two frames of images into the first backbone network model respectively to extract the respective feature maps corresponding to the two frames of images. Similarly, assuming that the second image set includes the tth frame image sampled at the current sampling time, the t-Δt2th frame image, the t-2Δt2th frame image, …, and the t-nΔt2th frame image, a total of n+1 frames of images, inputting the n+1 frames of images into the second backbone network model respectively to extract the respective feature maps corresponding to the n+1 frames of images.

[0067] After obtaining the feature maps corresponding to each frame of image in the first image set and the feature maps corresponding to each frame of image in the second image set, the feature maps are fused.

[0068] As shown in the middle of the figure, there are three places where feature fusion is needed, which are represented by plus signs in the figure, the first two of which are the feature fusion within the short branch and the long branch, and the last one is the feature fusion between the short branch and the long branch. The way of feature fusion is to add or splice the input feature maps. Figure 2

[0069] Among them, the addition processing does not change the dimension of the feature map, that is, the dimension of the fused feature map is the same as that of the fused feature map. The splicing processing will change the dimension of the feature map, that is, the dimension of the fused feature map is greater than that of the fused feature map.

[0070] For feature fusion processing using addition method: the feature maps corresponding to each frame of image in the first image set can be added first, and the feature maps corresponding to each frame of image in the second image set can be added, and then the fused feature maps corresponding to each image set are added again to obtain the final fused feature map.

[0071] For feature fusion processing using splicing method: the feature maps corresponding to each frame of image in the first image set can be spliced to obtain the spliced first feature map, and the feature maps corresponding to each frame of image in the second image set can be spliced to obtain the spliced second feature map, and then since the dimension of the second feature map is higher than that of the first feature map, the second feature map can be processed first. Dimension reduction, the dimension is reduced to the same as the first feature map, and then spliced with the first feature map to obtain the final fused feature map.

[0072] After that, the fused feature map is input into the head network model, and the position information of the target object in the next frame of image of the current frame is predicted by the head network model.

[0073] It should be noted that the above target detection model is trained to have the ability to identify multiple different categories of target objects, such as vehicles, people, streetlights, road posts, trees, and various objects, so the target objects contained in the above current frame image can include more than one type of object.

[0074] ​In practical applications, the first backbone network model and the second backbone network model can be parameter-shared, that is, both are the same network model with the same parameters, or can be two independent different network models. The two backbone network models can adopt model structures such as a residual network model ResNet-50, a Darknet53 model structure, and the like. The head network model can adopt model structures such as YOLO, SSD (Single Shot MultiBox Detector), Faster R-CNN (Faster Region-based Convolutional Neural Network), R-FCN (Region-based Convolutional Neural Network), and the like.

[0075] In summary, the embodiment of the present application proposes an influenza perception target detection scheme based on a long-short dual-branch network, wherein long and short are relative to the time receptive field, the long branch has a longer time receptive field and is mainly used to capture time sequence motion information, while the short branch has a relatively short time receptive field and focuses more on capturing spatial structure information, and finally the two are fused, which finally improves the detection accuracy of target objects showing different motion change characteristics and can better overcome the interference of non-uniform motion and non-linear motion of target objects.

[0076] In the above, the first sampling interval multiple and the second sampling interval multiple can be set in advance. In practical applications, they can be set artificially according to experience, or alternatively, they can be determined according to the generation environment information corresponding to the video stream.

[0077] The generation environment information can be information indicating the type of the location, the video collection time period, and the like, such as a highway, an urban road, a morning and evening rush hour period, and a holiday period.

[0078] For example, assuming that the current time is a rush hour period, and the positioning information of the vehicle terminal device on the vehicle determines that the vehicle is on a road segment in the city where congestion is prone to occur, a larger first sampling interval multiple and a smaller second sampling interval multiple can be set at this time, or even when it is determined that the current situation is a serious congestion situation in combination with the available road congestion information, the second sampling interval multiple and the second quantity can be set to 0, that is, the long branch can not be used at this time, and only the short branch can be used.

[0079] Assuming that the current time is a weekday, and the positioning information of the vehicle terminal device on the vehicle determines that the vehicle is on a highway, a smaller first sampling interval multiple and a larger second sampling interval multiple can be set at this time.

[0080] In another optional embodiment, the moving speed of the target object can also be determined according to the obtained multiple position information prediction results of the target object, and the first sampling interval multiple and the second sampling interval multiple are adjusted according to the moving speed of the target object.

[0081] In this embodiment, a first sampling interval multiple and a second sampling interval multiple can be pre-set by default, and then the values of the first sampling interval multiple and the second sampling interval multiple are dynamically adjusted according to the prediction results of the prediction process triggered at different video frame sampling moments.

[0082] For ease of description, it is assumed that m position information corresponding to the same target object is obtained based on m prediction processes performed at m sampling moments T1-Tm, and the moving speed of the target object in this period of time can be known based on the m position information. If the moving speed of the target object is fast (for example, greater than a set threshold), the first sampling interval multiple can be lowered and the second sampling interval multiple can be raised. If the moving speed of the target object is slow (for example, less than a set threshold), the first sampling interval multiple can be raised and the second sampling interval multiple can be lowered. However, it should be noted that the adjustment of the first sampling interval multiple has upper and lower limits and cannot be too large.

[0083] The training process of the target detection model is described below. It is assumed below that the target detection model to be trained is composed of a backbone network model A, a backbone network model B and a head network model.

[0084] The training process can include the following steps:

[0085] Multiple sample images sampled from a training sample video are obtained, wherein the class and position information of a target object are marked in each sample image;

[0086] For the first sample image, a first number of historical sample images are obtained according to the first sampling interval multiple to obtain a third image set, and a second number of historical sample images are obtained according to the second sampling interval multiple to obtain a fourth image set; wherein the first sample image is included in both the third image set and the fourth image set;

[0087] The multiple sample images in the third image set are respectively input into the backbone network model A to obtain the feature maps corresponding to the multiple sample images in the third image set, and the multiple sample images in the fourth image set are respectively input into the backbone network model B to obtain the feature maps corresponding to the multiple sample images in the fourth image set;

[0088] The feature maps corresponding to the multiple sample images in the third image set and the feature maps corresponding to the multiple sample images in the fourth image set are fused;

[0089] input the fused feature map into the head network model to obtain position information of the target object in the second sample image predicted by the head network model, wherein the second sample image is a sample image sampled at a next sampling time of the first sample image;

[0090] According to the position information of the target object marked in the second sample image and the position information of the target object in the second sample image predicted by the head network model, a loss function value of the target detection model is determined to train the target detection model according to the loss function value.

[0091] The above is a conventional training process. In the embodiment of the present application, a model training method as shown in Figure 3 is further provided to further expand the training set and improve the performance of the target detection model.

[0092] Figure 3 A flowchart of a target detection model training method provided in the embodiment of the present application is shown in Figure 3 , and the training method comprises the following steps:

[0093] 301. Obtain a plurality of sample images sampled from a training sample video, wherein the class and position information of a target object are marked in each sample image.

[0094] 302. Set a first group of sampling interval multiples and a second group of sampling interval multiples for simulating a target speed change situation of the target object.

[0095] 303. For a first sample image in the plurality of sample images, a first sample group and a second sample group are constructed respectively with the first group of sampling interval multiples and the second group of sampling interval multiples, and the selected probability value range and the prediction reference image corresponding to each of the first sample group and the second sample group are determined; wherein each group of sampling interval multiples includes a third sampling interval multiple for obtaining a first number of historical sample images and a fourth sampling interval multiple for obtaining a second number of historical sample images.

[0096] 304. Generate a random number, and according to the ownership relationship between the random number and the selected probability value range corresponding to each of the first sample group and the second sample group, determine a target sample group in the first sample group and the second sample group, and use the target sample group to train a target detection model corresponding to the target speed change situation.

[0097] In the training method provided by the embodiment, different speed change situations or different speed objects are simulated by randomly changing the sampling interval multiple corresponding to the short branch (the third sampling interval multiple can be denoted as Δt1) and the sampling interval multiple corresponding to the long branch (the fourth sampling interval multiple can be denoted as Δt2), which can expand the data set, so that the target detection model can learn different speed change characteristics to improve the model performance.

[0098] The training sample video does not need to be collected in real time, but can be a pre-obtained video. The training sample video is any one of a plurality of training sample videos. The training sample video is sampled at a set sampling interval to obtain a plurality of sample images, and the target objects of a set category are labeled in each sample image to mark the corresponding positions and categories of the target objects in the sample images. For example, different colors are used to represent different categories, and rectangular boxes are used to mark positions. In the embodiment of the application, it is assumed that the target objects in each training sample video actually move at a constant speed.

[0099] The first set of sampling interval multiples and the second set of sampling interval multiples correspond to two different speed situations, that is, the two sets of sampling interval multiples can reflect the change situation between the two different motions.

[0100] Each set of sampling interval multiples includes a third sampling interval multiple corresponding to the short branch and a fourth sampling interval multiple corresponding to the long branch.

[0101] For example, one motion situation is uniform motion, at this time, one third sampling interval multiple and one fourth sampling interval multiple are set to reflect this speed situation; another motion situation is deceleration motion, at this time, another third sampling interval multiple and another fourth sampling interval multiple can be set to reflect this speed situation. Thus, the speed change situation between uniform motion and deceleration motion is simulated.

[0102] When the model training strategy adopted is to simulate the target speed change situation of the above-mentioned uniform motion + deceleration motion, a sample group corresponding to each speed change situation needs to be constructed, which is referred to as a first sample group and a second sample group. For a certain frame of sample image (referred to as a first sample image) in the plurality of sample images, the first sample image is regarded as a "current frame", based on the setting of the two sets of sampling interval multiples, the first sample image can be used as a current frame to correspondingly generate two sample groups, the first sample group corresponds to the first set of sampling interval multiples, and the second sample group corresponds to the second set of sampling interval multiples. Each sample group includes a first sample image set for inputting the short branch and a second sample image set for inputting the long branch, and each sample group can also include a sample image as a prediction reference image.

[0103] With reference to the first image set and the second image set in the foregoing example, it can be understood that the first sample image set included in each sample group includes the first sample image and a first number of historical sample images obtained according to a third sampling interval multiple, and the second sample image set includes the first sample image and a second number of historical sample images obtained according to a fourth sampling interval multiple. Only because the third sampling interval multiple and the fourth sampling interval multiple corresponding to the two sample groups are different, the two sample image sets respectively contained in the two sample groups generated based on the same current frame (i.e., the first sample image) are different.

[0104] In addition, the value range of the selection probability corresponding to each sample group can be randomly set. Specifically, the above two sample groups generated based on the same first sample image as the current frame can be randomly assigned a selection probability value, respectively denoted as p1 and p2, to describe the probability at which the two sample groups are selected, respectively, and the sum of the two probability values is 1. Based on the two probability values, the value range of the selection probability corresponding to each sample group can be determined. For example, p1 = 0.9 and p2 = 0.1, which means that the first sample group is selected with a probability of 90%, and the second sample group is selected with a probability of 10%. The value range of the selection probability corresponding to each sample group can be set based on the ratio of the selection probabilities of the two sample groups in the range of 0 to 1. For example, the value range of the selection probability corresponding to the first sample group is [0-0.9], and the value range of the selection probability corresponding to the second sample group is [0.9-1], which matches the above ratio. For another example, the value range of the selection probability corresponding to the first sample group is [0-0.5] and [0.7-1], and the value range of the selection probability corresponding to the second sample group is [0.6-0.7]. This setting also conforms to the constraint of the above ratio, i.e., the selection probability of the first sample group is 9 times the selection probability of the second sample group.

[0105] After the above setting is completed, a random number in the range of 0 to 1 can be generated, and the target sample group selected at this time can be determined according to which value range of the selection probability corresponding to the sample group the random number belongs to. The target detection model corresponding to the target speed deformation situation is trained using the target sample group. That is, each frame image in the first sample image set contained in the target sample group is input into the backbone network model corresponding to the short branch to extract the feature map of each frame image, and each frame image in the second sample image set is input into the backbone network model corresponding to the long branch to extract the feature map of each frame image. Then, the feature maps are fused, and the fused feature map is input into the head network to obtain the prediction result: the position of the target object in the prediction reference image sampled after the first sample image.

[0106] It should be noted that in the embodiment, the above target speed change situation of simulating uniform motion + deceleration motion is taken as an example, each speed change situation is used to train a corresponding target detection model, that is, when multiple speed change situations are set, different multiple target detection models are trained.

[0107] In an optional embodiment, setting the first set of sampling interval multiples and the second set of sampling interval multiples for simulating the target speed change situation of the target object includes: setting the third sampling interval multiple contained in the first set of sampling interval multiples to be less than the third sampling interval multiple contained in the second set of sampling interval multiples, and setting the fourth sampling interval multiple contained in the first set of sampling interval multiples to be less than the fourth sampling interval multiple contained in the second set of sampling interval multiples. Determining the respective predicted reference images corresponding to the first sample set and the second sample set includes: determining the respective predicted reference images corresponding to the first sample set and the second sample set according to the target speed change situation.

[0108] Specifically, the training processes when simulating two different speed change situations are illustrated below. Figure 4 and Figure 5 respectively.

[0109] The first one: the target speed change situation is the change between uniform motion and deceleration motion.

[0110] At this time, the setting of the two sets of sampling interval multiples is as follows: the third sampling interval multiple contained in the first set of sampling interval multiples is set to be less than the third sampling interval multiple contained in the second set of sampling interval multiples, and the fourth sampling interval multiple contained in the first set of sampling interval multiples is set to be less than the fourth sampling interval multiple contained in the second set of sampling interval multiples. For example, the third sampling interval multiple (Δt1) in the first set of sampling interval multiples is 1, and the fourth sampling interval multiple (Δt2) in the first set of sampling interval multiples is 2; the third sampling interval multiple (Δt1) in the second set of sampling interval multiples is 2, and the fourth sampling interval multiple (Δt2) in the second set of sampling interval multiples is 4.

[0111] Moreover, the second sample image, which is the next frame sample image sampled after the first sample image, is determined to be contained in both the first sample set and the second sample set as a predicted reference image.

[0112] For ease of understanding, only the third sampling interval multiple contained in each of the two sets of sampling interval multiples is taken as an example for example-type illustration. Figure 4

[0113] In Figure 4 ​In the embodiment, it is assumed that the third sampling interval multiple (Δt1) in the first group of sampling interval multiples is 1, the third sampling interval multiple (Δt1) in the second group of sampling interval multiples is 2, it is assumed that the first sample image is the t-th frame sample image, and the first quantity is 1. Based on the above assumptions, as shown in the following table: Figure 4

[0114] The first sample image set corresponding to the short branch in the first sample group includes the t-th frame sample image (i.e., the first sample image) and the (t-1)-th frame sample image. The two frame sample images are combined to predict the corresponding position of the target object in the (t+1)-th frame sample image (i.e., the second sample image). The interval t-1, t, t+1 is set to simulate the case of uniform motion.

[0115] The first sample image set corresponding to the short branch in the second sample group includes the t-th frame sample image (i.e., the first sample image) and the (t-2)-th frame sample image. The two frame sample images are combined to predict the corresponding position of the target object in the (t+1)-th frame sample image (i.e., the second sample image). The interval t-2, t, t+1 is set to simulate the case of deceleration. Because it is assumed that the target detection model is used to predict the position of the target object in the next frame based on the current frame and the previous frame, at this time, the (t-2)-th frame is regarded as the last frame, and the next frame is still the (t+1)-th frame. Assuming that the target object is moving at a constant speed, assuming that the speed change amount of the (t-1)-th frame and the (t)-th frame is v1, and the speed change amount of the (t+1)-th frame and the (t)-th frame is v2, v1=v2. In the target detection model, the (t-2)-th frame and the (t)-th frame are regarded as two adjacent frames, compared with the two adjacent frames of the (t-1)-th frame and the (t)-th frame, the time span is larger, which means that the speed change value is larger, assuming that the speed change amount of the (t-2)-th frame and the (t)-th frame is v3, then v3>v1, then the change from v3 to v2 is deceleration.

[0116] The above two sample groups are referred to as selection probabilities p1, p2. Based on the probabilities, the selection method introduced above can be used to select a sample group in a certain training process to train the corresponding target detection model (for distinction, the target detection model corresponding to the first case is referred to as the first target detection model).

[0117] It can be understood that in the above first speed change case, for each frame sample image in the multiple frame sample images sampled from a certain training sample video, two sample groups corresponding to the frame sample image can be generated. In each training process, two sample groups corresponding to a certain sample image can be selected, and then a target sample group used this time can be determined from the two sample groups.

[0118] Second: the target speed change case is the change between uniform motion and acceleration. ​

[0119] At this time, the third sampling interval multiple contained in the first group of sampling interval multiples is less than the third sampling interval multiple contained in the second group of sampling interval multiples, and the fourth sampling interval multiple contained in the first group of sampling interval multiples is less than the fourth sampling interval multiple contained in the second group of sampling interval multiples.

[0120] And, the second sample image contained in the first sample group as a prediction reference image is determined, and the second sample image is a next frame sample image sampled after the first sample image. The third sample image contained in the second sample group as a prediction reference image is determined, and the third sample image is a frame sample image sampled after the second sample image.

[0121] For ease of understanding, only the third sampling interval multiple contained in each of the two groups of sampling interval multiples is taken as an example, and an exemplary description is made in Figure 5 .

[0122] In Figure 5 , it is assumed that the third sampling interval multiple (Δt1) in the first group of sampling interval multiples is 1, and the third sampling interval multiple (Δt1) in the second group of sampling interval multiples is 2. It is assumed that the first sample image is the t-th frame sample image, and the first number is 1. Based on the above assumptions, as shown in Figure 5 :

[0123] The first sample image set corresponding to the short branch in the first sample group includes: the t-th frame sample image (i.e. the first sample image), the (t-1)-th frame sample image. The two frame sample images are combined to predict the corresponding position of the target object in the (t+1)-th frame sample image (i.e. the second sample image). The interval t-1, t, t+1 is set to simulate the case of uniform motion.

[0124] The first sample image set corresponding to the short branch in the second sample group includes: the t-th frame sample image (i.e. the first sample image), the (t-2)-th frame sample image. The two frame sample images are combined to predict the corresponding position of the target object in the (t+2)-th frame sample image (i.e. the third sample image). The interval t-2, t, t+2 is set to simulate the case of accelerated motion, and further is the accelerated uniform motion case compared with the first sample group.

[0125] For the above two sample groups, the selected probabilities p1, p2 are respectively, based on the probability, using the selection method introduced above, a certain sample group can be selected in a certain training process to train the corresponding target detection model (for distinction, the target detection model corresponding to the second case is called the second target detection model).

[0126] Based on the above examples, assume that the target detection models corresponding to different target velocity deformation scenarios include a first target detection model and a second target detection model. In this case, both target detection models can be used in subsequent prediction tasks.

[0127] Specifically, in Figure 1 In the illustrated embodiment, after obtaining the first image set and the second image set, feature maps corresponding to each of the multiple frames in the first image set can be extracted using a first object detection model. Similarly, feature maps corresponding to each of the multiple frames in the second image set can be extracted, and these feature maps are fused. Based on the fused feature maps, the first position information of the target object in the next frame image is predicted. Then, feature maps corresponding to each of the multiple frames in the first image set and the second image set are extracted using a second object detection model. These feature maps are fused, and the second position information of the target object in the next frame image is predicted. Finally, based on the first and second position information, the target position information of the target object in the next frame image is determined. Optionally, the target position information is the union of the first and second position information.

[0128] The target detection method provided in this invention can be executed in the cloud, where multiple computing nodes (cloud servers) can be deployed. Each computing node has processing resources such as computing and storage. In the cloud, multiple computing nodes can be organized to provide a certain service; of course, a single computing node can also provide one or more services. The cloud provides this service by providing an external service interface, which users call to use the corresponding service.

[0129] According to the solution provided in this embodiment of the invention, the cloud can provide a service interface for target detection. Users can call this service interface through in-vehicle terminal devices to trigger a target detection request to the cloud. This request includes a video stream of the road environment captured in real-time by the in-vehicle terminal device. The cloud determines the computing node that responds to the request and uses the processing resources in that computing node to perform the following steps:

[0130] The current frame image corresponding to the current sampling time is obtained from the video stream, and the current frame image contains the target object;

[0131] acquire a first number of historical frame images at a first sampling interval multiple to obtain a first image set, and acquire a second number of historical frame images at a second sampling interval multiple to obtain a second image set; wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both contain the current frame image;

[0132] extract the respective feature maps of the multiple frame images in the first image set and the respective feature maps of the multiple frame images in the second image set;

[0133] fuse the respective feature maps of the multiple frame images in the first image set and the respective feature maps of the multiple frame images in the second image set;

[0134] predict the position information of the target object in the next frame image according to the fused feature maps.

[0135] The cloud can send the position information to the vehicle terminal device, and the vehicle terminal device can mark the position information in the next frame image when the next frame image is collected.

[0136] The above execution process can refer to the related descriptions in the foregoing other embodiments, and will not be described here.

[0137] For ease of understanding, exemplary descriptions are given in conjunction with Figure 6 . The user can call the target detection service through the vehicle terminal device E1 as shown in Figure 6 to upload the collected video stream. The service interface called by the user to call the service includes a software development kit (Software Development Kit, SDK), an application programming interface (Application Programming Interface, API), and the like. Figure 6 The API interface is shown in . In the cloud, as shown in the figure, it is assumed that the target detection service is provided by a service cluster E2, and the service cluster E2 includes at least one computing node. After receiving the request, the service cluster E2 performs the steps described in the foregoing embodiments to obtain the position information of the target object in the next frame image.

[0138] The target detection device of one or more embodiments of the present application will be described in detail below. Those skilled in the art can understand that these devices can be configured by using commercially available hardware components through the steps taught by the present solution.

[0139] Figure 7 A structure diagram of a target detection device provided by an embodiment of the present application is shown in Figure 7 , and the device includes a sampling module 11, an acquisition module 12, and a detection module 13.

[0140] a sampling module 11, configured to acquire a current frame image corresponding to a current sampling moment in a real-time video stream, the current frame image containing a target object.

[0141] an acquisition module 12, configured to acquire a first number of historical frame images to obtain a first image set at a first sampling interval multiple, and acquire a second number of historical frame images to obtain a second image set at a second sampling interval multiple, wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both contain the current frame image.

[0142] a detection module 13, configured to extract a plurality of feature maps respectively corresponding to a plurality of frame images in the first image set, and extract a plurality of feature maps respectively corresponding to a plurality of frame images in the second image set, and predict position information of the target object in a next frame image according to the plurality of feature maps respectively corresponding to the plurality of frame images in the first image set and the plurality of feature maps respectively corresponding to the plurality of frame images in the second image set.

[0143] Optionally, the detection module 13 is specifically configured to: fuse the plurality of feature maps respectively corresponding to the plurality of frame images in the first image set and the plurality of feature maps respectively corresponding to the plurality of frame images in the second image set, and predict the position information of the target object in the next frame image according to the fused feature maps.

[0144] Optionally, the apparatus further includes a control module configured to trigger a risk-avoiding operation corresponding to a set alarm condition if it is determined that the position information of the target object in the next frame image satisfies the set alarm condition.

[0145] Optionally, the apparatus further includes a determination module configured to determine the first sampling interval multiple and the second sampling interval multiple according to generation environment information corresponding to the video stream.

[0146] Optionally, the determination module is further configured to: determine a moving speed of the target object according to a plurality of position information prediction results of the target object that have been obtained, and adjust the first sampling interval multiple and the second sampling interval multiple according to the moving speed of the target object.

[0147] Optionally, the apparatus further comprises a training module configured to: obtain a plurality of sample images sampled from a training sample video, wherein each sample image is labeled with class and location information of a target object; set a first set of sampling interval multiples and a second set of sampling interval multiples for simulating a target speed change scenario of the target object; for a first sample image in the plurality of sample images, construct a first sample set and a second sample set using the first set of sampling interval multiples and the second set of sampling interval multiples respectively, and determine a selected probability value range and a prediction reference image corresponding to each of the first sample set and the second sample set; wherein each set of sampling interval multiples comprises a third sampling interval multiple for obtaining a first number of historical sample images and a fourth sampling interval multiple for obtaining a second number of historical sample images, the first set of sampling interval multiples and the second set of sampling interval multiples each contain different values of the third sampling interval multiple and the fourth sampling interval multiple; generate a random number, and determine a target sample set from the first sample set and the second sample set according to a belonging relationship between the random number and the selected probability value range corresponding to each of the first sample set and the second sample set; and train a target detection model corresponding to the target speed change scenario using the target sample set and the prediction reference image corresponding thereto. The target detection model is a neural network model.

[0148] Optionally, the training module is configured to: set the third sampling interval multiple included in the first set of sampling interval multiples to be smaller than the third sampling interval multiple included in the second set of sampling interval multiples, and set the fourth sampling interval multiple included in the first set of sampling interval multiples to be smaller than the fourth sampling interval multiple included in the second set of sampling interval multiples; and determine the prediction reference image corresponding to each of the first sample set and the second sample set according to the target speed change scenario.

[0149] Optionally, the target speed change scenario is a change between uniform motion and deceleration motion; and the training module is specifically configured to: determine that the second sample image, which is a next sample image sampled after the first sample image, is included in each of the first sample set and the second sample set as a prediction reference image; and train a first target detection model using the target sample set and the second sample image.

[0150] Optionally, the target speed change case is a change between uniform motion and accelerated motion; the training module is specifically configured to: determine that the second sample image in the first sample group is a prediction reference image, the second sample image being a next frame sample image sampled after the first sample image; and determine that the third sample image in the second sample group is a prediction reference image, the third sample image being a frame sample image sampled after the second sample image; and train the second target detection model using the target sample group and the prediction reference image corresponding to the target sample group.

[0151] Optionally, the target detection model corresponding to different target speed change cases includes a first target detection model and a second target detection model. Based on this, the detection module 13 is specifically configured to: after obtaining first position information of the target object in a next frame image by the first target detection model and obtaining second position information of the target object in the next frame image by the second target detection model, determine target position information of the target object in the next frame image according to the first position information and the second position information.

[0152] Figure 7 The apparatus can perform the steps in the foregoing embodiments, and the detailed execution process and technical effects can be referred to the descriptions in the foregoing embodiments, which will not be repeated here.

[0153] In one possible design, the above Figure 7 The target detection apparatus can be implemented as an electronic device. For example, Figure 8 The electronic device can include a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable code. When the executable code is executed by the processor 21, the processor 21 can implement at least the target detection method provided in the foregoing embodiments.

[0154] In an optional embodiment, the electronic device can be a vehicle-mounted terminal device.

[0155] In addition, an embodiment of the present disclosure provides a non-transitory machine readable storage medium, which stores executable code. When the executable code is executed by a processor of an electronic device, the processor can implement at least the target detection method provided in the foregoing embodiments.

[0156] The apparatus embodiments described above are merely schematic, and the units described as separate components can or can not be physically separated. Part or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments. Those of ordinary skill in the art can understand and implement without creative labor.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of the necessary general hardware platforms, and of course can also be implemented by means of the combination of hardware and software. Based on such understanding, the above technical solutions can be embodied in the form of computer products, and the present application can be in the form of computer program products implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0158] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A target detection method characterized by, The method comprises: obtaining a current frame image corresponding to a current sampling time in a real-time collected video stream, the current frame image containing a target object; obtaining a first number of historical frame images with a first sampling interval multiple to obtain a first image set, the first image set including the first number of historical frame images and the current frame image; and obtaining a second number of historical frame images with a second sampling interval multiple to obtain a second image set, the second image set including the second number of historical frame images and the current frame image; wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both contain the current frame image; extracting respective feature maps of multiple frame images in the first image set and respective feature maps of multiple frame images in the second image set; predicting position information of the target object in a next frame image according to the respective feature maps of the multiple frame images in the first image set and the respective feature maps of the multiple frame images in the second image set.

2. The method of claim 1, wherein, The predicting position information of the target object in the next frame image according to the respective feature maps of the multiple frame images in the first image set and the respective feature maps of the multiple frame images in the second image set comprises: fusing the respective feature maps of the multiple frame images in the first image set and the respective feature maps of the multiple frame images in the second image set; predicting the position information of the target object in the next frame image according to the fused feature maps.

3. The method of claim 1, wherein, The method further comprises: if the position information of the target object in the next frame image meets a set alarm condition, triggering a risk-avoiding operation corresponding to the alarm condition.

4. The method of claim 1, wherein, The method further comprises: determining the first sampling interval multiple and the second sampling interval multiple according to environment information corresponding to the video stream.

5. The method of claim 1, wherein, The method further comprises: determining a moving speed of the target object according to a plurality of position information prediction results of the target object that have been obtained; adjusting the first sampling interval multiple and the second sampling interval multiple according to the moving speed of the target object.

6. The method of claim 1, wherein, A training process of a target detection model for predicting position information of a target object in a next frame image comprises: obtaining a plurality of sample frame images sampled from a training sample video, wherein each sample frame image is marked with a class and position information of a target object; setting a first group of sampling interval multiples and a second group of sampling interval multiples for simulating target speed change scenarios of the target object; For a first sample image in the plurality of sample images, a first sample group and a second sample group are constructed respectively with the first set of sampling interval multiples and the second set of sampling interval multiples, and a selected probability value range and a prediction reference image corresponding to each of the first sample group and the second sample group are determined; wherein each set of sampling interval multiples includes a third sampling interval multiple for obtaining a first number of historical sample images and a fourth sampling interval multiple for obtaining a second number of historical sample images, and the first set of sampling interval multiples and the second set of sampling interval multiples each contain different values of the third sampling interval multiple and the fourth sampling interval multiple; A random number is generated, and a target sample group is determined from the first sample group and the second sample group according to a belonging relationship between the random number and the selected probability value range corresponding to each of the first sample group and the second sample group; The target detection model corresponding to the target speed deformation condition is trained using the target sample group and the prediction reference image corresponding thereto; the target detection model is a neural network model.

7. The method of claim 6, wherein, The first set of sampling interval multiples and the second set of sampling interval multiples for simulating the target speed change condition of the target object are set, including: The third sampling interval multiple included in the first set of sampling interval multiples is set to be smaller than the third sampling interval multiple included in the second set of sampling interval multiples, and the fourth sampling interval multiple included in the first set of sampling interval multiples is set to be smaller than the fourth sampling interval multiple included in the second set of sampling interval multiples; Determining the prediction reference image corresponding to each of the first sample group and the second sample group includes: Determining the prediction reference image corresponding to each of the first sample group and the second sample group according to the target speed change condition.

8. The method of claim 7, wherein, The target speed change condition is a change between uniform motion and deceleration motion; The determining the prediction reference image corresponding to each of the first sample group and the second sample group according to the target speed change condition includes: determining that the second sample image, which is a prediction reference image, is included in each of the first sample group and the second sample group, the second sample image being a next frame sample image sampled after the first sample image; The training of the target detection model corresponding to the target speed deformation condition using the target sample group and the prediction reference image corresponding thereto includes: Training a first target detection model using the target sample group and the second sample image.

9. The method of claim 7, wherein, The target speed change condition is a change between uniform motion and acceleration motion; The determining the respective prediction reference images of the first sample group and the second sample group according to the target speed change case comprises: determining that a second sample image in the first sample group is a prediction reference image, the second sample image being a next frame sample image sampled after the first sample image; and determining that a third sample image in the second sample group is a prediction reference image, the third sample image being a frame sample image sampled after the second sample image. The training of the target detection model corresponding to the target speed change case using the target sample group and the prediction reference images corresponding to the target sample group comprises: training a second target detection model using the target sample group and the prediction reference images corresponding to the target sample group.

10. The method of claim 6, wherein, The target detection models corresponding to different target speed change cases comprise a first target detection model and a second target detection model. After obtaining first position information of the target object in a next frame image by the first target detection model and obtaining second position information of the target object in the next frame image by the second target detection model, the method further comprises: determining target position information of the target object in the next frame image according to the first position information and the second position information.

11. A target detection method characterized by, comprises: receiving a request triggered by a vehicle terminal device by calling a target detection service, the request comprising a video stream collected by the vehicle terminal device in real time from a front road environment; using processing resources corresponding to the target detection service to perform the following steps: obtaining a current frame image corresponding to a current sampling time in the video stream, the current frame image comprising a target object; obtaining a first number of historical frame images at a first sampling interval multiple to obtain a first image set, the first image set comprising the first number of historical frame images and the current frame image; and obtaining a second number of historical frame images at a second sampling interval multiple to obtain a second image set, the second image set comprising the second number of historical frame images and the current frame image; wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both comprise the current frame image; extracting respective feature maps of multiple frame images in the first image set and extracting respective feature maps of multiple frame images in the second image set; predicting position information of the target object in a next frame image according to the respective feature maps of the multiple frame images in the first image set and the respective feature maps of the multiple frame images in the second image set.

12. A target detection method characterized by, The method applied to a vehicle terminal device comprises: obtaining a current frame image corresponding to a current sampling time in a video stream collected in real time from a front road environment, the current frame image comprising a target object; acquire a first number of historical frame images to obtain a first image set at a first sampling interval multiple, the first image set including the first number of historical frame images and the current frame image; and acquire a second number of historical frame images to obtain a second image set at a second sampling interval multiple, the second image set including the second number of historical frame images and the current frame image; wherein the first sampling interval multiple is less than the second sampling interval multiple, the first number is less than the second number, and the first image set and the second image set both include the current frame image; extract respective feature maps of a plurality of frame images in the first image set, and extract respective feature maps of a plurality of frame images in the second image set; predict position information of the target object in a next frame image according to the respective feature maps of the plurality of frame images in the first image set and the respective feature maps of the plurality of frame images in the second image set; perform automatic driving control processing according to the position information.

13. An electronic device, comprising: comprise: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the target detection method according to any one of claims 1 to 10.

14. A non-transitory machine-readable storage medium, comprising: the non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by the processor of the cloud server, the processor performs the target detection method according to any one of claims 1 to 10.

Images