Target object detection method, electronic device, vehicle and medium

Abstract

The embodiment of the invention provides a target object detection method, an electronic device, a vehicle and a medium, and the method comprises the steps: obtaining a target feature parameter of a target image frame based on the target image frame in a video stream and an adjacent image frame of the target image frame, and carrying out the detection of a target object based on the target feature parameter of the target image frame, and obtaining a detection result of the target object, namely, combining the feature parameters of the current image frame and the feature parameters of the adjacent image frames before and after the current image frame to form feature data including the spatio-temporal information, and finally detecting the feature data including the spatio-temporal information to obtain the detection result including the target object. Therefore, the space-time information of the image frames is considered, the motion change between the adjacent image frames is reflected, and the accuracy of detecting the moving target object is improved.

Description

Technical Field

[0001] The present application belongs to the field of intelligent detection technology, and specifically relates to a method, electronic device, vehicle and medium for detecting a target object. Background Art

[0002] In the field of intelligent detection technology, the detection of target objects is an important computer vision task. For example, in autonomous driving tasks, dangerous behaviors of pedestrians on the road can be detected in a timely manner.

[0003] In the prior art, a method based on traditional Histogram of Oriented Gradients (HOG) feature extraction and Support Vector Machine (SVM) classification is adopted. The gradient information of each pixel is calculated and a feature vector describing the local gradient direction of the image is generated. Then, the SVM classifier is used to train and optimize the feature vectors in the training set to obtain the best classification performance.

[0004] However, using the existing methods, the accuracy of detecting the target object is not high. Summary of the invention

[0005] Embodiments of the present application provide a method for detecting a target object, an electronic device, a vehicle, and a medium to solve the problem of low accuracy in detecting a target object in the prior art.

[0006] A first aspect of an embodiment of the present application provides a method for detecting a target object, the method comprising:

[0007] Based on a target image frame in a video stream and image frames adjacent to the target image frame, obtaining target feature parameters of the target image frame;

[0008] Based on the target feature parameters of the target image frame, a detection result of the target object is obtained.

[0009] Optionally, the method of obtaining target feature parameters of the target image frame based on the target image frame in the video stream and the adjacent image frames of the target image frame includes: acquiring initial feature parameters respectively corresponding to the target image frame and the adjacent image frames of the target image frame; and superimposing the initial feature parameters respectively corresponding to the target image frame and the adjacent image frames of the target image frame to obtain the target feature parameters of the target image frame.

[0010] Optionally, the overlay processing includes any one of the following processing: splicing processing; weighted overlay processing; convolution processing.

[0011] Optionally, the step of obtaining initial feature parameters respectively corresponding to the target image frame and the adjacent image frames of the target image frame includes: obtaining initial histogram of oriented gradients (HOG) features respectively corresponding to the target image frame and the adjacent image frames of the target image frame; and the step of superimposing the initial feature parameters respectively corresponding to the target image frame and the adjacent image frames of the target image frame to obtain target feature parameters of the target image frame includes: superimposing the HOG features respectively corresponding to the target image frame and the adjacent image frames of the target image frame to obtain target HOG features of the target image frame.

[0012] Optionally, the step of obtaining the initial oriented gradient histogram HOG features corresponding to the target image frame and the adjacent image frames of the target image frame includes: for the target image frame and any one of the adjacent image frames of the target image frame, obtaining the gradient direction and amplitude of each pixel point of the image frame; and obtaining the initial HOG features corresponding to the image frame based on the gradient direction and amplitude of each pixel point of the image frame.

[0013] Optionally, obtaining the initial HOG feature corresponding to the image frame based on the gradient direction and amplitude of each pixel point of the image frame includes: dividing the image frame into multiple sub-units; for each sub-unit, obtaining the gradient direction histogram of the sub-unit based on the gradient direction and amplitude of each pixel point in the sub-unit; and obtaining the initial HOG feature corresponding to the image frame based on the gradient direction histograms corresponding to the multiple sub-units.

[0014] Optionally, the gradient direction histogram of the subunit is obtained based on the gradient direction and amplitude of each pixel point in the subunit, including: quantizing the gradient direction into multiple direction intervals, each direction interval corresponds to a histogram; based on the gradient direction and amplitude of each pixel point in the subunit, filling the histogram corresponding to the gradient direction to obtain the gradient direction histogram of the subunit.

[0015] Optionally, filling the histogram corresponding to the gradient direction based on the gradient direction and amplitude of each pixel point in the sub-unit includes: for each pixel point in the sub-unit, determining a target histogram based on the gradient direction of the pixel point, the gradient direction of the pixel point being within a direction interval corresponding to the target histogram; and filling the target histogram according to the amplitude of the pixel point.

[0016] Optionally, the obtaining of the initial HOG features corresponding to the image frame based on the gradient direction histograms respectively corresponding to the multiple sub-units includes: combining the multiple sub-units into a block; obtaining the gradient direction histogram of the block based on the gradient direction histograms respectively corresponding to the multiple sub-units; normalizing the gradient direction histogram of the block; and obtaining the initial HOG features corresponding to the image frame based on the normalized gradient direction histogram of the block.

[0017] Optionally, obtaining the detection result of the target object based on the target feature parameters of the target image frame includes: performing detection window scanning on the target image frame to obtain the target feature parameters corresponding to each detection window; inputting the target feature parameters corresponding to each detection window into a target classifier to obtain the detection result of the target object.

[0018] Optionally, performing detection window scanning on the target image frame to obtain target feature parameters corresponding to each detection window includes: performing detection window scanning on the target image frame at different scales to obtain target feature parameters corresponding to each detection window.

[0019] Optionally, the target classifier includes: a primary classifier and an advanced classifier; inputting the target feature parameters corresponding to each detection window into the target classifier to obtain the detection result of the target object includes: inputting the target feature parameters corresponding to each detection window into the primary classifier to obtain a candidate detection window; inputting the target feature parameters corresponding to the candidate detection window into the advanced classifier to obtain the detection result of the target object.

[0020] Optionally, the detection result of the target object includes: multiple target detection windows, each target detection window includes the target object, and the method further includes: sorting the target detection windows based on the confidence of the target detection windows; obtaining the target detection window with the highest confidence as a reference detection window; and screening the detection results of the target object based on the degree of overlap between the reference detection window and other target detection windows.

[0021] Optionally, the detection results of the target object are screened based on the degree of overlap between the reference detection window and other target detection windows, including: determining the overlapping detection window based on the intersection-and-union ratio of the reference detection window and other target detection windows; deleting the overlapping detection window from the detection results of the target object and retaining the remaining target detection windows.

[0022] Optionally, determining the overlapping detection window based on the intersection-and-union ratio of the reference detection window and other target detection windows includes: for each other target detection window, if the intersection-and-union ratio of the reference detection window and the other target detection window is greater than a preset threshold, determining that the other target detection window is an overlapping detection window.

[0023] Optionally, the method further includes: training a candidate classifier based on the sample data until the candidate classifier converges to obtain the target classifier.

[0024] Optionally, the candidate classifier includes: a candidate primary classifier and a candidate advanced classifier, and the candidate primary classifier and the candidate advanced classifier are cascaded; training the candidate classifier based on sample data until the candidate classifier converges to obtain the target classifier, including: training the candidate primary classifier and the candidate advanced classifier step by step based on the sample data until the candidate primary classifier and the candidate advanced classifier converge to obtain the target classifier.

[0025] Optionally, before obtaining the target feature parameters of the target image frame based on the target image frame in the video stream and the adjacent image frames of the target image frame, the method further includes: preprocessing the video stream to obtain a preprocessing result, wherein the preprocessing result includes: the target image frame and the adjacent image frames of the target image frame.

[0026] Optionally, preprocessing the video stream to obtain a preprocessing result includes: segmenting the video stream to obtain a plurality of continuous image frames.

[0027] Optionally, segmenting the video stream to obtain a plurality of continuous image frames includes: segmenting the video stream and assigning consecutive frame numbers to the plurality of image frames obtained by segmentation; and associating consecutive image frames based on the frame numbers.

[0028] Optionally, the method further includes: performing grayscale processing on the multiple image frames to obtain multiple image frames after grayscale processing.

[0029] Optionally, the method further includes: normalizing the pixel points of the multiple image frames after the grayscale processing to obtain multiple normalized image frames.

[0030] A second aspect of an embodiment of the present application provides a device for detecting a target object, the device comprising:

[0031] An acquisition module, used for obtaining target feature parameters of the target image frame based on the target image frame in the video stream and the adjacent image frames of the target image frame;

[0032] The detection module obtains a detection result of the target object based on the target feature parameters of the target image frame.

[0033] Optionally, the acquisition module is specifically used to obtain initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame; superimpose the initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame to obtain target feature parameters of the target image frame.

[0034] Optionally, the overlay processing includes any one of the following processing: splicing processing; weighted overlay processing; convolution processing.

[0035] Optionally, the acquisition module is specifically used to obtain the initial oriented gradient histogram HOG features corresponding to the target image frame and the adjacent image frames of the target image frame; the initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame are superimposed to obtain the target feature parameters of the target image frame, including: superimposing the HOG features corresponding to the target image frame and the adjacent image frames of the target image frame to obtain the target HOG features of the target image frame.

[0036] Optionally, the acquisition module is specifically used to obtain the gradient direction and amplitude of each pixel point of the target image frame and any image frame adjacent to the target image frame; based on the gradient direction and amplitude of each pixel point of the image frame, obtain the initial HOG feature corresponding to the image frame.

[0037] Optionally, the acquisition module is specifically used to divide the image frame into multiple sub-units; for each sub-unit, based on the gradient direction and amplitude of each pixel point in the sub-unit, obtain the gradient direction histogram of the sub-unit; based on the gradient direction histograms corresponding to the multiple sub-units, respectively, obtain the initial HOG features corresponding to the image frame.

[0038] Optionally, the acquisition module is specifically used to quantize the gradient direction into multiple direction intervals, each direction interval corresponds to a histogram; based on the gradient direction and amplitude of each pixel point in the sub-unit, fill the histogram corresponding to the gradient direction to obtain the gradient direction histogram of the sub-unit.

[0039] Optionally, the acquisition module is specifically used to determine a target histogram for each pixel in the subunit based on the gradient direction of the pixel, and the gradient direction of the pixel is within a direction interval corresponding to the target histogram; and fill the target histogram according to the amplitude of the pixel.

[0040] Optionally, the acquisition module is specifically used to combine multiple sub-units into a block; obtain the gradient direction histogram of the block based on the gradient direction histograms corresponding to the multiple sub-units respectively; normalize the gradient direction histogram of the block; and obtain the initial HOG feature corresponding to the image frame based on the normalized gradient direction histogram of the block.

[0041] Optionally, the detection module is specifically used to perform detection window scanning on the target image frame to obtain target feature parameters corresponding to each detection window; input the target feature parameters corresponding to each detection window into a target classifier to obtain a detection result of the target object.

[0042] Optionally, the detection module is specifically used to perform detection window scanning on the target image frame at different scales to obtain target feature parameters corresponding to each detection window.

[0043] Optionally, the target classifier includes: a primary classifier and an advanced classifier;

[0044] The detection module is specifically used to input the target feature parameters corresponding to each detection window into the primary classifier to obtain a candidate detection window; and input the target feature parameters corresponding to the candidate detection window into the high-level classifier to obtain the detection result of the target object.

[0045] Optionally, the detection result of the target object includes: multiple target detection windows, each target detection window includes the target object, and the detection module is also used to sort the target detection windows based on the confidence of the target detection windows; obtain the target detection window with the highest confidence as a reference detection window; and filter the detection results of the target object based on the degree of overlap between the reference detection window and other target detection windows.

[0046] Optionally, the detection module is specifically used to determine an overlapping detection window based on an intersection-and-union ratio between the reference detection window and other target detection windows; delete the overlapping detection window from the detection result of the target object, and retain the remaining target detection windows.

[0047] Optionally, the detection module is specifically configured to determine, for each other target detection window, if an intersection-and-union ratio between the reference detection window and the other target detection window is greater than a preset threshold, that the other target detection window is an overlapping detection window.

[0048] Optionally, the detection module is further used to train a candidate classifier based on sample data until the candidate classifier converges to obtain the target classifier.

[0049] Optionally, the candidate classifier includes: a candidate primary classifier and a candidate advanced classifier, and the candidate primary classifier and the candidate advanced classifier are cascaded;

[0050] The detection module is specifically used to train the candidate primary classifiers and the candidate advanced classifiers step by step based on the sample data until the candidate primary classifiers and the candidate advanced classifiers converge to obtain the target classifier.

[0051] Optionally, the acquisition module is further used to preprocess the video stream to obtain a preprocessing result, wherein the preprocessing result includes: the target image frame and image frames adjacent to the target image frame.

[0052] Optionally, the acquisition module is specifically used to segment the video stream to obtain a plurality of continuous image frames.

[0053] Optionally, the acquisition module is specifically used to segment the video stream, assign consecutive frame numbers to the multiple image frames obtained by segmentation, and associate consecutive image frames based on the frame numbers.

[0054] Optionally, the acquisition module is further used to perform grayscale processing on the multiple image frames to obtain multiple image frames after grayscale processing.

[0055] Optionally, the acquisition module is further used to normalize the pixels of the multiple image frames after the grayscale processing to obtain multiple normalized image frames.

[0056] A third aspect of an embodiment of the present application provides an electronic device, comprising: a processor, the processor is used to connect to a memory, the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the target object detection method described in the first aspect above are implemented.

[0057] A fourth aspect of an embodiment of the present application provides a vehicle, comprising the electronic device as described in the third aspect above.

[0058] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the target object detection method described in the first aspect are implemented.

[0059] A sixth aspect of an embodiment of the present application provides a computer program product, which, when executed by a processor of a vehicle or a cloud server, implements the steps of the target object detection method described in the first aspect above.

[0060] The target object detection method, electronic device, vehicle and medium provided by the embodiments of the present application obtain the target feature parameters of the target image frame based on the target image frame in the video stream and the adjacent image frames of the target image frame, and obtain the detection result of the target object based on the target feature parameters of the target image frame, that is, the feature parameters of the current image frame and the adjacent image frames before and after it are combined to form feature data including spatiotemporal information, and finally these feature data containing spatiotemporal information are detected to obtain the detection result containing the target object. Thus, the spatiotemporal information of the image frame is taken into account, the motion changes between adjacent image frames are reflected, and the accuracy of detecting the moving target object is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of a process for detecting a target object provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a flow chart of another method for detecting a target object provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0066] Figure 6 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0067] Figure 7 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0068] Figure 8 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0069] Fig. 9 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0070] Fig.10 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0071] Fig.11 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0072] Fig.12 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0073] Fig.13 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0074] Fig.14 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0075] Fig.15 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application;

[0076] Fig.16 A schematic diagram of the structure of a target object detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0078] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0079] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0080] In the prior art, the gradient information of each pixel is calculated through the traditional HOG, and a feature vector describing the local gradient direction of the image is generated, and then a single SVM classifier is used to train these feature vectors. However, in the above technology, the traditional HOG feature extraction only considers the spatial gradient information of a single frame image, and ignores the spatiotemporal information between adjacent frames, resulting in poor detection of moving targets, and thus low accuracy of pedestrian detection. In response to the above problems, on the basis of the traditional HOG features, the spatiotemporal information of adjacent frames is combined to form a spatiotemporal HOG feature vector, thereby improving the ability to detect moving targets. Furthermore, the spatiotemporal HOG feature vector is trained by a cascade SVM classifier, thereby improving the accuracy and speed of detecting target objects.

[0081] The following describes the technical solution of the target object detection method of the application by taking several specific embodiments as examples:

[0082] Figure 1 A flow chart of a method for detecting a target object provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the process of the method of this embodiment is as follows:

[0083] S11: Based on a target image frame in a video stream and image frames adjacent to the target image frame, obtain target feature parameters of the target image frame.

[0084] The video stream contains the motion information of the target object, and the video stream is divided into continuous image frames, and each image frame has adjacent image frames before and after. In order to capture the motion information of the target object in the video stream, the feature parameters of the target image frame and its adjacent image frames before and after are respectively obtained, and superimposed processing is performed, and the superimposed feature parameters obtained are the target feature parameters of the target image frame.

[0085] S12: Acquire a detection result of the target object based on the target feature parameters of the target image frame.

[0086] The target feature parameters of the target image frame are detected by using a classifier through sliding scanning of a multi-scale detection window, and the classification result finally obtained is the detection result of the target object.

[0087] In this embodiment, the target feature parameters of the target image frame are obtained based on the target image frame in the video stream and the adjacent image frames of the target image frame, and the detection result of the target object is obtained based on the target feature parameters of the target image frame, that is, the feature parameters of the current image frame and the adjacent image frames before and after it are combined to form feature data including spatiotemporal information, and finally these feature data including spatiotemporal information are detected to obtain the detection result including the target object. Thus, the spatiotemporal information of the image frame is taken into account, the motion changes between adjacent image frames are reflected, and the accuracy of detecting the moving target object is improved.

[0088] Figure 2 A schematic diagram of a flow chart of another target object detection method provided in an embodiment of the present application, such as Figure 2 As shown, Figure 2 is Figure 1 Based on the above, a possible implementation of S11 is further described. The process of the method in this embodiment is as follows:

[0089] S111: Obtain initial feature parameters corresponding to the target image frame and image frames adjacent to the target image frame.

[0090] Optionally, initial histogram of oriented gradients (HOG) features corresponding to the target image frame and image frames adjacent to the target image frame are obtained.

[0091] Wherein, the initial feature parameter is the initial directional gradient direction HOG feature corresponding to the image frame, and the process of obtaining the initial HOG features corresponding to the target image frame and the adjacent image frames of the target image frame is the same, including the spatial gradient calculation of the image, the construction of the gradient direction histogram and the block normalization. Wherein the spatial gradient calculation includes the calculation of the gradient direction and the amplitude, the construction of the gradient direction histogram includes the division subunit and the quantization gradient direction, and the block normalization includes the combination subunit.

[0092] S112: Superimposing initial feature parameters corresponding to the target image frame and image frames adjacent to the target image frame to obtain target feature parameters of the target image frame.

[0093] Optionally, the overlay processing includes any one of the following processing: splicing processing; weighted overlay processing; convolution processing.

[0094] Among them, superposition is not just a simple addition, but the temporal relationship between multiple frames is integrated into the feature description through a specific feature combination, so as to reflect the information in the time dimension. For example, the gradient features of different frames can be integrated into a high-dimensional spatiotemporal feature vector through splicing, weight combination, convolution, etc., and the temporal changes in motion, speed, direction, etc. are integrated into the spatiotemporal descriptor by comparing the HOG features of adjacent frames. Through the process, not only the spatial features can be seen, but also the motion changes in the time dimension can be captured. Possible implementation methods of superposition processing include but are not limited to the following methods:

[0095] One possible implementation method is splicing processing. For the current target image frame and its adjacent frames, the initial HOG feature vectors of each frame can be spliced ​​together to construct a high-dimensional spatiotemporal feature vector.

[0096] Another possible implementation method is weighted superposition processing. Different weights can be assigned to target image frames at different times, such as the current image frame has the highest weight, and the weights of the adjacent frames are lower, thereby emphasizing the information at the current moment while retaining some time continuity.

[0097] Another possible implementation is convolution processing. 3D convolution or other spatiotemporal convolution can be used to capture feature changes in the time dimension. This method can more effectively reflect the motion changes between frames.

[0098] Optionally, the HOG features corresponding to the target image frame and the image frames adjacent to the target image frame are superimposed to obtain the target HOG features of the target image frame.

[0099] Among them, the target feature parameter is the target HOG feature corresponding to the target image frame. The target HOG feature obtained by superimposing the initial HOG features of the target image frame and its adjacent frames, as a spatiotemporal feature descriptor, not only contains the spatial gradient information of each target image frame, but also reflects the motion changes between adjacent frames, and can comprehensively capture the spatial and temporal information of the target object.

[0100] In this embodiment, by obtaining the initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame, the target feature parameters of the target image frame and the adjacent image frames of the target image frame are superimposed to obtain the target feature parameters of the target image frame. Thus, the spatiotemporal HOG features of the target image frame are determined, providing richer feature input for subsequent detection.

[0101] Figure 3 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Figure 3 As shown, Figure 3 is Figure 2 Based on the above, a possible implementation of S111 is further described. The process of the method in this embodiment is as follows:

[0102] S1111: For the target image frame and any one of the image frames adjacent to the target image frame, obtain the gradient direction and amplitude of each pixel point of the image frame.

[0103] Among them, the gradient is a measure of the change in image intensity, which can reveal the edge and texture information of the image. Common gradient operators include Sobel operator, Prewitt operator and Roberts operator. These differential operators detect edge and texture features by calculating the rate of change of pixels in the image. Select a suitable differential operator to calculate the spatial gradient, and calculate the gradient direction and amplitude of each pixel point based on the spatial gradient.

[0104] For example, the spatial gradient is a vector. For a two-dimensional image I(x, y), its spatial gradient includes the gradient component G in the horizontal direction. x and the vertical gradient component G y , G x It reflects the rate of change of image intensity along the x direction and is expressed as G y It reflects the rate of change of image intensity along the y direction and is expressed as Then, the gradient direction and magnitude of each pixel are calculated by spatial gradient. Specifically, the gradient direction and magnitude are obtained by calculating the difference of each pixel in the horizontal direction (x direction) and the vertical direction (y direction). Among them, the gradient direction describes the direction of change of the pixel point, which specifically reflects the direction of change of the image intensity in space, and the gradient magnitude describes the intensity of change of the pixel point, which specifically reflects the size of change of the image intensity in space. The gradient direction can be expressed as:

[0105]

[0106] Among them, θ represents the gradient direction, G x represents the horizontal gradient, G y Represents the vertical gradient. This function is the inverse tangent function and can correctly handle G x When it is zero, the returned angle range is [-180°, 180°]. And taking modulo 180 can adjust the angle range to [0,180°].

[0107] The gradient magnitude can be expressed as:

[0108]

[0109] Among them, M represents the gradient amplitude, G x represents the horizontal gradient, G y Represents the vertical gradient.

[0110] S1112: Based on the gradient direction and amplitude of each pixel point of the image frame, obtain an initial HOG feature corresponding to the image frame.

[0111] Among them, based on the gradient direction and amplitude of each pixel point of the image frame, the gradient direction histogram of each pixel point of the image can be obtained, and based on the gradient direction histogram of each pixel point, the initial HOG feature corresponding to the image frame can be obtained.

[0112] In this embodiment, the gradient direction and amplitude of each pixel of the target image frame and any image frame adjacent to the target image frame are obtained, and based on the gradient direction and amplitude of each pixel of the image frame, the initial HOG feature corresponding to the image frame is obtained. Thus, the gradient direction and amplitude of each pixel of the image frame are accurately obtained, and then the HOG feature of the image frame is obtained.

[0113] Figure 4 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Figure 4 As shown, Figure 4 is Figure 3 Based on the above, a possible implementation of S1112 is further described. The process of the method in this embodiment is as follows:

[0114] S41: Divide the image frame into a plurality of sub-units.

[0115] The image frame is divided into a plurality of sub-units (cells). For example, assuming that the size of the image frame is 640×480 pixels and the size of each sub-unit is set to 8×8 pixels, the image frame can be divided into 4800 sub-units.

[0116] S42: For each sub-unit, based on the gradient direction and amplitude of each pixel point in the sub-unit, obtain a gradient direction histogram of the sub-unit.

[0117] For each sub-unit, the gradient direction of each pixel is quantized, and based on the gradient direction histogram filled in each gradient direction interval, the gradient direction histogram of the sub-unit can be obtained.

[0118] S43: Based on the gradient direction histograms corresponding to the multiple sub-units, obtain the initial HOG features corresponding to the image frame.

[0119] Among them, based on the multiple sub-units, they can be combined into blocks, and based on the gradient direction histograms corresponding to the multiple sub-units respectively, the gradient direction histogram of the block can be obtained; based on the normalization processing of the gradient direction histogram of the block, the initial HOG feature corresponding to the image frame can be obtained.

[0120] In this embodiment, the image frame is divided into multiple sub-units, and for each sub-unit, the gradient direction histogram of the sub-unit is obtained based on the gradient direction and amplitude of each pixel point in the sub-unit, and the initial HOG features corresponding to the image frame are obtained based on the gradient direction histograms corresponding to the multiple sub-units. Thus, the gradient direction histogram of the sub-unit is accurately obtained, and the local features in the image are better captured.

[0121] Figure 5 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Figure 5 As shown, Figure 5 is Figure 4 Based on the above, a possible implementation of S42 is further described. The process of the method in this embodiment is as follows:

[0122] S421: quantizing the gradient direction into multiple direction intervals, each direction interval corresponding to a histogram.

[0123] The gradient direction of each pixel is quantized into multiple direction intervals. For example, assuming that the gradient direction is quantized into 9 direction intervals, each direction interval in the 9 direction intervals is approximately 20 degrees, and each direction interval corresponds to a gradient direction histogram.

[0124] S422: Filling a histogram corresponding to the gradient direction based on the gradient direction and amplitude of each pixel point in the sub-unit to obtain a gradient direction histogram of the sub-unit.

[0125] Optionally, for each pixel in the subunit, a target histogram is determined based on the gradient direction of the pixel, and the gradient direction of the pixel is within a direction interval corresponding to the target histogram; and the target histogram is filled according to the amplitude of the pixel.

[0126] Among them, as in the above example, for each pixel point in the sub-unit, the corresponding histograms are filled based on the 9 gradient direction intervals of the pixel point, and the histogram of each gradient direction interval of each pixel point can be obtained; based on the gradient direction histogram of each pixel point, the gradient direction histogram of the sub-unit can be obtained.

[0127] In this embodiment, the gradient direction is quantized into multiple direction intervals, each direction interval corresponds to a histogram, and the histogram corresponding to the gradient direction is filled based on the gradient direction and amplitude of each pixel point in the sub-unit to obtain the gradient direction histogram of the sub-unit. Thus, the gradient direction distribution in the sub-unit is intuitively displayed, and the local features of each sub-unit are effectively described.

[0128] Figure 6 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Figure 6 As shown, Figure 6 is Figure 4 Based on the above, a possible implementation of S43 is further described. The process of the method in this embodiment is as follows:

[0129] S431: Combine multiple sub-units into a block.

[0130] Wherein, a plurality of sub-units are combined into a block. For example, as in the above example, the image frame is divided into 4800 sub-units. Assuming that a block contains 4×4 sub-units, the plurality of sub-units can be combined into 300 blocks.

[0131] S432: Obtaining a gradient direction histogram of the block based on the gradient direction histograms respectively corresponding to the plurality of sub-units.

[0132] In the above example, for the 300 blocks, based on the gradient direction histograms corresponding to the 16 sub-units contained in each block, the gradient direction histogram of each block can be obtained.

[0133] S433: normalizing the gradient direction histogram of the block.

[0134] In order to eliminate the influence of illumination changes and improve the robustness and consistency of image feature description, the gradient direction histogram of the block is normalized.

[0135] S434: Obtaining an initial HOG feature corresponding to the image frame based on the normalized gradient direction histogram of the block.

[0136] Among them, the normalized gradient direction histograms of all blocks are concatenated into a long vector to form the initial HOG feature vector. In the above example, the gradient direction histogram of each subunit has 9 direction intervals, and each block contains 16 subunits. The normalized gradient direction histogram of each block has 9×16=144 elements, so the length of the initial HOG feature vector corresponding to the entire image frame is 300×144=43200.

[0137] In this embodiment, multiple sub-units are combined into a block, and based on the gradient direction histograms corresponding to the multiple sub-units, a gradient direction histogram of the block is obtained, the gradient direction histogram of the block is normalized, and based on the normalized gradient direction histogram of the block, the initial HOG feature corresponding to the image frame is obtained, thereby accurately obtaining the initial HOG feature of the image frame.

[0138] In the above embodiment, in order to capture motion information, the HOG features corresponding to the target image frame and the adjacent image frames of the target image frame are superimposed to obtain the target HOG features of the target image frame. For example, assuming that a video stream contains three consecutive image frames Frame 1, Frame 2, and Frame 3, assuming that a splicing process is used to splice the initial HOG feature vectors of each frame together to construct a high-dimensional spatiotemporal feature vector, assuming that Frame 1 detects the contours of some target objects, and these contours are supplemented in Frame 2 and Frame 3, so that the complete target object detection result can be obtained. Assuming that the size of each frame image is 64×128 pixels, the image is divided into sub-units of 8×8 pixels, and these sub-units are grouped into 2×2 blocks. Each gradient direction histogram contains 9 direction intervals, then 32 blocks can be obtained, and the number of normalized gradient direction histogram elements of each block is 36. Therefore, the length of the initial HOG feature vector of each frame image is 32×36=1152. The initial HOG feature vectors of adjacent Frame 1, Frame 2, and Frame 3 are combined to form a three-dimensional spatiotemporal feature vector, and the target HOG feature vector length is 1152×3=3456.

[0139] Figure 7 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Figure 7 As shown, Figure 7 is Figure 1 Based on the above, a possible implementation of S12 is further described. The process of the method in this embodiment is as follows:

[0140] S121: Perform detection window scanning on the target image frame to obtain target feature parameters corresponding to each detection window.

[0141] Optionally, the target image frame is subjected to detection window scanning at different scales to obtain target feature parameters corresponding to each detection window.

[0142] Specifically, a detection window scan is performed at different scales on each target image frame, and the entire image is searched for areas that may contain target objects, which specifically includes multi-scale search and sliding windows. Among them, multi-scale search is because the target object may appear at different distances and positions in the picture, resulting in different sizes in the picture, so it is necessary to detect at multiple scales, that is, by adjusting the detection windows of different sizes to capture target objects of different sizes. Usually, a series of predetermined window sizes are set, increasing or decreasing according to a certain ratio. For example, a scale step is set, that is, the scale factor is 1.5. If the initial detection window size is 64×128 pixels, the next adjusted detection window size is 96×192 pixels. Sliding window means that at each scale, the detection window is gradually slid on the image to cover the entire image. For example, a sliding step is set to 8 pixels. Assuming that the size of the target image frame is 640×480 pixels, for an original window size of 64×128 pixels, the number of slides in the width direction is The number of slides in the height direction is For a new scale window size of 96×192 pixels, the number of slides in the width direction is The number of slides in the height direction is For each target image frame, detection windows of different scales capture the target object in the image by sliding, and the detection window at each detection position extracts its corresponding target feature parameters.

[0143] S122: Input the target feature parameters corresponding to each detection window into a target classifier to obtain a detection result of the target object.

[0144] Optionally, the target classifier includes: a primary classifier and an advanced classifier.

[0145] Among them, the target classifier is a cascaded multi-level SVM classifier, that is, two or more SVM classifiers are cascaded together, including a primary SVM classifier and an advanced SVM classifier. The target feature parameters corresponding to each detection window are input into the trained cascade SVM classifier for level-by-level classification to obtain the detection result of the target object.

[0146] In this embodiment, by performing a detection window scan on the target image frame, target feature parameters corresponding to each detection window are obtained, and the target feature parameters corresponding to each detection window are input into a target classifier to obtain the detection result of the target object, thereby accurately classifying the detection window containing the target object from the target image frame.

[0147] Figure 8 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Figure 8 As shown, Figure 8 is Figure 7 Based on the above, a possible implementation of S122 is further described. The process of the method in this embodiment is as follows:

[0148] S1221: Input the target feature parameters corresponding to each detection window into the primary classifier to obtain a candidate detection window.

[0149] Among them, the primary classifier is a primary SVM classifier, which is designed as a simple structure. The target feature parameters corresponding to each detection window are input into the primary SVM classifier, which can quickly exclude most background areas and obvious non-target object areas to obtain preliminary screened candidate detection windows.

[0150] S1222: Input the target feature parameters corresponding to the candidate detection window into the high-level classifier to obtain the detection result of the target object.

[0151] Among them, the advanced classifier is an advanced SVM classifier, which is designed to be more complex and accurate. The target feature parameters corresponding to the candidate detection window are input into the advanced SVM classifier, and the advanced SVM classifier performs a more detailed judgment on it, further excludes non-target object areas, and obtains the detection result of the target object.

[0152] In this embodiment, the target feature parameters corresponding to each detection window are input into the primary classifier to obtain a candidate detection window, and the target feature parameters corresponding to the candidate detection window are input into the advanced classifier to obtain the detection result of the target object, thereby screening out the most accurate target object area in the target image frame.

[0153] In the above embodiment, optionally, the detection result of the target object includes: a plurality of target detection windows, each of which includes the target object.

[0154] Among them, during the detection process of the target classifier, multiple overlapping detection windows may be generated pointing to the same target object. In order to obtain the unique and most accurate position of the target object, the detection result of the target object needs to be post-processed. The non-maximum suppression (NMS) method can be used, including: confidence sorting and removing overlapping windows.

[0155] Fig. 9 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Fig. 9 As shown, Fig. 9 is Figure 8 On the basis of, further, also includes:

[0156] S91: Sort the target detection windows based on the confidence levels of the target detection windows.

[0157] Among them, each target detection window in the final target object detection result has a confidence score given by the target classifier, where the confidence score indicates the possibility that the target detection window contains the target object. All target detection windows are sorted according to the confidence score. The target detection window with a higher score is more likely to contain the target object.

[0158] For example, assuming that the detection data of the detection window on the target image frame is as shown in Table 1, the confidence scores of the detection windows A, B, C and D are 0.80, 0.65, 0.70 and 0.85 respectively, then the order of the four detection windows is C, D, B and A if they are sorted from high to low according to the confidence scores.

[0159] Table 1 Detection data of the detection window on the target image frame

[0160] Detection Window Upper left corner coordinates Lower right corner coordinates Confidence score A (200,200) (264,328) 0.65 B (150,150) (214,278) 0.70 C (100,100) (164,228) 0.85 D (102,102) (162,222) 0.80

[0161] S92: Acquire the target detection window with the highest confidence as the reference detection window.

[0162] Among them, the reference detection window is the target detection window that is most likely to contain the target object. For example, in the above example, it can be seen from Table 1 that the detection window C has the highest confidence score and is the detection window that is most likely to contain the target object, that is, the reference detection window.

[0163] S93: Filtering the detection results of the target object based on the degree of overlap between the reference detection window and other target detection windows.

[0164] The reference detection window that is most likely to contain the target object is checked, and based on the degree of overlap between the reference detection window and other target detection windows, other target detection windows that overlap with the reference detection window and have lower confidence scores are removed.

[0165] In this embodiment, the target detection windows are sorted based on the confidence of the target detection windows, and the target detection window with the highest confidence is obtained as a reference detection window. The detection results of the target object are screened based on the degree of overlap between the reference detection window and other target detection windows, thereby retaining the most likely target object position in the detection results.

[0166] Fig.10 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Fig.10 As shown, Fig.10 is Fig. 9Based on the above, a possible implementation of S93 is further described. The process of the method in this embodiment is as follows:

[0167] S931: Determine an overlapping detection window based on an intersection-over-union ratio between the reference detection window and other target detection windows.

[0168] Optionally, for each other target detection window, if an intersection-over-union ratio between the reference detection window and the other target detection window is greater than a preset threshold, the other target detection window is determined to be an overlapping detection window.

[0169] The degree of overlap is usually measured by calculating the intersection over union (IoU) between target detection windows. IoU refers to the ratio of the intersection area of ​​two regions to their union area. Assuming that the area of ​​the reference detection window is M and the area of ​​some other target detection window is N, then their intersection area is M∩N and their union area is M∪N. The intersection over union ratio between the two is expressed as When the IoU exceeds a preset threshold, the target detection window is considered to overlap with the reference target detection window.

[0170] As shown in Table 1, the IoU of the reference detection window C and the other three detection windows is calculated to screen whether there are overlapping detection windows. Assuming that the preset threshold of IoU is 0.5, it can be seen from Table 1 that there is no intersection between detection window C and detection window A, then IoU(C, A) = 0, which is less than the preset threshold of 0.5, so it is determined that detection window A and detection window C do not overlap; the intersection area of ​​detection window C and detection window B is the upper left corner coordinate = (x min ,y min )=(150,150), the lower right corner coordinate=(x max ,y max )=(164,228), the intersection area is 1108, and the union area is 31660, then is less than the preset threshold 0.5, so it is determined that detection window B and detection window C do not overlap. The intersection area of ​​detection window C and detection window D is the upper left corner coordinate = (x min ,y min )=(102,102), the coordinate of the lower right corner=(x max ,y max )=(162,222), the intersection area is 7200, and the union area is 8192, then It is greater than the preset threshold 0.5, so it is determined that the detection window D overlaps with the detection window C.

[0171] S932: Deleting the overlapping detection windows from the detection result of the target object, and retaining the remaining target detection windows.

[0172] Among them, as in the above example, detection window D is an overlapping detection window of detection window C, and detection window D is deleted from the detection result. Since the IoU values ​​of detection windows A and B with detection window C are all less than the preset threshold, it means that detection windows A, B and C all contain different target objects, so detection windows A and B are retained.

[0173] In this embodiment, the detection results of the target object are screened based on the degree of overlap between the reference detection window and other target detection windows, the overlapping detection windows are determined based on the intersection-and-union ratio of the reference detection window and other target detection windows, the overlapping detection windows are deleted from the detection results of the target object, and the remaining target detection windows are retained, thereby effectively reducing redundant detection windows and improving detection accuracy.

[0174] Fig.11 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Fig.11 As shown, Fig.11 is Figure 7 On the basis of, further, before S121, it also includes:

[0175] S1101: Train candidate classifiers based on sample data until the candidate classifiers converge to obtain the target classifier.

[0176] Optionally, the candidate classifier includes: a candidate primary classifier and a candidate advanced classifier, and the candidate primary classifier and the candidate advanced classifier are cascaded; the candidate primary classifier and the candidate advanced classifier are trained step by step based on sample data until the candidate primary classifier and the candidate advanced classifier converge to obtain the target classifier.

[0177] Specifically, the sample data is a continuous sequence of target image frames, including manually annotated target objects and non-target object samples, the candidate classifier is a cascade SVM classifier to be trained, and the target classifier is a trained cascade SVM classifier. The initial HOG features are extracted for each sample, and then the initial HOG features of adjacent samples are superimposed to form the target HOG features. The target HOG features of the sample data are used to train the candidate primary classifier and the candidate high-level classifier step by step until the candidate primary classifier and the candidate high-level classifier converge to obtain a trained cascade SVM classifier, i.e., the target classifier.

[0178] The step-by-step training of candidate classifiers is not a simple process of training the first level classifier and then directly training the next level. Instead, it is a process from simple to complex. Each level of training is to further refine the classification and improve the detection accuracy. First, a simple and fast primary SVM classifier is trained to quickly exclude most non-target object samples, and then more complex and refined advanced SVM classifiers are trained for the remaining candidate areas step by step. The complexity and accuracy of each level will increase.

[0179] Furthermore, the training of each level of classifier has its own convergence conditions, which are usually based on classification error, accuracy, stability of support vector boundaries, etc. When these conditions meet the preset standards, the training ends. Specifically, the convergence conditions usually include but are not limited to the following:

[0180] One convergence condition is error convergence. During training, the optimization algorithm adjusts the model parameters according to the classification error until the error converges to a certain set threshold. When the training error (such as the loss function of SVM) no longer decreases significantly, it indicates that the model has reached the optimal state.

[0181] Another convergence condition is classification accuracy. Convergence conditions can also be based on the accuracy of the classifier. For example, in a primary classifier, when the classifier can achieve a certain detection accuracy (such as the error on the validation set is lower than a certain threshold), the training is stopped and the classifier at this level is considered good enough. Stability of the support vector boundary: The SVM classifier distinguishes different categories by finding the best hyperplane. One of the convergence conditions of training is that the boundary of the support vector no longer changes significantly, that is, the best segmentation boundary is found.

[0182] Another convergence condition is the balance of positive and negative samples. In cascade training, especially for advanced classifiers, the ratio of positive and negative samples is usually adjusted according to the number of samples that pass the primary classifier. When the number of negative samples is greatly reduced and the distribution of positive and negative samples tends to be balanced, it means that the initial screening has been passed and subsequent training can be stopped.

[0183] In this embodiment, the target classifier is obtained by training the candidate classifier based on the sample data until the candidate classifier converges, and a trained cascade multi-level SVM classifier is obtained, which helps to improve the accuracy of detecting the target object.

[0184] In the above embodiment, the cascaded multi-level SVM classifier is trained in a step-by-step training manner under specific conditions. In addition, the cascaded multi-level SVM classifier can also be trained in a joint training manner, which means that the parameters of each level of the classifier need to be adjusted at the same time so that the entire cascade system can be optimized as a whole. Possible implementation methods of joint training include but are not limited to the following two methods:

[0185] One possible implementation is: joint loss function. Define a joint loss function to combine the errors of each level of classifiers (such as the loss function of SVM) into an overall loss. During optimization, minimize the joint loss function by adjusting the parameters of each level of classifiers at the same time. End-to-end training of cascade structure: In a way similar to deep learning, the output of each level of classifiers is used as part of the entire model to form an end-to-end training process. The parameters of SVM can be optimized using gradient descent, but this requires modifying the SVM so that it can be optimized under the joint training framework.

[0186] Another possible implementation method is multi-task learning. Each level of classifier is regarded as a different task. Through the multi-task learning method, different loss functions are designed to ensure that the tasks of each level of classifier are independent, but they are jointly optimized by sharing some parameters (such as the feature extraction part).

[0187] Among them, through joint training, it is possible to find the global optimal solution, so that all classifiers work together to avoid local optimality; joint training allows classifiers at different levels to share information, for example, the primary classifier can be optimized with the advanced classifier to improve the overall detection performance. However, joint training faces the challenges of increased computational complexity, difficulty in parameter coordination, high complexity and low efficiency.

[0188] Fig.12 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Fig.12 As shown, Fig.12 is Figure 1 On the basis of, further, before S11, it also includes:

[0189] S1201: Preprocess the video stream to obtain a preprocessing result, wherein the preprocessing result includes: the target image frame and image frames adjacent to the target image frame.

[0190] Among them, the video stream is a road condition video including but not limited to target objects such as pedestrians. In order to better capture image features, the video stream is preprocessed, and the preprocessing includes: image frame segmentation, image grayscale and normalization processing. The preprocessing result is continuous target image frames that have been grayscaled and normalized, wherein each target image frame contains adjacent image frames.

[0191] In this embodiment, the video stream is preprocessed to obtain a preprocessing result, which includes: the target image frame and the adjacent image frames of the target image frame, thereby obtaining a training sample image and a test sample image of the target classifier, thereby improving the efficiency and accuracy of the HOG feature extraction process.

[0192] Fig.13 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Fig.13 As shown, Fig.13 is Fig.12 On the basis of, further, S1201 includes:

[0193] S12011: Segment the video stream to obtain a plurality of continuous image frames.

[0194] Optionally, the video stream is segmented, and a plurality of image frames obtained by segmentation are consecutively numbered; and consecutive image frames are associated based on the frame numbers.

[0195] The continuous multiple image frames are image frame sequences with continuous timing identifiers, which not only number each frame, but also associate adjacent frames in timing, such as recording the time difference between the previous and next frames. For example, the video stream is processed into image frames with continuous numbers F1, F2, F3, ..., FN, where F1 is associated with F2, F2 is associated with F1 and F3, and so on. The target image frame and its previous and next frames are input into the processing algorithm as a whole to capture the dynamic characteristics of the target object. The video stream is segmented and processed into continuous multiple image frames, which not only considers the spatial information of each frame, but also captures the motion relationship between frames through the timing dimension. For example, a pedestrian moves from left to right, and this movement trend can be captured through continuous frame processing, rather than just the position of the pedestrian at a certain moment.

[0196] In this embodiment, the video stream is segmented to obtain a plurality of continuous image frames, thereby taking into account the temporal and spatial information of the image frames and improving the stability of the algorithm in the face of noise and illumination changes.

[0197] Fig.14 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Fig.14 As shown, Fig.14 is Fig.13 On the basis of, further, also includes:

[0198] S12012: grayscale the multiple image frames to obtain multiple image frames after grayscale processing.

[0199] Among them, color images usually require at least three color channels to represent information, while grayscale images only require one channel to represent brightness information. Grayscale processing of images can reduce the amount of data. Therefore, grayscale processing of the multiple image frames can simplify subsequent calculation processing.

[0200] In this embodiment, by performing grayscale processing on the multiple image frames, multiple image frames after grayscale processing are obtained, thereby saving a certain amount of calculation.

[0201] Fig.15 A flowchart of another method for detecting a target object provided in an embodiment of the present application is shown as follows: Fig.15 As shown, Fig.15 is Fig.14 On the basis of, further, also includes:

[0202] S12013: normalizing the pixel points of the plurality of image frames after the grayscale processing to obtain a plurality of normalized image frames.

[0203] The normalization process is to map the pixel values ​​of the plurality of image frames after the grayscale process to between 0 and 1, so as to improve the stability and consistency of the image feature data.

[0204] In this implementation, the pixel points of the plurality of image frames after the grayscale processing are normalized to obtain the plurality of normalized image frames, thereby improving the stability and consistency of the image features.

[0205] Fig.16 A target object detection device provided in an embodiment of the present application includes: an acquisition module 1601, used to obtain target feature parameters of the target image frame based on the target image frame in a video stream and adjacent image frames of the target image frame; a detection module 1602, used to obtain a detection result of the target object based on the target feature parameters of the target image frame.

[0206] Optionally, the acquisition module 1601 is specifically used to obtain initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame; superimpose the initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame to obtain the target feature parameters of the target image frame.

[0207] Optionally, the overlay processing includes any one of the following processing: splicing processing; weighted overlay processing; convolution processing.

[0208] Optionally, the acquisition module 1601 is specifically used to obtain the initial oriented gradient histogram HOG features corresponding to the target image frame and the adjacent image frames of the target image frame; the initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame are superimposed to obtain the target feature parameters of the target image frame, including: superimposing the HOG features corresponding to the target image frame and the adjacent image frames of the target image frame to obtain the target HOG features of the target image frame.

[0209] Optionally, the acquisition module 1601 is specifically used to obtain the gradient direction and amplitude of each pixel point of the target image frame and any image frame adjacent to the target image frame; based on the gradient direction and amplitude of each pixel point of the image frame, obtain the initial HOG feature corresponding to the image frame.

[0210] Optionally, the acquisition module 1601 is specifically used to divide the image frame into multiple sub-units; for each sub-unit, based on the gradient direction and amplitude of each pixel point in the sub-unit, obtain the gradient direction histogram of the sub-unit; based on the gradient direction histograms corresponding to the multiple sub-units, respectively, obtain the initial HOG features corresponding to the image frame.

[0211] Optionally, the acquisition module 1601 is specifically used to quantize the gradient direction into multiple direction intervals, each direction interval corresponds to a histogram; based on the gradient direction and amplitude of each pixel point in the sub-unit, fill the histogram corresponding to the gradient direction to obtain the gradient direction histogram of the sub-unit.

[0212] Optionally, the acquisition module 1601 is specifically used to determine a target histogram for each pixel in the subunit based on the gradient direction of the pixel, and the gradient direction of the pixel is within a direction interval corresponding to the target histogram; and fill the target histogram according to the amplitude of the pixel.

[0213] Optionally, the acquisition module 1601 is specifically used to combine multiple sub-units into a block; obtain the gradient direction histogram of the block based on the gradient direction histograms corresponding to the multiple sub-units respectively; normalize the gradient direction histogram of the block; and obtain the initial HOG feature corresponding to the image frame based on the normalized gradient direction histogram of the block.

[0214] Optionally, the detection module 1602 is specifically used to perform a detection window scan on the target image frame to obtain target feature parameters corresponding to each detection window; input the target feature parameters corresponding to each detection window into a target classifier to obtain a detection result of the target object.

[0215] Optionally, the detection module 1602 is specifically configured to perform detection window scanning on the target image frame at different scales to obtain target feature parameters corresponding to each detection window.

[0216] Optionally, the target classifier includes: a primary classifier and an advanced classifier;

[0217] The detection module 1602 is specifically used to input the target feature parameters corresponding to each detection window into the primary classifier to obtain a candidate detection window; and input the target feature parameters corresponding to the candidate detection window into the high-level classifier to obtain the detection result of the target object.

[0218] Optionally, the detection result of the target object includes: multiple target detection windows, each target detection window includes the target object, and the detection module 1602 is also used to sort the target detection windows based on the confidence of the target detection windows; obtain the target detection window with the highest confidence as a reference detection window; and filter the detection results of the target object based on the degree of overlap between the reference detection window and other target detection windows.

[0219] Optionally, the detection module 1602 is specifically used to determine the overlapping detection window based on the intersection-and-union ratio of the reference detection window and other target detection windows; delete the overlapping detection window from the detection result of the target object, and retain the remaining target detection windows.

[0220] Optionally, the detection module 1602 is specifically configured to determine, for each other target detection window, if an intersection-over-union ratio between the reference detection window and the other target detection window is greater than a preset threshold, that the other target detection window is an overlapping detection window.

[0221] Optionally, the detection module 1602 is further used to train a candidate classifier based on sample data until the candidate classifier converges to obtain the target classifier.

[0222] Optionally, the candidate classifier includes: a candidate primary classifier and a candidate advanced classifier, and the candidate primary classifier and the candidate advanced classifier are cascaded;

[0223] The detection module 1602 is specifically used to train the candidate primary classifiers and the candidate advanced classifiers step by step based on the sample data until the candidate primary classifiers and the candidate advanced classifiers converge to obtain the target classifier.

[0224] Optionally, the acquisition module 1601 is further configured to preprocess the video stream to obtain a preprocessing result, wherein the preprocessing result includes: the target image frame and image frames adjacent to the target image frame.

[0225] Optionally, the acquisition module 1601 is specifically configured to segment the video stream to obtain a plurality of continuous image frames.

[0226] Optionally, the acquisition module 1601 is specifically configured to segment the video stream, assign consecutive frame numbers to a plurality of image frames obtained by segmentation, and associate consecutive image frames based on the frame numbers.

[0227] Optionally, the acquisition module 1601 is further configured to perform grayscale processing on the multiple image frames to obtain multiple image frames after grayscale processing.

[0228] Optionally, the acquisition module 1601 is further configured to perform normalization processing on the pixels of the plurality of image frames after the grayscale processing to obtain a plurality of normalized image frames.

[0229] The device of this embodiment can be used to execute the technical solutions of the above-mentioned method embodiments accordingly. Its implementation principles and technical effects are similar and will not be repeated here.

[0230] An embodiment of the present application also provides an electronic device, including: a processor, the processor is used to connect to a memory, the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of any of the above-mentioned target object detection method embodiments are implemented.

[0231] An embodiment of the present application also provides a vehicle, comprising the electronic device as described above.

[0232] An embodiment of the present application further provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of any of the above-mentioned target object detection method embodiments are implemented.

[0233] An embodiment of the present application also provides a computer program product, which, when executed by a processor of a vehicle or a cloud server, implements the steps of any of the above-mentioned target object detection method embodiments.

[0234] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0235] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, disk, CD, etc.), including several instructions to enable a terminal or a network-side device to execute the methods described in each embodiment of the present application.

[0236] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of the present application and the scope of protection of the claims, and these implementation methods are all within the protection of the present application.

Claims

1. A method for detecting a target object, characterized in that: include: Based on a target image frame in a video stream and image frames adjacent to the target image frame, obtaining target feature parameters of the target image frame; Based on the target feature parameters of the target image frame, a detection result of the target object is obtained.

2. The method according to claim 1, characterized in that The step of obtaining target feature parameters of the target image frame based on the target image frame in the video stream and the adjacent image frames of the target image frame comprises: Acquire initial feature parameters corresponding to the target image frame and image frames adjacent to the target image frame respectively; The initial feature parameters corresponding to the target image frame and the image frames adjacent to the target image frame are superimposed to obtain the target feature parameters of the target image frame.

3. The method according to claim 2, characterized in that The superposition process includes any of the following processes: Splicing processing; Weighted overlay processing; Convolution processing.

4. The method according to claim 2, characterized in that: The obtaining of initial feature parameters respectively corresponding to the target image frame and the image frames adjacent to the target image frame comprises: Acquire initial oriented gradient histogram HOG features corresponding to the target image frame and the image frames adjacent to the target image frame respectively; The step of superimposing the initial feature parameters corresponding to the target image frame and the adjacent image frames of the target image frame to obtain the target feature parameters of the target image frame includes: The HOG features corresponding to the target image frame and the adjacent image frames of the target image frame are superimposed to obtain the target HOG features of the target image frame.

5. The method according to claim 4, characterized in that The step of acquiring initial directional gradient histogram HOG features corresponding to the target image frame and the image frames adjacent to the target image frame respectively includes: For the target image frame and any one of the image frames adjacent to the target image frame, obtain the gradient direction and amplitude of each pixel point of the image frame; Based on the gradient direction and amplitude of each pixel point of the image frame, an initial HOG feature corresponding to the image frame is obtained.

6. The method according to claim 5, characterized in that The obtaining of the initial HOG feature corresponding to the image frame based on the gradient direction and amplitude of each pixel point of the image frame includes: Dividing the image frame into a plurality of sub-units; For each subunit, based on the gradient direction and amplitude of each pixel point in the subunit, obtain a gradient direction histogram of the subunit; Based on the gradient direction histograms respectively corresponding to the multiple sub-units, an initial HOG feature corresponding to the image frame is obtained.

7. The method according to claim 6, characterized in that The step of obtaining a gradient direction histogram of the subunit based on the gradient direction and amplitude of each pixel point in the subunit comprises: Quantize the gradient direction into multiple direction intervals, each direction interval corresponds to a histogram; Based on the gradient direction and amplitude of each pixel point in the sub-unit, the histogram corresponding to the gradient direction is filled to obtain the gradient direction histogram of the sub-unit.

8. The method according to claim 7, characterized in that The filling of the histogram corresponding to the gradient direction based on the gradient direction and the amplitude of each pixel point in the subunit includes: For each pixel in the subunit, a target histogram is determined based on the gradient direction of the pixel, and the gradient direction of the pixel is within a direction interval corresponding to the target histogram; and the target histogram is filled according to the amplitude of the pixel.

9. The method according to claim 6, characterized in that The obtaining the initial HOG feature corresponding to the image frame based on the gradient direction histograms respectively corresponding to the multiple sub-units includes: Combine multiple subunits into blocks; Based on the gradient direction histograms respectively corresponding to the multiple sub-units, obtaining the gradient direction histogram of the block; Normalizing the gradient direction histogram of the block; Based on the normalized gradient direction histogram of the block, an initial HOG feature corresponding to the image frame is obtained.

10. The method according to claim 2, characterized in that The acquiring the detection result of the target object based on the target feature parameter of the target image frame includes: Performing a detection window scan on the target image frame to obtain target feature parameters corresponding to each detection window; The target feature parameters corresponding to each detection window are input into the target classifier to obtain the detection result of the target object.

11. The method according to claim 10, characterized in that The scanning of the target image frame by a detection window to obtain target feature parameters corresponding to each detection window includes: The target image frame is scanned by a detection window at different scales to obtain target feature parameters corresponding to each detection window.

12. The method according to claim 10, characterized in that The target classifier includes: a primary classifier and an advanced classifier; The step of inputting the target feature parameters corresponding to each detection window into a target classifier to obtain a detection result of the target object includes: Inputting the target feature parameters corresponding to each detection window into the primary classifier to obtain a candidate detection window; The target feature parameters corresponding to the candidate detection window are input into the high-level classifier to obtain the detection result of the target object.

13. The method according to claim 12, characterized in that The detection result of the target object includes: a plurality of target detection windows, each of which includes the target object. The method further includes: sorting the target detection windows based on the confidence levels of the target detection windows; The target detection window with the highest confidence is obtained as the reference detection window; The detection results of the target object are screened based on the degree of overlap between the reference detection window and other target detection windows.

14. The method according to claim 13, characterized in that The screening of the detection result of the target object based on the overlap degree between the reference detection window and other target detection windows includes: Determining an overlapping detection window based on an intersection-over-union ratio between the reference detection window and other target detection windows; The overlapping detection windows are deleted from the detection results of the target object, and the remaining target detection windows are retained.

15. The method according to claim 14, characterized in that The determining of the overlapping detection window based on the intersection-over-union ratio of the reference detection window and other target detection windows includes: For each other target detection window, if the intersection-over-union ratio between the reference detection window and the other target detection window is greater than a preset threshold, the other target detection window is determined to be an overlapping detection window.

16. The method according to claim 10, characterized in that Also includes: The candidate classifier is trained based on the sample data until the candidate classifier converges to obtain the target classifier.

17. The method according to claim 16, characterized in that The candidate classifiers include: a candidate primary classifier and a candidate advanced classifier, wherein the candidate primary classifier and the candidate advanced classifier are cascaded; Training a candidate classifier based on sample data until the candidate classifier converges to obtain the target classifier, including: Based on the sample data, the candidate primary classifiers and the candidate high-level classifiers are trained step by step until the candidate primary classifiers and the candidate high-level classifiers converge to obtain the target classifier.

18. The method according to claim 1, characterized in that Before obtaining the target feature parameters of the target image frame based on the target image frame in the video stream and the image frames adjacent to the target image frame, the method further includes: The video stream is preprocessed to obtain a preprocessing result, wherein the preprocessing result includes: the target image frame and image frames adjacent to the target image frame.

19. The method according to claim 18, characterized in that The preprocessing of the video stream to obtain a preprocessing result includes: The video stream is segmented to obtain a plurality of continuous image frames.

20. The method according to claim 19, characterized in that The video stream is segmented to obtain a plurality of continuous image frames, including: Segmenting the video stream, and assigning consecutive frame numbers to a plurality of image frames obtained by segmentation; Consecutive image frames are associated based on the frame numbers.

21. The method according to claim 19, characterized in that Also includes: Grayscale processing is performed on the multiple image frames to obtain multiple image frames after grayscale processing.

22. The method according to claim 19, characterized in that Also includes: The pixel points of the plurality of image frames after the grayscale processing are normalized to obtain a plurality of normalized image frames.

23. An electronic device, characterized in that: include: A processor, wherein the processor is used to be connected to a memory, wherein the memory stores a computer program / instruction that can be run on the processor, and when the computer program / instruction is executed by the processor, the steps of the target object detection method as described in any one of claims 1 to 22 are implemented.

24. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 23.

25. A computer-readable storage medium, characterized in that: The readable storage medium stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the method for detecting a target object as described in any one of claims 1 to 22 are implemented.

26. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the target object detection method according to any one of claims 1 to 22 are implemented.

Images