Multi-view target detection method, device, computer equipment and storage medium

Abstract

An embodiment of the present invention proposes a multi-view target detection method, apparatus, computer equipment and storage medium, which belong to the field of computer vision. The method includes: obtaining multiple local image blocks of an image to be tested, and image block features of each local image block, and using a preset random forest model based on each image block feature to divide the multiple local image blocks into multiple visual words, thereby calculating the voting score of each visual word for each candidate center position of the image to be tested, obtaining the voting combination weight of each candidate center position with respect to each visual word, and then calculating the total score of each candidate center position based on the voting combination weight and the voting score, and determining the target center position based on the total score of each candidate center position. The method can fully consider the perspective factor, thereby reducing the influence of interference such as target occlusion, deformation and angle change, and improving the accuracy of target detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a multi-view target detection method, apparatus, computer equipment and storage medium. Background Art

[0002] Object detection is a fundamental problem in computer vision. Its core task is to determine the location and size of a specific target within any given image using a target recognition algorithm and search strategy. While target detection technology has made significant progress, most target trackers use a global, holistic appearance model to extract features to determine the target's location. When the target is obscured, deformed, or experiences changes in angle, the tracker's tracking performance can be poor, leading to low target detection accuracy. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a multi-perspective target detection method, device, computer equipment and storage medium, which can improve the current target detection method, and can improve the problem of low target detection accuracy of traditional target detection methods.

[0004] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows.

[0005] In a first aspect, an embodiment of the present invention provides a multi-view target detection method, which adopts the following technical solution.

[0006] A multi-view target detection method, the method comprising:

[0007] Acquire multiple local image blocks of the image to be tested, and an image block feature of each of the local image blocks;

[0008] Based on the features of each of the image blocks, a preset random forest model is used to divide the multiple local image blocks into multiple visual words, wherein the local image blocks of the same visual word have the same appearance features and geometric positions;

[0009] Calculating a voting score of each visual word for each candidate center position of the image to be tested;

[0010] Obtaining a combined voting weight for each candidate center position with respect to each visual word, and calculating a total score for each candidate center position based on the combined voting weight and the voting score;

[0011] The target center position is determined according to the total score of each candidate center position.

[0012] Furthermore, the step of determining the target center position according to the total score of each candidate center position includes:

[0013] Obtaining a voting mean according to the total score of each candidate center position;

[0014] Scaling the image to be tested under multiple scale factors, and combining the total score to obtain a Hough map under each scale factor;

[0015] According to the Hough graph at each scale factor, a 3D Hough space is obtained;

[0016] The target center position is determined in the 3D Hough space using a mean shift algorithm in combination with the voting mean.

[0017] Furthermore, the step of obtaining a voting combination weight of each candidate center position with respect to each visual word includes:

[0018] Acquire multiple sample images of the verification object and the background at various viewing angles, and divide all the sample images into multiple visual subcategories;

[0019] A plurality of visual subclasses are obtained from leaf nodes of a preset random forest model, an expression vector for each sample image is generated based on the visual subclasses, and a subclass label is added to each sample image according to the visual subclass to which each sample image belongs;

[0020] Based on the expression vectors and the subclass labels of all the sample images, a multi-class linear classification algorithm is used to train a score weight for each visual subclass.

[0021] Furthermore, the step of dividing all the sample images into a plurality of visual subcategories includes:

[0022] Calculating the HOG features of each of the sample images, and embedding all of the HOG features into a 2D space using a local linear embedding algorithm, where the 2D space includes a plurality of sample points, each of which corresponds to one of the sample images;

[0023] In the 2D space, all the sample images are regularized onto a circle, and unsupervised clustering is performed on the sample images on the circle to obtain a plurality of visual subclasses.

[0024] Furthermore, the step of dividing the plurality of local image blocks into a plurality of visual words using a preset random forest model according to the features of each of the image blocks includes:

[0025] For each of the image block features, traverse the random forest model using the image block feature until the image block feature reaches a leaf node of the random forest model, each leaf node corresponding to a visual word;

[0026] According to the leaf nodes reached by all the image block features, the local image blocks corresponding to all the image block features reaching the same leaf node are regarded as a visual word.

[0027] Furthermore, the method further comprises the step of training a random forest model, which comprises:

[0028] Obtaining sample images of the sample object at different viewing angles, dividing each of the sample images into a plurality of local image blocks, and using all the local image blocks as a training set;

[0029] A random forest model is constructed and trained based on a training algorithm according to the features of each local image block, wherein the features include appearance features, perspective categories, offset vectors from the local image blocks to the target center, and classification weights of the local image blocks at each layer node of the random forest model.

[0030] Furthermore, the step of constructing and training a random forest model based on a training algorithm according to the features of each local image block includes:

[0031] Generating the first layer nodes of each tree of the random forest model according to the training algorithm and the characteristics of each of the local image blocks;

[0032] When each layer of nodes is generated, the splitting function is obtained according to the set of local image blocks in each node on the current layer;

[0033] Predicting and updating the classification weight of each of the local image blocks on the lower layer nodes according to the splitting function;

[0034] Each time a layer of nodes is generated, it is determined whether the current random forest model meets the termination condition. If so, the training is terminated to obtain the trained random forest model. If not, the features after the classification weights are updated according to each of the local image blocks, and the next layer of nodes of each tree of the random forest model is continued to be generated.

[0035] In a second aspect, an embodiment of the present invention provides a multi-view target detection device, which adopts the following technical solution.

[0036] A multi-view target detection device includes a preprocessing module, a matching module and a voting module;

[0037] The preprocessing module is used to obtain multiple local image blocks of the image to be tested, and the image block features of each local image block;

[0038] The matching module is configured to divide the plurality of local image blocks into a plurality of visual words based on the features of each of the image blocks using a preset random forest model, wherein the local image blocks of the same visual word have the same appearance features and geometric positions;

[0039] The voting module is configured to calculate the voting score of each visual word for each candidate center position of the image to be tested, obtain a combined voting weight of each candidate center position with respect to each visual word, calculate a total score of each candidate center position based on the combined voting weight and the voting score, and determine a target center position based on the total score of each candidate center position.

[0040] In a third aspect, an embodiment of the present invention provides a computer device that adopts the following technical solution.

[0041] A computer device includes a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the multi-view target detection method as described in the first aspect.

[0042] In a fourth aspect, an embodiment of the present invention provides a storage medium that adopts the following technical solution.

[0043] A storage medium stores a computer program, which, when executed by a processor, implements the multi-view target detection method as described in the first aspect.

[0044] The multi-perspective target detection method, device, computer equipment and storage medium provided by the embodiments of the present invention use a random forest model to divide multiple local image blocks of the image to be tested into multiple visual words, each visual word representing a perspective, thereby obtaining the voting score of each visual word (i.e., from each perspective) for the candidate center position of the image to be tested, that is, multiple perspectives vote on the candidate center position, and according to the voting combination weight (i.e., the voting weight of each perspective word with respect to the candidate center position), the total score of each candidate center is obtained, and then according to the total score, the target center position is determined, and the voting of the candidate center position by multiple perspectives is realized, and the total score is counted in a weighted manner, which can fully consider the perspective factor, thereby reducing the influence of interference such as target occlusion, deformation and angle change, and improving the accuracy of target detection.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart illustrating some steps of a multi-view target detection method provided by an embodiment of the present invention is shown.

[0048] Figure 2 A flowchart illustrating another part of the steps of the multi-view target detection method provided by an embodiment of the present invention is shown.

[0049] Figure 3 Shown Figure 2 Schematic diagram of the process of some sub-steps of step S20.

[0050] Figure 4 Shown Figure 1 Flow chart of some sub-steps of step S103.

[0051] Figure 5 Shown Figure 1 Flow chart of some sub-steps of step S107.

[0052] Figure 6 Shown Figure 5 Flow chart of some sub-steps of step S107-1.

[0053] Figure 7 Shown Figure 1 Flow chart of some sub-steps of step S109.

[0054] Figure 8 A block diagram of a multi-view target detection device provided by an embodiment of the present invention is shown.

[0055] Figure 9 A block diagram of a computer device provided by an embodiment of the present invention is shown.

[0056] Icons: 100 - multi-view target detection device; 110 - preprocessing module; 120 - matching module; 130 - voting module; 140 - computer equipment. DETAILED DESCRIPTION

[0057] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0058] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0059] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0060] Object detection is the most basic problem in the field of computer vision. Its core task is to use a certain object recognition algorithm and search strategy to determine the position and size of a specific object in any given image.

[0061] In recent years, the application of learning in computer vision has been increasing. Convolutional Neural Networks (CNNs), which leverage receptive fields and a layered, incremental feature plane processing approach, offer advantages in processing image data and have become a key tool in computer vision. Currently, researchers have proposed numerous CNN-based solutions for object detection.

[0062] CNN-based solutions can be roughly divided into three categories according to the target detection process. The first type of CNN-based solution uses a pure copper sliding window for target detection and uses CNN as a classifier to determine whether the input sub-window is the target or the background. This method requires judgment on all windows, which takes a long time to detect and has low detection efficiency. The second type of CNN-based solution performs target detection based on region selection. It first selects candidate areas that may be targets from the entire image, and then uses CNN to classify these and determine whether they are background or targets. This type of method relies too much on the region selection algorithm and is prone to missed detection or false detection. The third type of CNN-based solution achieves target detection by predicting the target position of the entire image. The CNN input is the entire image, and the output is the target position information.

[0063] Obviously, the trackers in most of the above methods use a global holistic appearance model to extract features to determine the target's position. When interference such as occlusion, image deformation, angle changes, lighting changes, and posture changes occur, the tracker's tracking performance is poor, resulting in low target detection accuracy.

[0064] Based on the above considerations, an embodiment of the present invention provides a multi-view object detection solution that can improve the low target detection accuracy problem of existing target detection solutions. The multi-view object detection solution provided by an embodiment of the present invention will be described in detail below.

[0065] In one embodiment, referring to Figure 1 , provides a multi-view target detection method, comprising the following steps. In this embodiment, the execution of the multi-view target detection method provided can be a server or a terminal, and an adaptive selection can be made according to the usage situation.

[0066] S101 , obtaining a plurality of local image blocks of an image to be tested and an image block feature of each local image block.

[0067] The image to be tested can be divided into multiple local image blocks with a certain pixel size, such as 16×16. It should be understood that the local image blocks may have overlapping parts or may not overlap. The local image blocks can be obtained by sliding a certain pixel size pixel by pixel in the image to be tested.

[0068] Image block features include appearance features and geometric positions. Appearance features include but are not limited to pixel intensity values, color space, and gradient-related features.

[0069] S103 , dividing the multiple local image blocks into multiple visual words using a preset random forest model according to the features of each image block.

[0070] Among them, the local image patches of the same visual word have the same appearance features and geometric positions.

[0071] It should be understood that the same appearance feature means that the parameters of the appearance feature are within the same range of values, and the same geometric position means that the parameters of the geometric position are within the same range of values.

[0072] Since the local image blocks of the same visual word have the same geometric position, the local image blocks of a visual word are from the same perspective, which can also be understood as a visual word representing a perspective.

[0073] Step S103 is to match each local image block with a visual word, determine the viewing angle of the local image block, and divide the local image block into multiple viewing angles.

[0074] S105 , calculating the voting score of each visual word for each candidate center position of the image to be tested.

[0075] The candidate center position may be any position in the image to be measured, or may be a candidate center position predicted by other center prediction methods, such as using a CNN to predict the candidate center position, or obtaining the candidate center position according to a preset rule.

[0076] S107 , obtaining the combined voting weights of each candidate center position with respect to each visual word, and calculating the total score of each candidate center position based on the combined voting weights and the voting scores.

[0077] The combined voting weight refers to the weight of each visual word's votes in the total score. The same visual word can have different weights for different candidate center positions. Alternatively, the same visual word can have the same weight for all candidate center positions.

[0078] S109: Determine the target center position based on the total score of each candidate center position.

[0079] Compared to existing object detection methods that suffer from low accuracy due to interference such as object occlusion, image deformation, angle changes, illumination changes, and posture changes, the multi-view object detection method utilizes a random forest model to divide multiple local image blocks of the image to be detected into multiple visual words, each representing a viewpoint. This method then votes on candidate center positions from multiple viewpoints, mitigating the effects of interference such as object occlusion, image deformation, angle changes, illumination changes, and posture changes to a certain extent.

[0080] Moreover, by assigning different weights to different visual words (perspectives) through voting combination weights, the total score of each candidate center is obtained, which can fully consider the perspective factor and thus improve the accuracy of target detection.

[0081] In order to obtain the random forest model used in the above step S103, the multi-view target detection method provided by the embodiment of the present invention further includes a step of training the random forest model. The execution subject of this step can be the same as that of the above steps S101-S109, or it can be different. Figure 2 , including the following steps.

[0082] S201 , obtaining sample images of sample objects at different viewing angles, dividing each sample image into a plurality of local image blocks, and using all the local image blocks as a training set.

[0083] The sample object is photographed from different perspectives to obtain sample images of different perspectives, and each sample image is divided into multiple image blocks of the same size.

[0084] S202: Construct and train a random forest model based on the features of each local image block and a training algorithm.

[0085] Among them, the features include appearance features, perspective categories, offset vectors from local image patches to the target center, and classification weights of local image patches on each layer of nodes in the random forest model.

[0086] The viewpoint category of each local image patch is consistent with the viewpoint of the sample image. The viewpoint category is the label of the local image patch. The target center is the target center of the sample object. Based on the center position of the local image patch, an offset vector from the local image patch to the target center can be calculated. The offset vector can be an offset vector in 2D space.

[0087] The training set can be expressed as: Among them, p i Representing the local image patch, f i Characterize the appearance characteristics of the local image block, y i Characterize the view category of the local image patch, d i Represents the offset vector from the center of the local image block to the center of the target, m represents the number of node layers of the random forest, Represents the classification weight of the local image block at each layer node, N represents the number of sample images, and O represents that each sample image is divided into O local image blocks.

[0088] Appearance features i It can be represented by the feature map of A channels, that is, f i ={f i 1 ,fi 2 ,...,f i A If the local image block is an image block extracted from the background image, there is no di. At the root node of the random forest (i.e., m = 0), the classification weights of all local image blocks can be uniformly initialized as:

[0089] It should be noted that after training is completed, the local image blocks that reach the leaf nodes of each tree of the random forest model can be used to define visual words.

[0090] After obtaining the training set, refer to Figure 3 , the above-mentioned step S202 may include the following steps.

[0091] S202-1, based on the training algorithm and the features of each local image block, generate nodes of each layer of each tree of the random forest model layer by layer based on the depth.

[0092] The training algorithm can be flexibly selected, for example, the Boosting algorithm and the Bagging algorithm. In this embodiment, the training algorithm is the Boosting algorithm, and the obtained random forest model is a Gradient Boosting Random Forests (GBRF) model improved based on the boosting algorithm.

[0093] S202-2, each time a layer of nodes is generated, a splitting function is obtained according to a set of local image blocks in each node on the current layer of nodes.

[0094] It should be understood that the splitting function is essentially the loss function used when training the random forest model in the embodiments of the present invention. The splitting function loops through the local image block sample set at the node and records the matching scores of the local image blocks (8*8 pixel units) in the sample. If the matching branch is greater than the threshold, the splitting function is 0, otherwise it is 1.

[0095] When generating nodes of a certain depth (depth can also be understood as the number of node layers), nodes are randomly selected to maximize class purity or minimize the squared deviation of the offset vector to learn the splitting function of non-terminal nodes at that depth.

[0096] S202-3, predicting and updating the classification weight of each local image block on the lower layer node according to the splitting function.

[0097] It should be noted that if the node division of a certain layer in the random forest model is defined based on minimizing the squared deviation of the offset vector, that is, the classification function is learned by minimizing the squared deviation of the offset vector, then the classification weight of the local image block is retained to the lower layer and does not need to be updated. That is, the classification weight update is only for the division layer defined based on the sample class purity.

[0098] S202-4, when each layer of nodes is generated, it is determined whether the current random forest model meets the termination condition. If not, step S202-5 is executed, otherwise step S202-6 is executed.

[0099] Among them, the termination conditions include any one or more of the following: (1) reaching the maximum depth of the set random forest model; (2) the number of local image blocks in the bottom-level nodes reaches the minimum requirement for division; (3) the class purity of the local image blocks on each node in the bottom layer reaches the maximum or minimum value. If the class purity of the positive image block reaches the maximum value, it is also necessary to meet the requirement that the deviation square difference of the corresponding offset vector reaches the minimum value.

[0100] S202-5, continuing to generate the next layer of nodes of each tree of the random forest model based on the features after the classification weights of each local image block are updated.

[0101] S202-6, terminate the training and obtain the trained random forest model.

[0102] The construction process of the random forest model is to continuously divide the local image patch set and train visual words (each time the lowest node is a visual word). The core idea is to alternately optimize the class purity of the image patch set on each tree node, or the square difference of the relative offset vector of the positive sample local image patch in the node, to split the local image patch sample set of the node, so that the class purity of the image patch set in the child node after the split continues to increase and the squared deviation gradually decreases.

[0103] Unlike the Hough Forest (HF) approach where each tree is constructed independently, in the embodiment of the present invention, when using GREF to train visual words, that is, when training the random forest model, all trees in the forest are grown layer by layer based on depth as a whole. By defining a global split function (loss function), the division of the local image block set at each layer node has the smallest classification error.

[0104] Furthermore, each time a layer of nodes is generated, the local image blocks are weighted according to the splitting function (prediction loss). That is, the classification weights of the local image blocks on the lower layer nodes are updated based on the current classification function value (prediction loss), so that the misclassified local image blocks have a larger classification weight in the lower layer division. This helps improve classification accuracy.

[0105] As GBRF is constructed layer by layer, the set of local image blocks on the node will be continuously divided according to its visual category or positive sample offset vector until it reaches a leaf node, so that the local image blocks reaching the leaf node have similar appearance features and geometric positions, and the leaf node can be regarded as a visual word.

[0106] Because the visual words obtained by the trained random forest model contain class distribution probabilities and offset vectors relative to the target center, they can be used not only for classification but also for determining the target center. In this embodiment of the present invention, the random forest model updates the image block partition weights, allowing the lower-level partitioning to pay more attention to misclassified local image blocks, effectively reducing classification errors and increasing the discriminability of visual words.

[0107] After training the random forest model, in one embodiment, refer to Figure 4 , step S103 may include the following steps.

[0108] S103-1, for each image block feature, traverse the random forest model using the image block feature until the image block feature reaches a leaf node of the random forest model.

[0109] Each leaf node corresponds to a visual word. Each local image block can be represented as: e =(q e ,f e ), p e Representing local image patches, q e Characterizes the geometric position of the center of the local image block in the image to be tested, f e Characterize the appearance features of local image blocks.

[0110] The random forest model is used to classify all local image blocks of the image to be tested according to the image block features. Local image blocks with close or similar features will reach the same leaf node to achieve classification.

[0111] S103-2: Based on the leaf nodes reached by all the image block features, the local image blocks corresponding to all the image block features reaching the same leaf node are regarded as a visual word.

[0112] The local image blocks that reach the same leaf node represent the same view category, that is, the same view.

[0113] Steps S103 - 1 and S103 - 2 actually match the local image blocks with the leaf nodes of the random forest model to divide the local image blocks of the image to be tested into multiple local image block sets (ie, multiple visual words).

[0114] After converting the local image block of the image to be tested into multiple visual words, a vote is performed on each candidate center position of the image to be tested, and the voting is performed in the form of voting points.

[0115] The method of calculating the voting points in step S105 can be flexibly selected, for example, voting according to preset rules, or voting using CNN.

[0116] In one embodiment, the voting score of the visual word for each candidate center position may be obtained through the class purity and offset vector set of the positive examples in the visual word (local image block set).

[0117] Assume that the visual word is W i , the candidate center position is x, then the votes of the visual words on the candidate center position are divided into Score(x|W i ,p e ), which can also be understood as the response value of the visual word to the candidate center position is R i (x,W i )=Score(x|W i ,p e ).

[0118] In one embodiment, Among them, C L Characterizes the class purity of a set of local image patches in a visual word, Δ i Represents the mean offset vector of each local image block in the visual word, w i The unit ratio representing the visual word vote is determined by the number of offset vectors in the visual word and the total number of offset vectors. b is the parameter and ζ(ρ) represents the exponential function. It returns 1 when ρ is true and 0 when ρ is false.

[0119] ρ=(C L >λ1)∧(Num i >λ2), λ1 and λ2 are used to constrain the minimum probability of the distribution of positive local image blocks that can participate in voting and the minimum number of offset vectors in visual words, respectively. i Represents the number of offset vectors in a visual word.

[0120] It should be noted that visual words are independent of each other, so the voting process of visual words for the candidate center position x in the image to be tested is also independent of each other. Therefore, the voting scores (response value vectors) of different visual words for the candidate center position x can be expressed as: R(x) = [R1(x),...,R Nw (x)] T .

[0121] After obtaining the vote scores of each visual word for each candidate center position and obtaining the combined vote weights in step S107, in one embodiment, the total score of each candidate center position can be expressed as: That is the formula for calculating the total score.

[0122] Among them, Test represents the image to be tested, Map(p e ) represents the local image patch p e The set of visual words, that is, the set of visual words, usually [W i ∈Map(p e )] has a value of 1, Represents the weight of the visual word with respect to the candidate center position x.

[0123] Through the above content, the total score of each candidate center position takes into account the different appearance forms of similar targets, such as perspective differences, thereby reducing the impact of interference and improving detection accuracy.

[0124] In one embodiment, the combined voting weight of each visual word with respect to the candidate center position can be expressed as: V =(βv,1,...,βV,Nw), where v∈[1,V] and V is the number of visual words.

[0125] Further, refer to Figure 5 , is a flowchart of some sub-steps of the above step S107, and the voting combination weight of each candidate center position with respect to each visual word is obtained through the following steps.

[0126] S107-1, obtaining multiple sample images of the verification object and background under various viewing angles, and dividing all the sample images into multiple visual subcategories.

[0127] All sample images of the verification object constitute the verification set Q, The verification object may be a vehicle.

[0128] S107-2, obtain multiple visual subclasses based on the leaf nodes of the preset random forest model, generate an expression vector for each sample image based on the visual subclasses, and add a subclass label to each sample image according to the visual subclass to which each sample image belongs.

[0129] Using the trained random forest model, after inputting the sample images into the random forest model, the sample images reaching the same leaf node are regarded as a visual subclass. Each sample image is expressed as a vector based on the visual subclass and the center position of the sample image to obtain the expression vector of each sample image.

[0130] The method for expressing each sample image as a vector based on the visual subclass and the center position of the sample image can be flexibly selected, for example, calculated according to a preset rule. In one embodiment, the expression vector of each sample image can be obtained by calculating the vector between the center position of the visual subclass and the center position of the sample image.

[0131] S107-3, based on the expression vectors and subclass labels of all sample images, use a multi-class linear classification algorithm to train the score weights of each visual subclass.

[0132] The multi-category linear classification algorithm is the SVM support vector machine. Through the SVM vector machine, the score weight of each visual subclass can be predicted, and the score weights of all visual subclasses can be combined to obtain the voting combined weight of each visual word about the candidate position center.

[0133] Further, refer to Figure 6 , the division of all sample images into multiple visual subcategories in the above step S107 - 1 can be achieved through the following steps.

[0134] S301, calculating the HOG features of each sample image, and using a local linear embedding algorithm to embed all HOG features into a 2D space.

[0135] The 2D space includes multiple sample points, each of which corresponds to a sample image. In essence, the sample point represents the sample image.

[0136] The HOG feature is used to describe the positive sample, and the obtained positive sample set is recorded as: Among them, N2 is the number of positive samples of Q, H k is the HOG feature of the kth positive sample.

[0137] It should be understood that the sample images in steps S301 - S302 are all positive sample images.

[0138] The HOG feature, or Histogram of Oriented Gradients (HOG), is a feature descriptor used for object detection in computer vision and image processing. The Local Linear Embedding (LLE) algorithm is a nonlinear dimensionality reduction algorithm that effectively preserves the original manifold structure of the reduced data.

[0139] Since the perspective of verification objects such as vehicles can change 360 ​​degrees, it is difficult to reflect this change in high-dimensional feature space. Therefore, the LLE algorithm is first used to embed the high-dimensional HOG features into 2D space. After the high-dimensional HOG features of multi-perspective vehicles are embedded in 2D space, the sample points are distributed in a ring shape, and the perspective changes smoothly along this ring.

[0140] S302 , in a 2D space, regularize all sample images onto a circle, perform unsupervised clustering on the sample images on the circle, and obtain multiple visual subclasses.

[0141] The unsupervised clustering used can be a clustering algorithm such as the k-means algorithm and the scikit-learn algorithm.

[0142] After step S302, multiple visual subclasses are obtained. At this time, the verification set Q can be redefined as: α represents the total number of visual subclasses, j=0 represents the background sample set, and the other subsets each represent a visual subclass sample set.

[0143] After the validation set is divided in this way, each sample image in the original validation set Q is set as a multi-class label That is, when the jth sample belongs to subclass Q v hour, otherwise The center position of the sample image when the sample size is normalized to Width×Height First, we need to use the visual word set Ω to extract the matching relationship between the local image block and the visual words in Ω, and calculate the visual word in each sample image. The voting response vector of the center position o; then, using R j (o) Expression sample image L j , which contains both the appearance features and center position information of the target. The final validation set can be defined as:

[0144] The sample images in the validation set are represented using HOG features. The traditional approach to automatic subclassing is to perform unsupervised clustering directly in the HOG feature space. However, high-dimensional feature clustering inherently lacks stability, and the resulting subclasses cannot guarantee viewpoint consistency, making them unsuitable for the multi-view vehicle detection problem addressed by the present invention. Therefore, in the present invention, after S301 and S302, unsupervised clustering is performed in 2D space to improve viewpoint consistency among subclasses.

[0145] Through the above steps S107 - 1 to S107 - 3 , and S301 and S302 , the voting combination weights can be obtained, so that the total score of each candidate center position can be calculated using the total score calculation formula.

[0146] On the basis of the above, in one embodiment, referring to Figure 7 , is a flowchart of the sub-steps of the above-mentioned step S109, including the following steps.

[0147] S109-1, obtain the voting mean based on the total score of each candidate center position.

[0148] S109-2, scaling the image to be tested under multiple scale factors, and combining the total scores to obtain a Hough map under each scale factor.

[0149] S109-3, obtain the 3D Hough space according to the Hough graphs under each scale factor.

[0150] The Hough graphs at each scale factor are stacked to generate a 3D Hough space, where the dimensions of the 3D Hough space can represent the horizontal coordinate, vertical coordinate, and scale, respectively.

[0151] S109-4, combined with the voting mean, uses the mean shift algorithm to determine the target center position in the 3D Hough space.

[0152] The traditional method first finds the candidate center location with the highest voting score, then uses a threshold comparison to determine whether it is the target center location. If it is the target center location, the target is calibrated with a detection box of size Width × Height centered at that location. This method is simple and easy to implement, but in practice, it lacks stability and is easily affected by loud noise.

[0153] The embodiment of the present invention uses the mean shift algorithm through the above steps S109-1 to S109-4 to determine the target center position in the 3D Hough space, which can improve stability and reduce noise interference, thereby improving detection accuracy.

[0154] The multi-view target detection method provided by the embodiment of the present invention is a weighted Hough voting detection method suitable for multi-view vehicle detection objects. It can not only reflect the differences between multi-view targets, but also effectively utilize data sharing between multi-view targets, greatly improving the stability, accuracy and anti-interference ability of target detection.

[0155] At the same time, during the detection process, local image blocks are extracted from the image to be tested, and the matching relationship between local image blocks and visual words is established using the pre-trained random forest model. That is, all local image blocks are divided into multiple visual words, and the possible target center positions are voted based on the voting scores and combined weights, which greatly improves the accuracy of target detection.

[0156] The multi-view object detection method provided by the present invention uses weighted voting to discriminate between multi-view objects using local image blocks. This method not only demonstrates the differences between multi-view objects but also demonstrates the sharing of local features across them. Furthermore, it reduces model redundancy and addresses issues encountered in the collection and segmentation of multi-view object samples.

[0157] The main contributions of the multi-view object detection method provided by the embodiment of the present invention include:

[0158] (1) Using random forest (GBRF) to improve the traditional Hough forest method of using random forest for local image block clustering;

[0159] (2) Clustering the local image blocks in the leaf nodes of the forest, a new compact description method of visual words is proposed to effectively reduce redundant and invalid voting units;

[0160] (3) Provide a weighted Hough voting detection method;

[0161] (4) Using unsupervised view subclassification and multi-category SVM, a new learning algorithm (calculation method) for multi-view voting combination weights is provided.

[0162] Experimental results show that the multi-view target detection method provided by the embodiment of the present invention significantly improves the accuracy of multi-target vehicle detection. At the same time, the target brightness of the Hough map generated by voting is concentrated, which effectively reduces redundant and messy voting units.

[0163] Based on the same concept as the above multi-view target detection method, in one embodiment, the present invention further provides a multi-view target detection device 100, referring to Figure 8 The multi-view object detection device 100 includes a preprocessing module 110 , a matching module 120 and a voting module 130 .

[0164] The pre-processing module 110 is configured to obtain a plurality of local image blocks of the image to be tested and an image block feature of each local image block.

[0165] The matching module 120 is configured to divide the multiple local image blocks into multiple visual words based on the features of each image block using a preset random forest model.

[0166] Among them, the local image patches of the same visual word have the same appearance features and geometric positions.

[0167] Voting module 130 is configured to calculate the voting score of each visual word for each candidate center position of the image to be tested, obtain the combined voting weight of each candidate center position with respect to each visual word, calculate the total score of each candidate center position based on the combined voting weight and the voting score, and determine the target center position based on the total score of each candidate center position.

[0168] The multi-view target detection device 100 uses a random forest model to divide multiple local image blocks of the image to be tested into multiple visual words, each visual word representing a perspective, so as to vote on the candidate center position from multiple perspectives, thereby reducing the impact of interference such as target occlusion, image deformation, angle change, lighting change and posture change to a certain extent.

[0169] For specific definitions of the multi-view object detection apparatus 100, please refer to the definitions of the multi-view object detection method above and will not be repeated here. Each module in the multi-view object detection apparatus 100 described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each of the modules described above may be embedded in or independent of the processor in the computer device 140 in hardware form, or may be stored in the memory of the computer device 140 in software form, so that the processor can call and execute the corresponding operations of each module.

[0170] In one embodiment, a computer device 140 is provided. The computer device 140 may be a terminal (in other embodiments, it may be a server), and its internal structure diagram may be as follows: Figure 9 As shown. The computer device 140 includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device 140 is used to provide computing and control capabilities. The memory of the computer device 140 includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device 140 is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a multi-view target detection method is implemented. The display screen of the computer device 140 can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device 140 can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device 140, or an external keyboard, touchpad or mouse.

[0171] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 140 to which the solution of the present application is applied. The specific computer device 140 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0172] In one embodiment, the multi-view object detection device 100 provided by the present application can be implemented in the form of a computer program. The computer program can be used in the following ways: Figure 9 The computer device 140 shown in FIG. 1 is executed on the computer device 140. The memory of the computer device 140 may store various program modules constituting the multi-view object detection apparatus 100, such as: Figure 8 The pre-processing module 110, matching module 120 and voting module 130 are shown. The computer program composed of various program modules enables the processor to execute the steps of the multi-view object detection method of each embodiment of the present application described in this specification.

[0173] For example, Figure 9 The computer device 140 shown can be Figure 8 The pre-processing module 110 in the multi-view object detection apparatus 100 shown in FIG. performs step S101. The computer device 140 may perform step S103 through the matching module 120. The computer device 140 may perform steps S105, S107, and S109 through the voting module 130.

[0174] In one embodiment, a computer device 140 is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program: obtaining multiple local image blocks of an image to be tested, and image block features of each local image block; dividing the multiple local image blocks into multiple visual words based on each image block feature using a preset random forest model, wherein the local image blocks of the same visual word have the same appearance features and geometric positions; calculating the voting score of each visual word for each candidate center position of the image to be tested, obtaining the voting combination weight of each candidate center position with respect to each visual word, calculating the total score of each candidate center position based on the voting combination weight and the voting score, and determining the target center position based on the total score of each candidate center position.

[0175] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining a voting mean based on the total score of each candidate center position; scaling the image to be tested at multiple scale factors, and combining the total scores to obtain a Hough map at each scale factor; obtaining a 3D Hough space based on the Hough map at each scale factor; and determining the target center position in the 3D Hough space using a mean-shift algorithm in combination with the voting mean.

[0176] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining multiple local image blocks of an image to be tested and image block features of each local image block; dividing the multiple local image blocks into multiple visual words based on each image block feature using a preset random forest model, wherein the local image blocks of the same visual word have the same appearance features and geometric positions; calculating the voting score of each visual word for each candidate center position of the image to be tested, obtaining the voting combination weight of each candidate center position with respect to each visual word, calculating the total score of each candidate center position based on the voting combination weight and the voting score, and determining the target center position based on the total score of each candidate center position.

[0177] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: obtaining a voting mean based on the total score of each candidate center position; scaling the image to be tested at multiple scale factors, and combining the total scores to obtain a Hough map at each scale factor; obtaining a 3D Hough space based on the Hough map at each scale factor; and determining the target center position in the 3D Hough space using a mean-shift algorithm in combination with the voting mean.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0179] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0180] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device 140 (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0181] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A multi-view target detection method, characterized in that: The method comprises: Acquire multiple local image blocks of the image to be tested, and an image block feature of each of the local image blocks; Based on the features of each of the image blocks, a preset random forest model is used to divide the multiple local image blocks into multiple visual words, wherein the local image blocks of the same visual word have the same appearance features and geometric positions; Calculating a voting score of each visual word for each candidate center position of the image to be tested; Obtaining a combined voting weight for each candidate center position with respect to each visual word, and calculating a total score for each candidate center position based on the combined voting weight and the voting score; Determining a target center position according to the total score of each candidate center position; The total score of each candidate center position is expressed as: , , , Characterize the geometric position of the center of the local image block in the image to be tested, Characterize the candidate center position, Characterizes the class purity of a set of local image patches in a visual word, Represents the mean offset vector of each local image block in the visual word, The unit ratio representing the visual word vote is determined by the number of offset vectors in the visual word and the total number of offset vectors. b is a parameter. represents the exponential function, Returns 1 if true. Returns 0 when false, which is used to constrain the minimum probability of the distribution of positive local image blocks that can participate in voting and the minimum number of offset vectors in visual words. Represents the number of offset vectors in the visual word, Characterize the image to be tested, Representation has local image patches The set of visual words, that is, the set of visual words, Representing visual words with respect to candidate center positions The weight of Representing sets of visual words.

2. The multi-view target detection method according to claim 1, wherein: The step of determining the target center position according to the total score of each candidate center position includes: Obtaining a voting mean according to the total score of each candidate center position; Scaling the image to be tested under multiple scale factors, and combining the total score to obtain a Hough map under each scale factor; According to the Hough graph at each scale factor, a 3D Hough space is obtained; The target center position is determined in the 3D Hough space using a mean shift algorithm in combination with the voting mean.

3. The multi-view target detection method according to claim 1 or 2, characterized in that: The step of obtaining the voting combination weight of each candidate center position with respect to each visual word comprises: Acquire multiple sample images of the verification object and the background at various viewing angles, and divide all the sample images into multiple visual subcategories; A plurality of visual subclasses are obtained from leaf nodes of a preset random forest model, an expression vector for each sample image is generated based on the visual subclasses, and a subclass label is added to each sample image according to the visual subclass to which each sample image belongs; Based on the expression vectors and the subclass labels of all the sample images, a multi-class linear classification algorithm is used to train a score weight for each visual subclass.

4. The multi-view target detection method according to claim 3, characterized in that: The step of dividing all the sample images into a plurality of visual subcategories comprises: Calculating the HOG features of each of the sample images, and embedding all of the HOG features into a 2D space using a local linear embedding algorithm, where the 2D space includes a plurality of sample points, each of which corresponds to one of the sample images; In the 2D space, all the sample images are regularized onto a circle, and unsupervised clustering is performed on the sample images on the circle to obtain a plurality of visual subclasses.

5. The multi-view target detection method according to claim 1 or 2, characterized in that: The step of dividing the plurality of local image blocks into a plurality of visual words using a preset random forest model according to the features of each of the image blocks comprises: For each of the image block features, traverse the random forest model using the image block feature until the image block feature reaches a leaf node of the random forest model, each leaf node corresponding to a visual word; According to the leaf nodes reached by all the image block features, the local image blocks corresponding to all the image block features reaching the same leaf node are regarded as a visual word.

6. The multi-view target detection method according to claim 1 or 2, characterized in that: The method further comprises the step of training a random forest model, the step comprising: Obtaining sample images of the sample object at different viewing angles, dividing each of the sample images into a plurality of local image blocks, and using all the local image blocks as a training set; A random forest model is constructed and trained based on a training algorithm according to the features of each local image block, wherein the features include appearance features, perspective categories, offset vectors from the local image blocks to the target center, and classification weights of the local image blocks at each layer node of the random forest model.

7. The multi-view target detection method according to claim 6, characterized in that: The step of constructing and training a random forest model based on a training algorithm according to the features of each local image block includes: Generating the first layer nodes of each tree of the random forest model according to the training algorithm and the characteristics of each of the local image blocks; When each layer of nodes is generated, the splitting function is obtained according to the set of local image blocks in each node on the current layer; Predicting and updating the classification weight of each of the local image blocks on the lower layer nodes according to the splitting function; Each time a layer of nodes is generated, it is determined whether the current random forest model meets the termination condition. If so, the training is terminated to obtain the trained random forest model. If not, the features after the classification weights are updated according to each of the local image blocks, and the next layer of nodes of each tree of the random forest model is continued to be generated.

8. A multi-view target detection device, characterized in that: Includes preprocessing module, matching module and voting module; The preprocessing module is used to obtain multiple local image blocks of the image to be tested, and the image block features of each local image block; The matching module is configured to divide the plurality of local image blocks into a plurality of visual words based on the features of each of the image blocks using a preset random forest model, wherein the local image blocks of the same visual word have the same appearance features and geometric positions; The voting module is configured to calculate a voting score for each visual word for each candidate center position of the image to be tested, obtain a combined voting weight for each candidate center position with respect to each visual word, calculate a total score for each candidate center position based on the combined voting weight and the voting score, and determine a target center position based on the total score for each candidate center position; The total score of each candidate center position is expressed as: , , , Characterize the geometric position of the center of the local image block in the image to be tested, Characterize the candidate center position, Characterizes the class purity of a set of local image patches in a visual word, Represents the mean offset vector of each local image block in the visual word, The unit ratio representing the visual word vote is determined by the number of offset vectors in the visual word and the total number of offset vectors. b is a parameter. represents the exponential function, Returns 1 if true. Returns 0 when false, which is used to constrain the minimum probability of the distribution of positive local image blocks that can participate in voting and the minimum number of offset vectors in visual words. Represents the number of offset vectors in the visual word, Characterize the image to be tested, Representation has local image patches The set of visual words, that is, the set of visual words, Representing visual words with respect to candidate center positions The weight of Representing sets of visual words.

9. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the multi-view target detection method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-view target detection method according to any one of claims 1 to 7 is implemented.

Images