A target detection method and device

Abstract

The present invention discloses a target detection method and device, the method comprising: obtaining the position information and discrimination probability of different detection objects of the original image, and calculating the occurrence probability of the position relationship between two detection objects corresponding to each detection object position relationship combination, wherein the detection object position relationship combination is composed of the position information of any two different detection objects. It is determined whether the discrimination probability of the two detection objects corresponding to each detection object position relationship combination and the occurrence probability of the position relationship between the two detection objects meet a preset threshold condition, and when the preset threshold condition is met, the position information of the target as a whole corresponding to the detection object position relationship combination is obtained. By adopting the embodiment of the present invention, it is possible to effectively combine the occurrence probability of the position relationship between different detection objects of the target and the discrimination probability of the detection object position information, accurately locate the position of the target as a whole, and thus improve the detection accuracy of the target as a whole.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a target detection method and device. Background Art

[0002] As the demand for target detection technology continues to increase, the accuracy of target detection has received widespread attention. Taking human targets as an example, in order to improve the accuracy, the existing human detection methods not only use pedestrian feature detection, but also use the features of pedestrian body parts, such as half-body features and head features, etc. The classifier is used to detect multiple body parts of pedestrians, and the positional relationship between the body parts and the whole body of the human body is used to screen the human body, and finally the human target detection results are output.

[0003] However, the inventors have discovered that the prior art has at least the following problems: the output scores of the classifiers used in the prior art to detect body parts or the entire human body are different from the physical meaning of the positional relationship of the body parts relative to the entire body, and there is no effective way to merge the two. Judgment can only be made based on human experience, resulting in low accuracy of pedestrian detection results. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a target detection method and device, which can effectively combine the occurrence probability of the position relationship between different detection objects of the target and the discrimination probability of the detection object position information, accurately locate the position of the entire target, thereby improving the detection accuracy of the entire target.

[0005] To achieve the above object, an embodiment of the present invention provides a target detection method, including:

[0006] Acquire the position information and discrimination probability of different detection objects of the original image; wherein the different detection objects include the target as a whole and at least one component constituting the target as a whole;

[0007] According to the position information and discrimination probability of each of the detection objects, all the detection object position relationship combinations are obtained by traversing; wherein the detection object position relationship combination is composed of the position information of any two different detection objects;

[0008] Calculating the occurrence probability of the positional relationship between the two detection objects corresponding to each of the detection object positional relationship combinations according to the pre-constructed probability model;

[0009] Determine whether the discrimination probability of the two detection objects corresponding to each of the detection object position relationship combinations and the occurrence probability of the position relationship between the two detection objects meet a preset threshold condition;

[0010] When a preset threshold condition is met, the position information of the entire target corresponding to the detection object position relationship combination is obtained.

[0011] As an improvement of the above solution, the probability model is constructed in the following way:

[0012] Acquire a training data set; wherein the training data set includes a plurality of sample images with detection object rectangular frames pre-labeled therein, wherein the detection object rectangular frames include a target rectangular frame corresponding to the target as a whole, and a sub-target rectangular frame corresponding to at least one component of the target as a whole;

[0013] Normalizing the sizes of all images within the target rectangular frame according to a preset scale;

[0014] Determine the position information of the target rectangular frame and its corresponding sub-target rectangular frame according to the normalized target rectangular frame;

[0015] According to the position information of any two detection object rectangular frames in each sample image, the occurrence probability of the position relationship between any two detection objects in the training data set is calculated to construct the probability model.

[0016] As an improvement of the above solution, the calculation of the occurrence probability of the positional relationship between any two detection objects in the training data set according to the position information of any two detection object rectangular frames in each sample image specifically includes:

[0017] According to the position information of the nth detection object rectangular frame and the mth detection object rectangular frame in the same sample image, the occurrence probability of the position relationship between the two detection objects corresponding to the nth detection object rectangular frame and the mth detection object rectangular frame is calculated by the following calculation formula:

[0018]

[0019] q=(x n,k -x m,k ,y n,k -y m,k );

[0020] Among them, (x n,k ,y n,k ) is the position information of the rectangular frame of the nth detection object, (x m,k ,y m,k ) is the position information of the mth detection object rectangular frame, K is the number of sample images in the training data set; K q,m,nis the number of sample images in the training data set where the nth detection object rectangular box appears at position q relative to the mth detection object rectangular box, n = 1, 2, ..., C; m = 1, 2, ..., C; and n ≠ m; C is the number of detection object rectangular boxes marked in the sample images.

[0021] As an improvement of the above scheme, the preset threshold condition is: at least one of the discrimination probabilities of the two detection objects corresponding to the detection object position relationship combination is greater than a preset first probability threshold, and the probability of occurrence of the position relationship between the two detection objects is greater than a preset second probability threshold, and the sum of the discrimination probability of the two detection objects and the probability of occurrence of the position relationship between the two detection objects is greater than a preset third probability threshold.

[0022] As an improvement of the above solution, when a preset threshold condition is met, obtaining the overall position information of the target corresponding to the position relationship combination of the detection objects specifically includes:

[0023] When the preset threshold conditions are met,

[0024] If one of the two detection objects corresponding to the detection object position relationship combination is the target as a whole, obtaining position information of the target as a whole;

[0025] If none of the two detection objects corresponding to the detection object position relationship combination is the target as a whole, the position information of the target as a whole is calculated according to the position relationship between the two detection objects.

[0026] As an improvement of the above solution, the acquisition of position information and discrimination probability of different detection objects in the original image specifically includes:

[0027] Acquire an original image, and perform feature extraction on the original image;

[0028] The features of the original image are respectively input into different pre-trained classifiers to obtain the position information and discrimination probability of the detection object detected by each classifier; wherein different classifiers are used to detect the position information and discrimination probability of different detection objects.

[0029] As an improvement of the above solution, the classifier for detecting the target as a whole is trained in the following way:

[0030] Acquire a training data set; wherein the training data set includes a plurality of sample images with detection object rectangular frames pre-labeled therein, wherein the detection object rectangular frames include a target rectangular frame corresponding to the entire target;

[0031] Normalizing the sizes of all images within the target rectangular frame according to a preset scale;

[0032] Feature extraction is performed on the normalized image within the target rectangular frame, and a classifier is trained using the extracted features to obtain the classifier for detecting the target as a whole.

[0033] As an improvement of the above solution, the detection object rectangular frame also includes a sub-target rectangular frame corresponding to at least one component of the entire target;

[0034] Then, the classifier for detecting components of the same category is trained as follows:

[0035] According to the normalized target rectangular frame, obtaining the size of the sub-target rectangular frame corresponding to the target rectangular frame;

[0036] According to the sizes of the sub-target rectangular boxes corresponding to all components of the same category in the training data set, the normalized scale sizes of the components of the same category are calculated, and the sizes of the images within the sub-target rectangular boxes corresponding to all components of the same category are normalized according to the normalized scale sizes;

[0037] Feature extraction is performed on the normalized image within the sub-target rectangular frame, and a classifier is trained using the extracted features to obtain the classifier for detecting components of the same category.

[0038] As an improvement of the above solution, after obtaining the overall position information of the target corresponding to the position relationship combination of the detection objects when the preset threshold condition is met, the method further includes:

[0039] The acquired location information of all the target entities is screened to obtain the location information of the target entity that meets the preset conditions.

[0040] As an improvement of the above solution, the obtained location information of all the targets is screened to obtain the location information of the targets that meet the preset conditions, which specifically includes:

[0041] A non-maximum suppression processing method is adopted to screen the position information of all the detected targets as a whole, so as to obtain the position information of the target as a whole that meets the preset conditions.

[0042] As an improvement of the above solution, before obtaining the position information and discrimination probability of different detection objects of the original image, it also includes:

[0043] Acquire an original image, and scale the original image to different scales;

[0044] Then, for the original images of different scales, the following steps are performed in sequence: obtaining the position information and discrimination probability of different detection objects of the original images.

[0045] The embodiment of the present invention further provides a target detection device, comprising:

[0046] A detection object detection module, used to obtain position information and discrimination probability of different detection objects in the original image; wherein the different detection objects include the target as a whole and at least one component constituting the target as a whole;

[0047] A position relationship combination acquisition module, used to traverse and obtain all the detection object position relationship combinations according to the position information and discrimination probability of each detection object; wherein the detection object position relationship combination is composed of the position information of any two different detection objects;

[0048] A probability calculation module, used to calculate the occurrence probability of the position relationship between the two detection objects corresponding to each of the detection object position relationship combinations according to a pre-built probability model;

[0049] A threshold condition judgment module, used to judge whether the discrimination probability of the two detection objects corresponding to each combination of the detection object position relationship and the occurrence probability of the position relationship between the two detection objects meet a preset threshold condition;

[0050] The target position acquisition module is used to acquire the position information of the entire target corresponding to the position relationship combination of the detection objects when a preset threshold condition is met.

[0051] An embodiment of the present invention also provides a target detection device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements any of the target detection methods described above when executing the computer program.

[0052] Compared with the prior art, the target detection method and device disclosed in the present invention obtains the position information and discrimination probability of different detection objects in the original image, and calculates the occurrence probability of the position relationship between the two detection objects corresponding to each combination of the detection object position relationship, and determines whether the discrimination probability of the two detection objects corresponding to each combination of the detection object position relationship and the occurrence probability of the position relationship between the two detection objects meet the preset threshold conditions. When the preset threshold conditions are met, the position information of the target as a whole corresponding to the combination of the detection object position relationship is obtained. In the process of target detection, the embodiment of the present invention takes into account the position relationship between different detection objects, and characterizes the position relationship between different detection objects in a probabilistic manner. Since the position relationship between different detection objects is characterized as a probability, the position information of the detection object is also characterized as a probability, which makes it easier to interpret the two probabilities and threshold judgments. The position information of the target as a whole is detected in combination with the discrimination probability and the occurrence probability of the detection object, and the position of the target as a whole can be more accurately located, thereby improving the detection accuracy of the target as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a schematic diagram of the steps of a target detection method provided in Embodiment 1 of the present invention;

[0054] Figure 2 Schematic diagram of the training steps of the probability model provided in the second embodiment of the present invention;

[0055] Figure 3 Schematic diagram of the training steps of a classifier for detecting an entire target provided by the third embodiment of the present invention;

[0056] Figure 4 It is a schematic diagram of the training steps of a classifier for detecting components of an entire target provided by the third embodiment of the present invention;

[0057] Figure 5 is a schematic diagram of the steps of another target detection method provided by Embodiment 4 of the present invention;

[0058] Figure 6 is a schematic diagram of the structure of a target detection device provided in Embodiment 5 of the present invention;

[0059] Figure 7 It is a structural schematic diagram of another target detection device provided in Example 6 of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1 , is a schematic diagram of the steps of a target detection method provided by the first embodiment of the present invention. The first embodiment of the present invention provides a target detection method, which is specifically performed through steps S11 to S15:

[0062] S11, obtaining position information and discrimination probability of different detection objects of the original image; wherein the different detection objects include the target as a whole and at least one component constituting the target as a whole.

[0063] It should be noted that the original image contains at least one target as a whole, and the target as a whole is composed of a plurality of components. The plurality of components into which the target as a whole is divided can be pre-set according to the type and structure of the target as a whole.

[0064] Exemplarily, taking a pedestrian as an example, the pedestrian target is the target as a whole to be detected and ultimately output by this method, and the components constituting the target as a whole may include: head, left arm, right arm, body, left leg and right leg.

[0065] It can be understood that the above scenarios are only examples. In order to improve the detection accuracy, the body can be further divided into components such as left shoulder, right shoulder, left hand, right hand, left foot, and right foot. No specific limitation is made here.

[0066] For example, taking a car as the target entity, the car target is the target entity to be detected and ultimately output by the method, and the components constituting the target entity may include: wheels, windows, doors, and the like.

[0067] It is understandable that the above scenario is only an example. In actual application, the components of the car can be divided more finely according to the detection requirements, and no specific limitation is made here.

[0068] In the embodiment of the present invention, each detection object (the entire target and its components) of the original image is detected and calculated to obtain multiple position information q and discrimination probability P corresponding to each detection object. cls The position information refers to the location of the detection object in the original image, such as coordinate information. The discrimination probability refers to the probability value that the detection object is located at a certain position in the original image.

[0069] Generally, when the probability value that the detection object is located at a certain position in the original image is greater than a preset threshold, such as 75%, the position information is considered to be the determined position of the detection object.

[0070] It should be noted that the method for calculating the position information and discrimination probability of the detection object can adopt the detection method in the prior art, such as using a pre-trained neural network model for detecting position information and discrimination probability, etc., which is not specifically limited here.

[0071] For the convenience of explanation, here we take pedestrians as the target as an example, and obtain the probability of pedestrians being located at position A as P cls 1. The probability of being at position B is P cls 2. The probability of being at position C is P cls 3; and at least one component of a pedestrian target, such as a head and a body, where the probability of the head being located at position D is P cls 4. The probability of being at position E is P cls 5; The probability of the body being at position F is P cls 6. The probability of being at position G is P cls 7 and so on.

[0072] S12. According to the position information and discrimination probability of each of the detection objects, traverse to obtain all the detection object position relationship combinations; wherein the detection object position relationship combination is composed of the position information of any two different detection objects.

[0073] According to the multiple pieces of position information and corresponding discrimination probabilities of each detection object obtained in step S11, a certain piece of position information of any two different detection objects is used to form the detection object position relationship combination.

[0074] For example, a pedestrian target and the pedestrian's head are taken as the two different detection objects, and the obtained detection object position relationship combinations are: six combinations of position A and position D, position A and position E, position B and position D, position B and position E, position C and position D, position C and position E. Similarly, the detection object position relationship combination corresponding to taking the pedestrian target and the pedestrian's body as the two different detection objects, and the detection object position relationship combination corresponding to taking the pedestrian target's head and body as the two different detection objects are obtained.

[0075] S13. Calculate the occurrence probability of the positional relationship between the two detection objects corresponding to each combination of the detection object positional relationship according to the pre-constructed probability model.

[0076] A probability model is pre-built to calculate the occurrence probability P of the position relationship between two detection objects corresponding to the position relationship combination of the detection objects. pos The occurrence probability P pos It is used to characterize the probability that the first detection object appears at position q relative to the second detection object in the detection object position relationship combination. The position q refers to the relative position between the two detection objects.

[0077] Generally speaking, the higher the probability of the positional relationship between two detection objects, the more normal the relative positional relationship between the two detection objects is. Then, the corresponding position information of the target as a whole constituted by the two detection objects is closer to the final target position to be detected.

[0078] Specifically, the position information of two detection objects in the detection object position relationship combination is input into the probability model for calculation and analysis to obtain the occurrence probability of the position relationship between the two detection objects.

[0079] S14: Determine whether the discrimination probability of the two detection objects corresponding to each combination of the detection object position relationship and the occurrence probability of the position relationship between the two detection objects meet a preset threshold condition.

[0080] Specifically, the discrimination probability P of the detection object obtained according to the previous step cls , and the probability of occurrence P of the positional relationship between two different detection objects pos , perform corresponding threshold condition judgment, so as to obtain a detection object position relationship combination that meets the threshold condition, and then determine the position information of the target as a whole corresponding to the detection object position relationship combination, so as to locate the determined position of the target as a whole to be detected on the original image.

[0081] Preferably, the preset threshold condition is: at least one of the discrimination probabilities of the two detection objects corresponding to the detection object position relationship combination is greater than a preset first probability threshold, and the probability of occurrence of the position relationship between the two detection objects is greater than a preset second probability threshold, and the sum of the discrimination probability of the two detection objects and the probability of occurrence of the position relationship between the two detection objects is greater than a preset third probability threshold.

[0082] In the embodiment of the present invention, each of the detection object relationship combinations is traversed to obtain the discrimination probability of the two detection objects i and j and the occurrence probability of the position relationship between the two detection objects, which is recorded as P cls i(pi),P cls j(pj),P pos (pj-pi,i,j), P obtained by taking all i and j values cls i(pi),Pcls j(pj),P pos (pj-pi,i,j) is recorded as set R, and the target is classified according to set R. This operation is defined as op, and the output is 1 if the target is classified as a whole to be detected, otherwise it is 0. Among them, pi is the position information of the i-th detection object, and pj is the position information of the j-th detection object.

[0083] For the convenience of explanation, the 0th detection object is recorded as the target itself. When op(R) is 1, p0 is returned as the position of the target.

[0084] Here, the pedestrian target is the 0th detection object and the head is the 1st detection object, then:

[0085] op(R)=op(Pa,Pb,Pc)=op(P cls 0(p0),P cls 1(p1),P pos (p1-p0,0,1));

[0086] When op(R) satisfies: Pa or Pb is greater than or lower than the probability threshold P1_thresh, and Pc is greater than the second probability threshold P2_thresh, and Pa+Pb+Pc is greater than the third probability threshold P3_thresh, it indicates that the detection object relationship combination meets the preset threshold condition.

[0087] S15. When a preset threshold condition is met, the position information of the entire target corresponding to the detection object position relationship combination is obtained.

[0088] Specifically, when there is a certain detection object relationship combination that satisfies the preset threshold condition, if one of the two detection objects corresponding to the detection object position relationship combination is the target as a whole, the position information of the target as a whole is obtained; if none of the two detection objects corresponding to the detection object position relationship combination is the target as a whole, the position information of the target as a whole is calculated according to the position relationship between the two detection objects.

[0089] As an example, when the detection object relationship combination includes the pedestrian target as a whole, its corresponding position information p0 is the target position finally detected; when the detection object relationship combination does not include the pedestrian target as a whole, such as the head and body, the position information of the pedestrian target is determined according to the corresponding position information of the head and body, and the target position finally detected is obtained.

[0090] Embodiment 1 of the present invention provides a target detection method, which obtains the position information and discrimination probability of different detection objects in the original image, and calculates the occurrence probability of the position relationship between the two detection objects corresponding to each combination of the detection object position relationship, and determines whether the discrimination probability of the two detection objects corresponding to each combination of the detection object position relationship and the occurrence probability of the position relationship between the two detection objects meet the preset threshold condition. When the preset threshold condition is met, the position information of the target as a whole corresponding to the detection object position relationship combination is obtained. In the process of target detection, the embodiment of the present invention takes into account the position relationship between different detection objects, and characterizes the position relationship between different detection objects in a probabilistic manner. Since the position relationship between different detection objects is characterized as a probability, the position information of the detection object is also characterized as a probability, which makes it easier to interpret the two probabilities and threshold judgment. The position information of the target as a whole is detected in combination with the discrimination probability and the occurrence probability of the detection object, so that the position of the target as a whole can be more accurately located, thereby improving the detection accuracy of the target as a whole.

[0091] See also Figure 2 , is a schematic diagram of the training steps of the probability model provided in the second embodiment of the present invention. The second embodiment of the present invention provides a target detection method, which is further implemented on the basis of the first embodiment. In the embodiment of the present invention, the probability model in step S13 is constructed by the following steps S21 to S24:

[0092] S21. Obtain a training data set; wherein the training data set includes a number of sample images that have been pre-labeled with detection object rectangular frames, and the detection object rectangular frames include a target rectangular frame corresponding to the target as a whole, and a sub-target rectangular frame corresponding to at least one component of the target as a whole.

[0093] S22: Normalize the sizes of all images within the target rectangular frame according to a preset scale.

[0094] S23. Determine position information of the target rectangular frame and its corresponding sub-target rectangular frame according to the normalized target rectangular frame.

[0095] S24. Calculate the occurrence probability of the positional relationship between any two detection objects in the training data set according to the position information of any two detection object rectangular frames in each sample image, so as to construct the probability model.

[0096] Specifically, a plurality of sample images containing a target are obtained, and rectangular frames of detection objects are marked on the sample images, including a target rectangular frame corresponding to the target as a whole and a sub-target rectangular frame corresponding to at least one component constituting the target as a whole.

[0097] Taking pedestrians as the overall target as an example, the several sample images all include pedestrian targets, and a target rectangular frame corresponding to the pedestrian target and a sub-target rectangular frame corresponding to at least one component of the pedestrian, such as the head, left arm, right arm, body, left leg and right leg, are marked on each sample image.

[0098] Furthermore, all images in the target rectangular frame in the plurality of sample images are normalized, and the width and height of the image in the target rectangular frame after size normalization are (normal_width, normal_height).

[0099] It can be understood that since the size of the entire target is normalized, the size of the rectangular frame of each component corresponding to the entire target, that is, the size of the sub-target rectangular frame, also changes accordingly. After normalization, the width and height of the i-th sub-target rectangular frame in the k-th sample image are marked as (part_width i,k ,part_height i,k ), the center position is marked as (x i,k ,y i,k ). Where k∈K, K is the number of sample images, i∈C, C is the number of different types of detection objects, that is, the number of components corresponding to a target as a whole plus the total number of the target as a whole. For example, when the components of a pedestrian target are set to be head, left arm, right arm, body, left leg and right leg, the value of C is 7.

[0100] Furthermore, according to the position information of the nth detection object rectangular frame and the mth detection object rectangular frame in the same sample image, the occurrence probability of the position relationship between the two detection objects corresponding to the nth detection object rectangular frame and the mth detection object rectangular frame is calculated by the following calculation formula:

[0101]

[0102] q=(x n,k -x m,k ,y n,k -y m,k );

[0103] Among them, (x n,k ,y n,k ) is the position information of the rectangular frame of the nth detection object, (x m,k ,y m,k ) is the position information of the mth detection object rectangular frame, K is the number of sample images in the training data set; K q,m,nis the number of sample images in the training data set where the nth detection object rectangular box appears at position q relative to the mth detection object rectangular box, n = 1, 2, ..., C; m = 1, 2, ..., C; and n ≠ m; C is the number of detection object rectangular boxes marked in the sample images.

[0104] For any two detection objects in each sample image, the above method is used to calculate the occurrence probability of the relative position relationship, and the relative position relationship q and the occurrence probability P between different detection objects are established. pos , thereby constructing the probability model.

[0105] Therefore, in the application stage, that is, in step S13, the probability model is used to calculate the relative position relationship (position q) between the two detection objects according to the position information of the two detection objects in the detection object relationship combination, and the occurrence probability of the corresponding position relationship between the two detection objects can be obtained.

[0106] By adopting the technical means of the embodiment of the present invention, the probability model is trained by acquiring a number of labeled sample images, and the training process is simple and effective.

[0107] As a preferred implementation, the third embodiment of the present invention provides a target detection method, which is further implemented on the basis of the first embodiment or the second embodiment. In the embodiment of the present invention, the step S11, that is, obtaining the position information and discrimination probability of different detection objects in the original image, is specifically performed through steps S111 to S112:

[0108] S111, acquiring an original image, and performing feature extraction on the original image.

[0109] S112, inputting the features of the original image into different pre-trained classifiers respectively to obtain the position information and discrimination probability of the detection object detected by each classifier; wherein different classifiers are used to detect the position information and discrimination probability of different detection objects.

[0110] Specifically, according to each pixel value on the original image, a feature map is calculated for the original image to obtain a feature map to be detected of the original image. Feature extraction is performed on the feature map to obtain a feature vector of the feature map, and then the feature vectors are respectively input into different pre-trained classifiers to obtain the position information and discrimination probability of the detection object detected by each classifier.

[0111] The different classifiers are pre-trained and used to detect different detection objects. The classifier corresponding to a detection object can output the location information and the corresponding discrimination probability of the detection object by analyzing and calculating the input feature vector.

[0112] As a preferred embodiment, see Figure 3 , is a schematic diagram of the training steps of a classifier for detecting an entire target provided by Embodiment 3 of the present invention. The classifier C0 for detecting the entire target is trained by the following steps S31 to S33:

[0113] S31, obtaining a training data set; wherein the training data set includes a plurality of sample images with detection object rectangular frames pre-labeled therein, and the detection object rectangular frames include a target rectangular frame corresponding to the entire target;

[0114] S32, normalizing the sizes of all images within the target rectangular frame according to a preset scale;

[0115] S33, extracting features from the normalized image within the target rectangular frame, and using the extracted features to train a classifier to obtain the classifier for detecting the target as a whole.

[0116] Specifically, a plurality of sample images containing a target are obtained, and rectangular frames of detection objects are marked on the plurality of sample images, including a target rectangular frame corresponding to the entire target.

[0117] Taking pedestrians as the overall target as an example, the several sample images all include pedestrian targets, and a target rectangular frame corresponding to the pedestrian target is marked on each sample image. The images in all target rectangular frames in the several sample images are normalized, and the width and height of the image in the target rectangular frame after size normalization are (normal_width, normal_height).

[0118] As an example, the size of the target rectangular frame is scaled to the same size of 32×32.

[0119] Furthermore, according to each pixel value on the normalized image within the target rectangular frame, a feature map of the image within the target rectangular frame is calculated, and then features are extracted from the feature map to obtain a feature vector of the feature map. Then, the feature vectors are respectively input into preset classifiers to train the classifiers.

[0120] Preferably, in order to improve the training accuracy of the classifier, the image within the target rectangular frame can be used as a positive sample image, and several image regions that do not contain the entire target can be obtained as negative sample images. The feature vector of each negative sample image is calculated and input into the classifier together, thereby realizing the training of the classifier.

[0121] Exemplarily, a neural network can be used as a classifier, and a loss function is calculated based on the predicted value output by the classifier and the actual rectangular box position, and the parameters of the neural network are updated using a gradient descent optimization algorithm to reduce the loss function until the loss function is minimized, thereby obtaining a trained classifier. The activation function of the last layer of the neural network is a sigmoid function, and the loss function is logistic loss, denoted as H(y,y'). The mathematical expressions of the sigmoid function and logistic loss are as follows:

[0122]

[0123] H(y,y′)=-y·log(y′)-(1-y)·log(1-y′);

[0124] Among them, y is the actual label, y' is the predicted value of the label, and x is the input of the last layer of the neural network. After the training of the neural network is completed, the output of the neural network, i.e., y', can be used as the discrimination probability of the classifier.

[0125] Further, see Figure 4 , is a schematic diagram of the training steps of a classifier for detecting the components of a target as a whole provided by Embodiment 3 of the present invention. The detection object rectangular frame also includes a sub-target rectangular frame corresponding to at least one component of the target as a whole. For example, in the above scenario where a pedestrian is used as the target as a whole, it also includes a sub-target rectangular frame corresponding to at least one component constituting the pedestrian, such as the head, left arm, right arm, body, left leg and right leg.

[0126] Then, the classifier Ci for detecting components of the same category is trained by the following steps S31' to S33':

[0127] S31′, according to the normalized target rectangular frame, obtaining the size of the sub-target rectangular frame corresponding to the target rectangular frame.

[0128] S32', calculate the normalized scale size of the components of the same category according to the size of the sub-target rectangular boxes corresponding to the components of the same category in the training data set, and normalize the size of the images within the sub-target rectangular boxes corresponding to the components of the same category according to the normalized scale size.

[0129] S33', extracting features from the normalized image within the sub-target rectangular frame, and using the extracted features to train a classifier to obtain the classifier for detecting components of the same category.

[0130] It can be understood that since the size of the entire target is normalized, the size of the rectangular frame of each component corresponding to the entire target, that is, the size of the sub-target rectangular frame, also changes accordingly. After normalization, the width and height of the i-th sub-target rectangular frame in the k-th sample image are marked as (part_width i,k ,part_height i,k ), the center position is marked as (x i,k ,y i,k ). Wherein, k∈K, K is the number of sample images, i∈C, C is the number of different types of detection objects.

[0131] Furthermore, the coordinates of the center position of the sub-target rectangular box relative to the target rectangular box, as well as the corresponding width and height, are calculated. The width and height of the sub-target rectangular boxes corresponding to the components of the same category in the training set (for example, the rectangular boxes corresponding to all heads, or the rectangular boxes corresponding to all bodies, etc.) are averaged and rounded to the normalized scale size of the sub-target rectangular box. The width and height corresponding to the normalized scale size are calculated as follows:

[0132]

[0133]

[0134] Among them, ceil represents the rounding operation.

[0135] Furthermore, feature extraction is performed on the normalized image within the sub-target rectangular frame, and classifier training is performed using the features of the sub-target rectangular frame corresponding to the components of the same category to obtain the classifier for detecting the components of the same category.

[0136] It can be understood that the training process of the classifier used to detect components of the same category is similar to the training process of the classifier used to detect the target as a whole, and will not be described in detail here.

[0137] By adopting the technical means of the embodiments of the present invention and performing different classifier training for different detection objects, including the target as a whole and multiple components, the position information of the detection object can be obtained more accurately, thereby improving the subsequent detection accuracy of the target as a whole.

[0138] As a preferred implementation, the embodiment of the present invention is further implemented on the basis of the embodiment 1. After step S15, the target detection method further includes step S16:

[0139] S16: Screening all the acquired location information of the target as a whole to obtain the location information of the target as a whole that meets the preset conditions.

[0140] Preferably, in order to improve the accuracy of positioning the target object, a non-maximum suppression processing method is used to screen all the acquired position information of the target as a whole to obtain the position information of the target as a whole that meets a preset threshold condition.

[0141] Specifically, all the target rectangular frames corresponding to the target as a whole that have been obtained are sorted in descending order according to the scores; the target frames sorted in descending order are traversed in turn. The currently traversed target frame is intersected with the target frames remaining than the current one to obtain a corresponding intersection point set, and the intersection-over-union ratio (IOU) of each two target frames is calculated based on the area of ​​the convex polygon composed of the intersection point set; the target frames whose IOU is greater than a preset threshold are filtered out, and the target frames whose IOU is less than the set threshold are retained; thereby obtaining the final target frame that meets the preset threshold condition.

[0142] Since the target classifier performs classification calculations based on the input feature vector, after the target detection operation of steps S11 to S15 is performed on the original image, when the discrimination probability meets the threshold condition, it is determined that the corresponding target frame exists and is output, and at this time, the position information of more than one target as a whole may be obtained. In this case, there may be a situation where the target object in the target frame output by the target classifier is incomplete or not representative enough. Therefore, in the embodiment of the present invention, all the target frames output by the target classifier are screened using a non-maximum suppression processing method, so as to obtain a target frame that meets the preset threshold condition as the target frame that can ultimately accurately locate the target object.

[0143] In other implementations, other post-processing methods may be used to process the obtained target frame of the entire target, for example, two adjacent complementary target frames may be spliced ​​to obtain a target frame including the complete target, etc., so as to further improve the accuracy of target detection. No specific limitation is made here.

[0144] By adopting the technical means of the embodiment of the present invention, the detection accuracy of the entire target can be effectively improved by further screening the position information of all the entire targets obtained after the threshold condition is met.

[0145] See also Figure 5 , is a schematic diagram of the steps of another target detection method provided by the fourth embodiment of the present invention. The fourth embodiment of the present invention is further implemented on the basis of the above-mentioned third embodiment, and the target detection method includes steps S41 to S46:

[0146] S41, obtaining an original image, and scaling the original image to different scales;

[0147] S42, sequentially executing the following steps for the original images of different scales: obtaining position information and discrimination probability of different detection objects of the original images; wherein the different detection objects include the target as a whole and at least one component constituting the target as a whole;

[0148] S43, according to the position information and the discrimination probability of each of the detection objects, traverse to obtain all the detection object position relationship combinations; wherein the detection object position relationship combination is composed of the position information of any two different detection objects;

[0149] S44, calculating the occurrence probability of the positional relationship between the two detection objects corresponding to each of the detection object positional relationship combinations according to the pre-built probability model;

[0150] S45, determining whether the discrimination probability of the two detection objects corresponding to each of the detection object position relationship combinations and the occurrence probability of the position relationship between the two detection objects meet a preset threshold condition;

[0151] S46. When a preset threshold condition is met, the position information of the entire target corresponding to the detection object position relationship combination is obtained.

[0152] In the embodiment of the present invention, the classifier C0 for detecting the position information and discrimination probability of the entire target obtained by the training method of the third embodiment is only suitable for detecting a detection object of one scale, such as the entire target of 32×32 size mentioned in the above scenario. Based on this, in order to be able to comprehensively detect target objects of different scales on the original image, it is necessary to perform multi-scale detection on the original image.

[0153] In one implementation, as described in Example 4 of the present invention, multiple different scales are pre-set, the original image is scaled to the multiple scales to obtain original images of different scales, and the target detection method provided in Example 1 is executed for the original image of each scale.

[0154] As an example, by reducing the original image, the target as a whole having a scale of 32×64 pixels on the original image is scaled to a target as a whole having a scale of 16×32 pixels. Thus, after detection using the classifier C0 trained in the above-mentioned third embodiment and the classifier Ci of the corresponding component, the position information corresponding to the target as a whole is finally obtained by further combining the determination of the probability of occurrence of the positional relationship.

[0155] In another implementation, in order to reduce the amount of calculation, the feature map of the original image can be directly obtained, and then the feature map of the original image can be scaled to different scales and the scaled details can be processed. Furthermore, feature extraction is performed on the feature maps of the original images of different scales in turn, and the classifier C0 trained in the above embodiment 3 and the classifier Ci of the corresponding component are input for detection, thereby realizing multi-scale detection of the original image.

[0156] In another embodiment, classifiers C0 and Ci corresponding to different scales may be pre-trained. For example, classifiers C0 for detecting the entire target with a scale of 32×64 pixels and the entire target with a scale of 16×32 pixels are trained respectively, as well as classifiers Ci at the corresponding scales. In the target detection process, classifiers of different scales are directly used to perform target detection on the original image, thereby achieving multi-scale detection of the original image.

[0157] It can be understood that the above implementation is only an example. In actual applications, the above method can be used to perform target detection of different scales on the original image according to actual conditions, or other processing methods can be used to perform multi-scale detection on the original image, which will not affect the beneficial effects achieved by the present invention.

[0158] By adopting the technical means of the embodiments of the present invention, multi-scale target detection is performed on the original image, so as to more accurately locate the overall position information of all targets and improve the detection accuracy of the targets.

[0159] See also Figure 6 , is a schematic diagram of the structure of a target detection device provided by the fifth embodiment of the present invention. The fifth embodiment of the present invention provides a target detection device 50, including a detection object detection module 51, a position relationship combination acquisition module 52, a probability calculation module 53, a threshold condition judgment module 54 and a target position acquisition module 55; wherein,

[0160] The detection object detection module 51 is used to obtain the position information and discrimination probability of different detection objects in the original image; wherein the different detection objects include the target as a whole and at least one component constituting the target as a whole;

[0161] The position relationship combination acquisition module 52 is used to traverse and obtain all the detection object position relationship combinations according to the position information and discrimination probability of each detection object; wherein the detection object position relationship combination is composed of the position information of any two different detection objects;

[0162] The probability calculation module 53 is used to calculate the occurrence probability of the position relationship between the two detection objects corresponding to each detection object position relationship combination according to the pre-built probability model;

[0163] The threshold condition judgment module 54 is used to judge whether the discrimination probability of the two detection objects corresponding to each combination of the detection object position relationship and the occurrence probability of the position relationship between the two detection objects meet the preset threshold condition;

[0164] The target position acquisition module 55 is used to acquire the position information of the entire target corresponding to the detection object position relationship combination when a preset threshold condition is met.

[0165] It should be noted that the target detection device provided in the fifth embodiment of the present invention is used to execute all the process steps of the target detection method provided in any one of the above-mentioned embodiments one to four. The working principles and beneficial effects of the two correspond one to one, and thus will not be described in detail.

[0166] See also Figure 7 , is a schematic diagram of the structure of another target detection device provided by Embodiment 6 of the present invention. Embodiment 6 of the present invention provides a target detection device 60, comprising a processor 61, a memory 62, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the target detection method provided by any one of Embodiments 1 to 4 above is implemented.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0168] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A target detection method, characterized in that: include: Acquire the position information and discrimination probability of different detection objects of the original image; wherein the different detection objects include the target as a whole and at least one component constituting the target as a whole; According to the position information and discrimination probability of each of the detection objects, all the detection object position relationship combinations are obtained by traversing; wherein the detection object position relationship combination is composed of the position information of any two different detection objects; Calculating the occurrence probability of the positional relationship between the two detection objects corresponding to each of the detection object positional relationship combinations according to the pre-constructed probability model; Determine whether the discrimination probability of the two detection objects corresponding to each of the detection object position relationship combinations and the occurrence probability of the position relationship between the two detection objects meet a preset threshold condition; When a preset threshold condition is met, obtaining the position information of the entire target corresponding to the position relationship combination of the detection objects; The calculating, based on the pre-built probability model, the occurrence probability of the positional relationship between the two detection objects corresponding to each of the detection object positional relationship combinations includes: The position information of the two detection objects in each detection object position relationship combination is input into the probability model for calculation and analysis to obtain the occurrence probability of the position relationship between the two detection objects; wherein the probability model is used to calculate the occurrence probability of the position relationship between the two detection objects corresponding to the detection object position relationship combination; the occurrence probability is used to characterize the probability of the first detection object appearing in a relative position relative to the second detection object in the detection object position relationship combination, and the relative position refers to the relative position between the two detection objects.

2. The target detection method according to claim 1, characterized in that: The probability model is constructed in the following way: Acquire a training data set; wherein the training data set includes a plurality of sample images with detection object rectangular frames pre-labeled therein, wherein the detection object rectangular frames include a target rectangular frame corresponding to the target as a whole, and a sub-target rectangular frame corresponding to at least one component of the target as a whole; Normalizing the sizes of all images within the target rectangular frame according to a preset scale; Determine the position information of the target rectangular frame and its corresponding sub-target rectangular frame according to the normalized target rectangular frame; According to the position information of any two detection object rectangular frames in each sample image, the occurrence probability of the position relationship between any two detection objects in the training data set is calculated to construct the probability model.

3. The target detection method according to claim 2, characterized in that: The calculating the occurrence probability of the positional relationship between any two detection objects in the training data set according to the position information of any two detection object rectangular frames in each sample image specifically includes: According to the position information of the nth detection object rectangular frame and the mth detection object rectangular frame in the same sample image, the occurrence probability of the position relationship between the two detection objects corresponding to the nth detection object rectangular frame and the mth detection object rectangular frame is calculated by the following calculation formula: q=(x n,k -x m,k ,and n,k -and m,k ); Among them, (x n,k ,y n,k ) is the position information of the rectangular frame of the nth detection object, (x m,k ,y m,k ) is the position information of the mth detection object rectangular frame, K is the number of sample images in the training data set; K q,m,n is the number of sample images in the training data set where the nth detection object rectangular box appears at position q relative to the mth detection object rectangular box, n = 1, 2, ..., C; m = 1, 2, ..., C; and n ≠ m; C is the number of detection object rectangular boxes marked in the sample images.

4. The target detection method according to claim 1, characterized in that: The preset threshold condition is: at least one of the discrimination probabilities of the two detection objects corresponding to the detection object position relationship combination is greater than a preset first probability threshold, and the probability of the occurrence of the position relationship between the two detection objects is greater than a preset second probability threshold, and the sum of the discrimination probability of the two detection objects and the probability of the occurrence of the position relationship between the two detection objects is greater than a preset third probability threshold.

5. The target detection method according to claim 1, characterized in that: When the preset threshold condition is met, obtaining the overall position information of the target corresponding to the position relationship combination of the detection objects specifically includes: When the preset threshold conditions are met, If one of the two detection objects corresponding to the detection object position relationship combination is the target as a whole, obtaining position information of the target as a whole; If none of the two detection objects corresponding to the detection object position relationship combination is the target as a whole, the position information of the target as a whole is calculated according to the position relationship between the two detection objects.

6. The target detection method according to claim 1, characterized in that: The obtaining of the position information and discrimination probability of different detection objects of the original image specifically includes: Acquire an original image, and perform feature extraction on the original image; The features of the original image are respectively input into different pre-trained classifiers to obtain the position information and discrimination probability of the detection object detected by each classifier; wherein different classifiers are used to detect the position information and discrimination probability of different detection objects.

7. The target detection method according to claim 6, characterized in that: The classifier for detecting the target as a whole is trained in the following way: Acquire a training data set; wherein the training data set includes a plurality of sample images with detection object rectangular frames pre-labeled therein, wherein the detection object rectangular frames include a target rectangular frame corresponding to the entire target; Normalizing the sizes of all images within the target rectangular frame according to a preset scale; Feature extraction is performed on the normalized image within the target rectangular frame, and a classifier is trained using the extracted features to obtain the classifier for detecting the target as a whole.

8. The target detection method according to claim 7, characterized in that: The detection object rectangular frame also includes a sub-target rectangular frame corresponding to at least one component of the entire target; Then, the classifier for detecting components of the same category is trained as follows: According to the normalized target rectangular frame, obtaining the size of the sub-target rectangular frame corresponding to the target rectangular frame; According to the sizes of the sub-target rectangular boxes corresponding to all components of the same category in the training data set, the normalized scale sizes of the components of the same category are calculated, and the sizes of the images within the sub-target rectangular boxes corresponding to all components of the same category are normalized according to the normalized scale sizes; Feature extraction is performed on the normalized image within the sub-target rectangular frame, and a classifier is trained using the extracted features to obtain the classifier for detecting components of the same category.

9. The target detection method according to claim 1, characterized in that: After obtaining the position information of the entire target corresponding to the position relationship combination of the detection objects when the preset threshold condition is met, the method further includes: The acquired location information of all the target entities is screened to obtain the location information of the target entity that meets the preset conditions.

10. The target detection method according to claim 8, characterized in that: The step of screening all the acquired location information of the target as a whole to obtain the location information of the target as a whole that meets the preset conditions specifically includes: A non-maximum suppression processing method is adopted to screen the position information of all the detected targets as a whole, so as to obtain the position information of the target as a whole that meets the preset conditions.

11. The target detection method according to claim 1, characterized in that: Before obtaining the position information and discrimination probability of different detection objects of the original image, the method further includes: Acquire an original image, and scale the original image to different scales; Then, for the original images of different scales, the following steps are performed in sequence: obtaining the position information and discrimination probability of different detection objects of the original images.

12. A target detection device, characterized in that: include: A detection object detection module, used to obtain position information and discrimination probability of different detection objects in the original image; wherein the different detection objects include the target as a whole and at least one component constituting the target as a whole; A position relationship combination acquisition module, used to traverse and obtain all the detection object position relationship combinations according to the position information and discrimination probability of each detection object; wherein the detection object position relationship combination is composed of the position information of any two different detection objects; A probability calculation module, used to calculate the occurrence probability of the position relationship between the two detection objects corresponding to each of the detection object position relationship combinations according to a pre-built probability model; A threshold condition judgment module, used to judge whether the discrimination probability of the two detection objects corresponding to each combination of the detection object position relationship and the occurrence probability of the position relationship between the two detection objects meet a preset threshold condition; A target position acquisition module, used to acquire the position information of the entire target corresponding to the position relationship combination of the detection objects when a preset threshold condition is met; The probability calculation module is specifically used for: The position information of the two detection objects in each detection object position relationship combination is input into the probability model for calculation and analysis to obtain the occurrence probability of the position relationship between the two detection objects; wherein the probability model is used to calculate the occurrence probability of the position relationship between the two detection objects corresponding to the detection object position relationship combination; the occurrence probability is used to characterize the probability of the first detection object appearing in a relative position relative to the second detection object in the detection object position relationship combination, and the relative position refers to the relative position between the two detection objects.

13. A target detection device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the target detection method according to any one of claims 1 to 11 when executing the computer program.

Images