An unsupervised clustering target object video search method

Abstract

The application discloses a target object video search method based on unsupervised clustering, and the specific process is as follows: S1, for a target object, an expression vector of the target object is extracted by directly setting a candidate box as a bounding box of the target pedestrian. Since the bounding box of the target pedestrian is given by a human being, the target object can be completely and accurately framed in the bounding box; S2, similarity calculation is performed between the target pedestrian and a to-be-searched pedestrian in a to-be-searched library, and the final similarity is equal to a cosine similarity weighted by a detection confidence. The detection confidence can effectively suppress false detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to a target object video search method based on unsupervised clustering. BACKGROUND

[0002] In a public security video monitoring system, it is required to lock the position of a target object in a video image and to find and locate a specific target object in public security video resources, so as to realize target object detection and target object re-identification.

[0003] In the search of a target object in a public security video monitoring system, image information of the target object is obtained according to the search, and further, position information and identity information of the target object in the video monitoring system are obtained, which are two functions required by an intelligent video monitoring system, namely, function positioning and identity recognition, so as to realize target object re-identification, otherwise, a large amount of public security video resources cannot play a role. The difficulty of target object search positioning and re-identification is that the complexity of light conditions causes difficulty in identifying video images, resulting in false detection and incorrect identification of the detection result. Different shooting angles of public security cameras will cause changes in the features of the target object, affecting the effect of target object re-identification. In addition, the image of the target object is partially or completely blocked by an obstacle, which will affect the accuracy of video detection and re-identification.

[0004] Currently, the way of searching for pedestrians in a video resource and the way of searching for videos are to use a binary classification loss function to supervise the radial module length and to use an online instance matching loss function to supervise the vector angle. In the model inference stage, the feature vector of the target pedestrian is normalized in length, and the similarity between the feature vector and the feature vector corresponding to any detection frame is calculated, and the similarity is determined by the length and angle of the vector, that is, the search result depends on the confidence of pedestrian detection and the cosine similarity of pedestrian re-identification. The higher the similarity is, the higher the detection confidence is, and the higher the identity similarity of the pedestrian is.

[0005] One-step pedestrian search model is based on the top convolutional layer, and a multi-task module for coordinate regression, classification and re-identification is stacked. The online instance matching model is a normalized fully connected layer stacked on the features of the global average pooling layer, which is used to generate a pedestrian re-identification feature vector. The boundary box regression and region classification loss function remain the same, and the pedestrian feature vector generated by the fully connected layer is supervised by the identity classification loss function.

[0006] However, both the model based on geometric decomposition representation and the original model are based on the same assumption that the position of the bounding box does not move much between frames. However, when the camera is in motion, the object moves fast, or the frame rate of the video is relatively low, the assumption does not hold. The original model alleviates this problem by adding a camera motion model as a pre-processing, which is a camera motion compensation. The camera motion compensation estimates an affine transformation matrix from the input images by iterative optimization, which is computationally expensive. SUMMARY

[0007] In view of the deficiencies of the prior art, the present application aims to provide a target object video search method based on unsupervised clustering.

[0008] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0009] A target object video search method based on unsupervised clustering, the specific process is as follows:

[0010] First, the representation vector of the candidate box is extracted by directly setting the candidate box as the bounding box of the target pedestrian. Since the bounding box of the target pedestrian is given by human, the bounding box is complete and accurate when the target object is completely framed. Then, the similarity between the target pedestrian and any detected pedestrian in the search library is calculated to obtain the cosine similarity between the target pedestrian and the searched pedestrian. The final similarity is equal to the cosine similarity weighted by the detection confidence. The calculation of the final similarity is as follows:

[0011]

[0012] is the cosine similarity between the target pedestrian and the searched pedestrian, is the detection confidence.

[0013] Further, the specific process of extracting the representation vector of the candidate frame is: the convolution feature map of each candidate frame is transformed into a vector by global average pooling, the feature vector of the pedestrian processes the interference noise of the misplacement area; by emphasizing the body part features of the pedestrian and suppressing the features of the misplacement area, the spatial information is utilized; the tensor is predicted from the top feature map, the 256-dimensional vector of all pixel positions is normalized and scaled into a geometric decomposition representation, and the spatial structure is still retained; the radial module length of each pixel position is used as a spatial attention to calibrate the importance of each pixel before the tensor is pooled into the final matching vector; the pixel-by-pixel cross-entropy loss function is used to supervise the radial module length of all positions; the pedestrian search dataset does not have pixel-by-pixel labels, and it is necessary to generate coarse-grained labels from the bounding box annotations; in the inference process, the probability map of each pixel position is obtained, and a single probability is still needed to measure the detection confidence when matching pedestrians, therefore, the average probability of all spatial positions is used to calculate the detection confidence; the search coarse-grained generation process is that, for each candidate frame, the intersection result of the target candidate frame and the labeled bounding box is calculated.

[0014] The present application has the following advantages:

[0015] 1. The detection quality of the present application is higher: the present application realizes a recall rate of 92.6% and an average precision of 86.8%; better detection quality indicates that the detection target in the present method is optimized more smoothly and efficiently. Thanks to the high-quality pedestrian bounding box, the final pedestrian search performance of the model of the present application is better.

[0016] 2. The method of the present application has stronger identity identification. The present method achieves a performance of 91.5% and 92.4% in the overall class average precision and the highest precision, and the performance of the detection frame is improved by 5.6% and 4.8%. The method of the present application has better re-identification precision. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 It is an online instance matching model used in the embodiment of the present application.

[0018] Figure 2 It is a bilinear interpolation model used in the embodiment of the present application.

[0019] Figure 3 It is a geometric decomposition representation model used in the embodiment of the present application.

[0020] Figure 4 It is a pixel-by-pixel geometric decomposition representation model used in the embodiment of the present application. DETAILED DESCRIPTION

[0021] The present application will be further described below in conjunction with the accompanying drawings. It should be noted that the present embodiment is based on the technical solution of the present application, and detailed implementation and specific operation processes are given, but the protection scope of the present application is not limited to the present embodiment.

[0022] The present embodiment provides a target object video search method based on unsupervised clustering, and the specific process is as follows:

[0023] First, the representation vector of the candidate box is extracted by directly setting the candidate box as the bounding box of the target pedestrian. Since the bounding box of the target pedestrian is given by a human, the bounding box is complete and accurate when it frames a target object. Then, the similarity between the target pedestrian and any detected pedestrian in the search library is calculated to obtain the cosine similarity between the target pedestrian and the searched pedestrian. The final similarity is equal to the cosine similarity weighted by the detection confidence.

[0024] The final similarity is calculated as follows:

[0025]

[0026] is the cosine similarity between the target pedestrian and the searched pedestrian, is the detection confidence.

[0027] The addition of the detection confidence can effectively suppress false detection. It has the same form as the class weighted similarity, but differs from the class weighted similarity in that the present embodiment does not only use the weighted similarity as a post-processing step, but uses it to explicitly decompose the pedestrian detection and the target function of the re-identification target in the training process.

[0028] As shown in Figure 1 The online instance matching model is shown in the figure. In the online instance matching model, the norm normalization full connection layer is stacked on the features of the global average pooling layer to generate the pedestrian re-identification feature vector. The bounding box regression and the region classification loss function are consistent with the loss function in the Faster R-CNN. The pedestrian feature vector generated by the full connection layer is supervised by the identity classification loss function. The convolution feature map of each candidate box is transformed into a vector by global average pooling, and the feature vector of the pedestrian processes the interference noise of the misaligned region.

[0029] As shown in Figure 4As shown, with spatial information, the features of the body parts of pedestrians are emphasized and the features of misalignment areas are suppressed. From the top feature map, a tensor is predicted, a 256-dimensional vector for all pixel positions is normalized and scaled into a geometric decomposition representation, still retaining the spatial structure. The radial module length of each pixel position is used as a spatial attention to calibrate the importance of each pixel before the tensor is pooled into the final matching vector. A pixel-wise cross-entropy loss function is used to supervise the radial module length of all positions. Unlike the standard semantic segmentation method, the pedestrian search dataset does not have pixel-wise labels, and coarse-grained labels need to be generated from the bounding box annotations. In the inference process, a probability map is obtained for each pixel position, and a single probability is still needed to measure the detection confidence when matching pedestrians. Therefore, the average probability of all spatial positions is used to calculate the detection confidence in this embodiment, as shown in Figure 2 The generation process of the search coarse-grained is as follows: for each candidate box, the intersection result of the target candidate box and the labeled bounding box is calculated.

[0030] As shown in Figure 3 The original region classification branch of Faster R-CNN is removed, and the length of the pedestrian feature vector is directly used as the confidence of the target background classification. The representation vector of the target is extracted by directly setting the candidate box as the bounding box of the target pedestrian. The similarity is equal to the cosine similarity weighted by the detection confidence. The addition of the detection confidence suppresses the false detection.

[0031] For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all these changes and modifications should be included in the protection scope of the claims of the present application.

Claims

1. A method of unsupervised clustering-based target object video search, characterized by, The specific process is as follows: First, the representation vector of the candidate frame is extracted by directly setting the candidate frame as the bounding box of the target pedestrian; since the bounding box of the target pedestrian is given by human, the bounding box is complete and accurate when it frames a target object; then, the similarity between the target pedestrian and any detected pedestrian in the search library is calculated, and the cosine similarity between the target pedestrian and the searched pedestrian is obtained; the final similarity is equal to the cosine similarity weighted by the detection confidence; the final similarity is calculated as follows: is a cosine similarity between the target pedestrian and the to-be-searched pedestrian, is a detection confidence; The specific process of extracting the representation vector of the candidate frame is as follows: the convolution feature map of each candidate frame is transformed into a vector by global average pooling, and the feature vector of the pedestrian is processed to eliminate the interference noise of the misaligned area; by emphasizing the features of the body part of the pedestrian and suppressing the features of the misaligned area, the spatial information is utilized; the tensor is predicted from the top feature map, the 256-dimensional vector of all pixel positions is normalized and scaled to a geometric decomposition representation, and the spatial structure is still retained; the radial module length of each pixel position is used as a spatial attention to calibrate the importance of each pixel before the tensor is pooled into the final matching vector; the pixel-by-pixel cross-entropy loss function is used to supervise the radial module length of all positions; The pedestrian search dataset does not have pixel-by-pixel labels, and the coarse-grained labels need to be generated from the bounding box annotations; In the inference process, the probability map of each pixel position is obtained, and a single probability is still needed to measure the detection confidence when matching pedestrians, so the average probability of all spatial positions is used to calculate the detection confidence; the search coarse-grained generation process is as follows: for each candidate frame, the intersection result of the target candidate frame and the labeled bounding box is calculated.

Images