Model training method, object recognition method and device, and storage medium

By establishing a sample set and training a cross-group distance model, the problem of pedestrian re-identification relying on data annotation in the existing technology is solved, and efficient identification of pedestrian re-identification across cameras is achieved.

CN113807122BActive Publication Date: 2024-11-22阿里巴巴(中国)网络技术有限公司
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202010529203.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-11
Publication Date
2024-11-22
Estimated Expiration
2040-06-11

AI Technical Summary

Technical Problem

The existing pedestrian re-identification technology relies on large amounts of data annotation, which is time-consuming and poor migration, making it difficult to achieve unsupervised cross-camera pedestrian re-identification.

Method used

By acquiring the images collected by the image acquisition device, a multi-group sample set is established, and the distance between samples is calculated using a single-group distance model and a cross-group distance model, and the cross-group distance model is initialized and continued to train the cross-group distance model to realize unsupervised pedestrian re-identification across cameras.

Benefits of technology

It realizes high recognition accuracy of unsupervised cross-camera pedestrian re-identification, avoids the dependence of data annotation, and improves the efficiency and migration of pedestrian re-identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113807122B_ABST
    Figure CN113807122B_ABST
Patent Text Reader

Abstract

A model training method, an object recognition method and device, and a storage medium, the method comprising acquiring images acquired by an image acquisition device, establishing multiple groups of sample sets according to the acquired images; performing association processing on the acquired images to obtain multiple video clips, establishing a single group of sample pair sets according to the video clips and the sample sets, and using the single group of sample pair sets to train a single group of distance models, the single group of distance models being used to calculate the distance between two samples belonging to the same sample set; establishing a first cross-group sample pair set and a second cross-group sample pair set according to the single group of distance models and the sample sets, initializing the training of the cross-group distance model using the single group of sample pair sets and the first cross-group sample pair set, and continuing to train the cross-group distance model using the second cross-group sample pair set. The present application realizes unsupervised cross-camera model training and object recognition, and the recognition accuracy is relatively high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to but is not limited to the field of image recognition technology, and in particular to a model training method, an object recognition method and device, and a storage medium. Background Art

[0002] At present, a large number of surveillance cameras are used in places with high crowd density and prone to security problems, such as supermarket anti-theft, park security, etc. Pedestrian re-identification, also known as pedestrian re-identification or re-verification, is a technology that determines whether pedestrians captured by non-overlapping cameras are the same pedestrian. Pedestrian re-identification technology makes it possible to retrieve surveillance videos for specific objects, which has important practical significance. According to the dependence on training data, pedestrian re-identification algorithms can be divided into pedestrian re-identification algorithms based on supervised learning, pedestrian re-identification algorithms based on semi-supervised learning, and pedestrian re-identification algorithms based on unsupervised learning. For the former two, users need to collect and annotate a large number of training samples in specific application scenarios, which is not only time-consuming but also has poor transferability. Unsupervised pedestrian re-identification does not rely on data annotation and has become a trend in the development of pedestrian re-identification technology. Summary of the invention

[0003] The present application provides a model training method, an object recognition method and device, and a storage medium, which can realize unsupervised cross-camera pedestrian re-recognition with relatively high recognition accuracy.

[0004] The embodiment of the present application provides a model training method, including: acquiring images acquired by an image acquisition device, and establishing multiple groups of sample sets according to the acquired images; performing association processing on the acquired images to obtain multiple video clips, establishing a single group of sample pair sets according to the video clips and the sample sets, and using the single group of sample pair sets to train a single group distance model, wherein the single group distance model is used to calculate the distance between two samples belonging to the same sample set; establishing a first cross-group sample pair set and a second cross-group sample pair set according to the single group distance model and the sample set, wherein the first cross-group sample pair set includes multiple first cross-group sample pairs, wherein the first cross-group sample pairs include first samples and second samples belonging to different sample sets, and the first The sample and the second sample are the samples that are closest to each other and searched in the sample set to which the other is located; the second cross-group sample pair set includes multiple second cross-group sample pairs, the second cross-group sample pairs include a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is the sample that is closest to the other of the third sample and the fourth sample and searched in the sample set to which the second sample is located; the cross-group distance model is initialized for training using the single-group sample pair set and the first cross-group sample pair set, and the cross-group distance model is continuously trained using the second cross-group sample pair set until the training of the cross-group distance model is completed, and the cross-group distance model is used to calculate the distance between two samples belonging to different sample sets.

[0005] In some possible implementations, establishing multiple groups of sample sets based on the acquired images includes: scaling the acquired images to preset pixel values; extracting multidimensional feature vectors of the scaled images; reducing the dimensionality of the extracted multidimensional feature vectors; and storing the feature vectors after dimensionality reduction as samples in the sample sets.

[0006] In some possible implementations, an image captured by the image acquisition device corresponds to one or more sets of sample sets.

[0007] In some possible implementations, the single group of sample pair sets includes a positive sample pair set and a negative sample pair set; the establishing of a single group of sample pair sets based on the video clip and the sample set includes: taking any two samples belonging to the same video clip as a positive sample pair, and a plurality of positive sample pairs corresponding to a group of sample sets constitute a positive sample pair set corresponding to the group of sample sets; taking any two samples belonging to different video clips as a negative sample pair, and a plurality of negative sample pairs corresponding to a group of sample sets constitute a negative sample pair set corresponding to the group of sample sets.

[0008] In some possible implementations, the single-group distance model and the cross-group distance model are both cross-view Figure 2 Sub-discriminant analysis distance model.

[0009] In some possible implementations, when the cross-group distance model is continuously trained using the second cross-group sample pair set, a loop convergence condition is that all second cross-group sample pairs are trained or the number of training times reaches a preset number of iterations.

[0010] An embodiment of the present application also provides a model training device, including a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the model training method as described in any one of the above.

[0011] An embodiment of the present application also provides a storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the model training method described in any one of the above are implemented.

[0012] An embodiment of the present application provides an object recognition method, including: acquiring an image to be recognized; performing object recognition on the image to be recognized using a pre-trained cross-group distance model, wherein the training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set, wherein the initial training sample set includes a single-group sample pair set and a first cross-group sample pair set, wherein the secondary training sample set includes a second cross-group sample pair set, wherein the first cross-group sample pair set includes a plurality of first cross-group sample pairs, wherein the first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and wherein the first sample and the second sample are samples that are closest to each other and are searched in the sample set to which the other belongs; wherein the second cross-group sample pair set includes a plurality of second cross-group sample pairs, wherein the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and wherein one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample and is searched in the sample set to which the other belongs.

[0013] The embodiment of the present application provides an object recognition method, including: a cloud server receives an image to be recognized input by a user, wherein the image to be recognized contains a target object; the cloud server uses a pre-trained cross-group distance model to perform object recognition on the image to be recognized, and outputs one or more images, wherein the output images contain an object matching the target object, the training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set, the initial training sample set includes a single-group sample pair set and a first cross-group sample pair set, the secondary training sample set includes a second cross-group sample pair set, the first cross-group sample pair set includes a plurality of first cross-group sample pairs, the first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes a plurality of second cross-group sample pairs, the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the other is located.

[0014] An embodiment of the present application further provides an object recognition device, including a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of any one of the object recognition methods described above.

[0015] An embodiment of the present application further provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the object recognition method as described in any one of the above are implemented.

[0016] The embodiment of the present application also provides a model training device, including a sample establishment module, a single-group training module and a cross-group training module, wherein: the sample establishment module is used to obtain images acquired by an image acquisition device, and establish multiple groups of sample sets according to the acquired images; the single-group training module is used to perform association processing on the acquired images to obtain multiple video clips, establish a single-group sample pair set according to the video clips and the sample set, and use the single-group sample pair set to train a single-group distance model, wherein the single-group distance model is used to calculate the distance between two samples belonging to the same sample set; the cross-group training module is used to establish a first cross-group sample pair set and a second cross-group sample pair set according to the single-group distance model and the sample set, wherein the first cross-group sample pair set includes multiple first cross-group sample pairs, and the first cross-group A sample pair includes a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other and are searched in the sample set to which the other is located; the second cross-group sample pair set includes multiple second cross-group sample pairs, the second cross-group sample pairs include a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is searched in the sample set to which the other is located and is closest to the other of the third sample and the fourth sample; a cross-group distance model is initialized and trained using a single-group sample pair set and the first cross-group sample pair set, and the cross-group distance model is continuously trained using the second cross-group sample pair set until the training of the cross-group distance model is completed, and the cross-group distance model is used to calculate the distance between two samples belonging to different sample sets.

[0017] The embodiment of the present application also provides an object recognition device, including an image acquisition module and an object recognition module, wherein: the image acquisition module is used to acquire an image to be recognized; the object recognition module is used to perform object recognition on the image to be recognized using a pre-trained cross-group distance model, the training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set, the initial training sample set includes a single-group sample pair set and a first cross-group sample pair set, the secondary training sample set includes a second cross-group sample pair set, the first cross-group sample pair set includes a plurality of first cross-group sample pairs, the first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes a plurality of second cross-group sample pairs, the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the other is located.

[0018] The model training method, object recognition method and device, and storage medium of the present application realize an unsupervised cross-camera model training method and pedestrian re-identification method by initializing the training of the cross-group distance model using a single group sample pair set and a first cross-group sample pair set and continuing to train the cross-group distance model using a second cross-group sample pair set, and the object recognition result accuracy of the present application is relatively high.

[0019] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0021] Figure 1 A flowchart of a model training method according to an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a processing process of a model training method according to an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a flow chart of an object recognition method according to an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a flow chart of another object recognition method according to an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a pedestrian re-identification result in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of a model training device according to an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of an object recognition device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0029] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.

[0030] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0031] like Figure 1 As shown, an embodiment of the present application provides a model training method, including steps 101 to 104.

[0032] Step 101 includes: acquiring images acquired by an image acquisition device, and establishing multiple groups of sample sets according to the acquired images.

[0033] In an exemplary embodiment, the image acquisition device may be a camera, an image sensor, or any other type of image acquisition device.

[0034] In an exemplary embodiment, the method further includes: performing pedestrian detection in the acquired image using a deformable parts model (DPM) algorithm to extract a pedestrian image.

[0035] The DPM algorithm is a very successful target detection algorithm and has become an important part of many classifiers, segmentation, human posture and behavior classification. The DPM algorithm can be regarded as an extension of the Histogram of Oriented Gradients (HOG), and the general idea is the same as HOG. First, the gradient direction histogram is calculated, and then the support vector machine (SVM) is used to train the gradient model of the object.

[0036] In an exemplary embodiment, an image captured by one image acquisition device may correspond to one set of sample sets; or, an image captured by one image acquisition device may correspond to multiple sets of sample sets.

[0037] In an exemplary embodiment, multiple groups of sample sets are established according to the acquired images, including:

[0038] Step 1011, scaling the acquired image to a preset pixel value;

[0039] Step 1012, extracting a multi-dimensional feature vector of the scaled image;

[0040] Step 1013, reducing the dimension of the extracted multi-dimensional feature vector;

[0041] Step 1014: store the feature vector after dimension reduction as a sample in the sample set.

[0042] In an exemplary embodiment, the preset pixel value may be 128×48 pixels, so that the scaled image occupies less storage space and can still display the features of the image more clearly.

[0043] In an exemplary embodiment, a 26960-dimensional feature vector can be extracted from each scaled pedestrian image using a local maximum probability (LOMO) algorithm. Compared with deep features, the feature vector extracted by the artificially designed LOMO operator is more efficient and stable, suitable for embedded deployment and not vulnerable to adversarial attacks.

[0044] The LOMO algorithm focuses on illumination and perspective issues. The Retinex image enhancement algorithm is used for image enhancement before feature extraction. The feature representation uses HSV (Hue, Saturation, Value) histogram and Scale Invariant Local Ternary Pattern (SILTP) histogram. The HSV space is the most commonly used color space for histograms. Its three components represent color (Hue), saturation (Saturation) and value (Value).

[0045] In an exemplary embodiment, the principal component analysis (PCA) algorithm is used to reduce the dimension of the extracted multi-dimensional feature vector. Exemplarily, the feature vector after dimensionality reduction may be 600-dimensional.

[0046] The PCA algorithm is one of the most important dimensionality reduction methods and is widely used in data compression, redundancy elimination, and data noise elimination. The PCA algorithm transforms the original data into a set of linearly independent representations of each dimension through linear transformation, thereby extracting the main linear components of the data.

[0047] Step 102 includes: performing association processing on the acquired images to obtain multiple video clips, establishing a single group of sample pair sets based on the video clips and the sample sets, and using the single group of sample pair sets to train a single group of distance models, which are used to calculate the distance between two samples belonging to the same sample set.

[0048] In an exemplary embodiment, a generalized maximum multiclique problem (GMMCP) tracking algorithm is used to perform association processing on the acquired images to obtain multiple video clips.

[0049] The GMMCP tracking algorithm provides a data association method based on a global framework. It first formulates the objective function with the help of binary integer programming, then adds virtual nodes to deal with the problem of missing targets, and proposes an acceleration method on this basis: aggregating virtual nodes. Finally, the maximum binary integer programming is used to solve the undirected graph, so that multiple groups are obtained at the same time, that is, the multi-target tracking is completed. In short, GMMCP converts multi-target tracking into a graph problem. First, the graph is constructed using weights, and the final trajectory is obtained by solving the graph.

[0050] In an exemplary embodiment, the single set of sample pairs includes a positive sample pair set and a negative sample pair set. The single set of sample pairs is established according to the video clip and the sample set, including:

[0051] Any two samples belonging to the same video clip are regarded as a positive sample pair, and multiple positive sample pairs corresponding to a set of sample sets constitute a positive sample pair set corresponding to the set of sample sets;

[0052] Any two samples belonging to different video clips are regarded as a negative sample pair, and multiple negative sample pairs corresponding to a set of sample sets constitute a negative sample pair set corresponding to the set of sample sets.

[0053] In an exemplary embodiment, the single-group distance model and the cross-group distance model established in the embodiment of the present application are both cross-view distance models. Figure 2 Cross-view Quadratic Discriminant Analysis (XQDA) distance model.

[0054] The XQDA distance model of the embodiment of the present application includes two types: a single-group distance model and a cross-group distance model. In an exemplary embodiment, the single-group distance model corresponding to the i-th group of samples is represented by d i (.,.), the single group distance model corresponding to the j-th group of samples is expressed as d j (.,.), the cross-group distance model from the i-th sample set to the j-th sample set is expressed as d i,j (.,.), the cross-group distance model from the j-th sample set to the i-th sample set is expressed as d j,i (.,.).

[0055] The XQDA distance model is proposed based on the Keep It Simple and Straightforward (KISSME) metric and the Bayesian face method. This method uses Gaussian models to fit the difference distribution of sample features within and between classes, and derives the Mahalanobis distance based on the log-likelihood ratio of two Gaussian distributions.

[0056] In the embodiment of the present application, for each sample set, any two samples in the same video clip are taken as positive sample pairs, and any two samples in different video clips are taken as negative sample pairs, to establish a single sample pair set S. For example, for the i-th sample set, the single sample pair set is S i . Use XQDA to train a single-group distance model d(x, y) on a single-group sample pair set S. For example, the single-group distance model corresponding to the i-th sample set is d i (x, y), where the superscript i represents the i-th sample set, and x and y represent different samples.

[0057] The expression of the XQDA distance model in the embodiment of the present application is (,)( T (y)) T M(W T(xy))), where W is the model matrix parameter and M is the model covariance parameter. W and M can be learned from training samples and the XQDA algorithm.

[0058] Step 103 includes: establishing a first cross-group sample pair set and a second cross-group sample pair set according to a single-group distance model and a sample set, the first cross-group sample pair set including multiple first cross-group sample pairs, the first cross-group sample pair including a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes multiple second cross-group sample pairs, the second cross-group sample pair including a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the other is located.

[0059] In the embodiment of the present application, the first sample and the second sample are samples that are closest to each other in the sample set where the other is located, which means that the second sample is the sample that is closest to the first sample in the sample set where the second sample is located, and the first sample is the sample that is closest to the second sample in the sample set where the first sample is located.

[0060] One of the third sample and the fourth sample is a sample that is searched in the sample set to which the fourth sample belongs and is closest to the other of the third sample and the fourth sample, which means that the fourth sample is a sample that is searched in the sample set to which the fourth sample belongs and is closest to the third sample, or the third sample is a sample that is searched in the sample set to which the third sample belongs and is closest to the fourth sample.

[0061] The embodiment of the present application uses the first cross-group sample pair set to collect first cross-group sample pairs with high credibility, and the first cross-group sample pairs in the first cross-group sample pair set can effectively initialize the cross-group distance model. Then, the cross-view collaborative learning method is used to iteratively label the second cross-group sample pairs between the two groups of sample sets, so that the cross-group distance model can be continuously trained.

[0062] In an exemplary embodiment, the first cross-group sample pair set includes a plurality of first cross-group sample pairs. For example, the first cross-group sample pair from the i-th sample set to the j-th sample set can be expressed as (x i , ), where i and j are both natural numbers and i≠j, x i is a sample in the i-th group of samples, is the search result in the jth sample set that matches x i The closest sample And the searched samples in the i-th group are The closest sample With x i For the same sample, Ω j is the jth sample set, Ω i is the i-th group of samples.

[0063] In an exemplary embodiment, the second cross-group sample pair set includes a plurality of second cross-group sample pairs, and the second cross-group sample pairs from the i-th sample set to the j-th sample set are (x i , ), where i and j are both natural numbers and i≠j, x i is a sample in the i-th group of samples, is the search result in the jth sample set that matches x i The closest sample Ω j is the jth group of samples.

[0064] Step 104 includes: initializing the training of a cross-group distance model using a single-group sample pair set and a first cross-group sample pair set, and continuing to train the cross-group distance model using a second cross-group sample pair set until the training of the cross-group distance model is completed. The cross-group distance model is used to calculate the distance between two samples belonging to different sample sets.

[0065] In the embodiment of the present application, first, multiple initial training sample pair sets of cross-group distance models are established. For example, all first cross-group sample pairs corresponding to the i-th sample set to the j-th sample set are added to the single-group sample pair set corresponding to the i-th sample set, and the initial training sample pair set S of the cross-group distance model from the i-th sample set to the j-th sample set is obtained. i,j =S i ∪S s , where S i is a single set of sample pairs corresponding to the i-th sample set (the single set of sample pairs includes a positive sample pair set and a negative sample pair set), S s is the first cross-group sample pair set from the i-th sample set to the j-th sample set. All the first cross-group sample pairs from the j-th sample set to the i-th sample set are added to the single sample pair set corresponding to the j-th sample set to obtain the initial training sample pair set S of the cross-group distance model from the j-th sample set to the i-th sample set. j,i =S j ∪S s′ , where S j is the single set of sample pairs corresponding to the jth set of samples (the single set of sample pairs includes a positive sample pair set and a negative sample pair set), S s ′ is the first cross-group sample pair set from the j-th sample set to the i-th sample set.

[0066] Then, the established initial training sample pair sets are used to initialize and train multiple cross-group distance models. i,j Initialize the training to obtain the cross-group distance model d from the i-th sample set to the j-th sample set i,j (.,.), using the initial training sample set S j,i Initialize the training to obtain the cross-group distance model d from the j-th sample set to the i-th sample set j,i (.,.).

[0067] All the second cross-group sample pairs that meet the conditions are added to the initial training sample pair set to obtain an updated training sample pair set, and the updated training sample pair set is used to continue training the cross-group distance model. For example, all the second cross-group sample pairs from the i-th sample set to the j-th sample set are added to the initial training sample pair set S i,j , get the updated training sample pair set, and continue to train the cross-group distance model d from the i-th sample set to the j-th sample set i,j (.,.); Update the training sample set S in the same way j,i , and then continue to train the model d j,i (.,.).

[0068] In an exemplary embodiment, when the cross-group distance model is continuously trained using the second cross-group sample pair set, the loop convergence condition may be that all second cross-group sample pairs are trained or the number of training times reaches a preset number of iterations. For example, the preset number of iterations may be set to 20 times.

[0069] In an exemplary embodiment, Figure 2 As shown, the user collects images and sends them to the server. The server uses the model training method to establish multiple sample sets, establishes a single sample pair set based on the sample sets, and uses the single sample pair set to train the single distance model. According to the single distance model and the sample set, a first cross-group sample pair set and a second cross-group sample pair set are established. The cross-group distance model is initialized with the first cross-group sample pair set, and the cross-group distance model is continued to be trained with the second cross-group sample pair set, and the trained cross-group distance model is output to the user.

[0070] In another exemplary embodiment, the user inputs an image set and its original image recognition model, and the server uses the model training method to establish a single-group distance model and a cross-group distance model for the image set input by the user and trains the established single-group distance model and cross-group distance model, and optimizes the image recognition model input by the user according to the trained cross-group distance model.

[0071] The present application also provides an object recognition method, such as Figure 3 As shown, the object recognition method includes steps 301 and 302.

[0072] Wherein, step 301 includes: obtaining an image to be identified.

[0073] Step 302 includes: using a pre-trained cross-group distance model to perform object recognition on the image to be recognized, the training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set, the initial training sample set includes a single-group sample pair set and a first cross-group sample pair set, the secondary training sample set includes a second cross-group sample pair set, the first cross-group sample pair set includes a plurality of first cross-group sample pairs, the first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes a plurality of second cross-group sample pairs, the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the other is located.

[0074] In an exemplary embodiment, the single set of sample pairs includes a positive sample pair set and a negative sample pair set. The positive sample pair set includes a plurality of positive sample pairs, each positive sample pair includes two samples belonging to the same video segment. The negative sample pair set includes a plurality of negative sample pairs, each negative sample pair includes two samples belonging to different video segments.

[0075] In an exemplary embodiment, the cross-group distance model is a cross-view Figure 2 Sub-discriminant analysis distance model.

[0076] The present application also provides an object recognition method, such as Figure 4 As shown, the object recognition method includes steps 401 and 402.

[0077] Wherein, step 401 includes: the cloud server receives an image to be identified input by a user, where the image to be identified includes a target object;

[0078] Step 402 includes: the cloud server uses a pre-trained cross-group distance model to perform object recognition on the image to be recognized, and outputs one or more images, wherein the output images include objects matching the target object, and the training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set, wherein the initial training sample set includes a single-group sample pair set and a first cross-group sample pair set, wherein the secondary training sample set includes a second cross-group sample pair set, wherein the first cross-group sample pair set includes a plurality of first cross-group sample pairs, wherein the first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and wherein the first sample and the second sample are samples that are closest to each other and are searched in the sample set to which the other belongs; wherein the second cross-group sample pair set includes a plurality of second cross-group sample pairs, wherein the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and wherein one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample and is searched in the sample set to which the other belongs.

[0079] In an exemplary embodiment, the user first sends a training image set to a cloud server. The cloud server uses the aforementioned model training method to establish an initial training sample set and a secondary training sample set for the training image set sent by the user, and uses the initial training sample set and the secondary training sample set to train the pre-established single-group distance model and cross-group distance model; thereafter, the user inputs a test image, and the cloud server performs object recognition on the test image input by the user according to the trained cross-group distance model, and outputs the recognition result that is closest to the test image input by the user to the user.

[0080] In an exemplary embodiment, Figure 5 As shown, the user inputs a prototype image segment of a pedestrian to be identified into a pre-trained cross-group distance model to obtain one or more identified pedestrian image segments close to the pedestrian.

[0081] Person Re-Identification (ReID), that is, re-identifying pedestrians, refers to the process of establishing a corresponding relationship between pedestrian images taken by different cameras with non-overlapping fields of view. When there is no overlap between the camera shooting ranges, the difficulty of retrieval increases due to the lack of continuous information. Therefore, person re-identification emphasizes the retrieval of specific pedestrians in videos across cameras. Person re-identification is mainly used in image retrieval and other aspects. Given a picture of a person (query image), it is a personnel comparison technology that uses the overall characteristics of pedestrians to find the one or more pictures belonging to him / her from multiple pictures (gallery images).

[0082] The embodiment of the present application uses the PRID 2011 standard pedestrian re-identification database to test the effectiveness of the pedestrian re-identification method of the present application. The data set was collected by the Austrian Institute of Technology for the purpose of promoting the research of pedestrian re-identification technology. The data set consists of images extracted from multiple pedestrian trajectories recorded by two different static surveillance cameras. These images differ due to changes in the camera viewpoint and factors such as illumination, pedestrian background and camera characteristics. Since the images are extracted from the trajectories, each person has several different postures in the image. The data set records the trajectories of 475 people from one perspective, the trajectories of 856 people from another perspective, and 245 people appear in both perspectives. The data set has filtered out some severely obscured pedestrians, pedestrians with less than five reliable images under one camera, and damaged images due to labeling errors.

[0083] Figure 5 : is a test effect diagram of the embodiment of the present application. Figure 5 As shown, the left side is the prototype image segment of the pedestrian to be identified, and the right side is the top ten results identified by the embodiment of the present application. For the first pedestrian prototype image segment, six recognition results are correct and four recognition results are wrong. For the second pedestrian prototype image segment, all ten recognition results are correct. Figure 5 It can be seen that the embodiments of the present application can relatively successfully identify identical pedestrians and similar pedestrians.

[0084] The pedestrian re-identification method of the embodiment of the present application is qualitatively compared and analyzed with the existing pedestrian re-identification method using the top k ranking accuracy (Rank-k accuracy) evaluation method to evaluate the re-identification accuracy of the pedestrian re-identification method of the embodiment of the present application and the existing pedestrian re-identification method. In this embodiment, k is 1, 5, 10, and 20. Rank-k means the accuracy rate of the first k images sorted by similarity that have the same ID as the query image (Query Image).

[0085] The experimental results show that on the PRID 2011 video pedestrian re-identification database, the accuracy of the embodiment of the present application has obvious advantages over other unsupervised methods, and the accuracy is not much different from that of the supervised learning method. Table 1 reports the recognition accuracy comparison results on the PRID 2011 database. As shown in Table 1, Euclidean (LOMO) represents an unsupervised pedestrian re-identification algorithm based on LOMO features and Euclidean distance, Salience represents an unsupervised pedestrian re-identification algorithm based on saliency features, CNN+XQDA represents a supervised pedestrian re-identification algorithm based on convolutional neural network features and XQDA distance learning, and XQDA (LOMO) represents a supervised pedestrian re-identification algorithm based on LOMO features and XQDA distance learning. It can be seen from Table 1 that the embodiment of the present application has achieved good results in pedestrian re-identification accuracy. Compared with the existing pedestrian re-identification methods, the re-identification accuracy of the pedestrian re-identification method provided by the embodiment of the present application is significantly better than that of the unsupervised pedestrian re-identification algorithm, and slightly lower than that of the supervised pedestrian re-identification algorithm. However, the embodiment of the present application does not rely on data annotation and has greater advantages in practice.

[0086]

[0087] Table 1

[0088] The object recognition method of the embodiment of the present application realizes an unsupervised cross-camera object recognition method by initializing the training of the cross-group distance model with a single-group sample pair set and a first cross-group sample pair set and continuing to train the cross-group distance model with a second cross-group sample pair set. Experimental results show that the recognition accuracy is relatively high.

[0089] An embodiment of the present application also provides a model training device, including a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the model training method as described in any one of the above.

[0090] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the model training method described in any one of the above are implemented.

[0091] An embodiment of the present application further provides an object recognition device, including a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of any one of the object recognition methods described above.

[0092] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the object recognition method as described in any one of the above are implemented.

[0093] like Figure 6 As shown, an embodiment of the present application also provides a model training device, including a sample establishment module 601, a single group training module 602 and a cross-group training module 603.

[0094] The sample establishment module 601 is used to acquire images acquired by an image acquisition device and establish multiple groups of sample sets according to the acquired images.

[0095] The single group training module 602 is used to perform association processing on the acquired images to obtain multiple video clips, establish a single group sample pair set based on the video clips and the sample set, and use the single group sample pair set to train a single group distance model, which is used to calculate the distance between two samples belonging to the same sample set.

[0096] The cross-group training module 603 is used to establish a first cross-group sample pair set and a second cross-group sample pair set according to a single-group distance model and a sample set, wherein the first cross-group sample pair set includes multiple first cross-group sample pairs, the first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other sample belongs; the second cross-group sample pair set includes multiple second cross-group sample pairs, the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the second sample belongs; the cross-group distance model is initialized and trained using the single-group sample pair set and the first cross-group sample pair set, and the cross-group distance model is continuously trained using the second cross-group sample pair set until the training of the cross-group distance model is completed, and the cross-group distance model is used to calculate the distance between two samples belonging to different sample sets.

[0097] In an exemplary embodiment, the sample establishment module 601 establishes multiple groups of sample sets based on the acquired images, including: scaling the acquired images to preset pixel values; extracting multidimensional feature vectors of the scaled images; reducing the dimension of the extracted multidimensional feature vectors; and storing the feature vectors after dimension reduction as samples in the sample set.

[0098] In an exemplary embodiment, a single group of sample pairs includes a positive sample pair set and a negative sample pair set; the single group training module 602 establishes a single group of sample pairs according to the video clip and the sample set, including: taking any two samples belonging to the same video clip as a positive sample pair, and a plurality of positive sample pairs corresponding to a group of sample sets constitute a positive sample pair set corresponding to the sample set; taking any two samples belonging to different video clips as a negative sample pair, and a plurality of negative sample pairs corresponding to a group of sample sets constitute a negative sample pair set corresponding to the sample set.

[0099] In an exemplary embodiment, both the single-group distance model and the cross-group distance model are cross-view Figure 2 Sub-discriminant analysis distance model.

[0100] In an exemplary embodiment, the first cross-group sample pair set includes a plurality of first cross-group sample pairs, and the first cross-group sample pair from the i-th sample set to the j-th sample set is (x i , ), where i and j are both natural numbers and i≠j, x i is a sample in the i-th group of samples, is the search result in the jth sample set that matches x i The closest sample And the searched samples in the i-th group are The closest sample With x i For the same sample, Ω j is the jth sample set, Ω i is the i-th group of samples, d j (.,.) is the single group distance model corresponding to the j-th sample set, d i (.,.) is the single group distance model corresponding to the i-th group of samples.

[0101] In an exemplary embodiment, the second cross-group sample pair set includes a plurality of second cross-group sample pairs, and the second cross-group sample pairs from the i-th sample set to the j-th sample set are (x i , ), where i and j are both natural numbers and i≠j, x i is a sample in the i-th group of samples, is the search result in the jth sample set that matches x i The closest sample Ω j is the jth sample set, d i (.,.) is the single group distance model corresponding to the i-th group of samples.

[0102] In an exemplary embodiment, when the cross-group distance model is continuously trained using the second cross-group sample pair set, the loop convergence condition is that all second cross-group sample pairs are trained or the number of training times reaches a preset number of iterations.

[0103] like Figure 7 As shown, the embodiment of the present application further provides an object recognition device, including an image acquisition module 701 and an object recognition module 702 .

[0104] The image acquisition module 701 is used to acquire an image to be identified.

[0105] The object recognition module 702 is used to use a pre-trained cross-group distance model to perform object recognition on the image to be recognized. The training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set. The initial training sample set includes a single-group sample pair set and a first cross-group sample pair set. The secondary training sample set includes a second cross-group sample pair set. The first cross-group sample pair set includes multiple first cross-group sample pairs. The first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes multiple second cross-group sample pairs. The second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the other is located.

[0106] In an exemplary embodiment, the single set of sample pairs includes a positive sample pair set and a negative sample pair set. The positive sample pair set includes a plurality of positive sample pairs, each positive sample pair includes two samples belonging to the same video segment. The negative sample pair set includes a plurality of negative sample pairs, each negative sample pair includes two samples belonging to different video segments.

[0107] In an exemplary embodiment, the cross-group distance model is a cross-view Figure 2 Sub-discriminant analysis distance model.

[0108] The object recognition device of the embodiment of the present application realizes unsupervised cross-camera object recognition by initializing the training of the cross-group distance model with a single-group sample pair set and a first cross-group sample pair set and continuing to train the cross-group distance model with a second cross-group sample pair set. Experimental results show that the recognition accuracy is relatively high.

[0109] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A model training method, characterized in that: include: Acquire images acquired by an image acquisition device, and establish multiple groups of sample sets according to the acquired images; The acquired images are subjected to association processing to obtain a plurality of video clips, a single group of sample pair sets is established according to the video clips and the sample sets, a single group of distance models is trained using the single group of sample pair sets, and the single group of distance models is used to calculate the distance between two samples belonging to the same sample set; wherein the single group of sample pair sets includes a positive sample pair set and a negative sample pair set; the establishment of the single group of sample pair sets according to the video clips and the sample sets includes: taking any two samples belonging to the same video clip as a positive sample pair, and a plurality of positive sample pairs corresponding to a group of sample sets constitute a positive sample pair set corresponding to the group of sample sets; taking any two samples belonging to different video clips as a negative sample pair, and a plurality of negative sample pairs corresponding to a group of sample sets constitute a negative sample pair set corresponding to the group of sample sets; According to the single-group distance model and the sample set, a first cross-group sample pair set and a second cross-group sample pair set are established, wherein the first cross-group sample pair set includes a plurality of first cross-group sample pairs, the first cross-group sample pairs include a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other and are searched in the sample set to which the other belongs; the second cross-group sample pair set includes a plurality of second cross-group sample pairs, the second cross-group sample pairs include a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is searched in the sample set to which the other belongs and is closest to the other of the third sample and the fourth sample; A cross-group distance model is initialized and trained using a single-group sample pair set and a first cross-group sample pair set, and the cross-group distance model is continuously trained using a second cross-group sample pair set until the training of the cross-group distance model is completed. The cross-group distance model is used to calculate the distance between two samples belonging to different sample sets.

2. The model training method according to claim 1, characterized in that: The step of establishing multiple groups of sample sets according to the acquired images comprises: Scaling the acquired image to a preset pixel value; Extracting a multi-dimensional feature vector of the scaled image; Perform dimensionality reduction on the extracted multi-dimensional feature vector; The feature vector after dimension reduction is stored as a sample in the sample set.

3. The model training method according to claim 1, characterized in that: An image captured by one of the image capture devices corresponds to one or more sets of sample sets.

4. The model training method according to claim 1, characterized in that: The single-group distance model and the cross-group distance model are both cross-view quadratic discriminant analysis distance models.

5. A model training device, characterized in that: It comprises a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the model training method as described in any one of claims 1 to 4.

6. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the model training method according to any one of claims 1 to 4.

7. An object recognition method, characterized in that: include: Obtain an image to be recognized; A pre-trained cross-group distance model is used to perform object recognition on an image to be recognized. The training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set. The initial training sample set includes a single-group sample pair set and a first cross-group sample pair set. The secondary training sample set includes a second cross-group sample pair set. The single-group sample pair set includes a positive sample pair set and a negative sample pair set. The positive sample pair set includes a plurality of positive sample pairs, each positive sample pair includes two samples belonging to the same video clip. The negative sample pair set includes a plurality of negative sample pairs, each negative sample pair includes two samples belonging to different video clips. The first cross-group sample pair set includes a plurality of first cross-group sample pairs, the first cross-group sample pair includes a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set where the other sample is located. The second cross-group sample pair set includes a plurality of second cross-group sample pairs, the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set where the other sample is located.

8. An object recognition method, characterized in that: include: The cloud server receives an image to be identified input by a user, wherein the image to be identified includes a target object; The cloud server uses a pre-trained cross-group distance model to perform object recognition on the image to be recognized, and outputs one or more images, wherein the output images contain objects matching the target object, the training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set, the initial training sample set includes a single-group sample pair set and a first cross-group sample pair set, the secondary training sample set includes a second cross-group sample pair set; the single-group sample pair set includes a positive sample pair set and a negative sample pair set, the positive sample pair set includes a plurality of positive sample pairs, each positive sample pair includes two samples belonging to the same video clip, and the negative sample pair set includes a plurality of negative sample pairs. In this pair, each negative sample pair includes two samples belonging to different video clips; the first cross-group sample pair set includes multiple first cross-group sample pairs, the first cross-group sample pairs include a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes multiple second cross-group sample pairs, the second cross-group sample pairs include a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the other is located.

9. An object recognition device, characterized in that: The invention comprises a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the object recognition method according to claim 7 or claim 8.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the object recognition method according to claim 7 or claim 8.

11. A model training device, characterized in that: It includes sample establishment module, single group training module and cross-group training module, among which: The sample establishment module is used to acquire images acquired by the image acquisition device and establish multiple groups of sample sets according to the acquired images; The single-group training module is used to perform association processing on the acquired images to obtain multiple video clips, establish a single-group sample pair set according to the video clips and the sample set, and use the single-group sample pair set to train a single-group distance model, and the single-group distance model is used to calculate the distance between two samples belonging to the same sample set; wherein the single-group sample pair set includes a positive sample pair set and a negative sample pair set; the establishment of the single-group sample pair set according to the video clips and the sample set includes: taking any two samples belonging to the same video clip as a positive sample pair, and a plurality of positive sample pairs corresponding to a group of sample sets constitute a positive sample pair set corresponding to the group of sample sets; taking any two samples belonging to different video clips as a negative sample pair, and a plurality of negative sample pairs corresponding to a group of sample sets constitute a negative sample pair set corresponding to the group of sample sets; The cross-group training module is used to establish a first cross-group sample pair set and a second cross-group sample pair set according to the single-group distance model and the sample set, the first cross-group sample pair set including multiple first cross-group sample pairs, the first cross-group sample pair including a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes multiple second cross-group sample pairs, the second cross-group sample pair includes a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the other is located; the cross-group distance model is initialized and trained using the single-group sample pair set and the first cross-group sample pair set, and the cross-group distance model is continuously trained using the second cross-group sample pair set until the training of the cross-group distance model is completed, and the cross-group distance model is used to calculate the distance between two samples belonging to different sample sets.

12. An object recognition device, characterized in that: It includes an image acquisition module and an object recognition module, wherein: The image acquisition module is used to acquire the image to be identified; The object recognition module is used to use a pre-trained cross-group distance model to perform object recognition on the image to be recognized. The training sample set of the cross-group distance model includes an initial training sample set and a secondary training sample set. The initial training sample set includes a single-group sample pair set and a first cross-group sample pair set. The secondary training sample set includes a second cross-group sample pair set; the single-group sample pair set includes a positive sample pair set and a negative sample pair set. The positive sample pair set includes a plurality of positive sample pairs, each positive sample pair includes two samples belonging to the same video clip, and the negative sample pair set includes a plurality of negative sample pairs, each negative sample pair includes samples belonging to different two samples of the same video clip; the first cross-group sample pair set includes multiple first cross-group sample pairs, the first cross-group sample pairs include a first sample and a second sample belonging to different sample sets, and the first sample and the second sample are samples that are closest to each other in the sample set to which the other is located; the second cross-group sample pair set includes multiple second cross-group sample pairs, the second cross-group sample pairs include a third sample and a fourth sample belonging to different sample sets, and one of the third sample and the fourth sample is a sample that is closest to the other of the third sample and the fourth sample in the sample set to which the first sample is located.

Citation Information

Patent Citations

  • Pedestrian re-identification method and device

    CN108960013A

  • Pedestrian re-identification method, system and device based on sorting metric function

    CN109492702A

  • Face recognition method, face recognition system, medium and electronic equipment

    CN109583332A

  • Target re-identification

    GB201711541D0

  • Depth discrimination network model method for pedestrian re-recognition in image or video

    WO2019010950A1