Target object identification method and device, storage medium and electronic device

Abstract

The present invention discloses a method and device for identifying a target object, a storage medium and an electronic device, wherein the method comprises: obtaining a picture to be identified containing a target object; identifying the target object contained in the picture to be identified based on a trained object recognition model; wherein the trained object recognition model is trained by machine learning using multiple sets of data, each of which includes: a training picture corresponding to a global feature with an increased weight and a target object identified in the training picture. With respect to related technologies, the traditional pedestrian recognition model based on a neural network is trained according to the local features of pedestrians, resulting in a poor ability to recognize pedestrians.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a method and device for identifying a target object, a storage medium and an electronic device. Background Art

[0002] Pedestrian re-identification is a sub-problem in the field of image retrieval. Pedestrians (faces, vehicles) under different cameras are stored in the database, the pedestrian of interest is input, the similarity between this pedestrian and the database image is compared, and then the pedestrians with the same ID as the pedestrian of interest are obtained based on the similarity sorting. Therefore, it is a computer vision technology that realizes pedestrian identity recognition by extracting and matching features of pedestrians in images or videos under different cameras. It has very important application value in the fields of smart retail, intelligent transportation, and public security criminal investigation. Due to the complex posture changes, rich perspective changes, and occlusion of the human body, the apparent features of pedestrians with the same identity may be quite different, while pedestrians with different identities and similar clothing may have similar apparent features. Therefore, compared with the face recognition task, the research on pedestrian re-identification is more difficult. The existing pedestrian re-identification methods can be roughly divided into three categories: supervised learning, semi-supervised learning, and unsupervised learning.

[0003] Considering the actual application scenarios and algorithm performance of engineering, supervised learning pedestrian re-identification is still the mainstream method at present. It mainly improves the robustness and discriminability of pedestrian features through the design of data enhancement, neural network, loss function and training strategy. In terms of network design, the multi-branch component network based on multi-granularity theory can extract rich local features of pedestrians through horizontal stripe segmentation of different granularities. In order to further extract rich pedestrian features, an attention mechanism layer is often inserted into the network structure. Therefore, the attention mechanism has been widely used in the field of pedestrian re-identification. However, the attention module of the existing neural network pays more and more attention to the core area, that is, the local features of pedestrians, so that the neural network can only be trained according to the local features of pedestrians during the training process, resulting in poor training effect and poor recognition ability of pedestrians.

[0004] Regarding related technologies, the traditional neural network-based pedestrian recognition model is trained according to the local features of pedestrians, resulting in poor pedestrian recognition capabilities. No effective solution has been proposed yet.

[0005] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention

[0006] Embodiments of the present invention provide a method and device for identifying a target object, a storage medium, and an electronic device to at least solve the problem that a traditional neural network-based pedestrian recognition model is trained according to the local features of pedestrians, resulting in poor pedestrian recognition capabilities.

[0007] According to one aspect of an embodiment of the present invention, there is provided a method for identifying a target object, comprising: obtaining a picture to be identified containing a target object; identifying the target object contained in the picture to be identified based on a trained object recognition model; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data comprises: a training picture corresponding to a global feature with an increased weight and a target object identified in the training picture.

[0008] Furthermore, based on the trained object recognition model, before identifying the target object contained in the picture to be identified, the method also includes: dividing the feature matrix of the training picture according to the target dimension to obtain multiple activation feature matrices, wherein the training picture is used to train the object recognition model; for each activation feature matrix of the multiple activation feature matrices, clustering the each activation feature matrix using a clustering algorithm to obtain multiple activation feature matrices with marked positions corresponding to the multiple activation feature matrices, wherein the data of the activation feature matrix at the marked positions is used to indicate the global features of the object in the training picture; training the object recognition model through the multiple activation feature matrices with marked positions to obtain a trained object recognition model.

[0009] Furthermore, a clustering algorithm is used to perform clustering processing on each activation feature matrix to obtain multiple activation feature matrices with marked positions corresponding to the multiple activation feature matrices, including: obtaining a target threshold corresponding to each activation feature matrix according to a preset algorithm; determining a binary matrix corresponding to each activation feature matrix according to the target threshold; and clustering the binary matrix using a clustering algorithm to determine the marked positions corresponding to each activation feature matrix.

[0010] Furthermore, determining the binary matrix corresponding to each activated feature matrix according to the target threshold includes: creating a first target matrix, wherein the first target matrix has the same matrix structure as the activated feature matrix, and all element values ​​in the first target matrix are 0; when there is a target element in the activated feature matrix, setting the element value of the position corresponding to the element in the first target matrix to 1 to obtain a second target matrix, wherein the element value of the target element is greater than the target threshold; and determining the second target matrix as a binary matrix.

[0011] Furthermore, the binary matrix is ​​clustered using a clustering algorithm to determine the marking position corresponding to each activated feature matrix, including: determining multiple position areas corresponding to the binary matrix through a clustering algorithm, wherein the position areas include: a first position area, a second position area, the ratio of elements with element values ​​of 1 in the first position area to all elements in the first position area is greater than a first preset ratio, and the ratio of elements with element values ​​of 0 in the second position area to all elements in the second position area is greater than a second preset ratio; connecting multiple first position areas and determining a first area of ​​the connected first position area; connecting multiple second position areas and determining a second area of ​​the connected second position area; determining the marking position of the position area with a larger area between the first area and the second area, and using the marking position of the position area as the marking position corresponding to each activated feature matrix.

[0012] Furthermore, the object recognition model is trained by the multiple activation feature matrices with marked positions to obtain a trained object recognition model, including: for each activation feature matrix with marked positions in the multiple activation feature matrices with marked positions, increasing the weight of the data of each activation feature matrix with marked positions at the marked positions according to a preset rule to obtain a plurality of weights corresponding to the plurality of activation feature matrices with marked positions; and the object recognition model is trained according to the plurality of weights and the plurality of activation feature matrices with marked positions to obtain a trained object recognition model.

[0013] Furthermore, based on the trained object recognition model, the target object contained in the picture to be recognized is recognized, including: obtaining a target picture, wherein the target picture contains the target object to be recognized, and the image to be recognized contains multiple objects; inputting the picture to be recognized and the target picture into the trained object recognition model to recognize the target object from the multiple objects in the image to be recognized.

[0014] According to another aspect of an embodiment of the present invention, a device for identifying a target object is also provided, including: an acquisition module, used to acquire a picture to be identified containing a target object; and a recognition module, used to identify the target object contained in the picture to be identified based on a trained object recognition model; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training picture corresponding to a global feature with an increased weight and a target object identified in the training picture.

[0015] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned target object recognition method when running.

[0016] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the target object recognition method through the computer program.

[0017] Through the present invention, a picture to be identified containing a target object is obtained; based on a trained object recognition model, the target object contained in the picture to be identified is identified; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training picture corresponding to a global feature with an increased weight and a target object identified in the training picture. In other words, the object recognition model will focus on training the global features of the target object in the training picture. The above technical solution solves the problem that the traditional pedestrian recognition model based on a neural network is trained according to the local features of pedestrians, resulting in poor recognition ability of pedestrians. Then the pedestrian recognition model is trained according to the global features of pedestrians, so that the trained pedestrian recognition model can accurately identify pedestrians, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal of a target object identification method according to an embodiment of the present invention;

[0020] Figure 2 is a flow chart of a method for identifying a target object according to an embodiment of the present invention (I);

[0021] Figure 3 is a schematic diagram of a location area of ​​a target object recognition method according to an embodiment of the present invention;

[0022] Figure 4 is a schematic flow chart of a target object identification method according to an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of training a traditional pedestrian recognition model according to a target object recognition method according to an embodiment of the present invention;

[0024] Figure 6 is a structural block diagram of a target object recognition device according to an embodiment of the present invention (I);

[0025] Figure 7 2 is a structural block diagram of a target object recognition device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a target object identification method according to an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target object recognition method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] This embodiment provides a method for identifying a target object. Figure 2 is a flow chart of a target object recognition method according to an embodiment of the present invention, and the flow chart includes the following steps:

[0032] Step S202: obtaining a picture to be identified containing a target object;

[0033] Step S202: Based on the trained object recognition model, the target object contained in the image to be recognized is recognized; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training image corresponding to a global feature with an increased weight and a target object identified in the training image.

[0034] It should be noted that the above global features are global features of the target object.

[0035] Through the above steps, a picture to be identified containing a target object is obtained; based on the trained object recognition model, the target object contained in the picture to be identified is identified; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training picture corresponding to a global feature with an increased weight and a target object identified in the training picture. In other words, the object recognition model will focus on training the global features of the target object in the training picture. The above technical solution solves the problem that the traditional pedestrian recognition model based on a neural network is trained according to the local features of pedestrians, resulting in poor recognition ability of pedestrians. Then the pedestrian recognition model is trained according to the global features of pedestrians, so that the trained pedestrian recognition model can accurately identify pedestrians, thereby improving the recognition accuracy.

[0036] For a better understanding, based on the trained object recognition model, before identifying the target object contained in the to-be-recognized picture, the following steps are further performed:

[0037] Step 1: dividing the feature matrix of the training image according to the target dimension to obtain multiple activation feature matrices, wherein the training image is used to train the object recognition model;

[0038] It should be noted that the target dimension can be [B,H,W], where B represents the number, H represents the height, and W represents the width.

[0039] Step 2: for each of the multiple activation feature matrices, clustering processing is performed on the activation feature matrix using a clustering algorithm to obtain multiple activation feature matrices with marked positions corresponding to the multiple activation feature matrices, wherein the data of the activation feature matrix at the marked position is used to indicate the global features of the object in the training image;

[0040] It should be noted that the above-mentioned marking position is specifically used to mark the position of the element in the activation feature matrix. For example, the marking position can be: the first row, the first column; it can also be the first row or the first column.

[0041] Step three: training the object recognition model using the multiple activated feature matrices with marked positions to obtain a trained object recognition model.

[0042] It should be noted that the above objects include pedestrians in the picture, but are not limited to pedestrians. For better understanding, the objects in the embodiments of the present application are taken as pedestrians. The global features of the above objects can be understood as the features of all limbs of the pedestrian, and the local features of the pedestrian can be understood as the features of a part of the limbs of the pedestrian, such as the head, arms, etc.

[0043] It should be noted that the above-mentioned step 2 has multiple execution methods. In an optional embodiment, it can be implemented in the following way: obtaining the target threshold corresponding to each activation feature matrix according to a preset algorithm; determining the binary matrix corresponding to each activation feature matrix according to the target threshold; clustering the binary matrix using a clustering algorithm to determine the mark position corresponding to each activation feature matrix.

[0044] That is to say, for each of the multiple activation feature matrices, clustering processing is required. For each activation feature matrix, a predetermined algorithm will be used to determine the target threshold corresponding to the activation feature evidence based on the element values ​​in the activation feature evidence. Then, the binary matrix corresponding to the activation feature matrix will be determined based on the target threshold. After the binary matrix of the activation feature matrix is ​​determined, the binary matrix will be clustered using a clustering algorithm to determine the mark position corresponding to the activation feature matrix.

[0045] Furthermore, in order to better understand the determination of the binary matrix corresponding to the activation feature matrix according to the target threshold, it can be achieved in the following way: create a first target matrix, wherein the first target matrix has the same matrix structure as the activation feature matrix, and all element values ​​in the first target matrix are 0; when there is a target element in the activation feature matrix, the element value of the position corresponding to the element in the first target matrix is ​​set to 1 to obtain a second target matrix, wherein the element value of the target element is greater than the target threshold; and determine the second target matrix as a binary matrix.

[0046] In other words, the structure of the activation feature matrix needs to be determined first. If the activation feature matrix is ​​a 24-order matrix, a 24-order first target matrix needs to be created, and all element values ​​of the first target matrix are set to 0. Then, starting from the first row and first column of the activation feature matrix, determine in turn whether the element values ​​of the elements at the corresponding positions of the activation feature matrix are greater than the target threshold. If so, set the element values ​​at the corresponding positions of the activation feature map in the first target matrix to 1, until the element values ​​of all elements in the activation feature matrix are determined, and then the binary matrix corresponding to the activation feature matrix can be determined according to the target threshold.

[0047] Further, in an optional embodiment, the binary matrix is ​​clustered using a clustering algorithm to determine the mark position corresponding to each activated feature matrix, which is achieved in the following manner: multiple position areas corresponding to the binary matrix are determined by a clustering algorithm, wherein the position areas include: a first position area, a second position area, and the ratio of elements with element values ​​of 1 in the first position area to all elements in the first position area is greater than a first preset ratio, and the ratio of elements with element values ​​of 0 in the second position area to all elements in the second position area is greater than a second preset ratio; connecting multiple first position areas and determining a first area of ​​the connected first position areas; connecting multiple second position areas and determining a second area of ​​the connected second position areas; determining the mark position of the position area with a larger area between the first area and the second area, and using the mark position of the position area as the mark position corresponding to each activated feature matrix.

[0048] For a better understanding, the following is a specific description. The neural network can cluster the binary matrix according to the clustering algorithm, and then determine multiple position areas from the binary matrix. For a better understanding of the above-mentioned first position area and second position area, the following is a specific description. Assuming that the binary matrix is ​​a 9-order matrix, it is determined through clustering that the binary matrix has 5 first position areas and 4 second position areas. For a better understanding, Figure 3 is a schematic diagram of a location area of ​​a target object recognition method according to an embodiment of the present invention. Figure 3This is just a simplified graph for understanding the scheme of the embodiment of the present application. The specific clustering result is not the case. It is assumed that the areas numbered 2, 4, 5, 6, and 8 are the first position areas, and the remaining numbered areas are the second position areas. There are 9 elements in the first position area and the second position area, and the element values ​​are 0 or 1. Among the 9 elements in the first position area, most of the element values ​​are 1, that is, the ratio of the number of elements with an element value of 1 to the number of elements in the first position area is greater than the first preset ratio, and the first preset ratio can be 0.8 or 0.9. Among the 9 elements in the second position area, most of the element values ​​are 0, that is, the ratio of the number of elements with an element value of 0 to the number of elements in the second position area is greater than the second preset ratio, and the second preset ratio can be 0.8 or 0.9. Then, multiple first position areas can be connected to determine the first area of ​​the connected first position area, and multiple second position areas can be connected to determine the second area of ​​the connected second position area. If the first area is greater than the second area, the mark position of the area corresponding to the first area is determined, and the mark position is used as the mark position corresponding to the activation feature matrix. If the second area is larger than the first area, the marking position of the area corresponding to the second area is determined, and the marking position is used as the marking position corresponding to the activation feature matrix. Figure 3 In , the marked positions of the regions numbered 2, 4, 5, 6, and 8 are used as the marked positions corresponding to the activation feature matrix.

[0049] After the corresponding marking positions of multiple activation feature matrices are determined, the object recognition model can be trained. Specifically, it is achieved in the following way: for each activation feature matrix with a marking position among the multiple activation feature matrices with marking positions, the weight of the data of each activation feature matrix with a marking position at the marking position is increased according to a preset rule to obtain multiple weights corresponding to the multiple activation feature matrices with marking positions; the object recognition model is trained according to the multiple weights and the multiple activation feature matrices with marking positions to obtain a trained object recognition model.

[0050] That is to say, the data of the activated feature matrix at the marked position is used to represent the global features of the object. If the object in the training picture is a person, it is used to represent the global features of the person. Therefore, when training the object recognition model, it is necessary to increase the weight of the data of each activated feature matrix at the marked position. This can reduce the training time of the model and increase the model's recognition rate of the object.

[0051] Furthermore, based on the trained object recognition model, the target object contained in the picture to be recognized is recognized, and it is necessary to obtain the target picture, wherein the target picture contains the target object to be recognized, and the image to be recognized contains multiple objects; the picture to be recognized and the target picture are input into the trained object recognition model to recognize the target object from the multiple objects in the image to be recognized.

[0052] That is, now there is a target image, and there is a target object in the target image. For better understanding, the object is understood as a person, that is, there is a person A in the target image. There is also a picture with recognition, and there are many people in the picture to be recognized. After the picture to be recognized and the target picture are input into the trained object recognition model, A can be recognized from the multiple people in the image to be recognized.

[0053] Obviously, the above-described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. In order to better understand the above-mentioned target object recognition method, the above-mentioned process is described below in conjunction with the embodiments, but it is not intended to limit the technical solution of the embodiments of the present invention, specifically:

[0054] In an optional embodiment, the present application discloses a pedestrian re-identification method based on feature clustering attention, which aims to enrich the expressive power of pedestrian features, improve the feature robustness and discriminability of pedestrian features, so that the attention area can focus on the overall area of ​​the pedestrian (equivalent to the pedestrian recognition model based on the traditional neural network in the above embodiment is trained according to the global features of the pedestrian), thereby improving the accuracy of pedestrian recognition.

[0055] To achieve the above objectives, a feature clustering attention module is designed, which can be inserted into the existing pedestrian re-identification method. Figure 4 is a flow chart of a target object identification method according to an embodiment of the present invention. It should be noted that: Figure 4 The dashed arrows in the middle represent the processes related to neural network training. The recognition process is as follows:

[0056] Step S402: obtaining a to-be-searched image (equivalent to the target image in the above embodiment) and a base image (equivalent to the to-be-recognized image in the above embodiment);

[0057] Step S404: input into a multi-branch convolutional network (equivalent to the trained object recognition model in the above embodiment);

[0058] Step S406: extracting global features and local features;

[0059] Step S408: feature fusion and similarity calculation;

[0060] Step S410: Pedestrians (equivalent to the objects in the above embodiment) are retrieved and visualized.

[0061] For better understanding, the detailed description is as follows:

[0062] S1: Building a hierarchical and progressive multi-branch network

[0063] The basic network structure of this network is a residual network structure similar to ResNet50, which mainly includes 4 residual block convolutional layers, denoted as L1, L2, L3 and L4 respectively. The multi-branch network implements a hierarchical sharing mechanism and contains four branches, namely P1, P2, P3 and P4, which share the network structure and parameters of the L1 layer and its previous layers.

[0064] The P1 branch is mainly composed of L2, L3 and L4. As a trunk branch, it does not perform horizontal stripe division. The P1 branch directly accesses the network after the shared L1 layer;

[0065] The P2 branch is mainly composed of L3 and L4 residual block convolution layers that have been deeply copied. The feature map output by the L2 layer of the P1 branch (equivalent to the feature matrix in the above embodiment) will be sent to the P2 branch. The P2 branch contains a total of 1 global feature sub-branch and 2 fine-grained local feature sub-branches; the local feature sub-branch is obtained by dividing the feature map output by L4 of P2 into horizontal stripe blocks that are equally divided into two.

[0066] The P3 branch mainly contains the L4 residual block convolution layer after deep copy. The feature map output by the L3 layer of the P2 branch will be sent to the P3 branch. The P3 branch contains a global feature sub-branch and two fine-grained local feature sub-branches. The local feature sub-branch is obtained by dividing the feature map output by the L4 of the P3 into three equal horizontal stripes.

[0067] The P4 branch mainly contains the L4 residual block convolution layer after deep copy. The feature map of the L4 output of the P4 branch will be divided into four equal horizontal stripe blocks, which belongs to a finer stripe segmentation and requires higher semantic information. Therefore, the feature map output by the L3 layer of the P1 branch will be sent to the P4 branch. The P4 branch contains a total of 1 global feature sub-branch and 4 fine-grained local feature sub-branches;

[0068] In addition to the residual convolution layer, each sub-branch of the above-mentioned Part branch also includes an independent sampling layer, feature embedding layer and classification layer. Among them, the sampling layer consists of maximum sampling and mean sampling operations. The output features of the two sampling operations are added to obtain the fused 2048-dimensional features and input them into the feature embedding layer. The embedding layer consists of a 1×1 convolution layer and a batch normalization BN layer, which is used to reduce the dimension of the features output by the sampling layer, and finally obtain 256-dimensional features.

[0069] The above multi-branch network will form 13 sub-branches, including 4 coarse-grained global feature extraction branches and 9 fine-grained local feature extraction branches.

[0070] The downsampling step size of the L4 layer of the P1 branch remains unchanged, that is, the scale of the output feature map of this layer is 1 / 2 of the scale of the input feature map of this layer. In order to make the input feature maps of the P2, P3 and P4 branches that perform stripe blocking retain more feature information, the downsampling step size of the L4 layer of these branches is set to stride=1. The scale of the input features and output features of the L4 layer remains unchanged.

[0071] From the above network structure description, we can see that the P2 branch is connected from the first convolution layer (i.e., L2) of the P1 branch, the P3 branch is connected from the first residual convolution layer (i.e., L3) of the P2 branch, and the P4 branch is connected from the second convolution layer (i.e., L3) of the P1 branch. This network topology design has the design characteristics of hierarchical sharing. For example, the P1 branch and the P2 branch share the L1 convolution layer and the L2 convolution layer of the residual network, while P2 and P3 share the L3 convolution layer in addition to the L1 and L2 convolution layers. Compared with the traditional design of connecting multiple branches from the same residual convolution layer, it can effectively reduce the number of network parameters and improve the training efficiency of the network.

[0072] S2: Building a feature attention module

[0073] Figure 5 is a schematic diagram of training a traditional pedestrian recognition model according to a target object recognition method according to an embodiment of the present invention, such as Figure 5 As shown in the figure, the existing pedestrian re-identification method only distinguishes the target based on the hottest area. Intuitively, this information is not sufficient and will lead to inadequate robustness. Therefore, this clustering attention module is designed, which connects the various non-continuous attention areas based on the density clustering DBSCAN algorithm, and then expands the attention area based on learnable parameters and expands the influence on this area.

[0074] The algorithm flow is as follows:

[0075] The activated feature map (equivalent to the activated feature matrix in the above embodiment) of each feature map (equivalent to the feature matrix in the above embodiment) in the batch has a dimension of [B, H, W] (equivalent to the target dimension in the above embodiment);

[0076] Automatically update the weight parameter w based on back propagation, with the dimension [B,H,W];

[0077] A core point threshold is set based on the activated feature map (equivalent to the target threshold in the above embodiment), and a binary map is established (equivalent to the binary matrix in the above embodiment); specifically, a threshold is selected in the feature map, and the binary map is determined according to the threshold, and values ​​less than the threshold are set to 0, and values ​​greater than the threshold are set to 1.

[0078] Clustering is performed based on the DBSCAN algorithm; specifically, the foreground, i.e., target features, are clustered using a clustering algorithm to ensure that the attention area is expanded.

[0079] Get the area with the largest area and update the feature map with the weight parameter w (equivalent to training the recognition model).

[0080] S3: Network training

[0081] Prepare training samples, use the training samples to train the multi-branch network model, and save it. In the method of the present invention, each branch of the multi-branch network performs a separate classification loss calculation, and each sub-branch performs an adaptive adjustment parameter γ to achieve the purpose of adaptively learning the spherical feature embedding space. In addition, a triple loss function is used for distance metric learning to improve the discriminability of features. Specifically, in addition to the global features output by the global feature branches of P1, P2, P3 and P4, the 2 fine-grained local features cascaded in the P2 branch, the 3 fine-grained local features cascaded in the P3 branch, and the 4 fine-grained local features cascaded in the P4 branch are also individually calculated for triple loss, that is, a total of 7 global features are subject to the constraints and training of triple loss. In addition, during the training process, considering the difference in feature embedding space learned by the classification loss of the spherical constraint and the distance metric loss, a training strategy is adopted in which only the classification loss is used in one training cycle and the classification loss and the distance metric loss are used in the next training cycle.

[0082] The above-mentioned fine-grained features include 2 local features obtained by dividing the stripe block into two equal parts, 3 local features obtained by dividing the stripe block into three equal parts, and 4 local features obtained by dividing the stripe block into four equal parts, for a total of 9 fine-grained local features.

[0083] The preparation of training samples includes detecting pedestrian targets from scene images and labeling pedestrian samples. The images input to the multi-branch network will be adjusted to a certain size.

[0084] The training of the multi-branch network model using the training samples includes initializing the multi-branch network parameters, training the network for a specified number of cycles using the Adam optimizer, and saving the network parameters after the training is completed. The trained network and parameters can be used for model deployment for inference and testing.

[0085] S4: Feature fusion and similarity calculation

[0086] By dividing the human body feature map into horizontal stripe blocks, combined with the multi-branch network design and training strategy of the present invention, it is possible to extract feature information with rich granularity, enhance the robustness and discriminability of pedestrian features, and thus improve the accuracy of pedestrian recognition. When the network training is completed for inference or testing, the global or local features output by the 13 sub-branches of the multi-branch network are cascaded and normalized as the fused multi-granularity pedestrian feature expression, and the similarity of the pedestrian features is calculated and sorted. Since the features have been normalized, there is no problem of choosing between the Euclidean distance measure and the cosine distance measure. The cosine similarity calculation formula for pedestrian features is: The smaller the calculated value, the greater the similarity.

[0087] S5: Pedestrian retrieval and visualization

[0088] In order to perform the pedestrian re-identification task, after feature extraction and similarity calculation for the query image Query obtained by the data acquisition device and the base library Gallery to be retrieved, the similarity score is obtained according to the S4 calculation formula, the retrieval results are sorted, and the sorting results and scores matching the Query are visualized.

[0089] In addition, the above technical solutions of the embodiments of the present invention expand the pedestrian's attention area, improve the ability to express pedestrian features, and thus improve the recognition accuracy; improve the feature robustness and discriminability of the joint expression of global features and local features, thereby improving the generalization performance; the network can quickly learn the target area, so that the learning process can converge quickly and reduce training time.

[0090] The embodiment of the present application introduces the clustering algorithm in traditional machine learning into the deep learning network to enhance the interpretability of deep learning; the clustering algorithm is introduced into the field of attention mechanism, which can effectively make the target focus on the specified area compared with the channel attention, spatial attention and other methods in the prior art, thereby greatly improving the training efficiency of the network.

[0091] In the process of training the pedestrian recognition model, the accuracy of model recognition can also be improved by changing the loss function. In terms of loss function, the classic cross entropy classification loss function is used to constrain the training of the convolutional neural network. Specifically, the spherical constraint classification loss function can be applied to the pedestrian re-identification task to help the neural network learn a more intuitive feature embedding space. In addition, when the amount of data is limited, a loss function based on distance metric learning can also be introduced, including: Triplet Loss, which adds a distance constraint factor (Margin) to make the feature clusters of pedestrians with the same identity more cohesive and the class centers of different pedestrian categories more separated, thereby helping the network learn more discriminative global features or local fine-grained features.

[0092] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0093] In the present embodiment, a target object recognition device is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the descriptions that have been made are not repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the equipment described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0094] Figure 6 1 is a structural block diagram of a target object recognition device according to an embodiment of the present invention (I), the device comprising:

[0095] An acquisition module 62 is used to acquire a picture to be identified containing a target object;

[0096] The recognition module 64 is used to recognize the target object contained in the picture to be recognized based on the trained object recognition model; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training picture corresponding to the global feature with increased weight and the target object identified in the training picture.

[0097] Through the above module, a picture to be identified containing a target object is obtained; based on the trained object recognition model, the target object contained in the picture to be identified is identified; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training picture corresponding to a global feature with an increased weight and a target object identified in the training picture. In other words, the object recognition model will focus on training the global features of the target object in the training picture. The above technical solution solves the problem that the traditional pedestrian recognition model based on a neural network is trained according to the local features of pedestrians, resulting in poor recognition ability of pedestrians. Then the pedestrian recognition model is trained according to the global features of pedestrians, so that the trained pedestrian recognition model can accurately identify pedestrians, thereby improving the recognition accuracy.

[0098] also, Figure 7 is a structural block diagram of a target object recognition device according to an embodiment of the present invention (II), Figure 7 A clustering training module 66 is also included.

[0099] Optionally, the training module 66 is also used to divide the feature matrix of the training picture according to the target dimension to obtain multiple activation feature matrices, wherein the training picture is used to train the object recognition model; for each of the multiple activation feature matrices, a clustering algorithm is used to perform clustering processing on each activation feature matrix to obtain multiple activation feature matrices with marked positions corresponding to the multiple activation feature matrices, wherein the data of the activation feature matrix at the marked positions is used to indicate the global features of the objects in the training picture; the object recognition model is trained by the multiple activation feature matrices with marked positions to obtain a trained object recognition model.

[0100] Optionally, the training module 66 is also used to obtain the target threshold corresponding to each activation feature matrix according to a preset algorithm; determine the binary matrix corresponding to each activation feature matrix according to the target threshold; and cluster the binary matrix using a clustering algorithm to determine the marking position corresponding to each activation feature matrix.

[0101] Optionally, the training module 66 is also used to create a first target matrix, wherein the first target matrix has the same matrix structure as the activation feature matrix, and all element values ​​in the first target matrix are 0; when there is a target element in the activation feature matrix, the element value of the position corresponding to the element in the first target matrix is ​​set to 1 to obtain a second target matrix, wherein the element value of the target element is greater than the target threshold; and the second target matrix is ​​determined to be a binary matrix.

[0102] Optionally, the training module 66 is also used to determine multiple position areas corresponding to the binary matrix through a clustering algorithm, wherein the position areas include: a first position area, a second position area, the ratio of elements with element values ​​of 1 in the first position area to all elements in the first position area is greater than a first preset ratio, and the ratio of elements with element values ​​of 0 in the second position area to all elements in the second position area is greater than a second preset ratio; connecting multiple first position areas and determining a first area of ​​the connected first position areas; connecting multiple second position areas and determining a second area of ​​the connected second position areas; determining a mark position of the position area with a larger area between the first area and the second area, and using the mark position of the position area as the mark position corresponding to each activated feature matrix.

[0103] Optionally, the training module 66 is also used to increase the weight of the data of each activation feature matrix with a marked position in the multiple activation feature matrices with marked positions according to a preset rule, so as to obtain multiple weights corresponding to the multiple activation feature matrices with marked positions; and train the object recognition model according to the multiple weights and the multiple activation feature matrices with marked positions to obtain a trained object recognition model.

[0104] Optionally, the recognition module 64 is also used to obtain a target picture, wherein the target picture contains the target object to be identified, and the image to be identified contains multiple objects; the image to be identified and the target picture are input into the trained object recognition model to identify the target object from the multiple objects in the image to be identified.

[0105] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0106] S1, obtain the image to be identified containing the target object;

[0107] S2, based on the trained object recognition model, identifying the target object contained in the image to be identified; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training image corresponding to a global feature with an increased weight and a target object identified in the training image.

[0108] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0109] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0110] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0111] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0112] S1, obtain the image to be identified containing the target object;

[0113] S2, based on the trained object recognition model, identifying the target object contained in the image to be identified; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training image corresponding to a global feature with an increased weight and a target object identified in the training image.

[0114] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0115] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0116] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying a target object, characterized in that: include: Get the image to be identified containing the target object; Based on the trained object recognition model, the target object contained in the picture to be recognized is recognized; wherein the trained object recognition model is trained by machine learning using multiple sets of data, and each set of data in the multiple sets of data includes: a training picture corresponding to the global feature with increased weight and a target object identified in the training picture; Wherein, before identifying the target object contained in the to-be-identified picture based on the trained object recognition model, the method further includes: Determine multiple position areas corresponding to the binary matrix by a clustering algorithm, wherein the position areas include: a first position area and a second position area, the ratio of elements with element values ​​of 1 in the first position area to all elements in the first position area is greater than a first preset ratio, and the ratio of elements with element values ​​of 0 in the second position area to all elements in the second position area is greater than a second preset ratio; connecting a plurality of the first location areas, and determining a first area of ​​the connected first location areas; connecting a plurality of the second position areas, and determining a second area of ​​the connected second position areas; Determine the mark position of the location area with the larger area between the first area and the second area, and use the mark position of the location area as the mark position corresponding to each activation feature matrix; wherein the binary matrix is ​​determined based on the target threshold corresponding to each activation feature matrix obtained based on a preset algorithm; each activation feature matrix is ​​obtained by dividing the feature matrix of the training image based on the target dimension; the data of the activation feature matrix at the mark position is used to indicate the global features of the object in the training image.

2. The method according to claim 1, characterized in that Before identifying the target object contained in the to-be-identified picture based on the trained object recognition model, the method further includes: The object recognition model is trained by using a plurality of activation feature matrices with marked positions to obtain a trained object recognition model.

3. The method according to claim 1, characterized in that Determining a binary matrix corresponding to each activation feature matrix according to the target threshold comprises: Creating a first target matrix, wherein the first target matrix has the same matrix structure as the activation feature matrix, and all element values ​​in the first target matrix are 0; In the case where a target element exists in the activation feature matrix, the element value of the position corresponding to the element in the first target matrix is ​​set to 1, so as to obtain a second target matrix, wherein the element value of the target element exists to be greater than the target threshold; The second target matrix is ​​determined as a binary matrix.

4. The method according to claim 2, characterized in that: The object recognition model is trained by using a plurality of activation feature matrices with marked positions to obtain a trained object recognition model, including: For each activation feature matrix with a marked position among the plurality of activation feature matrices with marked positions, increase the weight of the data of each activation feature matrix with a marked position at the marked position according to a preset rule to obtain a plurality of weights corresponding to the plurality of activation feature matrices with marked positions; The object recognition model is trained according to the multiple weights and the multiple activation feature matrices with marked positions to obtain a trained object recognition model.

5. The method according to claim 1, characterized in that Based on the trained object recognition model, the target object contained in the to-be-recognized picture is recognized, including: Acquire a target image, wherein the target image contains the target object to be identified, and the image to be identified contains multiple objects; The image to be identified and the target image are input into the trained object recognition model to identify the target object from multiple objects in the image to be identified.

6. A target object recognition device, characterized in that: include: An acquisition module is used to acquire a picture to be identified containing a target object; A recognition module, for recognizing a target object contained in the image to be recognized based on a trained object recognition model; wherein the trained object recognition model is trained by machine learning using multiple sets of data, each set of data in the multiple sets of data including: a training image corresponding to a global feature with an increased weight and a target object identified in the training image; The device also includes: a training module, which is used to determine multiple position areas corresponding to a binary matrix through a clustering algorithm, wherein the position areas include: a first position area, a second position area, and the ratio of elements with element values ​​of 1 in the first position area to all elements in the first position area is greater than a first preset ratio, and the ratio of elements with element values ​​of 0 in the second position area to all elements in the second position area is greater than a second preset ratio; connecting multiple first position areas and determining a first area of ​​the connected first position area; connecting multiple second position areas and determining a second area of ​​the connected second position area; determining a mark position of a position area with a larger area between the first area and the second area, and using the mark position of the position area as the mark position corresponding to each activation feature matrix; wherein the binary matrix is ​​determined based on a target threshold corresponding to each activation feature matrix obtained by a preset algorithm; each activation feature matrix is ​​obtained by dividing the feature matrix of the training image based on a target dimension; and the data of the activation feature matrix at the mark position is used to indicate the global features of the object in the training image.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 5 when executed.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 5 through the computer program.

Images