Pedestrian image recognition method and device

Abstract

The present invention provides a pedestrian image recognition method and device, the method comprising: obtaining multiple pedestrian images; extracting a two-dimensional feature matrix of each pedestrian image; obtaining the most pixel-level attention feature vector of each pedestrian image according to the two-dimensional feature matrix of each pedestrian image; obtaining a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories; obtaining the most pixel-level attention feature matrix of each pedestrian image according to the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image; obtaining a third feature vector according to the most pixel-level attention feature vector of each pedestrian image and the most pixel-level attention feature matrix; based on the third feature vector, finding and outputting the pedestrian image closest to the target pedestrian image from multiple pedestrian images. The present invention can accurately identify the target pedestrian from a large number of pedestrian images, and has strong robustness.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a pedestrian image recognition method and device. Background Art

[0002] With the nationwide layout of retirement communities, there are more and more retirement communities, and the activities in the communities are also rich and colorful. The elderly need a variety of customized services and lives. As the number of community cameras gradually increases, more and more videos are produced. How to more efficiently use these photos or videos to better serve high-net-worth customers living in the community in the future is an increasingly urgent problem that needs to be solved. Therefore, identifying pedestrians from these photos or videos is an urgent problem to be solved.

[0003] In the existing technology, the ReID algorithm is often used for pedestrian recognition. However, the data of the ReID algorithm in actual application scenarios is very complex. Due to the differences between different camera devices, pedestrians have both rigid and flexible characteristics, and their appearance is easily affected by clothing, scale, occlusion, posture and perspective, so high requirements are placed on the ReID algorithm. Because when the algorithm is applied to the scene, there are often more complex scenes and occlusion problems, which makes the pedestrian image recognition of the ReID algorithm not robust. Summary of the invention

[0004] The embodiment of the present invention provides a pedestrian image recognition method for accurately identifying a target pedestrian from a large number of pedestrian images with strong robustness. The method includes:

[0005] Acquire multiple pedestrian images;

[0006] Extract the two-dimensional feature matrix F of each pedestrian image;

[0007] According to the two-dimensional feature matrix F of each pedestrian image, the most pixel-level attention feature vector F1 of each pedestrian image is obtained;

[0008] Obtain a preset number of categories and a pedestrian local category matrix P pre-trained based on the number of categories;

[0009] According to the pedestrian local category matrix and the two-dimensional feature matrix F of each pedestrian image, the most pixel-level attention feature matrix of each pedestrian image is obtained;

[0010] Obtain a third feature vector F4 according to the most pixel-level attention feature vector F1 and the most pixel-level attention feature matrix of each pedestrian image;

[0011] Based on the third eigenvector, the pedestrian image closest to the target pedestrian image is found from multiple pedestrian images and output.

[0012] The embodiment of the present invention provides a pedestrian image recognition device, which is used to accurately recognize a target pedestrian from a large number of pedestrian images and has strong robustness. The device includes:

[0013] A pedestrian image acquisition module, used to acquire multiple pedestrian images;

[0014] A two-dimensional feature extraction module is used to extract a two-dimensional feature matrix F of each pedestrian image;

[0015] A pixel-level attention feature vector extraction module is used to obtain a pixel-level attention feature vector F1 of each pedestrian image according to a two-dimensional feature matrix F of each pedestrian image;

[0016] A pedestrian local category matrix acquisition module is used to obtain a preset number of categories and a pedestrian local category matrix P pre-trained based on the number of categories;

[0017] A pixel-level attention feature matrix acquisition module is used to obtain the pixel-level attention feature matrix of each pedestrian image according to the pedestrian local category matrix and the two-dimensional feature matrix F of each pedestrian image;

[0018] A third feature vector obtaining module, used for obtaining a third feature vector F4 according to the most pixel-level attention feature vector F1 and the most pixel-level attention feature matrix of each pedestrian image;

[0019] The retrieval module is used to find the pedestrian image closest to the target pedestrian image from multiple pedestrian images based on the third eigenvector and output it.

[0020] An embodiment of the present invention further proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned pedestrian image recognition method when executing the computer program.

[0021] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned pedestrian image recognition method.

[0022] In an embodiment of the present invention, multiple pedestrian images are obtained; a two-dimensional feature matrix of each pedestrian image is extracted; based on the two-dimensional feature matrix of each pedestrian image, a most pixel-level attention feature vector of each pedestrian image is obtained; a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories are obtained; based on the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image, a most pixel-level attention feature matrix of each pedestrian image is obtained; based on the most pixel-level attention feature vector of each pedestrian image and the most pixel-level attention feature matrix, a third feature vector is obtained; based on the third feature vector, a pedestrian image closest to the target pedestrian image is found from multiple pedestrian images and output. In the above process, the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image can be obtained. The principle is to aggregate pixels of the same attribute, and the pre-trained pedestrian local category matrix makes it possible to find features of different parts of the human body in a weakly supervised manner, thereby improving the robustness in complex scenes and improving the accuracy of pedestrian recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0024] Figure 1 Flow chart of a method for pedestrian image recognition in an embodiment of the present invention;

[0025] Figure 2 A flowchart of obtaining a pedestrian image in an embodiment of the present invention;

[0026] Figure 3 This is a flow chart of extracting a two-dimensional feature matrix in an embodiment of the present invention;

[0027] Figure 4 A flowchart of obtaining the most pixel-level attention feature vector in an embodiment of the present invention;

[0028] Figure 5 A flowchart of obtaining a pixel-level attention feature matrix in an embodiment of the present invention;

[0029] Figure 6 This is a flow chart of obtaining the third eigenvector in an embodiment of the present invention;

[0030] Figure 7 is a flowchart of image retrieval in an embodiment of the present invention;

[0031] Figure 8Schematic diagram of a pedestrian image recognition device according to an embodiment of the present invention;

[0032] Fig. 9 Schematic diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0034] In the description of this specification, the terms "include", "including", "have", "contain", etc. are all open terms, which mean including but not limited to. The descriptions with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. The order of steps involved in each embodiment is used to schematically illustrate the implementation of the present application, and the order of steps is not limited and can be appropriately adjusted as needed.

[0035] Figure 1 FIG. 1 is a flow chart of a method for identifying a pedestrian image in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0036] Step 101, obtaining multiple pedestrian images;

[0037] Step 102, extracting a two-dimensional feature matrix F of each pedestrian image;

[0038] Step 103, obtaining the most pixel-level attention feature vector F1 of each pedestrian image according to the two-dimensional feature matrix F of each pedestrian image;

[0039] Step 104, obtaining a preset number of categories and a pedestrian local category matrix P pre-trained based on the number of categories;

[0040] Step 105, obtaining the pixel-level attention feature matrix of each pedestrian image according to the pedestrian local category matrix and the two-dimensional feature matrix F of each pedestrian image;

[0041] Step 106, obtaining a third feature vector F4 according to the most pixel-level attention feature vector F1 and the most pixel-level attention feature matrix of each pedestrian image;

[0042] Step 107 : Based on the third eigenvector, find a pedestrian image closest to the target pedestrian image from multiple pedestrian images and output the pedestrian image.

[0043] In an embodiment of the present invention, the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image can be obtained. The principle is to aggregate pixels with the same attributes, and the pre-trained pedestrian local category matrix makes it possible to find the features of different parts of the human body in a weakly supervised manner, thereby improving the robustness in complex scenes and improving the accuracy of pedestrian recognition.

[0044] In the specific implementation, in steps 102 to 106, it is implemented based on the pedestrian pixel and local attention model PPAM (Pixel-Aware and Part-Aware Atention Model), which is an optimization of the ReID algorithm. The PPAM model can be based on the attention model extraction method of the pedestrian image to obtain the most pixel-level attention feature vector F1. At the same time, it further utilizes the pixel-level features to pre-train a pedestrian local category matrix to obtain the most pixel-level attention feature matrix. Finally, a third feature vector can be obtained for retrieval, that is, to find the pedestrian image closest to the target pedestrian image and output it.

[0045] Figure 2 The flowchart of obtaining pedestrian images in an embodiment of the present invention is as follows. In one embodiment, obtaining multiple pedestrian images includes:

[0046] Step 201, obtaining video data within a set time period;

[0047] Step 202, extracting frames from the video data to obtain a plurality of initial images;

[0048] Step 203, obtaining target frames of multiple pedestrians from multiple initial images;

[0049] Step 204 , after cropping the target frame of each pedestrian into a pedestrian picture, transform it into a pedestrian image of a preset size.

[0050] In the above embodiment, after obtaining the video data of the camera of the set time length, the video data is framed to obtain multiple initial images. After that, a target detection algorithm is used to obtain the target frame of the pedestrian. There may be multiple target detection algorithms, for example, the yolov5 algorithm. After obtaining the target frame of the pedestrian, the pedestrian is cropped into a pedestrian picture according to the target frame, and the picture is uniformly transformed into a (256, 128, 3) image and a pedestrian image of a preset size.

[0051] Figure 3This is a flow chart of extracting a two-dimensional feature matrix in an embodiment of the present invention. In one embodiment, extracting a two-dimensional feature matrix of each pedestrian image includes:

[0052] Step 301, extracting a feature matrix of each pedestrian image;

[0053] Step 302: Flatten the feature matrix of each pedestrian image into a corresponding two-dimensional feature matrix.

[0054] In the above embodiment, when extracting the feature matrix of each pedestrian image, the model used is not unique. For example, resnet50 can be used, and the last pooling layer and fully connected layer of resnet50 are removed to obtain a feature matrix, for example, a feature matrix with a feature length, width and number of channels of (16, 8, 2048).

[0055] The background area of ​​the pedestrian image has different characteristics, which increases the difficulty of obtaining robust features of the target pedestrian. Therefore, the embodiment of the present invention adopts the self-attention mechanism in the PPAM model to capture the complete image context information, and a pixel-level context-aware feature matrix can be obtained. Among them, a (1, 1) convolution is used to convert the feature matrix into a smaller dimension (16, 8, 512), and then the feature matrix is ​​flattened into a two-dimensional feature matrix F with a size of (128, 512), where 128 represents the multiplication of length and width, and 512 represents the number of channels.

[0056] Figure 4 The flowchart of obtaining the most pixel-level attention feature vector in an embodiment of the present invention is as follows. In one embodiment, according to the two-dimensional feature matrix of each pedestrian image, obtaining the most pixel-level attention feature vector of each pedestrian image includes:

[0057] Step 401, calculating the attention weight of each pixel in each two-dimensional feature matrix;

[0058] Step 402, for each pedestrian image, based on the attention weights between each pixel in the two-dimensional feature matrix corresponding to the pedestrian image, a pixel feature vector that aggregates the pixel context information of all positions in the two-dimensional feature matrix is ​​calculated;

[0059] Step 403, concatenate all pixel feature vectors in the two-dimensional feature matrix corresponding to each pedestrian image, and use two fully connected layers to obtain the most pixel-level attention feature vector of each pedestrian image.

[0060] In the above embodiment, the attention weight between each pixel in the two-dimensional feature matrix F is first calculated according to the following formula:

[0061]

[0062]

[0063] Among them, s i，j is the attention weight between pixel i, i, j∈1, 2, 3.....128; f i is the feature vector of pixel i in the two-dimensional feature matrix F; W q , W k There are two projection matrices, both of size (512, 512).

[0064] Then, the following formula is used to calculate the pixel feature vector that aggregates the pixel context information of all positions in the two-dimensional feature matrix:

[0065]

[0066] in, is the pixel feature vector, size is (1, 512); W v is the projection matrix, size is (512, 512).

[0067] Finally, the pixel-level attention feature vector F1 of each pedestrian image is obtained, and its size is (128, 512).

[0068] In step 104, a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories are obtained. The pedestrian local category matrix is ​​used to extract local categories. In the embodiment of the present invention, the pedestrian image is divided into multiple categories using an attention mechanism. The number of categories is related to the complexity of the scene. For example, it can be divided into 8 parts, that is, the number of categories is 8. Therefore, the size of the trained pedestrian local category matrix P is (8, 512).

[0069] Figure 5 The flowchart of obtaining the most pixel-level attention feature matrix in an embodiment of the present invention is as follows. In one embodiment, according to the pedestrian local category matrix and the two-dimensional feature matrix F of each pedestrian image, the most pixel-level attention feature matrix of each pedestrian image is obtained, including:

[0070] Step 501: for each pedestrian image, the weight parameter p of each category is calculated according to the pedestrian local category matrix P and the two-dimensional feature matrix F corresponding to the pedestrian image. i，j ;

[0071] Step 502: According to the weight parameter p of each category i，j , calculate the pixel feature vector that aggregates all category information corresponding to each pedestrian image

[0072] Step 503: All the category feature vectors corresponding to each pedestrian image are They are concatenated together and two fully connected layers are used to obtain the pixel-level attention feature matrix.

[0073] In the above embodiment, the following formula is used to calculate the weight parameter p of each category: i，j :

[0074]

[0075]

[0076] Among them, p i，j is the weight parameter for each category, i∈1, 2, 3.....8; j∈1, 2, 3.....128; p is the feature matrix of each category; f is the feature vector of each pixel in the two-dimensional feature matrix F; W qq , W kk There are two projection matrices, both of size (512, 512).

[0077] Use the following formula, according to the weight parameter p of each category i，j , calculate the pixel feature vector that aggregates all category information corresponding to each pedestrian image

[0078]

[0079] in, The size of W is (1, 512); vv is the projection matrix, with a size of (512, 512).

[0080] Figure 6 This is a flow chart of obtaining the third eigenvector in an embodiment of the present invention. In one embodiment, the third eigenvector F4 is obtained according to the most pixel-level attention feature vector F1 and the most pixel-level attention feature matrix of each pedestrian image, including:

[0081] Step 601, performing global average pooling on the most pixel-level attention feature matrix corresponding to each pedestrian image to obtain a category information vector P1;

[0082] Step 602, performing global average pooling on the most pixel-level attention feature vector F1 of each pedestrian image to obtain a first feature vector F2;

[0083] Step 603, the category information vector P1 of each pedestrian image is multiplied by the first feature vector F2 using the sigmoid function activation point to obtain the second feature vector F3;

[0084] Step 604: concatenate the second feature vector F3 and the first feature vector F2 of each pedestrian image to obtain a third feature vector F4.

[0085] In the above embodiment, the size of F2 is (1, 512) and the size of F4 is (1, 1024).

[0086] Figure 7 Flow chart of image retrieval in an embodiment of the present invention. In one embodiment, based on the third feature vector, a pedestrian image closest to a target pedestrian image is found from multiple pedestrian images, including:

[0087] Step 701, respectively calculating the Euclidean distance between the third feature vector F4 of each pedestrian image and the target pedestrian image;

[0088] Step 702 : Determine the pedestrian image with the smallest Euclidean distance as the pedestrian image closest to the target pedestrian image.

[0089] In the above embodiment, steps 701 and 702 may use any efficient and fast feature vector search tool, such as milvus.

[0090] The pedestrian image recognition method proposed in the embodiment of the present invention can be applied to the following two scenarios.

[0091] Scenario 1: When an elderly person is lost, recent photos of the elderly person are found, and the elderly person's coding features are extracted as the target pedestrian image. Video data from the community and nearby cameras are retrieved and compared with the target pedestrian image to identify all pedestrian images of the elderly person, thereby finding all the elderly person's movement trajectories and the last scene they passed through, providing strong clues for finding the elderly person or searching for the elderly person.

[0092] Scenario 2: There are many medical models currently, such as the fall prevention model, which can provide a description of the risk of the elderly falling through various indicators. The improvement and accuracy of the model requires data of more dimensions, such as the graphics of the elderly. Therefore, the method of the present invention can be used to quickly annotate and analyze the movement trajectories of the elderly, and to extract more movement characteristics of the elderly, such as the number of exercises, the number of times they go out, the time they go out, etc., to provide more data for the algorithm personnel of the fall prevention model, thereby providing better medical predictive capabilities for the elderly care community.

[0093] In the method proposed in the embodiment of the present invention, multiple pedestrian images are obtained; a two-dimensional feature matrix of each pedestrian image is extracted; based on the two-dimensional feature matrix of each pedestrian image, the most pixel-level attention feature vector of each pedestrian image is obtained; a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories are obtained; based on the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image, the most pixel-level attention feature matrix of each pedestrian image is obtained; based on the most pixel-level attention feature vector of each pedestrian image and the most pixel-level attention feature matrix, a third feature vector is obtained; based on the third feature vector, the pedestrian image closest to the target pedestrian image is found from multiple pedestrian images and output. In the above process, the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image can be obtained. The principle is to aggregate pixels with the same attributes, and the pre-trained pedestrian local category matrix makes it possible to find the features of different parts of the human body in a weakly supervised manner, thereby improving the robustness in complex scenes and improving the accuracy of pedestrian recognition.

[0094] The embodiment of the present invention further provides a pedestrian image recognition device, the principle of which is similar to that of the pedestrian image recognition method, and will not be described in detail here.

[0095] Figure 8 Schematic diagram of a pedestrian image recognition device according to an embodiment of the present invention. Figure 8 As shown, the device comprises:

[0096] A pedestrian image acquisition module 801 is used to acquire multiple pedestrian images;

[0097] A two-dimensional feature extraction module 802 is used to extract a two-dimensional feature matrix of each pedestrian image;

[0098] The most pixel-level attention feature vector extraction module 803 is used to obtain the most pixel-level attention feature vector of each pedestrian image according to the two-dimensional feature matrix of each pedestrian image;

[0099] A pedestrian local category matrix obtaining module 804 is used to obtain a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories;

[0100] The most pixel-level attention feature matrix obtaining module 805 is used to obtain the most pixel-level attention feature matrix of each pedestrian image according to the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image;

[0101] A third feature vector obtaining module 806 is used to obtain a third feature vector according to the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image;

[0102] The retrieval module 807 is used to find the pedestrian image closest to the target pedestrian image from multiple pedestrian images based on the third feature vector and output the pedestrian image.

[0103] In one embodiment, the pedestrian image acquisition module is specifically used for:

[0104] Get video data within the set duration;

[0105] Extracting frames from the video data to obtain a plurality of initial images;

[0106] Obtain target frames of multiple pedestrians from multiple initial images;

[0107] After cropping the target frame of each pedestrian into a pedestrian picture, it is transformed into a pedestrian image of a preset size.

[0108] In one embodiment, the two-dimensional feature extraction module is specifically used for:

[0109] Extract the feature matrix of each pedestrian image;

[0110] Flatten the feature matrix of each pedestrian image into a corresponding two-dimensional feature matrix.

[0111] In one embodiment, the pixel-level attention feature vector extraction module is specifically used to:

[0112] Calculate the attention weights for each pixel in each two-dimensional feature matrix;

[0113] For each pedestrian image, based on the attention weights between each pixel in the two-dimensional feature matrix corresponding to the pedestrian image, a pixel feature vector that aggregates the pixel context information of all positions in the two-dimensional feature matrix is ​​calculated;

[0114] All pixel feature vectors in the two-dimensional feature matrix corresponding to each pedestrian image are concatenated together, and two fully connected layers are used to obtain the most pixel-level attention feature vector for each pedestrian image.

[0115] In one embodiment, the pixel-level attention feature matrix acquisition module is specifically used to:

[0116] For each pedestrian image, a weight parameter of each category is calculated according to the pedestrian local category matrix and the two-dimensional feature matrix corresponding to the pedestrian image;

[0117] According to the weight parameters of each category, the pixel feature vector that aggregates all category information corresponding to each pedestrian image is calculated;

[0118] All category feature vectors corresponding to each pedestrian image are concatenated together, and two fully connected layers are used to obtain the pixel-level attention feature matrix.

[0119] In one embodiment, the third feature vector obtaining module is specifically used for:

[0120] Perform global average pooling on the pixel-level attention feature matrix corresponding to each pedestrian image to obtain the category information vector;

[0121] Perform global average pooling on the most pixel-level attention feature vector of each pedestrian image to obtain the first feature vector;

[0122] The category information vector of each pedestrian image is multiplied by the first eigenvector using the sigmoid function to activate the dot multiplication to obtain the second eigenvector;

[0123] The second eigenvector and the first eigenvector of each pedestrian image are concatenated together to obtain the third eigenvector.

[0124] In one embodiment, the retrieval module is specifically used to:

[0125] Calculate the Euclidean distance between the third eigenvector of each pedestrian image and the target pedestrian image respectively;

[0126] The pedestrian image with the smallest Euclidean distance is determined to be the pedestrian image closest to the target pedestrian image.

[0127] To summarize, in the device proposed in the embodiment of the present invention, the pedestrian image acquisition module is used to acquire multiple pedestrian images; the two-dimensional feature extraction module is used to extract the two-dimensional feature matrix of each pedestrian image; the most pixel-level attention feature vector extraction module is used to obtain the most pixel-level attention feature vector of each pedestrian image based on the two-dimensional feature matrix of each pedestrian image; the pedestrian local category matrix acquisition module is used to obtain a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories; the most pixel-level attention feature matrix acquisition module is used to obtain the most pixel-level attention feature matrix of each pedestrian image based on the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image; the third feature vector acquisition module is used to obtain the third feature vector based on the most pixel-level attention feature vector of each pedestrian image and the most pixel-level attention feature matrix; the retrieval module is used to find and output the pedestrian image closest to the target pedestrian image from multiple pedestrian images based on the third feature vector. In the above process, the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image can be obtained. The principle is to aggregate pixels with the same attributes, and the pre-trained pedestrian local category matrix makes it possible to find the features of different parts of the human body in a weakly supervised manner, thereby improving the robustness in complex scenes and improving the accuracy of pedestrian recognition.

[0128] An embodiment of the present application also provides a computer device, Fig. 9Schematic diagram of a computer device in an embodiment of the present invention. The computer device can implement all steps in the pedestrian image recognition method in the above embodiment. The electronic device specifically includes the following contents:

[0129] Processor (processor) 901, memory (memory) 902, communication interface (CommunicationsInterface) 903 and bus 904;

[0130] The processor 901, the memory 902, and the communication interface 903 communicate with each other through the bus 904; the communication interface 903 is used to realize information transmission between relevant devices such as the server-side device, the detection device, and the user-side device;

[0131] The processor 901 is used to call the computer program in the memory 902, and when the processor executes the computer program, all the steps in the pedestrian image recognition method in the above embodiment are implemented.

[0132] An embodiment of the present application also provides a computer-readable storage medium that can implement all the steps in the pedestrian image recognition method in the above embodiment. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all the steps in the pedestrian image recognition method in the above embodiment.

[0133] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A pedestrian image recognition method, characterized in that: include: Acquire multiple pedestrian images; Extract the two-dimensional feature matrix of each pedestrian image; According to the two-dimensional feature matrix of each pedestrian image, the most pixel-level attention feature vector of each pedestrian image is obtained; The pixel-level attention feature vector is obtained by calculating the pixel feature vectors that aggregate the pixel context information of all positions of the two-dimensional feature matrix based on the attention weight between each pixel and splicing them together using two fully connected layers; Obtaining a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories; According to the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image, the most pixel-level attention feature matrix of each pedestrian image is obtained; The pixel-level attention feature matrix of each pedestrian image is obtained by calculating the weight parameters of each category of the pedestrian local category matrix, calculating the pixel feature vectors corresponding to each pedestrian image that aggregate all category information, and then splicing them together using two fully connected layers; Obtaining a third feature vector according to the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image; Based on the third eigenvector, the pedestrian image closest to the target pedestrian image is found from multiple pedestrian images and output.

2. The pedestrian image recognition method according to claim 1, characterized in that: Get multiple pedestrian images, including: Get video data within the set duration; Extracting frames from the video data to obtain a plurality of initial images; Obtain target frames of multiple pedestrians from multiple initial images; After cropping the target frame of each pedestrian into a pedestrian picture, it is transformed into a pedestrian image of a preset size.

3. The pedestrian image recognition method according to claim 1, characterized in that: Extract the two-dimensional feature matrix of each pedestrian image, including: Extract the feature matrix of each pedestrian image; Flatten the feature matrix of each pedestrian image into a corresponding two-dimensional feature matrix.

4. The pedestrian image recognition method according to claim 1, characterized in that: According to the two-dimensional feature matrix of each pedestrian image, the most pixel-level attention feature vector of each pedestrian image is obtained, including: Calculate the attention weights for each pixel in each two-dimensional feature matrix; For each pedestrian image, based on the attention weights between each pixel in the two-dimensional feature matrix corresponding to the pedestrian image, a pixel feature vector that aggregates the pixel context information of all positions in the two-dimensional feature matrix is ​​calculated; All pixel feature vectors in the two-dimensional feature matrix corresponding to each pedestrian image are concatenated together, and two fully connected layers are used to obtain the most pixel-level attention feature vector for each pedestrian image.

5. The pedestrian image recognition method according to claim 1, characterized in that: According to the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image, the most pixel-level attention feature matrix of each pedestrian image is obtained, including: For each pedestrian image, a weight parameter of each category is calculated according to the pedestrian local category matrix and the two-dimensional feature matrix corresponding to the pedestrian image; According to the weight parameters of each category, the pixel feature vector that aggregates all category information corresponding to each pedestrian image is calculated; All category feature vectors corresponding to each pedestrian image are concatenated together, and two fully connected layers are used to obtain the pixel-level attention feature matrix.

6. The pedestrian image recognition method according to claim 1, characterized in that: According to the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image, the third feature vector is obtained, including: Perform global average pooling on the pixel-level attention feature matrix corresponding to each pedestrian image to obtain the category information vector; Perform global average pooling on the most pixel-level attention feature vector of each pedestrian image to obtain the first feature vector; The category information vector of each pedestrian image is multiplied by the first eigenvector using the sigmoid function to activate the dot multiplication to obtain the second eigenvector; The second eigenvector and the first eigenvector of each pedestrian image are concatenated together to obtain the third eigenvector.

7. The pedestrian image recognition method according to claim 1, characterized in that: Based on the third eigenvector, find the pedestrian image closest to the target pedestrian image from multiple pedestrian images, including: Calculate the Euclidean distance between the third eigenvector of each pedestrian image and the target pedestrian image respectively; The pedestrian image with the smallest Euclidean distance is determined to be the pedestrian image closest to the target pedestrian image.

8. A pedestrian image recognition device, characterized in that: A pedestrian image acquisition module, used to acquire multiple pedestrian images; A two-dimensional feature extraction module is used to extract a two-dimensional feature matrix of each pedestrian image; A pixel-level attention feature vector extraction module is used to obtain the pixel-level attention feature vector of each pedestrian image according to the two-dimensional feature matrix of each pedestrian image; The pixel-level attention feature vector is obtained by calculating the pixel feature vectors that aggregate the pixel context information of all positions of the two-dimensional feature matrix based on the attention weight between each pixel and splicing them together using two fully connected layers; A pedestrian local category matrix acquisition module is used to obtain a preset number of categories and a pedestrian local category matrix pre-trained based on the number of categories; A pixel-level attention feature matrix acquisition module is used to obtain the pixel-level attention feature matrix of each pedestrian image according to the pedestrian local category matrix and the two-dimensional feature matrix of each pedestrian image; The pixel-level attention feature matrix of each pedestrian image is obtained by calculating the weight parameters of each category of the pedestrian local category matrix, calculating the pixel feature vectors corresponding to each pedestrian image that aggregate all category information, and then splicing them together using two fully connected layers; A third feature vector obtaining module, used for obtaining a third feature vector according to the most pixel-level attention feature vector and the most pixel-level attention feature matrix of each pedestrian image; The retrieval module is used to find the pedestrian image closest to the target pedestrian image from multiple pedestrian images based on the third eigenvector and output it.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method according to any one of claims 1 to 7.

Images