A Gait Contour Extraction Method and System Based on Depthwise Separable Convolution

Abstract

A gait contour extraction method based on depthwise separable convolution, comprising: preprocessing an input sequence of gait images; extracting features from the input sequence of gait images using an atrous convolutional neural network; correcting the edge fine-grainedness of the feature map; fusing the extracted shallow feature images and deep feature images; obtaining the original image size through upsampling; and outputting a gait contour map. The present invention uses depthwise separable convolution to replace the ordinary convolutional neural network, greatly reducing the number of parameters in the encoder and decoder parts and accelerating the extraction speed; performing unstable sorting on the points on the feature map and re-predicting the chaotic boundary classification, greatly improving the accuracy and integrity of edge segmentation.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to a gait contour extraction method and system based on depthwise separable convolution. Background Art

[0002] Gait is a biometric feature. Compared with other biometric features such as iris, palmprint, vein, etc., gait has significant advantages such as non-contact, remotely detectable, non-invasive, and difficult to hide, and can be widely applied in industries such as security, transportation, and industry. In the existing gait recognition technology, first, a gait image sequence is generated through object detection and tracking technology, then gait contour extraction is performed, and finally gait recognition is performed. The quality of the gait black and white contour map determines the accuracy of gait recognition. Currently, the existing image segmentation algorithms have been oscillating and deforming at the edges of the segmented regions, and their edges have not been well preserved, resulting in inaccurate edge segmentation. When applied to the gait recognition scenario, the output black and white contour map also has problems such as incomplete and fragmented edges, which seriously affect the accuracy of gait recognition. Summary of the Invention

[0003] In view of the above problems, the present invention is proposed to provide a gait contour extraction method and system based on depthwise separable convolution that overcomes the above problems or at least partially solves the above problems.

[0004] In order to solve the above technical problems, the embodiments of the present application disclose the following technical solutions:

[0005] A gait contour extraction method based on depthwise separable convolution, comprising:

[0006] S100. Preprocess the input gait image sequence;

[0007] S200. Extract features from the input gait image sequence using an atrous convolutional neural network;

[0008] S300. Correct the edge fine-grainedness of the feature map;

[0009] S400. Perform feature fusion on the extracted shallow feature image and deep feature image;

[0010] S500. Obtain the original image size through upsampling;

[0011] S600. Output the gait contour map.

[0012] Further, in S100, preprocessing the input gait image sequence includes: converting the input image size to 513*513*3.

[0013] Further, in S200, a dilated convolutional neural network is used to extract features from the input gait image sequence, including the following steps:

[0014] S201. Use Xception65 as the backbone network;

[0015] S202. Extract features from the input image sequence through a multi-layer serial dilated convolutional neural network, output feature S1, and divide S1 into two parts S1-1 and S1-2;

[0016] S203. Use a multi-layer parallel dilated convolutional neural network with different rates to extract features from the output S1-2, and perform feature compression on the output feature map through 1*1 depthwise separable convolution to obtain feature S2.

[0017] Further, in S300, the feature map is corrected at the edge fine-grained level, including the following steps:

[0018] Perform unstable sorting on the points on the S1-1 feature map and select K points;

[0019] In the Xception65 network, the outputs are c1, c2, c3, c4; where c1 is the feature map at a higher resolution (1 / 4), and c4 is the final feature map (1 / 16). Extract the corresponding features of the selected K points on these two maps;

[0020] Glue together the features at the corresponding positions of these K points;

[0021] Use MLP for subdivision prediction to assign these points to different classes, and replace the unstable points in the output with the prediction results; output feature map S1-1-1;

[0022] According to the above method, perform instability sorting on S2 as well, find the K most unstable points, perform class prediction and correction on these K points, and output feature map S2-1.

[0023] Further, in S300, perform unstable sorting on the points on the S1-1 feature map and select K points, where K is 8192.

[0024] Further, in S400, feature fusion is performed on the extracted shallow feature image and deep feature image, including:

[0025] For S1-1-1 Figure 1 *1 depthwise separable convolution is used to perform feature compression to obtain feature S1-1-2;

[0026] Perform bilinear interpolation for upsampling S2-1 once to obtain S2-2;

[0027] Fuse the features of S1-1-2 and S2-2 and output S3.

[0028] Furthermore, in S500, the method for obtaining the original image size through upsampling is as follows:

[0029] Pass S3 through a 3*3 depthwise separable convolutional neural network to obtain S4;

[0030] Perform bilinear interpolation for upsampling S4 once to obtain S5.

[0031] Furthermore, in S600, output the gait contour map and output the binary black and white gait contour map.

[0032] The present invention also discloses a gait contour extraction system based on depthwise separable convolution, which is characterized by including a gait image sequence preprocessing unit, a gait image feature extraction unit, an edge fine-grained correction unit, an image feature fusion unit, an upsampling unit, and a gait contour map output unit; wherein:

[0033] The gait image sequence preprocessing unit is used to preprocess the input gait image sequence;

[0034] The gait image feature extraction unit is used to extract features from the input gait image sequence by using a dilated convolutional neural network;

[0035] The edge fine-grained correction unit is used to correct the edge fine-grained of the feature map;

[0036] The image feature fusion unit is used to fuse the extracted shallow feature image and deep feature image;

[0037] The upsampling unit is used to obtain the original image size through upsampling;

[0038] The gait contour map output unit is used to output the gait contour map.

[0039] The beneficial effects of the above technical solutions provided by the embodiments of the present invention at least include:

[0040] A gait contour extraction method based on depthwise separable convolution disclosed by the present invention includes: preprocessing an input gait image sequence; extracting features from the input gait image sequence using an atrous convolutional neural network; correcting the edge fine-grainedness of the feature map; fusing the extracted shallow feature image and deep feature image; obtaining the original image size through upsampling; and outputting a gait contour map. The present invention uses depthwise separable convolution to replace the ordinary convolutional neural network, greatly reducing the number of parameters in the encoder and decoder parts and accelerating the extraction speed; unstable sorting is performed on the points on the feature map, and the chaotic boundary classification is re-predicted, greatly improving the accuracy and integrity of edge segmentation.

[0041] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0042] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0043] Figure 1 It is a flowchart of a gait contour extraction method based on depthwise separable convolution in Embodiment 1 of the present invention;

[0044] Figure 2 It is a flowchart of correcting the edge fine-grainedness of the feature map in Embodiment 1 of the present invention. Detailed Embodiments

[0045] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0046] To solve the problems existing in the prior art, an embodiment of the present invention provides a gait contour extraction method and system based on depthwise separable convolution.

[0047] Embodiment 1

[0048] This embodiment discloses a gait contour extraction method based on depthwise separable convolution, as Figure 1 , including:

[0049] S100. Preprocess the input gait image sequence; specifically, in this embodiment, preprocessing the input gait image sequence includes: converting the size of the input image to 513*513*3.

[0050] S200. Use a dilated convolutional neural network to extract features from the input gait image sequence; in S200 of this embodiment, using a dilated convolutional neural network to extract features from the input gait image sequence includes the following steps:

[0051] S201. Use Xception65 as the backbone network;

[0052] S202. Extract features from the input image sequence through a multi-layer serial dilated convolutional neural network, output feature S1, and at the same time divide S1 into two parts S1-1 and S1-2;

[0053] S203. Use a multi-layer parallel dilated convolutional neural network with different rates to extract features from the output S1-2, and perform feature compression on the output feature map through 1*1 depthwise separable convolution to obtain feature S2.

[0054] S300. Correct the edge fine-grainedness of the featuremap; in S300 of this embodiment, correcting the edge fine-grainedness of the featuremap includes the following steps:

[0055] Perform unstable sorting on the points on the S1-1 featuremap, such as Figure 2 , select K points, and set K to 8192 in this training;

[0056] In the Xception65 network, the outputs are c1, c2, c3, c4; where c1 is the featuremap (1 / 4) at a higher resolution, and c4 is the final featuremap (1 / 16). Extract the corresponding features of the selected K points on these two maps;

[0057] Glue the features at the corresponding positions of these K points together;

[0058] Use MLP for subdivision prediction to make these points belong to different classes, and replace the unstable points in the output with the prediction results; output feature map S1-1-1;

[0059] According to the above method, perform instability sorting on S2 as well, find the K most unstable points, perform class prediction and correction on these K points, and output feature map S2-1.

[0060] S400. Perform feature fusion on the extracted shallow feature image and deep feature image; in S400 of this embodiment, performing feature fusion on the extracted shallow feature image and deep feature image includes:

[0061] Perform feature compression on the S1-1-1 Figure 1 *1 depthwise separable convolution to obtain feature S1-1-2;

[0062] Perform bilinear interpolation on S2-1 for one upsampling to obtain S2-2;

[0063] Fuse the features of S1-1-2 and S2-2 and output S3.

[0064] S500. Obtain the original image size through upsampling; in S500 of this embodiment, the method for obtaining the original image size through upsampling is:

[0065] Pass S3 through a 3*3 depthwise separable convolutional neural network to obtain S4;

[0066] Perform bilinear interpolation on S4 for one upsampling to obtain S5.

[0067] S600. Output the gait contour map. In S600 of this embodiment, output the gait contour map and output the gait black and white binary contour map.

[0068] This embodiment also discloses a gait contour extraction system based on depthwise separable convolution, including a gait image sequence preprocessing unit, a gait image feature extraction unit, an edge fine-grained correction unit, an image feature fusion unit, an upsampling unit, and a gait contour map output unit; where:

[0069] The gait image sequence preprocessing unit is used to preprocess the input gait image sequence;

[0070] The gait image feature extraction unit is used to extract features from the input gait image sequence by using an atrous convolutional neural network;

[0071] The edge fine-grained correction unit is used to correct the edge fine-grained of the feature map;

[0072] The image feature fusion unit is used to perform feature fusion on the extracted shallow feature image and deep feature image;

[0073] The upsampling unit is used to obtain the original image size through upsampling;

[0074] The gait contour map output unit is used to output the gait contour map.

[0075] Specifically, the working principles of the gait image sequence preprocessing unit, the gait image feature extraction unit, the edge fine-grained correction unit, the image feature fusion unit, the upsampling unit, and the gait contour map output unit have been described in detail above and will not be elaborated here.

[0076] A gait contour extraction method based on depthwise separable convolution disclosed in this embodiment includes: preprocessing the input gait image sequence; extracting features from the input gait image sequence using an atrous convolutional neural network; performing edge fine-grained correction on the feature map; fusing the extracted shallow feature image and deep feature image; obtaining the original image size through upsampling; and outputting a gait contour map. The present invention uses depthwise separable convolution to replace the ordinary convolutional neural network, greatly reducing the number of parameters in the encoder and decoder parts and accelerating the extraction speed; performing unstable sorting on the points on the feature map and re-predicting the chaotic boundary classification, greatly improving the accuracy and integrity of edge segmentation.

[0077] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of the present disclosure. The appended method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy recited.

[0078] In the above detailed description, various features are combined in a single embodiment to simplify the present disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are expressly stated in each claim. On the contrary, as reflected in the appended claims, the present invention lies in a state less than all the features of the single disclosed embodiment. Therefore, the appended claims are hereby expressly incorporated into the detailed description, where each claim stands alone as a separate preferred embodiment of the present invention.

[0079] Those skilled in the art should also understand that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments herein can be implemented as electronic hardware, computer software, or a combination thereof. To clearly illustrate the interchangeability of hardware and software, the above description of the various illustrative components, blocks, modules, circuits, and steps has been generally described in terms of their functions. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in a flexible manner for each particular application, but such implementation decisions should not be construed as departing from the scope of the present disclosure.

[0080] The steps of the methods or algorithms described in connection with the embodiments of this specification may be embodied directly as hardware, software modules executed by a processor, or a combination thereof. The software modules may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The ASIC may be located in a user terminal. Of course, the processor and the storage medium may also exist as discrete components in the user terminal.

[0081] For a software implementation, the techniques described in this application may be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes may be stored in a memory unit and executed by a processor. The memory unit may be implemented within the processor or outside the processor, and in the latter case, it is communicatively coupled to the processor by various means, which are well known in the art.

[0082] The foregoing description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but those of ordinary skill in the art should recognize that the various embodiments can be further combined and arranged. Accordingly, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, this term is inclusive in a manner similar to the term "including" as interpreted when used as a transitional word in a claim. In addition, any use of the term "or" in the claims or specification is to mean "non-exclusive or".

Claims

1. A gait contour extraction method based on depthwise separable convolution, characterized in that Including: S100. Preprocess the input gait image sequence; S200. Use a dilated convolutional neural network to extract features from the input gait image sequence. In S200, using a dilated convolutional neural network to extract features from the input gait image sequence includes the following steps: S201. Use Xception65 as the backbone network; S202. Extract features from the input image sequence through a multi-layer serial dilated convolutional neural network, output feature S1, and at the same time divide S1 into two parts S1-1 and S1-2; S203. Feature extraction is performed on the output S1-2 using a multi-layer parallel dilated convolutional neural network with different rates, and the output feature map is compressed through 1 1 depthwise separable convolution to obtain the feature S2; S300. Perform edge fine-grained correction on the featuremap. In S300, performing edge fine-grained correction on the featuremap includes the following steps: Perform unstable sorting on the points on the S1-1 featuremap and select K points; In the Xception65 network, the outputs are c1, c2, c3, c4; where c1 is the featuremap (1 / 4) at a higher resolution, and c4 is the final featuremap (1 / 16). Extract the corresponding features of the selected K points on these two maps; Glue together the features at the corresponding positions of these K points; Use MLP for subdivision prediction to assign these points to different classes, and replace the unstable points in the output with the prediction results; output feature map S1-1-1; According to the above method, perform instability sorting on S2 as well, find the K most unstable points, perform class prediction and correction on these K points, and output feature map S2-1; S400. Perform feature fusion on the extracted shallow feature image and deep feature image; S500. Obtain the original image size through upsampling; S600. Output the gait contour map.

2. The gait silhouette extraction method based on depthwise separable convolution according to claim 1, characterized in that In S100, preprocess the input gait image sequence, including: convert the input image size to 513 513 by 3 size.

3. A gait silhouette extraction method based on depthwise separable convolution according to claim 1, characterized in that In S300, perform unstable sorting on the points on the S1-1 featuremap and select K points. K is 8192.

4. The gait silhouette extraction method based on depthwise separable convolution according to claim 1, characterized in that, In S400, performing feature fusion on the extracted shallow feature image and deep feature image includes: For Figure 1 of S1-1-1 1 Feature compression is performed on the depthwise separable convolution to obtain feature S1-1-2; Perform one upsampling on S2-1 through bilinear interpolation to obtain S2-2; Fuse the features of S1-1-2 and S2-2 and output S3.

5. The gait silhouette extraction method based on depthwise separable convolution according to claim 4, wherein, In S500, the method of obtaining the original image size through upsampling is: Pass S3 through 3 3 depthwise separable convolutional neural networks to obtain S4; Perform one upsampling on S4 through bilinear interpolation to obtain S5.

6. The gait contour extraction method based on depthwise separable convolution according to claim 1, characterized in that, In S600, output the gait contour map, and output the gait black and white binary contour map.

7. A gait contour extraction system based on depthwise separable convolution, which implements the gait contour extraction method described in any one of the above claims 1-6, characterized in that Including a gait image sequence preprocessing unit, a gait image feature extraction unit, an edge fine-grained correction unit, an image feature fusion unit, an upsampling unit, and a gait contour map output unit; where: The gait image sequence preprocessing unit is used to preprocess the input gait image sequence; The gait image feature extraction unit is used to extract features from the input gait image sequence using a dilated convolutional neural network; The edge fine-grained correction unit is used to perform edge fine-grained correction on the featuremap; The image feature fusion unit is used to perform feature fusion on the extracted shallow feature image and deep feature image; The upsampling unit is used to obtain the original image size through upsampling; A gait profile graph output unit for outputting a gait profile graph.

Images