A method, device, electronic device, and storage medium for reconstructing an occluded gait

Abstract

The present invention relates to a method, device, electronic device and storage medium for reconstructing an occluded gait. The method for reconstructing an occluded gait of the present invention comprises: obtaining an occluded gait sequence to be reconstructed; inputting the occluded gait sequence into a generator of a trained generative adversarial network to obtain a repaired gait sequence; wherein the training method of the generative adversarial network comprises the following steps: selecting part of the samples from the unoccluded gait data set as the original sample set, and adding occlusion features to the samples in the original sample set as a training set; putting the samples of the training set into the generator to obtain a repaired sample set; putting the repaired sample set and the original sample set into the discriminator, and using the loss function to calculate the difference between the repaired sample set and the original sample set; adversarially training the generator and the discriminator until the loss function converges, and saving the trained generative adversarial network. The method for reconstructing an occluded gait of the present invention reconstructs a more complete human body structure.

Description

Technical Field

[0001] The present invention relates to the technical field of gait recognition, and in particular to a method, device, electronic device and storage medium for reconstructing an occluded gait. Background Art

[0002] Gait refers to a person's walking pattern, which is related to a person's muscle memory, bone structure and the influence of the nerve center controlled by the cerebellum. As a biological feature that can be identified from a distance, gait recognition has important application value in public security criminal investigation, danger warning and other fields.

[0003] However, gait recognition methods are affected by various covariates, such as coats, backpacks, camera angles, and occlusions. These covariates alter the appearance of the gait, leading to reduced gait recognition accuracy. This phenomenon poses significant obstacles to the real-world application of gait recognition. For example, in train stations, supermarkets, schools, and on the road, a person's gait is often obscured by other pedestrians or objects. Therefore, solving this problem has become a challenging problem. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a method, device, electronic device and storage medium for reconstructing occluded gait, using body part-based input and skeleton information to calculate the loss function, guiding the generation network to pay more attention to local reconstruction information, and reconstructing a more complete human body structure.

[0005] In a first aspect, the present invention provides a method for reconstructing an occluded gait, comprising the following steps:

[0006] Obtain the occluded gait sequence to be reconstructed;

[0007] Inputting the occluded gait sequence into the generator of the trained generative adversarial network to obtain a repaired gait sequence;

[0008] The training method of the generative adversarial network comprises the following steps:

[0009] Select some samples from the unobstructed gait dataset as the original sample set, and add occlusion features to the samples in the original sample set as the training set;

[0010] Put the samples of the training set into the generator to obtain a repaired gait sequence, i.e., a repaired sample set;

[0011] Putting the repaired sample set and the original sample set into a discriminator, and using a loss function to calculate the difference between the repaired sample set and the original sample set; the loss function is calculated based on the input of the body part and the skeleton information;

[0012] The generator and the discriminator are trained adversarially until the loss function converges, and the trained generative adversarial network is saved.

[0013] Furthermore, the loss function of the discriminator includes a body part loss function, a skeleton information loss function and a triplet loss function;

[0014] The body part loss function is based on the repair sample S in the repair sample set I and the true sample S in the original sample set GT The distance of the body block is calculated;

[0015] The skeleton information loss function is based on the repair sample S in the repair sample set I and the true sample S in the original sample set GT The skeleton graph distance is calculated;

[0016] The triplet loss function is based on the repair sample S in the repair sample set I Based on the true sample S in the original sample set GT Constructed positive sample S pos and negative samples S neg Calculated.

[0017] Furthermore, the triple loss function is:

[0018]

[0019] Among them, L-triplet is the triple loss function, margin is a hyperparameter, D(S I )、D(S pos ) and D(S neg ) are to repair the sample S I , positive sample S pos and negative samples S neg The output of the input to the discriminator.

[0020] Furthermore, the steps for calculating the body part loss function are:

[0021] The repair sample S I The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg;

[0022] The true sample S GT The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg;

[0023] Use the following formula to calculate the repair sample S respectively I Each subgraph is related to the true sample SGT The distance of each corresponding subgraph of :

[0024]

[0025]

[0026]

[0027]

[0028]

[0029] Among them, i is the current training sample and n is the number of data sets used in training.

[0030] The body part loss function L-patch is calculated using the following formula:

[0031] L-patch=L-head+L-leftbody+L-rightbody+L-leftleg+L-rightleg.

[0032] Furthermore, the calculation steps of the skeleton information loss function are:

[0033] For the repair sample S I and the true sample S GT Perform skeleton extraction to generate skeleton images respectively;

[0034] The skeleton information loss function L-skeleton is calculated using the following formula:

[0035]

[0036] Among them, L-skeleton is the skeleton information loss function, i is the current training sample, and n is the number of data sets used in training.

[0037] Furthermore, residual connections are used in the generator network to add the output of the first layer network to the output of the second to last layer network as the input of the last convolutional layer, and to add the input of the third layer network to the output of the fourth to last layer network as the input of the third to last layer.

[0038] Furthermore, a dilated convolution is introduced into the generator;

[0039] The dilated convolution introduces a hyperparameter called the dilation rate, which defines the spacing between values when the convolution kernel processes the data:

[0040] An ordinary 3*3 two-dimensional convolution kernel can sample a 3*3 feature map at a time; a 3*3 two-dimensional void convolution with an expansion rate of 2 can sample a 5*5 feature map at a time.

[0041] In a second aspect, the present invention further provides a device for reconstructing an occluded gait, comprising:

[0042] A gait sequence acquisition module, used to acquire the occluded gait sequence to be reconstructed;

[0043] a gait sequence restoration module, configured to input the occluded gait sequence into a generator of a trained generative adversarial network to obtain a restored gait sequence;

[0044] The training method of the generative adversarial network comprises the following steps:

[0045] Select some samples from the unobstructed gait dataset as the original sample set, and add occlusion features to the samples in the original sample set as the training set;

[0046] Put the samples of the training set into the generator to obtain a repaired gait sequence, i.e., a repaired sample set;

[0047] Putting the repaired sample set and the original sample set into a discriminator, and using a loss function to calculate the difference between the repaired sample set and the original sample set; the loss function is calculated based on the input of the body part and the skeleton information;

[0048] The generator and the discriminator are trained adversarially until the loss function converges, and the trained generative adversarial network is saved.

[0049] In a third aspect, the present invention further provides an electronic device, comprising:

[0050] at least one memory and at least one processor;

[0051] The memory is used to store one or more programs;

[0052] When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of the method for reconstructing an occluded gait as described in any one of the first aspects of the present invention.

[0053] In a fourth aspect, the present invention further provides a computer-readable storage medium,

[0054] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for reconstructing an occluded gait as described in any one of the first aspects of the present invention are implemented.

[0055] The present invention provides a method, device, electronic device and storage medium for reconstructing occluded gait, which uses input based on body parts to calculate the loss function, guiding the generation network to pay more attention to local reconstruction information. The loss function is calculated using skeleton information to guide the generation network to reconstruct a more complete human body structure. Skip connection is used in the generator network, and the residual connection method allows the network to utilize the information of the unoccluded part in the previous gait sequence. By introducing dilated convolution in the generator, the receptive field can be increased without losing information during downsampling. Each convolution output contains a larger range of information. Compared with previous technologies, the reconstructed image quality is better and the structure of the reconstructed image is more complete.

[0056] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the training steps of the generative adversarial network used in the present invention;

[0058] Figure 2 is an exemplary unoccluded gait silhouette sequence;

[0059] Figure 3 is an exemplary (static) gait silhouette sequence after artificial occlusion;

[0060] Figure 4 is an exemplary (dynamic) gait silhouette sequence after artificial occlusion;

[0061] Figure 5 This is an example of a gait silhouette sequence reconstructed by the generator of a generative adversarial network.

[0062] Figure 6 A network framework diagram used in one embodiment of the present invention;

[0063] Figure 7 A schematic structural diagram of a device for reconstructing occluded gait provided by the present invention. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0065] It should be clear that the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.

[0066] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present application. The singular forms "a," "the," and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0067] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0068] In addition, in this application, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0069] To address the problems in the background technology, an embodiment of the present application provides a method for reconstructing an occluded gait, which includes the following steps:

[0070] S01: Obtain the occluded gait sequence to be reconstructed.

[0071] S02: Inputting the occluded gait sequence into the generator of the trained generative adversarial network to obtain a repaired gait sequence.

[0072] like Figure 1 As shown, the training method of the generative adversarial network includes the following steps:

[0073] S11: Select some samples from the unobstructed gait dataset as the original sample set, and add occlusion features to the samples in the original sample set as the training set.

[0074] In a specific embodiment, the unoccluded gait profile sequence is as follows: Figure 2As shown in the figure, multiple unobstructed gait profile sequences constitute the original sample set; the gait profile sequence after artificial occlusion (static) is as follows Figure 3 As shown, the gait profile sequence after artificial occlusion (dynamic) is as follows Figure 4 As shown in Figure 2, multiple gait profile sequences after artificial occlusion constitute the training set.

[0075] S12: Put the samples of the training set into the generator to obtain a repaired gait sequence, that is, a repaired sample set.

[0076] In a specific embodiment, the reconstructed gait profile sequence is as follows: Figure 5 As shown in Figure 2, multiple reconstructed gait profile sequences constitute the restoration sample set.

[0077] S13: Putting the repaired sample set and the original sample set into a discriminator, and using a loss function to calculate the gap between the repaired sample set and the original sample set; the loss function is calculated based on the input of the body part and the skeleton information.

[0078] S14: adversarially train the generator and the discriminator until the loss function converges, and save the trained generative adversarial network.

[0079] In a preferred embodiment, Figure 6 As shown, the loss function of the discriminator includes body part loss function, skeleton information loss function and triplet loss function.

[0080] The body part loss function is based on the repair sample S in the repair sample set. I and the true sample S in the original sample set GT The distance of the body block is calculated.

[0081] The body part loss function enables the network to pay more attention to the local information of the reconstructed image to enhance the local consistency between the reconstructed image and the real image.

[0082] Preferably, the steps for calculating the body part loss function are:

[0083] S21: The repair sample S I The human body silhouette is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg.

[0084] S22: The true sample S GT The human body silhouette is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg.

[0085] S23: Use the following formula to calculate the repair sample S I Each subgraph is related to the true sample S GTThe distance of each corresponding subgraph of :

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] Among them, i is the current training sample and n is the number of data sets used in training.

[0092] S24: Calculate the body part loss function L-patch using the following formula:

[0093] L-patch=L-head+L-leftbody+L-rightbody+L-leftleg+L-rightleg.

[0094] The skeleton information loss function is based on the repair sample S in the repair sample set I and the true sample S in the original sample set GT The skeleton graph distance is calculated.

[0095] The skeleton information loss function enables the reconstruction network to pay attention to the integrity of the human body structure during the reconstruction process.

[0096] Preferably, the calculation steps of the skeleton information loss function are:

[0097] S31: Repair the sample S I and the true sample S GT Skeleton extraction is performed separately to generate skeleton images.

[0098] In a specific embodiment, the skeleton extraction method is a binary image thinning method, and the implementation steps are as follows:

[0099] Binary image thinning is the process of removing some points from the original binary image while maintaining the original image skeleton, thinning it into lines with a width of one pixel. Whether a pixel can be removed is determined based on its eight adjacent pixels. The specific conditions are: 1. Internal points cannot be deleted. 2. Isolated points cannot be deleted. 3. Line endpoints cannot be deleted.

[0100] S32: Calculate the skeleton information loss function L-skeleton using the following formula:

[0101]

[0102] Among them, L-skeleton is the skeleton information loss function, i is the current training sample, and n is the number of data sets used in training.

[0103] The triplet loss function is based on the repair sample S in the repair sample set I Based on the true sample S in the original sample set GT Constructed positive sample S pos and negative samples S neg Calculated.

[0104] For a true sample S GT , construct a corresponding positive sample data set and negative sample data set, the positive sample is the different gait videos of the same person walking, and the negative sample is the gait videos of different people walking. I And positive and negative sample data are input into the discriminator, and the output value is D(S I )、D(S pos ) and D(S neg ), calculate the triplet loss function L-triplet, the calculation formula is:

[0105]

[0106] Among them, L-triplet is the triple loss function, margin is a hyperparameter, D(S I )、D(S pos ) and D(S neg ) are to repair the sample S I , positive sample S pos and negative samples S neg The output of the input to the discriminator.

[0107] Combine L-patch, L-skeleton and L-triplet to improve the generator's reconstruction effect on occluded areas.

[0108] In a preferred embodiment, residual connections are used in the generator network to add the output of the first layer of the network to the output of the second to last layer of the network as the input of the last convolutional layer, and to add the input of the third layer of the network to the output of the fourth to last layer of the network as the input of the third to last layer.

[0109] The role of residual connection: The residual connection method allows the network to utilize the information of the unoccluded part of the gait sequence.

[0110] In a more preferred embodiment, dilated convolution is also introduced into the generator:

[0111] The dilated convolution introduces a hyperparameter called dilation rate, which defines the spacing between values when the convolution kernel processes data.

[0112] An ordinary 3*3 two-dimensional convolution kernel can sample a 3*3 feature map at a time.

[0113] A 3*3 two-dimensional hole convolution with a dilation rate of 2 can sample a 5*5 feature map at a time.

[0114] The role of dilated convolution is to increase the receptive field without losing information during downsampling, so that each convolution output contains a wider range of information.

[0115] The present application also provides a device for reconstructing an obscured gait. Figure 7 As shown, the occluded gait reconstruction device 400 includes:

[0116] A gait sequence acquisition module 401 is used to acquire an occluded gait sequence to be reconstructed;

[0117] a gait sequence repair module 402 for inputting the occluded gait sequence into a generator of a trained generative adversarial network to obtain a repaired gait sequence;

[0118] The training method of the generative adversarial network comprises the following steps:

[0119] Select some samples from the unobstructed gait dataset as the original sample set, and add occlusion features to the samples in the original sample set as the training set;

[0120] Put the samples of the training set into the generator to obtain a repaired gait sequence, i.e., a repaired sample set;

[0121] Putting the repaired sample set and the original sample set into a discriminator, and using a loss function to calculate the difference between the repaired sample set and the original sample set; the loss function is calculated based on the input of the body part and the skeleton information;

[0122] The generator and the discriminator are trained adversarially until the loss function converges, and the trained generative adversarial network is saved.

[0123] Preferably, the loss function of the discriminator includes a body part loss function, a skeleton information loss function and a triplet loss function;

[0124] The body part loss function is based on the repair sample S in the repair sample set I and the true sample S in the original sample set GT The distance of the body block is calculated;

[0125] The skeleton information loss function is based on the repair sample S in the repair sample set I and the true sample S in the original sample set GT The skeleton graph distance is calculated;

[0126] The triplet loss function is based on the repair sample S in the repair sample set I Based on the true sample S in the original sample set GT Constructed positive sample S pos and negative samples S neg Calculated.

[0127] Preferably, the triple loss function is:

[0128]

[0129] Among them, L-triplet is the triple loss function, margin is a hyperparameter, D(S I )、D(S pos ) and D(S neg ) are to repair the sample S I , positive sample S pos and negative samples S neg The output of the input to the discriminator.

[0130] Preferably, the steps for calculating the body part loss function are:

[0131] The repair sample S I The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg;

[0132] The true sample S GT The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg;

[0133] Use the following formula to calculate the repair sample S respectively I Each subgraph is related to the true sample S GT The distance of each corresponding subgraph of :

[0134]

[0135]

[0136]

[0137]

[0138]

[0139] Among them, i is the current training sample and n is the number of data sets used in training.

[0140] The body part loss function L-patch is calculated using the following formula:

[0141] L-patch=L-head+L-leftbody+L-rightbody+L-leftleg+L-rightleg.

[0142] Preferably, the repair sample S I and the true sample S GT Perform skeleton extraction to generate skeleton images respectively;

[0143] The skeleton information loss function L-skeleton is calculated using the following formula:

[0144]

[0145] Among them, L-skeleton is the skeleton information loss function, i is the current training sample, and n is the number of data sets used in training.

[0146] Preferably, residual connections are used in the generator network to add the output of the first layer network to the output of the second to last layer network as the input of the last convolutional layer, and to add the input of the third layer network to the output of the fourth to last layer network as the input of the third to last layer.

[0147] Preferably, a dilated convolution is introduced into the generator;

[0148] The dilated convolution introduces a hyperparameter called the dilation rate, which defines the spacing between values when the convolution kernel processes the data:

[0149] An ordinary 3*3 two-dimensional convolution kernel can sample a 3*3 feature map at a time; a 3*3 two-dimensional void convolution with an expansion rate of 2 can sample a 5*5 feature map at a time.

[0150] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0151] An embodiment of the present application further provides an electronic device, including:

[0152] at least one memory and at least one processor;

[0153] The memory is used to store one or more programs;

[0154] When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of the aforementioned method for reconstructing an occluded gait.

[0155] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0156] The present application also provides a computer-readable storage medium.

[0157] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for reconstructing an occluded gait are implemented.

[0158] Computer-usable storage media includes permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0159] The present invention provides a method, device, electronic device and storage medium for reconstructing occluded gait, which uses input based on body parts to calculate the loss function, guiding the generation network to pay more attention to local reconstruction information. The loss function is calculated using skeleton information to guide the generation network to reconstruct a more complete human body structure. Skip connection is used in the generator network, and the residual connection method allows the network to utilize the information of the unoccluded part in the previous gait sequence. By introducing dilated convolution in the generator, the receptive field can be increased without losing information during downsampling. Each convolution output contains a larger range of information. Compared with previous technologies, the reconstructed image quality is better and the structure of the reconstructed image is more complete.

[0160] The above-described embodiments merely illustrate several implementations of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, and all such variations and improvements fall within the scope of protection of the present invention.

Claims

1. A method for reconstructing occluded gait, characterized in that: The following steps are involved: Obtaining the occluded gait sequence to be reconstructed; Inputting the occluded gait sequence into the generator of the trained generative adversarial network to obtain a repaired gait sequence; The training method of the generative adversarial network comprises the following steps: Select some samples from the unobstructed gait dataset as the original sample set, and add occlusion features to the samples in the original sample set as the training set; Put the samples of the training set into the generator to obtain a repaired gait sequence, i.e., a repaired sample set; Putting the repaired sample set and the original sample set into a discriminator, and using a loss function to calculate the difference between the repaired sample set and the original sample set; the loss function is calculated based on the input of the body part and the skeleton information; Training the generator and the discriminator in an adversarial manner until the loss function converges, and saving the trained generative adversarial network; The loss function of the discriminator includes a body part loss function, a skeleton information loss function and a triplet loss function; The body part loss function is based on the repair sample in the repair sample set Compared with the true samples in the original sample set The distance of the body block is calculated; The skeleton information loss function is based on the repair sample in the repair sample set Compared with the true samples in the original sample set The skeleton graph distance is calculated; The triplet loss function is based on the repair samples in the repair sample set Based on the true samples in the original sample set Constructed positive samples and negative samples Calculated; The calculation steps of the body part loss function are: The repair sample The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg; The true sample The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg; Use the following formula to calculate the repair samples respectively Each subgraph is related to the true sample The distance of each corresponding subgraph of : Among them, i is the current training sample, n is the number of data sets used in training; The body part loss function L-patch is calculated using the following formula: ； The calculation steps of the skeleton information loss function are: The repair sample and the true sample Perform skeleton extraction to generate skeleton images respectively; The skeleton information loss function L-skeleton is calculated using the following formula: Among them, L-skeleton is the skeleton information loss function, i is the current training sample, and n is the number of data sets used in training.

2. The method for reconstructing occluded gait according to claim 1, characterized in that: The triplet loss function is: Among them, L-triplet is the triple loss function, margin is a hyperparameter, D( )、D( ) and D( ) are the repair samples , positive samples and negative samples The output of the input to the discriminator.

3. The method for reconstructing an occluded gait according to claim 1, wherein: The generator network uses residual connections to add the output of the first layer of the network to the output of the second-to-last layer of the network as the input of the last convolutional layer, and to add the input of the third layer of the network to the output of the fourth-to-last layer of the network as the input of the third-to-last layer.

4. The method for reconstructing an occluded gait according to claim 1, wherein: The generator introduces dilated convolution; The dilated convolution introduces a hyperparameter called the dilation rate, which defines the spacing between values when the convolution kernel processes the data: An ordinary 3*3 two-dimensional convolution kernel can sample a 3*3 feature map at a time; a 3*3 two-dimensional void convolution with an expansion rate of 2 can sample a 5*5 feature map at a time.

5. A device for reconstructing occluded gait, characterized in that: include: A gait sequence acquisition module, used to acquire the occluded gait sequence to be reconstructed; a gait sequence restoration module, configured to input the occluded gait sequence into a generator of a trained generative adversarial network to obtain a restored gait sequence; The training method of the generative adversarial network comprises the following steps: Select some samples from the unobstructed gait dataset as the original sample set, and add occlusion features to the samples in the original sample set as the training set; Put the samples of the training set into the generator to obtain a repaired gait sequence, i.e., a repaired sample set; Putting the repaired sample set and the original sample set into a discriminator, and using a loss function to calculate the difference between the repaired sample set and the original sample set; the loss function is calculated based on the input of the body part and the skeleton information; Training the generator and the discriminator in an adversarial manner until the loss function converges, and saving the trained generative adversarial network; The loss function of the discriminator includes a body part loss function, a skeleton information loss function and a triplet loss function; The body part loss function is based on the repair sample in the repair sample set Compared with the true samples in the original sample set The distance of the body block is calculated; The skeleton information loss function is based on the repair sample in the repair sample set Compared with the true samples in the original sample set The skeleton graph distance is calculated; The triplet loss function is based on the repair samples in the repair sample set Based on the true samples in the original sample set Constructed positive samples and negative samples Calculated; The calculation steps of the body part loss function are: The repair sample The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg; The true sample The human body outline is divided into five sub-graphs according to the body parts: head, left side of the body, right side of the body, left leg, and right leg; Use the following formula to calculate the repair samples respectively Each subgraph is related to the true sample The distance of each corresponding subgraph of : Among them, i is the current training sample, n is the number of data sets used in training; The body part loss function L-patch is calculated using the following formula: ； The calculation steps of the skeleton information loss function are: The repair sample and the true sample Perform skeleton extraction to generate skeleton images respectively; The skeleton information loss function L-skeleton is calculated using the following formula: Among them, L-skeleton is the skeleton information loss function, i is the current training sample, and n is the number of data sets used in training.

6. An electronic device, characterized in that: include: at least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of the method for reconstructing an occluded gait as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for reconstructing an occluded gait according to any one of claims 1 to 4 are implemented.

Images