Dynamic micro-change identification method based on multi-granularity feature fusion and weighted entropy loss

Abstract

The invention discloses a dynamic micro-change identification method based on multi-granularity feature fusion and weighted entropy loss, uses an optical flow graph to better extract dynamic micro-change features and classify the dynamic micro-change features, and belongs to the technical field of machine learning and deep learning. According to the invention, block weighting is carried out on the optical flow graph, and the optical flow is enhanced by using the result of the attention module in the form of weight. According to the method, a local feature prediction vector is extracted from a weighted start-vertex optical flow block, a global feature prediction vector is extracted from a start-vertex optical flow graph and a vertex-offset optical flow graph, the global feature prediction vector and the local feature prediction vector are fused, and the distinguishing capability is improved through multi-granularity feature fusion. And the recognition accuracy is improved. In addition, a weighted cross entropy loss function is used, weighting is carried out according to different types of training loss, and the problem that the number of different types of training samples is greatly different is solved.

Description

Technical Field

[0001] The invention relates to a dynamic micro-change recognition method based on multi-granularity feature fusion and weighted entropy loss, and belongs to the technical field of machine learning and deep learning. Background Art

[0002] Dynamic micro-changes refer to the subtle changes in the characteristic morphology, detail texture, etc. (and dynamic micro-changes such as changes in facial features, crack shape, object direction or speed) that occur in a specific scenario. Dynamic micro-changes refer to the subtle adjustments made by individuals in terms of speed, strength, direction, rhythm, etc. when performing a task or activity. These changes may originate from changes in the individual's internal physiological state, psychological state or cognitive strategy, or may be subtly affected by external environmental factors such as light, temperature, social atmosphere, etc. They can be either unconscious natural reactions or conscious control after training. The main task of identifying dynamic micro-changes is to predict the change category of the given fragment. The identification of dynamic micro-changes is of great value in some cases, such as facial expression recognition, sports training, and medical operations. Although the importance of dynamic micro-changes has gradually attracted the attention of the academic community, related research still faces many challenges. These include but are not limited to the limitations of precise collection and analysis technology of motion data, the difficulty in extracting universal laws caused by individual differences, and how to accurately identify and interpret these micro-changes in a complex and changing social environment. Many dynamic micro-change recognition methods based on machine learning have been proposed, but these methods cannot effectively extract recognizable features from dynamic micro-changes. Some methods directly analyze and process the original fragments when using the attention module to extract the identification features of dynamic micro-changes. For some fragments with very small changes, the performance of the attention module is limited, affecting the accuracy of dynamic micro-change recognition. Some methods only classify and identify dynamic micro-change fragments from a global or local perspective, lacking a comprehensive analysis of the two, which also reduces the accuracy of dynamic micro-change recognition. During the training process of the method, the single use of the cross-loss entropy function leads to the problem of sample imbalance, resulting in problems such as overfitting and underfitting, which reduces the final performance of the method. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and to propose a dynamic micro-change recognition method based on multi-granularity feature fusion and weighted entropy loss.

[0004] The technical solution provided by the present invention is as follows: a dynamic micro-change recognition method based on multi-granularity feature fusion and weighted entropy loss, characterized in that it includes the following steps: Step S1, preprocessing each dynamic micro-change segment in the data set; Step S2, calculating the start-vertex optical flow map between the start frame and the vertex frame according to the preprocessed data set, weighting the start-vertex optical flow map blocks to obtain weighted start-vertex optical flow blocks, extracting the feature vector of each weighted start-vertex optical flow block, and merging them to obtain the local feature prediction vector of the dynamic micro-change segment; Step S3, calculating the optical flow graph between the vertex frame and the offset frame and the start-vertex optical flow graph based on the preprocessed data set Figure 1 After block division, convolution and pooling operations, the start-vertex optical flow block features and vertex-offset optical flow block features are extracted respectively. The attention module is used to further extract the start-vertex optical flow block relationship vector and the vertex-offset optical flow block relationship vector. The fully connected layer and the normalization layer are used to compress and refine the two relationship vectors to obtain the start-vertex optical flow prediction vector and the vertex-offset optical flow prediction vector. After merging the two optical flow prediction vectors, the global feature prediction vector is obtained. Step S4, obtaining a prediction vector for the dynamic slight change segment according to the local feature prediction vector and the global feature prediction vector; Step S5: Use the weighted cross entropy loss function to train the dynamic micro-change recognition method.

[0005] Further, in step S1: For a dynamic micro-change segment, the feature points on the first video frame in the dynamic micro-change segment are first detected, and the main area of ​​the first video frame is cropped according to these feature points. Then, the remaining video frames in the dynamic micro-change segment are cropped according to the area of ​​the first video frame to obtain a preprocessed dynamic micro-change segment; then, each dynamic micro-change segment in the data set is preprocessed according to the above method to obtain a preprocessed data set.

[0006] Furthermore, in step S2: Step S21, using the TV-L1 algorithm to calculate the start-to-vertex optical flow graph between the start frame and the vertex frame of the pre-processed dynamic slight change segment ; The start-vertex optical flow graph The grid is divided into multiple square start-vertex optical flow blocks, which are sent to the block weight generation module at the same time to calculate the weight of each start-vertex optical flow block; In the block weight generation module, the 2D convolution operation starts the vertex optical flow graph The channel dimension of each start-vertex optical flow block is expanded, and the attention module captures the relationship between these start-vertex optical flow blocks to obtain the output vector of the attention module. ; Output vector of the attention module After being processed by the fully connected layer and the normalized layer, the dimension is reduced, and the activation function Sigmoid is used to generate the output vector of the attention module. Calculate and finally get the weight of the start-vertex optical flow block : ; ; in, Represents the weight of the start-vertex optical flow block, weight Each value in is, in turn, the weight of each start-vertex optical flow block; represents the output vector of the attention module, represents a two-dimensional convolution operation, represents the attention module, represents the fully connected layer, represents the normalization layer, represents the activation function; Step S22, weighting each start-vertex optical flow block by multiplying each start-vertex optical flow block by the weight of the corresponding position in the weight vector to obtain each weighted start-vertex optical flow block, which is expressed as follows: ; ; in, Indicates Row and The start-vertex optical flow block of the column, Indicates Row and The weighted start-vertex optical flow block of the column, is the serial number, Indicates the maximum number of start-vertex optical flow block arrangements in the horizontal direction, The weight vector of the start-vertex optical flow block is The weight of the start-vertex optical flow block; Step S23, after performing a weighted operation on each start-vertex optical flow block, extracting a feature vector of each weighted start-vertex optical flow block, and splicing the feature vectors of each weighted start-vertex optical flow block together, and finally obtaining a local feature of the dynamic slight change segment; Specifically include: Through two-dimensional convolution and maximum pooling operations, the activation function Extract the feature vector of each weighted start-vertex optical flow block; flatten the feature vector of each weighted start-vertex optical flow block into a one-dimensional vector, connect all the flattened one-dimensional vectors together using the vector dimension connection operation, and obtain the local feature vector of the dynamic micro-change fragment : ; ; in, is the local feature vector of the dynamic micro-change fragment obtained by concatenating the feature vectors of all weighted start-vertex optical flow blocks, Indicates Row and The feature vector of the weighted start-vertex optical flow block of the column, is the vector dimension concatenation operation, represents the maximum number of weighted start-vertex optical flow block arrangements in the horizontal direction, Indicates the maximum number of weighted start-vertex optical flow block arrangements in the vertical direction, is the maximum pooling operation, is the activation function, Flatten operation for vectors; Step S24: local feature vector After calculations in the fully connected layer and the normalized layer, the prediction results for each category are obtained, that is, the local feature prediction vector : ; in, represents the fully connected layer, represents the normalization layer, Represents an average pooling operation.

[0007] Furthermore, in step S3: Step S31, using the TV-L1 algorithm to calculate the vertex-offset optical flow graph between the vertex frame and the offset frame of the pre-processed dynamic micro-change segment ; The start-vertex optical flow graph and vertex-offset optical flow graph The grid is divided into multiple square optical flow blocks, and two-dimensional convolution and maximum pooling operations are performed. The process is as follows: ; ; in represents the start-vertex optical flow block feature, represents the vertex-offset optical flow block feature, represents the start-vertex optical flow graph, represents the vertex-offset optical flow graph, Indicates that the input optical flow graph is divided into multiple square optical flow blocks according to the grid. represents the maximum pooling operation, Represents a two-dimensional convolution operation; Step S32, using the attention module to further extract the start-vertex optical flow block features The relationship vector between and vertex-offset optical flow block features The relationship vector between: ; ; in, represents the start-vertex optical flow block relationship vector, represents the vertex-offset optical flow block relationship vector, represents a vector flattening operation, represents the attention module; Step S33, using the calculation of the fully connected layer and the normalized layer, the start-vertex optical flow block relationship vector and vertex-offset optical flow block relation vector After compression and refinement, we finally get the start-vertex optical flow prediction vector and the vertex-offset optical flow prediction vector: ; ; in, represents the start-vertex optical flow prediction vector, represents the vertex-offset optical flow prediction vector; represents the fully connected layer, represents the normalization layer; Step S34: The start-vertex optical flow prediction vector and vertex-offset optical flow prediction vector Add together to get the global feature prediction vector for the dynamic micro-change segment : .

[0008] Furthermore, in step S4: the global feature prediction vector and the local feature prediction vector Add together to get the prediction vector for the dynamic micro-change segment : .

[0009] Furthermore, the weighted cross entropy loss function in step S5 is : ; in, Indicates the number of dynamic micro-change segments, Indicates the number of classifications of dynamic micro-change segments, Indicates a dynamic micro-change fragment, Indicates the true category of the dynamic micro-change segment. The category is equal to hour, The value is 1, otherwise it is 0. Indicates dynamic micro-change fragments Belongs to category The predicted probability when Indicates category The corresponding loss weight.

[0010] The beneficial effects of the present invention are as follows: the present invention adopts a block-weighted approach to the optical flow graph. After the optical flow graph is divided into blocks, an attention module is used to extract the relationship between all optical flow blocks, and a weight for each optical flow block is generated and weighted. By enhancing the optical flow in the form of weights using the results of the attention module, the problem of a small number of samples when the attention module is used directly is avoided. The present invention analyzes the dynamic micro-change content from both global and local perspectives, extracts local feature prediction vectors from all weighted start-vertex optical flow blocks, and extracts global feature prediction vectors from the start-vertex optical flow graph and the vertex-offset optical flow graph, and fuses the global and local feature prediction vectors to better improve the recognition performance. A weighted cross entropy loss function is also used to weight the training loss of the dynamic micro-change segment according to the category of each dynamic micro-change segment, overcoming the problem of excessive differences in the number of different categories of dynamic micro-change segments, i.e., imbalance. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0012] The specific implementation modes of the present invention are described in detail below: like Figure 1 As shown, the dynamic micro-change recognition method based on multi-granularity feature fusion and weighted entropy loss includes the following steps: Step S1, the data set is composed of a number of dynamic micro-change segments, each of which corresponds to a category and is annotated using label alignment. Each dynamic micro-change segment is composed of a number of video frames arranged in sequence from front to back.

[0013] Preprocess each dynamic micro-change segment in the dataset: For a dynamic micro-change segment, first detect the feature points on the first video frame in the dynamic micro-change segment, and crop the main area of ​​the first video frame according to these feature points. Then, crop the remaining video frames in the dynamic micro-change segment according to the area of ​​the first video frame to obtain a preprocessed dynamic micro-change segment. Then, each dynamic micro-change segment in the data set is preprocessed according to the above method to obtain a preprocessed data set.

[0014] Step S2, the first video frame of each dynamic micro-change segment of the preprocessed data set is called the starting frame, the last video frame is called the offset frame, and the video frame with the largest expression intensity in the dynamic micro-change segment recorded in the preprocessed data set is called the vertex frame.

[0015] The start-vertex optical flow map between the start frame and the vertex frame is calculated, and then the start-vertex optical flow map is weighted in blocks to obtain weighted start-vertex optical flow blocks. The feature vector of each weighted start-vertex optical flow block is then extracted and merged to obtain the local feature prediction vector of the dynamic micro-change fragment.

[0016] Specifically include: Step S21, first use the TV-L1 algorithm to calculate the start-vertex optical flow graph between the start frame and the vertex frame of the pre-processed dynamic slight change segment Then, the start-vertex optical flow graph The grid is divided into multiple square start-vertex optical flow blocks, which are sent to the block weight generation module at the same time to calculate the weight of each start-vertex optical flow block; In the block weight generation module, the two-dimensional convolution operation first converts the start-vertex optical flow graph The channel dimension of each start-vertex optical flow block is expanded, and then the attention module captures the relationship between these start-vertex optical flow blocks to obtain the output vector of the attention module Next, the output vector of the attention module is After being processed by the fully connected layer and the normalization layer, the dimension is reduced, and then the activation function Sigmoid is used to activate the output vector of the attention module. Calculate and finally get the weight of the start-vertex optical flow block : ; ; in, Represents the weight of the start-vertex optical flow block, weight Each value in is, in turn, the weight of each start-vertex optical flow block; represents the output vector of the attention module, represents a two-dimensional convolution operation, represents the attention module, represents the fully connected layer, represents the normalization layer, Represents the activation function.

[0017] Step S22, weighting each start-vertex optical flow block by multiplying each start-vertex optical flow block by the weight of the corresponding position in the weight vector to obtain each weighted start-vertex optical flow block, which is expressed as follows: ; ; in, Indicates Row and The start-vertex optical flow block of the column, Indicates Row and The weighted start-vertex optical flow block of the column, is the serial number, Indicates the maximum number of start-vertex optical flow block arrangements in the horizontal direction, The weight vector of the start-vertex optical flow block is The weight of the start-vertex optical flow block.

[0018] Step S23, after performing a weighted operation on each start-vertex optical flow block, extract the feature vector of each weighted start-vertex optical flow block, and splice the feature vectors of each weighted start-vertex optical flow block together, and finally obtain the local features of the dynamic slight change segment.

[0019] Specifically include: Through two-dimensional convolution and maximum pooling operations, the activation function The feature vector of each weighted start-vertex optical flow block is extracted. Then, the feature vector of each weighted start-vertex optical flow block is flattened into a one-dimensional vector, and all the flattened one-dimensional vectors are connected together using the vector dimension connection operation to obtain the local feature vector of the dynamic micro-change segment. : ; ; in, is the local feature vector of the dynamic micro-change fragment obtained by concatenating the feature vectors of all weighted start-vertex optical flow blocks, Indicates Row and The feature vector of the weighted start-vertex optical flow block of the column, is the vector dimension concatenation operation, represents the maximum number of weighted start-vertex optical flow block arrangements in the horizontal direction, Indicates the maximum number of weighted start-vertex optical flow block arrangements in the vertical direction, is the maximum pooling operation, is the activation function, Performs a vector flattening operation.

[0020] Step S24: local feature vector After calculations in the fully connected layer and the normalized layer, the prediction results for each category are obtained, that is, the local feature prediction vector : ; in, represents the fully connected layer, represents the normalization layer, Represents an average pooling operation.

[0021] Step S3, first calculate the optical flow graph between the vertex frame and the offset frame, and the start-vertex optical flow Figure 1 The image is processed from beginning to end, and after block, convolution and pooling operations, the start-vertex optical flow block features and vertex-offset optical flow block features are extracted respectively. The attention module is used to continue to extract the start-vertex optical flow block relationship vector and the vertex-offset optical flow block relationship vector. Finally, the fully connected layer and the normalization layer are used to compress and refine the two relationship vectors to obtain the start-vertex optical flow prediction vector and the vertex-offset optical flow prediction vector. After merging the two optical flow prediction vectors, the global feature prediction vector is obtained.

[0022] Specifically include: Step S31, using the TV-L1 algorithm to calculate the vertex-offset optical flow graph between the vertex frame and the offset frame of the pre-processed dynamic micro-change segment Then the start-vertex optical flow graph and vertex-offset optical flow graph The grid is divided into multiple square optical flow blocks, and then two-dimensional convolution and maximum pooling operations are performed. The process is as follows: ; ; in represents the start-vertex optical flow block feature, represents the vertex-offset optical flow block feature, represents the start-vertex optical flow graph, represents the vertex-offset optical flow graph, Indicates that the input optical flow graph is divided into multiple square optical flow blocks according to the grid. represents the maximum pooling operation, Represents a two-dimensional convolution operation.

[0023] Step S32, using the attention module to further extract the start-vertex optical flow block features The relationship vector between and vertex-offset optical flow block features The relationship vector between: ; ; in, represents the start-vertex optical flow block relationship vector, represents the vertex-offset optical flow block relationship vector, represents a vector flattening operation, Represents the attention module.

[0024] Step S33, using the calculation of the fully connected layer and the normalized layer, the start-vertex optical flow block relationship vector and vertex-offset optical flow block relation vector After compression and refinement, we finally get the start-vertex optical flow prediction vector and the vertex-offset optical flow prediction vector: ; ; in, represents the start-vertex optical flow prediction vector, Represents the vertex-offset optical flow prediction vector. represents the fully connected layer, represents a normalization layer.

[0025] Step S34: The start-vertex optical flow prediction vector and vertex-offset optical flow prediction vector Add together to get the global feature prediction vector for the dynamic micro-change segment : .

[0026] Step S4: global feature prediction vector and the local feature prediction vector Add together to get the prediction vector for the dynamic micro-change segment : .

[0027] Step S5: Considering that the proportion of dynamic micro-change segments of different categories in the data set varies greatly and there is an imbalance problem, a weighted cross entropy loss function is used to train the proposed dynamic micro-change recognition method. The loss function is recorded as . On the basis of the traditional cross entropy loss function, the training loss of the dynamic slight change fragments is weighted according to the category of each dynamic slight change fragment. The larger the proportion of the number of dynamic slight change fragments corresponding to the category in the data set, the smaller the weight assigned, and the smaller the proportion, the more weight assigned.

[0028] The details are as follows: ; in, Indicates the number of dynamic micro-change segments, Indicates the number of classifications of dynamic micro-change segments, Indicates a dynamic micro-change fragment, Indicates the true category of the dynamic micro-change segment. The category is equal to hour, The value is 1, otherwise it is 0. Indicates dynamic micro-change fragments Belongs to category The predicted probability when Indicates category The corresponding loss weight.

[0029] It should be understood that the parts not elaborated in detail in this specification belong to the prior art. The above embodiments are only descriptions of the preferred implementation methods of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made by ordinary engineers and technicians in this field to the technical solution of the present invention should fall within the protection scope determined by the claims of the present invention.

Claims

1. A dynamic micro-change recognition method based on multi-granularity feature fusion and weighted entropy loss, characterized in that: It includes the following steps: Step S1, preprocessing each dynamic micro-change segment in the data set; Step S2, calculating the start-vertex optical flow map between the start frame and the vertex frame according to the preprocessed data set, weighting the start-vertex optical flow map blocks to obtain weighted start-vertex optical flow blocks, extracting the feature vector of each weighted start-vertex optical flow block, and merging them to obtain the local feature prediction vector of the dynamic micro-change segment; Step S3, calculating the optical flow map between the vertex frame and the offset frame according to the preprocessed data set, and extracting the start-vertex optical flow block features and the vertex-offset optical flow block features respectively through block division, convolution and pooling operations together with the start-vertex optical flow map, using the attention module to continue to extract the start-vertex optical flow block relationship vector and the vertex-offset optical flow block relationship vector, using the fully connected layer and the normalization layer to compress and refine the two relationship vectors, and obtaining the start-vertex optical flow prediction vector and the vertex-offset optical flow prediction vector, and after merging the two optical flow prediction vectors, obtaining the global feature prediction vector; Step S4, obtaining a prediction vector for the dynamic slight change segment according to the local feature prediction vector and the global feature prediction vector; Step S5: Use the weighted cross entropy loss function to train the dynamic micro-change recognition method.

2. The dynamic micro-change identification method based on multi-granularity feature fusion and weighted entropy loss according to claim 1, characterized in that: In the step S1: For a dynamic micro-change segment, the feature points on the first video frame in the dynamic micro-change segment are first detected, and the main area of ​​the first video frame is cropped according to these feature points. Then, the remaining video frames in the dynamic micro-change segment are cropped according to the area of ​​the first video frame to obtain a preprocessed dynamic micro-change segment; then, each dynamic micro-change segment in the data set is preprocessed according to the above method to obtain a preprocessed data set.

3. The dynamic micro-change identification method based on multi-granularity feature fusion and weighted entropy loss as claimed in claim 1, characterized in that: In step S2: Step S21, using the TV-L1 algorithm to calculate the start-to-vertex optical flow graph between the start frame and the vertex frame of the pre-processed dynamic micro-change segment ; The start-vertex optical flow graph The grid is divided into multiple square start-vertex optical flow blocks, which are sent to the block weight generation module at the same time to calculate the weight of each start-vertex optical flow block; In the block weight generation module, the 2D convolution operation starts the vertex optical flow graph The channel dimension of each start-vertex optical flow block is expanded, and the attention module captures the relationship between these start-vertex optical flow blocks to obtain the output vector of the attention module. ; Output vector of the attention module After being processed by the fully connected layer and the normalized layer, the dimension is reduced, and the activation function Sigmoid is used to generate the output vector of the attention module. Calculate and finally get the weight of the start-vertex optical flow block : ； ； in, Represents the weight of the start-vertex optical flow block, weight Each value in is, in turn, the weight of each start-vertex optical flow block; represents the output vector of the attention module, represents a two-dimensional convolution operation, represents the attention module, represents the fully connected layer, represents the normalization layer, represents the activation function; Step S22, weighting each start-vertex optical flow block by multiplying each start-vertex optical flow block by the weight of the corresponding position in the weight vector to obtain each weighted start-vertex optical flow block, which is expressed as follows: ； ； in, Indicates Row and The start-vertex optical flow block of the column, Indicates Row and The weighted start-vertex optical flow block of the column, is the serial number, Indicates the maximum number of start-vertex optical flow block arrangements in the horizontal direction, The weight vector of the start-vertex optical flow block is The weight of the start-vertex optical flow block; Step S23, after performing a weighted operation on each start-vertex optical flow block, extracting a feature vector of each weighted start-vertex optical flow block, and splicing the feature vectors of each weighted start-vertex optical flow block together, and finally obtaining a local feature of the dynamic slight change segment; Specifically include: Through two-dimensional convolution and maximum pooling operations, the activation function Extract the feature vector of each weighted start-vertex optical flow block; flatten the feature vector of each weighted start-vertex optical flow block into a one-dimensional vector, connect all the flattened one-dimensional vectors together using the vector dimension connection operation, and obtain the local feature vector of the dynamic micro-change fragment : ； ； in, is the local feature vector of the dynamic micro-change fragment obtained by concatenating the feature vectors of all weighted start-vertex optical flow blocks, Indicates Row and The feature vector of the weighted start-vertex optical flow block of the column, is the vector dimension concatenation operation, represents the maximum number of weighted start-vertex optical flow block arrangements in the horizontal direction, Indicates the maximum number of weighted start-vertex optical flow block arrangements in the vertical direction, is the maximum pooling operation, is the activation function, Flatten operation for vectors; Step S24, local feature vector After calculations in the fully connected layer and the normalized layer, the prediction results for each category are obtained, that is, the local feature prediction vector : ； in, represents the fully connected layer, represents the normalization layer, Represents an average pooling operation.

4. The dynamic micro-change identification method based on multi-granularity feature fusion and weighted entropy loss according to claim 1, characterized in that: In step S3: Step S31, using the TV-L1 algorithm to calculate the vertex-offset optical flow graph between the vertex frame and the offset frame of the pre-processed dynamic micro-change segment ; The start-vertex optical flow graph and vertex-offset optical flow graph The grid is divided into multiple square optical flow blocks, and two-dimensional convolution and maximum pooling operations are performed. The process is as follows: ； ； in represents the start-vertex optical flow block feature, represents the vertex-offset optical flow block feature, represents the start-vertex optical flow graph, represents the vertex-offset optical flow graph, Indicates that the input optical flow graph is divided into multiple square optical flow blocks according to the grid. represents the maximum pooling operation, Represents a two-dimensional convolution operation; Step S32, using the attention module to further extract the start-vertex optical flow block features The relationship vector between and vertex-offset optical flow block features The relationship vector between: ； ； in, represents the start-vertex optical flow block relationship vector, represents the vertex-offset optical flow block relationship vector, represents a vector flattening operation, represents the attention module; Step S33, using the calculation of the fully connected layer and the normalized layer, the start-vertex optical flow block relationship vector and vertex-offset optical flow block relation vector After compression and refinement, we finally get the start-vertex optical flow prediction vector and the vertex-offset optical flow prediction vector: ； ； in, represents the start-vertex optical flow prediction vector, represents the vertex-offset optical flow prediction vector; represents the fully connected layer, represents the normalization layer; Step S34: The start-vertex optical flow prediction vector and vertex-offset optical flow prediction vector Add together to get the global feature prediction vector for the dynamic micro-change segment : 。 5. The dynamic micro-change identification method based on multi-granularity feature fusion and weighted entropy loss as claimed in claim 1, characterized in that: In step S4: the global feature prediction vector and the local feature prediction vector Add together to get the prediction vector for the dynamic micro-change segment : 。 6. The dynamic micro-change identification method based on multi-granularity feature fusion and weighted entropy loss according to claim 1, characterized in that: The weighted cross entropy loss function in step S5 : ； in, Indicates the number of dynamic micro-change segments, Indicates the number of classifications of dynamic micro-change segments, Indicates a dynamic micro-change fragment, Indicates the true category of the dynamic micro-change segment. The category is equal to hour, The value is 1, otherwise it is 0. Indicates dynamic micro-change fragments Belongs to category The predicted probability when Indicates category The corresponding loss weight.

Images