A deep learning-based dynamic image feature capturing method

Abstract

The application discloses a dynamic image feature capturing method based on deep learning, and relates to the technical field of image recognition. Through optimization of a deep convolutional neural network model, a convolutional layer in the deep convolutional neural network model which does not work or has a negative effect on a recognition result is removed, so that the calculation amount of the deep convolutional neural network model is reduced, and the accuracy of the deep convolutional neural network model is further improved on the basis of having completed training. Meanwhile, after a technical personnel deletes a feature marking frame of an error recognition through a tool, corresponding contrast video frames and original video frames are input as training samples of the deep convolutional neural network model, so that the recognition accuracy of the deep convolutional neural network model is more accurate.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically a method for capturing dynamic image features based on deep learning. Background Technology

[0002] Image features mainly include color features, texture features, shape features, and spatial relationship features. Color features are global features that describe the surface properties of the scene corresponding to the image or image region. Texture features are also global features that describe the surface properties of the scene corresponding to the image or image region. Shape features have two types of representation methods: contour features and region features. Contour features of an image mainly target the outer boundary of an object, while region features of an image relate to the entire shape region. Spatial relationship features refer to the spatial position or relative direction relationship between multiple targets segmented in an image. These relationships can also be divided into connection relationships, overlap relationships, and inclusion relationships, etc.

[0003] In existing technologies, when extracting features from dynamic images, the recognition results can easily become confused or erroneous when multiple features appear in the dynamic image. How to obtain more accurate dynamic image feature capture results is a problem we need to solve. To this end, we now provide a dynamic image feature capture method based on deep learning. Summary of the Invention

[0004] The purpose of this invention is to provide a dynamic image feature capture method based on deep learning.

[0005] The objective of this invention can be achieved through the following technical solution: a dynamic image feature capture method based on deep learning, comprising the following steps:

[0006] Step S1: Optimize the deep convolutional neural network model that has already been trained;

[0007] Step S2: Capture the dynamic image and preprocess the obtained dynamic image;

[0008] Step S3: Input the preprocessed dynamic image into the optimized deep convolutional neural model and output the feature capture results of the dynamic image;

[0009] Step S4: Based on the output feature capture results, form the movement trajectory of the features to be captured in the dynamic image.

[0010] Furthermore, the training process of a deep convolutional neural network model includes:

[0011] Construct a deep convolutional neural network model and train the constructed deep convolutional neural network; the deep convolutional neural network model consists of convolutional layers, pooling layers, and activation function layers;

[0012] The weights of each convolutional layer are set, and the type of impact of the convolutional layer on the deep convolutional neural network model is determined based on the weight coefficients of each convolutional layer.

[0013] Convolutional layers with negative or no impact are labeled, and the labeled convolutional layers are then removed to complete the optimization of the deep convolutional neural network model.

[0014] Furthermore, the preprocessing of the obtained dynamic images includes:

[0015] Convert the obtained dynamic image into a grayscale dynamic image;

[0016] Convert a grayscale dynamic image into several video frames;

[0017] Obtain the grayscale values ​​of each region within the video frame converted from each grayscale dynamic image, and associate the obtained grayscale values ​​with the corresponding positions within the video frame;

[0018] Establish a two-dimensional coordinate system, map video frames into the two-dimensional coordinate system, and mark the video frames mapped into the two-dimensional coordinate system as the original video frames;

[0019] All the original video frames obtained are input into the optimized deep convolutional neural network model.

[0020] Furthermore, after the original video frame is input into the deep convolutional neural network model, the regions corresponding to the features that need to be captured within the original video frame are marked.

[0021] The original video frames after marking are output to obtain the feature capture results of the dynamic image;

[0022] The output original video frame is marked as the reference video frame, and the reference video frame is mapped into a two-dimensional coordinate system. The reference video frame is then overlaid with the corresponding original video frame in the two-dimensional coordinate system to obtain the coordinate range of the marked feature in the two-dimensional coordinate system.

[0023] Furthermore, multiple features that need to be captured can exist within the same video frame. When there are at least two features that need to be identified within the video frame, the labels of the different features are all different.

[0024] Furthermore, the process of forming the movement trajectory of the features to be captured within the dynamic image includes:

[0025] Mark all the features that need to be captured in each comparison video frame;

[0026] Based on the time corresponding to each reference video frame, a corresponding timestamp is generated, and the generated timestamp is associated with all features captured within the corresponding reference video frame;

[0027] Then, the features marked in each comparison video frame are classified according to the different markings to form the corresponding feature sets;

[0028] To determine if there are at least two features in the feature set that belong to the same reference video frame, we need to find out if there is a misidentification.

[0029] Then, the corresponding video frame to which the feature belongs is retrieved, and technicians use tools to delete the incorrectly identified feature bounding boxes, and synchronize the deletion results to the feature set;

[0030] Features within the same feature set are sorted according to the time order of their associated timestamps, and the corresponding movement trajectories are formed in a two-dimensional coordinate system according to the sorting results. The formed movement trajectories are then mapped onto the dynamic image.

[0031] Compared with existing technologies, the beneficial effects of this invention are as follows: By optimizing the convolutional layers, convolutional layers that have no effect or negative effect on the recognition results in the deep convolutional neural network model are eliminated. On the one hand, this reduces the computational load of the deep convolutional neural network model, and on the other hand, it can further improve the accuracy of the deep convolutional neural network model based on the completed training. At the same time, after the technicians use tools to delete the incorrectly identified feature boxes, and use the corresponding control video frames and the original video frames as training samples for the deep convolutional neural network model, the recognition accuracy of the deep convolutional neural network model can also be improved. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0033] like Figure 1 As shown, a deep learning-based method for capturing dynamic image features includes the following steps:

[0034] Step S1: Optimize the deep convolutional neural network model that has already been trained;

[0035] Step S2: Capture the dynamic image and preprocess the obtained dynamic image;

[0036] Step S3: Input the preprocessed dynamic image into the optimized deep convolutional neural model and output the feature capture results of the dynamic image;

[0037] Step S4: Based on the output feature capture results, form the movement trajectory of the features to be captured in the dynamic image.

[0038] It should be further explained that, in the specific implementation process, the training process of the deep convolutional neural network model includes:

[0039] Construct a deep convolutional neural network model and train the constructed deep convolutional neural network; the deep convolutional neural network model consists of convolutional layers, pooling layers, and activation function layers;

[0040] Each convolutional layer is labeled and denoted as i, where i = 1, 2, ..., n, and n is an integer;

[0041] The weights of each convolutional layer are set, and the weight coefficients of each convolutional layer are denoted as Q. i ;

[0042] The convolutional layers are optimized based on the weight coefficients of each layer, i.e.:

[0043] Obtain the recognition accuracy of the trained deep convolutional neural network and mark the recognition accuracy of the trained deep convolutional neural network model as the initial recognition accuracy, denoted as CS;

[0044] The convolutional layers are removed sequentially, and the recognition accuracy of the deep convolutional neural network model after the corresponding convolutional layers are removed is obtained. This recognition accuracy is recorded as the reference recognition accuracy.

[0045] For example:

[0046] The convolutional layer labeled i=1 is removed, and the recognition accuracy of the deep convolutional neural network model containing only convolutional layers labeled i=2 to i=n is obtained.

[0047] Then, the convolutional layer labeled i=2 is removed, and the recognition accuracy of the deep convolutional neural network model containing only convolutional layers labeled i=1, i=3 to i=n is obtained, and so on.

[0048] After removing the convolutional layer labeled i, the reference recognition accuracy of the deep convolutional neural network model is denoted as CZ. i ;

[0049] The weight coefficients of the convolutional layer labeled i

[0050] If Q iIf the value is greater than 1, it means that the convolutional layer labeled i has a negative impact on the recognition accuracy of the deep convolutional neural network model.

[0051] If Q i =1 indicates that the convolutional layer with label i has no effect on the recognition accuracy of the deep convolutional neural network model;

[0052] If Q i If <1, it means that the convolutional layer labeled i has a positive impact on the recognition accuracy of the deep convolutional neural network model.

[0053] Convolutional layers with negative or no impact are labeled, and the labeled convolutional layers are removed to complete the optimization of the deep convolutional neural network model.

[0054] By optimizing the convolutional layers, convolutional layers that have no effect or negative impact on the recognition results in the deep convolutional neural network model can be removed. This reduces the computational cost of the deep convolutional neural network model and further improves its accuracy based on the completed training.

[0055] It should be further explained that, in the specific implementation process, the preprocessing of the obtained dynamic images includes:

[0056] The obtained dynamic image is converted into a grayscale dynamic image; it should be further noted that, in the specific implementation process, the dynamic image is a continuous video segment, and the grayscale dynamic image is a continuous video segment obtained after grayscale processing of the dynamic image;

[0057] The grayscale dynamic image is converted into several video frames, and each video frame is labeled as j, j = 1, 2, ..., m, where m is an integer;

[0058] Obtain the grayscale values ​​of each region within the video frame converted from each grayscale dynamic image, and associate the obtained grayscale values ​​with the corresponding positions within the video frame;

[0059] Establish a two-dimensional coordinate system, map video frames into the two-dimensional coordinate system, and mark the video frames mapped into the two-dimensional coordinate system as the original video frames;

[0060] All the original video frames obtained are input into the optimized deep convolutional neural network model.

[0061] It should be further explained that, in the specific implementation process, after the original video frame is input into the deep convolutional neural network model, the regions corresponding to the features that need to be captured in the original video frame are marked.

[0062] The original video frames after marking are output to obtain the feature capture results of the dynamic image;

[0063] The output original video frame is marked as the reference video frame, and the reference video frame is mapped into a two-dimensional coordinate system. The reference video frame is then overlaid with the corresponding original video frame in the two-dimensional coordinate system to obtain the coordinate range of the marked feature in the two-dimensional coordinate system. It should be further noted that in the specific implementation process, there may be multiple features to be captured in the same video frame, and these features may be similar. When there are at least two features to be identified in the video frame, the labels of the different features are different.

[0064] It should be further explained that, in the specific implementation process, the formation process of the movement trajectory of the features to be captured within the dynamic image includes:

[0065] Mark all the features that need to be captured in each comparison video frame;

[0066] Based on the time corresponding to each reference video frame, a corresponding timestamp is generated, and the generated timestamp is associated with all features captured within the corresponding reference video frame;

[0067] Then, the features marked in each comparison video frame are classified according to the different markings to form the corresponding feature sets;

[0068] To determine whether there are at least two features in the feature set that belong to the same reference video frame;

[0069] If there are no at least two features belonging to the same reference video frame, it means that there are no misidentified features;

[0070] If at least two features belong to the same reference video frame, then the corresponding features are marked and corresponding feature bounding boxes are generated.

[0071] The corresponding video frames containing the obtained features are retrieved, and technicians use tools to delete the incorrectly identified feature bounding boxes, and the deletion results are synchronized to the feature set.

[0072] By leveraging the relationship between timestamps and reference video frames, as well as the uniqueness of the same feature within the same reference video frame, when the same feature within the same video frame is misidentified, the misidentified portion can be marked, and technicians can process the marked misidentified portion to obtain accurate feature capture results.

[0073] Features within the same feature set are sorted according to the time order of their associated timestamps, and the corresponding movement trajectories are formed in a two-dimensional coordinate system according to the sorting results. The formed movement trajectories are then mapped onto the dynamic image.

[0074] It should be further explained that, in the specific implementation process, after the technicians delete the incorrectly identified feature bounding boxes using tools, they mark the corresponding reference video frame and the original video frame corresponding to the reference video frame. The reference video frame and the original video frame are then used as training samples for the deep convolutional neural network model, thereby further training the deep convolutional neural network model and making the recognition accuracy of the deep convolutional neural network model more accurate.

[0075] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for capturing dynamic image features based on deep learning, characterized in that, Includes the following steps: Step S1: Optimize the deep convolutional neural network model that has already been trained; Step S2: Capture the dynamic image and preprocess the obtained dynamic image; Step S3: Input the preprocessed dynamic image into the optimized deep convolutional neural model and output the feature capture results of the dynamic image; Step S4: Based on the output feature capture results, form the movement trajectory of the features to be captured within the dynamic image; The process of forming the movement trajectory of the features to be captured within a dynamic image includes: Mark all the features that need to be captured in each comparison video frame; Based on the time corresponding to each reference video frame, a corresponding timestamp is generated, and the generated timestamp is associated with all features captured within the corresponding reference video frame; Then, the features marked in each comparison video frame are classified according to the different markings to form the corresponding feature sets; To determine if there are at least two features in the feature set that belong to the same reference video frame, we need to find out if there is a misidentification. Then, the corresponding video frame to which the feature belongs is retrieved, and technicians use tools to delete the incorrectly identified feature bounding boxes, and synchronize the deletion results to the feature set; Features within the same feature set are sorted according to the time order of their associated timestamps, and the corresponding movement trajectories are formed in a two-dimensional coordinate system according to the sorting results. The formed movement trajectories are then mapped onto the dynamic image.

2. The method for capturing dynamic image features based on deep learning according to claim 1, characterized in that, The training process of a deep convolutional neural network model includes: Construct a deep convolutional neural network model and train the constructed deep convolutional neural network; the deep convolutional neural network model consists of convolutional layers, pooling layers, and activation function layers; The weights of each convolutional layer are set, and the type of impact of the convolutional layer on the deep convolutional neural network model is determined based on the weight coefficients of each convolutional layer. Convolutional layers with negative or no impact are labeled, and the labeled convolutional layers are then removed to complete the optimization of the deep convolutional neural network model.

3. The method for capturing dynamic image features based on deep learning according to claim 2, characterized in that, The preprocessing of the acquired dynamic images includes: Convert the obtained dynamic image into a grayscale dynamic image; Convert a grayscale dynamic image into several video frames; Obtain the grayscale values ​​of each region within the video frame converted from each grayscale dynamic image, and associate the obtained grayscale values ​​with the corresponding positions within the video frame; Establish a two-dimensional coordinate system, map video frames into the two-dimensional coordinate system, and mark the video frames mapped into the two-dimensional coordinate system as the original video frames; All the original video frames obtained are input into the optimized deep convolutional neural network model.

4. The method for capturing dynamic image features based on deep learning according to claim 3, characterized in that, After the original video frame is input into the deep convolutional neural network model, the regions corresponding to the features that need to be captured in the original video frame are marked. The original video frames after marking are output to obtain the feature capture results of the dynamic image; The output original video frame is marked as the reference video frame, and the reference video frame is mapped into a two-dimensional coordinate system. The reference video frame is then overlaid with the corresponding original video frame in the two-dimensional coordinate system to obtain the coordinate range of the marked feature in the two-dimensional coordinate system.

5. The method for capturing dynamic image features based on deep learning according to claim 4, characterized in that, Multiple features that need to be captured can exist within the same video frame. When there are at least two features that need to be identified within the video frame, the labels of the different features are not the same.

Images