Image recognition method and device based on lightweight dual-channel feature progressive extraction

Abstract

The present invention relates to an image recognition method based on lightweight dual-channel progressive feature extraction, including: S10, obtaining a target instance image; S20, performing image recognition on the target instance image through a pre-trained LDC-PLE recognition model to obtain a recognition result. The present invention can improve the performance, versatility and applicability of traditional PLE models by fusing two pre-trained models such as ResNet with PLE and performing dual-channel extraction on the data set, while finding a balance in terms of computational efficiency and feature expression, and is particularly suitable for multi-task learning and transfer learning scenarios. This fusion method gives full play to the advantages of both and improves the performance of deep learning models in various computer vision tasks. Adding a convolutional layer before the FC layer of PLE can significantly reduce the number of parameters, that is, lightweight, making the model suitable for high-dimensional data, while enhancing the model's ability to process structured data. In addition, considering the particularity of the data source for ship recognition, the present invention is particularly suitable for ship recognition.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to an image recognition method and device based on lightweight dual-channel feature progressive extraction. Background Art

[0002] At present, Progressive Layered Extraction (PLE) networks have been applied to artificial intelligence multi-task learning. When the two tasks to be learned have a certain correlation, the performance of multi-task learning is improved by gradually extracting hierarchical features, solving the problem of difficulty in optimizing multiple goals at the same time, that is, the improvement of one task indicator is accompanied by the decline of other task indicators.

[0003] However, there are some potential defects in the current use of Progressive Layered Extraction (PLE), including the limitation of extracting low-level features, high computational complexity, limitation of universality, and insufficient transfer learning ability. At the same time, the FC (Fully Connected) layer used in the PLE network is only suitable for low-dimensional data and small number of parameters, and performs poorly for structured data. The use of PLE alone in image recognition and the use of FC layer alone in PLE have the disadvantage of not being able to efficiently extract data features, thereby realizing multi-task classification of images. Summary of the invention

[0004] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide an image recognition method and device based on lightweight dual-channel feature progressive extraction.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] Specifically, an image recognition method based on lightweight dual-channel feature progressive extraction is proposed, including the following:

[0007] S10, obtaining a target instance image;

[0008] S20, performing image recognition on the target instance image through the pre-trained LDC-PLE recognition model to obtain a recognition result;

[0009] Specifically, the process of building a pre-trained LDC-PLE recognition model includes:

[0010] S21, obtaining a plurality of related data sets from different data sources, and annotating the data sets according to instance-level categories to obtain annotated data sets;

[0011] S22. Build two ResNets as two independent sub-models;

[0012] S23, connecting the output tensor of ResNet to the input tensor of the PLE network, passing the preliminary feature representation extracted from ResNet to the input layer of the PLE network as the input of the PLE, wherein the output data of the first ResNet is connected to the task A module of the PLE network, and the output data of the second ResNet is connected to the task B module of the PLE network, and the shared task module switches the input data of the first and second ResNet in turn, and trains the model based on the labeled data set;

[0013] S24. Add at least one convolutional layer before the FC layer of the bottom CGC customized gating network layer of the PLE network model to obtain a pre-trained LDC-PLE recognition model.

[0014] Furthermore, the method further comprises:

[0015] Before performing image recognition on the target instance image through the pre-trained LDC-PLE recognition model, a preprocessing operation is also performed on the target instance image, and the preprocessing operation includes resizing, normalizing and data enhancement processing on the target instance image.

[0016] Furthermore, specifically, two convolutional layers are added before the FC layer of the bottom CGC custom gated network layer of the PLE network model to obtain a pre-trained LDC-PLE recognition model. The specific processing process includes:

[0017] Assume that the preprocessed image is a 28×28 grayscale image, denoted as I, where I(i, j) represents the pixel value at position (i, j) in I, and passes through a 3×3 convolution kernel K in the convolution layer. (1) Calculate the output C (1) :

[0018]

[0019] Among them, C (1) (i, j) represents the value at position (i, j) in the convolution output. In the pooling layer, the maximum pooling operation is performed to retain the maximum value in each 2×2 window, that is:

[0020] P (1) (i,j)=max(C (1) (2i,2j),C (1) (2i,2j+1),C (1) (2i+1,2j),C (1) (2i+1,2j+1))

[0021] Where P (1)(i,j) represents the value at position (i,j) in the pooled output;

[0022] The operation of the second convolution layer is defined in the same way as above, using a 3×3 convolution kernel K (2) Calculate the convolution kernel output C (2) , use the maximum pooling operation again to retain the maximum value in each 2×2 window and get the pooling output P (2) , then P (2) Flattened into a one-dimensional vector for feature extraction by CGC of PLE multi-layer.

[0023] Furthermore, specifically, in the preprocessing operation, normalization is to scale pixel values ​​to a specific range, and data enhancement includes flipping, rotating, and cropping operations on the target instance image.

[0024] Further, specifically, multiple related data sets from different data sources include optical imaging, infrared imaging, thermal imaging, and grayscale images.

[0025] The present invention also proposes an image recognition device based on lightweight dual-channel feature progressive extraction, comprising:

[0026] Data acquisition module, used to obtain target instance images;

[0027] The image recognition module is used to perform image recognition on the target instance image through the pre-trained LDC-PLE recognition model to obtain the recognition result.

[0028] The present invention also proposes a ship image recognition device based on lightweight dual-channel feature progressive extraction, and applies the image recognition device based on lightweight dual-channel feature progressive extraction, wherein the target instance picture acquired by the data acquisition module is a ship image.

[0029] The beneficial effects of the present invention are:

[0030] The present invention proposes an image recognition method based on lightweight dual-channel feature progressive extraction. By fusing two pre-trained models such as ResNet with PLE and performing dual-channel extraction on the data set, the performance, versatility and applicability of the traditional PLE model can be improved, while finding a balance between computational efficiency and feature expression, which is particularly suitable for multi-task learning and transfer learning scenarios. This fusion method gives full play to the advantages of both and improves the performance of deep learning models in various computer vision tasks. Adding a convolutional layer before the FC layer of PLE can significantly reduce the number of parameters, that is, lightweight, making the model suitable for high-dimensional data, while enhancing the model's ability to process structured data. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0032] Figure 1 It is a flow chart of the image recognition method based on lightweight dual-channel feature progressive extraction of the present invention;

[0033] Figure 2 The figure shows a flow chart of the present invention introducing the first two ResNets of PLE to realize dual-channel feature extraction;

[0034] Figure 3 The flowchart of the present invention introducing convolution operation before the FC layer of CGC at the bottom of PLE is shown;

[0035] Figure 4 Shown is a schematic diagram of the network structure of the LDC-PLE recognition model proposed in the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0037] In multi-task learning, different tasks may compete or interfere with each other. The model needs to find a proper balance between multiple tasks so that it can achieve good performance on different tasks. This may lead to difficulties in generalizing to new tasks or unseen data. The feature extraction of the convolutional layer of LDC-PLE enables the model to better adapt to the input of different data sources. This helps to improve the model's generalization ability to different data sources.

[0038] The advantage of the LDC-PLE method is that it solves the technical problems of feature hierarchy, computational complexity and training speed by introducing pre-trained ResNet for feature extraction, and also solves the technical problems of data source differences, information loss and generalization difficulties by introducing convolutional layers at the bottom of the original network for feature extraction. This enables the model to better understand image details and hierarchical structures, adapt to different data sources, improve model performance and generalization capabilities, speed up convergence, and make it a more powerful multi-task learning model.

[0039] Reference Figure 1 , Figure 2 , Figure 3 as well as Figure 4 , Embodiment 1, based on the above idea, the present invention proposes an image recognition method based on lightweight dual-channel feature progressive extraction, including the following:

[0040] S10, obtaining a target instance image;

[0041] S20, performing image recognition on the target instance image through the pre-trained LDC-PLE recognition model to obtain a recognition result;

[0042] Specifically, the process of building a pre-trained LDC-PLE recognition model includes:

[0043] S21, obtaining a plurality of related data sets from different data sources, and annotating the data sets according to instance-level categories to obtain annotated data sets;

[0044] S22. Build two ResNets as two independent sub-models;

[0045] S23, connecting the output tensor of ResNet to the input tensor of the PLE network, passing the preliminary feature representation extracted from ResNet to the input layer of the PLE network as the input of the PLE, wherein the output data of the first ResNet is connected to the task A module of the PLE network, and the output data of the second ResNet is connected to the task B module of the PLE network, and the shared task module switches the input data of the first and second ResNet in turn, and trains the model based on the labeled data set;

[0046] S24. Add at least one convolutional layer before the FC layer of the bottom CGC customized gating network layer of the PLE network model to obtain a pre-trained LDC-PLE recognition model.

[0047] In this embodiment, the formula of the residual block in ResNet can be expressed as:

[0048]

[0049] Where x represents the input, Represents the output of the residual mapping function, and y represents the final output.

[0050] For example: the input x is a feature tensor, which can represent an image or the feature representation obtained by the previous layer. Represents the residual map calculated by operations such as convolution and activation functions. The output of this function is a feature vector with the same dimension as the input x. According to the ResNet formula Mapping function between input x and residual Add them together to get the output y.

[0051] Compared with the prior art, the present invention mainly has the following process improvements:

[0052] The first two ResNets of PLE are introduced for dual-channel feature extraction, which includes the following steps:

[0053] Step 1: Build two ResNet submodules, each for feature extraction of two different data sets.

[0054] Step 2: Concatenate the features of the two data sources by channel and connect the output of ResNet with the input of PLE.

[0055] The convolution operation is introduced before the FC layer of CGC at the bottom of PLE, which includes the following steps:

[0056] Step 1: Introduce one or more convolutional layers at the initial level of the deep neural network.

[0057] The convolution layer usually includes convolution operations and pooling operations to extract local features from the input data. The specific steps are:

[0058] Step 1.1: Apply a convolution operation to the input data. The convolution kernel slides over the input data, performing a convolution operation to detect and extract features from different locations of the input data. These features are captured as feature maps.

[0059] Step 1.2: Pooling can be applied after the convolution operation. Pooling helps reduce the dimension of the feature map and extract more abstract features. Common pooling operations include maximum pooling and average pooling.

[0060] Step 2: Pass the feature map to the subsequent layers of the neural network.

[0061] Through the above process, the dual-channel feature extraction of the first two convolutional neural networks of PLE was introduced to solve the following technical problems:

[0062] First, PLE usually focuses on hierarchical feature extraction, because PLE used alone focuses more on the extraction of high-level features, which may not fully capture the low-level features in the original image. This may lead to performance degradation in some tasks when low-level features are required. LDC-PLE can fully utilize the advantages of ResNet, while extracting features in a more refined and customized manner through the hierarchical approach of PLE, thereby achieving better performance on specific tasks.

[0063] At the same time, using PLE alone may not handle multi-task learning problems well. This may also lead to a decline in the performance of the model on some tasks. LDC-PLE can improve the representation of the input image, so that the hierarchical feature extraction of PLE can better understand and analyze the image, which helps the model better understand the detailed information and hierarchical structure of the image.

[0064] Finally, using PLE alone may limit the performance of the model in new domains. Because networks such as ResNet have already been transferred learned. Inputting data into LDC-PLE that has been transferred learned can take advantage of these learned common features. This helps the model converge faster on new tasks and provides better initial feature representations.

[0065] At the same time, the convolution operation is introduced before the FC layer of CGC to achieve lightweight and solve the following technical problems:

[0066] Inputs from different data sources have different characteristics and structures, and relevant features need to be captured and extracted effectively. By introducing convolutional layers, LDC-PLE can better process inputs from different data sources and extract useful information, thereby providing richer feature representations for subsequent task learning.

[0067] The fully connected layer in the PLE before the improvement fully connects the high-dimensional features of the input data, which may lead to information loss and redundancy. The feature extraction process before the introduction of the convolutional layer allows the model to extract more representative features in the initial stage. The convolutional layer helps to preserve the local structure and characteristics of the input data and reduce information loss.

[0068] As a preferred embodiment of the present invention, the method further comprises:

[0069] Before performing image recognition on the target instance image through the pre-trained LDC-PLE recognition model, a preprocessing operation is also performed on the target instance image, and the preprocessing operation includes resizing, normalizing and data enhancement processing on the target instance image.

[0070] As a preferred embodiment of the present invention, specifically, two convolutional layers are added before the FC layer of the bottom CGC custom gated network layer of the PLE network model to obtain a pre-trained LDC-PLE recognition model. The specific processing process includes:

[0071] Assume that the preprocessed image is a 28×28 grayscale image, denoted as I, where I(i, j) represents the pixel value at position (i, j) in I, and passes through a 3×3 convolution kernel K in the convolution layer. (1) Calculate the output C (1) :

[0072]

[0073] Among them, C (1) (i, j) represents the value at position (i, j) in the convolution output. In the pooling layer, the maximum pooling operation is performed to retain the maximum value in each 2×2 window, that is:

[0074] P (1) (i,j)=max(C (1) (2i,2j),C (1) (2i,2j+1),C (1) (2i+1,2j),C (1) (2i+1,2j+1))

[0075] Where P (1) (i,j) represents the value at position (i,j) in the pooled output;

[0076] The operation of the second convolution layer is defined in the same way as above, using a 3×3 convolution kernel K (2) Calculate the convolution kernel output C (2) , use the maximum pooling operation again to retain the maximum value in each 2×2 window and get the pooling output P (2) , then P (2) Flattened into a one-dimensional vector for feature extraction by CGC of PLE multi-layer.

[0077] As a preferred implementation of the present invention, specifically, in the preprocessing operation, normalization is to scale the pixel values ​​to a specific range, and data enhancement includes flipping, rotating, and cropping operations on the target instance image.

[0078] As a preferred embodiment of the present invention, specifically, the multiple related data sets from different data sources include optical imaging, infrared imaging, thermal imaging, and grayscale images.

[0079] The present invention also proposes an image recognition device based on lightweight dual-channel feature progressive extraction, comprising:

[0080] Data acquisition module, used to obtain target instance images;

[0081] The image recognition module is used to perform image recognition on the target instance image through the pre-trained LDC-PLE recognition model to obtain the recognition result.

[0082] The present invention also proposes a ship image recognition device based on lightweight dual-channel feature progressive extraction, and applies the image recognition device based on lightweight dual-channel feature progressive extraction, wherein the target instance picture acquired by the data acquisition module is a ship image.

[0083] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0084] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0085] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.

[0086] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. Image recognition method based on lightweight dual-channel feature progressive extraction, It is characterized in that Includes the following: S10, obtaining a target instance image; S20, performing image recognition on the target instance image through the pre-trained LDC-PLE recognition model to obtain a recognition result; Specifically, the process of building a pre-trained LDC-PLE recognition model includes: S21, obtaining a plurality of related data sets from different data sources, and annotating the data sets according to instance-level categories to obtain annotated data sets; S22. Build two ResNets as two independent sub-models; S23, connecting the output tensor of ResNet to the input tensor of the PLE network, passing the preliminary feature representation extracted from ResNet to the input layer of the PLE network as the input of the PLE, wherein the output data of the first ResNet is connected to the task A module of the PLE network, and the output data of the second ResNet is connected to the task B module of the PLE network, and the shared task module switches the input data of the first and second ResNet in turn, and trains the model based on the labeled data set; S24. Add at least one convolutional layer before the FC layer of the bottom CGC customized gating network layer of the PLE network model to obtain a pre-trained LDC-PLE recognition model.

2. The image recognition method based on lightweight dual-channel feature progressive extraction according to claim 1, It is characterized in that The method further comprises, Before performing image recognition on the target instance image through the pre-trained LDC-PLE recognition model, a preprocessing operation is also performed on the target instance image, and the preprocessing operation includes resizing, normalizing and data enhancement processing on the target instance image.

3. The image recognition method based on lightweight dual-channel feature progressive extraction according to claim 2, It is characterized in that Specifically, two convolutional layers are added before the FC layer of the bottom layer CGC custom gated network layer of the PLE network model to obtain a pre-trained LDC-PLE recognition model. The specific processing process includes: Assume that the preprocessed image is a 28×28 grayscale image, denoted as I, where I(i, j) represents the pixel value at position (i, j) in I, and passes through a 3×3 convolution kernel K in the convolution layer. (1) Calculate the output C (1) : Among them, C (1) (i, j) represents the value at position (i, j) in the convolution output. In the pooling layer, the maximum pooling operation is performed to retain the maximum value in each 2×2 window, that is: <h2 style=";text-align:left;direction:ltr">P<h2 style=";text-align:left;direction:ltr"> (1) <h2 style=";text-align:left;direction:ltr"> (i,j) = max(C<h2 style=";text-align:left;direction:ltr"> (1) <h2 style=";text-align:left;direction:ltr"> (2i,2j),C<h2 style=";text-align:left;direction:ltr"> (1) <h2 style=";text-align:left;direction:ltr"> (2i,2j+1),C<h2 style=";text-align:left;direction:ltr"> (1) <h2 style=";text-align:left;direction:ltr"> (2i+1,2j),C<h2 style=";text-align:left;direction:ltr"> (1) <h2 style=";text-align:left;direction:ltr"> (2i+1,2j+1)) Where P (1) (i,j) represents the value at position (i,j) in the pooled output; The operation of the second convolution layer is defined in the same way as above, using a 3×3 convolution kernel K (2) Calculate the convolution kernel output C (2) , use the maximum pooling operation again to retain the maximum value in each 2×2 window and get the pooling output P (2) , then P (2) Flattened into a one-dimensional vector for feature extraction by CGC of PLE multi-layer.

4. The image recognition method based on lightweight dual-channel feature progressive extraction according to claim 3, It is characterized in that Specifically, in the preprocessing operation, normalization is to scale the pixel values ​​to a specific range, and data enhancement includes flipping, rotating, and cropping operations on the target instance image.

5. The image recognition method based on lightweight dual-channel feature progressive extraction according to claim 1, It is characterized in that Specifically, multiple related data sets from different data sources include optical imaging, infrared imaging, thermal imaging, and grayscale images.

6. Image recognition device based on lightweight dual-channel feature progressive extraction, It is characterized in that include: Data acquisition module, used to obtain target instance images; The image recognition module is used to perform image recognition on the target instance image through the pre-trained LDC-PLE recognition model to obtain the recognition result.

7. Ship image recognition device based on lightweight dual-channel progressive feature extraction, It is characterized in that The image recognition device based on lightweight dual-channel progressive feature extraction as described in claim 6 above is applied, wherein the target instance image acquired by the data acquisition module is a ship image.

Images