Image feature extraction method and device, equipment and medium

Abstract

The embodiment of the invention provides an image feature extraction method and device, equipment and a medium, and the method comprises the steps: carrying out the feature extraction of a to-be-processed image through a first convolution module of a convolutional neural network; the first convolution module comprises a first convolution unit and a first residual unit; at least two second convolution units of the first residual unit are used for performing convolution processing by adopting an n * n convolution kernel under the condition that the number of output channels is not changed; performing feature extraction on the first image features by using a second convolution module; the second convolution module comprises a third convolution unit and a second residual unit; performing feature extraction on the second image feature by using a third convolution module to obtain a third image feature; the third convolution module comprises a cross-stage local unit; and performing feature extraction on the third image features by using a fourth convolution module. According to the embodiment of the invention, the method can achieve the low-parameter and high-precision feature extraction of a target, and can remarkably improve the performance and efficiency of image recognition.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, apparatus, device, and medium for extracting image features. Background Technology

[0002] Image recognition refers to the technology of using computers to process, analyze, and understand images in order to identify targets and objects of various patterns. Image feature extraction techniques directly affect the accuracy, adaptability, and stability of image recognition; therefore, image feature extraction techniques are of paramount importance. Summary of the Invention

[0003] This application provides an image feature extraction method that can achieve low-parameter, high-precision feature extraction, thereby significantly improving the performance and efficiency of image recognition.

[0004] Accordingly, embodiments of this application also provide an image feature extraction device, an electronic device, and a machine-readable medium to ensure the implementation and application of the above methods.

[0005] To address the aforementioned problems, this application discloses a method for extracting image features, the method comprising:

[0006] Receive the image to be processed;

[0007] The first convolutional module of a convolutional neural network is used to extract features from the image to be processed, thereby obtaining first image features. The first convolutional module includes at least one first convolutional unit and at least one first residual unit connected in sequence. The at least one first convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The first main branch of the first residual unit includes at least two second convolutional units, which are used to perform convolution processing with an n×n convolutional kernel without changing the number of output channels. n is a positive integer greater than 1.

[0008] The second convolutional module of a convolutional neural network is used to extract features from the first image features to obtain second image features. The second convolutional module includes at least one third convolutional unit and at least one second residual unit connected in sequence. The at least one third convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The second main branch of the second residual unit includes a fourth convolutional unit and a fifth convolutional unit. The fourth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing, and the fifth convolutional unit is used to decrease the number of output channels of the image features based on convolution processing.

[0009] The third convolutional module of a convolutional neural network is used to extract features from the second image features to obtain the third image features. The third convolutional module includes at least one sixth convolutional unit and at least one cross-stage local unit connected in sequence. The sixth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The corresponding input image features are divided into first features and second features according to the channels. The cross-stage local unit is used to perform convolution processing on the first features and the second features respectively, and to concatenate the channel numbers of the convolution processing results corresponding to the first features and the second features respectively. The grouping parameter values ​​of the convolutional units in the first convolutional module, the second convolutional module, and the third convolutional module are all 1, or consistent with the number of input channels.

[0010] The fourth convolutional module of the convolutional neural network is used to extract features from the third image features to obtain the fourth image features; the fourth convolutional module adopts an inverted residual structure; the inverted residual structure contains convolutional units with grouping parameter values ​​greater than 1;

[0011] The fourth image feature is output as the feature extraction result.

[0012] This application also discloses an image feature extraction device, the device comprising:

[0013] The receiving module is used to receive the image to be processed;

[0014] A first feature extraction module is used to extract features from the image to be processed using a first convolutional module of a convolutional neural network to obtain first image features. The first convolutional module includes at least one first convolutional unit and at least one first residual unit connected in sequence. The at least one first convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The first main branch of the first residual unit includes at least two second convolutional units, which are used to perform convolution processing with an n×n convolutional kernel without changing the number of output channels; n is a positive integer greater than 1.

[0015] The second feature extraction module is used to extract features from the first image features using the second convolution module of the convolutional neural network to obtain second image features. The second convolution module includes at least one third convolution unit and at least one second residual unit connected in sequence. The at least one third convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The second main branch of the second residual unit includes a fourth convolution unit and a fifth convolution unit. The fourth convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing, and the fifth convolution unit is used to decrease the number of output channels of the image features based on convolution processing.

[0016] The third feature extraction module is used to extract features from the second image features using the third convolution module of the convolutional neural network to obtain the third image features. The third convolution module includes at least one sixth convolution unit and at least one cross-stage local unit connected in sequence. The sixth convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The corresponding input image features are divided into first features and second features according to the channels. The cross-stage local unit is used to perform convolution processing on the first features and the second features respectively, and to concatenate the channel numbers of the convolution processing results corresponding to the first features and the second features respectively. The grouping parameter values ​​of the convolution units in the first convolution module, the second convolution module, and the third convolution module are all 1, or consistent with the number of input channels.

[0017] The fourth feature extraction module is used to extract features from the third image features using the fourth convolution module of the convolutional neural network to obtain the fourth image features; the fourth convolution module adopts an inverted residual structure; the inverted residual structure contains convolutional units with grouping parameter values ​​greater than 1;

[0018] The output module is used to output the fourth image features as the feature extraction result.

[0019] Optionally, the first residual unit further includes: a first shortcut branch and a first fusion unit;

[0020] The first fusion unit is used to fuse the main branch output of the first main branch and the shortcut branch output of the first shortcut branch without changing the number of output channels.

[0021] Optionally, the kernel size of the fourth convolutional unit is larger than the kernel size of the fifth convolutional unit.

[0022] Optionally, the cross-stage local unit includes: a first branch and a second branch;

[0023] The first branch includes a seventh convolutional unit and at least one third residual unit; the seventh convolutional unit is used to reduce the number of output channels for image features based on convolution processing; the third main branch of the third residual unit includes an eighth convolutional unit and a ninth convolutional unit, the eighth convolutional unit and the ninth convolutional unit being used to perform convolution processing without changing the number of output channels.

[0024] Optionally, the inverted residual structure includes: a third main branch, a third shortcut branch, and a third fusion unit;

[0025] The third main branch includes: a tenth convolutional unit and a depthwise separable convolutional unit; the tenth convolutional unit is used to increase the number of output channels for image features based on convolution processing;

[0026] The third fusion unit is used to fuse the main branch output of the third main branch and the shortcut branch output of the third shortcut branch without changing the number of output channels.

[0027] Optionally, the number of output channels for the first image feature is a first number, the number of output channels for the second image feature is a second number, the number of output channels for the third image feature is a third number, and the number of output channels for the fourth image feature is a fourth number;

[0028] The second quantity is greater than the first quantity, the third quantity is greater than the second quantity, and the fourth quantity is greater than the third quantity.

[0029] Optionally, the device further includes:

[0030] The recognition module is used to determine the recognition result corresponding to the image to be processed based on the feature extraction result.

[0031] This application also discloses an electronic device, including: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform the method described in this application.

[0032] This application also discloses a machine-readable medium storing executable code thereon, which, when executed, causes a processor to perform the method described in this application.

[0033] The embodiments of this application have the following advantages:

[0034] In the technical solution of this application embodiment, the first convolutional module, the second convolutional module, the third convolutional module and the fourth convolutional module are designed in the convolutional neural network by taking advantage of the characteristic that the number of parameters of the convolutional neural network gradually increases.

[0035] First, the structure of the first convolutional module enhances the expressive power and richness of image features without significantly increasing the number of parameters. Specifically, the first convolutional unit increases the number of output channels for the input image features through convolution processing, which helps to initially enrich the representation of image features. At least two second convolutional units in the first main branch of the first residual unit use n×n convolutional kernels for convolution processing without changing the number of output channels. On the one hand, not changing the number of output channels means that the number of parameters will not increase significantly due to the increase in the number of channels. On the other hand, performing convolution processing on image features with a fixed number of channels allows for in-depth mining of local features of the image to be processed without introducing too many parameters, further enriching the expressive power of image features.

[0036] Secondly, the second convolutional module enhances feature extraction capabilities by expanding the network width rather than increasing its depth, thereby improving the inference speed of the convolutional neural network on the Neural Processing Unit (NPU). Specifically, the third convolutional unit increases the number of output channels for the corresponding input image features through convolution, thus expanding the width of the convolutional neural network. The fourth convolutional unit in the second residual unit further increases the number of output channels to expand the network width even more significantly. The fifth convolutional unit, however, reduces the number of output channels for image features through convolution. This approach of first increasing and then decreasing allows for the selection and integration of image features, strengthening useful features and suppressing irrelevant ones, thereby improving the expressive power of image features.

[0037] Furthermore, the third convolutional module introduces a cross-stage local unit to reduce the number of parameters. This cross-stage local unit divides the input image features into first and second features according to channels, performs convolution processing on each feature separately, and then concatenates the convolution results by the number of channels. This processing can not only extract different features precisely within a local range and avoid parameter redundancy, but also enhance feature representation capabilities through feature fusion. In the entire convolutional neural network, the third convolutional module is in the middle stage. By effectively reducing the number of parameters, refining the processing, and fusing features, it improves the network's operating efficiency and accuracy, providing strong support for achieving more accurate image recognition tasks.

[0038] Furthermore, to significantly reduce the number of parameters, the fourth convolutional module employs an inverse residual structure. Given that the first to third convolutional modules have successfully captured extremely rich image features, and that the largest proportion of network parameters is concentrated in the higher layers, the fourth convolutional module utilizes an inverse residual structure in the higher layers of the convolutional neural network, which can significantly reduce the number of parameters. This approach keeps the overall number of parameters in the convolutional neural network at a low level. In this way, on the one hand, it improves the accuracy of the convolutional neural network when processing complex image recognition tasks, and on the other hand, it avoids, to some extent, problems such as excessive consumption of computational resources and overfitting caused by an excessive number of parameters.

[0039] Furthermore, the grouping parameter of the convolutional layer controls the grouping method of the input channels. When the grouping parameter value is 1, it indicates that all input channels are treated as a whole and will not be divided into multiple independent groups for convolution calculation. In the first, second, and third convolutional modules described in this embodiment, the grouping parameter value of the convolutional units is all 1. This means that during the convolution operation, the various channels of the input image features are interconnected and participate in the calculation together. In this way, this embodiment can fully utilize the correlation and complementarity between different channels in the input image features, thereby extracting feature information more comprehensively. This, in turn, helps to improve the feature extraction and expressive capabilities of the convolutional neural network.

[0040] In summary, the four convolutional modules in this embodiment work closely together. While strictly controlling the number of parameters, the first convolutional module initially enriches the image feature representation; the second convolutional module expands the network width to enhance feature extraction; the third convolutional module introduces cross-stage local unit fusion features and reduces the number of parameters; and the fourth convolutional module uses an inverse residual structure to further reduce the number of parameters at higher levels. Therefore, this embodiment achieves the goal of low-parameter, high-precision feature extraction, thereby significantly improving the performance and efficiency of image recognition. On the one hand, the low parameters significantly reduce the consumption of computing resources, allowing the image recognition system to run on a wider range of devices, especially performing well in resource-constrained environments. On the other hand, high-precision feature extraction ensures the accuracy of image recognition, enabling more accurate identification of various objects in the image. Attached Figure Description

[0041] Figure 1 This is a schematic flowchart of the steps of an image feature extraction method according to an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the structure of the first convolutional module according to an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the structure of the first residual unit according to an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the structure of the second convolution module according to an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the structure of the second residual unit according to an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the structure of the third convolution module according to an embodiment of this application;

[0047] Figure 7 This is a schematic diagram of the structure of a cross-stage local unit according to an embodiment of this application;

[0048] Figure 8 This is a schematic diagram of the inverted residual structure according to an embodiment of this application;

[0049] Figure 9 This is a schematic diagram of the structure of an image feature extraction device according to an embodiment of this application;

[0050] Figure 10 This is a schematic diagram of the structure of an apparatus provided in one embodiment of this application. Detailed Implementation

[0051] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The embodiments of this application can be applied to the field of image processing technology to improve the accuracy of image recognition.

[0053] This application does not limit the specific scenarios for image recognition. For example, image recognition scenarios may include face recognition and object recognition. Face recognition can be applied to security checks, identity verification, and mobile payments. Object recognition may include product recognition, which can play a role in the circulation of goods, especially in unmanned retail fields such as unmanned shelves and smart retail cabinets.

[0054] The image recognition process can specifically include the following steps: image acquisition, image preprocessing, feature extraction, and classification.

[0055] The image acquisition stage is used to obtain image data to be identified. Image data can come from various sources, such as cameras, scanners, and satellite remote sensing devices.

[0056] Image data can be affected by factors such as noise, uneven lighting, and inconsistent sizes, leading to quality issues. Image preprocessing addresses these quality problems to improve image quality and prepare it for subsequent feature extraction. For example, for noisy images, preprocessing can employ filtering techniques such as Gaussian filtering and median filtering to remove noise.

[0057] The feature extraction stage is used to extract image features that can represent objects in the preprocessed image. These image features should be representative, discriminative, and stable to facilitate subsequent classification and recognition.

[0058] The classification stage uses classification algorithms to categorize image features and determine which category an object in the image belongs to. For example, in a face recognition scenario, the extracted facial features are compared with a pre-stored face feature database, or the extracted facial features can be input into a pre-trained classification model, which will output a classification result, determining whether the face belongs to a known or unknown identity.

[0059] Currently, lightweight convolutional neural networks (hereinafter referred to as lightweight networks) are widely used for image feature extraction. These lightweight networks employ techniques such as depthwise separable convolution and grouped convolution, effectively reducing the number of parameters and computational load in the image feature extraction process, thus enabling their application on resource-constrained embedded and mobile devices.

[0060] However, depthwise separable convolution performs independent convolution operations on each input channel, while grouped convolution divides the input channels into multiple groups, with each group performing convolution operations independently. This limits information exchange between groups during the convolution process. Consequently, lightweight networks suffer from insufficient cross-channel information fusion during feature extraction, resulting in less rich and nuanced captured features, thus reducing the richness and expressive power of image features.

[0061] To address the technical problem of low richness and expressive power of image features in related technologies, embodiments of this application provide an image feature extraction method, which specifically includes the following steps:

[0062] Receive the image to be processed;

[0063] The first convolutional module of the convolutional neural network is used to extract features from the image to be processed to obtain first image features. The first convolutional module includes at least one first convolutional unit and at least one first residual unit connected in sequence. The at least one first convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The first main branch of the first residual unit includes at least two second convolutional units. The at least two second convolutional units are used to perform convolution processing with an n×n convolutional kernel without changing the number of output channels. n is a positive integer greater than 1. For example, to keep the number of output channels unchanged, the input and output channels can be set to be equal.

[0064] The second convolutional module of the convolutional neural network is used to extract features from the first image features to obtain second image features. The second convolutional module includes at least one third convolutional unit and at least one second residual unit connected in sequence. The at least one third convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The second main branch of the second residual unit includes a fourth convolutional unit and a fifth convolutional unit. The fourth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing, and the fifth convolutional unit is used to decrease the number of output channels of the image features based on convolution processing.

[0065] The third convolutional module of the convolutional neural network is used to extract features from the second image features to obtain the third image features. The third convolutional module includes at least one sixth convolutional unit and at least one cross-stage local unit connected in sequence. The sixth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The corresponding input image features are divided into first features and second features according to channels. For example, the first features and second features can be evenly divided according to channels. The cross-stage local unit is used to perform convolution processing on the first features and second features respectively, and to concatenate the channel numbers of the convolution processing results corresponding to the first features and second features respectively. The grouping parameter values ​​of the convolutional units in the first convolutional module, the second convolutional module and the third convolutional module are all 1, or consistent with the number of input channels.

[0066] The fourth convolutional module of the convolutional neural network is used to extract features from the third image features to obtain the fourth image features; the fourth convolutional module adopts an inverted residual structure; the inverted residual structure contains convolutional units with grouping parameter values ​​greater than 1.

[0067] The fourth image feature is output as the feature extraction result.

[0068] This application embodiment utilizes a convolutional neural network (CNN) to extract features from the image to be processed. In a CNN, as the network depth increases, the number of output channels is typically increased to extract more complex features. However, increasing the number of output channels leads to an increase in the number of parameters, as more parameters are needed to process and learn the various image features represented by different channels. Therefore, CNNs have the characteristic of a gradually increasing number of parameters.

[0069] This application embodiment utilizes the characteristic of convolutional neural networks (CNNs) where the number of parameters gradually increases, and designs a first convolutional module, a second convolutional module, a third convolutional module, and a fourth convolutional module within the CNN. These four convolutional modules can be components of the CNN.

[0070] First convolutional module

[0071] The first convolutional module includes at least one first convolutional unit and at least one first residual unit connected in sequence. This structure of the first convolutional module enhances the expressive power and richness of image features without significantly increasing the number of parameters.

[0072] The first convolutional unit increases the number of output channels for the input image features through convolution processing, which helps to initially enrich the representation of image features.

[0073] At least two second convolutional units in the first main branch of the first residual unit are convolved using n×n convolutional kernels without changing the number of output channels. On one hand, keeping the number of output channels constant means that the number of parameters will not increase significantly with the increase in the number of channels. On the other hand, convolving image features with a fixed number of channels allows for deeper mining of local features of the image without introducing too many parameters, further enriching the expressive power of image features. For example, it can better extract local feature information such as edge details and textures in the image, thereby enhancing the richness of image features.

[0074] Since the number of output channels in the first convolutional module is usually relatively small, according to the formula for calculating the number of parameters in a convolutional layer (number of input channels × number of output channels × kernel side length × kernel side length), the number of parameters will not be large. Therefore, the first convolutional module can effectively improve the expressive power and richness of image features while controlling the overall increase in the number of parameters.

[0075] Second convolution module

[0076] The second convolutional module includes at least one third convolutional unit and at least one second residual unit connected in sequence. This structure enhances feature extraction capabilities by expanding the network width rather than increasing the network depth, thereby improving the inference speed of convolutional neural networks on the Neural Processing Unit (NPU). The NPU, as a hardware accelerator specifically designed for artificial intelligence computing tasks, plays a crucial role in improving computational efficiency.

[0077] The third convolutional unit uses convolution to increase the number of output channels for the corresponding input image features, thereby expanding the width of the convolutional neural network. The fourth convolutional unit in the second residual unit further increases the number of output channels to expand the network width to a greater extent. The fifth convolutional unit, on the other hand, reduces the number of output channels for image features through convolution. This approach of first increasing and then decreasing the number of output channels enables the filtering and integration of image features, strengthening useful image features and suppressing irrelevant ones, thus improving the expressive power of image features.

[0078] When performing inference on an NPU, increasing network depth often leads to a significant increase in computational cost and memory usage, thereby reducing inference speed. In contrast, within a certain range, increasing network width has a smaller impact on inference speed. This is because increasing network width allows for better utilization of parallel computing resources, enabling the NPU to process feature calculations across multiple channels more efficiently. Simultaneously, shallower network structures are easier to optimize and deploy, thus reducing latency during inference.

[0079] Third convolution module

[0080] The third convolutional module introduces a cross-stage local unit to reduce the number of parameters. This cross-stage local unit divides the input image features into first and second features according to channels, performs convolution processing on each feature separately, and then concatenates the convolution results by the number of channels. This processing can not only extract different features finely within a local range and avoid parameter redundancy, but also enhance feature representation capabilities through feature fusion. In the entire convolutional neural network, the third convolutional module is in the middle stage. By effectively reducing the number of parameters, finely processing and fusing features, it improves the network's operating efficiency and accuracy, providing strong support for achieving more accurate image recognition tasks.

[0081] Fourth convolution module

[0082] To significantly reduce the number of parameters, the fourth convolutional module employs an inverted residual structure. Given that the first three convolutional modules have successfully captured extremely rich image features, and that the largest proportion of network parameters is concentrated in the higher layers, the fourth convolutional module uses an inverted residual structure in the higher layers of the convolutional neural network, thereby significantly reducing the number of parameters. This approach keeps the overall number of parameters in the convolutional neural network at a low level. In this way, on the one hand, it improves the accuracy of the convolutional neural network when processing complex image recognition tasks, and on the other hand, it avoids, to some extent, the problems of excessive computational resource consumption and overfitting caused by an excessive number of parameters.

[0083] Method Example 1

[0084] refer to Figure 1 The diagram illustrates a step-by-step flowchart of an image feature extraction method according to an embodiment of this application. The method specifically includes the following steps:

[0085] Step 101: Receive the image to be processed;

[0086] Step 102: Using the first convolutional module of the convolutional neural network, feature extraction is performed on the above-mentioned image to be processed to obtain the first image features; the first convolutional module specifically includes: at least one first convolutional unit and at least one first residual unit connected in sequence; the at least one first convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing; the first main branch of the first residual unit specifically includes: at least two second convolutional units, the at least two second convolutional units are used to perform convolution processing with an n×n convolutional kernel without changing the number of output channels; n is a positive integer greater than 1;

[0087] Step 103: Using the second convolutional module of the convolutional neural network, feature extraction is performed on the first image features to obtain second image features. The second convolutional module specifically includes: at least one third convolutional unit and at least one second residual unit connected in sequence. The at least one third convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The second main branch of the second residual unit includes: a fourth convolutional unit and a fifth convolutional unit. The fourth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing, and the fifth convolutional unit is used to decrease the number of output channels of the image features based on convolution processing. In the embodiment, the number of output channels after the increase and decrease can be kept equal to the number of output channels before the increase.

[0088] Step 104: Using the third convolutional module of the convolutional neural network, feature extraction is performed on the second image features to obtain the third image features. The third convolutional module specifically includes: at least one sixth convolutional unit and at least one cross-stage local unit connected in sequence. The sixth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The corresponding input image features are divided into first features and second features according to the channels. The cross-stage local unit is used to perform convolution processing on the first features and the second features respectively, and to concatenate the channel numbers of the convolution processing results corresponding to the first features and the second features respectively. The grouping parameter values ​​of the convolutional units in the first convolutional module, the second convolutional module and the third convolutional module are all 1, or consistent with the number of input channels.

[0089] Step 105: Using the fourth convolutional module of the convolutional neural network, feature extraction is performed on the third image features to obtain the fourth image features; the fourth convolutional module adopts an inverted residual structure; the inverted residual structure contains convolutional units with grouping parameter values ​​greater than 1.

[0090] Step 106: Output the fourth image features as the feature extraction result.

[0091] Figure 1The method embodiment shown can extract features from the image to be processed, and the obtained feature extraction results can be used for image processing tasks such as image recognition.

[0092] Figure 1 The steps included in the illustrated method embodiments can be executed by either the client or the server. It is understood that the embodiments of this application are for... Figure 2 The specific entity that performs the method embodiment shown is not limited.

[0093] In practical applications, the client can run on the primary operating system, and compatibility layer software can also run on the primary operating system. The compatibility layer software can be used to provide a compatibility layer environment.

[0094] In step 101, the image to be processed can be a source image directly acquired by the image source device. Alternatively, the image to be processed can be a preprocessed version of the source image. It is understood that the embodiments of this application do not limit the specific image to be processed.

[0095] This application embodiment can utilize a convolutional neural network to extract features from the image to be processed. Therefore, this application embodiment can utilize a convolutional neural network to receive the image to be processed.

[0096] A convolutional neural network (CNN) is a deep learning model specifically designed for processing data with a grid structure, such as images and audio. It automatically extracts features from the data through components such as convolutional layers, pooling layers, fully-connected layers, and activation functions.

[0097] The convolutional layer is the core component of a convolutional neural network. It contains at least one convolutional kernel (also called a filter). A convolutional kernel is a small matrix; for example, in image processing, its size might be 1×1, 3×3, 5×5, etc.

[0098] As the convolution kernel slides across the input image (image features), it performs a weighted summation operation on the local region it covers. For example, given a 5×5 input image and a 3×3 convolution kernel, the kernel starts from the top left corner of the image, moves one pixel at a time (with a stride of 1), calculates the sum of its product with the pixel values ​​of the covered 3×3 region, and obtains a new value. This value constitutes an element of the output image features.

[0099] Taking image recognition as an example, if the input is an image of handwritten digits, different convolutional kernels can extract different features, such as edges and lines. One convolutional kernel may be sensitive to vertical edges, while another may be sensitive to horizontal edges.

[0100] Convolution operations feature weight sharing, meaning that the weights of the same convolution kernel remain unchanged as it slides across the entire input data. This significantly reduces the number of parameters in the model, lowers computational costs, and also allows the model to better capture translation-invariant features in the data. For example, the handwritten digit "7" can have similar edge features extracted through a convolution kernel regardless of its location in the image.

[0101] Pooling layers are used to reduce the dimensionality of data while retaining important feature information. Common pooling operations include max pooling and average pooling.

[0102] In max pooling, the pooling window (e.g., 2×2) slides across the image features, selecting the maximum value within the window as the output each time. For example, for a 2×2 pooling window, if the local region of the image features is [1,2,3,4] (where the semicolon represents a line break), then the output after max pooling is 4.

[0103] Average pooling calculates the average pixel values ​​within the pooling window as the output. Pooling operations can reduce the dimensionality of data, decrease subsequent computation, and are robust to a certain degree of translation and deformation. For example, in image recognition, even if the position of an object in an image changes slightly, the features after pooling can still remain relatively stable.

[0104] Fully connected layers are typically located at the end of a convolutional neural network. After the preceding convolution and pooling operations, the extracted image features are flattened into a one-dimensional vector. This one-dimensional vector is then fed into the fully connected layer, where each neuron is connected to all neurons in the previous layer. The fully connected layer can synthesize and classify the extracted features. For example, in a 10-class image recognition task, the output of the fully connected layer might be a 10-dimensional vector, where each dimension represents the probability that the input image belongs to a certain class (such as the digits 0-9).

[0105] In convolutional neural networks, activation functions are used to introduce non-linearity. Without activation functions, a combination of multiple neural networks remains a linear model and cannot fit complex data well. A common activation function is ReLU (Rectified Linear Unit), which is expressed as f(x) = max(0,x). When the input x is greater than 0, the output is equal to x; when the input x is less than or equal to 0, the output is 0.

[0106] This application primarily focuses on the design of convolutional layers in convolutional neural networks (CNNs), aiming to achieve the goal of low-parameter, high-precision feature extraction. Based on the characteristic of CNNs where the number of parameters gradually increases, this application meticulously designs a first, second, third, and fourth convolutional module within the CNN. The convolutional layer can be one of the components of the first to fourth convolutional modules.

[0107] In step 202, the first convolution module of the convolutional neural network is used to extract features from the image to be processed to obtain the first image features.

[0108] Reference Figure 2 The diagram illustrates the structure of a first convolution module according to an embodiment of this application, which receives an image to be processed and outputs first image features. The first convolution module specifically includes: a first convolution unit A201, a first convolution unit B202, and a first residual unit 203 connected in sequence.

[0109] The first convolutional unit A201 and the first convolutional unit B202 are used to increase the number of output channels for the corresponding input image features based on convolution processing.

[0110] In convolutional neural networks, for a color image to be processed, there are typically three input channels (RGB, red, green, and blue), while the number of output channels determines the number of feature maps after the convolution operation. Each output channel can be viewed as extracting a different feature. The number of output channels can be equal to the number of convolutional kernels. In a convolutional layer, each convolutional kernel slides across the input data to perform a convolution operation, generating one output channel (feature map).

[0111] Therefore, the number of convolution kernels can be set in this application to control the number of output channels, such as increasing, decreasing, or maintaining (not changing) the number of output channels.

[0112] Specifically, increasing the number of output channels means that the number of output channels is greater than the number of input channels. By setting the number of convolutional kernels to be greater than the number of input channels, the number of output channels can be increased, enabling the extraction of features from more dimensions. This is helpful for complex tasks, but computational resources and data volume must be considered to avoid overfitting.

[0113] Reducing the number of output channels means that the number of output channels is less than the number of input channels. This can be achieved by setting the number of convolutional kernels to be less than the number of input channels, which simplifies the data and reduces computation, but it's crucial to ensure that key features are preserved for subsequent tasks.

[0114] Maintaining the number of output channels means that the number of output channels equals the number of input channels. This can be achieved by setting the number of convolutional kernels to be equal to the number of input channels.

[0115] In a practical implementation, assuming the input channel count of the image to be processed is 3, representing the red, green, and blue channels, the first convolutional unit A201 can initially increase the number of output channels from 3 to A, and the first convolutional unit B202 can decrease the number of output channels from A to B, where A is greater than 3 and B is less than A. Alternatively, the first convolutional units A201 and B202 can gradually increase the number of output channels to A.

[0116] Reference Figure 3 The diagram shows a structural schematic of a first residual unit according to an embodiment of this application, which specifically includes: a first main branch, a first shortcut branch, and a first fusion unit;

[0117] The first main branch specifically includes at least two second convolution units, which are used to perform convolution processing with an n×n convolution kernel without changing the number of output channels; n is a positive integer greater than 1, and n can be a value such as 3, 5 or 7. Figure 3 In this context, at least two second convolutional units specifically include: second convolutional unit A and second convolutional unit B. On one hand, not changing the number of output channels means that the number of parameters will not increase significantly due to the increase in the number of channels. On the other hand, performing convolution processing on image features with a fixed number of channels allows for in-depth mining of the local features of the image to be processed without introducing too many parameters, further enriching the expressive power of image features.

[0118] The first fusion unit is used to fuse the main branch output of the first main branch and the shortcut branch output of the first shortcut branch without changing the number of output channels.

[0119] It should be noted that the first convolutional module may contain one or more first residual units, and the output image features of the last first residual unit can be used as the first image features.

[0120] In specific implementations, the first or second convolutional unit in the embodiments of this application can fall under the category of convolutional units. These convolutional units can perform convolution operations using convolutional layers. The parameters of these convolutional units specifically include: weight parameters, bias parameters, dilation parameters, padding parameters, grouping parameters, kernel size, and stride parameters, etc.

[0121] The weight parameters specifically include: the number of output channels, the number of input channels, the width of the convolution kernel, and the height of the convolution kernel.

[0122] Bias parameters typically correspond to the number of output channels. Each output channel has an independent bias term used to adjust the result after convolution computation. The purpose of the bias term is to allow the convolutional unit to perform a certain translation adjustment on the output during feature extraction, enabling the model to better fit the data. For example, after the convolution result is calculated, the corresponding bias term is added to each element of each output channel.

[0123] Dilated convolution is a special type of convolution that expands the receptive field of a convolution without increasing the kernel size by setting a dilation rate. When the dilation rate is 1, it's a normal convolution operation. For example, the receptive field is the area covered by the kernel size. If the dilation rate is greater than 1, such as 2, the kernel skips some intermediate pixels during computation, thus expanding the receptive field. This operation is useful in tasks that require acquiring broader contextual information.

[0124] The grouping parameter indicates the number of groups in the group convolution. When the grouping parameter is 1, the convolution is a regular convolution operation. When the grouping parameter is greater than 1, the convolution is a grouped convolution. Grouped convolution is a method that divides the input and output channels into several groups and then performs convolution within each group.

[0125] For example, if the number of input channels is 6, the number of output channels is 6, and the grouping parameter is set to 2, then the input channels will be divided into two groups of 3 channels each, and the output channels will also be divided into two groups of 3 channels each. Then, convolution operations will be performed within each group. Grouped convolution can reduce computation and improve the efficiency and performance of the model to some extent.

[0126] Padding adds extra pixels (usually 0) to the edges of the input data. The purpose of the padding parameter is to control the size of the output feature map, while also allowing the convolutional kernel to completely cover the edge portions of the input data.

[0127] The stride parameter determines the distance the convolution kernel moves across the input data each time.

[0128] Referring to Table 1, examples of parameters for the convolutional units in the first convolutional module are shown. The parameters in Table 1 specifically include: the number of output channels, the number of input channels, the width of the convolutional kernel, the height of the convolutional kernel, the grouping parameter, and the stride parameter. It is understood that those skilled in the art can determine the parameter values ​​in Table 1 according to actual application requirements.

[0129] Table 1

[0130]

[0131] It should be noted that, in the embodiments of this application, an activation unit can be set after the convolutional unit. This activation unit is used to introduce nonlinear factors into the convolutional neural network. After processing by the convolutional unit, the output of the convolutional unit may still be in a linear space, making it difficult to fit complex data distributions and feature relationships. The activation unit performs a nonlinear transformation on the output of the convolutional unit through an activation function, enabling the convolutional neural network to learn richer and more complex feature representations. For example, a first activation unit can be set after the first convolutional unit A. Another example is that a second activation unit can be set after the first convolutional unit B. Yet another example is that a third activation unit can be set after the second convolutional unit A.

[0132] In step 103, the second convolution module of the convolutional neural network is used to extract features from the first image features to obtain the second image features.

[0133] Reference Figure 4 This diagram illustrates the structure of a second convolutional module according to an embodiment of this application. The module receives first image features and outputs second image features. Specifically, the second convolutional module includes a third convolutional unit 401 and at least one second residual unit 402 connected in sequence. Figure 4 The diagram shows a second residual unit; it can be understood that there can be multiple second residual units.

[0134] The third convolution unit 401 is used to increase the number of output channels for the corresponding input image features based on convolution processing.

[0135] Reference Figure 5 The diagram shows a structural schematic of the second residual unit according to an embodiment of this application, which specifically includes: a second main branch, a second shortcut branch, and a second fusion unit.

[0136] The second main branch specifically includes a fourth convolutional unit and a fifth convolutional unit. The fourth convolutional unit is used to increase the number of output channels for the corresponding input image features based on convolution processing, while the fifth convolutional unit is used to decrease the number of output channels for image features based on convolution processing. This method of first increasing and then decreasing the number of output channels enables the filtering and integration of image features, strengthening useful image features and suppressing irrelevant image features, thereby improving the expressive power of image features. It should be noted that the kernel sizes used in the fourth and fifth convolutional units can be the same or different.

[0137] The second fusion unit is used to fuse the main branch output of the second main branch and the shortcut branch output of the second shortcut branch without changing the number of output channels.

[0138] Referring to Table 2, examples of parameters for the convolutional units in the second convolutional module are shown. The parameters in Table 2 specifically include: the number of output channels, the number of input channels, the width of the convolutional kernel, the height of the convolutional kernel, the grouping parameter, and the stride parameter. It is understood that those skilled in the art can determine the parameter values ​​in Table 2 according to actual application requirements.

[0139] Table 2

[0140]

[0141] It should be noted that, in the embodiments of this application, a fourth activation unit may be set after the third convolutional unit. Alternatively, a fifth activation unit may be set after the fourth convolutional unit.

[0142] In step 104, the third convolution module of the convolutional neural network is used to extract features from the second image features to obtain the third image features.

[0143] Reference Figure 6 The diagram illustrates the structure of a third convolutional module according to an embodiment of this application, which receives second image features and outputs third image features. Specifically, the third convolutional module includes a sixth convolutional unit 601 and a cross-stage local unit 602 connected in sequence.

[0144] The sixth convolutional unit 601 is used to increase the number of output channels for the corresponding input image features based on convolution processing.

[0145] The corresponding input image features are divided into first features and second features according to channels; the aforementioned cross-stage local unit 602 is used to perform convolution processing on the first features and second features respectively, and to concatenate the convolution processing results corresponding to the first features and second features according to the number of channels. The first feature and the second feature have the same number of input channels.

[0146] In one example, assuming the input image features have 2y input channels, they can be divided into a first feature and a second feature, such that the first and second features each have y input channels. After each feature undergoes its own convolution process, the output channels of the convolution results for the first and second features are both y. Then, concatenating these two convolution results yields a concatenated output with 2y output channels.

[0147] The above processing can not only extract different features in a precise local range and avoid parameter redundancy, but also enhance the feature expression capability through feature fusion.

[0148] Reference Figure 7The diagram shows a structural schematic of a cross-stage local unit according to an embodiment of this application. The cross-stage local unit specifically includes: a first branch, a second branch, and a splicing unit.

[0149] The first branch includes a seventh convolutional unit and at least one third residual unit; the seventh convolutional unit is used to reduce the number of output channels for image features based on convolution processing; the third main branch of the third residual unit includes an eighth convolutional unit and a ninth convolutional unit, which are used to perform convolution processing without changing the number of output channels; the stitching unit is used to stitch the output of the first branch and the output of the second branch together.

[0150] It is understood that the third residual unit may also include: a third shortcut branch and a third fusion unit. The third fusion unit is used to fuse the main branch output of the third main branch and the shortcut branch output of the third shortcut branch without changing the number of output channels.

[0151] The second branch may include convolutional units and activation units. It is understood that the embodiments of this application do not limit the specific second branch.

[0152] Referring to Table 3, examples of parameters for the convolutional units in the third convolutional module are shown. The parameters in Table 3 specifically include: the number of output channels, the number of input channels, the width of the convolutional kernel, the height of the convolutional kernel, the grouping parameter, and the stride parameter. It is understood that those skilled in the art can determine the parameter values ​​in Table 3 according to actual application requirements.

[0153] Table 3

[0154]

[0155] It should be noted that, in the embodiments of this application, a sixth activation unit may be set after the sixth convolutional unit. Alternatively, a seventh activation unit may be set after the seventh convolutional unit. Yet another example is that an eighth activation unit may be set after the eighth convolutional unit.

[0156] In one example, the input image features of the cross-stage local unit can originate from the sixth activation unit, assuming its output channel count is 144. In this embodiment, the corresponding input image features can be divided into a first feature and a second feature according to channels, with the first and second features having an output channel count of 72. The aforementioned cross-stage local unit 602 is used to perform convolution processing on the first and second features respectively, and to concatenate the convolution processing results corresponding to the first and second features according to the number of channels, resulting in an output channel count of 144 for the concatenated result.

[0157] It should be noted that there can be multiple third convolutional modules in the embodiments of this application. This means that in the architecture of a convolutional neural network, the same type of third convolutional module can be reused according to actual needs to enhance the network's ability to extract and express specific features.

[0158] Different third convolutional modules can contain the same convolutional structure, indicating that they are similar in their overall design and composition. For example, they may all contain the same number of convolutional layers, the same type of activation function, etc. This design allows the network to process data in a similar way at different locations and stages, thus realizing a modular design philosophy and improving the network's scalability and maintainability.

[0159] However, despite having the same convolutional structure, the parameters of the convolutional units can differ. This is crucial because different parameters mean that each third convolutional module will perform differently when processing specific data. Differences in parameters allow each third convolutional module to adapt to different data features and task requirements. For example, different convolutional units may have different weight values ​​and bias terms, which will cause them to extract different features when performing convolution operations on the input data.

[0160] In step 105, the fourth convolutional module of the convolutional neural network is used to extract features from the third image features to obtain the fourth image features. The fourth convolutional module uses an inverted residual structure in the high-level layers of the convolutional neural network, which can significantly reduce the number of parameters.

[0161] Reference Figure 8 The diagram shows a schematic of the inverted residual structure according to an embodiment of this application. The inverted residual structure specifically includes: a third main branch, a third shortcut branch, and a third fusion unit.

[0162] The third main branch specifically includes: a tenth convolutional unit and a depthwise separable convolutional unit; the tenth convolutional unit is used to increase the number of output channels for image features based on convolution processing;

[0163] The third fusion unit is used to fuse the main branch output of the third main branch and the shortcut branch output of the third shortcut branch without changing the number of output channels.

[0164] The tenth convolutional unit can be used to increase the number of output channels based on convolution. Depthwise separable convolutional units typically include depthwise convolutional units and pointwise convolutional units. Depthwise convolutional units maintain the same number of output channels, while pointwise convolutional units restore the number of output channels. In the process of the inverse residual structure moving from low-dimensional to high-dimensional and then back to low-dimensional, expanding the dimensions allows depthwise separable convolutions to better extract features, maintaining good feature representation capabilities while reducing computational cost, especially performing well in scenarios with high computational resource requirements.

[0165] It should be noted that the fourth convolutional module may also include several eleventh convolutional units connected before the inverse residual structure. These eleventh convolutional units are used to increase the number of output channels for image features based on convolution processing.

[0166] For example, several eleventh convolutional units specifically include: eleventh convolutional unit A, eleventh convolutional unit B, eleventh convolutional unit C, and eleventh convolutional unit D.

[0167] Referring to Table 4, examples of parameters for the convolutional units in the fourth convolutional module are shown. The parameters in Table 4 specifically include: the number of output channels, the number of input channels, the width of the convolutional kernel, the height of the convolutional kernel, the grouping parameter, and the stride parameter. It is understood that those skilled in the art can determine the parameter values ​​in Table 4 according to actual application requirements.

[0168] Table 4

[0169]

[0170]

[0171] It should be noted that, in the embodiments of this application, an activation unit may be set after the convolution unit of the fourth convolution module. It is understood that the embodiments of this application do not limit the specific activation unit.

[0172] In the specific implementation, the number of output channels for the first image feature is the first number, the number of output channels for the second image feature is the second number, the number of output channels for the third image feature is the third number, and the number of output channels for the fourth image feature is the fourth number; therefore, the second number is greater than the first number, the third number is greater than the second number, and the fourth number is greater than the third number. Starting from the image to be processed, after passing through four convolutional modules, the increase in the number of channels corresponds to the gradual extraction of features from low-level to high-level. Low-level features may include basic image edges, color, and other information, while as the number of channels increases, high-level features such as object shape and semantic information are gradually extracted. This enables the convolutional neural network to analyze images at different levels and better understand the content of the image.

[0173] In step 106, this embodiment of the application can support at least one output method for the feature extraction results. For example, the feature extraction results can be output to a subsequent classification model, or the feature extraction results can be output in the form of a file.

[0174] This application embodiment can also determine the recognition result corresponding to the image to be processed based on the feature extraction result. For example, in a face recognition scenario, the facial features as the feature extraction result are compared with a pre-stored face feature database, or the facial features as the feature extraction result can be input into a pre-trained classification model, and the classification model will output a classification result to determine whether the face belongs to a known identity or an unknown identity. It is understood that this application embodiment does not limit the specific classification model.

[0175] In summary, the image feature extraction method of this application embodiment utilizes the characteristic of the convolutional neural network that the number of parameters gradually increases, and designs a first convolutional module, a second convolutional module, a third convolutional module and a fourth convolutional module in the convolutional neural network.

[0176] First, the structure of the first convolutional module enhances the expressive power and richness of image features without significantly increasing the number of parameters. Specifically, the first convolutional unit increases the number of output channels for the input image features through convolution processing, which helps to initially enrich the representation of image features. At least two second convolutional units in the first main branch of the first residual unit use n×n convolutional kernels for convolution processing without changing the number of output channels. On the one hand, not changing the number of output channels means that the number of parameters will not increase significantly due to the increase in the number of channels. On the other hand, performing convolution processing on image features with a fixed number of channels allows for in-depth mining of local features of the image to be processed without introducing too many parameters, further enriching the expressive power of image features.

[0177] Secondly, the second convolutional module enhances feature extraction capabilities by expanding the network width rather than increasing its depth, thereby improving the inference speed of the convolutional neural network on the Neural Processing Unit (NPU). Specifically, the third convolutional unit increases the number of output channels for the corresponding input image features through convolution, thus expanding the width of the convolutional neural network. The fourth convolutional unit in the second residual unit further increases the number of output channels to expand the network width even more significantly. The fifth convolutional unit, however, reduces the number of output channels for image features through convolution. This approach of first increasing and then decreasing allows for the selection and integration of image features, strengthening useful features and suppressing irrelevant ones, thereby improving the expressive power of image features.

[0178] Furthermore, the third convolutional module introduces a cross-stage local unit to reduce the number of parameters. This cross-stage local unit divides the input image features into first and second features according to channels, performs convolution processing on each feature separately, and then concatenates the convolution results by the number of channels. This processing can not only extract different features precisely within a local range and avoid parameter redundancy, but also enhance feature representation capabilities through feature fusion. In the entire convolutional neural network, the third convolutional module is in the middle stage. By effectively reducing the number of parameters, refining the processing, and fusing features, it improves the network's operating efficiency and accuracy, providing strong support for achieving more accurate image recognition tasks.

[0179] Furthermore, to significantly reduce the number of parameters, the fourth convolutional module employs an inverse residual structure. Given that the first to third convolutional modules have successfully captured extremely rich image features, and that the largest proportion of network parameters is concentrated in the higher layers, the fourth convolutional module utilizes an inverse residual structure in the higher layers of the convolutional neural network, which can significantly reduce the number of parameters. This approach keeps the overall number of parameters in the convolutional neural network at a low level. In this way, on the one hand, it improves the accuracy of the convolutional neural network when processing complex image recognition tasks, and on the other hand, it avoids, to some extent, problems such as excessive consumption of computational resources and overfitting caused by an excessive number of parameters.

[0180] Furthermore, the grouping parameter of the convolutional layer controls the grouping method of the input channels. When the grouping parameter value is 1, it indicates that all input channels are treated as a whole and will not be divided into multiple independent groups for convolution calculation. In the first, second, and third convolutional modules described in this embodiment, the grouping parameter value of the convolutional units is all 1. This means that during the convolution operation, the various channels of the input image features are interconnected and participate in the calculation together. In this way, this embodiment can fully utilize the correlation and complementarity between different channels in the input image features, thereby extracting feature information more comprehensively. This, in turn, helps to improve the feature extraction and expressive capabilities of the convolutional neural network.

[0181] The grouping parameter values ​​of the convolutional units in the first, second, and third convolutional modules are consistent with the number of input channels, which can reduce the number of parameters and speed up inference.

[0182] In summary, the four convolutional modules in this embodiment work closely together. While strictly controlling the number of parameters, the first convolutional module initially enriches the image feature representation; the second convolutional module expands the network width to enhance feature extraction; the third convolutional module introduces cross-stage local unit fusion features and reduces the number of parameters; and the fourth convolutional module uses an inverse residual structure to further reduce the number of parameters at higher levels. Therefore, this embodiment achieves the goal of low-parameter, high-precision feature extraction, thereby significantly improving the performance and efficiency of image recognition. On the one hand, the low parameters significantly reduce the consumption of computing resources, allowing the image recognition system to run on a wider range of devices, especially performing well in resource-constrained environments. On the other hand, high-precision feature extraction ensures the accuracy of image recognition, enabling more accurate identification of various objects in the image.

[0183] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0184] Based on the above embodiments, this embodiment also provides an image feature extraction device, referring to... Figure 9 The schematic diagram shown indicates that the device specifically includes: a receiving module 901, a first feature extraction module 902, a second feature extraction module 903, a third feature extraction module 904, a fourth feature extraction module 905, and an output module 906.

[0185] The receiving module 901 is used to receive the image to be processed;

[0186] The first feature extraction module 902 is used to extract features from the image to be processed using the first convolution module of a convolutional neural network to obtain first image features. The first convolution module includes at least one first convolution unit and at least one first residual unit connected in sequence. The at least one first convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The first main branch of the first residual unit includes at least two second convolution units, which are used to perform convolution processing with an n×n convolution kernel without changing the number of output channels. n is a positive integer greater than 1.

[0187] The second feature extraction module 903 is used to extract features from the first image features using the second convolution module of the convolutional neural network to obtain second image features. The second convolution module includes at least one third convolution unit and at least one second residual unit connected in sequence. The at least one third convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The second main branch of the second residual unit includes a fourth convolution unit and a fifth convolution unit. The fourth convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing, and the fifth convolution unit is used to decrease the number of output channels of the image features based on convolution processing.

[0188] The third feature extraction module 904 is used to extract features from the second image features using the third convolution module of the convolutional neural network to obtain the third image features. The third convolution module includes at least one sixth convolution unit and at least one cross-stage local unit connected in sequence. The sixth convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The corresponding input image features are divided into first features and second features according to the channels. The cross-stage local unit is used to perform convolution processing on the first features and the second features respectively, and to concatenate the channel numbers of the convolution processing results corresponding to the first features and the second features respectively. The grouping parameter values ​​of the convolution units in the first convolution module, the second convolution module, and the third convolution module are all 1, or consistent with the number of input channels.

[0189] The fourth feature extraction module 905 is used to extract features from the third image features using the fourth convolution module of the convolutional neural network to obtain the fourth image features; the fourth convolution module adopts an inverted residual structure; the inverted residual structure contains convolutional units with grouping parameter values ​​greater than 1;

[0190] The output module 906 is used to output the fourth image features as the feature extraction result.

[0191] Optionally, the first residual unit further includes: a first shortcut branch and a first fusion unit;

[0192] The first fusion unit is used to fuse the main branch output of the first main branch and the shortcut branch output of the first shortcut branch without changing the number of output channels.

[0193] Optionally, the kernel size of the fourth convolutional unit is larger than the kernel size of the fifth convolutional unit.

[0194] Optionally, the cross-stage local unit includes: a first branch and a second branch;

[0195] The first branch includes a seventh convolutional unit and at least one third residual unit; the seventh convolutional unit is used to reduce the number of output channels for image features based on convolution processing; the third main branch of the third residual unit includes an eighth convolutional unit and a ninth convolutional unit, the eighth convolutional unit and the ninth convolutional unit being used to perform convolution processing without changing the number of output channels.

[0196] Optionally, the inverted residual structure includes: a third main branch, a third shortcut branch, and a third fusion unit;

[0197] The third main branch includes: a tenth convolutional unit and a depthwise separable convolutional unit; the tenth convolutional unit is used to increase the number of output channels for image features based on convolution processing;

[0198] The third fusion unit is used to fuse the main branch output of the third main branch and the shortcut branch output of the third shortcut branch without changing the number of output channels.

[0199] Optionally, the number of output channels for the first image feature is a first number, the number of output channels for the second image feature is a second number, the number of output channels for the third image feature is a third number, and the number of output channels for the fourth image feature is a fourth number;

[0200] The second quantity is greater than the first quantity, the third quantity is greater than the second quantity, and the fourth quantity is greater than the third quantity.

[0201] Optionally, the device further includes:

[0202] The recognition module is used to determine the recognition result corresponding to the image to be processed based on the feature extraction result.

[0203] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0204] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In this application, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0205] The embodiments of this disclosure can be implemented as an apparatus configured as desired using any suitable hardware, firmware, software, or any combination thereof, including electronic devices such as terminal devices and servers (clusters). Figure 10An exemplary apparatus 1100 is schematically shown that can be used to implement the various embodiments described in this application.

[0206] In one embodiment, Figure 10 An exemplary device 1100 is shown, which includes one or more processors 1102, a control module (chipset) 1104 coupled to at least one of the processors 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory / storage device 1108 coupled to the control module 1104, one or more input / output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.

[0207] Processor 1102 may include one or more single-core or multi-core processors, and processor 1102 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, device 1100 can serve as a terminal device, server (cluster), or other device as described in the embodiments of this application.

[0208] In some embodiments, apparatus 1100 may include one or more computer-readable media (e.g., memory 1106 or non-volatile memory / storage device 1108) having instructions 1114 and one or more processors 1102 that are combined with the one or more computer-readable media and configured to execute instructions 1114 to implement modules and thus perform the actions described in this disclosure.

[0209] In one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1102 and / or any suitable device or component communicating with the control module 1104.

[0210] The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0211] Memory 1106 may be used, for example, to load and store data and / or instructions 1114 for device 1100. In one embodiment, memory 1106 may include any suitable volatile memory, such as suitable DRAM (Dynamic Random Access Memory). In some embodiments, memory 1106 may include double data rate type quad synchronous dynamic random access memory.

[0212] In one embodiment, the control module 1104 may include one or more input / output controllers to provide an interface to the non-volatile memory / storage device 1108 and (one or more) input / output devices 1110.

[0213] For example, non-volatile memory / storage device 1108 may be used to store data and / or instructions 1114. Non-volatile memory / storage device 1108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives, one or more optical disk drives, and / or one or more digital universal optical disk drives).

[0214] The non-volatile memory / storage device 1108 may include storage resources that are physically part of a device on which the device 1100 is mounted, or that are accessible by the device but do not necessarily have to be part of the device. For example, the non-volatile memory / storage device 1108 may be accessed via a network via one or more input / output devices 1110.

[0215] One or more input / output devices 1110 may provide an interface for device 1100 to communicate with any other suitable device. Input / output devices 1110 may include communication components, audio components, sensor components, etc. Network interface 1112 may provide an interface for device 1100 to communicate via one or more networks. Device 1100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi (Wireless Fidelity), 2G (2-Generation wireless telephone technology), 3G (3-Generation wireless telephone technology), 4G (4-Generation wireless telephone technology), 5G (5-Generation wireless telephone technology), etc., or combinations thereof.

[0216] In one embodiment, at least one of the processors 1102 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 1104. In one embodiment, at least one of the processors 1102 may be logically packaged with one or more controllers of the control module 1104 to form a system-in-package. In one embodiment, at least one of the processors 1102 may be integrated with the logic of one or more controllers of the control module 1104 on the same die. In one embodiment, at least one of the processors 1102 may be integrated with the logic of one or more controllers of the control module 1104 on the same die to form a system-on-a-chip.

[0217] In various embodiments, device 1100 may be, but is not limited to, a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, touchscreen device, netbook, etc.). In various embodiments, device 1100 may have more or fewer components and / or different architectures. For example, in some embodiments, device 1100 includes one or more cameras, a keyboard, a liquid crystal display screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0218] The detection device may use a main control chip as a processor or control module, and sensor data, position information, etc. may be stored in a memory or non-volatile memory / storage device. The sensor group may be used as an input / output device, and the communication interface may include a network interface.

[0219] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0220] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0221] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0222] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0223] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0224] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0225] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0226] The foregoing has provided a detailed description of an image feature extraction method and apparatus, an electronic device, and a machine-readable medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application sets based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for extracting image features, characterized in that, The method includes: Receive the image to be processed; The first convolutional module of a convolutional neural network is used to extract features from the image to be processed, thereby obtaining first image features. The first convolutional module includes at least one first convolutional unit and at least one first residual unit connected in sequence. The at least one first convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The first main branch of the first residual unit includes at least two second convolutional units, which are used to perform convolution processing with an n×n convolutional kernel without changing the number of output channels. n is a positive integer greater than 1. The second convolutional module of a convolutional neural network is used to extract features from the first image features to obtain second image features. The second convolutional module includes at least one third convolutional unit and at least one second residual unit connected in sequence. The at least one third convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The second main branch of the second residual unit includes a fourth convolutional unit and a fifth convolutional unit. The fourth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing, and the fifth convolutional unit is used to decrease the number of output channels of the image features based on convolution processing. The third convolutional module of a convolutional neural network is used to extract features from the second image features to obtain the third image features. The third convolutional module includes at least one sixth convolutional unit and at least one cross-stage local unit connected in sequence. The sixth convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The corresponding input image features are divided into first features and second features according to the channels. The cross-stage local unit is used to perform convolution processing on the first features and the second features respectively, and to concatenate the channel numbers of the convolution processing results corresponding to the first features and the second features respectively. The grouping parameter values ​​of the convolutional units in the first convolutional module, the second convolutional module, and the third convolutional module are all 1, or consistent with the number of input channels. The fourth convolutional module of the convolutional neural network is used to extract features from the third image features to obtain the fourth image features; the fourth convolutional module adopts an inverted residual structure; the inverted residual structure contains convolutional units with grouping parameter values ​​greater than 1; The fourth image feature is output as the feature extraction result.

2. The method according to claim 1, characterized in that, The first residual unit further includes: a first shortcut branch and a first fusion unit; The first fusion unit is used to fuse the main branch output of the first main branch and the shortcut branch output of the first shortcut branch without changing the number of output channels.

3. The method according to claim 1, characterized in that, The kernel size of the fourth convolutional unit is larger than the kernel size of the fifth convolutional unit.

4. The method according to claim 1, characterized in that, The cross-stage local unit includes: a first branch and a second branch; The first branch includes a seventh convolutional unit and at least one third residual unit; the seventh convolutional unit is used to reduce the number of output channels for image features based on convolution processing; the third main branch of the third residual unit includes an eighth convolutional unit and a ninth convolutional unit, the eighth convolutional unit and the ninth convolutional unit being used to perform convolution processing without changing the number of output channels.

5. The method according to claim 1, characterized in that, The inverted residual structure includes: a third main branch, a third shortcut branch, and a third fusion unit; The third main branch includes: a tenth convolutional unit and a depthwise separable convolutional unit; the tenth convolutional unit is used to increase the number of output channels for image features based on convolution processing; The third fusion unit is used to fuse the main branch output of the third main branch and the shortcut branch output of the third shortcut branch without changing the number of output channels.

6. The method according to any one of claims 1 to 5, characterized in that, The number of output channels for the first image feature is a first number, the number of output channels for the second image feature is a second number, the number of output channels for the third image feature is a third number, and the number of output channels for the fourth image feature is a fourth number; The second quantity is greater than the first quantity, the third quantity is greater than the second quantity, and the fourth quantity is greater than the third quantity.

7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Based on the feature extraction results, the recognition result corresponding to the image to be processed is determined.

8. An image feature extraction device, characterized in that, The device includes: The receiving module is used to receive the image to be processed; A first feature extraction module is used to extract features from the image to be processed using a first convolutional module of a convolutional neural network to obtain first image features. The first convolutional module includes at least one first convolutional unit and at least one first residual unit connected in sequence. The at least one first convolutional unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The first main branch of the first residual unit includes at least two second convolutional units, which are used to perform convolution processing with an n×n convolutional kernel without changing the number of output channels; n is a positive integer greater than 1. The second feature extraction module is used to extract features from the first image features using the second convolution module of the convolutional neural network to obtain second image features. The second convolution module includes at least one third convolution unit and at least one second residual unit connected in sequence. The at least one third convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The second main branch of the second residual unit includes a fourth convolution unit and a fifth convolution unit. The fourth convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing, and the fifth convolution unit is used to decrease the number of output channels of the image features based on convolution processing. The third feature extraction module is used to extract features from the second image features using the third convolution module of the convolutional neural network to obtain the third image features. The third convolution module includes at least one sixth convolution unit and at least one cross-stage local unit connected in sequence. The sixth convolution unit is used to increase the number of output channels of the corresponding input image features based on convolution processing. The corresponding input image features are divided into first features and second features according to the channels. The cross-stage local unit is used to perform convolution processing on the first features and the second features respectively, and to concatenate the channel numbers of the convolution processing results corresponding to the first features and the second features respectively. The grouping parameter values ​​of the convolution units in the first convolution module, the second convolution module, and the third convolution module are all 1, or consistent with the number of input channels. The fourth feature extraction module is used to extract features from the third image features using the fourth convolution module of the convolutional neural network to obtain the fourth image features; the fourth convolution module adopts an inverted residual structure; the inverted residual structure contains convolutional units with grouping parameter values ​​greater than 1; The output module is used to output the fourth image features as the feature extraction result.

9. An electronic device, characterized in that, include: processor; and A memory having executable code stored thereon, which, when executed, causes the processor to perform the method as described in any one of claims 1-7.

10. A machine-readable medium having executable code stored thereon, which, when executed, causes a processor to perform the method as claimed in any one of claims 1-7.

Images