Feature extraction model training method, image processing method, device, and medium

Abstract

The application discloses a training method of a feature extraction model, an image processing method, equipment and a medium. The training method comprises the following steps: obtaining a sample image; performing feature extraction on the sample image by using a student feature extraction model and a teacher feature extraction model respectively, and obtaining multiple initial feature maps of different sizes, wherein the complexity of the student feature extraction model is less than that of the teacher feature extraction model; performing feature fusion on the multiple initial feature maps by using the student feature extraction model and the teacher feature extraction model respectively, and obtaining target feature maps corresponding to the initial feature maps, wherein the number of the target feature maps is the same as that of the initial feature maps; and adjusting parameters in the student feature extraction model according to the similarity loss between the multiple target feature maps obtained by using the student feature extraction model and the multiple target feature maps obtained by using the teacher feature extraction model. The above scheme can improve the accuracy of the student feature extraction model in the subsequent image processing process.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a training method for a feature extraction model and an image processing method, device and medium. Background Technology

[0002] Image processing modules (such as models for object detection, object recognition, and object classification) are finding increasingly widespread applications. Object detection or object recognition is a crucial task in computer vision, aiming to detect and locate specific objects in images or videos. However, object detection, object recognition, and object classification models are generally quite complex, making them unsuitable for deployment on resource-constrained or embedded devices. Conversely, models with lower complexity often suffer from accuracy issues, resulting in lower accuracy for object detection, object recognition, and object classification. Therefore, there is an urgent need for a model that not only has lower complexity and can be deployed on embedded devices but also boasts high detection performance and efficient inference speed. Summary of the Invention

[0003] This application provides at least one training method for a feature extraction model, as well as an image processing method, device, and medium.

[0004] This application provides a training method for a feature extraction model, comprising: acquiring sample images; extracting features from the sample images using a student feature extraction model and a teacher feature extraction model respectively, obtaining multiple initial feature maps of different sizes, wherein the complexity of the student feature extraction model is less than that of the teacher feature extraction model; fusing the multiple initial feature maps using the student feature extraction model and the teacher feature extraction model respectively, obtaining target feature maps corresponding to each initial feature map, wherein the number of target feature maps is the same as the number of initial feature maps; adjusting the parameters in the student feature extraction model using the similarity loss between the multiple target feature maps obtained by the student extraction model and the multiple target feature maps obtained by the teacher feature extraction model, thereby enabling the trained student feature extraction model to perform feature extraction and feature fusion on the image to be processed in subsequent image processing.

[0005] This application provides a training device for a feature extraction model, comprising: an acquisition module for acquiring sample images; an extraction module for extracting features from the sample images using a student feature extraction model and a teacher feature extraction model respectively, to obtain multiple initial feature maps of different sizes, wherein the complexity of the student feature extraction model is less than that of the teacher feature extraction model; a fusion module for fusing the multiple initial feature maps using the student feature extraction model and the teacher feature extraction model respectively, to obtain target feature maps corresponding to each initial feature map, wherein the number of target feature maps is the same as the number of initial feature maps; and an adjustment module for adjusting the parameters in the student feature extraction model using the similarity loss between the multiple target feature maps obtained by the student extraction model and the multiple target feature maps obtained by the teacher feature extraction model, wherein the trained student feature extraction model can perform feature extraction and feature fusion on the image to be processed in subsequent image processing.

[0006] This application provides an image processing method, including: acquiring an image to be processed; processing the image to be processed using a student feature extraction model in an image processing model to obtain multiple target feature maps corresponding to the image to be processed, wherein the student feature extraction model is trained based on the above training method; and using a detection head model in an image processing model to perform image processing based on the multiple target feature maps corresponding to the image to be processed to obtain the image processing result of the image to be processed.

[0007] This application provides an image processing apparatus, comprising: an image acquisition module for acquiring an image to be processed; a first image processing module for processing the image to be processed using a student feature extraction model in an image processing model to obtain multiple target feature maps corresponding to the image to be processed, wherein the student feature extraction model is trained based on the above-mentioned training method; and a second image processing module for performing image processing based on the multiple target feature maps corresponding to the image to be processed using a detection head model in an image processing model to obtain an image processing result of the image to be processed.

[0008] This application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-described feature extraction model training method and / or image processing method.

[0009] This application provides a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the training method and / or image processing method of the above-mentioned feature extraction model.

[0010] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0013] Figure 1 This is a flowchart illustrating an embodiment of the training method for the feature extraction model of this application;

[0014] Figure 2 This is a schematic diagram of the structure of the first channel transformation submodule of each channel transformation module in one embodiment of the training method of the feature extraction model of this application;

[0015] Figure 3 This is a schematic diagram of the structure of the second channel transformation submodule of each channel transformation module in one embodiment of the training method of the feature extraction model of this application;

[0016] Figure 4 This is a schematic diagram of the structure of each dimensionality reduction module in one embodiment of the training method of the feature extraction model of this application;

[0017] Figure 5 This is a schematic diagram of the relationship between target feature maps in one embodiment of the training method of the feature extraction model of this application;

[0018] Figure 6 This is a schematic flowchart of an embodiment of the image processing method of this application;

[0019] Figure 7 This is a schematic diagram of the structure of a fractal network, illustrating an embodiment of the training method for the feature extraction model of this application.

[0020] Figure 8This is a schematic diagram illustrating the structure of a fractal network in one embodiment of the image processing method of this application;

[0021] Figure 9 This is a schematic diagram of the structure of an embodiment of the training device for the feature extraction model of this application;

[0022] Figure 10 This is a schematic diagram of the structure of an embodiment of the image processing apparatus of this application;

[0023] Figure 11 This is a schematic diagram of the structure of an embodiment of the electronic device of this application;

[0024] Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0025] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0026] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0027] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0028] This application provides training methods and / or image processing methods and apparatus for feature extraction models. The application scenarios of these training methods and / or image processing methods include, but are not limited to, object recognition analysis and security monitoring analysis. For example, in security monitoring analysis, sample images can be facial information, vehicle information, animal information, etc., that need to be identified in security monitoring. The detection or recognition result can be the wearing status of accessories in facial information, such as whether a mask is worn, whether glasses are worn, etc., as the target object set according to task requirements. The execution entity of the training methods and / or image processing methods for feature extraction models can be a training device and / or an image processing device for feature extraction models. For example, the training device and / or image processing device can be a terminal device, a server, or other processing device. The terminal device can be a device used for training the feature extraction model, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, etc. In some possible implementations, the training method and / or image processing method of the feature extraction model can be implemented by the processor calling computer-readable instructions stored in memory.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the training method for the feature extraction model of this application.

[0030] Specifically, the training method for a feature extraction model may include the following steps:

[0031] Step S11: Obtain sample images.

[0032] A feature extraction model can include multiple models or modules, including multiple backbone modules and multiple intermediate connection modules. Each module can be a network module or an algorithm module. The models or modules are as follows, and will not be elaborated here.

[0033] Sample images can be two-dimensional or three-dimensional. They can be static or dynamic, and can also be extracted from video. In some applications, sample images can be used for image processing. There can be multiple sample images, some containing a target object and others not. Image processing can include object detection, object recognition, and object classification. For example, object detection can detect whether a face is wearing a mask. Some sample images can be faces without masks, and some can be faces with masks. For instance, sample images can be facial images captured by a camera, advertising posters, emojis, user interface designs, comics, etc. This application uses facial images as an example.

[0034] Sample images can be obtained through various methods, including offline camera capture, downloading from public datasets, and video file extraction. In some applications, offline capture is a viable method. This involves organizing on-site collection of facial sample images. Depending on the task requirements, different groups can be selected, and specific equipment or cameras can be used to capture the participants' facial images during surveys, experiments, or activities. With the participants' consent, the facial sample images can be properly processed and stored. In other applications, sample images can be obtained by downloading from public datasets. These datasets can be object detection datasets such as the WiderFace Face Detection Dataset, the AIZOO Mask Recognition Dataset, and the Real-World Masked Face Dataset (RMFD), containing facial images of the target objects. Finally, in some applications, sample images can be obtained by extracting from video files. Image processors can be used to extract the desired facial sample images from video files containing facial images.

[0035] For example, sample images can be obtained by using an open-source computer vision library (OpenCV) to extract and split pre-prepared video data from a surveillance scene into single-frame image sequences, and then annotating them using relevant software. To enhance the generalization of the model, images from the AIZOO and RMFD datasets are further added to form a face mask detection dataset. The sample images can be composed of the processed frame image sequence data, AIZOO dataset, and RMFD dataset images to form the face mask detection dataset. Before inputting the sample images into the feature extraction model, the face mask detection dataset can be preprocessed, and the preprocessed face mask detection dataset can be used as the sample images input into the feature extraction model. The preprocessing of the face mask detection dataset can be achieved by dividing the dataset into training, validation, and test sets in an 8:1:1 ratio, and then processing the training and validation sets. This processing can include translation, rotation, scaling, etc. However, methods such as vertical flipping and mosaic enhancement are not suitable for face detection; other image processing methods can be applied depending on the task requirements. The processed training set can be used as sample images to input into the feature extraction model.

[0036] This is just an example and does not limit the method of obtaining the sample images.

[0037] Step S12: Use the student feature extraction model and the teacher feature extraction model to extract features from the sample image respectively, and obtain multiple initial feature maps of different sizes. The complexity of the student feature extraction model is less than that of the teacher feature extraction model.

[0038] The feature extraction model can include a student feature extraction model and a teacher feature extraction model. Sample images can be input into the student feature extraction model and the teacher feature extraction model, respectively, for feature extraction. Specifically, sample images can be input into the backbone module of the student feature extraction model, which outputs multiple initial feature maps of different sizes; similarly, sample images can be input into the backbone module of the teacher feature extraction model, which also outputs multiple initial feature maps of different sizes. Knowledge distillation is performed between the teacher and student feature extraction models. Here, the specific structure of the backbone module in the teacher feature extraction model is the same as that in the student feature extraction model, but the number of channels differs. For ease of description, this scheme uses the structure of the student feature extraction model as an example; the structure of the backbone module in the teacher feature extraction model can be referenced from the student feature extraction model.

[0039] Feature extraction from sample images can be performed using any network or algorithm with feature extraction capabilities. The feature extraction network can be a lightweight network (e.g., ShuffleNet-v2), a mobile network (e.g., MobileNet), a Visual Geometry Group Network (VGGNet), a Residual Network (ResNet), or a variant thereof. For example, it could be different versions of the lightweight network ShuffleNet-v2, such as ShuffleNet-v2-2x and ShuffleNet-v2-0.5x.

[0040] Both student and teacher feature extraction models are models capable of extracting features from sample images. The student feature extraction model is less complex than the teacher model. The teacher model may have more channels, more layers, deeper layers, or larger convolutional kernels compared to the student model. In some applications, the student and teacher feature extraction models have the same structure, but the teacher model has more channels. In other applications, the teacher model includes all the structures of the student model, and may also include other structures, such as a deeper model where the number of channels is greater than or equal to that of the student model. The teacher model may have more parameters, higher computational cost, larger memory usage, or longer inference time compared to the student model. The initial feature map can be represented as (H, W, C), where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels. The initial feature maps can be obtained separately through student feature extraction models and teacher feature extraction models. It's understandable that the initial features obtained after undergoing the same processing by both student and teacher feature extraction models are... Figure 1 The initial feature maps obtained after different processing levels by the student feature extraction model and the teacher feature extraction model are not the same size.

[0041] In some applications, the teacher feature extraction model can be directly modified by reducing the number of layers or the depth to obtain a student feature extraction model with the same model structure. In other applications, the width of each layer or the number of parameters in the teacher feature extraction model can be reduced to obtain a student feature extraction model with the same model structure. In still other applications, model compression techniques can be used to derive a student feature extraction model from the teacher feature extraction model through knowledge distillation. The student and teacher feature extraction models have identical model structures. In knowledge distillation, the teacher feature extraction model is typically a complex and accurate model, while the student feature extraction model is a simplified one. By passing the prediction results (soft labels) or intermediate representations (soft targets) of the teacher feature extraction model to the student feature extraction model as auxiliary targets for training, the student feature extraction model can learn the knowledge from the teacher feature extraction model. For example, this application could involve obtaining a student feature extraction model from a teacher feature extraction model through knowledge distillation.

[0042] It can be argued that the student feature extraction model can learn richer information from the teacher feature extraction model. The student feature extraction model can achieve similar performance to the teacher feature extraction model while maintaining relatively low complexity. The student feature extraction model, obtained by knowledge distillation of the teacher feature extraction model, reduces memory usage and computational costs. Through knowledge distillation, the student feature extraction model learns richer information from the teacher feature extraction model, including soft labels and intermediate representations. This additional knowledge transfer helps the student feature extraction model generalize better on both the training and test sets, improving model performance. The student feature extraction model can learn how to resist noise in the input data from the teacher feature extraction model, thus improving its robustness. Due to its simplified model complexity, the student feature extraction model has a higher inference speed compared to the teacher feature extraction model. This is highly beneficial for deploying trained student feature extraction models to applications on edge devices, mobile devices, or resource-constrained environments.

[0043] Step S13: Use the student feature extraction model and the teacher feature extraction model to fuse multiple initial feature maps to obtain target feature maps corresponding to each initial feature map. The number of target feature maps is the same as the number of initial feature maps.

[0044] Multiple initial feature maps can be input into the intermediate connection modules of the student feature extraction model and the teacher feature extraction model, respectively, for feature fusion to obtain target feature maps. Specifically, multiple initial feature maps corresponding to the student feature extraction model can be input into the intermediate connection module of the student feature extraction model, and multiple initial feature maps corresponding to the teacher feature extraction model can be input into the intermediate connection module of the teacher feature extraction model. The number of target feature maps is the same as the number of initial feature maps. Knowledge distillation is performed between the teacher feature extraction model and the student feature extraction model. Here, the specific structure of the intermediate connection module in the teacher feature extraction model is the same as that in the student feature extraction model, but the number of channels differs. For ease of description, this scheme uses the structure of the student feature extraction model as an example; the structure of the intermediate connection module in the teacher feature extraction model can be referenced from the student feature extraction model.

[0045] The target feature map is the output of the feature extraction model. It can be a feature map corresponding to each of the initial feature maps obtained by fusing multiple initial feature maps. Feature fusion of multiple initial feature maps can be performed using any network or algorithm with feature fusion capabilities. In some applications, intermediate connection modules can perform feature fusion operations such as weighted summation, attention mechanisms, and averaging on the input feature maps. In other applications, intermediate connection modules can combine with a path aggregation network (PAN) to obtain multi-scale feature maps and perform connections and fusion between adjacent feature maps.

[0046] Step S14: Using the similarity loss between multiple target feature maps obtained by the student feature extraction model and multiple target feature maps obtained by the teacher feature extraction model, the parameters in the student feature extraction model are adjusted. After training, the student feature extraction model can perform feature extraction and feature fusion on the image to be processed in subsequent image processing.

[0047] Multiple target feature maps can be the target feature maps output by the student feature extraction model and the target feature maps output by the teacher feature extraction model. For multiple target feature maps, different similarity calculation methods can be used to obtain the similarity loss between them. In some application scenarios, similarity calculation methods can include cosine similarity, Euclidean distance, Manhattan distance, etc. Different similarity calculation methods can be selected according to requirements. In some application scenarios, the parameters in the student feature extraction model can be adjusted and optimized based on generative adversarial networks (GANs). Based on GANs, the target feature maps output by the intermediate connection modules of the teacher feature extraction model can be used as real samples, and the teacher feature extraction model can be used as the discriminator. Based on GANs, the target feature maps output by the intermediate connection modules of the student feature extraction model can be used as fake samples, and the student feature extraction model can be used as the discriminator. The similarity between real and fake samples is calculated. The student feature extraction model and the teacher feature extraction model perform self-adversarial distillation learning, training on a face mask dataset, ultimately obtaining a high-precision student feature extraction model. The trained student feature extraction model can then perform feature extraction and feature fusion on the image to be processed in subsequent image processing.

[0048] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0049] In some embodiments, the student feature extraction model and the teacher feature extraction model include a channel transformation module, which includes multiple cascaded channel transformation modules and multiple mapping modules. Each initial feature map includes an initial feature map corresponding to each channel transformation module. Step S12 may include the following steps: First, the sample image is preprocessed using the student feature extraction model and the teacher feature extraction model respectively to obtain a preprocessed feature map. Then, for the first channel transformation module, the preprocessed feature map is processed to obtain the feature map output by the first channel transformation module. Next, for each non-first channel transformation module, the feature map output by the previous channel transformation module is processed to obtain the feature map corresponding to each non-first channel transformation module. Subsequently, the feature maps output by each channel transformation module are mapped using each mapping module to obtain the initial feature map corresponding to each channel transformation module. The number of channels in the initial feature map corresponding to each channel transformation module is less than the number of channels in the feature map output by each channel transformation module.

[0050] The backbone module of the student feature extraction model can include a channel transformation module. This module can consist of multiple cascaded channel transformation modules and multiple mapping modules. A channel transformation module can be composed of a first channel transformation module and multiple non-first channel transformation modules, a first mapping module, and multiple non-first mapping modules. The channel transformation module processes the feature maps using depthwise separable convolution and channel shuffle operations. Depthwise separable convolution involves convolutional operations on channels within each group, resulting in independent convolutional feature maps for each channel group. This reduces computational complexity and increases gradient propagation efficiency. Channel shuffle involves dividing the input feature map into several equal channel groups and shuffling the feature maps from different groups to promote information exchange and feature integration between different groups. In some application scenarios, the channel transformation module can be composed of different versions of a lightweight network (ShuffleNet-v2). For example, ShuffleNet-v2-0.5x can be used as the channel transformation module in the student feature extraction model. ShuffleNet-v2-2x can be used as the channel transformation module in the teacher feature extraction model.

[0051] Each mapping module maps the feature maps output by the channel transformation modules to obtain the initial feature maps corresponding to each channel transformation module. In some applications, the mapping module can be a pooling layer, which can downsample the feature maps, reducing their size while retaining the main features. In some applications, the mapping module can be a fully connected layer. In some applications, the mapping module can be a convolutional layer. For example, the mapping modules in the student feature extraction model and the teacher feature extraction model can be convolutional layers with the same structure, specifically 1×1 convolutional layers.

[0052] First, the sample images are preprocessed using both the student feature extraction model and the teacher feature extraction model to obtain preprocessed feature maps. Specifically, the core module of the student feature extraction model may include a preprocessing module. Each preprocessing module can perform preliminary processing on the sample images, reducing the computational load in subsequent processing steps within each core module. For example, the preprocessing module may be a Conv-BN-Manpooling module. This preprocessing module can consist of a convolutional layer (Conv), a batch normalization layer (BN), and a pooling layer (Manpooling). The sample images can be input into the preprocessing modules of both the student and teacher feature extraction models. First, the sample images are input into the convolutional layer of the preprocessing module, where a downsampling operation with a kernel size of 3 and a stride of 2 is performed. Then, the downsampled sample images are input into the batch normalization layer for batch normalization, standardizing the image features. Subsequently, a max pooling operation with a stride of 2 is performed on the feature maps after batch normalization, resulting in an output feature map size of 80×80. It is understandable that the preprocessed feature maps obtained by preprocessing the sample images using both the student and teacher feature extraction models are both 80×80 pixels in size. Since the teacher and student feature extraction models have the same structure but different numbers of channels, this application only describes the size of the model feature maps when describing the execution of each module in the feature extraction model. That is, when describing the student feature extraction model, this application only describes the size of the model feature maps.

[0053] In some embodiments, each channel conversion module includes a first channel conversion submodule with a step size of 2 and a second channel conversion submodule with a step size of 1.

[0054] Each channel transformation module can be a first channel transformation submodule with a stride of 2 and a second channel transformation submodule with a stride of 1, cascaded sequentially. In some application scenarios, both the first and second channel transformation submodules can perform depthwise separable convolution and channel shuffle operations.

[0055] In some application scenarios, the first and second channel transformation submodules within each channel transformation module can be used to perform channel transformation processing on the feature maps of the input channel transformation module, achieving information propagation and integration, thereby enhancing the model's representational capabilities and propagation effects. This can improve the performance of student and teacher feature extraction models while reducing computational complexity.

[0056] For example, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of the first channel transformation submodule of each channel transformation module in one embodiment of the training method of the feature extraction model of this application. The first channel transformation submodule can be a Shuffle-block (stride=2) submodule. The first channel transformation submodule may include a first separation module, a first processing module, a first convolution module, a first transformation module, a second convolution module, a second transformation module, a first cascade module, and a first shuffling module. Specifically, the first separation module performs a channel split operation on the input feature map to obtain two sets of feature maps corresponding to the first separation module.

[0057] The first processing module can be a 1×1 convolution-BN-ReLU module. It performs 1×1 convolution (1×1 convolution), batch normalization (BN), and non-linear activation (ReLU) on the feature maps corresponding to one set of the first separation modules to obtain the feature maps corresponding to the first processing modules. The first convolution module can be a 3x3 DWConv-BN module with a stride of 2. It performs dimensionality reduction on the feature maps corresponding to the first processing modules to obtain the feature maps corresponding to the first convolution modules. The dimensionality reduction performed by the first convolution module can be a depthwise separable convolution with a kernel size of 3×3 and a stride of 2 (3x3 DWConv, stride = 2), followed by batch normalization (BN). The first transformation module can be a 1×1 convolution-BN-ReLU module. It performs 1×1 convolution (1×1 convolution), batch normalization (BN), and non-linear activation (ReLU) on the feature maps corresponding to the first convolution modules to obtain the feature maps corresponding to the first transformation modules.

[0058] The second convolutional module can be a 3x3 DWConv-BN module with a stride of 2. The second convolutional module is used to reduce the dimensionality of the feature maps corresponding to the first separating module, resulting in the feature map corresponding to the second convolutional module. The dimensionality reduction performed by the second convolutional module can be a depthwise separable convolution with a kernel size of 3×3 and a stride of 2 (3x3 DWConv, stride = 2), followed by batch normalization (BN). The second transformation module can be a 1×1 conv-BN-ReLU module. The second transformation module is used to perform 1×1 convolution (1×1 conv), batch normalization (BN), and non-linear activation (ReLU) on the feature map corresponding to the second convolutional module, resulting in the feature map corresponding to the second transformation module.

[0059] Specifically, the first concatenation module performs channel concatenation (Concat) on the feature maps corresponding to the first transformation module and the second transformation module to obtain the feature map corresponding to the first concatenation module. The first shuffle module then performs channel shuffle on the feature map corresponding to the first concatenation module to obtain the advanced features corresponding to the first channel transformation submodule.

[0060] For example, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of the second channel transformation submodule of each channel transformation module in one embodiment of the training method of the feature extraction model of this application. The second channel transformation submodule can be a Shuffle-block (stride=1) submodule. The second channel transformation submodule may include a second separation module, a second processing module, a third convolution module, a third transformation module, a second concatenation module, and a second shuffling module. Specifically, the second separation module performs a channel split operation on the input feature map to obtain two sets of feature maps corresponding to the second separation module.

[0061] The second processing module can be a 1×1 conv-BN-ReLU module. It performs 1×1 convolution (1×1 conv), batch normalization (BN), and non-linear activation (ReLU) on a set of feature maps corresponding to the second separation module to obtain the feature map corresponding to the second processing module. The third convolution module can be a 3x3 DWConv-BN module with stride = 1. It performs depthwise separable convolution on the feature maps corresponding to the second processing module to obtain the feature map corresponding to the third convolution module. The depthwise separable convolution performed by the third convolution module can be a 3x3 DWConv depthwise separable convolution with a kernel size of 3×3 and a stride of 1 (3x3 DWConv, stride = 1), followed by batch normalization (BN). The third transformation module can be a 1×1 conv-BN-ReLU module. The feature map corresponding to the third convolution module is processed by 1×1 convolution (1×1conv), batch normalization (BN), and non-linear activation (ReLU) using the third transformation module to obtain the feature map corresponding to the third transformation module.

[0062] Specifically, the second concatenation module performs channel concatenation (Concat) on the feature map corresponding to the third transformation module and another set of feature maps corresponding to the second separation module to obtain the feature map corresponding to the second concatenation module. The second shuffling module then performs channel shuffling (Channel shuffle) on the feature map corresponding to the second concatenation module to obtain the feature maps corresponding to each channel transformation module.

[0063] Then, for the first channel transformation module, the preprocessed feature map is processed to obtain the feature map output by the first channel transformation module. The preprocessed feature map is then input into the first channel transformation module, and after depthwise separable convolution and channel shuffling operations, the feature map output by the first channel transformation module is obtained.

[0064] The first channel transformation module can be the one among the various channel transformation modules that is directly connected to the preprocessing module. In some applications, the preprocessed feature map is input into the channel transformation module by performing dimensionality reduction on the feature map to obtain a feature map with half the size. In other applications, the preprocessed feature map is input into the channel transformation module by performing depthwise separable convolution and channel shuffling operations on the preprocessed feature map using the first channel transformation module to obtain the feature map output by the first channel transformation module.

[0065] In some embodiments, the above-described processing of the preprocessed feature map by the first channel transformation module to obtain the feature map output by the first channel transformation module may include the following steps: First, the preprocessed feature map is processed using the first channel transformation submodule to obtain a first advanced feature map. Then, the first advanced feature map is processed using the second channel transformation submodule to obtain the feature map output by the first channel transformation module.

[0066] For the first channel transformation module in the channel transformation module, the preprocessed feature map is processed using the first channel transformation submodule within the first channel transformation module to obtain a first advanced feature map. The first advanced feature map is then processed using the second channel transformation submodule within the first channel transformation module to obtain the feature map output by the first channel transformation module. For example, the preprocessed feature map can be processed by performing a depthwise separable convolution and a channel shuffle operation through the first channel transformation submodule within the first channel transformation module to obtain the first advanced feature map. Alternatively, the first channel transformation module can process a preprocessed feature map with a feature map size of 80×80 to obtain a feature map output by the first channel transformation module with a feature map size of 40×40.

[0067] Next, for each non-first channel transformation module, the feature map output by the previous channel transformation module is processed to obtain the feature map corresponding to each non-first channel transformation module.

[0068] A non-first channel transformation module can be any channel transformation module in the channel transformation module that is not directly connected to the preprocessing module. The preceding channel transformation module corresponding to this non-first channel transformation module can be either the first channel transformation module or a non-first channel transformation module. In some application scenarios, processing the feature map output by the preceding channel transformation module can involve dimensionality reduction to obtain a feature map with half the size. In some application scenarios, processing the feature map output by the preceding channel transformation module can involve inputting the feature map output by the preceding first channel transformation module corresponding to this non-first channel transformation module into the non-first channel transformation module for processing, thus obtaining the feature map corresponding to the non-first channel transformation module. In some application scenarios, processing the feature map output by the preceding channel transformation module can involve inputting the feature map output by the preceding non-first channel transformation module corresponding to this non-first channel transformation module into the non-first channel transformation module for processing, thus obtaining the feature map corresponding to the non-first channel transformation module.

[0069] In some embodiments, the above-described processing of the feature map output by the previous channel transformation module to obtain the feature map corresponding to each non-first channel transformation module may include the following steps: First, using the first channel transformation submodule, the feature map output by the previous channel transformation module is processed to obtain a second advanced feature map. Then, using the second channel transformation submodule, the second advanced feature map is processed to obtain the feature map output by the non-first channel transformation module.

[0070] For non-first channel transformation modules in the channel transformation module, the feature map output by the previous channel transformation module is processed by the non-first channel transformation module to obtain the feature map corresponding to each non-first channel transformation module. Specifically, the first channel transformation submodule in the non-first channel transformation module processes the feature map output by the previous channel transformation module to obtain the second advanced feature map. The second channel transformation submodule in the non-first channel transformation module processes the second advanced feature map to obtain the feature map corresponding to each non-first channel transformation module.

[0071] For example, the feature map output from the previous channel transformation module can be processed by the first channel transformation submodule in a non-first channel transformation module, performing a depthwise separable convolution operation and a channel shuffle operation to obtain a second advanced feature map. Alternatively, a non-first channel transformation module can be used to process the feature map output from the first channel transformation module with a feature map size of 40×40 to obtain a feature map output from a non-first channel transformation module with a feature map size of 20×20. Or, a non-first channel transformation module can be used to process the feature map output from a non-first channel transformation module with a feature map size of 20×20 to obtain a feature map output from a non-first channel transformation module with a feature map size of 10×10.

[0072] Understandably, different sizes of preprocessed feature maps can be set according to requirements to obtain feature maps of different sizes output by each channel transformation module.

[0073] In some application scenarios, the number of channels in the initial feature map corresponding to each channel transformation module is less than the number of channels in the feature map output by each channel transformation module. For example, in the student feature extraction model, the number of channels in the initial feature map corresponding to each channel transformation module can be 96. In the teacher feature extraction model, the number of channels in the initial feature map corresponding to each channel transformation module can be 512. As long as the number of channels in the initial feature map corresponding to each channel transformation module is less than the number of channels in the feature map output by each channel transformation module, the specific number of channels in the feature map output by each channel transformation module is not limited here.

[0074] It is understandable that the step of mapping the feature maps output by each channel transformation module to obtain the initial feature map corresponding to each channel transformation module can be achieved by reducing the number of channels of the feature maps output by each channel transformation module to a preset value for the initial feature map corresponding to each channel transformation module, without changing the size of the feature maps. For example, the feature maps output by each channel transformation module pass through each mapping module, and each mapping module can be a 1×1 convolutional layer. The initial feature map corresponding to the first channel transformation module can be (40, 40, C). The initial feature maps corresponding to subsequent channel transformation modules can be (20, 20, C) or (10, 10, C).

[0075] It can be considered that, considering the subsequent deployment of the student feature extraction model in monitoring and security equipment, the size of the network model, the computational resources used in the model inference process, the memory access cost, and the model inference speed are very important. In the ShuffleNet-v2 network, the channel transformation module uses depthwise separable convolution and channel shuffle operations, which significantly reduces the computational load of the network while ensuring network accuracy. In this application, different versions of the ShuffleNet-v2 network are used in the backbone modules of the student feature extraction model and the teacher feature extraction model, respectively.

[0076] In some embodiments, step S13 may include the following steps: First, one of the initial feature maps is used as the starting feature map, and the feature maps other than the starting feature map are used as intermediate feature maps. The starting feature map has only one initial feature map of adjacent size. Then, for each intermediate feature map, the intermediate feature map is fused with the fusion feature map corresponding to the previous adjacent starting feature map or the previous adjacent intermediate feature map to obtain the fusion feature map corresponding to each intermediate feature map. Next, the starting feature map and the fusion feature maps corresponding to each intermediate feature map are fused to obtain each target feature map.

[0077] The initial feature maps have different sizes. In some application scenarios, the intermediate connection module in the student feature extraction model is used to fuse the features of each initial feature map to obtain the target feature maps. For example, one of the initial feature maps is used as the starting feature map, and the feature maps other than the starting feature map are used as intermediate feature maps. The starting feature map only has one initial feature map of adjacent size. It is understood that in some application scenarios, the starting feature map here can be the initial feature map corresponding to the first channel transformation module. In some application scenarios, the starting feature map here can also be the initial feature map corresponding to the last non-first channel transformation module among the non-first channel transformation modules. The last non-first channel transformation module can be the module furthest from the first channel transformation module among the non-first channel transformation modules of the channel transformation module. For example, this application uses the initial feature map corresponding to the first channel transformation module as the starting feature map.

[0078] In some embodiments, the step of fusing the intermediate feature map with the previous adjacent starting feature map or the fusion feature map corresponding to the previous adjacent intermediate feature map to obtain the fusion feature map corresponding to each intermediate feature map may be: using the intermediate connection module in the student feature extraction model to fuse the intermediate feature map, the fusion feature map corresponding to each intermediate feature map, and the starting feature map to obtain the fusion feature map corresponding to each intermediate feature map in the student feature extraction model.

[0079] In some application scenarios, the first intermediate feature map can be fused with the starting feature map to obtain a fused feature map. The first intermediate feature map can be the one whose size is closest to the starting feature map among all intermediate feature maps. In other application scenarios, for non-first intermediate feature maps, they can be fused with the fused feature map corresponding to their preceding adjacent intermediate feature map to obtain a fused feature map.

[0080] In some embodiments, the step of fusing the intermediate feature map with the preceding adjacent starting feature map or the fusion feature map corresponding to the preceding adjacent intermediate feature map to obtain the fusion feature map corresponding to each intermediate feature map may include the following steps: First, for the first intermediate feature map, concatenate the starting feature map after the dimensionality reduction module to obtain the fusion feature map corresponding to the first intermediate feature map. Then, for each non-first intermediate feature map, concatenate the fusion feature map corresponding to the preceding adjacent intermediate feature map after the dimensionality reduction module to obtain the fusion feature map corresponding to each non-first intermediate feature map.

[0081] The dimensionality reduction module can perform dimensionality reduction processing on the feature maps of the input dimensionality reduction modules. For example, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of each dimensionality reduction module in one embodiment of the training method for the feature extraction model of this application. The dimensionality reduction module can be a Ghost block module. Specifically, the dimensionality reduction module may include a first channel processing submodule, a depthwise convolution submodule, a second channel processing submodule, and a concatenation submodule. The first and second channel processing submodules can be Ghost Module modules. Ghost Module modules can be used to reduce the number of parameters and computational complexity of the feature extraction model. Specifically, the Ghost Module module corresponding to the first channel processing submodule can be used to increase the number of channels in the feature map. The Ghost Module module corresponding to the second channel processing submodule can be used to reduce the number of channels in the feature map. The first channel processing submodule is used to process the feature maps of each input dimensionality reduction module to obtain the feature map corresponding to the first channel processing submodule. The depthwise convolution submodule can be a Dwconv(stride=2) module. The depthwise convolution submodule is used to perform a depthwise convolution with a stride of 2 on the feature map corresponding to the first channel processing submodule to obtain the feature map corresponding to the depthwise convolution submodule. The second-channel processing submodule performs channel processing on the feature map corresponding to the depthwise convolution submodule to obtain the feature map corresponding to the second-channel processing submodule. The concatenation submodule can be an ADD module. The concatenation submodule performs feature fusion (ADD operation) on the feature map corresponding to the second-channel processing submodule and the feature maps of each input dimensionality reduction module to obtain the dimensionality reduction feature maps.

[0082] Cascading can increase the number of channels in a feature map. The third cascading module in the intermediate connection module can be a concatenate module. This third cascading module performs a concatenation operation (concatenate) on the feature maps from each dimensionality reduction module and the intermediate feature maps to obtain a fused feature map corresponding to each intermediate feature map.

[0083] In some application scenarios, for the first intermediate feature map, a dimensionality reduction module is used to reduce the dimensionality of the starting feature map to obtain the dimensionality-reduced feature map corresponding to the first intermediate feature map. A third concatenation module is then used to concatenate the dimensionality-reduced feature maps corresponding to the first intermediate feature map to obtain the fused feature map corresponding to the first intermediate feature map. For example, the starting feature map can be (40, 40, C). The first intermediate feature map can be (20, 20, C). The fused feature map corresponding to the first intermediate feature map can be (20, 20, C).

[0084] In some application scenarios, for non-first intermediate feature maps, a dimensionality reduction module processes the fused feature map corresponding to the preceding adjacent intermediate feature map to obtain dimensionality-reduced feature maps for each non-first intermediate feature map. A third concatenation module then concatenates these dimensionality-reduced feature maps to obtain fused feature maps for each non-first intermediate feature map. For example, the fused feature map corresponding to the preceding adjacent intermediate feature map could be the fused feature map (20, 20, C) corresponding to the first intermediate feature map. A non-first intermediate feature map could be (10, 10, C). The fused feature map corresponding to this non-first intermediate feature map could be (10, 10, C).

[0085] Next, feature fusion is performed on the initial feature map and the fused feature maps corresponding to each intermediate feature map to obtain each target feature map.

[0086] The target feature map can be the output obtained by fusing features from the intermediate connection modules of the feature extraction model, which are input from each initial feature map. The intermediate connection modules in the student feature extraction model are used to fuse the initial feature map and the corresponding fused feature maps of each intermediate feature map to obtain the target feature maps corresponding to the student feature extraction model. In some application scenarios, the target feature map can be obtained by fusing features from the initial feature map and the fused feature map corresponding to the first intermediate feature map. In some application scenarios, the target feature map can be obtained by fusing features from the fused feature map corresponding to the first intermediate feature map and the fused feature map corresponding to a non-first intermediate feature map. In some application scenarios, the target feature map can be obtained by fusing features from the fused feature map corresponding to a non-first intermediate feature map and the fused feature map corresponding to a non-first intermediate feature map.

[0087] In some embodiments, the above-described feature fusion of the starting feature map and the fused feature maps corresponding to each intermediate feature map to obtain each target feature map may include the following steps: First, the fused feature map corresponding to the last intermediate feature map is used as the starting fused feature map, and the other fused feature maps are used as intermediate fused feature maps. Second, for each intermediate fused feature map, the intermediate fused feature map is fused with the advanced feature map corresponding to the previous adjacent starting fused feature map or the previous adjacent intermediate fused feature map to obtain the advanced feature map corresponding to each intermediate fused feature map. Then, the starting feature map is fused with the last advanced feature map to obtain the advanced feature map corresponding to the starting feature map. Next, based on the starting fused feature map and each advanced feature map, the target feature map corresponding to the starting fused feature map and the target feature map corresponding to each advanced feature map are obtained.

[0088] The last intermediate feature map can be an initial feature map with only one adjacent size, excluding the starting feature map. The last intermediate feature map can be the feature map with the smallest feature map size among all intermediate feature maps. The starting fusion feature map can be the fusion feature map corresponding to the last intermediate feature map. The intermediate fusion feature map can be the fusion feature map corresponding to all intermediate feature maps except the last one.

[0089] Secondly, for each intermediate fused feature map, the intermediate fused feature map is fused with the advanced feature map corresponding to the previous adjacent starting fused feature map or the previous adjacent intermediate fused feature map to obtain the advanced feature map corresponding to each intermediate fused feature map.

[0090] The preceding neighboring intermediate fused feature map can be an intermediate fused feature map that is adjacent to the current intermediate fused feature map but performs fewer feature fusion steps. The intermediate connection module in the student feature extraction model is used to perform feature fusion between this intermediate fused feature map and the advanced feature map corresponding to the initial fused feature map or the preceding neighboring intermediate fused feature map, to obtain the advanced feature maps corresponding to each intermediate fused feature map in the student feature extraction model.

[0091] In some application scenarios, for the first intermediate fused feature map, it can be fused with the starting fused feature map to obtain the advanced feature map corresponding to the first intermediate fused feature map. The first intermediate fused feature map can be the one whose feature map size is closest to the starting fused feature map among all intermediate fused feature maps. In some application scenarios, for a non-first intermediate fused feature map, it can be fused with the advanced feature map corresponding to the preceding adjacent first intermediate fused feature map to obtain the advanced feature map corresponding to the non-first intermediate fused feature map. In some application scenarios, for a non-first intermediate fused feature map, it can be fused with the advanced feature map corresponding to the preceding adjacent non-first intermediate fused feature map to obtain the advanced feature map corresponding to the non-first intermediate fused feature map.

[0092] Downsampling can be used to reduce the dimensionality of the input feature map. Concatenation can be used to concatenate multiple input feature maps, increasing the number of channels in the feature map. In some application scenarios, the downsampling module and the fourth concatenation module in the intermediate connection module are used to process the intermediate fused feature map with the advanced feature map corresponding to the starting fused feature map or the previous adjacent intermediate fused feature map, to obtain the advanced feature maps corresponding to each intermediate feature map and the advanced feature map corresponding to the starting feature map. For example, the fused feature map corresponding to the last intermediate feature map can be used as the starting fused feature map, and the other fused feature maps can be used as intermediate fused feature maps. The downsampling module can be a Downsample module. It performs downsampling and dimensionality reduction on the input feature map to obtain the feature map corresponding to the downsampled module. The fourth concatenation module in the intermediate connection module can be a Concatenate module. It performs concatenation on the input feature map to obtain the feature map corresponding to the fourth concatenation module.

[0093] In some embodiments, the step of fusing the intermediate fused feature map with the preceding adjacent initial fused feature map or the advanced feature map corresponding to the preceding adjacent intermediate fused feature map to obtain the advanced feature map corresponding to each intermediate fused feature map includes the following steps: First, for the first intermediate fused feature map, the downsampled initial fused feature map is concatenated to obtain the advanced feature map corresponding to the first intermediate fused feature map. Second, for each non-first intermediate fused feature map, the downsampled advanced feature map corresponding to the preceding adjacent intermediate fused feature map is concatenated to obtain the advanced feature map corresponding to each non-first intermediate fused feature map.

[0094] In some application scenarios, for the first intermediate fused feature map, the downsampling module is used to downsample and reduce the dimensionality of the initial fused feature map to obtain the downsampled feature map corresponding to the initial fused feature map. The fourth concatenation module is then used to concatenate the downsampled feature map corresponding to the initial fused feature map and the first intermediate fused feature map to obtain the advanced feature map corresponding to the first intermediate fused feature map.

[0095] For each non-first intermediate fused feature map, a downsampling module is used to downsample the preceding adjacent intermediate fused feature map to obtain the downsampled feature map corresponding to each non-first intermediate fused feature map. The fourth concatenation module is then used to concatenate the downsampled feature map corresponding to each non-first intermediate fused feature map with the non-first intermediate fused feature map itself to obtain the advanced feature map corresponding to each non-first intermediate fused feature map. In some application scenarios, for the non-first intermediate fused feature map, the downsampling module performs downsampling and dimensionality reduction processing on the advanced feature map corresponding to the preceding adjacent first intermediate fused feature map to obtain the downsampled feature map corresponding to the preceding adjacent first intermediate fused feature map. The fourth concatenation module is then used to concatenate the downsampled feature map corresponding to the preceding adjacent first intermediate fused feature map with the non-first intermediate fused feature map to obtain the advanced feature map corresponding to the non-first intermediate fused feature map. In some application scenarios, for the non-first intermediate fused feature map, a downsampling module is used to perform downsampling and dimensionality reduction on the advanced feature map corresponding to the preceding adjacent non-first intermediate fused feature map, resulting in a downsampled feature map corresponding to the preceding adjacent non-first intermediate fused feature map. A fourth concatenation module is then used to concatenate the downsampled feature map corresponding to the preceding adjacent non-first intermediate fused feature map and the non-first intermediate fused feature map to obtain the advanced feature map corresponding to the non-first intermediate fused feature map.

[0096] Then, the initial feature map and the last advanced feature map are fused to obtain the advanced feature map corresponding to the initial feature map.

[0097] The last advanced feature map can be the feature map that has undergone the most feature fusion steps among the advanced feature maps corresponding to the intermediate fused feature maps. The intermediate connection module in the student feature extraction model is used to fuse the last advanced feature map with the initial feature map to obtain the advanced feature map corresponding to the initial feature map in the student feature extraction model.

[0098] In some embodiments, the step of fusing the initial feature map with the last advanced feature map to obtain the advanced feature map corresponding to the initial feature map includes the following steps: for the initial feature map, concatenating the last advanced feature map after downsampling to obtain the advanced feature map corresponding to the initial feature map.

[0099] In some application scenarios, for the initial feature map, the last advanced feature map among the advanced feature maps corresponding to each non-first intermediate fused feature map is input into the downsampling module for downsampling processing to obtain the downsampled feature map corresponding to the last advanced feature map. The fourth concatenation module is then used to concatenate the downsampled feature map corresponding to the last advanced feature map and the initial feature map to obtain the advanced feature map corresponding to the initial feature map.

[0100] Next, based on the initial fused feature map and each advanced feature map, the target feature map corresponding to the initial fused feature map and the target feature map corresponding to each advanced feature map are obtained.

[0101] The method for feature fusion of the initial fusion feature map and each advanced feature map can be set according to requirements. In some application scenarios, the feature fusion method can be weighted summation fusion, maximum value fusion, attention mechanism fusion, etc., respectively, on the initial fusion feature map and each advanced feature map.

[0102] In some embodiments, the step of obtaining the target feature map corresponding to the initial fusion feature map and the target feature map corresponding to each advanced feature map based on the initial fusion feature map and each advanced feature map may include the following steps: processing the initial fusion feature map and each advanced feature map through a dimensionality reduction module to obtain the target feature map corresponding to the initial fusion feature map and the target feature map corresponding to each advanced feature map.

[0103] The initial fused feature map and each advanced feature map are processed by a dimensionality reduction module to reduce the dimensionality of the input dimensionality reduction module's feature map. In some application scenarios, the dimensionality reduction module in the intermediate connection module is used to process the initial fused feature map and each advanced feature map to obtain the target feature map corresponding to the initial fused feature map and each advanced feature map. Specifically, processing the initial fused feature map using the dimensionality reduction module yields the corresponding dimensionality-reduced feature map, which is also the target feature map corresponding to the initial fused feature map. Similarly, processing each advanced feature map using the dimensionality reduction module yields the corresponding dimensionality-reduced feature map, which is also the target feature map corresponding to each advanced feature map.

[0104] It can be argued that introducing a dimensionality reduction module into the intermediate connection module in the above scheme can reduce the computational load of the feature extraction model.

[0105] In some embodiments, prior to step S14 above, the training method further includes the following steps:

[0106] First, the similarity between the target feature maps in each target feature map group is obtained. Each target feature map group includes a target feature map obtained from a student feature extraction model and a target feature map obtained from a teacher feature extraction model. The initial feature map sizes of the two target feature maps in the target feature map group are the same. Then, based on the similarity scores, the similarity loss is determined.

[0107] The target feature map set includes target feature maps obtained from a student feature extraction model and a target feature map obtained from a teacher feature extraction model, with each model's processing steps corresponding one-to-one. In some application scenarios, the correlation between the target feature maps obtained from the student feature extraction model and the teacher feature extraction model can be pre-established. For example, such as... Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the association relationship between target feature maps in one embodiment of the training method for the feature extraction model of this application. The association relationship between target feature maps can be two target feature maps obtained by each module in each target feature map group, including the teacher feature extraction model and the student feature extraction model, through the same processing steps.

[0108] In some application scenarios, the target feature maps in each target feature map group can be the target feature map P1 corresponding to the first initial feature map M1 in the student feature extraction model, and the target feature map P1 corresponding to the first initial feature map M1 in the teacher feature extraction model. In other application scenarios, the target feature maps in each target feature map group can be the target feature map P2 corresponding to a non-first initial feature map M2 in the student feature extraction model, and the target feature map P2 corresponding to a non-first initial feature map M2 in the teacher feature extraction model. The target feature maps output by the teacher feature extraction model can be P1, P2, and P3 corresponding to the teacher feature extraction model. Correspondingly, the target feature maps output by the student feature extraction model can be P1, P2, and P3 corresponding to the student feature extraction model. It can be understood that the processing steps for each target feature map P1 obtained by the student feature extraction model and the target feature map P1 obtained by the teacher feature extraction model correspond one-to-one. The target feature map P2 obtained from the student feature extraction model and the target feature map P2 obtained from the teacher feature extraction model are shown in the target feature map set. The processing steps for each target feature map P2 obtained from the two models correspond one-to-one. These will not be elaborated here.

[0109] This method combines self-distillation training with generative adversarial networks (GANs). Using the GAN, the target feature maps P1, P2, and P3 output by the intermediate connection modules of the teacher feature extraction model can be used as real samples, with the teacher feature extraction model acting as the discriminator. Using the GAN, the target feature maps P1, P2, and P3 output by the intermediate connection modules of the student feature extraction model can be used as fake samples, with the student feature extraction model acting as the discriminator. The student and teacher feature extraction models undergo self-adversarial distillation learning to train on a face mask dataset, ultimately resulting in a high-precision student feature extraction model. In some applications, this can be achieved by calculating the similarity loss between P1 corresponding to the teacher feature extraction model and P1 corresponding to the student feature extraction model in the target feature map group. In other applications, this can be achieved by calculating the similarity loss between P2 corresponding to the teacher feature extraction model and P2 corresponding to the student feature extraction model in the target feature map group. In still other applications, multiple similarity losses can be obtained by calculating the similarity between target feature maps in multiple target feature map groups. In some application scenarios, depending on the requirements, the similarity of multiple target feature map groups can be averaged to determine whether the average value reaches a preset value.

[0110] In some application scenarios, the target feature maps P1, P2, and P3 obtained from the teacher feature extraction model can be further fused to obtain richer fused features. These richer fused features can then be used as updated target feature maps P1, P2, and P3 from the teacher feature extraction model. The updated target feature maps P1, P2, and P3 from the teacher feature extraction model are used as real samples. The target feature maps P1, P2, and P3 from the student feature extraction model are used as fake samples. The above self-adversarial distillation training is performed, and the similarity between the target feature maps in each target feature map group is calculated.

[0111] Understandably, this self-adversarial distillation training method can make the target feature map output by the student feature extraction model closer to the target feature map output by the teacher feature extraction model. In other words, it indirectly transfers the features learned by the teacher feature extraction model for the training set to the student feature extraction model. Without increasing the number of parameters and computation in the student feature extraction model, it improves the features learned by the student feature extraction model, which is beneficial for feature extraction and feature fusion of the image to be processed in subsequent image processing.

[0112] Please see Figure 6 , Figure 6 This is a schematic flowchart of an embodiment of the image processing method of this application. Specifically, the image processing method may include the following steps:

[0113] Step S61: Obtain the image to be processed.

[0114] There are multiple images to be processed. Depending on the task requirements, the images to be processed can be images that contain the target object or images that do not contain the target object.

[0115] Step S62: The image to be processed is processed using the student feature extraction model in the image processing model to obtain multiple target feature maps corresponding to the image to be processed. The student feature extraction model is trained based on the above training method embodiment.

[0116] The optimized student feature extraction model obtained through the above feature extraction model is used as the student feature extraction model in the image processing model. In some application scenarios, the weight parameters of the high-precision student feature extraction model obtained through training are saved, and the student feature extraction model is further quantized, that is, the weight parameters stored in float16 are converted into weight parameters stored in int8. In some application scenarios, the input sample image of the student feature extraction model in the above feature extraction model is changed to the image to be processed. The other execution steps of the student feature extraction model in the label detection model are the same as those of the student feature extraction model in the feature extraction model, and will not be repeated here.

[0117] Step S63: Using the detection head model in the image processing model, perform image processing based on multiple target feature maps corresponding to the image to be processed to obtain the image processing result of the image to be processed.

[0118] The detection head model can be used for image processing tasks to extract the location and bounding box information of targets in an image and to identify target objects. There can be multiple image processing results. These results can represent the presence of a target object in the image being processed and related information about that object. In some applications, the detection head model can be a region convolutional neural network (R-CNN series models), a real-time image processing model (e.g., You Only Look Once, YOLO series models), an image processing model (e.g., Single Shot MultiBox Detector, SSD), a feature pyramid network-based image processing model (RetinaNet), and other models used to perform image processing tasks.

[0119] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0120] In some embodiments, step S63 may include the following steps: image processing includes target classification. First, multiple target feature maps are convolved using a first compact convolution to obtain multiple convolutional feature maps. Then, the multiple convolutional feature maps are input into a second compact convolution to obtain a classification result, which indicates whether a target object exists or does not exist in the image to be processed. Finally, the multiple convolutional feature maps are input into a third compact convolution to obtain the location of the target object in the image to be processed and the confidence level of that location.

[0121] Image processing can involve classifying objects within an image. A detection head model can perform first, second, and third compact convolutions, applying these convolutions multiple times to various target features in the image to obtain the classification result, the target object's location, and the confidence score at that location. The classification result indicates whether the target object exists or not in the image. The location confidence score represents the match value between a given bounding box and the target object. In some applications, different image processing tasks can be configured according to requirements.

[0122] For example, an image processing model is deployed on a security monitoring system to detect in real time whether people in a specific scene are wearing masks. The image to be processed can be a face image. Using the student feature extraction model in the image processing model, feature extraction and feature fusion are performed on the face image to obtain multiple target feature maps. The first compact convolution, second compact convolution, and third compact convolution can all be represented as a compact convolution operation with a kernel size of 1×1 (1×1Comconv). The first compact convolution is used to convolve multiple target feature maps to obtain multiple convolutional feature maps. In some application scenarios, after performing the first compact convolution on multiple target feature maps to obtain multiple convolutional feature maps, multiple convolution operations can be performed on the multiple convolutional feature maps to obtain updated convolutional feature maps. The image processing model can perform the second compact convolution and the third compact convolution on the multiple convolutional feature maps respectively. In some application scenarios, the image processing model can perform the second compact convolution on the convolutional feature maps to obtain classification results. The classification result corresponds to the classification task `cls` in image processing. The classification result indicates whether the target object in the face image is wearing a mask. In some applications, the image processing model can perform a third compact convolution on the convolutional feature map to obtain the target object location and its confidence score. The target object location and its confidence score correspond to the regression task `reg` in image processing. The target object location can be the position of the mask in the face image, and the confidence score can be the matching value between the mask and a given bounding box.

[0123] Understandably, in image processing tasks, regression (reg) and classification (cls) are mutually exclusive. This application utilizes a first compact convolution, a second compact convolution, and a third compact convolution to process the regression and classification tasks separately, thereby optimizing the image processing task. The image processing model in this application significantly reduces the number of parameters and complexity of the model while maintaining algorithm accuracy.

[0124] To better understand the feature extraction model provided in this solution, please refer to [link / reference]. Figure 7 . Figure 7 This is a schematic diagram of the structure of a fractal network, illustrating an embodiment of the training method for the feature extraction model of this application.

[0125] A feature extraction model can include multiple backbone modules and multiple intermediate connection modules. The model can include a student feature extraction model and a teacher feature extraction model. The student feature extraction model can include one backbone module and one intermediate connection module. The teacher feature extraction model can also include one backbone module and one intermediate connection module.

[0126] In some application scenarios, the backbone modules of a student feature extraction model may include a preprocessing module and a channel transformation module. The channel transformation module may include multiple cascaded channel transformation modules and multiple mapping modules. Each channel transformation module includes a first channel transformation submodule with a stride of 2 and a second channel transformation submodule with a stride of 1. Specifically, the first channel transformation submodule within a single channel transformation module may include a first separation module, a first processing module, a first convolution module, a first transformation module, a second convolution module, a second transformation module, a first cascade module, and a first shuffling module. Similarly, the second channel transformation submodule within a single channel transformation module may include a second separation module, a second processing module, a third convolution module, a third transformation module, a second cascade module, and a second shuffling module. The intermediate linking modules in the student feature extraction model may include multiple dimensionality reduction modules, multiple third-level cascade modules, multiple downsampling modules, and multiple fourth-level cascade modules. A single dimensionality reduction module may include a first channel processing submodule, a depthwise convolution submodule, a second channel processing submodule, and a concatenation submodule.

[0127] In some application scenarios, the student feature extraction model and the teacher feature extraction model have the same structure. The complexity of the student feature extraction model is lower than that of the teacher feature extraction model. Knowledge distillation is performed between the teacher and student feature extraction models. Here, the specific structure of the backbone module and intermediate connection module in the teacher feature extraction model is the same as that in the student feature extraction model, and will not be repeated here.

[0128] The input to the feature extraction model can be a sample image. The student feature extraction model within the feature extraction model is used to extract and fuse features from the sample image, resulting in target feature maps corresponding to the student feature extraction model. Similarly, the teacher feature extraction model within the feature extraction model is used to extract and fuse features from the sample image, resulting in target feature maps corresponding to the teacher feature extraction model.

[0129] In some application scenarios, using the student feature extraction model within a feature extraction model to extract and fuse features from sample images to obtain the target feature maps corresponding to the student feature extraction model can be achieved through the following steps: First, the backbone module of the student feature extraction model extracts features from the sample image to obtain initial feature maps. Second, the intermediate connection module of the student feature extraction model fuses the initial feature maps to obtain the target feature maps.

[0130] First, the backbone module of the student feature extraction model is used to extract features from the sample image, obtaining initial feature maps. For example, the preprocessing module in the backbone module preprocesses the sample image, obtaining a preprocessed feature map. The channel transformation module in the backbone module then processes the preprocessed feature map, obtaining initial feature maps. Specifically, the first channel transformation module in the channel transformation module processes the preprocessed feature map, obtaining the feature map output by the first channel transformation module. More specifically, the first channel transformation submodule in the first channel transformation module processes the preprocessed feature map, obtaining a first advanced feature map. The second channel transformation submodule in the first channel transformation module processes the first advanced feature map, obtaining the feature map output by the first channel transformation module. The first mapping module maps the feature map output by the first channel transformation module, obtaining the initial feature map corresponding to the first channel transformation module. Furthermore, the non-first channel transformation modules in the channel transformation module process the feature map output by the previous channel transformation module, obtaining feature maps corresponding to each non-first channel transformation module. Specifically, the first channel transformation submodule within the non-first channel transformation modules processes the feature map output by the previous channel transformation module to obtain a second-order feature map. The second-order feature map is then processed using the second channel transformation submodule within the non-first channel transformation modules to obtain the feature maps corresponding to each non-first channel transformation module. Finally, each non-first mapping module maps the feature maps corresponding to each non-first channel transformation module to obtain the initial feature map corresponding to each non-first channel transformation module. It is understood that the number of channels in the initial feature map corresponding to each channel transformation module is less than the number of channels in the feature map output by each channel transformation module.

[0131] Next, the intermediate connection module in the student feature extraction model is used to fuse the features of each initial feature map to obtain each target feature map. For example, one of the initial feature maps is used as the starting feature map, and the feature maps other than the starting feature map are used as intermediate feature maps. The starting feature map only has one initial feature map of adjacent dimensions. It can be understood that the starting feature map here can be the initial feature map corresponding to the first channel transformation module, or it can be the initial feature map corresponding to the last non-first channel transformation module. This application uses the initial feature map corresponding to the first channel transformation module as an example. The intermediate connection module in the student feature extraction model is used to fuse the features of the starting feature map and the fused feature maps corresponding to each intermediate feature map to obtain each target feature map.

[0132] The intermediate feature maps are processed using the dimensionality reduction module and the third concatenation module within the intermediate connection module to obtain the corresponding fused feature maps. Specifically, for the first intermediate feature map, the dimensionality reduction module processes the starting feature map to obtain a dimensionality-reduced feature map. The third concatenation module then concatenates these dimensionality-reduced feature maps to obtain the corresponding fused feature map. For non-first intermediate feature maps, the dimensionality reduction module processes the fused feature map corresponding to the preceding adjacent intermediate feature map to obtain dimensionality-reduced feature maps for each non-first intermediate feature map. The third concatenation module then concatenates these dimensionality-reduced feature maps to obtain the corresponding fused feature maps for each non-first intermediate feature map.

[0133] The downsampling module and the fourth concatenation module in the intermediate connection module process the fused feature map and the starting feature map corresponding to each intermediate feature map to obtain the advanced feature map corresponding to each intermediate feature map and the advanced feature map corresponding to the starting feature map. For example, the fused feature map corresponding to the last intermediate feature map is used as the starting fused feature map, and the other fused feature maps are used as intermediate fused feature maps. Specifically, for the first intermediate fused feature map, the downsampling module performs downsampling processing on the first intermediate fused feature map to obtain the downsampled feature map corresponding to the first intermediate fused feature map. The fourth concatenation module concatenates the downsampled feature map corresponding to the first intermediate fused feature map with the first intermediate fused feature map to obtain the advanced feature map corresponding to the first intermediate fused feature map. For each non-first intermediate fused feature map, the downsampling module performs downsampling processing on the preceding adjacent intermediate fused feature map to obtain the downsampled feature map corresponding to each non-first intermediate fused feature map. The fourth concatenation module concatenates the downsampled feature map corresponding to each non-first intermediate fused feature map with each non-first intermediate fused feature map to obtain the advanced feature map corresponding to each non-first intermediate fused feature map. Specifically, for the initial feature map, the last advanced feature map among the advanced feature maps corresponding to each non-first intermediate fused feature map is input into the downsampling module for downsampling processing to obtain the downsampled feature map corresponding to the last advanced feature map. The fourth concatenation module is then used to concatenate the downsampled feature map corresponding to the last advanced feature map and the initial feature map to obtain the advanced feature map corresponding to the initial feature map.

[0134] The dimensionality reduction module in the intermediate connection module processes the initial fused feature map and each advanced feature map to obtain the target feature map corresponding to the initial fused feature map and each advanced feature map. Specifically, processing the initial fused feature map using the dimensionality reduction module yields the corresponding dimensionality-reduced feature map, which is also the target feature map. Similarly, processing each advanced feature map using the dimensionality reduction module yields the corresponding dimensionality-reduced feature map, which is also the target feature map.

[0135] In some application scenarios, knowledge distillation is performed between the teacher feature extraction model and the student feature extraction model. The specific structure of the backbone module and intermediate connection module in the teacher feature extraction model is the same as that in the student feature extraction model. The specific execution methods of each module in the backbone module and intermediate connection module of the teacher feature extraction model are also the same as those in the student feature extraction model. Therefore, the specific execution steps for using the teacher feature extraction model to extract and fuse features from sample images to obtain the target feature maps corresponding to the teacher feature extraction model are not elaborated here.

[0136] To better understand the image processing model provided in this solution, please refer to [link / reference]. Figure 8 . Figure 8 This is a schematic diagram of the structure of a fractal network shown in an embodiment of the image processing method of this application.

[0137] The image processing model can include a student feature extraction model and a detection head model. The detection head model in the image processing model can include a first compact convolutional module, a second compact convolutional module, and a third compact convolutional module. The specific structure of the student feature extraction model in the image processing model is as described above, and will not be repeated here.

[0138] By utilizing the similarity loss between multiple target feature maps obtained from the student feature extraction model and multiple target feature maps obtained from the teacher feature extraction model, the parameters in the student feature extraction model are adjusted. The trained student feature extraction model can then perform feature extraction and feature fusion on the image to be processed in subsequent image processing. In some application scenarios, the feature extraction and feature fusion performed by the trained student feature extraction model in subsequent image processing can be handled within the image processing model itself.

[0139] The input to an image processing model can be the image to be processed. In some application scenarios, the student feature extraction model in the image processing model is used to extract and fuse features from the image to be processed, resulting in multiple target feature maps corresponding to the image to be processed in the student feature extraction model. Here, the execution steps of the student feature extraction model in the image processing model to extract and fuse features from the image to be processed, resulting in multiple target feature maps corresponding to the image to be processed, are consistent with those of the student feature extraction model in the feature extraction model described above. It can be understood that the input sample image of the student feature extraction model in the above feature extraction model is changed to the image to be processed, and the other execution steps are consistent with those of the student feature extraction model in the feature extraction model, which will not be elaborated here.

[0140] In some application scenarios, a detection head model in an image processing model is used to process multiple target feature maps to obtain the image processing result of the image to be processed. The image processing result can include classification results, location, and confidence scores. A first compact convolutional module is used to convolve multiple target feature maps to obtain multiple convolutional feature maps. A second compact convolutional module is used to convolve multiple convolutional feature maps to obtain the classification result. The classification result indicates whether a target object exists or does not exist in the image to be processed. A third compact convolutional module is used to convolve multiple convolutional feature maps to obtain the location and confidence score in the image processing result. The location and confidence score in the image processing result represent the location of the target object in the image to be processed and the confidence score of that location.

[0141] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0142] Please see Figure 9 , Figure 9This is a schematic diagram of an embodiment of the training device for the feature extraction model of this application. The training device 90 for the feature extraction model includes an acquisition module 91, an extraction module 92, a fusion module 93, and an adjustment module 94. The acquisition module 91 is used to acquire sample images; the extraction module 92 is used to extract features from the sample images using both the student feature extraction model and the teacher feature extraction model, respectively, to obtain multiple initial feature maps of different sizes, wherein the complexity of the student feature extraction model is less than that of the teacher feature extraction model; the fusion module 93 is used to fuse the multiple initial feature maps using both the student feature extraction model and the teacher feature extraction model, respectively, to obtain target feature maps corresponding to each initial feature map, wherein the number of target feature maps is the same as the number of initial feature maps; the adjustment module 94 is used to adjust the parameters in the student feature extraction model using the similarity loss between the multiple target feature maps obtained by the student extraction model and the multiple target feature maps obtained by the teacher feature extraction model, so that the trained student feature extraction model can perform feature extraction and feature fusion on the image to be processed in subsequent image processing.

[0143] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0144] Please see Figure 10 , Figure 10 This is a schematic diagram of an embodiment of the image processing apparatus of this application. The training device 100 for the feature extraction model includes an image acquisition module 101, a first image processing module 102, and a second image processing module 103. The image acquisition module 101 is used to acquire the image to be processed; the first image processing module 102 is used to process the image to be processed using a student feature extraction model in the image processing model to obtain multiple target feature maps corresponding to the image to be processed, wherein the student feature extraction model is trained based on the above-described training method; the second image processing module 103 is used to perform image processing based on the multiple target feature maps corresponding to the image to be processed using a detection head model in the image processing model to obtain the image processing result of the image to be processed.

[0145] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0146] For details on the functions performed by each module, please refer to the training methods of the feature extraction model and / or the image processing methods; these will not be elaborated upon here.

[0147] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 110 includes a memory 111 and a processor 112. The processor 112 is used to execute program instructions stored in the memory 111 to implement the steps in the above-described embodiment of the feature extraction model training method and / or image processing method. In a specific implementation scenario, the electronic device 110 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 110 may also include mobile devices such as laptops and tablets, which are not limited here.

[0148] Specifically, processor 112 controls itself and memory 111 to implement the steps in the above-described feature extraction model training method and / or image processing method embodiments. Processor 112 may also be referred to as a CPU (Central Processing Unit). Processor 112 may be an integrated circuit chip with signal processing capabilities. Processor 112 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 112 may be implemented using integrated circuit chips.

[0149] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0150] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 120 stores program instructions 1201 thereon. When the program instructions 1201 are executed by a processor, they implement the steps in the training method and / or image processing method embodiments of any of the above-described feature extraction models.

[0151] The above scheme trains the student feature extraction model using the teacher feature extraction model. This results in a high similarity between the feature maps obtained by the trained student model and those output by the teacher model. Furthermore, the student model has lower complexity, enabling it to achieve higher feature extraction accuracy and be applied to resource-constrained devices. Additionally, this scheme extracts feature maps of different sizes from both models and then fuses them to obtain the corresponding target feature maps. The similarity loss is determined by comparing the similarity between multiple target feature maps output by the two models, which is more accurate than using the similarity between a single feature map. This results in more accurate outputs from the trained student feature extraction model.

[0152] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0153] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0155] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0156] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A training method for a feature extraction model, characterized in that, include: Acquire sample images; The sample image is used to extract features using a student feature extraction model and a teacher feature extraction model respectively, resulting in multiple initial feature maps of different sizes. The complexity of the student feature extraction model is less than that of the teacher feature extraction model. The method involves fusing features from multiple initial feature maps using student feature extraction models and teacher feature extraction models to obtain target feature maps corresponding to each initial feature map. This includes: using one of the initial feature maps as a starting feature map, and using the feature maps other than the starting feature map as intermediate feature maps, where the starting feature map has only one initial feature map of adjacent size; for each intermediate feature map, fusing the intermediate feature map with the fused feature map corresponding to the previous adjacent starting feature map or the previous adjacent intermediate feature map to obtain a fused feature map corresponding to each intermediate feature map; and fusing the starting feature map with the fused feature maps corresponding to each intermediate feature map to obtain target feature maps, where the number of target feature maps is the same as the number of initial feature maps. By using the similarity loss between multiple target feature maps obtained by the student feature extraction model and multiple target feature maps obtained by the teacher feature extraction model, the parameters in the student feature extraction model are adjusted. After training, the student feature extraction model can perform feature extraction and feature fusion on the image to be processed in subsequent image processing.

2. The training method according to claim 1, characterized in that, Before adjusting the parameters in the student feature extraction model using the similarity loss between the multiple target feature maps obtained by the student feature extraction model and the multiple target feature maps obtained by the teacher feature extraction model, the training method further includes: The similarity between the target feature maps in each target feature map group is obtained. Each target feature map group includes a target feature map obtained by the student feature extraction model and a target feature map obtained by the teacher feature extraction model. The initial feature map sizes of the two target feature maps in the target feature map group are the same. Based on each of the aforementioned similarities, the similarity loss is determined.

3. The method according to claim 1, characterized in that, For each of the intermediate feature maps, feature fusion is performed between the intermediate feature map and the fusion feature map corresponding to the previous adjacent starting feature map or the previous adjacent intermediate feature map to obtain the fusion feature map corresponding to each of the intermediate feature maps, including: For the first intermediate feature map, the initial feature map after the dimensionality reduction module is concatenated to obtain the fused feature map corresponding to the first intermediate feature map; For each non-first intermediate feature map, the fused feature map corresponding to the previous adjacent intermediate feature map after the dimensionality reduction module is concatenated to obtain the fused feature map corresponding to each non-first intermediate feature map.

4. The training method according to claim 1, characterized in that, The step of fusing features between the initial feature map and the fused feature maps corresponding to each of the intermediate feature maps to obtain each of the target feature maps includes: The fused feature map corresponding to the last intermediate feature map is used as the starting fused feature map, and the other fused feature maps are used as intermediate fused feature maps. For each of the intermediate fusion feature maps, the intermediate fusion feature map is fused with the preceding adjacent starting fusion feature map or the advanced feature map corresponding to the preceding adjacent intermediate fusion feature map to obtain the advanced feature map corresponding to each of the intermediate fusion feature maps. The initial feature map and the last advanced feature map are fused to obtain the advanced feature map corresponding to the initial feature map; Based on the initial fusion feature map and each of the advanced feature maps, the target feature map corresponding to the initial fusion feature map and the target feature map corresponding to each of the advanced feature maps are obtained.

5. The training method according to claim 4, characterized in that, For each of the intermediate fused feature maps, feature fusion is performed between the intermediate fused feature map and the advanced feature map corresponding to the previous adjacent initial fused feature map or the previous adjacent intermediate fused feature map to obtain the advanced feature map corresponding to each of the intermediate fused feature maps, including: For the first intermediate fused feature map, the downsampled initial fused feature map is concatenated to obtain the advanced feature map corresponding to the first intermediate fused feature map; For each non-first intermediate fusion feature map, the advanced feature map corresponding to the previous adjacent intermediate fusion feature map after downsampling is concatenated to obtain the advanced feature map corresponding to each non-first intermediate fusion feature map. The step of fusing the initial feature map with the last advanced feature map to obtain the advanced feature map corresponding to the initial feature map includes: For the initial feature map, the last advanced feature map after downsampling is concatenated to obtain the advanced feature map corresponding to the initial feature map.

6. The training method according to claim 4, characterized in that, The step of obtaining the target feature map corresponding to the initial fusion feature map and the target feature maps corresponding to each of the advanced feature maps based on the initial fusion feature map includes: The initial fusion feature map and each of the advanced feature maps are processed by a dimensionality reduction module to obtain the target feature map corresponding to the initial fusion feature map and the target feature map corresponding to each of the advanced feature maps.

7. The training method according to claim 1, characterized in that, The student feature extraction model and the teacher feature extraction model both include a channel transformation module, which includes multiple cascaded channel transformation modules and multiple mapping modules. Each initial feature map includes an initial feature map corresponding to each of the channel transformation modules. The process of extracting features from the sample image using the student feature extraction model and the teacher feature extraction model respectively yields multiple initial feature maps of different sizes, including: The sample images are preprocessed using the student feature extraction model and the teacher feature extraction model respectively to obtain preprocessed feature maps; For the first channel transformation module, the preprocessed feature map is processed to obtain the feature map output by the first channel transformation module; For each non-first channel transformation module, the feature map output by the previous channel transformation module is processed to obtain the feature map corresponding to each non-first channel transformation module. The feature maps output by each of the channel transformation modules are mapped using the mapping modules to obtain the initial feature maps corresponding to each of the channel transformation modules. The number of channels in the initial feature maps corresponding to each of the channel transformation modules is less than the number of channels in the feature maps output by each of the channel transformation modules.

8. The training method according to claim 7, characterized in that, Each of the aforementioned channel transformation modules includes a first channel transformation submodule with a step size of 2 and a second channel transformation submodule with a step size of 1. For the first channel transformation module, the preprocessed feature map is processed to obtain the feature map output by the first channel transformation module, including: The preprocessed feature map is processed using the first channel transformation submodule to obtain a first advanced feature map; The first advanced feature map is processed using the second channel transformation submodule to obtain the feature map output by the first channel transformation module; For each non-first channel transformation module, the feature map output by the previous channel transformation module is processed to obtain the feature map corresponding to each non-first channel transformation module, including: The first channel transformation submodule is used to process the feature map output by the previous channel transformation module to obtain the second advanced feature map. The second channel transformation submodule is used to process the second advanced feature map to obtain the feature map output by the non-first channel transformation module.

9. An image processing method, characterized in that, include: Obtain the image to be processed; The image to be processed is processed using a student feature extraction model in an image processing model to obtain multiple target feature maps corresponding to the image to be processed. The student feature extraction model is trained based on the training method described in any one of claims 1 to 8. Using the detection head model in the image processing model, image processing is performed based on multiple target feature maps corresponding to the image to be processed, to obtain the image processing result of the image to be processed.

10. The image processing method according to claim 9, characterized in that, The image processing includes target classification. The image processing model utilizes a detection head model to perform image processing based on multiple target feature maps corresponding to the image to be processed, obtaining the image processing result of the image to be processed, including: The multiple target feature maps are convolved using a first compact convolution to obtain multiple convolutional feature maps; The multiple convolutional feature maps are input into a second compact convolution to obtain a classification result, which is used to indicate whether a target object exists or does not exist in the image to be processed. Furthermore, the multiple convolutional feature maps are input into a third compact convolution to obtain the position of the target object in the image to be processed and the confidence level of the position.

11. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, include: The system contains a program file that, when executed by a processor, is used to implement the method as described in any one of claims 1-10.

Images