Image classification methods, training methods, devices and equipment based on classification models

Abstract

This application provides an image classification method, training method, apparatus, and device based on a classification model, comprising: acquiring an image to be processed; inputting the image to be processed into a preset classification model; and acquiring a convolutional input vector from the input layer of the classification model; compressing and extracting features from the convolutional input vector based on a downsampling module in the classification model to obtain a downsampling output vector; extracting features from the downsampling output vector based on a normal module in the classification model to obtain an image feature vector; and classifying the image feature vector based on a fully connected layer in the classification model to obtain the image category of the image to be processed. This application reduces the amount of computational data required for feature extraction from the image to be processed, improves the computational speed of feature extraction, and ensures the comprehensiveness and accuracy of feature extraction.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image classification method, training method, apparatus and device based on a classification model. Background Technology

[0002] With the development of concepts such as intelligent driving and smart cities, more and more hardware devices are classifying images through built-in high-efficiency neural network processing units (GPUs, NPUs, etc.). Current classification processing usually uses convolutional neural networks with convolutional layers, intermediate layers, and fully connected layers.

[0003] However, the inventors discovered that current image classification methods based on convolutional neural networks typically require feature extraction of all pixels in an image to obtain feature vectors, and then classifying the image based on these feature vectors. This results in hardware devices running convolutional neural networks needing to perform a huge amount of computation, leading to low image classification speed and a heavy computational burden. Summary of the Invention

[0004] This application provides an image classification method, training method, apparatus, and device based on a classification model, which solves the problem that current image classification methods require a large number of calculations, resulting in low image classification speed and high computational burden.

[0005] Firstly, this application provides an image classification method based on a classification model, including:

[0006] The process involves acquiring an image to be processed, inputting the image to be processed into a preset classification model, and obtaining a convolutional input vector from the input layer of the classification model; wherein the classification model is a convolutional neural network model; and the convolutional input vector is used to represent the image to be processed.

[0007] The downsampling module in the classification model compresses and extracts features from the convolutional input vector to obtain a downsampling output vector; wherein, the downsampling output vector is a feature vector for preliminary feature extraction of the compressed image to be processed, and a representation of the abstracted image to be processed.

[0008] Based on the common module in the classification model, feature extraction is performed on the downsampled output vector to obtain the image feature vector; wherein, the image feature vector is used to characterize the shape and contour of the object in the image to be processed;

[0009] The image feature vector is classified based on the fully connected layer in the classification model to obtain the image category of the image to be processed; wherein the image category reflects the type or kind of object in the image to be processed.

[0010] In the above scheme, the step of compressing and extracting features from the convolutional input vector based on the downsampling module in the classification model to obtain the downsampled output vector includes:

[0011] The convolutional input vector is obtained based on the sampling input layer in the downsampling module;

[0012] Based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and the compressed convolutional input vector is used to extract features to obtain a sampling feature vector; wherein, the sampling feature vector is a feature vector for preliminary feature extraction of the compressed image to be processed;

[0013] Based on the pooling module in the downsampling module, the convolutional input vector is pooled to obtain a pooling vector; wherein, the pooling vector represents the abstracted image to be processed;

[0014] The sampled feature vector and the pooling vector are combined to obtain the downsampled output vector.

[0015] In the above scheme, the step of extracting features from the downsampled output vector based on the ordinary module in the classification model to obtain the image feature vector includes:

[0016] The downsampled output vector is obtained based on the ordinary input layer in the ordinary module.

[0017] The downsampled output vector is split based on the ordinary splitting and compression module in the ordinary module, and features are extracted and merged on the split downsampled output vector with an ordinary stride to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to characterize the compressed downsampled output vector;

[0018] The ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and the split ordinary compressed convolutional data is subjected to feature extraction and merging with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is the feature vector for preliminary feature extraction of the compressed downsampled output vector;

[0019] The image feature vector is obtained by summing the downsampled output vector from the ordinary input layer and the ordinary feature vector.

[0020] Secondly, this application provides a training method for a classification model applied to image classification, comprising:

[0021] A training image is obtained, the training image is input into a preset initial model, and a convolutional input vector is obtained from the input layer of the initial model; wherein, the initial model is a convolutional neural network model; the convolutional input vector is used to represent the training image;

[0022] The convolutional input vector is compressed and its features are extracted based on the downsampling module in the initial model to obtain a downsampling training vector; wherein, the downsampling training vector is a feature vector for preliminary feature extraction of the compressed image to be trained, and a representation of the abstracted image to be trained;

[0023] Based on the common modules in the initial model, feature extraction is performed on the downsampled training vector to obtain the image training vector; wherein, the image training vector is used to characterize the shape and contour of the object in the image to be trained;

[0024] The image training vectors are classified based on the fully connected layers in the initial model to obtain the image category of the image to be trained; wherein, the image category reflects the type or category of the object in the image to be trained;

[0025] If the difference between the image category of the image to be trained and the training label of the image to be trained is less than a preset training threshold, then the initial model is set as the classification model.

[0026] In the above scheme, the step of compressing and extracting features from the convolutional input vector based on the downsampling module in the initial model to obtain the downsampled training vector includes:

[0027] The convolutional input vector is obtained based on the sampling input layer in the downsampling module;

[0028] Based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and the compressed convolutional input vector is used to extract features to obtain a sampling feature vector; wherein, the sampling feature vector is a feature vector for preliminary feature extraction of the compressed image to be trained;

[0029] Based on the pooling module in the downsampling module, the convolutional input vector is pooled to obtain a pooling vector; wherein, the pooling vector represents the abstracted image to be trained;

[0030] The sampled feature vector and the pooling vector are combined to obtain the downsampled training vector.

[0031] In the above scheme, the step of extracting features from the downsampled training vector based on the ordinary module in the initial model to obtain the image training vector includes:

[0032] The downsampled training vector is obtained based on the ordinary input layer in the ordinary module.

[0033] The downsampled training vector is split based on the ordinary splitting and compression module in the ordinary module, and features are extracted and merged on the split downsampled training vector with an ordinary stride to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to represent the compressed downsampled training vector.

[0034] The ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and the split ordinary compressed convolutional data is subjected to feature extraction and merging with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is a feature vector for preliminary feature extraction of the compressed downsampled training vector.

[0035] The image training vector is obtained by summing the downsampled training vectors from the ordinary input layer and the ordinary feature vectors.

[0036] Thirdly, this application provides an image classification device based on a classification model, comprising:

[0037] An input module is used to acquire an image to be processed, input the image to be processed into a preset classification model, and acquire a convolutional input vector from the input layer of the classification model; wherein, the classification model is a convolutional neural network model; and the convolutional input vector is used to represent the image to be processed.

[0038] The first processing module is used to compress and extract features from the convolutional input vector based on the downsampling module in the classification model to obtain a downsampled output vector; wherein, the downsampled output vector is a feature vector for preliminary feature extraction of the compressed image to be processed, and a representation of the abstracted image to be processed;

[0039] The second processing module is used to extract features from the downsampled output vector based on the ordinary module in the classification model to obtain the image feature vector; wherein, the image feature vector is used to characterize the shape and contour of the object in the image to be processed;

[0040] The third processing module is used to classify the image feature vector based on the fully connected layer in the classification model to obtain the image category of the image to be processed; wherein the image category reflects the type or kind of object in the image to be processed.

[0041] Fourthly, this application provides a training apparatus for a classification model applied to image classification, comprising:

[0042] The training input module is used to acquire the image to be trained, input the image to be trained into a preset initial model, and acquire a convolutional input vector from the input layer of the initial model; wherein, the initial model is a convolutional neural network model; and the convolutional input vector is used to represent the image to be trained.

[0043] The first training module is used to compress and extract features from the convolutional input vector based on the downsampling module in the initial model to obtain a downsampled training vector; wherein, the downsampled training vector is a feature vector for preliminary feature extraction of the compressed image to be trained, and a representation of the abstracted image to be trained;

[0044] The second training module is used to extract features from the downsampled training vector based on the ordinary module in the initial model to obtain the image training vector; wherein, the image feature vector is used to characterize the shape and contour of the object in the image to be trained;

[0045] The third training module is used to classify the image feature vector based on the fully connected layer in the initial model to obtain the image category of the image to be trained; wherein, the image category reflects the type or category of the object in the image to be trained;

[0046] The model generation module is used to set the initial model as the classification model if the difference between the image category of the image to be trained and the training label of the image to be trained is less than a preset training threshold.

[0047] Fifthly, this application provides a computer device, including: a processor and a memory communicatively connected to the processor;

[0048] The memory stores computer-executed instructions;

[0049] The processor executes computer execution instructions stored in the memory to implement the method described above.

[0050] Sixthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described method.

[0051] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0052] This application provides an image classification method, training method, apparatus, and device based on a classification model. The application compresses the received feature vectors using a downsampling module to reduce the amount of data in the feature vectors, thereby eliminating redundant or invalid data. This reduces the computational data required for feature extraction in subsequent ordinary modules and improves the speed of feature extraction. By setting a feature extraction function in the downsampling module to extract features from the compressed convolutional input vector, the application assists ordinary modules in recognizing objects in the convolutional input vector, thereby improving the computational accuracy of the subsequently generated neural network model.

[0053] The received feature vector is extracted by a normal module to obtain feature vectors that can characterize the coverage area and contour of the object in the downsampled output vector. Since the downsampled output vector is fully computed, the comprehensiveness and accuracy of feature extraction are ensured. Therefore, the obtained image feature vector contains the downsampled output vector and the feature vectors that characterize the contour and coverage area of ​​the object in the downsampled output vector, thus ensuring the comprehensiveness of the output image feature vector.

[0054] By using a fully connected layer to classify the image feature vectors that describe the shape and contour of an object, the accuracy of image classification is ensured. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0056] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0057] Figure 2 A flowchart of Embodiment 1 of an image classification method based on a classification model provided in this application;

[0058] Figure 3 A flowchart of Embodiment 2 of a training method for a classification model applied to image classification provided in this application;

[0059] Figure 4 A schematic diagram of the program module of an image classification device based on a classification model provided in this application, according to embodiment 3;

[0060] Figure 5 A schematic diagram of the program modules of Embodiment 4 of a training device for a classification model applied to image classification, provided in this application.

[0061] Figure 6This is a schematic diagram of the hardware structure of the computer device in the computer device of the present invention.

[0062] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0063] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0064] Please see Figure 1 The specific application scenario of this application is as follows:

[0065] The classification model in this application includes: a regular module 2 and a downsampling module 3, as well as an input layer 4 and a fully connected layer 5; wherein, M downsampling modules 3 and N regular modules 2 are sequentially connected to obtain a stage combination 6, and L stage combinations 6 are sequentially connected to obtain a neural network model 7, where M, N, and K are natural numbers. , , The resulting neural network model is as follows: Figure 1 As shown.

[0066] Specifically, the ordinary module 2 includes: an ordinary input layer 21, an ordinary splitting and compression module 22, and an ordinary expansion and merging module 23;

[0067] The ordinary splitting and compression module 22 includes: ordinary target splitting module 221, ordinary compression convolution module 222, ordinary compression merging module 223 and ordinary merging convolution module 224.

[0068] The ordinary dilation and merging module 23 includes: ordinary convolution splitting module 231, ordinary dilation convolution module 232 and ordinary dilation and merging module 233.

[0069] The downsampling module 3 includes: a sampling input layer 31, a feature extraction module 32 and a pooling module 33 connected to the sampling input layer 31.

[0070] The feature module 32 includes: a sampling splitting and compression module 321 and a sampling dilation and merging module 322;

[0071] Pooling module 33 includes: average pooling module 331 and pooling convolution kernel 332.

[0072] The sampling splitting and compression module 321 includes: a sampling target splitting module 3211, a sampling compression convolution module 3212, a sampling compression merging module 3213, and a sampling merging convolution module 3214;

[0073] The sampling dilation and merging module 322 includes: a sampling convolution splitting module 3221, a sampling dilation convolution module 3222, and a sampling dilation and merging module 3223.

[0074] Based on factors such as accuracy requirements, data computation volume, server computing power, and result accuracy requirements, neural network models with different numbers of downsampling modules and / or ordinary modules are constructed, namely: large version models and small version models.

[0075] The construction method of the large-version model includes: sequentially connecting M downsampling modules with N ordinary modules to obtain stage combinations, and sequentially connecting K stage combinations to obtain a neural network model, including:

[0076] Connecting M1 downsampling modules and N1 ordinary modules sequentially yields the first-stage combination; connecting K1 first-stage combinations sequentially yields the larger version model;

[0077] The method for constructing the minor version model includes: sequentially connecting M2 downsampling modules and N2 ordinary modules to obtain the second-stage combination; sequentially connecting K2 second-stage combinations to obtain the minor version model; where M1, M2, N1, N2, K1, and K2 are natural numbers; M1≥M2, N1≥N2, and K1≥K2.

[0078] For example, the settings for the downsampling module and the normal module are shown in the table below:

[0079]

[0080] Wherein, conv1 is the model input layer, Stage2 is the second stage, Stage2_0 is the downsampling module in the second stage, Stage2_n is the regular module in the second stage, Stage2_0-repeat is the number of downsampling modules in the second stage, and Stage2_n is the number of regular modules in the second stage; and so on, to obtain Stage3 and Stage4; Conv5 is the model output layer, pool is the pooling layer, and FC is the fully connected layer.

[0081] Ultimately, the major version model GogglesNet-L consists of: the model input layer, the second stage, the third stage, the fourth stage, the model output layer, the pooling layer, and the fully connected layer, connected in sequence; the second stage consists of one downsampling module and seven regular modules connected in sequence; the third stage consists of one downsampling module and thirteen regular modules connected in sequence; and the fourth stage consists of one downsampling module and two regular modules connected in sequence.

[0082] The minor version of the model GogglesNet-S consists of: a model input layer, a second stage, a third stage, a fourth stage, a model output layer, a pooling layer, and a fully connected layer, connected in sequence; the second stage consists of one downsampling module and three regular modules connected in sequence; the third stage consists of one downsampling module and six regular modules connected in sequence; and the fourth stage consists of one downsampling module and two regular modules connected in sequence.

[0083] The advantages of the obtained major version model GogglesNet-L and minor version model GogglesNet-S compared with the current mainstream mobile networks are shown in the table below:

[0084]

[0085] Compared to several well-known mobile networks, our proposed method has certain advantages on various hardware devices such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), and NPU (Neural Network Processing Unit) for classification tasks with similar accuracy. Especially on NPU, the larger version of the model is 30% faster than RegNetX, and the smaller version is 88% faster than MobileNet.

[0086] The technical solution of this application and how the technical solution of this application solves the problems of the prior art will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0087] Example 1:

[0088] Please see Figure 2 This application provides an image classification method based on a classification model, including:

[0089] S101: Obtain the image to be processed, input the image to be processed into a preset classification model, and obtain the convolutional input vector from the input layer of the classification model; wherein, the classification model is a convolutional neural network model; the convolutional input vector is used to represent the image to be processed.

[0090] In this step, the input layer is used to preprocess or convolve the input image to be processed, and obtain the convolutional input vector representing the image to be processed.

[0091] Convolutional Neural Networks (CNNs) are a type of neural network that uses convolutional kernels to extract important information from data, employs image padding strategies to preserve as much information as possible from the original data, and uses pooling and other strategies to avoid overfitting (a great many strategies have been proposed to avoid overfitting).

[0092] The classification model in this application is a convolutional neural network adapted for the Winograd algorithm. Originating in 1980, the Winograd algorithm, proposed by Shmuel Winograd, is an algorithm designed to reduce the computational cost of FIR filters. It states that for an FIR filter with *n* outputs and *n* parameters, only *n* multiplications are needed instead of *n* multiplications. The Winograd algorithm is suitable for convolutional kernels with a kernel size of 3×3 or larger. Therefore, by setting the kernel size for feature extraction in both the ordinary and downsampling modules to K×K, where K is a natural number and K≥3, the ordinary and downsampling modules are better adapted to the Winograd algorithm, thereby improving the running speed of the neural network model.

[0093] The Winograd algorithm is a deep learning convolution optimization algorithm. Therefore, neural network models built based on the Winograd algorithm will have higher running speed while ensuring comprehensiveness and accuracy of computation. This allows the obtained neural network model to ensure that mobile hardware devices can run the model while also ensuring the accuracy of neural network computation.

[0094] S102: Based on the downsampling module in the classification model, the convolutional input vector is compressed and features are extracted to obtain the downsampled output vector; where the downsampled output vector is the feature vector for preliminary feature extraction of the compressed image to be processed, and the image to be processed after abstraction.

[0095] In this step, the received feature vector is compressed by the downsampling module to reduce the amount of data in the feature vector, thereby eliminating redundant or invalid data in the feature vector. This reduces the amount of computational data required for the subsequent ordinary module to extract features from the image to be processed, and improves the computational speed of feature extraction. By setting the feature extraction function in the downsampling module to extract features from the compressed convolution input vector, the ordinary module is assisted in recognizing objects in the convolution input vector, so as to improve the computational accuracy of the subsequently generated neural network model.

[0096] In a preferred embodiment, the convolutional input vector is compressed and features are extracted based on the downsampling module in the classification model to obtain a downsampled output vector, including:

[0097] The convolutional input vector is obtained based on the sampling input layer in the downsampling module;

[0098] Based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and the compressed convolutional input vector is used to extract features to obtain the sampling feature vector; where the sampling feature vector is the feature vector for preliminary feature extraction of the compressed image to be processed.

[0099] Based on the pooling module in the downsampling module, the convolution input vector is pooled to obtain a pooling vector; where the pooling vector represents the abstracted image to be processed;

[0100] The downsampled output vector is obtained by summing the sampled feature vector and the pooling vector.

[0101] In this example, the feature module extracts features from the convolutional input vector in the sampling input layer by means of a sampling stride. In this embodiment, the sampling stride is 2. Therefore, the width and height of the downsampled feature vector obtained by the feature module are half of the width and height of the sampling input layer, respectively, so as to achieve the technical effect of compressing the convolutional input vector.

[0102] Simultaneously, the feature module is also used to extract features from the convolution input vector to obtain a sampled feature vector. The sampled feature vector is then integrated with the pooling vector output by the pooling module to obtain a downsampled feature vector. Finally, the sampled feature vector is integrated with the convolution input vector to obtain the downsampled feature vector. This ensures that the downsampled feature vector contains the convolution input vector and feature vectors representing the contour and coverage of objects in the convolution input vector, thus ensuring the comprehensiveness of the output downsampled feature vector.

[0103] Meanwhile, a downsampling module is obtained by connecting the sampling input layer and the output of the feature module. When training this downsampling module and performing reverse iteration on a downsampling module, the reverse iterator can not only adjust the weights in the downsampling module, but also bypass the feature module through the pooling module to reach the sampling input layer. It is then passed through the sampling input layer to the downsampling module or ordinary module located before this downsampling module, and the weights in the preceding ordinary module or downsampling module are adjusted. This achieves the comprehensiveness of the reverse propagation of the reverse iterator, ensuring that each ordinary module and downsampling module can be iteratively adjusted.

[0104] Specifically, based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and features are extracted from the compressed convolutional input vector to obtain a sampling feature vector, including:

[0105] The sampling splitting and compression module in the sampling feature module splits the convolution input vector, and extracts and merges the features of the split convolution input vector with a sampling stride to obtain sampled compressed convolution data; wherein, the sampled compressed convolution data is used to characterize the compressed image to be processed;

[0106] The sampling feature module splits the sampling compressed convolutional data based on the sampling dilation and merging module, and extracts and merges the features of the split sampling compressed convolutional data with the sampling stride to obtain the sampling feature vector.

[0107] In this example, a sampling splitting and compression module is used to split the convolutional input vector into several sampled target data segments. This avoids overfitting that can occur when training by directly extracting features from the entire convolutional input vector. Simultaneously, the sampling splitting and compression module compresses and extracts features from the sampled target data segments with a sampling stride, and then merges the feature vectors extracted from each sampled target data segment to complete the feature extraction of the convolutional input vector, resulting in sampled compressed convolutional data. The compression of the sampled target data segments eliminates invalid and redundant data.

[0108] The sampling dilation and merging module splits the sampled compressed convolutional data into several sampled convolutional sub-data to avoid overfitting when training feature extraction directly from the entire sampled compressed convolutional data. Simultaneously, the module dilates and extracts features from the sampled convolutional sub-data with a sampling stride, and merges the resulting feature vectors to complete feature extraction from the convolutional input vector, obtaining sampled dilated convolutional data. Merging multiple sampled dilated convolutional data yields a sampled feature vector. Specifically, dilation of the sampled convolutional sub-data fills the positions of previously invalid or redundant data with pre-defined dilated feature values, separating the feature values ​​representing the object from other parts of the convolutional input vector, thereby improving feature extraction efficiency.

[0109] Specifically, based on the pooling module in the downsampling module, the convolutional input vector is pooled to obtain a pooled vector, including:

[0110] The average pooling module in the pooling module is used to perform pooling processing on the convolutional input vector to obtain average pooled data; where the average pooled data represents the abstracted image to be processed.

[0111] Based on the pooling convolution kernel in the pooling module, the average pooled data is convolved to obtain the pooling vector.

[0112] In this example, the convolutional input vector is pooled using an average pooling module and a pooling convolution kernel to obtain a pooling vector, thus avoiding overfitting in the downsampling module. The pooling vector and the sampled feature vector are summed to improve the robustness of weight changes in the downsampling module.

[0113] Furthermore, the convolutional input vector is split based on the sampling splitting and compression module in the sampling feature module, and features are extracted and merged from the split convolutional input vectors with a sampling stride to obtain sampled compressed convolutional data, including:

[0114] Based on the sampling target splitting module in the sampling splitting compression module, the convolution input vector is split to obtain at least one sampling target splitting data; wherein, the sampling target splitting data represents a part of the image to be processed;

[0115] Based on the sampling and compression module in the sampling and splitting compression module, each sampled target split data is subjected to compression and convolution processing to obtain at least one sampled and splitting compressed convolution data; wherein, the sampled and splitting compressed convolution data represents the compressed part of the image to be processed;

[0116] Based on the sampling split compression module, the sampling split compression convolutional data are merged to obtain a sampling compressed merged data; wherein, the sampling compressed merged data represents the compressed part of the image to be processed after merging.

[0117] Based on the sampling merging convolution module in the sampling splitting compression module, the sampled compressed and merged data is processed by convolution to obtain a sampled compressed convolution data.

[0118] In this example, the sampling target splitting module is used to split a convolutional input vector to obtain at least one sampling target splitting data, so as to avoid training by directly using the entire convolutional input vector for feature extraction, which would lead to overfitting when using the feature module.

[0119] The sampling compression convolution module is used to perform compression convolution processing on each sampled target split data to obtain at least one sampled split compressed convolution data. It is used to compress the sampled target split data with a sampling stride and extract features in the sampling stage of feature extraction. The feature vectors after feature extraction of each sampled target split data are merged to complete the feature extraction of the convolution input vector and obtain the sampled split compressed convolution data.

[0120] The sampling compression and merging module is used to merge sampled, split, compressed, and convolutional data to obtain a single sampled, compressed, and merged data.

[0121] The sampling and merging convolution module is used to perform convolution processing on the sampled and compressed merged data to obtain a sampled and compressed convolutional data; it is used to perform overall feature extraction on the sampled and compressed merged data with a sampling stride, so as to realize feature extraction in the sampling stage of feature extraction, thereby improving the accuracy of feature extraction.

[0122] Different weights can be set for each layer in the sampling and merging convolution module to extract and merge features from the sampled, compressed, and merged data using the weights of different channels, thus achieving the technical effect of weighted summation of extracted features.

[0123] Furthermore, the sampled compressed convolutional data is split based on the sampled dilation and merging module in the sampled feature module, and features are extracted and merged from the split sampled compressed convolutional data with a sampling stride to obtain a sampled feature vector, including:

[0124] Based on the sampling convolution splitting module in the sampling feature module, the sampling compressed convolution data is split to obtain at least one sample convolution split data; wherein, the sample convolution split data represents the compressed image to be processed after further splitting;

[0125] Based on the sampling dilated convolution module in the sampling feature module, each sampled convolution split data is subjected to dilated convolution processing to obtain at least one sampled dilated convolution data; wherein, the sampled dilated convolution data represents the compressed image to be processed after being split again to restore the dimension to the convolution input vector.

[0126] Based on the sampling dilation and merging module in the sampling feature module, the sampled dilation convolution data are merged to obtain a sampling feature vector.

[0127] In this example, the sampling convolution splitting module is used to split a sampled compressed convolutional data into at least one sampled convolution split data to avoid training by directly using the entire sampled compressed convolutional data for feature extraction, which could lead to overfitting when using the feature module.

[0128] The sampling dilated convolution module performs dilation convolution processing on each sampled convolutional split data to obtain at least one sampled dilated convolutional data. Simultaneously, the sampling dilation merging module performs dilation processing and feature extraction on the sampled convolutional split data with a sampling stride, and merges the obtained feature vectors to complete the feature extraction of the convolutional input vector and obtain sampled dilated convolutional data. By dilating the sampled convolutional split data, preset dilation feature values ​​are filled in the positions of the original invalid or redundant data in the sampled target split data, so as to separate the feature values ​​representing the object in the convolutional input vector from the feature values ​​of other parts before representing the object, thereby improving the feature extraction effect.

[0129] The sampling dilation merging module is used to merge sampled dilated convolution data to obtain a sampled feature vector.

[0130] For example, a feature with a channel of c, a height of h, and a width of w is used as the input vector for convolution.

[0131] The sampling target splitting module splits the convolutional input vector into n groups of features along the channel dimension, resulting in n groups of features with c / n channels, h height, and w width, which are the sampling target splitting data.

[0132] The sampling compression convolution module splits the data for each sampled target and compresses it using a stride of 2 and a compression ratio of r. 3. Process the data using convolution to obtain n sets of features with c / (n×r) channels, h / 2 height, and w / 2 width. That is, sample, split, and compress the convolutional data.

[0133] The sampling, compression, and merging module merges the sampled, split, compressed, and convolutional data along the channel dimension to obtain features with a channel size of c / r, a height of h / 2, and a width of w / 2, i.e., sampled, compressed, and merged data.

[0134] The sampling and merging convolution module uses 3x3 convolutions with equal input and output. 3. Convolutional processing of multiple sampled and compressed data results in features with channels c / r, height h / 2, and width w / 2, i.e., sampled and compressed convolutional data.

[0135] The sampling convolution splitting module splits the sampled compressed convolutional data into n groups of features along the channel dimension, resulting in n groups of features with channels c / n / r, height h / 2, and width w / 2, i.e., the sampled convolution splitting data.

[0136] The sampling dilated convolution module splits each sampled convolutional data set into sub-data sets and dilates them using a 3x3 dilation factor of r. The data is processed by 3 convolutions to obtain n sets of features with c / n channels, h / 2 height, and w / 2 width, which is called sampled dilated convolution data.

[0137] The sampling dilation and merging module merges the sampled dilated convolutional data along the channel dimension to obtain features with c channels, h / 2 height, and w / 2 width. That is, the sampled feature vector.

[0138] S103: Based on the ordinary module in the classification model, feature extraction is performed on the downsampled output vector to obtain the image feature vector; wherein, the image feature vector is used to characterize the shape and contour of the object in the image to be processed.

[0139] In this step, a common module is used to extract features from the received feature vector to obtain feature vectors in the downsampled output vector that can characterize the coverage area and contour of the object in the downsampled output vector.

[0140] In this example, the downsampled output vector in the ordinary input layer is extracted using a normal module with a normal step size. In this embodiment, the normal step size is 1. Therefore, the width and height dimensions of the normal feature vector obtained by the normal module are consistent with the width and height dimensions of the ordinary input layer. This ensures that the downsampled output vector is fully processed and the corresponding feature vector is obtained, thus ensuring the comprehensiveness and accuracy of feature extraction. Therefore, it ensures that the image feature vector contains the downsampled output vector and the feature vector representing the contour and coverage of the object in the downsampled output vector, ensuring the comprehensiveness of the output image feature vector.

[0141] Meanwhile, a normal module is obtained by connecting the normal input layer and the output of the normal module. When training the normal module and performing reverse iteration on a normal module, the reverse iterator can not only adjust the weights in the normal module, but also bypass the normal module through the output of the normal module to reach the normal input layer, and then pass it through the normal input layer to the normal module or downsampling module located in front of the normal module, and adjust the weights in the previous normal module or downsampling module. This realizes the comprehensiveness of the reverse propagation of the reverse iterator and ensures that every normal module and downsampling module can be iteratively adjusted.

[0142] In a preferred embodiment, feature extraction is performed on the downsampled output vector based on a common module in the classification model to obtain an image feature vector, including:

[0143] The downsampled output vector is obtained based on the ordinary input layer in the ordinary module;

[0144] The downsampled output vector is split based on the ordinary splitting and compression module in the ordinary module, and features are extracted and merged on the split downsampled output vector with ordinary stride to obtain ordinary compressed convolutional data; wherein, ordinary compressed convolutional data is used to represent the compressed downsampled output vector;

[0145] The ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and the split ordinary compressed convolutional data is subjected to feature extraction and merging with ordinary stride to obtain ordinary feature vector; where the ordinary feature vector is the feature vector of the compressed downsampled output vector after preliminary feature extraction.

[0146] The image feature vector is obtained by summing the downsampled output vector and the ordinary feature vector from the ordinary input layer.

[0147] In this example, a standard splitting and compression module is used to split the downsampled output vector into several standard target data segments. This avoids overfitting that can occur when training by directly using the entire downsampled output vector for feature extraction. Simultaneously, the standard splitting and compression module compresses and extracts features from the standard target data segments with a standard stride. The feature vectors extracted from each standard target data segment are then merged to complete the feature extraction of the downsampled output vector, resulting in standard compressed convolutional data. Compression of the standard target data segments eliminates invalid and redundant data.

[0148] The ordinary dilation and merging module is used to split the ordinary compressed convolutional data into several ordinary convolutional split data to avoid overfitting caused by directly training the ordinary module through feature extraction from the entire ordinary compressed convolutional data. Simultaneously, the ordinary dilation and merging module performs dilation processing and feature extraction on the ordinary convolutional split data with a normal stride, and merges the resulting feature vectors to complete the feature extraction of the downsampled output vector, obtaining ordinary dilated convolutional data. Merging multiple ordinary dilated convolutional data yields a normal feature vector. Specifically, the dilation processing of the ordinary convolutional split data fills the positions of the original invalid or redundant data in the ordinary target split data with pre-set dilation feature values, facilitating the separation of the feature values ​​representing the object in the downsampled output vector from the feature values ​​of other parts before representing the object, thereby improving the feature extraction effect.

[0149] Specifically, the downsampled output vector is split based on the ordinary splitting and compression module in the ordinary module, and features are extracted and merged from the split downsampled output vector with an ordinary stride to obtain ordinary compressed convolutional data, including:

[0150] Based on the ordinary target splitting module in the ordinary splitting compression module, the downsampled output vector is split to obtain at least one ordinary target splitting data; wherein, the ordinary target splitting data represents a part of the downsampled attribute vector;

[0151] The ordinary compression convolution module in the ordinary split compression module is used to perform compression convolution processing on each ordinary target split data to obtain at least one ordinary split compressed convolution data; wherein, the ordinary split compressed convolution data represents the compressed partial downsampled output vector;

[0152] Based on the ordinary compression and merging module in the ordinary split compression module, the ordinary split compressed convolutional data is merged to obtain ordinary compressed merged data; wherein, the ordinary compressed merged data represents the merged compressed partial downsampled output vector;

[0153] Based on the ordinary merging convolution module in the ordinary splitting compression module, ordinary compressed and merged data is processed by convolution to obtain ordinary compressed convolution data.

[0154] In this example, the ordinary target splitting module is used to split a downsampled output vector to obtain at least one ordinary target splitting data, so as to avoid the overfitting of the ordinary module caused by directly training the feature extraction from the entire downsampled output vector.

[0155] The ordinary compression convolution module is used to perform compression convolution processing on each ordinary target segmentation data to obtain at least one ordinary segmentation compressed convolution data. It is used to compress the ordinary target segmentation data with an ordinary stride and extract features in the ordinary stage of feature extraction. The feature vectors after feature extraction of each ordinary target segmentation data are merged to complete the feature extraction of the downsampled output vector and obtain the ordinary segmentation compressed convolution data.

[0156] The standard compression and merging module is used to merge standard split and compressed convolutional data to obtain a single standard compressed and merged data.

[0157] The ordinary merging convolution module is used to perform convolution processing on ordinary compressed and merged data to obtain ordinary compressed convolution data; it is used to perform overall feature extraction on ordinary compressed and merged data with ordinary stride, thereby realizing the second stage of feature extraction and improving the accuracy of feature extraction.

[0158] In this module, different weights can be set for each layer in the ordinary convolutional merging module, so as to extract and merge features from ordinary compressed and merged data by using the weights of different channels as weights, and obtain the technical effect of feature extraction weighted summation.

[0159] Specifically, the ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and features are extracted and merged from the split ordinary compressed convolutional data with an ordinary stride to obtain ordinary feature vectors, including:

[0160] Based on the ordinary convolution splitting module in the ordinary dilation and merging module, the ordinary compressed convolution data is split to obtain at least one ordinary convolution split data; wherein, the ordinary convolution split data represents the compressed downsampled output vector after further splitting;

[0161] Based on the ordinary dilated convolution module in the ordinary dilated merging module, each ordinary convolution split data is subjected to dilated convolution processing to obtain at least one ordinary dilated convolution data; wherein, the ordinary dilated convolution data represents the compressed downsampled output vector that is re-splitted to restore the dimension of the convolution input vector;

[0162] Based on the ordinary dilation merging module in the ordinary dilation merging module, ordinary dilation convolution data are merged to obtain an ordinary feature vector.

[0163] In this example, the ordinary convolution splitting module is used to split an ordinary compressed convolutional data into at least one ordinary convolution split data to avoid overfitting of the ordinary module by directly training the feature extraction from the entire ordinary compressed convolutional data.

[0164] The ordinary dilated convolution module is used to perform dilated convolution processing on each ordinary convolution split data to obtain at least one ordinary dilated convolution data. At the same time, the ordinary dilation merging module performs dilation processing and feature extraction on the ordinary convolution split data with an ordinary stride, and merges the obtained feature vectors to complete the feature extraction of the downsampled output vector and obtain ordinary dilated convolution data. By performing dilation processing on the ordinary convolution split data, preset dilation feature values ​​are filled in the positions of the original invalid or redundant data in the ordinary target split data, so as to separate the feature values ​​representing the object in the downsampled output vector from the feature values ​​of other parts before representing the object, thereby improving the feature extraction effect.

[0165] The ordinary dilation merge module is used to merge ordinary dilated convolution data to obtain an ordinary feature vector.

[0166] For example, a feature with channel c, height h, and width w is used as a downsampled output vector.

[0167] The ordinary target splitting module splits the downsampled output vector into n groups of features along the channel dimension, resulting in n groups of features with c / n channels, h height, and w width, which are the ordinary target splitting data.

[0168] The standard compressed convolutional module splits each set of ordinary target data into three parts, each using a stride of 1 and a compression ratio of r. 3. Process it with convolution to obtain n groups of features with channels c / (n×r), height h, and width w, which is: ordinary splitting and compressing convolution data.

[0169] The ordinary compression and merging module merges the ordinary split and compressed convolutional data along the channel dimension to obtain features with channels c / r, height h, and width w, i.e., ordinary compressed and merged data.

[0170] The standard merge convolution module uses 3x3 convolutions with equal input and output. 3. Convolution processes multiple ordinary compressed and merged data to obtain features with channels c / r, height h, and width w, i.e., ordinary compressed convolution data.

[0171] The ordinary convolution splitting module splits ordinary compressed convolution data into n groups of features along the channel dimension, resulting in n groups of features with a channel size of c / (n×r), a height of h, and a width of w, i.e., ordinary convolution splitting data.

[0172] The standard dilated convolution module splits each set of data into three parts, each with a dilation ratio of r. 3. Process it with convolution to obtain n sets of features with c / n channels, h height and w width, that is: ordinary dilated convolution data.

[0173] The ordinary dilation and merging module merges the ordinary dilated convolution data along the channel dimension to obtain features with c channels, h height, and w width. That is, ordinary feature vectors.

[0174] Add the ordinary feature vector and the downsampled output vector to obtain a feature with channel c, height h, and width w.

[0175] S104: Based on the fully connected layer in the classification model, the image feature vector is classified to obtain the image category of the image to be processed; where the image category reflects the type or kind of object in the image to be processed.

[0176] In this step, fully connected layers (FC) act as a "classifier" in the entire convolutional neural network. If operations such as convolutional layers, pooling layers, and activation functions map the raw data to the hidden feature space, then fully connected layers map the learned "distributed feature representation" to the sample label space.

[0177] Example 2:

[0178] Please see Figure 3 This application provides a training method for a classification model applied to image classification, comprising:

[0179] S201: Obtain the image to be trained, input the image to be trained into a preset initial model, and obtain the convolutional input vector from the input layer of the initial model; wherein, the initial model is a convolutional neural network model; the convolutional input vector is used to represent the image to be trained.

[0180] In this step, the input layer is used to preprocess or convolve the input image to be trained, so as to obtain a convolutional input vector representing the image to be trained.

[0181] Convolutional Neural Networks (CNNs) are a type of neural network that uses convolutional kernels to extract important information from data, employs image padding strategies to preserve as much information as possible from the original data, and uses pooling and other strategies to avoid overfitting (a great many strategies have been proposed to avoid overfitting).

[0182] The classification model in this application is a convolutional neural network adapted for the Winograd algorithm. Originating in 1980, the Winograd algorithm, proposed by Shmuel Winograd, is an algorithm designed to reduce the computational cost of FIR filters. It states that for an FIR filter with *n* outputs and *n* parameters, only *n* multiplications are needed instead of *n* multiplications. The Winograd algorithm is suitable for convolutional kernels with a kernel size of 3×3 or larger. Therefore, by setting the kernel size for feature extraction in both the ordinary and downsampling modules to K×K, where K is a natural number and K≥3, the ordinary and downsampling modules are better adapted to the Winograd algorithm, thereby improving the running speed of the neural network model.

[0183] The Winograd algorithm is a deep learning convolution optimization algorithm. Therefore, neural network models built based on the Winograd algorithm will have higher running speed while ensuring comprehensiveness and accuracy of computation. This allows the obtained neural network model to ensure that mobile hardware devices can run the model while also ensuring the accuracy of neural network computation.

[0184] S202: Based on the downsampling module in the initial model, the convolutional input vector is compressed and features are extracted to obtain the downsampled training vector; wherein, the downsampled training vector is the feature vector of the compressed training image after preliminary feature extraction, and the training image after representation abstraction.

[0185] In a preferred embodiment, the convolutional input vector is compressed and its features are extracted based on the downsampling module in the initial model to obtain a downsampled training vector, including:

[0186] The convolutional input vector is obtained based on the sampling input layer in the downsampling module;

[0187] Based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and the compressed convolutional input vector is used to extract features to obtain the sampling feature vector; where the sampling feature vector is the feature vector for preliminary feature extraction of the compressed training image.

[0188] Based on the pooling module in the downsampling module, the convolutional input vector is pooled to obtain a pooling vector; where the pooling vector represents the abstracted image to be trained;

[0189] The downsampled training vector is obtained by summing the sampled feature vector and the pooling vector.

[0190] In this example, the feature module extracts features from the convolutional input vector in the sampling input layer by means of a sampling stride. In this embodiment, the sampling stride is 2. Therefore, the width and height of the downsampled feature vector obtained by the feature module are half of the width and height of the sampling input layer, respectively, so as to achieve the technical effect of compressing the convolutional input vector.

[0191] Simultaneously, the feature module is also used to extract features from the convolution input vector to obtain a sampled feature vector. The sampled feature vector is then integrated with the pooling vector output by the pooling module to obtain a downsampled feature vector. Finally, the sampled feature vector is integrated with the convolution input vector to obtain the downsampled feature vector. This ensures that the downsampled feature vector contains the convolution input vector and feature vectors representing the contour and coverage of objects in the convolution input vector, thus ensuring the comprehensiveness of the output downsampled feature vector.

[0192] Meanwhile, a downsampling module is obtained by connecting the sampling input layer and the output of the feature module. When training this downsampling module and performing reverse iteration on a downsampling module, the reverse iterator can not only adjust the weights in the downsampling module, but also bypass the feature module through the pooling module to reach the sampling input layer. It is then passed through the sampling input layer to the downsampling module or ordinary module located before this downsampling module, and the weights in the preceding ordinary module or downsampling module are adjusted. This achieves the comprehensiveness of the reverse propagation of the reverse iterator, ensuring that each ordinary module and downsampling module can be iteratively adjusted.

[0193] Specifically, based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and features are extracted from the compressed convolutional input vector to obtain the sampling feature vector, including:

[0194] The sampling splitting and compression module in the sampling feature module splits the convolutional input vector, and extracts and merges the features of the split convolutional input vector with a sampling stride to obtain sampled compressed convolutional data; wherein, the sampled compressed convolutional data is used to represent the compressed training image;

[0195] The sampling feature module splits the sampling compressed convolutional data based on the sampling dilation and merging module, and extracts and merges the features of the split sampling compressed convolutional data with the sampling stride to obtain the sampling feature vector.

[0196] In this example, a sampling splitting and compression module is used to split the convolutional input vector into several sampled target data segments. This avoids overfitting that can occur when training by directly extracting features from the entire convolutional input vector. Simultaneously, the sampling splitting and compression module compresses and extracts features from the sampled target data segments with a sampling stride, and then merges the feature vectors extracted from each sampled target data segment to complete the feature extraction of the convolutional input vector, resulting in sampled compressed convolutional data. The compression of the sampled target data segments eliminates invalid and redundant data.

[0197] The sampling dilation and merging module splits the sampled compressed convolutional data into several sampled convolutional sub-data to avoid overfitting when training feature extraction directly from the entire sampled compressed convolutional data. Simultaneously, the module dilates and extracts features from the sampled convolutional sub-data with a sampling stride, and merges the resulting feature vectors to complete feature extraction from the convolutional input vector, obtaining sampled dilated convolutional data. Merging multiple sampled dilated convolutional data yields a sampled feature vector. Specifically, dilation of the sampled convolutional sub-data fills the positions of previously invalid or redundant data with pre-defined dilated feature values, separating the feature values ​​representing the object from other parts of the convolutional input vector, thereby improving feature extraction efficiency.

[0198] Specifically, based on the pooling module in the downsampling module, the convolutional input vector is pooled to obtain a pooled vector, including:

[0199] The average pooling module in the pooling module is used to perform pooling processing on the convolutional input vector to obtain average pooled data; where the average pooled data represents the abstracted image to be trained.

[0200] Based on the pooling convolution kernel in the pooling module, the average pooled data is convolved to obtain the pooling vector.

[0201] In this example, the convolutional input vector is pooled using an average pooling module and a pooling convolution kernel to obtain a pooling vector, thus avoiding overfitting in the downsampling module. The pooling vector and the sampled feature vector are summed to improve the robustness of weight changes in the downsampling module.

[0202] Furthermore, the convolutional input vector is split based on the sampling splitting and compression module in the sampling feature module, and features are extracted and merged from the split convolutional input vectors with a sampling stride to obtain sampled compressed convolutional data, including:

[0203] Based on the sampling target splitting module in the sampling splitting compression module, the convolution input vector is split to obtain at least one sampling target splitting data; wherein, the sampling target splitting data represents a part of the image to be trained;

[0204] Based on the sampling and splitting compression module in the sampling and splitting compression module, each sampled target splitting data is subjected to compression and convolution processing to obtain at least one sampled and splitting compressed convolution data; wherein, the sampled and splitting compressed convolution data represents the compressed part of the image to be trained;

[0205] Based on the sampling split compression module, the sampling split compression convolutional data are merged to obtain a sampling compressed merged data; wherein, the sampling compressed merged data represents the compressed part of the training image after merging.

[0206] Based on the sampling merging convolution module in the sampling splitting compression module, the sampled compressed and merged data is processed by convolution to obtain a sampled compressed convolution data.

[0207] In this example, the sampling target splitting module is used to split a convolutional input vector to obtain at least one sampling target splitting data, so as to avoid training by directly using the entire convolutional input vector for feature extraction, which would lead to overfitting when using the feature module.

[0208] The sampling compression convolution module is used to perform compression convolution processing on each sampled target split data to obtain at least one sampled split compressed convolution data. It is used to compress the sampled target split data with a sampling stride and extract features in the sampling stage of feature extraction. The feature vectors after feature extraction of each sampled target split data are merged to complete the feature extraction of the convolution input vector and obtain the sampled split compressed convolution data.

[0209] The sampling compression and merging module is used to merge sampled, split, compressed, and convolutional data to obtain a single sampled, compressed, and merged data.

[0210] The sampling and merging convolution module is used to perform convolution processing on the sampled and compressed merged data to obtain a sampled and compressed convolutional data; it is used to perform overall feature extraction on the sampled and compressed merged data with a sampling stride, so as to realize feature extraction in the sampling stage of feature extraction, thereby improving the accuracy of feature extraction.

[0211] Different weights can be set for each layer in the sampling and merging convolution module to extract and merge features from the sampled, compressed, and merged data using the weights of different channels, thus achieving the technical effect of weighted summation of extracted features.

[0212] Furthermore, the sampled compressed convolutional data is split based on the sampled dilation and merging module in the sampled feature module, and features are extracted and merged from the split sampled compressed convolutional data with a sampling stride to obtain a sampled feature vector, including:

[0213] Based on the sampling convolution splitting module in the sampling feature module, the sampling compressed convolution data is split to obtain at least one sampling convolution split data; wherein, the sampling convolution split data represents the compressed training image after further splitting;

[0214] Based on the sampling dilated convolution module in the sampling feature module, each sampled convolution split data is subjected to dilated convolution processing to obtain at least one sampled dilated convolution data; wherein, the sampled dilated convolution data represents the compressed training image that has been re-splitted to the dimension of the convolution input vector.

[0215] Based on the sampling dilation and merging module in the sampling feature module, the sampled dilation convolution data are merged to obtain a sampling feature vector.

[0216] In this example, the sampling convolution splitting module is used to split a sampled compressed convolutional data into at least one sampled convolution split data to avoid training by directly using the entire sampled compressed convolutional data for feature extraction, which could lead to overfitting when using the feature module.

[0217] The sampling dilated convolution module performs dilation convolution processing on each sampled convolutional split data to obtain at least one sampled dilated convolutional data. Simultaneously, the sampling dilation merging module performs dilation processing and feature extraction on the sampled convolutional split data with a sampling stride, and merges the obtained feature vectors to complete the feature extraction of the convolutional input vector and obtain sampled dilated convolutional data. By dilating the sampled convolutional split data, preset dilation feature values ​​are filled in the positions of the original invalid or redundant data in the sampled target split data, so as to separate the feature values ​​representing the object in the convolutional input vector from the feature values ​​of other parts before representing the object, thereby improving the feature extraction effect.

[0218] The sampling dilation merging module is used to merge sampled dilated convolution data to obtain a sampled feature vector.

[0219] For example, a feature with a channel of c, a height of h, and a width of w is used as the input vector for convolution.

[0220] The sampling target splitting module splits the convolutional input vector into n groups of features along the channel dimension, resulting in n groups of features with c / n channels, h height, and w width, which are the sampling target splitting data.

[0221] The sampling compression convolution module splits the data for each sampled target and compresses it using a stride of 2 and a compression ratio of r. 3. Process the data using convolution to obtain n sets of features with c / (n×r) channels, h / 2 height, and w / 2 width. That is, sample, split, and compress the convolutional data.

[0222] The sampling, compression, and merging module merges the sampled, split, compressed, and convolutional data along the channel dimension to obtain features with a channel size of c / r, a height of h / 2, and a width of w / 2, i.e., sampled, compressed, and merged data.

[0223] The sampling and merging convolution module uses 3x3 convolutions with equal input and output. 3. Convolutional processing of multiple sampled and compressed data results in features with channels c / r, height h / 2, and width w / 2, i.e., sampled and compressed convolutional data.

[0224] The sampling convolution splitting module splits the sampled compressed convolutional data into n groups of features along the channel dimension, resulting in n groups of features with channels c / n / r, height h / 2, and width w / 2, i.e., the sampled convolution splitting data.

[0225] The sampling dilated convolution module splits each sampled convolutional data set into sub-data sets and dilates them using a 3x3 dilation factor of r. The data is processed by 3 convolutions to obtain n sets of features with c / n channels, h / 2 height, and w / 2 width, which is called sampled dilated convolution data.

[0226] The sampling dilation and merging module merges the sampled dilated convolutional data along the channel dimension to obtain features with c channels, h / 2 height, and w / 2 width. That is, the sampled feature vector.

[0227] S203: Based on the ordinary modules in the initial model, feature extraction is performed on the downsampled training vector to obtain the image training vector; wherein, the image training vector is used to represent the shape and contour of the object in the image to be trained.

[0228] In this step, a common module is used to extract features from the received feature vectors to obtain feature vectors in the downsampled training vectors that can characterize the coverage area and contour of the objects in the downsampled training vectors.

[0229] In this example, the normal module extracts features from the downsampled training vector in the normal input layer using a normal step size. In this embodiment, the normal step size is 1. Therefore, the width and height of the normal feature vector obtained by the normal module are consistent with the width and height of the normal input layer, so as to realize the comprehensive calculation of the downsampled training vector and obtain the corresponding feature vector, thus ensuring the comprehensiveness and accuracy of feature extraction.

[0230] By integrating ordinary feature vectors and downsampled training vectors, ordinary feature vectors are obtained. This ensures that the ordinary feature vectors contain the downsampled training vectors and feature vectors that represent the contours and coverage of objects in the downsampled training vectors, thus ensuring the comprehensiveness of the output ordinary feature vectors.

[0231] Meanwhile, a normal module is obtained by connecting the normal input layer and the output of the normal module. When training the normal module and performing reverse iteration on a normal module, the reverse iterator can not only adjust the weights in the normal module, but also bypass the normal module through the output of the normal module to reach the normal input layer, and then pass it through the normal input layer to the normal module or downsampling module located in front of the normal module, and adjust the weights in the previous normal module or downsampling module. This realizes the comprehensiveness of the reverse propagation of the reverse iterator and ensures that every normal module and downsampling module can be iteratively adjusted.

[0232] In a preferred embodiment, feature extraction is performed on the downsampled training vector based on the common modules in the initial model to obtain the image training vector, including:

[0233] Obtain downsampled training vectors based on the ordinary input layer in the ordinary module;

[0234] The downsampled training vector is split based on the ordinary splitting and compression module in the ordinary module, and the features of the split downsampled training vector are extracted and merged with ordinary stride to obtain ordinary compressed convolutional data; wherein, ordinary compressed convolutional data is used to represent the compressed downsampled training vector.

[0235] The ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and the split ordinary compressed convolutional data is extracted and merged with ordinary stride to obtain ordinary feature vector; where the ordinary feature vector is the feature vector of the compressed downsampled training vector after preliminary feature extraction.

[0236] The image training vector is obtained by summing the downsampled training vectors and ordinary feature vectors from the ordinary input layer.

[0237] In this example, a standard splitting and compression module is used to split the downsampled training vector into several standard target data segments. This avoids overfitting that can occur when directly training by extracting features from the entire downsampled training vector. Simultaneously, the standard splitting and compression module compresses and extracts features from the standard target data segments with a standard stride. The feature vectors extracted from each standard target data segment are then merged to complete the feature extraction of the downsampled training vector, resulting in standard compressed convolutional data. Compression of the standard target data segments eliminates invalid and redundant data.

[0238] The ordinary dilation and merging module is used to split the ordinary compressed convolutional data into several ordinary convolutional split data to avoid overfitting caused by directly training the ordinary module through feature extraction from the entire ordinary compressed convolutional data. Simultaneously, the ordinary dilation and merging module performs dilation processing and feature extraction on the ordinary convolutional split data with a normal stride, and merges the resulting feature vectors to complete feature extraction from the downsampled training vector, obtaining ordinary dilated convolutional data. Merging multiple ordinary dilated convolutional data yields ordinary feature vectors. Specifically, the dilation processing of the ordinary convolutional split data fills the positions of previously invalid or redundant data in the ordinary target split data with pre-set dilation feature values, facilitating the separation of the feature values ​​representing the object in the downsampled training vector from the feature values ​​of other parts before representing the object, thereby improving the feature extraction effect.

[0239] Specifically, the downsampled training vectors are split using the standard splitting and compression module within the standard module, and features are extracted and merged from the split downsampled training vectors using a standard stride to obtain standard compressed convolutional data, including:

[0240] Based on the common target splitting module in the common splitting compression module, the downsampled training vector is split to obtain at least one common target splitting data; wherein, the common target splitting data represents a part of the downsampled attribute vector;

[0241] The ordinary compression convolution module in the ordinary split compression module is used to perform compression convolution processing on each ordinary target split data to obtain at least one ordinary split compressed convolution data; wherein, the ordinary split compressed convolution data represents the compressed partial downsampled training vector;

[0242] Based on the ordinary compression and merging module in the ordinary split compression module, the ordinary split compressed convolutional data is merged to obtain ordinary compressed merged data; wherein, the ordinary compressed merged data represents the merged compressed partial downsampled training vector;

[0243] Based on the ordinary merging convolution module in the ordinary splitting compression module, ordinary compressed and merged data is processed by convolution to obtain ordinary compressed convolution data.

[0244] In this example, the ordinary target splitting module is used to split a downsampled training vector to obtain at least one ordinary target splitting data, so as to avoid the overfitting of the ordinary module caused by directly training the feature extraction from the entire downsampled training vector.

[0245] The ordinary compression convolution module is used to perform compression convolution processing on each ordinary target segmentation data to obtain at least one ordinary segmentation compressed convolution data. It is used to compress the ordinary target segmentation data with an ordinary stride and extract features in the ordinary stage of feature extraction. The feature vectors after feature extraction of each ordinary target segmentation data are merged to complete the feature extraction of the downsampled training vector and obtain the ordinary segmentation compressed convolution data.

[0246] The standard compression and merging module is used to merge standard split and compressed convolutional data to obtain a single standard compressed and merged data.

[0247] The ordinary merging convolution module is used to perform convolution processing on ordinary compressed and merged data to obtain ordinary compressed convolution data; it is used to perform overall feature extraction on ordinary compressed and merged data with ordinary stride, thereby realizing the second stage of feature extraction and improving the accuracy of feature extraction.

[0248] In this module, different weights can be set for each layer in the ordinary convolutional merging module, so as to extract and merge features from ordinary compressed and merged data by using the weights of different channels as weights, and obtain the technical effect of feature extraction weighted summation.

[0249] Specifically, the ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and features are extracted and merged from the split ordinary compressed convolutional data with an ordinary stride to obtain ordinary feature vectors, including:

[0250] Based on the ordinary convolution splitting module in the ordinary dilation and merging module, the ordinary compressed convolution data is split to obtain at least one ordinary convolution split data; wherein, the ordinary convolution split data represents the compressed downsampled training vector after further splitting.

[0251] Based on the ordinary dilated convolution module in the ordinary dilation merging module, each ordinary convolution split data is subjected to dilation convolution processing to obtain at least one ordinary dilated convolution data; wherein, the ordinary dilated convolution data represents the compressed downsampled training vector that is re-splitted to restore the dimension of the convolution input vector;

[0252] Based on the ordinary dilation merging module in the ordinary dilation merging module, ordinary dilation convolution data are merged to obtain an ordinary feature vector.

[0253] In this example, the ordinary convolution splitting module is used to split an ordinary compressed convolutional data into at least one ordinary convolution split data to avoid overfitting of the ordinary module by directly training the feature extraction from the entire ordinary compressed convolutional data.

[0254] The ordinary dilated convolution module is used to perform dilated convolution processing on each ordinary convolution split data to obtain at least one ordinary dilated convolution data. At the same time, the ordinary dilation merging module performs dilation processing and feature extraction on the ordinary convolution split data with an ordinary stride, and merges the obtained feature vectors to complete the feature extraction of the downsampled training vector and obtain ordinary dilated convolution data. By performing dilation processing on the ordinary convolution split data, preset dilated feature values ​​are filled in the positions of the original invalid or redundant data in the ordinary target split data, so as to separate the feature values ​​representing the object in the downsampled training vector from the feature values ​​of other parts before representing the object, thereby improving the feature extraction effect.

[0255] The ordinary dilation merge module is used to merge ordinary dilated convolution data to obtain an ordinary feature vector.

[0256] For example, a feature with channel c, height h, and width w is used as a downsampled training vector.

[0257] The ordinary target splitting module splits the downsampled training vector into n groups of features along the channel dimension, resulting in n groups of features with c / n channels, h height, and w width, which are the ordinary target splitting data.

[0258] The standard compressed convolutional module splits each set of ordinary target data into three parts, each using a stride of 1 and a compression ratio of r. 3. Process it with convolution to obtain n groups of features with channels c / (n×r), height h, and width w, which is: ordinary splitting and compressing convolution data.

[0259] The ordinary compression and merging module merges the ordinary split and compressed convolutional data along the channel dimension to obtain features with channels c / r, height h, and width w, i.e., ordinary compressed and merged data.

[0260] The standard merge convolution module uses 3x3 convolutions with equal input and output. 3. Convolution processes multiple ordinary compressed and merged data to obtain features with channels c / r, height h, and width w, i.e., ordinary compressed convolution data.

[0261] The ordinary convolution splitting module splits ordinary compressed convolution data into n groups of features along the channel dimension, resulting in n groups of features with a channel size of c / (n×r), a height of h, and a width of w, i.e., ordinary convolution splitting data.

[0262] The standard dilated convolution module splits each set of data into three parts, each with a dilation ratio of r. 3. Process it with convolution to obtain n sets of features with c / n channels, h height and w width, that is: ordinary dilated convolution data.

[0263] The ordinary dilation and merging module merges the ordinary dilated convolution data along the channel dimension to obtain features with c channels, h height, and w width. That is, ordinary feature vectors.

[0264] Add the ordinary feature vector and the downsampled training vector to obtain a feature with channel c, height h, and width w.

[0265] S204: Classify the image training vectors based on the fully connected layers in the initial model to obtain the image category of the image to be trained; whereby the image category reflects the type or kind of object in the image to be trained.

[0266] In this step, fully connected layers (FC) act as a "classifier" in the entire convolutional neural network. If operations such as convolutional layers, pooling layers, and activation functions map the raw data to the hidden feature space, then fully connected layers map the learned "distributed feature representation" to the sample label space.

[0267] S205: If the difference between the image category and the training label of the image to be trained is less than the preset training threshold, then the initial model is set as a classification model.

[0268] In this step, the training labels reflect the true image categories of the images to be trained; the training threshold reflects the initial model that generates image categories based on the images to be trained, enabling it to classify the images to be trained and obtain classification results that match the training labels.

[0269] Example 3:

[0270] Please see Figure 4 This application provides an image classification device 1 based on a classification model, comprising:

[0271] The input module 11 is used to acquire the image to be processed, input the image to be processed into a preset classification model, and obtain the convolutional input vector from the input layer of the classification model; wherein, the classification model is a convolutional neural network model; the convolutional input vector is used to represent the image to be processed;

[0272] The first processing module 12 is used to compress and extract features from the convolutional input vector based on the downsampling module in the classification model to obtain a downsampled output vector; wherein, the downsampled output vector is a feature vector for preliminary feature extraction of the compressed image to be processed, and a representation of the abstracted image to be processed;

[0273] The second processing module 13 is used to extract features from the downsampled output vector based on the ordinary module in the classification model to obtain an image feature vector; wherein, the image feature vector is used to characterize the shape and contour of the object in the image to be processed;

[0274] The third processing module 14 is used to classify the image feature vector based on the fully connected layer in the classification model to obtain the image category of the image to be processed; wherein, the image category reflects the type or kind of object in the image to be processed.

[0275] Preferably, the first processing module 12 includes:

[0276] The first input unit 121 is used to obtain the convolutional input vector based on the sampling input layer in the downsampling module;

[0277] The first compression and extraction unit 122 is used to compress the convolutional input vector in the sampling input layer with a sampling stride based on the sampling feature module in the downsampling module, and to extract features from the compressed convolutional input vector to obtain a sampling feature vector; wherein, the sampling feature vector is a feature vector for preliminary feature extraction of the compressed image to be processed.

[0278] The first pooling unit 123 is used to perform pooling processing on the convolutional input vector based on the pooling module in the downsampling module to obtain a pooling vector; wherein, the pooling vector represents the abstracted image to be processed;

[0279] The first aggregation unit 124 is used to aggregate the sampled feature vector and the pooling vector to obtain the downsampled output vector.

[0280] Preferably, the first compression and extraction unit 122 includes:

[0281] The first compression extraction subunit 1221 is used to split the convolution input vector based on the sampling split compression module in the sampling feature module, and to extract and merge the features of the split convolution input vector with a sampling stride to obtain sampled compressed convolution data; wherein, the sampled compressed convolution data is used to characterize the compressed image to be processed;

[0282] The first dilation extraction subunit 1222 is used to split the sampled compressed convolutional data based on the sampled dilation and merging module in the sampled feature module, and extract and merge the features of the split sampled compressed convolutional data with the sampling stride to obtain the sampled feature vector.

[0283] Preferably, the first pooling unit 123 includes:

[0284] The first average pooling unit 1231 is used to perform pooling processing on the convolutional input vector based on the average pooling module in the pooling module to obtain average pooled data; wherein, the average pooling data represents the abstracted image to be processed.

[0285] The first convolutional pooling unit 1232 is used to perform convolution processing on the average pooled data based on the pooling convolution kernel in the pooling module to obtain the pooling vector.

[0286] Preferably, the first compression extraction subunit 1221 includes:

[0287] The first compression and splitting subunit 12211 is used to split the convolution input vector based on the sampling target splitting module in the sampling splitting compression module to obtain at least one sampling target splitting data; wherein, the sampling target splitting data represents a part of the image to be processed;

[0288] The first compression processing subunit 12212 is used to perform compression convolution processing on each sampled target split data based on the sampled compression convolution module in the sampled split compression module to obtain at least one sampled split compressed convolution data; wherein, the sampled split compressed convolution data represents the compressed part of the image to be processed.

[0289] The first compression and merging subunit 12213 is used to merge the sampled split compression convolutional data into a sampled compressed merged data based on the sampled split compression module in the sampled compression and merging module; wherein, the sampled compressed merged data represents the merged compressed part of the image to be processed;

[0290] The first compression convolution subunit 12214 is used to perform convolution processing on the sampled compressed and merged data based on the sampled split compression module in the sampled merge convolution module to obtain a sampled compressed convolution data.

[0291] Preferably, the first expansion extraction subunit 1222 includes:

[0292] The first dilation splitting subunit 12221 is used to split the sampled compressed convolutional data into at least one sampled convolutional split data based on the sampled convolutional splitting module in the sampled feature module; wherein the sampled convolutional split data represents the compressed image to be processed after further splitting.

[0293] The first dilation processing subunit 12222 is used to perform dilation convolution processing on each sampled convolution split data based on the sampled dilation convolution module in the sampled feature module to obtain at least one sampled dilation convolution data; wherein, the sampled dilation convolution data represents the compressed image to be processed after being split again to restore the dimension to the convolution input vector.

[0294] The first dilation and merging subunit 12223 is used to merge sampled dilation and merging convolutional data based on the sampled dilation and merging module in the sampled feature module to obtain a sampled feature vector.

[0295] Preferably, the second processing module 13 includes:

[0296] The second input unit 131 is used to obtain the downsampled output vector based on the ordinary input layer in the ordinary module;

[0297] The second compression extraction unit 132 is used to split the downsampled output vector based on the ordinary split compression module in the ordinary module, and to extract and merge the features of the split downsampled output vector with an ordinary stride to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to characterize the compressed downsampled output vector;

[0298] The second dilation extraction unit 133 is used to split the ordinary compressed convolutional data based on the ordinary dilation merging module in the ordinary module, and to extract and merge the features of the split ordinary compressed convolutional data with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is the feature vector for preliminary feature extraction of the compressed downsampled output vector.

[0299] The second summarization unit 134 is used to summarize the downsampled output vector and ordinary feature vector in the ordinary input layer to obtain the image feature vector.

[0300] Preferably, the second compression and extraction unit 132 includes:

[0301] The second compression and splitting subunit 1321 is used to split the downsampled output vector based on the ordinary target splitting module in the ordinary splitting compression module to obtain at least one ordinary target splitting data; wherein, the ordinary target splitting data represents a portion of the downsampled attribute vector;

[0302] The second compression processing subunit 1322 is used to perform compression convolution processing on each ordinary target split data based on the ordinary compression convolution module in the ordinary split compression module to obtain at least one ordinary split compressed convolution data; wherein, the ordinary split compressed convolution data represents the compressed partial downsampled output vector;

[0303] The second compression merging subunit 1323 is used to merge ordinary split-compressed convolutional data into ordinary compressed merged data based on the ordinary compression merging module in the ordinary split compression module; wherein, the ordinary compressed merged data represents the merged compressed partial downsampled output vector.

[0304] The second compression convolution subunit 1324 is used to perform convolution processing on ordinary compressed and merged data based on the ordinary merge convolution module in the ordinary split compression module to obtain ordinary compressed convolution data.

[0305] Preferably, the second expansion extraction subunit 133 includes:

[0306] The second dilation splitting subunit 1331 is used to split ordinary compressed convolutional data into at least one ordinary convolutional split data based on the ordinary convolutional splitting module in the ordinary dilation merging module; wherein, the ordinary convolutional split data represents the compressed downsampled output vector after further splitting.

[0307] The second dilation processing subunit 1332 is used to perform dilation convolution processing on each ordinary convolution split data based on the ordinary dilation convolution module in the ordinary dilation merging module to obtain at least one ordinary dilation convolution data; wherein, the ordinary dilation convolution data represents the compressed downsampled output vector that is re-splitted to restore the dimension of the convolution input vector.

[0308] The second dilation merging subunit 1333 is used to merge ordinary dilated convolution data to obtain an ordinary feature vector based on the ordinary dilation merging module in the ordinary dilation merging module.

[0309] Example 4:

[0310] Please see Figure 5 This application provides a training device 8 for a classification model applied to image classification, comprising:

[0311] The training input module 81 is used to acquire the image to be trained, input the image to be trained into a preset initial model, and obtain the convolutional input vector from the input layer of the initial model; wherein, the initial model is a convolutional neural network model; the convolutional input vector is used to represent the image to be trained;

[0312] The first training module 82 is used to compress and extract features from the convolutional input vector based on the downsampling module in the initial model to obtain a downsampled training vector; wherein, the downsampled training vector is a feature vector for preliminary feature extraction of the compressed training image and a representation of the training image after abstraction.

[0313] The second training module 83 is used to extract features from the downsampled training vectors based on the ordinary modules in the initial model to obtain image training vectors; wherein, the image feature vectors are used to characterize the shape and contour of objects in the image to be trained;

[0314] The third training module 84 is used to classify the image feature vectors based on the fully connected layers in the initial model to obtain the image category of the image to be trained; wherein, the image category reflects the type or kind of object in the image to be trained;

[0315] The model generation module 85 is used to set the initial model as a classification model if the difference between the image category and the training label of the image to be trained is less than a preset training threshold.

[0316] Preferably, the first training module 82 includes:

[0317] The first training input unit 821 is used to obtain the convolutional input vector based on the sampling input layer in the downsampling module;

[0318] The first training compression and extraction unit 822 is used to compress the convolutional input vector in the sampling input layer with a sampling stride based on the sampling feature module in the downsampling module, and to extract features from the compressed convolutional input vector to obtain a sampling feature vector; wherein, the sampling feature vector is a feature vector for preliminary feature extraction of the compressed training image.

[0319] The first training pooling unit 823 is used to perform pooling processing on the convolutional input vector based on the pooling module in the downsampling module to obtain a pooling vector; wherein, the pooling vector represents the abstracted image to be trained;

[0320] The first training aggregation unit 824 is used to aggregate the sampled feature vectors and pooling vectors to obtain downsampled training vectors.

[0321] Preferably, the first training compression and extraction unit 822 includes:

[0322] The first training compression extraction subunit 8221 is used to split the convolution input vector based on the sampling split compression module in the sampling feature module, and to extract and merge the features of the split convolution input vector with a sampling stride to obtain sampled compressed convolution data; wherein, the sampled compressed convolution data is used to characterize the compressed training image;

[0323] The first training dilation extraction subunit 8222 is used to split the sampled compressed convolutional data based on the sampled dilation merging module in the sampled feature module, and extract and merge the features of the split sampled compressed convolutional data with the sampling stride to obtain the sampled feature vector.

[0324] Preferably, the first training pooling unit 823 includes:

[0325] The first training average pooling unit 8231 is used to perform pooling processing on the convolutional input vector based on the average pooling module in the pooling module to obtain average pooling data; wherein, the average pooling data represents the abstracted image to be trained.

[0326] The first training convolutional pooling unit 8232 is used to perform convolution processing on the average pooled data based on the pooling convolution kernel in the pooling module to obtain the pooling vector.

[0327] Preferably, the first training compression extraction subunit 8221 includes:

[0328] The first training compression and splitting subunit 82211 is used to split the convolutional input vector based on the sampling target splitting module in the sampling splitting compression module to obtain at least one sampling target splitting data; wherein, the sampling target splitting data represents a part of the image to be trained;

[0329] The first training compression processing subunit 82212 is used to perform compression convolution processing on each sampled target split data based on the sampled compression convolution module in the sampled split compression module to obtain at least one sampled split compressed convolution data; wherein, the sampled split compressed convolution data represents the compressed part of the image to be trained.

[0330] The first training compression and merging subunit 82213 is used to merge the sampled split compression convolutional data into a sampled compressed merged data based on the sampled split compression module in the sampled compression and merging module; wherein, the sampled compressed merged data represents the compressed part of the training image after merging.

[0331] The first training compression convolution subunit 82214 is used to perform convolution processing on the sampled compressed and merged data based on the sampled split compression module in the sampled merge convolution module to obtain a sampled compressed convolution data.

[0332] Preferably, the first training dilation extraction subunit 8222 includes:

[0333] The first training dilation splitting subunit 82221 is used to split the sampled compressed convolutional data based on the sampled convolutional splitting module in the sampled feature module to obtain at least one sampled convolutional splitting data; wherein, the sampled convolutional splitting data represents the compressed training image after further splitting.

[0334] The first training dilation processing subunit 82222 is used to perform dilation convolution processing on each sampled convolution split data based on the sampled dilation convolution module in the sampled feature module to obtain at least one sampled dilation convolution data; wherein, the sampled dilation convolution data represents the compressed training image after being split again to restore the dimension to the convolution input vector.

[0335] The first training dilation and merging subunit 82223 is used to merge sampled dilation and merging convolutional data based on the sampled dilation and merging module in the sampled feature module to obtain a sampled feature vector.

[0336] Preferably, the second training module 83 includes:

[0337] The second training input unit 831 is used to obtain downsampled training vectors based on the ordinary input layer in the ordinary module;

[0338] The second training compression extraction unit 832 is used to split the downsampled training vector based on the ordinary split compression module in the ordinary module, and to extract features from the split downsampled training vector with an ordinary stride and merge them to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to represent the compressed downsampled training vector.

[0339] The second training dilation extraction unit 833 is used to split the ordinary compressed convolutional data based on the ordinary dilation merging module in the ordinary module, and to extract and merge the features of the split ordinary compressed convolutional data with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is a feature vector for preliminary feature extraction of the compressed downsampled training vector.

[0340] The second training aggregation unit 834 is used to aggregate the downsampled training vectors and ordinary feature vectors in the ordinary input layer to obtain the image training vectors.

[0341] Preferably, the second training compression and extraction unit 832 includes:

[0342] The second training compression and splitting subunit 8321 is used to split the downsampled training vector based on the ordinary target splitting module in the ordinary splitting and compression module to obtain at least one ordinary target splitting data; wherein, the ordinary target splitting data represents a part of the downsampled attribute vector;

[0343] The second training compression processing subunit 8322 is used to perform compression convolution processing on each ordinary target split data based on the ordinary compression convolution module in the ordinary split compression module to obtain at least one ordinary split compressed convolution data; wherein, the ordinary split compressed convolution data represents the compressed partial downsampled training vector.

[0344] The second training compression and merging subunit 8323 is used to merge ordinary split-compressed convolutional data into ordinary compressed and merged data based on the ordinary compression and merging module in the ordinary split-compression module; wherein, the ordinary compressed and merged data represents the merged compressed partial downsampled training vector.

[0345] The second training compression convolution subunit 8324 is used to perform convolution processing on ordinary compressed and merged data based on the ordinary merge convolution module in the ordinary split compression module to obtain ordinary compressed convolution data.

[0346] Preferably, the second training dilation extraction subunit 833 includes:

[0347] The second training dilation splitting subunit 8331 is used to split ordinary compressed convolutional data into at least one ordinary convolution splitting data based on the ordinary convolution splitting module in the ordinary dilation merging module; wherein, the ordinary convolution splitting data represents the compressed downsampled training vector after further splitting.

[0348] The second training dilation processing subunit 8332 is used to perform dilation convolution processing on each ordinary convolution split data based on the ordinary dilation convolution module in the ordinary dilation merging module to obtain at least one ordinary dilation convolution data; wherein, the ordinary dilation convolution data represents the compressed downsampled training vector that is re-splitted to restore the dimension of the convolution input vector.

[0349] The second training dilation merging subunit 8333 is used to merge ordinary dilated convolution data to obtain an ordinary feature vector based on the ordinary dilation merging module in the ordinary dilation merging module.

[0350] Example 5:

[0351] To achieve the above objectives, this application also provides a computer device 9, including: a processor 92 and a memory 91 communicatively connected to the processor 92; the memory stores computer-executed instructions;

[0352] The processor executes computer execution instructions stored in memory 91 to implement the aforementioned image classification method based on a classification model and the training method for the classification model applied to image classification. The components of the image classification device based on the classification model and the training device for the classification model applied to image classification can be distributed across different computer devices. Computer device 9 can be a smartphone, tablet, laptop, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple application servers), etc. The computer device in this embodiment includes, but is not limited to, a memory 91 and a processor 92 that can communicate with each other via a system bus. Figure 6 As shown. It should be noted that, Figure 6Only computer devices with components are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead. In this embodiment, memory 91 (i.e., readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, memory 91 may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, memory 91 may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Of course, memory 91 may also include both internal storage units and external storage devices of the computer device. In this embodiment, the memory 91 is typically used to store the operating system and various application software installed on the computer device, such as the program code of the image classification device based on the classification model and the training device for the classification model applied to image classification in Embodiments 3 and 4. In addition, the memory 91 can also be used to temporarily store various types of data that have been output or will be output. The processor 92 in some embodiments may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 92 is typically used to control the overall operation of the computer device. In this embodiment, the processor 92 is used to run the program code stored in the memory 91 or process data, for example, to run the image classification device based on the classification model and the training device for the classification model applied to image classification, to implement the image classification method based on the classification model and the training method for the classification model applied to image classification in the above embodiments.

[0353] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application. It should be understood that the processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor. The memory may include high-speed RAM, and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disk, etc.

[0354] To achieve the above objectives, this application also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores computer-executable instructions. When the program is executed by processor 92, it implements the corresponding function. The computer-readable storage medium of this embodiment is used to store computer-executable instructions for implementing a classification model-based image classification method and a training method for the classification model applied to image classification. When executed by processor 92, it implements the classification model-based image classification method and the training method for the classification model applied to image classification of the above embodiments.

[0355] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0356] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0357] This application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image classification method based on a classification model and the training method for the classification model applied to image classification.

[0358] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0359] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0360] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An image classification method based on a classification model, characterized in that, include: The process involves acquiring an image to be processed, inputting the image to be processed into a preset classification model, and obtaining a convolutional input vector from the input layer of the classification model; wherein the classification model is a convolutional neural network model; and the convolutional input vector is used to represent the image to be processed. The downsampling module in the classification model compresses and extracts features from the convolutional input vector to obtain a downsampling output vector; wherein, the downsampling output vector is a feature vector for preliminary feature extraction of the compressed image to be processed, and a representation of the abstracted image to be processed. Based on the common module in the classification model, feature extraction is performed on the downsampled output vector to obtain an image feature vector; wherein, the image feature vector is used to characterize the shape and contour of the object in the image to be processed; The image feature vector is classified based on the fully connected layer in the classification model to obtain the image category of the image to be processed; wherein, the image category reflects the type or kind of object in the image to be processed; The step of extracting features from the downsampled output vector based on the ordinary module in the classification model to obtain an image feature vector includes: The downsampled output vector is obtained based on the ordinary input layer in the ordinary module. The downsampled output vector is split based on the ordinary splitting and compression module in the ordinary module, and features are extracted and merged on the split downsampled output vector with an ordinary stride to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to characterize the compressed downsampled output vector; The ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and the split ordinary compressed convolutional data is subjected to feature extraction and merging with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is the feature vector for preliminary feature extraction of the compressed downsampled output vector; The image feature vector is obtained by summing the downsampled output vector from the ordinary input layer and the ordinary feature vector.

2. The image classification method according to claim 1, characterized in that, The downsampling module in the classification model performs compression and feature extraction on the convolutional input vector to obtain a downsampled output vector, including: The convolutional input vector is obtained based on the sampling input layer in the downsampling module; Based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and the compressed convolutional input vector is used to extract features to obtain a sampling feature vector; wherein, the sampling feature vector is a feature vector for preliminary feature extraction of the compressed image to be processed; Based on the pooling module in the downsampling module, the convolutional input vector is pooled to obtain a pooling vector; wherein, the pooling vector represents the abstracted image to be processed; The sampled feature vector and the pooling vector are combined to obtain the downsampled output vector.

3. A training method for a classification model applied to image classification, characterized in that, include: A training image is obtained, the training image is input into a preset initial model, and a convolutional input vector is obtained from the input layer of the initial model; wherein, the initial model is a convolutional neural network model; the convolutional input vector is used to represent the training image; The convolutional input vector is compressed and its features are extracted based on the downsampling module in the initial model to obtain a downsampling training vector; wherein, the downsampling training vector is a feature vector for preliminary feature extraction of the compressed image to be trained, and a representation of the abstracted image to be trained; Based on the common modules in the initial model, feature extraction is performed on the downsampled training vector to obtain the image training vector; wherein, the image training vector is used to characterize the shape and contour of the object in the image to be trained; The image training vectors are classified based on the fully connected layers in the initial model to obtain the image category of the image to be trained; wherein, the image category reflects the type or category of the object in the image to be trained; If the difference between the image category of the image to be trained and the training label of the image to be trained is less than a preset training threshold, then the initial model is set as the classification model; The step of extracting features from the downsampled training vector based on the ordinary modules in the initial model to obtain the image training vector includes: The downsampled training vector is obtained based on the ordinary input layer in the ordinary module. The downsampled training vector is split based on the ordinary splitting and compression module in the ordinary module, and features are extracted and merged on the split downsampled training vector with an ordinary stride to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to represent the compressed downsampled training vector. The ordinary compressed convolutional data is split based on the ordinary dilation and merging module in the ordinary module, and the split ordinary compressed convolutional data is subjected to feature extraction and merging with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is a feature vector for preliminary feature extraction of the compressed downsampled training vector. The image training vector is obtained by summing the downsampled training vectors from the ordinary input layer and the ordinary feature vectors.

4. The training method according to claim 3, characterized in that, The step of compressing and extracting features from the convolutional input vector based on the downsampling module in the initial model to obtain a downsampled training vector includes: The convolutional input vector is obtained based on the sampling input layer in the downsampling module; Based on the sampling feature module in the downsampling module, the convolutional input vector in the sampling input layer is compressed with a sampling stride, and the compressed convolutional input vector is used to extract features to obtain a sampling feature vector; wherein, the sampling feature vector is a feature vector for preliminary feature extraction of the compressed image to be trained; Based on the pooling module in the downsampling module, the convolutional input vector is pooled to obtain a pooling vector; wherein, the pooling vector represents the abstracted image to be trained; The sampled feature vector and the pooling vector are combined to obtain the downsampled training vector.

5. An image classification device based on a classification model, characterized in that, include: An input module is used to acquire an image to be processed, input the image to be processed into a preset classification model, and acquire a convolutional input vector from the input layer of the classification model; wherein, the classification model is a convolutional neural network model; and the convolutional input vector is used to represent the image to be processed. The first processing module is used to compress and extract features from the convolutional input vector based on the downsampling module in the classification model to obtain a downsampled output vector; wherein, the downsampled output vector is a feature vector for preliminary feature extraction of the compressed image to be processed, and a representation of the abstracted image to be processed; The second processing module is used to extract features from the downsampled output vector based on the ordinary module in the classification model to obtain an image feature vector; wherein the image feature vector is used to characterize the shape and contour of the object in the image to be processed; The third processing module is used to classify the image feature vector based on the fully connected layer in the classification model to obtain the image category of the image to be processed; wherein, the image category reflects the type or category of the object in the image to be processed; The second processing module includes: The second input unit is used to obtain the downsampled output vector based on the ordinary input layer in the ordinary module; The second compression and extraction unit is used to split the downsampled output vector based on the ordinary splitting and compression module in the ordinary module, and to extract features from the split downsampled output vector with an ordinary stride and merge them to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to characterize the compressed downsampled output vector; The second dilation extraction unit is used to split the ordinary compressed convolutional data based on the ordinary dilation merging module in the ordinary module, and to extract and merge the features of the split ordinary compressed convolutional data with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is a feature vector for preliminary feature extraction of the compressed downsampled output vector. The second summarization unit is used to summarize the downsampled output vector in the ordinary input layer and the ordinary feature vector to obtain the image feature vector.

6. A training device for a classification model applied to image classification, characterized in that, include: The training input module is used to acquire the image to be trained, input the image to be trained into a preset initial model, and acquire a convolutional input vector from the input layer of the initial model; wherein, the initial model is a convolutional neural network model; and the convolutional input vector is used to represent the image to be trained. The first training module is used to compress and extract features from the convolutional input vector based on the downsampling module in the initial model to obtain a downsampled training vector; wherein, the downsampled training vector is a feature vector for preliminary feature extraction of the compressed image to be trained, and a representation of the abstracted image to be trained; The second training module is used to extract features from the downsampled training vector based on the ordinary module in the initial model to obtain an image training vector; wherein the image training vector is used to characterize the shape and contour of the object in the image to be trained; The third training module is used to classify the image training vectors based on the fully connected layers in the initial model to obtain the image category of the image to be trained; wherein, the image category reflects the type or category of objects in the image to be trained; The model generation module is used to set the initial model as the classification model if the difference between the image category of the image to be trained and the training label of the image to be trained is less than a preset training threshold. The second training module includes: The second training input unit is used to obtain the downsampled training vector based on the ordinary input layer in the ordinary module; The second training compression and extraction unit is used to split the downsampled training vector based on the ordinary splitting and compression module in the ordinary module, and to extract features from the split downsampled training vector with an ordinary stride and merge them to obtain ordinary compressed convolutional data; wherein, the ordinary compressed convolutional data is used to characterize the compressed downsampled training vector. The second training dilation extraction unit is used to split the ordinary compressed convolutional data based on the ordinary dilation merging module in the ordinary module, and to extract and merge the features of the split ordinary compressed convolutional data with an ordinary stride to obtain an ordinary feature vector; wherein, the ordinary feature vector is a feature vector for preliminary feature extraction of the compressed downsampled training vector. The second training aggregation unit is used to aggregate the downsampled training vectors and the ordinary feature vectors in the ordinary input layer to obtain the image training vectors.

7. A computer device, characterized in that, include: A processor and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 4.

Images