A lightweight network-based object classification method

Abstract

This invention provides an object classification method based on lightweight networks, comprising the following steps: constructing a lightweight network, acquiring and processing sample images, training the lightweight network using the processed sample images, obtaining a trained lightweight network, and using the trained lightweight network to extrapolate the image of the object to be identified: extracting multiple different image features by dilated convolution of the image of the object to be identified according to multiple preset dilation rates, then assigning weights to the image features using adaptive weighting and performing feature fusion, finally outputting the classification result. This invention provides an object classification method based on lightweight networks, solving the problem of low classification accuracy achieved by existing object classification techniques in low-resource environments.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to an object classification method based on lightweight networks. Background Technology

[0002] Artificial intelligence (AI) is a technical science that studies and develops theories, methods, technologies, and application systems that can simulate, extend, and expand human intelligence. Its research aims to enable intelligent machines to hear (speech recognition, machine translation, etc.), see (image recognition, text recognition, etc.), speak (speech synthesis, human-computer dialogue, etc.), think (human-computer chess, theorem proving, etc.), learn (machine learning, knowledge representation, etc.), and act (robots, self-driving cars, etc.). One of the most popular commercial AI technologies currently is the ability to perform AI locally, without requiring devices to be connected to the internet for large-scale computation in cloud data centers. This allows for real-time environmental perception, human-computer interaction, and decision-making control without an internet connection.

[0003] Artificial intelligence systems can be trained using data from well-functioning machines that have experienced malfunctions, enabling them to identify potential problems after acquiring real-time data. While neural networks running on servers are currently a popular choice, they are not the only available AI solution, such as deep neural networks for embedded systems. However, deep neural networks for embedded systems have very high requirements for computing power and resources, leading to increased system power consumption; processors supporting deep neural network acceleration are typically complex SoCs integrating multiple architectures, making them extremely expensive.

[0004] Therefore, most existing object classification technologies use complex and expensive chips, which are difficult to implement on low-resource and low-cost chips and are not suitable for single tasks, easily leading to resource waste. On the other hand, object classification technologies that can be implemented in low-resource environments use simple models, single training and evaluation strategies, and can achieve lower classification accuracy. Summary of the Invention

[0005] To overcome the technical shortcomings of existing object classification technologies, which suffer from low classification accuracy in low-resource environments, this invention provides an object classification method based on lightweight networks.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] An object classification method based on lightweight networks includes the following steps:

[0008] A lightweight network is constructed, sample images are collected and processed, and the lightweight network is trained using the processed sample images to obtain a trained lightweight network. The trained lightweight network is then used to extrapolate from images of the objects to be recognized.

[0009] Based on multiple preset dilation rates, dilated convolution is performed on the image of the object to be identified to extract multiple different image features. Then, the image features are weighted using an adaptive weighting method and feature fusion is performed to finally output the classification result.

[0010] In the above scheme, a lightweight network is used to perform dilated convolution on the image of the object to be identified according to multiple preset dilation rates to extract multiple different levels of image features. This enhances the object classification ability while maintaining high real-time performance. The network structure is simple and has low computational cost, enabling better real-time and efficient object classification on embedded terminal devices. At the same time, by fusing image features at different levels, the ability to perceive image details is improved.

[0011] Preferably, the lightweight network includes a first convolutional layer, a normalization layer, a first block layer, a second block layer, a pooling layer, a second convolutional layer, and a third block layer connected in sequence; it also includes a first dilated convolutional layer, a second dilated convolutional layer, a third dilated convolutional layer, a fusion layer, and a fully connected layer; wherein,

[0012] The branch output of the first block layer is connected to the input of the first dilated convolutional layer, the branch output of the second block layer is connected to the input of the second dilated convolutional layer, the branch output of the third block layer is connected to the input of the third dilated convolutional layer, the outputs of the first dilated convolutional layer, the second dilated convolutional layer, and the third dilated convolutional layer are respectively connected to the input of the fusion layer, and the output of the fusion layer is connected to the input of the fully connected layer.

[0013] Preferably, the first block layer, the second block layer, and the third block layer each include a depthwise separable layer, an upscaling convolutional layer, a regularized convolutional layer, a downsampling layer, an downscaling convolutional layer, two fully connected layers, an attention layer, and two residual layers; in any block layer,

[0014] A depthwise separable layer, an upscaling convolutional layer, a regularized convolutional layer, a downsampling layer, a downsampling convolutional layer, a fully connected layer, and another fully connected layer are connected in sequence. The output of the depthwise separable layer is also connected to the input of another fully connected layer through a residual layer. The output of the regularized convolutional layer is also connected to the input of a fully connected layer through another residual layer. The output of the upscaling convolutional layer is also connected to the input of the downscaling convolutional layer through a delimiter layer.

[0015] Preferably, assuming the image size of the object to be identified is X*X, the output size is W*W, the kernel size is N*N, the stride is Stride, the padding is Padding, and the dilation rate is L, then the definition of dilated convolution is...

[0016]

[0017] Preferably, the new feature matrix O obtained after feature fusion is:

[0018] O=α*F 1 +β*F 2 +γ*F 3

[0019] α,β,γ∈[0,1]

[0020] α+β+γ=1

[0021] Among them, F 1 F represents the feature matrix output by the first block layer. 2 F represents the feature matrix output by the second block layer. 3 This represents the feature matrix output by the third Block layer, and α represents F. 1 The weights, β represents F 2 The weights, γ represents F 3 The weight.

[0022] Preferably, the steps for processing sample images include:

[0023] A1: Convert the sample image to grayscale to obtain a grayscale image;

[0024] A2: Randomly divide the grayscale image into training sample images and test sample images;

[0025] A3: Perform image augmentation on the training sample images to obtain enhanced training sample images;

[0026] A4: Perform Gaussian pyramid downsampling on the enhanced training sample images to obtain downsampled training sample images.

[0027] Preferably, the image augmentation operation includes: randomly rotating vertically, randomly rotating horizontally, randomly adding Gaussian filtering, randomly cropping, randomly erasing the image with a 20% probability, and randomly changing the brightness, contrast, and saturation of the image with a 30% probability.

[0028] Preferably, Gaussian pyramid downsampling of the enhanced training sample image specifically involves: using a Gaussian low-pass filter to perform Gaussian smoothing on the enhanced training sample image, and then sequentially processing the Gaussian smoothed enhanced training sample image through downsampling, max pooling, average pooling, label smoothing, and SGD optimizer.

[0029] Preferably, the steps for training a lightweight network include:

[0030] B1: Set the loss function, optimal loss function value, threshold for the number of times the optimal loss function value is replaced, and maximum number of training iterations for the lightweight network;

[0031] B2: Input the downsampled training sample images into the lightweight network to obtain the current loss function value;

[0032] B3: Compare the current loss function value with the optimal loss function value. If the current loss function value is less than the optimal loss function value, replace the optimal loss function value with the current loss function value and record the number of replacements. Otherwise, do not perform a replacement.

[0033] B4: Compare the number of replacements with the threshold for the number of replacements of the optimal loss function value. If the number of replacements is less than the threshold for the number of replacements of the optimal loss function value, return to step S2 for the next round of training; otherwise, the lightweight network training is complete, and the trained lightweight network is obtained.

[0034] Preferably, when training a lightweight network, the Dropout picking method is used in the convolutional layer, normalization layer, pooling layer, fully connected layer, and attention layer.

[0035] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0036] This invention provides an object classification method based on a lightweight network. By using a lightweight network to perform dilated convolution on images of objects to be identified according to multiple preset dilation rates, multiple image features at different levels are extracted. This method enhances the object classification ability while maintaining high real-time performance. The network structure is simple and has low computational cost, enabling better real-time and efficient object classification on embedded terminal devices. At the same time, by fusing image features at different levels, the ability to perceive image details is improved. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the network structure of the lightweight network in this invention. Detailed Implementation

[0038] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0039] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0040] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0041] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] Example 1

[0043] like Figure 1 As shown, an object classification method based on lightweight networks includes the following steps:

[0044] A lightweight network (Mixture-MobileNetV3) is constructed. Sample images are acquired and processed. The lightweight network (Mixture-MobileNetV3) is trained using the processed sample images, resulting in a pre-trained lightweight network (Mixture-MobileNetV3). The pre-trained lightweight network (Mixture-MobileNetV3) is then used to extrapolate from images of the objects to be recognized.

[0045] Based on multiple preset dilation rates, dilated convolution is performed on the image of the object to be identified to extract multiple different image features. Then, the image features are weighted using an adaptive weighting method and feature fusion is performed to finally output the classification result.

[0046] In the specific implementation process, a lightweight network (Mixture-MobileNetV3) is used to perform dilated convolution on the image of the object to be identified according to multiple preset dilation rates, thereby extracting image features at multiple different levels. This enhances the object classification ability while maintaining high real-time performance. The network structure is simple and has low computational cost, enabling better real-time and efficient object classification on embedded terminal devices. At the same time, by fusing image features at different levels, the ability to perceive image details is improved.

[0047] Example 2

[0048] An object classification method based on lightweight networks includes the following steps:

[0049] A lightweight network (Mixture-MobileNetV3) is constructed. Sample images are acquired and processed. The lightweight network (Mixture-MobileNetV3) is trained using the processed sample images, resulting in a pre-trained lightweight network (Mixture-MobileNetV3). The pre-trained lightweight network (Mixture-MobileNetV3) is then used to extrapolate from images of the objects to be recognized.

[0050] Based on multiple preset dilation rates, dilated convolution is performed on the image of the object to be identified to extract multiple different image features. Then, the image features are weighted using an adaptive weighting method and feature fusion is performed to finally output the classification result.

[0051] More specifically, the lightweight network (Mixture-MobileNetV3) includes a first convolutional layer, a normalization layer, a first block layer, a second block layer, a pooling layer, a second convolutional layer, and a third block layer connected in sequence; it also includes a first dilated convolutional layer, a second dilated convolutional layer, a third dilated convolutional layer, a fusion layer, and a fully connected layer; wherein,

[0052] The branch output of the first block layer is connected to the input of the first dilated convolutional layer, the branch output of the second block layer is connected to the input of the second dilated convolutional layer, the branch output of the third block layer is connected to the input of the third dilated convolutional layer, the outputs of the first dilated convolutional layer, the second dilated convolutional layer, and the third dilated convolutional layer are respectively connected to the input of the fusion layer, and the output of the fusion layer is connected to the input of the fully connected layer.

[0053] More specifically, the first block layer, the second block layer, and the third block layer each include a depthwise separable layer, an upscaling convolutional layer, a regularized convolutional layer, a downsampling layer, an downscaling convolutional layer, two fully connected layers, an attention layer, and two residual layers; in any block layer,

[0054] A depthwise separable layer, an upscaling convolutional layer, a regularized convolutional layer, a downsampling layer, a downsampling convolutional layer, a fully connected layer, and another fully connected layer are connected in sequence. The output of the depthwise separable layer is also connected to the input of another fully connected layer through a residual layer. The output of the regularized convolutional layer is also connected to the input of a fully connected layer through another residual layer. The output of the upscaling convolutional layer is also connected to the input of the downscaling convolutional layer through a delimiter layer.

[0055] More specifically, assuming the image size of the object to be identified is X*X, the output size is W*W, the kernel size is N*N, the stride is Stride, the padding is Padding, and the dilation rate is L, then the dilated convolution is defined as follows:

[0056]

[0057] More specifically, the new feature matrix O obtained after feature fusion is:

[0058] O=α*F 1 +β*F 2 +γ*F 3

[0059] α,β,γ∈[0,1]

[0060] α+β+γ=1

[0061] Among them, F 1 F represents the feature matrix output by the first block layer. 2 F represents the feature matrix output by the second block layer. 3 This represents the feature matrix output by the third Block layer, and α represents F. 1 The weights, β represents F 2 The weights, γ represents F 3 The weight.

[0062] Example 3

[0063] An object classification method based on lightweight networks includes the following steps:

[0064] A lightweight network (Mixture-MobileNetV3) is constructed. Sample images are acquired and processed. The lightweight network (Mixture-MobileNetV3) is trained using the processed sample images, resulting in a pre-trained lightweight network (Mixture-MobileNetV3). The pre-trained lightweight network (Mixture-MobileNetV3) is then used to extrapolate from images of the objects to be recognized.

[0065] Based on multiple preset dilation rates, dilated convolution is performed on the image of the object to be identified to extract multiple different image features. Then, the image features are weighted using an adaptive weighting method and feature fusion is performed to finally output the classification result.

[0066] More specifically, the steps for processing sample images include:

[0067] A1: Convert the sample image to grayscale to obtain a grayscale image;

[0068] In practice, the acquired color images of the objects are 320×240 RGB256 format color images, with each pixel occupying 16 bits of storage space. Red, green, and blue occupy bits 0-4, 5-10, and 11-15, respectively. A weighted average of the R, G, and B values ​​is applied. Since the human eye is most sensitive to green, followed by red, and least sensitive to blue, a weighting of wG > wR > wB will result in a more easily recognizable grayscale image. The weights for converting the color image to grayscale are wB = 0.114 (blue), wG = 0.578 (green), and wR = 0.299 (red); "end" indicates the end of a grayscale image.

[0069] A2: Randomly divide the grayscale image into training sample images and test sample images;

[0070] In practice, the grayscale images are randomly divided into training sample images and test sample images in an 8:2 ratio;

[0071] A3: Perform image augmentation on the training sample images to obtain enhanced training sample images;

[0072] A4: Perform Gaussian pyramid downsampling on the enhanced training sample images to obtain downsampled training sample images.

[0073] More specifically, image augmentation operations include: random vertical rotation, random horizontal rotation, random addition of Gaussian filtering, random cropping, random image erasure with a 20% probability, and random changes to the image's brightness, contrast, and saturation with a 30% probability.

[0074] More specifically, Gaussian pyramid downsampling of the augmented training sample images involves: using a Gaussian low-pass filter to smooth the augmented training sample images using Gaussian smoothing; and then sequentially processing the Gaussian smoothed augmented training sample images through downsampling, max pooling, average pooling, label smoothing, and the SGD optimizer.

[0075] In the specific implementation, each training sample image is randomly rotated vertically, horizontally, Gaussian filtered, and cropped. Several enhanced training sample images are generated from a single training sample image, increasing the scale and complexity of the training data and improving the accuracy of the neural network. Image downsampling generally involves two steps: applying a Gaussian convolution kernel to the image; and deleting all even-numbered rows and columns. This yields downsampled training sample images. Label smoothing is also used, which encourages representing training examples of the same class as close clusters. This results in the loss of information about the similarity between instances of different classes in the logits (the logarithm of the ratio of an event occurring to the event not occurring), which is necessary to improve the model's classification accuracy without reducing the model's generalization and calibration capabilities. The core idea of ​​label smoothing is to transform the hard target (the original label, i.e., the numerical values ​​0, 1, ...) into a soft target (a flexible label after weighted averaging). During training, a region of moments in the image is randomly selected and its pixels are erased. This process generates training images with different degrees of occlusion, reducing the risk of overfitting and making the model robust to occlusion. Random erasure requires no parameter learning and can be used in conjunction with most deep learning recognition models. It randomly changes the brightness, contrast, and saturation of the image. This allows the classified object image to exhibit different levels of detail, simulating the real-world appearance of the classified object under various environments and lighting conditions. By sequentially performing Gaussian pyramid downsampling, average pooling, and max pooling operations on each enhanced training sample image, the size of the training data is reduced, thus decreasing computational costs.

[0076] More specifically, the steps for training the lightweight network (Mixture-MobileNetV3) include:

[0077] B1: Set the loss function, optimal loss function value, threshold for the number of times the optimal loss function value is replaced, and maximum number of training iterations for the lightweight network (Mixture-MobileNetV3); use fast random weighted averaging for the lightweight network (Mixture-MobileNetV3);

[0078] In practice, the threshold for the number of times the optimal loss function value is replaced is 20 times, and the maximum number of training iterations is 50,000. The optimal loss function value can be set as needed.

[0079] In practice, the key to Fast Stochastic Weighted Average (FSA) lies in using a learning rate schedule with a longer period to increase the distance between the average weights and the diversity of the corresponding predictions; averaging the weights of multiple networks within each period (while stochastic weighted average only averages the learning rate corresponding to each period); the SGD optimizer (Stochastic Gradient Descent) randomly selects a set of samples, trains it, updates it once according to the gradient, then selects another set and updates it again. With a very large sample size, it may be possible to obtain a model with an acceptable loss value without training all samples. Weight updates no longer involve traversing the entire dataset, but instead select a single sample. Generally, its step size is smaller than that of gradient descent because gradient descent uses the exact gradient, allowing it to iterate significantly towards the global optimum (when the problem is convex). Stochastic gradient descent, however, uses an approximate gradient, or sometimes it may not be moving in the direction of gradient descent at all, thus progressing more slowly. This also means that compared to gradient descent, it is less prone to getting trapped in local optima.

[0080] B2: Input the downsampled training sample images into the lightweight network (Mixture-MobileNetV3) to obtain the current loss function value;

[0081] In practice, the PolyLoss loss function is used to calculate the loss function value of the downsampled training images. The PolyLoss formula is:

[0082] L = -log(P) t )+x j *(1-P t ) i

[0083] PolyLoss is a monotonically decreasing function on [0,1], which can be expressed using x. j Regulation function.

[0084] B3: Compare the current loss function value with the optimal loss function value. If the current loss function value is less than the optimal loss function value, replace the optimal loss function value with the current loss function value and record the number of replacements. Otherwise, do not perform a replacement.

[0085] B4: Compare the number of replacements with the threshold for the number of replacements of the optimal loss function value. If the number of replacements is less than the threshold for the number of replacements of the optimal loss function value, return to step S2 for the next round of training. Otherwise, the lightweight network (Mixture-MobileNetV3) training is completed, and the trained lightweight network (Mixture-MobileNetV3) is obtained.

[0086] In this embodiment, the test.py file is used to test the test set data and evaluate its performance.

[0087] More specifically, when training the lightweight network (Mixture-MobileNetV3), Dropout is used in the convolutional layers, normalization layers, pooling layers, fully connected layers, and attention layers.

[0088] In practice, Dropout is a technique that randomly disconnects neurons, used only during model training. It improves the model's fitting ability by randomly deleting a portion of the neurons in a layer. Specifically, 1%-5% of neurons are randomly disconnected during each iteration of training. Because the neurons disconnected in Dropout are different each time, this technique reduces the likelihood of neurons coupling together to extract the same feature, allowing the network to make correct inferences even with missing information. Furthermore, when processing test set samples, the network's inferences are actually the average of the inferences made by the subnetworks at different neuron disconnection points, thus helping to prevent overfitting.

[0089] More specifically, the lightweight network (Mixture-MobileNetV3) includes a first convolutional layer, a normalization layer, a first block layer, a second block layer, a pooling layer, a second convolutional layer, and a third block layer connected in sequence; it also includes a first dilated convolutional layer, a second dilated convolutional layer, a third dilated convolutional layer, a fusion layer, and a fully connected layer; wherein,

[0090] The branch output of the first block layer is connected to the input of the first dilated convolutional layer, the branch output of the second block layer is connected to the input of the second dilated convolutional layer, the branch output of the third block layer is connected to the input of the third dilated convolutional layer, the outputs of the first dilated convolutional layer, the second dilated convolutional layer, and the third dilated convolutional layer are respectively connected to the input of the fusion layer, and the output of the fusion layer is connected to the input of the fully connected layer.

[0091] In the specific implementation, the block layer uses a residual network and attention layer design. Residual units in the residual network can be implemented with skip connections, meaning the unit's input is directly added to its output. The residual network effectively solves the degradation problem of deep neural networks. Considering its use on embedded devices, it was redesigned to reduce computational cost with the same number of layers while maintaining accuracy. During model training, image information is continuously lost. The residual layer is used to fuse high-dimensional and low-dimensional image features, ensuring the model can grasp global image information through continuous iteration. Dilated convolutional layers are used to extract feature information from different layers of the backbone network. Dilated convolutions with different dilation rates process the extracted feature information, thereby improving the utilization of image spatial information and obtaining more semantic and localization information. The fusion layer assigns different weight coefficients to the processed feature information through adaptive weight assignment, resulting in richer feature information.

[0092] The h-swish activation function is set up in the first convolutional layer, normalization layer, first block layer, second block layer, pooling layer, second convolutional layer, third block layer, and fully connected layer. The data processed in each layer is used only once, introducing a non-linear factor. This allows the lightweight network (Mixture-MobileNetV3) to fit all non-linear functions. It helps prevent slow training and gradients from gradually approaching zero, and saturation smoothness plays an important role in optimization and generalization. The formula for the h-swish activation function is:

[0093]

[0094] ReLU6 = min(6, max(0, x))

[0095] Where x is an element of the image feature matrix.

[0096] The normalization layer, known as Batch Normalization (BN), accelerates training, provides some regularization against overfitting, and allows for larger learning rates. However, it generally requires a larger batch size to maintain both speed and accuracy. As the parameters of the earlier layers change, the input distribution of each layer changes during training, complicating the training of deep neural networks. This slows down training, requires lower learning rates and parameter initialization for each layer, and makes training models with convergent non-linearity difficult. As part of the model architecture, the BN layer performs normalization on each training mini-batch during training, normalizing the input distribution of each layer to N(0,1) to reduce so-called "internal covariate drift." Batch normalization allows the model to use higher learning rates and is less careful during initialization. It also acts as a regularization mechanism. More precisely, BN reduces the L constant in the gradient of the neural network, making it easier for the network to learn, for example, allowing for larger learning rates. Intuitively, reducing the L constant of the gradient makes the loss function smoother, resulting in more predictable and stable gradients.

[0097] The image, after being downsampled by a Gaussian pyramid, is input into a lightweight network (Mixture-MobileNetV3). First, a dimensionality-increasing operation is performed through the first convolutional layer to obtain the first feature v1. The first feature v2 is then passed through a normalization layer to obtain the third feature v3. The third feature v3 is then passed through the first block layer to obtain the fourth feature v4. The fourth feature v4 is then passed through the second block layer to obtain the fifth feature v5. The fifth feature v5 is then passed through a pooling layer to obtain the sixth feature v6. The sixth feature v6 is then passed through the second convolutional layer to obtain the seventh feature v7. The seventh feature v7 is then passed through the third block layer to obtain the eighth feature v8. The fourth feature v4 is then passed through the first dilated convolutional layer to obtain the ninth feature v9. The fifth feature v5 is then passed through the second dilated convolutional layer to obtain the tenth feature v9. 10 The eighth feature v8 is used to obtain the eleventh feature v through the third dilated convolutional layer. 11 Ninth feature v9, tenth feature v 10 and the eleventh feature v 11 The twelfth feature v is obtained through the fusion layer. 12 ; Twelfth feature v 12 The thirteenth feature v is obtained through a fully connected layer. 13 Among them, the thirteenth feature v 13 The dimension is the number of categories of objects to be identified.

[0098] Before scaling up the network layers, this embodiment performs Deep Compression weight pruning and Quantization weight quantization techniques. Deep Compression weight pruning retains only some important connections. In the parameter distribution of a typical convolutional layer, most weights are concentrated around 0, contributing little to the network. By pruning, the unimportant, excessively small weights around 0 are set to 0, preventing them from being activated. The focus is then placed on training other non-zero weights, thereby compressing the network size while maintaining its accuracy.

[0099] Quantization, a weighting technique, allows for the sharing of weights. This weighting is based on weight clustering, reducing the number of weights that need to be stored by discretizing the continuously distributed weights. Similar to the k-means method, it clusters the weights at each layer, ensuring that weights belonging to the same cluster share the same uniform weight.

[0100] This also includes saving and loading the weight parameters of the trained lightweight network (Mixture-MobileNetV3) onto the mobile device:

[0101] S4.1: Save the weight parameters of the trained lightweight network (Mixture-MobileNetV3) as an h5 file;

[0102] S4.2: Parse the h5 file to obtain the matrixed network parameters of the lightweight network (Mixture-MobileNetV3);

[0103] S4.3: Use STM32Cube.AI software to convert the h5 file into the main files "mian.c" and "app_x-cude-ai.c" and other project files, and write the matrix network parameters into the "mian.c" file in the form of a data stream;

[0104] S4.4: Deploy the STM32Cube.AI software on an ARM-based STM32 device using "mian.c" and "app_x-cude-ai.c" and other project files.

[0105] Image processing and network training are both performed on the AI ​​server side. However, the weight parameters of the lightweight network (Mixture-MobileNetV3) cannot be directly used on embedded devices. These neural networks need to be converted into a matrix form with less data and lower computational complexity. The model structure, model weights, training configuration (loss function, optimizer, etc.), and optimizer state are saved to an h5 file for easy retraining if training is interrupted.

[0106] In this embodiment, the software platform used is: Ubuntu 20.04.5 operating system, Python 3.8, JBrainsPyCharm 2021x64, STM32Cube.AI; the hardware platform is: AMD 3960x processor, 128G memory, and NVIDIA GeForce RTX 3080Ti graphics card.

[0107] Using tested weights, a neural network is transformed using STM32Cube.AI. STM32Cube.AI converts the pre-trained neural network into C code that can run on an STM32 microcontroller, calling optimized function libraries. The resulting C code can be embedded into an STM32 device, allowing real-time object classification by inputting the object image to be recognized.

[0108] After replacing the corresponding old files in the embedded platform project file with "mobilenetmodel_init.c" and "mobilenetmodel_init.h", modify the object category names defined in the embedded platform project file to the names of the currently trained object categories.

[0109] The lightweight network (Mixture-MobileNetV3) uses an election strategy to perform real-time inference on the object to be identified and obtain the recognition result. The election strategy, which trades time for accuracy, reduces the impact of human factors and equipment on prediction accuracy by comprehensively evaluating the results of multiple inferences.

[0110] More specifically, the first block layer, the second block layer, and the third block layer each include a depthwise separable layer, an upscaling convolutional layer, a regularized convolutional layer, a downsampling layer, an downscaling convolutional layer, two fully connected layers, an attention layer, and two residual layers; in any block layer,

[0111] A depthwise separable layer, an upscaling convolutional layer, a regularized convolutional layer, a downsampling layer, a downsampling convolutional layer, a fully connected layer, and another fully connected layer are connected in sequence. The output of the depthwise separable layer is also connected to the input of another fully connected layer through a residual layer. The output of the regularized convolutional layer is also connected to the input of a fully connected layer through another residual layer. The output of the upscaling convolutional layer is also connected to the input of the downscaling convolutional layer through a delimiter layer.

[0112] More specifically, assuming the image size of the object to be identified is X*X, the output size is W*W, the kernel size is N*N, the stride is Stride, the padding is Padding, and the dilation rate is L, then the dilated convolution is defined as follows:

[0113]

[0114] More specifically, the new feature matrix O obtained after feature fusion is:

[0115] O=α*F 1 +β*F 2 +γ*F 3

[0116] α,β,γ∈[0,1]

[0117] α+β+γ=1

[0118] Among them, F 1 F represents the feature matrix output by the first block layer. 2 F represents the feature matrix output by the second block layer. 3 This represents the feature matrix output by the third Block layer, and α represents F. 1 The weights, β represents F 2 The weights, γ represents F 3 The weight.

[0119] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An object classification method based on lightweight networks, characterized in that, Includes the following steps: A lightweight network is constructed, sample images are collected and processed, and the lightweight network is trained using the processed sample images to obtain a trained lightweight network. The trained lightweight network is then used to extrapolate from images of the objects to be recognized. Based on multiple preset dilation rates, dilated convolution is performed on the image of the object to be identified to extract multiple different image features. Then, the image features are assigned weights in an adaptive weighting manner and feature fusion is performed to finally output the classification result. The lightweight network comprises a first convolutional layer, a normalization layer, a first block layer, a second block layer, a pooling layer, a second convolutional layer, and a third block layer connected in sequence; it also includes a first dilated convolutional layer, a second dilated convolutional layer, a third dilated convolutional layer, a fusion layer, and a fully connected layer; wherein, The branch output of the first block layer is connected to the input of the first dilated convolutional layer, the branch output of the second block layer is connected to the input of the second dilated convolutional layer, the branch output of the third block layer is connected to the input of the third dilated convolutional layer, the outputs of the first dilated convolutional layer, the second dilated convolutional layer, and the third dilated convolutional layer are respectively connected to the input of the fusion layer, and the output of the fusion layer is connected to the input of the fully connected layer.

2. The object classification method based on lightweight networks according to claim 1, characterized in that, The first, second, and third block layers each include a depthwise separable layer, an upscaling convolutional layer, a regularized convolutional layer, a downsampling layer, an downscaling convolutional layer, two fully connected layers, an attention layer, and two residual layers; in any block layer, A depthwise separable layer, an up-dimensional convolutional layer, a regularized convolutional layer, a downsampling layer, a down-dimensional convolutional layer, a fully connected layer, and another fully connected layer are connected in sequence. The output of the depthwise separable layer is also connected to the input of another fully connected layer through a residual layer. The output of the regularized convolutional layer is also connected to the input of a fully connected layer through another residual layer. The output of the up-dimensional convolutional layer is also connected to the input of the down-dimensional convolutional layer through an attention layer.

3. The object classification method based on lightweight networks according to claim 1, characterized in that, Assuming the image of the object to be identified has dimensions X*X, the output size is W*W, the kernel size is N*N, the stride is Stride, the padding is Padding, and the dilation rate is L, then the definition of dilated convolution is...

4. The object classification method based on lightweight networks according to claim 1, characterized in that, The new feature matrix O obtained after feature fusion is: O=α*F 1 +β*F 2 +γ*F 3 α,β,γ∈[0,1] α+β+γ=1 Among them, F 1 F represents the feature matrix output by the first block layer. 2 F represents the feature matrix output by the second block layer. 3 This represents the feature matrix output by the third Block layer, and α represents F. 1 The weights, β represents F 2 The weights, γ represents F 3 The weight.

5. The object classification method based on lightweight networks according to claim 1, characterized in that, The steps for processing sample images include: A1: Convert the sample image to grayscale to obtain a grayscale image; A2: Randomly divide the grayscale image into training sample images and test sample images; A3: Perform image augmentation on the training sample images to obtain enhanced training sample images; A4: Perform Gaussian pyramid downsampling on the enhanced training sample images to obtain downsampled training sample images.

6. The object classification method based on lightweight networks according to claim 5, characterized in that, Image augmentation operations include: random vertical rotation, random horizontal rotation, random Gaussian filtering, random cropping, random image erasure with a 20% probability, and random changes to the image's brightness, contrast, and saturation with a 30% probability.

7. The object classification method based on lightweight networks according to claim 5, characterized in that, The Gaussian pyramid downsampling of the augmented training sample images is specifically performed by: using a Gaussian low-pass filter to smooth the augmented training sample images using Gaussian smoothing, and then processing the Gaussian smoothed augmented training sample images sequentially through downsampling, max pooling, average pooling, label smoothing, and SGD optimizer.

8. The object classification method based on lightweight networks according to claim 5, characterized in that, The steps for training a lightweight network include: B1: Set the loss function, optimal loss function value, threshold for the number of times the optimal loss function value is replaced, and maximum number of training iterations for the lightweight network; B2: Input the downsampled training sample images into the lightweight network to obtain the current loss function value; B3: Compare the current loss function value with the optimal loss function value. If the current loss function value is less than the optimal loss function value, replace the optimal loss function value with the current loss function value and record the number of replacements. Otherwise, do not perform a replacement. B4: Compare the number of replacements with the threshold for the number of replacements of the optimal loss function value. If the number of replacements is less than the threshold for the number of replacements of the optimal loss function value, return to step S2 for the next round of training; otherwise, the lightweight network training is complete, and the trained lightweight network is obtained.

9. The object classification method based on lightweight networks according to claim 2, characterized in that, When training lightweight networks, Dropout is used in convolutional layers, normalization layers, pooling layers, fully connected layers, and attention layers.

Images