Picture classification method based on convolutional neural network parameter sharing light-weight model

Abstract

This invention discloses an image classification method based on a lightweight convolutional neural network (CNN) parameter sharing model, belonging to the field of image classification technology. This image classification method based on a lightweight convolutional neural network parameter sharing model includes the following steps: obtaining CNN parameters according to the image classification task; comparing and analyzing device performance with predefined device performance, and performing a lightweight sharing evaluation analysis based on the performance comparison analysis results; and performing lightweight processing of CNN parameters based on the threshold comparison analysis results. This invention compares and analyzes device performance with predefined device performance, performs a lightweight sharing evaluation analysis based on the performance comparison analysis results, and performs lightweight processing of CNN parameters based on the threshold comparison analysis results, thereby improving the hardware applicability of lightweight CNN parameters and solving the problem of insufficient hardware applicability of lightweight CNN parameters in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to an image classification method based on a lightweight model with parameter sharing in a convolutional neural network. Background Technology

[0002] With the rapid development of artificial intelligence technology, especially the successful application of deep learning in computer vision, traditional deep learning models, such as Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), while performing excellently in many tasks, typically have a huge number of parameters and require massive computation. The high computational complexity and storage requirements of deep learning models limit their application in resource-constrained scenarios, particularly in embedded devices, mobile devices, and edge computing platforms. For example, the classic image super-resolution model SRGAN has millions or even hundreds of millions of parameters and requires high-performance GPUs to run.

[0003] Based on this, research on lightweighting deep learning models mainly focuses on the following aspects: network structure optimization, parameter quantization, model pruning, and knowledge distillation. As the scale and complexity of deep neural networks continue to increase, researchers are committed to minimizing the computational and storage requirements of the models while maintaining accuracy.

[0004] For example, the invention patent with publication number CN115861714A discloses a classification method for weft-knitted fabrics based on a lightweight convolutional neural network, which includes: acquiring sample images of various weft-knitted fabrics and establishing a sample dataset; standardizing each sample image to establish a standardized dataset; dividing the standardized dataset into a training set, a validation set, and a test set; constructing a neural network model; processing the sample images of the training set using a weight-sharing strategy and fusing the obtained feature vectors to obtain a fused feature vector; classifying the input sample image group according to the obtained fused feature vector; evaluating the training effect using the loss value of the training set and the prediction accuracy of the validation set, adjusting the hyperparameters of the neural network model, retaining the model with the highest prediction accuracy of the validation set as the optimal neural network model, inputting the test set samples into the optimal neural network model, and determining the final neural network model based on the prediction accuracy of the test set.

[0005] For example, the invention patent announcement CN113780146B discloses a hyperspectral image classification method and system based on lightweight neural architecture search. This method includes: constructing a supernet using modular lightweight candidate operations; making the discrete candidate operations on the edges continuous through weighted hybrid operations; performing two-layer optimization of the supernet using gradient optimization; and accelerating the model search speed through subnet weight sharing. Subsequently, during the optimization process, a greedy decision is used to select a non-discrete edge, retaining the operation with the largest architecture parameter on that edge and deleting the remaining operations on that edge. The remaining network constitutes a new supernet. The new supernet is iteratively optimized using the above method. As the edges are continuously discretized and operations on the edges are deleted, the supernet is continuously simplified, ultimately yielding a lightweight deep neural network architecture for hyperspectral image classification.

[0006] However, in the process of implementing the inventive technical solution in the embodiments of this application, it was found that the above-mentioned technology has at least the following technical problems: In existing technologies, traditional small convolutional kernels increase the receptive field through depth stacking, but this leads to a sharp increase in computational cost, storage resource requirements, and model training difficulty, especially when processing high-resolution images. Increased network depth not only reduces computational efficiency but also easily leads to gradient vanishing and model overfitting. Meanwhile, as image data becomes increasingly complex, existing network architectures are proving inadequate for global feature extraction. Large kernel convolutions can cover a larger area at once, reducing layer stacking and thus having a natural advantage in global information extraction. However, using large kernel convolutions presents a series of new challenges. Large kernel convolutions lead to a sharp increase in the number of parameters, resulting in excessive computational resource consumption. Furthermore, large kernel convolutions have more stringent requirements for memory usage and parallel computing in hardware implementation, placing higher demands on hardware deployment in practical applications. Therefore, how to maintain the advantages of global feature extraction while optimizing the computational efficiency and hardware adaptability of large kernel convolutions has become an urgent problem to solve, highlighting the issue of insufficient hardware applicability for lightweight convolutional neural network parameters. Summary of the Invention

[0007] This application provides an image classification method based on a lightweight convolutional neural network parameter sharing model, which solves the problem of insufficient hardware applicability of lightweight convolutional neural network parameters in the prior art, and achieves the effect of improving the hardware applicability of lightweight convolutional neural network parameters.

[0008] This application provides an image classification method based on a lightweight convolutional neural network parameter sharing model, comprising the following steps: receiving an image classification task; matching convolutional neural network parameters according to the image classification task; matching the corresponding device performance according to the convolutional neural network parameters; comparing and analyzing the device performance with predefined device performance; performing a lightweight sharing evaluation analysis based on the performance comparison analysis results; comparing and analyzing the lightweight sharing evaluation analysis with predefined hardware resource thresholds; and performing lightweight convolutional neural network parameter processing based on the threshold comparison analysis results.

[0009] Furthermore, the specific process of obtaining convolutional neural network parameters based on image classification task matching is as follows: receiving an image classification task and obtaining the image classification category, image size, and number of images; obtaining predefined image classification matching rules through an image classification task matching database; and matching the convolutional kernel size and number of convolutional neural network layers based on the image classification category, image size, number of images, the computing resource performance of the corresponding device running the convolutional neural network, and the maximum time required for image classification using the predefined image classification matching rules.

[0010] Furthermore, the specific process of comparing and analyzing the device performance with the predefined device performance is as follows: directly obtain the predefined device performance matching rules through the image classification task matching database; obtain the device performance based on the convolution kernel size and the number of layers of the convolutional neural network through the predefined device performance matching rules; if the device performance is greater than or equal to the predefined device performance, it is determined that the device performance is overloaded; if the device performance is less than the predefined device performance, it is determined that the device performance is not overloaded.

[0011] Furthermore, the shared lightweight assessment analysis based on the performance comparison analysis results specifically includes: if it is determined that the device performance is not overloaded, then no shared lightweight assessment analysis is performed, and the image classification task is performed by a convolutional neural network with the obtained convolutional neural network kernel size and convolutional neural network layer number; if it is determined that the device performance is overloaded, then a shared lightweight assessment analysis is performed.

[0012] Furthermore, the shared lightweight assessment analysis is performed, specifically including: if the device performance is determined to be overloaded, a shared lightweight assessment analysis is performed; the execution rate parameters of the convolutional neural network, the memory bandwidth utilization parameters of the convolutional neural network running device, and the storage parameters of the convolutional neural network running device are collected through performance analysis tools; and the basic performance value of the convolutional neural network running device is obtained by comprehensively analyzing the execution rate parameters of the convolutional neural network, the memory bandwidth utilization parameters of the convolutional neural network running device, and the storage parameters of the convolutional neural network running device.

[0013] Furthermore, the comparison analysis between the shared lightweight evaluation analysis and the predefined hardware resource threshold specifically includes: if the basic performance value of the convolutional neural network running device is less than the predefined hardware resource threshold, then a second shared lightweight evaluation analysis is performed; if the basic performance value of the convolutional neural network running device is greater than or equal to the predefined hardware resource threshold, then a first shared lightweight optimization adjustment is performed on the convolutional neural network.

[0014] Furthermore, the first shared lightweight optimization adjustment specifically includes: obtaining predefined left and right unitary matrices through an image classification task matching database, and setting the predefined left and right unitary matrices as shared parameters of the convolutional kernels in the convolutional neural network; obtaining different predefined feature value matrices through the image classification task matching database, and setting the different predefined feature value matrices as feature parameters of the convolutional kernels in the corresponding convolutional neural networks; thereby obtaining the parameters of the shared lightweight convolutional neural network, and denoteing the convolutional neural network after the first shared lightweight optimization adjustment as the shared lightweight convolutional neural network, and performing the image classification task through the shared lightweight convolutional neural network.

[0015] Furthermore, a second shared lightweight evaluation analysis is performed, specifically as follows: if the average computation rate of the convolutional neural network is less than or equal to the lower limit threshold of the computation rate, relevant personnel are notified and an alarm is issued; if the computation rate of the convolutional neural network is greater than the lower limit threshold, the computation rate parameters, maximum image processing throughput, and cache hit rate parameters of the convolutional neural network are collected using performance analysis tools; by comprehensively analyzing the computation rate parameters, maximum image processing throughput, and cache hit rate parameters, a correction value for the running quality of the convolutional neural network is obtained; by comprehensively analyzing the basic performance value of the device running the convolutional neural network and the correction value for the running quality of the convolutional neural network, a comprehensive running value of the convolutional neural network is obtained; and by comparing the comprehensive running value of the convolutional neural network with a predefined comprehensive running threshold, a comparison analysis result of the comprehensive running threshold is obtained.

[0016] Furthermore, the obtained comprehensive running threshold comparison analysis results specifically include: if the comprehensive running value of the convolutional neural network is less than or equal to the predefined comprehensive running threshold, then the convolutional neural network is subjected to the first shared lightweight optimization adjustment; if the comprehensive running value of the convolutional neural network is greater than the predefined comprehensive running threshold, then no adjustment is made.

[0017] Furthermore, the first shared lightweight optimization adjustment of the convolutional neural network also includes: training the shared lightweight convolutional neural network with data to obtain the accuracy of the shared lightweight convolutional neural network; if the accuracy of the shared lightweight convolutional neural network is greater than or equal to a predefined accuracy threshold, no adjustment is made, and the image classification task is performed through the shared lightweight convolutional neural network; if the accuracy of the shared lightweight convolutional neural network is less than the predefined accuracy threshold, relevant personnel are notified to perform a predefined shared lightweight backoff adjustment of the convolutional neural network parameters.

[0018] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. This invention compares and analyzes the performance of the device with that of a predefined device, performs a shared lightweight evaluation analysis based on the performance comparison analysis results, and performs lightweight processing of convolutional neural network parameters based on the threshold comparison analysis results. This achieves the effect of improving the hardware applicability of lightweight convolutional neural network parameters and solves the problem of insufficient hardware applicability of lightweight convolutional neural network parameters in the prior art.

[0019] 2. By comparing shared lightweight evaluation analysis with predefined hardware resource thresholds and making different optimization adjustments based on the results, it is possible to ensure that convolutional neural networks can run efficiently on various hardware platforms while maintaining high accuracy. This helps to optimize overall performance, reduce costs, and ultimately improve the reliability of convolutional neural network operation.

[0020] 3. By running a comprehensive threshold comparison analysis, and adjusting the first shared lightweight optimization based on the results or keeping it unchanged, we identify and optimize those factors that may cause performance fluctuations to ensure the stable operation of the convolutional neural network. When the comprehensive value is less than or equal to the predefined threshold, we can avoid unnecessary resource consumption through lightweight optimization, thereby improving resource utilization efficiency. Furthermore, by retraining and testing the model accuracy after lightweight adjustment, we ensure that the model does not lose accuracy excessively during the lightweighting process. Attached Figure Description

[0021] Figure 1 A flowchart illustrating the image classification method based on a lightweight model with shared parameters of a convolutional neural network, provided in this application embodiment. Detailed Implementation

[0022] This application provides an image classification method based on a lightweight convolutional neural network parameter sharing model, which solves the problem of insufficient hardware applicability of lightweight convolutional neural network parameters in the prior art, and improves the hardware applicability of lightweight convolutional neural network parameters.

[0023] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0024] like Figure 1 The diagram shows a flowchart of an image classification method based on a lightweight convolutional neural network parameter sharing model provided in this application. The method includes the following steps: receiving an image classification task; matching convolutional neural network parameters according to the image classification task; matching the corresponding device performance according to the convolutional neural network parameters; comparing and analyzing the device performance with predefined device performance; performing a shared lightweight evaluation analysis based on the performance comparison analysis results; comparing and analyzing the shared lightweight evaluation analysis with predefined hardware resource thresholds; and performing convolutional neural network parameter lightweighting processing based on the threshold comparison analysis results.

[0025] In this embodiment, conventional convolutional methods for image classification extract features using small convolutional kernels (3×3 or 5×5). Although stacking multiple layers can increase the receptive field, this method is inefficient.

[0026] Each convolutional operation can only extract features within a local range, thus it is weak in extracting global information that depends on long distances and is difficult to effectively capture large-scale features in images.

[0027] While traditional convolutions have small kernel sizes, the computational cost and number of parameters increase significantly with network depth. This is especially true when processing high-resolution images, where stacked convolutional layers with small kernels consume a large amount of computational resources.

[0028] With smaller convolutional kernels, existing convolutional neural networks typically stack many layers of small convolutional kernels to increase the receptive field, resulting in very deep models and increased training difficulty. Furthermore, the vanishing or exploding gradient problem is more likely to occur, especially without proper regularization or residual structure.

[0029] Prioritizing local features and using small convolutional kernels, which focus more on extracting local features, often results in poor performance for tasks that need to capture long-range dependencies, such as scene understanding and global pattern analysis. Local convolutional operations cannot effectively handle the correlation between features at long distances.

[0030] Existing large-kernel convolution methods have a large number of parameters and high computational complexity: the use of large convolution kernels (such as 7×7 or 11×11) directly leads to a sharp increase in the number of parameters and computational cost. Compared with small-kernel convolution, large-kernel convolution involves more pixels in each calculation, resulting in increased memory usage and computational overhead. This is a significant drawback for resource-constrained hardware environments (such as embedded devices).

[0031] The computational model of large kernel convolution is not friendly to some existing hardware accelerators and may not be able to fully utilize the parallel computing capabilities of existing hardware, resulting in a decrease in hardware efficiency.

[0032] Furthermore, the parameters of the convolutional neural network are obtained based on the image classification task. The specific process is as follows: receive the image classification task and obtain the image classification category, image size, and number of images; obtain predefined image classification matching rules through the image classification task matching database; and obtain the convolution kernel size and the number of convolutional neural network layers by matching the image classification category, image size, number of images, the computing resource performance of the corresponding device running the convolutional neural network, and the maximum time required for image classification according to the predefined image classification matching rules.

[0033] In this embodiment, different predefined convolutional neural network parameters are matched according to the size of the image classification task. Factors considered in the predefined image classification matching rules include: image size, number of images, and real-time requirements. Specific examples of predefined image classification matching rules are as follows: The image classification categories are divided into three types: small tasks (e.g., 100 images of 100KB each), small image size (e.g., 64x64 pixels), convolution kernel size (usually using smaller convolution kernels, such as 3x3 or 5x5), and network depth (shallow network structure, such as 2-4 layers).

[0034] LeNet for small tasks or simple custom small networks.

[0035] The image classification categories are divided into 7 types: medium-sized tasks (e.g., 100 images of 1MB each), medium-sized images (e.g., 128x128 pixels), medium-sized convolutional kernels (e.g., 5x5 or 7x7), and medium-depth networks (e.g., 8-16 layers).

[0036] For medium-sized tasks, VGG-16 (using stacked small convolutional kernels) or ResNet-18 are suitable.

[0037] The image classification categories are 13, including large tasks (e.g., 100 images of 100MB each), large image size (e.g., 1024x1024 pixels), large convolution kernel size (each kernel is between 31*31 and 101*101), and network depth (e.g., more than 16 layers).

[0038] Furthermore, the device performance is compared and analyzed with predefined device performance. The specific process is as follows: predefined device performance matching rules are directly obtained from the image classification task matching database; the device performance is obtained according to the kernel size and the number of layers of the convolutional neural network through the predefined device performance matching rules; if the device performance is greater than or equal to the predefined device performance, the device performance is judged to be overloaded; if the device performance is less than the predefined device performance, the device performance is judged to be not overloaded.

[0039] In this embodiment, the parameter size of the convolutional neural network is determined by the storage performance of the running device. In specific technical applications, the default parameter size of the applied convolutional neural network is sufficient for normal operation on the corresponding running device, and the parameter size will not exceed the storage performance limit of the device.

[0040] The predefined device performance refers to the performance of the device used for the specific processing steps of the image classification in the lightweight convolutional neural network parameter-sharing model. The predefined device performance can be directly extracted from the image classification task matching database.

[0041] Device performance data, including CPU, GPU, memory, and storage. Matching rules are then defined based on these parameters and device performance data. For example, for a specific kernel size and number of layers, the minimum required GPU memory and CPU performance are specified.

[0042] Furthermore, a shared lightweight assessment analysis is conducted based on the performance comparison analysis results. Specifically, if it is determined that the device performance is not overloaded, the shared lightweight assessment analysis is not conducted, and the image classification task is performed using a convolutional neural network with the obtained convolutional neural network kernel size and convolutional neural network layer number. If it is determined that the device performance is overloaded, the shared lightweight assessment analysis is conducted.

[0043] In this embodiment, if it is determined that the device performance is not overloaded, it means that the currently used convolutional neural network can run normally on the corresponding device and no adjustment is needed; if it is determined that the device performance is overloaded, it means that the currently used convolutional neural network cannot run normally on the corresponding device due to performance limitations and adjustment is needed.

[0044] Furthermore, a shared lightweight assessment and analysis is conducted, specifically including: if the device performance is determined to be overloaded, a shared lightweight assessment and analysis is performed; the computational rate parameters of the convolutional neural network, the memory bandwidth utilization parameters of the convolutional neural network running device, and the storage parameters of the convolutional neural network running device are collected through performance analysis tools; and the basic performance values ​​of the convolutional neural network running device are obtained by comprehensively analyzing the computational rate parameters of the convolutional neural network, the memory bandwidth utilization parameters of the convolutional neural network running device, and the storage parameters of the convolutional neural network running device.

[0045] In this embodiment, the performance analysis tools specifically include NVIDIA's CUDA toolkit and cuDNN library, which can obtain the GPU's floating-point computing power (FLOPS), memory bandwidth, and execution time of convolution operations.

[0046] TensorFlow's Profiler tool can obtain detailed performance data of a convolutional neural network.

[0047] The maximum storage size can be obtained by using Performance Monitor, a monitoring tool in Windows, to determine the maximum memory bandwidth utilization of the convolutional neural network's operating device if the device is a computer. This tool records and displays memory usage data, allowing you to determine the maximum memory bandwidth utilization of the device running the convolutional neural network.

[0048] The detection time of the convolutional neural network (CNN) running device is divided into different CNN running device detection time periods according to the predefined size of the CNN running data detection time, and these CNN running device detection time periods are numbered sequentially. This indicates the time period number of the device detecting the convolutional neural network. This represents the total number of device detection time periods during which the convolutional neural network is running. .

[0049] The average computation rate of a convolutional neural network is obtained by measuring the number of floating-point operations performed per unit time when the convolutional neural network is actually running.

[0050] The maximum memory bandwidth utilization of the device running the convolutional neural network is obtained by monitoring the memory bandwidth usage of the device when executing the convolutional neural network.

[0051] The maximum storage size required for a convolutional neural network (CNN) to run is obtained by calculating the total size of the CNN's parameters, including weights, biases, and other trainable parameters.

[0052] Indicates the first The baseline performance values ​​of the convolutional neural network running device during the detection period of each convolutional neural network running device.

[0053] ; Represents the natural constant; This represents the number of output channels of a convolutional neural network; This represents the number of input channels in a convolutional neural network. This indicates the kernel size of the convolutional neural network; Indicates the first The average execution rate of a convolutional neural network (CNN) over a given time period is measured by the device. The unit is FLOPS (floating-point operations per second), which measures the speed of performing floating-point operations.

[0054] Indicates the first The maximum memory bandwidth utilization of the convolutional neural network running device during the detection period.

[0055] This represents the threshold for memory bandwidth utilization on the device running the convolutional neural network. The threshold is obtained directly from an image classification task matching database.

[0056] Indicates the first The maximum storage size required by a convolutional neural network (CNN) for a given time period during which the CNN operates.

[0057] This represents the storage threshold of the device on which the convolutional neural network (CNN) runs. The storage threshold is obtained directly from the image classification task matching database.

[0058] The larger the storage space required for a convolutional neural network (CNN) to run, the larger the required RAM of the device. The parameters of a CNN (such as storage requirements) are closely related to the device's performance evaluation parameters (such as storage capacity, bandwidth, and speed), as they collectively determine whether the device can effectively run a specific CNN model. When designing and deploying a CNN, it is essential to ensure that the network's requirements match the device's performance to guarantee stable operation and good performance.

[0059] The number of parameters in a convolutional neural network (CNN) directly affects the maximum storage size required by the device on which it runs. The storage size refers to the space required to store the model on disk or in memory. The required storage size is directly proportional to the number of parameters, each typically represented by a floating-point number or an integer. Parameters are usually stored in bytes. When estimating storage size, the data type of the model's weights and biases must also be considered, typically floating-point numbers. For example, a neural network model with 10 million parameters, each represented by 4 bytes (32-bit floating-point numbers), would require approximately 40 megabytes (40 MB) of storage.

[0060] The weighting factor represents the computational speed of the convolutional neural network relative to the baseline performance value of the device running the convolutional neural network, and the degree of correction of the computational speed of the convolutional neural network relative to the baseline performance value of the device running the convolutional neural network.

[0061] The maximum memory bandwidth utilization of the convolutional neural network operating device is a weighting factor relative to the basic performance value of the convolutional neural network operating device, and the maximum memory bandwidth utilization of the convolutional neural network operating device is a correction factor relative to the basic performance value of the convolutional neural network operating device.

[0062] The weighting factors of the computation speed of the convolutional neural network to the basic performance value of the convolutional neural network running device and the maximum memory bandwidth utilization of the convolutional neural network running device to the basic performance value of the convolutional neural network running device are obtained directly from the image classification task matching database through a pre-defined mapping relationship.

[0063] The throughput of a convolutional neural network refers to the amount of input data that a convolutional neural network can process within a predefined time. It is measured in data processed per second.

[0064] For example, a mapping set is constructed of the convolutional neural network throughput and its corresponding weighting factors of the execution rate of the convolutional neural network to the basic performance value of the convolutional neural network running device, and the maximum memory bandwidth utilization rate of the convolutional neural network running device to the basic performance value of the convolutional neural network running device. The mapping relationship between the real-time convolutional neural network throughput and its corresponding weighting factors of the execution rate of the convolutional neural network to the basic performance value of the convolutional neural network running device, and the maximum memory bandwidth utilization rate of the convolutional neural network running device to the basic performance value of the convolutional neural network running device, is either one-to-one or many-to-one.

[0065] The number of output channels, the number of input channels, and the size of the convolutional kernel in a convolutional neural network directly affect the number of parameters and computational complexity of the convolutional layer.

[0066] The average rate of performing convolutional neural network calculations is a metric that measures how quickly a device can perform convolutional neural network calculations.

[0067] The maximum memory bandwidth utilization of a device running a convolutional neural network is an indicator that measures the efficiency of memory bandwidth usage.

[0068] The more parameters a convolutional neural network has, the larger the storage space required, and theoretically, the more computational resources it needs.

[0069] High computing speeds may require higher memory bandwidth to support data transfer, and higher computing speeds and more efficient memory bandwidth utilization can improve throughput.

[0070] These parameters are interdependent and together determine the performance of a convolutional neural network on a specific device.

[0071] Furthermore, the shared lightweight evaluation analysis is compared with the predefined hardware resource threshold. Specifically, if the basic performance value of the device running the convolutional neural network is less than the predefined hardware resource threshold, a second shared lightweight evaluation analysis is performed; if the basic performance value of the device running the convolutional neural network is greater than or equal to the predefined hardware resource threshold, a first shared lightweight optimization adjustment is performed on the convolutional neural network.

[0072] In this embodiment, the first shared lightweight optimization adjustment specifically involves depthwise separable convolution. This is achieved through parameter sharing between convolutional kernels in the depthwise convolution part and parameter sharing within the convolutional kernels in the pointwise convolution part. Parameter compression is achieved by setting shared left and right unitary matrices and unique eigenvalues ​​for each group of convolutional kernels or within each convolutional kernel. While the device's software performance can be managed through distributed resource scheduling, the actual computational capacity in specific applications is ultimately determined by the device's hardware performance.

[0073] Furthermore, the first shared lightweight optimization adjustment specifically includes: obtaining predefined left and right unitary matrices through the image classification task matching database, and setting the predefined left and right unitary matrices as shared parameters of the convolutional kernels in the convolutional neural network; obtaining different predefined feature value matrices through the image classification task matching database, and setting the different predefined feature value matrices as feature parameters of the convolutional kernels in the corresponding convolutional neural networks; thereby obtaining the parameters of the shared lightweight convolutional neural network, and denoteing the convolutional neural network after the first shared lightweight optimization adjustment as the shared lightweight convolutional neural network, and performing the image classification task through the shared lightweight convolutional neural network.

[0074] In this embodiment, the left and right unitary matrices are shared among all convolutional kernels, while the eigenvalue matrix is ​​unique to each convolutional kernel.

[0075] The eigenvalue matrix only needs to learn a set of eigenvalues ​​for each convolution kernel, instead of the entire convolution kernel.

[0076] By rotating the unitary matrix and adjusting the eigenvalues ​​of the diagonal matrix, the number of parameters can be reduced while maintaining model performance.

[0077] The implementation is based on the existing backbone. Existing large kernel convolution methods have kernel sizes between 31*31 and 101*101. It is implemented using depthwise separable convolution, which consists of two parts: depthwise convolution and pointwise convolution.

[0078] For depthwise convolution kernels, the number of parameters in existing depthwise convolutions is: ; Indicates the number of parameters in a depthwise convolution; Indicates the size of the depthwise convolution kernel; This indicates the number of input / output channels for depthwise convolution.

[0079] For pointwise convolution kernels, the number of parameters for existing pointwise convolutions is: ; Indicates the number of parameters in the pointwise convolution; This represents the number of input channels for pointwise convolution; This represents the number of output channels for pointwise convolution.

[0080] The total number of parameters in a convolutional neural network is: ; This represents the total number of parameters in a convolutional neural network.

[0081] This approach uses a parameter-sharing method to process convolutional kernels. First, for depthwise convolution, the kernel is divided into left and right unitary matrices and eigenvalues. Multiple kernels are implemented by keeping the left and right unitary matrices unchanged (sharing the same set of left and right unitary matrices) and only changing the intermediate eigenvalues. During training, the kernels are trained using the shared left and right unitary matrices and multiple eigenvalues. Second, for pointwise convolution, since the kernel is (1,1), this scheme combines multiple channels of (1,1) kernels into a matrix. This matrix can be divided into multiple smaller square matrices. Each square matrix is ​​decomposed to obtain the left and right unitary matrices and eigenvalues. The left and right unitary matrices are shared, while the eigenvalues ​​are unique to each square matrix. Changing the eigenvalues ​​yields different matrices. The resulting multiple smaller matrices are then concatenated to form the desired convolutional kernel.

[0082] After the first shared lightweight optimization adjustment, the number of parameters of the lightweight convolutional neural network is: ; This represents the total number of parameters in a lightweight convolutional neural network. Indicates the size of the depthwise convolution kernel. Represents the feature values ​​of the depthwise convolution kernel; This represents the number of input / output channels in a depthwise convolution. This represents the number of output channels for pointwise convolution. Represents the eigenvalues ​​of the pointwise convolution kernel; It is important to note that the size of the deep convolution kernel is generally large in large kernel convolution operations, which makes the total number of parameters of a lightweight convolutional neural network much smaller than the total number of parameters of a convolutional neural network.

[0083] A lightweight algorithm based on parameter sharing ensures optimal computational efficiency and inference performance of convolutional neural networks across various application scenarios. This method combines lightweight schemes at different levels with a large-kernel convolutional method improved through specific structural design. While maintaining a large receptive field for global feature extraction, it significantly reduces computational complexity and the number of parameters. This invention successfully reduces the number of model parameters, thereby lowering memory usage and model storage requirements, achieving model lightweighting. This innovation provides higher efficiency and convenience for practical deployment and adapts to resource-constrained device environments.

[0084] Further, a second shared lightweight evaluation and analysis is conducted. The specific process is as follows: if the average computation rate of the convolutional neural network (CNN) is less than or equal to the lower limit threshold, relevant personnel are notified and an alarm is issued; if the CNN computation rate exceeds the lower limit threshold, CNN computation rate parameters, maximum image processing throughput, and cache hit rate parameters are collected using performance analysis tools; by comprehensively analyzing these parameters, a CNN operation quality correction value is obtained; by comprehensively analyzing the CNN operation device's base performance value and the CNN operation quality correction value, a comprehensive CNN operation value is obtained; and by comparing the comprehensive CNN operation value with a predefined comprehensive operation threshold, a comprehensive operation threshold comparison analysis result is obtained.

[0085] In this embodiment, if the average computation rate of the convolutional neural network is less than or equal to the lower limit threshold of the computation rate of the convolutional neural network, it indicates that the large kernel convolution of the current convolutional neural network may not be suitable for the architecture of the device hardware, resulting in low algorithm efficiency. In this case, continuing to reduce the parameters is not very meaningful, the device cannot run well, and relevant personnel need to be notified to make adjustments.

[0086] Indicates the first The convolutional neural network (CNN) running quality correction value is a value generated during a specific time period of the CNN running device. This value is used to represent the degree of quality correction during the CNN running process.

[0087] ; Indicates the first The maximum image processing throughput of a convolutional neural network during a given time period.

[0088] Indicates the first The minimum image processing throughput of a convolutional neural network during a given time period.

[0089] Indicates the first The standard value of image processing throughput for a given time period is determined by the operation of a convolutional neural network.

[0090] Indicates the first The maximum computational rate of the convolutional neural network during the device's detection period.

[0091] Indicates the first The minimum computational rate of the convolutional neural network during the detection period of the device running the convolutional neural network.

[0092] Indicates the first The average execution rate of a convolutional neural network (CNN) over a given time period is measured by the device. The unit is FLOPS (floating-point operations per second), which measures the speed of performing floating-point operations.

[0093] Image processing throughput refers to the amount of data a system can process within a predefined time. In convolutional neural networks, it typically refers to the number of images or data batches that can be processed within a predefined time.

[0094] Computational rate refers to the number of computational operations completed within a predefined time. Specifically, it refers to the number of floating-point operations that can be performed per unit of time.

[0095] The predefined time is the detection time period of the convolutional neural network running in the actual calculation.

[0096] Throughput focuses on the overall output capacity of a system, i.e., the speed at which the final result is produced. Computational rate, on the other hand, focuses on the efficiency of computing resource utilization, i.e., the speed at which computational operations are executed. Throughput is limited by several aspects of the system, including computing power, memory bandwidth, I / O operations, data preprocessing, and post-processing. Computational rate is primarily limited by the performance of computing units, such as the computing power of a CPU or GPU.

[0097] The weighting factor represents the maximum image processing throughput as a factor of the convolutional neural network's performance quality correction value, and the degree of correction the maximum image processing throughput has on the convolutional neural network's performance quality correction value.

[0098] The weight factor represents the maximum computational speed of the convolutional neural network in relation to the correction value of the convolutional neural network's running quality, and the degree of correction of the maximum computational speed of the convolutional neural network in relation to the correction value of the convolutional neural network's running quality.

[0099] The weighting factors of the maximum image processing throughput for the convolutional neural network running quality correction value and the weighting factors of the maximum convolutional neural network computation speed for the convolutional neural network running quality correction value are obtained directly from the image classification task matching database through a pre-defined mapping relationship.

[0100] For example, a mapping set is constructed of the weighting factors of CPU utilization and its corresponding maximum image processing throughput for the correction value of convolutional neural network running quality, and the weighting factors of the maximum execution rate of convolutional neural network computation for the correction value of convolutional neural network running quality. The mapping relationship between the real-time CPU utilization and its corresponding maximum image processing throughput for the correction value of convolutional neural network running quality, and the weighting factors of the maximum execution rate of convolutional neural network computation for the correction value of convolutional neural network running quality, is either one-to-one or many-to-one.

[0101] Indicates the first The maximum cache hit rate of the device executing the convolutional neural network during the specified time period.

[0102] Indicates the first The minimum cache hit rate of the device executing the convolutional neural network during the specified time period.

[0103] Indicates the first The standard value of the cache hit rate of the device executing the convolutional neural network during the detection period.

[0104] Indicates the first The overall value of the convolutional neural network operation during the detection period of the device.

[0105] ; Indicates the first The quality correction value for the convolutional neural network operation during the detection period of the device.

[0106] Indicates the first The baseline performance values ​​of the convolutional neural network running device during the detection period of each convolutional neural network running device.

[0107] Setting a lower limit for the computational rate of a convolutional neural network (CNN) indicates that the current CNN is computationally inefficient and may not be suitable for the device's hardware architecture. This may necessitate a reconsideration of the network design or hardware selection.

[0108] Throughput and computation rate are two key metrics for measuring system performance. Throughput focuses on output capability, while computation rate focuses on computational efficiency.

[0109] High computing speed usually leads to high throughput, and high cache hit rate can reduce data access latency, thereby improving computing speed.

[0110] If a high cache hit rate is low, it may lead to a decrease in computing speed.

[0111] The overall performance value of the convolutional neural network reflects the overall performance of the device within a specific detection time period.

[0112] These parameters are interconnected and collectively determine the performance of a convolutional neural network on specific hardware. Understanding the relationships between these parameters helps identify performance bottlenecks, thereby guiding network optimization and hardware configuration adjustments.

[0113] Furthermore, the results of the comparative analysis of the overall running threshold are obtained, specifically including: if the overall running value of the convolutional neural network is less than or equal to the predefined overall running threshold, then the convolutional neural network is adjusted by the first shared lightweight optimization; if the overall running value of the convolutional neural network is greater than the predefined overall running threshold, then no adjustment is made.

[0114] In this embodiment, if the overall running value of the convolutional neural network is less than or equal to the predefined overall running threshold, it indicates that in the actual deployment of the convolutional neural network, the algorithm efficiency fluctuates greatly due to issues such as hardware bottlenecks, unbalanced load, data access patterns, and system interference. Although the total number of parameters of the convolutional neural network is not large, the overall convolutional neural network still needs to be lightweight.

[0115] Certain hardware resources (such as memory bandwidth or computing units) may become bottlenecks, causing performance fluctuations; in multi-GPU or multi-core systems, uneven task allocation may cause some devices or cores to be overloaded while others are idle; different data access patterns may cause fluctuations in memory access efficiency, affecting performance; other system processes or tasks may interfere with the operation of CNN, causing performance fluctuations.

[0116] Furthermore, the first shared lightweight optimization adjustment of the convolutional neural network includes: training the shared lightweight convolutional neural network with data to obtain its accuracy; if the accuracy of the shared lightweight convolutional neural network is greater than or equal to a predefined accuracy threshold, no adjustment is made, and the image classification task is performed using the shared lightweight convolutional neural network; if the accuracy of the shared lightweight convolutional neural network is less than the predefined accuracy threshold, relevant personnel are notified to perform a predefined shared lightweight backoff adjustment of the convolutional neural network parameters.

[0117] In this embodiment, training the shared lightweight convolutional neural network with data to obtain its accuracy includes the following steps: To train a shared lightweight convolutional neural network, the training data can be collected using Python libraries such as Pandas, NumPy, and OpenCV. The training data refers to the image data in the image classification task. The training data dataset is divided into training set, validation set, and test set.

[0118] Initialization parameters: Randomly initialize network weights or use weights from a pre-trained model.

[0119] Choosing a loss function: Cross-entropy loss function is typically used for classification tasks.

[0120] Choose an optimizer, such as Adam or SGD, to update network weights.

[0121] Set training parameters such as learning rate, batch size, and number of iterations.

[0122] The accuracy of a model is evaluated on the test set, which is typically done by calculating the ratio of the number of correct predictions to the total number of test samples.

[0123] train_data, val_data, test_data = load_and_split_data(); Network design and lightweighting; model = design_lightweight_cnn(); Model training; for epoch in range(num_epochs); for batch in train_data; inputs, labels = batch; predictions = model.forward(inputs); loss = calculate_loss(predictions, labels); model.backward(loss); model.update_weights(optimizer); Periodic verification val_accuracy = evaluate_model(model, val_data); Model evaluation; test_accuracy = evaluate_model(model, test_data); Model deployment; export_model(model); share_model_with_users().

[0124] An example of the process for performing lightweight rollback adjustments for shared predefined convolutional neural network parameters is as follows: Notify team members responsible for model maintenance and optimization via email, instant messaging software, etc.; determine which version of model parameters to roll back to; prepare the corresponding data and code environment; restore the predefined convolutional neural network parameters from the backup or the original model; test the performance of the rolled-back model on the validation set; confirm whether the model's accuracy has reached the predetermined threshold, and repeat the process.

[0125] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0129] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0130] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A picture classification method based on a convolutional neural network parameter sharing lightweight model, characterized in that, Includes the following steps: Receive image classification tasks and obtain convolutional neural network parameters based on the image classification tasks; Based on the parameters of the convolutional neural network, the corresponding device performance is matched, and the device performance is compared and analyzed with the predefined device performance. Based on the performance comparison and analysis results, a shared lightweight evaluation and analysis is performed. The shared lightweight evaluation analysis is compared with the predefined hardware resource threshold analysis, and the convolutional neural network parameters are lightweighted based on the threshold comparison analysis results. The comparison analysis between the shared lightweight evaluation and the predefined hardware resource threshold specifically includes: If the basic performance value of the device running the convolutional neural network is less than the predefined hardware resource threshold, then a second shared lightweight evaluation analysis is performed. If the basic performance value of the device on which the convolutional neural network runs is greater than or equal to a predefined hardware resource threshold, then the convolutional neural network will be subjected to the first shared lightweight optimization adjustment. The first shared lightweight optimization adjustment specifically includes: The predefined left and right unitary matrices are obtained by matching the database through the image classification task, and then set as the shared parameters of the convolution kernel in the convolutional neural network. Different predefined feature value matrices are obtained by matching the database through image classification tasks, and these different predefined feature value matrices are set as the feature parameters of the convolution kernels in the corresponding convolutional neural networks. Thus, the parameters of the shared lightweight convolutional neural network are obtained. The first shared lightweight optimized convolutional neural network is denoted as the shared lightweight convolutional neural network. The image classification task is performed through the shared lightweight convolutional neural network.

2. The image classification method based on a lightweight model with parameter sharing in a convolutional neural network as described in claim 1, characterized in that, The specific process of obtaining convolutional neural network parameters based on image classification tasks is as follows: Receive image classification tasks and obtain the image classification category, image size, and number of images; Retrieve predefined image classification matching rules from the image classification task matching database; Based on the image classification category, image size, number of images, the computing resource performance of the corresponding device running the convolutional neural network, and the maximum time required for the corresponding image classification, the kernel size and number of layers of the convolutional neural network are obtained through predefined image classification matching rules.

3. The image classification method based on a lightweight model with parameter sharing in a convolutional neural network as described in claim 1, characterized in that, The specific process of comparing and analyzing the device performance with the predefined device performance is as follows: Predefined device performance matching rules can be directly obtained from the image classification task matching database; Device performance is obtained by using predefined device performance matching rules based on the kernel size and number of layers of the convolutional neural network. If the equipment performance is greater than or equal to the predefined equipment performance, then the equipment performance is judged to be overloaded; If the device performance is less than the predefined device performance, then the device performance is determined not to be overloaded.

4. The image classification method based on a lightweight model with parameter sharing in a convolutional neural network as described in claim 1, characterized in that, The shared lightweight evaluation analysis based on the performance comparison analysis results specifically includes: If it is determined that the device performance is not overloaded, then the shared lightweight assessment analysis will not be performed. Instead, the image classification task will be performed by the convolutional neural network with the kernel size and the number of layers obtained by matching the convolutional neural network. If the equipment is determined to be overloaded, a shared lightweight assessment and analysis will be performed.

5. The image classification method based on a lightweight model with parameter sharing in a convolutional neural network as described in claim 4, characterized in that, The aforementioned process involves a shared lightweight assessment and analysis, specifically including: If the equipment is determined to be overloaded, a shared lightweight assessment and analysis will be performed. The computational rate parameters of the convolutional neural network, the memory bandwidth utilization parameters of the device running the convolutional neural network, and the storage parameters of the device running the convolutional neural network were collected through performance analysis tools. By comprehensively analyzing the computational rate parameters of the convolutional neural network, the memory bandwidth utilization parameters of the convolutional neural network operating device, and the storage parameters of the convolutional neural network operating device, the basic performance values ​​of the convolutional neural network operating device are obtained.

6. The image classification method based on a lightweight model with parameter sharing in a convolutional neural network as described in claim 1, characterized in that, The second shared lightweight evaluation analysis is then performed, and the specific process is as follows: If the average computation rate of executing a convolutional neural network is less than or equal to the lower limit threshold for the computation rate of executing a convolutional neural network, relevant personnel will be notified and an alarm will be issued. If the computation rate of the convolutional neural network is greater than the lower limit threshold of the computation rate of the convolutional neural network, the computation rate parameters of the convolutional neural network, the maximum image processing throughput, and the cache hit rate parameters are collected through performance analysis tools. By comprehensively analyzing the computation rate parameters of the convolutional neural network, the maximum image processing throughput, and the cache hit rate parameters, the correction value for the running quality of the convolutional neural network is obtained. By comprehensively analyzing the basic performance values ​​of the convolutional neural network operating equipment and the correction values ​​for the operating quality of the convolutional neural network, a comprehensive operating value of the convolutional neural network is obtained; Based on the comparison and analysis of the overall value of the convolutional neural network operation with the predefined overall value threshold, the results of the overall value threshold comparison and analysis are obtained.

7. The image classification method based on a lightweight model with parameter sharing in a convolutional neural network as described in claim 6, characterized in that, The obtained comprehensive threshold comparison analysis results specifically include: If the overall value of the convolutional neural network is less than or equal to the predefined overall value threshold, then the convolutional neural network is subjected to the first shared lightweight optimization adjustment. If the overall value of the convolutional neural network exceeds the predefined overall value threshold, no adjustment will be made.

8. The image classification method based on a lightweight model with parameter sharing in a convolutional neural network as described in claim 7, characterized in that, The first shared lightweight optimization adjustment of the convolutional neural network also includes: The accuracy of the shared lightweight convolutional neural network is obtained by training it with data. If the accuracy of the shared lightweight convolutional neural network is greater than or equal to the predefined accuracy threshold of the shared lightweight convolutional neural network, no adjustment is made, and the image classification task is performed through the shared lightweight convolutional neural network. If the accuracy of the shared lightweight convolutional neural network is less than the predefined accuracy threshold, relevant personnel will be notified to perform a rollback adjustment of the predefined convolutional neural network parameters.

Images