Image data processing method and system based on lightweight sparse neural network

Abstract

The invention provides an image data processing method and system based on a lightweight sparse neural network, and relates to the technical field of machine learning, and the method comprises the steps: obtaining to-be-processed image data; performing data enhancement and tensor standardization preprocessing of random flipping and random cutting on the image data in sequence to obtain standard tensor data meeting model input requirements; inputting the standard tensor data into the lightweight sparse neural network model; according to the model, local edge and texture features and global semantic features are extracted step by step from an image through a plurality of sparse convolution layers; and mapping the extracted physical features into category confidence through a full connection layer, and finally outputting a classification result through a Softmax function. Through a gradient-guided differential variation strategy and GPU parallel optimization, efficient image processing is realized on a mobile terminal, an embedded device and an edge computing device with limited resources, computing overhead and memory occupation are reduced, and meanwhile, high-precision classification performance is kept.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to an image data processing method and system based on a lightweight sparse neural network. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In recent years, deep learning-based image processing technologies, such as image classification, object detection, and character recognition, have been widely applied in scenarios such as intelligent surveillance, autonomous driving, and mobile document scanning and recognition (e.g., license plates, doorplates, and tickets). These applications typically require real-time or near-real-time image processing on resource-constrained hardware such as mobile terminals, embedded systems, or edge computing devices. However, to achieve high accuracy, current mainstream convolutional neural networks (such as VGG and ResNet) are often designed as parameter-intensive deep models, which leads to the following technical problems: First, the massive size and computational overhead of the model limit its deployment in resource-constrained environments. For example, in real-time license plate recognition using vehicle cameras or offline document digitization on mobile phones, the large model can lead to unacceptable memory consumption and computational latency, severely impacting system responsiveness and user experience.

[0004] Secondly, existing model compression and acceleration methods have significant shortcomings in addressing the aforementioned issues in image processing tasks. Traditional iterative pruning methods (such as the IMP algorithm based on the "lottery hypothesis") can obtain sparse models, but their search process is slow to converge, prone to getting trapped in local optima, and is a sequential process, resulting in low efficiency and difficulty in quickly generating efficient sparse subnetworks for different image tasks. Furthermore, although evolutionary algorithms possess global search capabilities and can find better sparse structures, their inherent drawbacks of slow convergence and high computational cost make them unsuitable for direct application to high-dimensional image neural network optimization.

[0005] Therefore, the existing technology lacks a method that can efficiently and automatically search for and generate high-performance lightweight sparse neural networks for complex image processing tasks. This has become a key bottleneck for the widespread application of advanced image recognition technology on resource-constrained hardware such as mobile terminals, embedded systems, or edge computing devices. Summary of the Invention

[0006] To address at least one of the technical problems mentioned above, particularly the high computational overhead, high memory consumption, and low energy efficiency issues encountered when deploying deep learning models on edge computing and mobile devices, this invention proposes an image data processing method and system based on lightweight sparse neural networks.

[0007] The first aspect of this invention provides an image data processing method based on a lightweight sparse neural network, applicable to mobile terminals, embedded devices, or edge computing devices, comprising: Acquire the image data to be processed; The image data is sequentially subjected to random flipping, random cropping, data augmentation, and tensor normalization preprocessing to obtain normalized tensor data that meets the model input requirements; The canonical tensor data is input into a lightweight sparse neural network model; the model extracts physical features from the image step by step through multiple sparse convolutional layers, including local edge and texture features extracted by the lower convolutional layers, and global semantic features integrated by the higher convolutional layers; the local edge and texture features are used to characterize the basic structural information in the image, and the global semantic features are used to characterize the category attributes of the image; The extracted physical features are mapped to class confidence scores through a fully connected layer, and the classification results are finally output through the Softmax function.

[0008] Furthermore, the lightweight sparse neural network model is obtained through the following training process: A dense neural network with shared weights and multiple consecutive masks are randomly initialized in the GPU's global memory. By starting the CUDA kernel, scheduling the GPU's streaming multiprocessors and CUDA cores, element-wise multiplication is performed on the weight tensor and each mask tensor to generate multiple sparse sub-networks to form the initial population. The discrete binary masks are continuously relaxed to become learnable parameters and introduced into the objective function for joint optimization with the network weights. The objective function includes a loss term and a sparsity regularization term based on the continuously relaxed masks. The joint gradient is calculated on the GPU using automatic differentiation techniques. The network weights and the continuous relaxation mask are optimized using a gradient-guided differential mutation strategy. The strategy first assigns different differential mutation directions based on the individual fitness, and then introduces gradient information to retrain the mutated individuals to accelerate convergence. The optimized continuous mask is discretized into a Boolean tensor and stored in the GPU's global memory as the final sparse connection mask. Then, the network weights are rolled back to the historical iteration state, and the remaining non-zero weights are fine-tuned using the gradient descent algorithm with the parallel computing power of the GPU. Finally, a lightweight sparse neural network model that can run efficiently on actual hardware is generated.

[0009] Furthermore, the image data includes any one or more of the following: handwritten digit images, natural scene images, and street view character images; The local edge features correspond to the contour information of objects in the image. In handwritten digit images, they are represented as the edges of digit strokes; in natural scene images, they are represented as object boundaries; and in street scene character images, they are represented as character outlines. The texture features correspond to the pattern information of the object surface in the image. In natural scene images, they are manifested as texture changes on the object surface, and in street scene character images, they are manifested as the thickness and spacing of the character strokes. The global semantic features correspond to the category information of objects in the image, and local features are integrated through high-level semantic representation to identify the overall content.

[0010] Furthermore, the objective function is expressed as:

[0011] in, Represents network The loss, Used to control the balance between loss and number of parameters For the Sigmoid function, Temperature parameter is a learnable parameter. Controls the degree of convergence of the mask variable to binary 0 or 1.

[0012] Furthermore, the initial population includes Population of individuals Each individual Defined as:

[0013] Among them, mask parameters Controlling sparse structures, These are shared model weights.

[0014] Furthermore, the gradient-guided differential mutation strategy includes the following sub-steps: Calculate the normalized fitness value for each individual in the current population; Select the individual with the best fitness value in the current population; Based on the relationship between the normalized fitness value and the preset threshold, different differential mutation strategies are assigned to individuals; For the intermediate individuals generated after differential mutation, use their gradient information to perform one or more gradient descent updates.

[0015] Furthermore, the specific implementation of different differential mutation strategies involves employing a random differential mutation strategy if an individual's normalized fitness value is lower than a preset threshold.

[0016] in, As a variable factor, This refers to the new individual generated after the current mutation operation. Two individuals are randomly selected from the population, and ; If an individual's normalized fitness value is higher than or equal to the preset threshold, a guided exploration strategy toward the optimal individual is adopted:

[0017] in, This is the point with the highest fitness value in the population.

[0018] Furthermore, the training process of the sparse subnetwork includes: Input dataset Population size mask initial Temperature coefficient Penalty coefficient Total number of training rounds Number of iterations per round Backward Algebra ; Generate the initial population ,in , ; for arrive Wheel, execute: for arrive In the next iteration, execute: from Medium-sampled mini-batch training samples ; For each individual Execute a gradient-guided differential mutation strategy; Update parameters , Update temperature parameters ; Perform a soft rollback update mask: Reset to , Reset to 1; Determine the mask structure rollback weight ; Gradient descent algorithm is used for sparse subnetworks Make fine adjustments; Return to the final model: .

[0019] A second aspect of the present invention provides an image data processing system based on a lightweight sparse neural network, comprising: The data acquisition module is used to acquire the image data to be processed; The data preprocessing module is used to perform random flipping, random cropping, data augmentation, and tensor normalization preprocessing on the image data in sequence to obtain normalized tensor data that meets the model input requirements. The model inference module is used to input the canonical tensor data into a lightweight sparse neural network model. This model extracts physical features from the image step by step through multiple sparse convolutional layers, including local edge and texture features extracted by the lower convolutional layers, and global semantic features integrated by the higher convolutional layers. The local edge and texture features are used to represent the basic structural information in the image, and the global semantic features are used to represent the category attributes of the image. The extracted physical features are mapped to category confidence scores through a fully connected layer, and finally the classification result is output through the Softmax function.

[0020] A third aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the image data processing method based on a lightweight sparse neural network as described in the first aspect of the present invention.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of the image data processing method based on a lightweight sparse neural network as described in the first aspect of the present invention.

[0022] A fifth aspect of the present invention provides a computer program product comprising software code, wherein the program in the software code performs the steps of the image data processing method based on a lightweight sparse neural network as described in the first aspect of the present invention.

[0023] Compared with existing technologies, the image data processing method and system based on lightweight sparse neural networks provided by this invention have the following advantages: (1) To address the technical problems of high computational overhead, high memory consumption and low search efficiency when deploying models in resource-constrained hardware environments, this invention uses GPUs to perform sparse subnetwork search and optimization in parallel on computing chips, randomly initializes dense neural networks with shared weights and multiple continuous masks, generates multiple sparse subnetworks to form an initial population, and uses a gradient-guided differential mutation strategy for parallel optimization. This method can obtain multiple deployable lightweight network structures in a single training process, significantly reducing the computational load and storage requirements of the model in mobile terminals, embedded devices and edge computing scenarios, and effectively improving the inference speed, energy efficiency and real-time response performance on the device side.

[0024] (2) To address the problem of sparse neural networks easily getting trapped in local optima and having slow convergence speed during training, this invention proposes a gradient-guided differential mutation optimization method. On a GPU platform, discrete binary masks are continuously relaxed into learnable parameters and optimized in conjunction with network weights. Simultaneously, a fitness-guided differential mutation strategy is combined to introduce gradient information for retraining mutated individuals, balancing global exploration and local convergence, thereby achieving parallel training and automatic structure adjustment of multiple networks. This method can quickly generate sparse structures with high computational efficiency and hardware friendliness during the search process, adapting to different hardware platforms (such as GPUs and edge AI chips) without additional complex search procedures, significantly improving the system's deployment flexibility and resource utilization.

[0025] (3) To address the technical problem of significant performance degradation of models under high compression ratios in resource-constrained environments such as mobile terminals and embedded systems, this invention discretizes the optimized continuous mask into binary form and backtracks to fine-tune the weights. By discretizing the continuous mask into binary form to generate the final sparse subnetwork and backtracking the network weights to the historical iteration state for fine-tuning, the model performance is effectively maintained while preserving high sparsity (above 80%). On the CIFAR-10 and SVHN datasets, the test accuracy reaches 90.70% and 95.76%, respectively, generating a lightweight neural network model suitable for hardware deployment. This achieves the best balance between model compression and performance in resource-constrained environments such as mobile terminals and embedded systems. Attached Figure Description

[0026] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0027] Figure 1 The flowchart illustrates the steps of an image data processing method based on a lightweight sparse neural network, as provided in Embodiment 1 of the present invention.

[0028] Figure 2 This is a flowchart illustrating the training process of a lightweight sparse neural network model provided in Embodiment 1 of the present invention.

[0029] Figure 3 The curve showing the change in accuracy of the method during the search process on the MNIST dataset LeNet-5 provided in Embodiment 1 of the present invention is shown.

[0030] Figure 4 The curve showing the change in test accuracy of the LeNet-5 sparse subnetwork on the MNIST dataset during the fine-tuning stage is provided in Embodiment 1 of the present invention.

[0031] Figure 5The curve showing the change in accuracy of the method during the search process on the CIFAR10 dataset ResNet18 provided in Embodiment 1 of the present invention.

[0032] Figure 6 The curve showing the change in test accuracy of the ResNet18 sparse subnetwork on the CIFAR10 dataset during the fine-tuning stage, as provided in Embodiment 1 of the present invention.

[0033] Figure 7 The curve showing the change in accuracy of the method during the search process on the CIFAR100 dataset ResNet18 provided in Embodiment 1 of the present invention is shown.

[0034] Figure 8 The curve showing the change in test accuracy of the ResNet18 sparse subnetwork on the CIFAR100 dataset during the fine-tuning stage, as provided in Embodiment 1 of the present invention.

[0035] Figure 9 The curve showing the change in accuracy of the method during the search process on the SVHN dataset ResNet18 provided in Embodiment 1 of the present invention is shown.

[0036] Figure 10 This is a comparison chart of the test accuracy of sparse subnetworks of various methods on the ResNet18 dataset of the SVHN dataset, provided in Embodiment 1 of the present invention.

[0037] Figure 11 The curves showing the change in individual population test accuracy during the search process of this method on VGG16 provided in Embodiment 1 of the present invention.

[0038] Figure 12 The curve showing the change in test accuracy of the sparse subnetwork on VGG16 during the fine-tuning stage, provided in Embodiment 1 of the present invention.

[0039] Figure 13 This is the verification accuracy of the optimal subnetwork of ResNet18 under different sparsities provided in Embodiment 1 of the present invention.

[0040] Figure 14 The impact of different backoff periods on the testing accuracy of sparse subnetworks is provided in Embodiment 1 of the present invention.

[0041] Figure 15 This is a schematic diagram of an image data processing system based on a lightweight sparse neural network provided in Embodiment 2 of the present invention. Detailed Implementation

[0042] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0043] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0044] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0045] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0046] Before describing the embodiments of the present invention in detail, the technical terms involved in the present invention are explained and defined as follows: 1. Sparse subnetwork: refers to a network whose total number of parameters is significantly reduced after applying a mask to the shared weights.

[0047] 2. Population: In evolutionary algorithms, this term refers to a set of candidate solutions to be optimized. Therefore, the initial population here specifically refers to the set of sparse subnetworks with different structures generated at the beginning of the algorithm using a set of randomly generated continuous masks. These subnetworks will serve as the starting point for the differential evolutionary algorithm optimization.

[0048] Example 1 like Figure 1 This embodiment provides an image data processing method based on a lightweight sparse neural network, applicable to mobile terminals, embedded devices, or edge computing devices, including: Acquire the image data to be processed; The image data is sequentially subjected to random flipping, random cropping, data augmentation, and tensor normalization preprocessing to obtain normalized tensor data that meets the model input requirements; The canonical tensor data is input into a lightweight sparse neural network model; the model extracts physical features from the image step by step through multiple sparse convolutional layers, including local edge and texture features extracted by the lower convolutional layers, and global semantic features integrated by the higher convolutional layers; the local edge and texture features are used to characterize the basic structural information in the image, and the global semantic features are used to characterize the category attributes of the image; The extracted physical features are mapped to class confidence scores through a fully connected layer, and the classification results are finally output through the Softmax function.

[0049] like Figure 2 The lightweight sparse neural network model is obtained through the following training process: A dense neural network with shared weights and multiple consecutive masks are randomly initialized in the GPU's global memory. By starting the CUDA kernel, scheduling the GPU's streaming multiprocessors and CUDA cores, element-wise multiplication is performed on the weight tensor and each mask tensor to generate multiple sparse sub-networks to form the initial population. The discrete binary masks are continuously relaxed to become learnable parameters and introduced into the objective function for joint optimization with the network weights. The objective function includes a loss term and a sparsity regularization term based on the continuously relaxed masks. The joint gradient is calculated on the GPU using automatic differentiation techniques. The network weights and the continuous relaxation mask are optimized using a gradient-guided differential mutation strategy. The strategy first assigns different differential mutation directions based on the individual fitness, and then introduces gradient information to retrain the mutated individuals to accelerate convergence. The optimized continuous mask is discretized into a Boolean tensor and stored in the GPU's global memory as the final sparse connection mask. Then, the network weights are rolled back to the historical iteration state, and the remaining non-zero weights are fine-tuned using the gradient descent algorithm with the parallel computing power of the GPU. Finally, a lightweight sparse neural network model that can run efficiently on actual hardware is generated.

[0050] A lightweight sparse neural network model randomly initializes a dense neural network with shared weights and multiple consecutive masks in the GPU's global memory. By launching the CUDA kernel and scheduling the GPU's streaming multiprocessors and CUDA cores, element-wise multiplication is performed on the weight tensor and each mask tensor to generate multiple sparse subnetworks to form the initial population. A gradient-guided differential mutation strategy is used to optimize the network weights and consecutively relaxed masks. This technical solution solves the problems of sparse neural networks easily getting trapped in local optima and slow convergence during training. It achieves the technical effect of rapidly generating sparse structures with high computational efficiency and hardware friendliness on the GPU platform, significantly improving system deployment flexibility and resource utilization.

[0051] Specifically, the image data includes any one or more of the following: handwritten digit images, natural scene images, and street view character images; The local edge features correspond to the contour information of objects in the image. In handwritten digit images, they are represented as the edges of digit strokes; in natural scene images, they are represented as object boundaries; and in street scene character images, they are represented as character outlines. The texture features correspond to the pattern information of the object surface in the image. In natural scene images, they are manifested as texture changes on the object surface, and in street scene character images, they are manifested as the thickness and spacing of the character strokes. The global semantic features correspond to the category information of objects in the image, and local features are integrated through high-level semantic representation to identify the overall content.

[0052] The lightweight sparse neural network model provided by this invention can be applied to the processing of handwritten digit images, natural scene images, and street scene character images. By connecting physical features with specific scenes, the practicality and interpretability of the lightweight sparse neural network model are enhanced.

[0053] Specifically, the objective function is expressed as:

[0054] in, Represents network The loss, Used to control the balance between loss and number of parameters For the Sigmoid function, Temperature parameter is a learnable parameter. Controls the degree of convergence of the mask variable to binary 0 or 1.

[0055] This invention controls the convergence of the mask variable by using learnable parameters and temperature parameters in the objective function, balancing model loss and the number of parameters, effectively maintaining model performance while preserving high sparsity, and achieving the best balance between model compression and performance.

[0056] Specifically, the initial population includes Population of individuals Each individual Defined as:

[0057] Among them, mask parameters Controlling sparse structures, These are shared model weights.

[0058] By controlling the sparse structure through mask parameters and sharing model weights, a population of individuals with different sparse connection structures is constructed. This solves the problem that a single network structure cannot adapt to diverse deployment requirements, enabling the acquisition of multiple deployable lightweight network structures during a single training process, thus improving the applicability of the method.

[0059] Specifically, the gradient-guided differential mutation strategy includes the following sub-steps: Calculate the normalized fitness value for each individual in the current population; Select the individual with the best fitness value in the current population; Based on the relationship between the normalized fitness value and the preset threshold, different differential mutation strategies are assigned to individuals; For the intermediate individuals generated after differential mutation, use their gradient information to perform one or more gradient descent updates.

[0060] Specifically, the different differential mutation strategies are assigned as follows: if an individual's normalized fitness value is lower than a preset threshold, a random differential mutation strategy is adopted.

[0061] in, As a variable factor, This refers to the new individual generated after the current mutation operation. Two individuals are randomly selected from the population, and ; If an individual's normalized fitness value is higher than or equal to the preset threshold, a guided exploration strategy toward the optimal individual is adopted:

[0062] in, This is the point with the highest fitness value in the population.

[0063] A gradient-guided differential mutation strategy is adopted, assigning different differential mutation directions based on individual fitness, and incorporating gradient information to retrain mutated individuals. This solves the problems of slow convergence and high computational cost in evolutionary algorithms, achieving a technical effect of balancing global exploration and local convergence, and accelerating the sparse structure search process.

[0064] Specifically, the training process of the sparse subnetwork includes: Input dataset Population size mask initial Temperature coefficient Penalty coefficient Total number of training rounds Number of iterations per round Backward Algebra ; Generate the initial population ,in , ; for arrive Wheel, execute: for arrive In the next iteration, execute: from Medium-sampled mini-batch training samples ; For each individual Execute a gradient-guided differential mutation strategy; Update parameters , Update temperature parameters ; Perform a soft rollback update mask: Reset to , Reset to 1; Determine the mask structure rollback weight ; Gradient descent algorithm is used for sparse subnetworks Make fine adjustments; Return to the final model: .

[0065] In one specific embodiment, the present invention was carried out on a server equipped with a 16 vCPU Intel Xeon Platinum 8352V processor and an RTX 4090 (24GB) GPU (24GB VRAM), using the Ubuntu 20.04 operating system and the PyTorch 2.3.0 deep learning framework, along with the CUDA 12.1 acceleration library.

[0066] This invention evaluates the advantages of the GDE-CS algorithm in sparse subnetwork search capabilities on GPUs by comparing its performance with mainstream network sparsification methods such as the IMP algorithm and continuous sparsification CS algorithm on different network architectures, and by comparing the corresponding dense networks as performance references. This invention selects three typical neural network architectures: LeNet-5, ResNet-18, and VGG16, and conducts experiments on their respective datasets.

[0067] The effectiveness of the method (GDE-CS algorithm) provided in this invention was tested on the MNIST dataset and the classic LeNet-5 model. The LeNet-5 model is relatively small (approximately 60,000 parameters) and adopts a 7-layer structure (including 2 convolutional layers, 2 pooling layers, and 3 fully connected layers). This model systematically introduces key concepts such as local receptive field, weight sharing, and downsampling.

[0068] The MNIST dataset contains 70,000 28×28 pixel grayscale images of handwritten digits (0–9), each image accompanied by a corresponding digit label. This dataset is primarily used for handwritten digit recognition in document processing such as bank and express delivery slips, and tax invoices, to automatically recognize handwritten digit input and improve the automation of information entry. It is also frequently used to validate the real-time recognition capabilities of models on low-power devices (such as Jetson Nano, Raspberry Pi, and FPGA modules).

[0069] This experiment sets the population size to 5 individuals (i.e., ... =5), and the GDE-CS algorithm was used to perform a sparsity search on the original dense network, resulting in 5 different sparse subnetworks. Their performance on the test set and validation set and their corresponding sparsity are shown in Table 1.

[0070] Table 1. Performance evaluation results of the sparse population in the LeNet-5 model

[0071] As can be observed from the table, all five sub-networks achieved classification accuracy comparable to the original dense model while maintaining high sparsity. Among them, the fifth sub-network, with a sparsity of 76.43%, still achieved a test accuracy of 99.10% and a validation accuracy of 99.21%.

[0072] For performance changes of the GDE-CS algorithm during training on the MNIST dataset and the LeNet-5 model, please refer to [link / reference]. Figure 3 and Figure 4 ,exist Figure 3 The diagram shows the test accuracy curves during the first round of training. It can be seen that all five individuals exhibited a good and rapid convergence trend in the first round of training, indicating that the gradient-guided differential mutation strategy can accelerate the convergence speed of sparse structures and improve the stability and accuracy of the initial training. Figure 4 The results show the change in test accuracy after fine-tuning each individual sparse subnetwork after pruning, indicating that the searched sparse structure has good potential for performance recovery. It can be seen that the GDE-CS algorithm exhibits stable test accuracy and strong generalization ability in both the sparse structure search and fine-tuning stages of the small network architecture LeNet-5.

[0073] To further verify the sparsity effect of GDE-CS, this experiment compared its optimal subnetwork with other current sparsity methods, including the original dense network (Dense), the IMP algorithm, and the CS algorithm. The comparison results are shown in Table 2.

[0074] Table 2. Performance comparison of different sparse algorithms on the LeNet-5 model

[0075] As can be observed from the table, GDE-CS achieved the best test loss and test accuracy on the LeNet-5 model and the MNIST dataset, found the lottery network, and slightly improved accuracy compared to mainstream sparse algorithms such as IMP and CS, while achieving a good balance between accuracy and sparsity.

[0076] The results in summary show that the GDE-CS algorithm can effectively maintain or even improve model performance while maintaining high model sparsity, demonstrating superior sparse structure search capabilities.

[0077] The effectiveness of the published method was tested on the CIFAR10 and CIFAR100 datasets and the SVHN dataset on the ResNet-18 model. The ResNet model, through the design of residual blocks, allows information to be passed across layers in the network, formally achieving identity mapping through "shortcut connections," thereby promoting more stable gradient propagation. This allows the network to scale to hundreds or even thousands of layers while maintaining good training performance. A typical ResNet structure consists of multiple stacked residual blocks, with the most common variants including ResNet-18, ResNet-34, and deeper models such as ResNet-50, ResNet-101, and ResNet-152.

[0078] The CIFAR-10 dataset contains diverse images with complex backgrounds, and is often used to verify the performance of convolutional neural networks in recognizing multiple natural scenes, as well as to verify the robustness of the model in recognizing targets such as vehicles, pedestrians, and transportation vehicles. It can also be used to achieve rapid deployment and efficient inference performance evaluation of embedded visual recognition systems on edge computing devices.

[0079] CIFAR-100 is divided into 100 fine-grained categories, with more subtle differences between image categories. It is suitable for evaluating the feature representation ability, transfer learning ability, and multi-task learning effect of deep networks in complex scenes, such as distinguishing and labeling multiple targets in intelligent monitoring, drone vision and other systems; and verifying the recognition ability of the model in "coarse to fine" classification problems.

[0080] Each image in the SVHN (Street View House Numbers) dataset contains one or more street view house numbers (0–9) with precise label information. Compared to MNIST, SVHN images are closer to the real world, containing complex backgrounds, different lighting conditions, and font variations, thus making the task more challenging. The task involves recognizing numerical information such as house numbers, license plates, and road signs in street view or vehicle-mounted cameras; and validating the recognition performance of a lightweight character recognition network in complex natural scenes using mobile terminal OCR (Optical Character Recognition).

[0081] This experiment focuses on searching for sparse subnetworks on the ResNet-18 convolutional neural network architecture. Table 3 shows the performance of multiple sparse subnetworks obtained by the GDE-CS algorithm on the test and validation sets on three datasets.

[0082] Table 3. Performance evaluation results of the ResNet18 model on different datasets with sparse populations.

[0083] As shown in the table, the GDE-CS algorithm successfully searched for subnetworks with sparsity ranging from 80% to 87% and exhibited excellent performance on all three datasets, demonstrating strong robustness. On CIFAR-100, despite the increased task complexity, the accuracy of the sparse subnetworks remained comparable to that of dense networks, indicating that the GDE-CS algorithm can effectively search for structurally valid sparse subnetworks even when facing multi-class classification tasks.

[0084] For the training performance of the GDE-CS algorithm on the ResNet-18 model on the CIFAR-10 and CIFAR-100 datasets, please refer to [link / reference]. Figure 5 , Figure 6 , Figure 7 and Figure 8 From 5 and Figure 7 As can be seen, in the first round of training, all five individuals achieved fast and stable convergence on both datasets, verifying the effectiveness of the proposed gradient-guided differential mutation strategy in accelerating structural convergence and improving the stability of initial training. Figure 6 and Figure 8 This reflects the changes in test accuracy of each sparse sub-network during the fine-tuning phase. It can be observed that the pruned models can recover and maintain high performance levels after further training, indicating that the sparse structure generated by the GDE-CS algorithm has good recoverability and adaptability. Overall, the algorithm demonstrates stable optimization results and strong generalization ability on different datasets under the ResNet architecture.

[0085] To verify the adaptability and robustness of the GDE-CS algorithm in different task scenarios, this experiment will further explore its sparse subnetwork search performance on ResNet18 and compare it with IMP, CS and the original dense model. Table 4 shows the model performance comparison of each algorithm while maintaining high sparsity.

[0086] Table 4. Performance comparison of ResNet18 model on different datasets using different sparse algorithms

[0087] The results show that GDE-CS achieved good performance on the CIFAR-10 and SVHN datasets with the lowest test losses of 0.3159 and 0.1674, respectively. While maintaining sparsity above 80%, it achieved test accuracies of 90.70% and 95.76%, respectively, slightly better than or on par with dense networks, demonstrating excellent sparse structure search capabilities and strong robustness. On the CIFAR-100 dataset, GDE-CS's test accuracy was slightly lower than other methods, but it still showed a good balance between loss and sparsity, indicating that it still possesses certain compression effects and optimization potential in complex multi-class tasks. (See also...) Figure 9 The GDE-CS algorithm maintains high test accuracy on the SVHN dataset, and in the early stages of training, GDE-CS outperforms other algorithms in test accuracy, demonstrating strong generalization ability and fast convergence speed.

[0088] The effectiveness of the published method was tested on the VGG16 model. VGG employs a multi-layer small convolutional kernel design strategy. This design increases the network depth while maintaining the same receptive field, thereby improving the model's expressive power and further optimizing the performance of the neural network. There are several versions of VGG, the most commonly used being VGG16, which indicates 16 convolutional layers. Other models include VGG11, VGG13, and VGG19. The CIFAR-10 dataset was selected as the evaluation benchmark, and the loss, accuracy, and corresponding sparsity of the sparse population searched by GDE-CS on the test and validation sets were recorded. The results are shown in Table 5.

[0089] Table 5. Performance evaluation results of sparse populations in the VGG16 model

[0090] As can be observed from the table, although the sparsity of the obtained sparse subnetworks all exceeded 94%, demonstrating significant compression effects, the test accuracy was slightly lower than that of the dense model, remaining generally between 88% and 89%. This indicates that the model's expressive power is somewhat limited under extremely high sparsity. For performance curves of the GDE-CS algorithm on VGG16 during the search process and sparse subnetwork fine-tuning stage, please refer to [reference needed]. Figure 11 and Figure 12 As can be seen, the performance differences among the sub-networks are small, and the diversity and differences of GDE-CS in population search are not reflected.

[0091] Next, this experiment further verifies the sparse subnetwork search performance of the GDE-CS algorithm on VGG16, and compares and analyzes it with IMP, CS and the original dense model. Table 6 shows the model performance comparison of each algorithm while maintaining high sparsity.

[0092] Table 6. Performance comparison of different sparse algorithms under the VGG16 model

[0093] As can be observed from the table, the GDE-CS algorithm achieves the highest sparsity (94.04%) while having the smallest number of parameters, only 2.88M, thus compressing the model size and demonstrating good compression performance. However, its test accuracy is slightly lower than that of the CS and Dense models, indicating a slight performance loss under extremely high sparsity.

[0094] To further investigate the impact of sparsity on model performance, this experiment sets the total number of training epochs to 5 on the ResNet-18 architecture and the CIFAR-10 dataset, and adjusts the initial value of the mask variable. This indirectly controls the final sparsity of the subnetwork during the search process. Subnetworks with different sparsities were ultimately obtained, and their performance results are shown in Table 7.

[0095] Table 7. Sparsity and performance variations of ResNet-18 under different initial mask values

[0096] As can be clearly observed from the table, as the initial mask value gradually decreases from 0, the model sparsity gradually increases, from 80.12% to 99.08%. However, at the same time, the model's accuracy on both the test and validation sets shows a significant downward trend. (See also...) Figure 13 When the sparsity is 80%, sparse subnetworks can achieve the same accuracy as dense networks.

[0097] This demonstrates that while excessive sparsity further compresses the model size, it also weakens the model's expressive power and generalization performance. Therefore, in practical applications, a trade-off must be struck between compression ratio and performance to avoid blindly pursuing extremely high sparsity and causing performance collapse. This experiment fully demonstrates the stability of the GDE-CS algorithm under different sparsity control conditions, and verifies the crucial impact of setting a reasonable initial mask value on the final sub-network performance. The flexibility of the sparsity control mechanism also reflects the application potential of GDE-CS in real-world deployment scenarios.

[0098] Finally, the impact of the rewind epoch on the training performance of the sparse subnetwork was investigated. Experiments were conducted on the ResNet-18 model using the GDE-CS algorithm at two sparsity levels: approximately 85% and approximately 93%. The effect of different rewind epoch values ​​on the final model performance was examined. The experimental results are listed in Table 8.

[0099] Table 8. Impact of backoff period on the performance of ResNet-18 high-sparseness subnetworks

[0100] The results from the two sets of experiments show that as the backoff period increases, the accuracy of the model on the validation set generally increases and the loss gradually decreases. This indicates that a longer backoff training phase can better preserve the important weight information from the early training, provide a more stable initialization for the sparse subnetwork, and help it recover and improve performance.

[0101] When the sparsity is further increased to approximately 93%, the performance of the subnetwork becomes more sensitive to changes in the backoff period. With a short backoff period (e.g., rewind=2), the model accuracy is significantly lower; while performance improves when rewind is increased to 24 or 30.

[0102] The impact of different backoff periods on the performance of sparse subnetworks is discussed in the following section. Figure 14 While the overall trend is that performance improves with increasing backoff period, slight fluctuations were observed in some experiments, indicating that an excessively long backoff period may lead to initialization that is mismatched with the current training phase, affecting the final convergence result. Therefore, appropriately selecting the backoff period value is particularly crucial for high-sparse models.

[0103] Example 2 like Figure 15 This embodiment provides an image data processing system based on a lightweight sparse neural network, including: The data acquisition module is used to acquire the image data to be processed; The data preprocessing module is used to perform random flipping, random cropping, data augmentation, and tensor normalization preprocessing on the image data in sequence to obtain normalized tensor data that meets the model input requirements. The model inference module is used to input the canonical tensor data into a lightweight sparse neural network model. This model extracts physical features from the image step by step through multiple sparse convolutional layers, including local edge and texture features extracted by the lower convolutional layers, and global semantic features integrated by the higher convolutional layers. The local edge and texture features are used to represent the basic structural information in the image, and the global semantic features are used to represent the category attributes of the image. The extracted physical features are mapped to category confidence scores through a fully connected layer, and finally the classification result is output through the Softmax function.

[0104] Example 3 Embodiment 3 of the present invention provides an electronic device.

[0105] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the image data processing method based on a lightweight sparse neural network as described in Embodiment 1 of the present invention.

[0106] The detailed steps are the same as those of the image data processing method based on lightweight sparse neural networks provided in Example 1, and will not be repeated here.

[0107] Example 4 Embodiment 4 of the present invention provides a computer-readable storage medium.

[0108] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the image data processing method based on a lightweight sparse neural network as described in Embodiment 1 of the present invention.

[0109] The detailed steps are the same as those of the image data processing method based on lightweight sparse neural networks provided in Example 1, and will not be repeated here.

[0110] Example 5 Embodiment 5 of the present invention provides a computer program product.

[0111] A computer program product includes software code, wherein the program in the software code performs the steps of the image data processing method based on a lightweight sparse neural network as described in Embodiment 1 of the present invention.

[0112] The detailed steps are the same as those of the image data processing method based on lightweight sparse neural networks provided in Example 1, and will not be repeated here.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. An image data processing method based on a lightweight sparse neural network, characterized in that, Applied to mobile terminals, embedded devices, or edge computing devices, including: Acquire the image data to be processed; The image data is sequentially subjected to random flipping, random cropping, data augmentation, and tensor normalization preprocessing to obtain normalized tensor data that meets the model input requirements; The canonical tensor data is input into a lightweight sparse neural network model; the model extracts physical features from the image step by step through multiple sparse convolutional layers, including local edge and texture features extracted by the lower convolutional layers, and global semantic features integrated by the higher convolutional layers; the local edge and texture features are used to characterize the basic structural information in the image, and the global semantic features are used to characterize the category attributes of the image; The extracted physical features are mapped to class confidence scores through a fully connected layer, and the classification results are finally output through the Softmax function.

2. The method as described in claim 1, characterized in that, The lightweight sparse neural network model is obtained through the following training process: A dense neural network with shared weights and multiple consecutive masks are randomly initialized in the GPU global memory. By starting the CUDA kernel, scheduling the GPU's streaming multiprocessors and CUDA cores, element-wise multiplication is performed on the weight tensor and each mask tensor to generate multiple sparse subnetworks to form the initial population. The discrete binary mask is continuously relaxed to become a learnable parameter, and then introduced into the objective function for joint optimization with the network weights. The objective function includes a loss term and a sparsity regularization term based on the continuously relaxed mask. The joint gradient is calculated on the GPU using automatic differentiation. The network weights and the continuous relaxation mask are optimized using a gradient-guided differential mutation strategy. The strategy first assigns different differential mutation directions based on the individual fitness, and then introduces gradient information to retrain the mutated individuals to accelerate convergence. The optimized continuous mask is discretized into a Boolean tensor and stored in the GPU's global memory as the final sparse connection mask. Then, the network weights are rolled back to the historical iteration state, and the remaining non-zero weights are fine-tuned using the gradient descent algorithm with the parallel computing power of the GPU. Finally, a lightweight sparse neural network model that can run efficiently on actual hardware is generated.

3. The method as described in claim 1, characterized in that, The image data includes any one or more of the following: handwritten digit images, natural scene images, and street view character images; The local edge features correspond to the contour information of objects in the image. In handwritten digit images, they are represented as the edges of digit strokes; in natural scene images, they are represented as object boundaries; and in street scene character images, they are represented as character outlines. The texture features correspond to the pattern information of the object surface in the image. In natural scene images, they are manifested as texture changes on the object surface, and in street scene character images, they are manifested as the thickness and spacing of the character strokes. The global semantic features correspond to the category information of objects in the image, and local features are integrated through high-level semantic representation to identify the overall content.

4. The method as described in claim 2, characterized in that, The objective function is expressed as: in, Represents network The loss, Used to control the balance between loss and number of parameters For the Sigmoid function, Temperature parameter is a learnable parameter. Controls the degree of convergence of the mask variable to binary 0 or 1.

5. The method as described in claim 2, characterized in that, The initial population includes Population of individuals Each individual Defined as: Among them, mask parameters Controlling sparse structures, These are shared model weights.

6. The method as described in claim 2, characterized in that, The gradient-guided differential mutation strategy includes the following sub-steps: Calculate the normalized fitness value for each individual in the current population; Select the individual with the best fitness value in the current population; Based on the relationship between the normalized fitness value and the preset threshold, different differential mutation strategies are assigned to individuals; For the intermediate individuals generated after differential mutation, use their gradient information to perform one or more gradient descent updates.

7. An image data processing system based on a lightweight sparse neural network, characterized in that, include: The data acquisition module is used to acquire the image data to be processed; The data preprocessing module is used to perform random flipping, random cropping, data augmentation, and tensor normalization preprocessing on the image data in sequence to obtain normalized tensor data that meets the model input requirements. The model inference module is used to input the canonical tensor data into a lightweight sparse neural network model. The model extracts physical features from the image step by step through multiple sparse convolutional layers, including local edge and texture features extracted by the lower convolutional layers, and global semantic features integrated by the higher convolutional layers. The local edge and texture features are used to characterize the basic structural information in the image, and the global semantic features are used to characterize the category attributes of the image. The extracted physical features are mapped to category confidence through a fully connected layer, and the classification result is finally output through the Softmax function.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the image data processing method based on a lightweight sparse neural network as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the image data processing method based on a lightweight sparse neural network as described in any one of claims 1 to 6.

10. A computer program product, comprising software code, characterized in that, The program in the software code performs the steps of the image data processing method based on a lightweight sparse neural network as described in any one of claims 1 to 6.

Images